Show HN: I trained a 125M model to autocomplete piano on-device
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
Discussion Highlights (20 comments)
yoouareperfect
This is really awesome thanks for sharing
evalystai
The idea is awesome! :) However there's definitely much room for improvement, first of all rythm and composition (so there's some sense of musical form).
isoprophlex
Amazing idea! Gonna hook this up to my little synthesizer and blast some square wave arpeggiated ML music!
heikkilevanto
I would love something like that, except that I play the melody, and it produces proper 3-4 part accompaniment, preferably in good baroque style. Extra bonus if it could also write it into a file in a format suitable for music editing programs.
subhajeet2107
This is so amazing, can you improve the quality of generation at the cost of notes per seconds ? No one can play 108 notes/sec anyways, maybe you can train the model to do CoT for better quality
devonsolomon
This is really fun. Scaler 3 starts with a chord progression and lets you break it down into musical performances and parts. Useful for ideation when producing. Would be fun to get a midi clock going and play some chords on my piano and have my synth start jamming along with the bass and my keyboard doing some performance. Or any combination of the above.
pil0u
Even after a few years deep into AI, I find your application absolutely magic. This is very inspiring, thank you for sharing.
timmb
Reminds me of Francois Pachet’s Continuator (all the way back in 2003, using hierarchical markov models) https://www.francoispachet.fr/continuator/
lvbyte
Any way to share the actual model? I don't have an iPhone to run the app on.
leobg
I don’t have MIDI. How about whistling or playing the piano via microphone? Sounds easy. Another 6 month rabbit hole? :)
goda90
Reminds me of this project to generate every melody possible algorithmically in order to fight music copyright lawsuits. https://allthemusic.info/
jwr
> Eventually I used Gemini 3.5 Flash for pairwise evaluation But, but… wouldn't that be… (gasp) DISTILLATION? Fun project!
davidajackson
How would you expand this to support elements like attack ("velocity of the key-down" in piano speak), grace notes, timing etc. Would each of those be part of this model or another model? How would you model an arbitrary element (pedal, duration, etc...)
arkmm
Very cool! Can you say a little bit about the size of the DPO training examples and how long training took?
hliyan
Ah, MIDI files. The only type of music you could realistically download from the internet back in the day, and you had to wake up at ungodly hours so that your dialup modem would not rack up a massive phone bill.
gaya3bollineni
Running a 125M model on-device at that speed is impressive. How much did you have to optimize the model to get that performance on an iPhone?
speedgoose
Talking about AI music with some live human MIDI inputs, Magenta Realtime 2 was released a few weeks ago and is pretty fun. https://magenta.withgoogle.com/magenta-realtime-2
the_black_hand
Cool work. I tried using LLMs to parse sheet music and they are really bad.
aghilmort
really incredible work! great use case, impeccable learning strategy, congrats!
mandeepj
Gemma 4 E2B was too heavy for your needs?