Hacker News new | past | comments | ask | show | jobs | submit login
Piano Practice Software Progress (jacquesmattheij.com)
314 points by jacquesm 10 months ago | hide | past | favorite | 188 comments

Must share my favorite piece of piano + software conjunction: Pianoteq [0] see the video and hear how amazing it is [1]

Pianoteq makes digital pianos sound like a real piano without using pre-recorded samples, but by instead generating sound via an advanced model.

[0] https://www.modartt.com/

[1] https://www.youtube.com/watch?v=wvGTsIkdsBU

Pianoteq is very impressive. It doesn't sound quite as good as a real piano 'live' but it sounds better than most real pianos when recorded.

I have another interesting setup at home that I'll do a blog post on one of these days that works extremely well with Pianoteq and other virtual pianos, costs peanuts and gives you some amazing capabilities. Stay tuned! (pun intended ;) ).

Pianoteq models pianos very accurately, but in my opinion, it doesn't reproduce the effect of using the best condenser mics and the talent of pro audio engineers.

Top-of-the-line sampled piano VSTs may lack the dynamic range and versatility of Pianoteq, or its ability to reproduce sympathetic resonance and partial pedaling, but they at least capture not only a great piano but also the added effect of pro mics mixed/recorded by pros in a pro studio.

It's a matter of time, really. They are doing some pretty impressive modeling there sooner or later the difference between that and reality will be on the level of gold-plated plugs for your stereo.

Yeah... and I'd also add that a pro could make Pianotek 7 sound awesome.

I'm not an audio engineer, so I just use Abbey Road's Yamaha VST - its default settings sound great, no extra work for me.

I'm really eager to see if I can replicate the Raspberry Pi setup with Pianoteq!


That would make that one of the highest quality and cheapest digital pianos available.

Inspiring, thank you!

You are so right on this, it's an incredible piece of software and easily my favourite sounding piano emulation VST.

The best part is the download is really, really small, compared to the multi-gigabyte sampled solution.

My favourite hardware solution is the Roland series of modelling, I currently use an RD88 [0], which sounds great, and is really good for practice (and it has built in speakers which is a handy addition), and it's small!

The FP series is also good and less expensive.

[0] https://www.roland.com/au/products/rd-88/

Is there decent open source examples of physical piano modeling ?

Pianoteq makes mathematical modelling of pianos. I think of it some what like raytracing is for graphics, but sound tracing is for a piano.

Pianoteq is very good!

I've been on the search for the perfect Piano VST forever and I absolutely found it with Pianoteq 7.

Unlike all the other ones I've tried where I get ear fatigue after a few minutes due to some issue I can never consciously identify, with Pianoteq I can play for HOURS.

Not trying to shill for them, honest. I just really really like it.

I've found the same with all of the digital pianos except for the Yamaha I ended up buying and if not for that one I would have probably not embarked on this project at all, I really hate it when the sound isn't right. Now it still doesn't feel right (especially not when you hit a bass note), but I can live with that and with headphones on at least I don't irritate everybody else here.

But the most fun in practicing is on the real piano.

Damn, that's really cool!

Also, thanks for showing me that YT channel. That soothing voice and the clean style alone makes it worth a subscribe, heh.

Does anyone know how this compares to Spectronic's Keyscape?

I’ve played piano my entire life, but I’ve never learned to play sheet music, and have no interest in it. I play by ear and I mostly improvise. Just mentioning this because sometimes it seems that people think the way you learn to play an instrument is by learning to play a score. No! These things are about a different as learning to program and learning to type in a program from a magazine. Even the idea that you could grade how well someone plays seems antithetical to the joy of expressing yourself through music.

Coming from a background as a professional music performer and educator (now a software engineer), seeing highly-upvoted comments like this one that are so confident and yet so completely wrong is a great reminder that you should always take what you read in an internet comment section with a grain of salt, no matter how many people are nodding virtually in agreement.

Of course, there's nothing wrong with anybody learning to play piano entirely by ear and never picking up a music score. If that brings you enjoyment, that's truly fantastic, and I mean that sincerely. But for the vast majority of pianists, being unable to read sheet music will cut you off from many genres of music entirely, make in-person instruction mostly impossible, render all written pedagogical resources inaccessible to you, and enormously limit your ability to play in ensembles. Even jazz pianists who improvise and play by ear for all of their meaningful playing can read music; in fact you'd probably find that most of the really good ones are incredible sight-readers.

> These things are about a different as learning to program and learning to type in a program from a magazine.

I think a better analogy is probably something like "these things are about as different as being able to understand a spoken language, and being able to speak and write it".

Couldn't agree more. Advanced piano pieces often come with nontrivial cords and multiple voices. Those without proper ear training can hardly recognize even a single cord, let alone replicate a whole piece just by ear. Genius born with perfect pitch may do the magic without training, but they are extreme outliers and their experience can't be generalized to the wider population. For most people, inability to read music will severely limit their reach in future.

Right, but a well trained ear is exactly what those who don't focus on reading music tend to have ("instead" I might add since someone focusing on reading music doesn't necessarily need to understand it, although understanding of course helps a great deal).

It's interesting that you talk about replicating pieces. This is a "peculiarly Western" way of treating musicianship (and even in the Western world it applies primarily to classical musicians). In most of the world, musicianship is first and foremost judged by ability to improvise and to perform an orally transmitted repertoire of music. Music tends to be made in an improvisatory manner, but within the rules and constraints of a particular style.

It really depends on what kind of musician you want to be. Do you want to play Western classical music, or professionally in recording studios then reading is probably essential. If you just want to make music, it might still be handy and practical but by no means required.

It's tricky.

I started piano lessons last year late in my life with explicit purpose to learn music theory and apply it to my limited and plateaued guitar skills.

It took several weeks to persuade my teacher that "learning music theory" is not the same thing as "learning sheet music".

I want to learn truths and relationships and connections which are separate and independent from any specific culturally and historically burdened notation.

Notation has its place and I won't claim its useless, of course its not... But i do see too many instructors think it a mandatory step when it isn't (FWIW, I've been studying music theory for a year now with tremendous weekly enlightenment and still cannot read sheet music and it's not my I'm ediate goal. If anything I find that way madness lies - math and relationships and insights of music theory are beautiful and universal and eye opening. Sheet music is a crap ton of inconsistencies we are stuck with, giving privileged view to a random scale and basicly hindering true understanding. I want to build as much understanding as I can before getting stuck in C major as a random baseline :-)

So I would say music theory to sheet music is at best math theory to written numbering system. And both are separate from any practical skill that utilizes them - just like you CAN be a great blacksmith or craftsperson with developed intuitive uderstanding of your matter, without learning blueprints and its notation (though it doesn't hurt and for some things it's necessary)

Sounds to me very much like "I want to learn truths and relationships about natural language that are independent of any specific culture and history", i.e. independent of any particular language and concepts like alphabets and spelling. Such things exist (like Chomsky's generative grammar), but they are of limited use in learning any particular language.

Music theory without culture and history would have to leave out things like scales, chords, chord progressions, tuning systems (like our 12-tone equal temperament), etc., since they vary between cultures and over time. I'm not entirely sure what's left, maybe the harmonic series?

Sheet music is similar to math notation or written language, and simpler to learn than either of those. It's not the only possible notation, but it's widely used and more compact than say guitar tabs or a MIDI piano roll. If you can't read/write any notation at all you will be limited by how much you can memorize, writing things down is a time-honored tradition for rememering details for yourself as well as for sharing it with others.

So I would suggest that music students learn sheet music plus any other notation that's relevant to their instrument, for the same reason I would suggest that English learners learn to read and write despite English spelling being a crapton of inconsistencies; it gives a lot more options for a modest amount of extra effort.

Can you go through life without being able to read and write, sure. I just don't see why you would want to.

>I want to learn truths and relationships and connections which are separate and independent from any specific culturally and historically burdened notation.

Well there is no such thing. Western musical tradition is very different (and has different "truths") than Indian classical music for instance.

> giving privileged view to a random scale

The piano gives that privileged view as well.

There have been some attempts to remedy this (Janko) but nothing that really succeeded. The inertia to change is tremendous.

Agreed. I'm torn between obtaining an isomorphic I out device... And practicality of only being to play at home :-/

I have one here if you want to mess around with it you are welcome to come visit (Netherlands, hope you are close).

It is interesting, for want of a better word, it's like Dvorak to Qwerty only much worse.

Hah, thanks, appreciated.. I'm in Canada though so it might be impractical :)

Oh, yes, that would be a bit tricky. Well, consider yourself invited anyway if you ever make it to Europe. You wouldn't be the first HN'er from the other side of the Atlantic either to visit and that one got out alive ;)

> seeing highly-upvoted comments like this one that are so confident and yet so completely wrong

Looking back at my comment and scratching my head. In what way could it even be wrong? I’m literally just offering my personal experience from a life of playing piano, in reaction to the implicit assumption in the post that learning to play the piano means learning to play by reading a score.

I’m not against reading sheet music, but I’m against the idea that you somehow must do it to play this instrument, because I know it’s absolutely wrong. I’m not cut off from any genre I’m interested in playing, I’ve been able to receive in-person instructions, and I’ve certainly played in bands. I’m not really sure what “written pedagogical resources” about playing the piano would be, so not sure what to say about that.

> background as a professional music performer and educator (now a software engineer)

For what it’s worth, this is a reasonably accurate description of me as well.

> Looking back at my comment and scratching my head. In what way could it even be wrong?

Your original comments gives the impression that reading scores is "bad" somehow. The analogy of "These things are about a different as learning to program and learning to type in a program from a magazine" gives off the wrong impression. I play piano and I get what you mean, music is much more than playing a score. But the score is just a medium to learn a song. It's not "typing a program from a magazine", it's more towards "reading an algorithm description and writing the code".

> I’m not cut off from any genre I’m interested in playing,

The "I'm interested in playing" part is important. I don't think trying to play some classical piano pieces by ear is going to be easy, for example.

Is it necessary? No, of course it isn't necessary to be able to read sheet music. But it's pretty useful, not that hard, and will make a lot of things easier. You could make analogies diminishing every way to learn music (e.g., learning by ear is just like looking at a program your buddy wrote and writing the same, you're just imitating; or learning chord notation is just like writing in scratch, you're limited to the blocks someone created before) but they're not useful at all. While it worked for you, most people will actually benefit from having multiple ways to learn music.

> not that hard

I'd beg to differ. To read a moderately complex piece at the speed at which it is played while playing is tougher than most other skills that I've acquired. If it weren't hard then it probably wouldn't be the major reason lots of people give up music, the notation is inconsistent, hard to read, requires mode shifts, requires a lot of attention and can get extremely cluttered. It is anything but easy, but of course, once you've mastered it completely it might feel easy. Just like computer programming feels easy to me. But that doesn't mean that it is easy. It's just something I've been doing all my life so the underlying complexity has been long ago internalized to a level where I'm not really thinking about the code, just about the problem I want to solve.

> To read a moderately complex piece at the speed at which it is played while playing is tougher than most other skills that I've acquired

Playing moderately complex pieces will be tough, no matter the method. Also, you're using the score to learn it, in most cases by the time you're able to play it at the correct speed you don't need to read every note, you use the score as a cue and guide. And some pieces fit with different methods, for example I find it more difficult to play pop songs by sheet music than by ear (or ear + chord notation for the harmony). On the other hand I recall Satie pieces, they're pretty easy to read but I'd really struggle a lot if I wanted to play them by ear.

> If it weren't hard then it probably wouldn't be the major reason lots of people give up music

Is it though? I'd say that the major reason lots of people give up music is because it's harder than they think, and because there usually is a disconnect between what the student expects and what the teacher wants or teaches.

> once you've mastered it completely it might feel easy

This also applies to your point. I think people would get frustrated with their professor if their way of teaching pieces was just playing it and saying "now play it" without telling them what the notes are. Playing by ear is not easy, and it's really tough for people that haven't developed a musical ear and don't know any musical theory yet. At least when reading there's a set of instructions that you can follow and advance on that.

There is a balance between memorizing/finger memory and reading across piano players.

I think the skill you are talking about is sight reading, which isn't necessarily something that is required to play the piano at a high level. No matter what, you still need to practice. A lot.

I think I'm aware of that :) That's why I wrote this software in the first place, see title of the article!

Have you got interested in alternative music notations? I've dug around and it turns out people have thought about the problem. Have you ever tried any of them?

Yes, very much so. There is this Japanese one that I really like:


> Just mentioning this because sometimes it seems that people think the way you learn to play an instrument is by learning to play a score.

I think this is what can be a little misleading, depending on what “learn to play” means.

Yes, anyone can “play” an instrument without formal instruction/training, but it will definitely limit your abilities and potential (for the average person and most above average people).

As someone that took very little formal training and can play piano by ear relatively well and can pick out and play many tunes, my abilities and potential are quite limited. I can also read music (I’m more formally trained as a trombonist), but I’m super slow at reading and playing piano music.

Looking back, I now wish I had learned more formally.

I’m speculating this was one of the points the GP was trying to point out.

Per my bigger post, a lot of people are conflating "music theory" or "formal training", with "sheet notation". You will be limited performer if you don't develop an understanding of music theory at some level yes. But I've successfully challenged my music Instructor to teach me music theory without sheet music for the last year... They really aren't as inseparable as sometimes people assume :)

Are you learning music theory without learning how to read music at all?

If so, how, for example, are you talking about the concept of a dominant 7th chord? Or a ii-V-I?

For example, here is a music theory author that recognizes that two are separate, and therefore has two books:

Music Theory - which explicitly does not require nor teach sheet music: https://www.amazon.ca/gp/product/1986061833/

How to read music https://www.amazon.ca/gp/product/1546933301/

I have half a dozen other "music theory" books which in actuality spend 70% of the time on "reading music"; worse, this boring, discouraging, counter-intuitive cluttered part is typically first in the book (which means that interested minds will give up before ever getting to "the good stuff" :( ).

Basically I had to fight uphill battle with majority of professionals to actually learn something interesting and useful and insightful, as opposed to memorize sheet music (or memorize music theory terms without understanding / reasons why). I may one day decide to come back to sheet music, for many valid reasons; but reading sheet music is 100% not needed to discuss music theory - at least for me!

Edit: I noticed you had similar conversations in the past; it seems I had similar needs/perspective/experience as "jeofken" (https://news.ycombinator.com/item?id=25812227).

I'd be eager to continue conversation via email / chat to obtain your further perspective, if you'd be willing:)

interesting, I wish amazon would let me see inside the music theory book.

I'm just curious if they are still using note names and how this teacher is going about things... Because yes it's totally true you don't need to be able to read sheet music in order to understand theory well

Look at a jazz lead sheet for example, where chord voicings aren't usually spelled out (big band being an exception sometimes). Or in analysis e.g I - V7 - iv

>>how, for example, are you talking about the concept of a dominant 7th chord? Or a ii-V-I?

I mean, exactly like that :).

You don't need sheet music specifically to talk about notes and chords and scales and modes. It's just an ingrained unquestioned assumption that we do. And personally, I found it getting in the way of music theory.

There are 12 notes in Western equal temperament and you can wonderfully beautifully transpose and move things back and forth; sheet music locks you into a seeming 7 notes on a seemingly privileged scale, and makes it far far harder to develop an intuitive understanding between notes, intervals, chords, etc. I find most of my music teachers learned these things by rote, unfortunately, rather than method or relationships :-/

happy to continue the conversations.

I don't hold this assumption that you name, and it's not a part of formalized music education. which actually does emphasize method and relationships, versus rote learning. I'm curious why you think anything else would be the case. how much exposure do you have to this world? I hear that you have worked with music teachers - in what context?

also, curious to understand how many of them were coming from a jazz background?

learning to read music on the staff has no relationship to 7 notes on a privileged scale, so I'm not sure what you mean here. no note is given priority over any other note, in any clef. what makes you think that sheet music locks someone into 7 notes?

I don't intend for this to sound condescending - how well can you read music? are you speaking about all of this from the perspective of someone who has read about reading music or someone who actually understands how to read?

your comments (and those that you refer to) seem to sound like those that deride reading music as some kind of crutch or limitation in music education. I see these amusing comments a fair amount on this website.

being able to read music is having an ability to speak a shared language. like English, but with a lot smaller alphabet and an incredibly smaller set of rules. it's funny to me the degree of resistance I hear to this from people who often want to "disrupt" this language without actually understanding why it exists in the first place.

to use an analogy from mathematics, students usually benefit from learning multiplication before learning algebra as a way to see examples of logic based symbol manipulation. theory is the algebra, reading music is multiplication. that's why theory is usually taught to people who have a rudimentary understanding of the language of music.

Learning to play by a score is very different from getting instructions or training.

> in reaction to the implicit assumption in the post that learning to play the piano means learning to play by reading a score

And even that wasn't implied, I'm well aware of many people playing piano at a level that I can only dream of that couldn't read a score if their lives depended on it.

Of course, if they could read scores, they'd be more capable (and more employable) pianists for it, all else being equal.

You were literally just saying that people who learn to play an instrument and express themselves through music, if they learned how to read, were no more musicians than some who can’t actually program is a programmer.

I suppose you can argue it’s just an opinion, so therefore while it might sound condescending, arrogant and profoundly self-centred, it isn’t wrong as such. The problem with that is it wasn’t just an opinion, it was an argument. I would say that learning to become a concert pianist it a completely different thing to typing in programs from magazines, and so you are very much wrong to say that it is.

I have absolutely not said that if you learn how to read a score you’re not a musician. I compared learning to play an instrument by first learning to read a score to learning to program by first learning to type in a program accurately from a magazine. The similarity is that you’re trying to learn something that, while hard to master, can very quickly be fun and creative, but first you decided to learn to do a different thing that is just as hard and very tedious and only tangentially relevant. For a lot of learners, it’s likely to be a turn-off, or a distraction.

Concert pianist are not beginners, and they have chosen to focus on the type of repertoire that is completely centered around sheet music, so they are way, way outside where this analogy makes sense.

Quite a few outstanding jazz performers couldn't read music. https://www.reddit.com/r/Jazz/comments/2hpzzp/who_are_some_o...

This is not to say the ability to read music somehow hurts your musical abilities. Sometimes thing are simply not that correlated. E.g. having an absolute pitch - does it help to become a great musician/composer? No one knows.

From that page: "do you even really need to read music to become a good jazz musician? It seems like everyone tells you to NOT rely on it anyways if you're just starting out,and to transcribe every sound you've ever heard in your life."

(Jazz musician here) I found the OP's "I’ve never learned to play sheet music, and have no interest in it" strange - because for me, being able to write music is far more useful than just to be able to read it. (Although reading is super-useful also, whatever the genre.) I hear something I like in the street, or in my head, or on a recording - I write it down! the notes, rhythms, harmonies. How do you do that if you can't "read music"?

Not to mention transcribing, i.e. writing out tunes and improvisations. When they're more than a certain speed, learning from just playing along with it becomes impossible, and you really have to write it down first before you start to play it.

Not really quite a few. If you look at the number of performers on that page compared to the total number of working jazz musicians, it's a completely insignificant percentage.

I have never met a working jazz musician who didn't read music, or, more fundamentally, didn't have an incredibly solid understanding of western music theory.

Here's a somewhere-in-the-middle perspective. I've recently been learning the mandolin, as part of a community orchestra. I can kind of read sheet music (for the piano at least) having learnt a little piano in high school. So give me some sheet music and I can work it out after a couple of tries. I can also mostly work out a song by ear (perhaps after being given a few notes). Both are really important, and use different neural pathways and feedback mechanisms (eyes -> hands vs ears -> hands).

There's a huge difference between reading music and sight reading music.

Being able to read music is important because it is how musical ideas are communicated with precision.

Sight reading is a skill that is only useful in certain professional contexts and adds nothing to one's musical ability. You can safely dispense with sight reading as a skill if you will not be using it.

> speak and write

Read and write.

Sheet music is about reproducibility, and is a means to quickly learn a piece. You learn new pieces MUCH MUCH MUCH faster once you have decent sight reading skills. It also allows you to put in your own notes, compose your own passages or variations, and have them available for reading years later. It also gives you a common vocabulary and framework for talking about musical and instrument techniques and common patterns in music.

Learning an instrument without learning how to read music is like learning to code without learning anything anything about programming theory and methodology, and without going back to look at any of your past work. Yes, you can do it, but you'll cut yourself off at the knees with all the bad habits you pick up, and any ability to deeply reason about it will be coincidental (just a "gut" feeling most of the time).

Do yourself a favour and do it right. Get a teacher. Learn proper posture (stops you from getting tired or injuring yourself), proper techniques (allows you to play more complex things with less effort required), and a good training regimen (so you can get maximum coverage of all techniques available to you at a manageable pace).

People ask me how I got so good at guitar in so short a time, when they've been plucking away at it for years, even decades. It's simply because I chose to find a teacher FIRST, and an instrument SECOND, and went through all the fundamentals, starting with very boring and basic pieces like https://www.youtube.com/watch?v=5ysOIJFm3Rg

> Learning an instrument without learning how to read music is like learning to code without learning anything anything about programming theory and methodology, and without going back to look at any of your past work.

No: learning an instrument without learning how to read music is like learning to speak without learning to read, and doesn't imply anything about "reproducibility" or "theory" or "methodology", as all of those things existed before we had written language.

People who don't know how to read are still able to form beautiful and coherent thoughts/tunes and repeat what other people say/play... entire oral/audial traditions exist, and you would be hard pressed to find anything written down from some cultures.

More to the point: the way people write isn't even the same as the way people talk, and that isn't to say the people who are talking are somehow worse at the language; find some sheet music for an Irish jig and then see if even a single musician is literally playing what was actually written and I think you'll be surprised that sheet music doesn't even cause "reproducibility" outside of some music traditions that cared about that (such as classical orchestra).

And, hell: using sheet music can itself be a "bad habit". The greatest musicians I admire hear some music and then just join in and start playing it themselves, as they are fluent in music; and sure: some of them can also read sheet music. On the other side, I experience people who are somehow crippled by the lack of sheet music: who you whistle a tune to, and then they need sheet music to play it, as if they are some kind of player piano, and that's it... imagine if you couldn't speak--even if merely repeating what someone else just said--without first having written down what you are going to say... it would be a bit crippling, no?

(None of this has anything to do with your points about finding an instructor, learning proper technique, having a good training refining, etc. but you will find a ton of instructors who don't concentrate on sheet music... particularly with guitar, an entire instrument where sheet music isn't common at all, as the vast majority of use of the guitar in music people want to play is based on chord patterns.)

I think different individuals learn different ways. I had piano lessons from a young age, and everything was done off of sheet music. I would spend all this time reading through note by note, chord by chord, working my way through. But even after practicing reading music for over 10 years, I ALWAYS learn to play a song much, much faster by hearing it than by trying to read the music. My brain just can't automatically look at the position of black dots and translate that to which keys to press, I have to actively think about it, but once I can hear the music, my brain says, "I want to hear this note, so that's what note that dot must correspond to".

Imagine trying to learn to read English, and instead of seeing words, you see a bunch of letters and have to decipher each letter. I'd almost describe it as dyslexia for musical notation or something (which isn't a problem for me when reading text). I can recognize C,D,E,F,G pretty instantly because of their position relative to the bottom bar of the treble clef, but as soon as you start getting above that, I start having to count spaces relative to a known position, because everything internal looks like a big jumble. So G-B-E (on the treble clef) is easy because I see G as the base, and then the other two notes are space by two, but if it's just B-E without the context of the G, I have to stop and figure out exactly where those notes fall. And as soon as I get below C (on the treble clef), I start having to stop to count the separator lines.

But if I can hear what I'm trying to play, I can usually just jump to the correct note/chord. Maybe I'll have to stop and experiment to get the right chord periodically, but I don't have to stop and analyze the position of each dot on the score. To me, the greatest value of sheet music for me has always been in keeping place, so I associate the location of specific patterns on a specific page, and based on all the context, and know that I'm supposed to be at this point in the song that I've already taken the time to memorize beforehand. But I'm pretty much never paying attention to the actual notes on the notation at that point.

I’ve been playing piano for over 40 years, the first 10 years through lessons. I’m not good, as in I can’t and do not want to perform for people. I play for personal pleasure which means I might play it once every few weeks.

But things like sight reading come very naturally to me. I read music the way that I read a language. I don’t have to think hard at all to recognize notes, chords, etc and then to play them. So my ability to pick up a new song is faster than even my wife, who is about an order of magnitude better than me. She can transpose songs, learn songs by listening to them, the whole gamut. But in that one small area of sight reading, I can pick up a moderately complex song pretty quickly relative to my wife, despite the fact that I practice much less frequently than her. It must have something yo do with how the brain is wired or some sort of hand eye coordination, but it’s very interesting how I perceive sheet music vs her.

> I can recognize C,D,E,F,G pretty instantly because of their position relative to the bottom bar of the treble clef, but as soon as you start getting above that, I start having to count spaces relative to a known position, because everything internal looks like a big jumble.

It was the same for me until I really started practicing sight reading (there are special books for that). Just like learning the alphabet and how to read as a child, it took a couple of years of doing sight reading many days a week before I got good enough to sight read musical pieces. Becoming fluent at reading takes a lot of practice. And like learning to read and write in any language, it's best to do it at the same time while you are learning to speak and understand.

This was the level I was at, which is why I wrote this software to begin with, it started out as a re-write of pianobooster, which is a very neat program with a bunch of hard to fix basic ideas. Now, about a year later it is very far ahead of pianobooster, and I've learned to sightread much better than I ever could have achieved with pianobooster.

I started learning piano just over a year ago and picked up reading sheet music decently in an extremely short amount of time.

The gaps is the treble clef are

    | F | A | C | E |

The bass clef is the same but shifted down one gap (and the highest note is G)

  F | A | C | E | G |

And then middle C is, well, C. Just remembering that FACE goes in two places you get:

    bass clef          (C)  treble clef
  F | A | C | E | G |   |   | F | A | C | E |
And from that it's easy to go to the closest note and the count up/down one note and gradually memorize more. This was, at least to me, a drop dead simple way to memorize where everything goes.

This is a little better, but don't do this. You still need to calculate what are the notes outside of F A C E. Just memorize each note independently, it may take longer but it is much easier after that. If you continue using clutches like this, you'll forever have to do the translation in your head, which takes a lot of time and effort.

The comparison with learning English is spot on. The way you learn your native language is by simply growing up in a context where you are forced to try to use the sounds of the language to make yourself understood. You start with the intuition; writing, spelling, and word classes come much, much later, when you’re really an expert on the language already.

For some reason when people learn a foreign language they tend to start with the written language, and actually holding a conversation or understanding a native speaker is often a less prominent part of learning. This, to me, is a lot like thinking that learning music starts with reading a score and studying up on music theory, when you actually already have an intuitive understanding of music because you’ve listened to it all your life, and should probably focus on building similar intuition for expressing yourself with an instrument.

Are you learning 50 min long difficult classical pieces by ear?

Nope, I pretty much stopped once I got to college. At some point I decided I was going to learn Rhapsody in Blue on the piano, and got the sheet music (book) and started practicing. I didn't make it too far, but I made it a lot farther by trying to imitate what I heard than trying to play it from the book.

This is only "true" if you want to pursue a classical repertoire. For anything else you don't need to know music theory to make music or read it for that matter. A good ear and instinctive knowledge is far more valuable. Just look outside of classical music, plenty of examples where the musician has no idea of the theoretical part, but has a great grasp of it practically.

"A good ear" and grasp of music theory go hand in hand. Strongly disagree that the latter is limited to classical music. The best musicians in jazz and pop music absolutely know how to incorporate the circle of fifths, types of cadences, Roman numeral harmony, etc., in their playing. That's... music theory! While there are musicians who can make it without that, they are the exception, not the rule.

Indeed, "theory" is not an all-or-nothing affair. There's a level of "theory" that's just learning why certain intervals are harmonious in the 12 tone system, and the names of things. I certainly learned those things, but if asked whether I know "theory," my answer is no. Virtually everything I know about harmony in jazz is due to hearing and recognizing recurring patterns.

In my case, I've gotten through 40 years of performing with jazz groups, so in some sense I'm doing OK, but I also know that I struggle with ultra-modern jazz harmonies. This came into pretty sharp relief when I played with some musicians who were composing all of their own tunes. I reach the end of my mental map, and then I have to fake it, or improvise directly from the melody.

But I agree that "ear" and perception of harmonic structure are closely related. It's hard to describe, and might make a psychologist cringe, but a musician develops a "mental ear." And I wouldn't recommend my approach to a young player. Most people want to become proficient in fewer than 40 years. ;-) There are things I can't do. I can't compose or arrange anything worth playing. Without exception, every musician I've played with who could compose or arrange decent jazz material has a music degree.

This is a bit like saying that all the best speakers know a lot of grammar. Maybe they do, but that’s not why they are able to put together complete sentences, let alone why they are able to move an audience with a speech.

Some of the best Jazz pianists started out as classical pianists. Friedrich Gulda for instance, and Keith Jarrett.

I beg to differ. Music theory is essential regardless of whether you're playing off lead sheets, playing by ear, improvising, or reading scores.

You can't get good at any of them beyond a certain point without having your theory nailed down.

I would add that music theory and sight-reading are orthogonal. My partner learned piano at a young age, and she can follow a sheet but won't know what chords she's playing. I learned guitar by ear but I'm always thinking about intervals/chords/modes. Obviously having both of these skills would be great.

Jazz was a genre mainly taught as an oral tradition, and of course improvisation was at the heart of it. Musicians played what they thought sounded good. That still didn’t stop George Russell’s book The Lydian Chromatic Concept of Tonal Organization from becoming hugely popular and influential among jazz musicians in the ’50s. Even when performers have an intuitive understanding of music, they can still benefit from explicit discussion of theory.

Also, a lot of jazz musicians wanted to eventually learn musical notation at least so that they could write their own lead sheets for copyright-claim purposes.

There are some great musicians who have no formal training, but far more who actually do have formal training.

Musical pedagogy follows classical music out of tradition, but there are plenty of contemporary pieces available as well.

The thing about the classical pieces is that they're good showcases and practice pieces for the fundamental techniques of music (hundreds of years of development will do that), which you absolutely will use in your musical career, regardless of whether you're even aware of it.

The difference is that when you can read and speak music, you can read, understand, and construct music far more easily than you could without the named concepts, nomenclature, and writing system. It's no different from the power that language and writing in general confers. An illiterate person can make himself understood, but a learned person can do so much more with far less effort.

> Musical pedagogy follows classical music out of tradition

i think at some levels, of pedagogy, there's also an element of "and because it is cheap to free".

my piano teacher bemoans how we only teach music from "dead white men" (a common refrain in some parts of the internet), but is hesitant to suggest i spend money to purchase anything, instead, referring me to IMSLP for everything.

if you want music from living folks, it is still under copyright, regardless of their color or gender. that costs more.

There are other reasons to learn written music, even if you're popular musician. For example, you may want to become a studio musician, so you need to learn to play quickly (in a few minutes) a complex piece of music to perform immediately. Studio musicians need to do this all the time, and reading from a score is the easiest way to achieve it. You may need to write music for other musicians (for example, wind instrument players and pianists). Finally, reading music will help you to learn theory and have a better understanding of music.

> instinctive knowledge

There’s no such thing!

I think of this as analogous to self-supervised pretraining followed by training on a smaller labeled set. When you study theory you can ground it on music you've listened to throughout your life.

Also to improvise confidently you have to internalize the theory, not just understand it and memorize it at a conscious level.

I'd say "intuitive knowledge" is a good way to sum this up.

> It also gives you a common vocabulary and framework for talking about musical and instrument techniques and common patterns in music.

This right here. Even if you want to play rock music it can help. Guitarists and bassists have tablature, which makes working with each other easy. But tab is foreign to other musicians.

I’ve a background in music theory, so there’s been dozens of times I’ve had to act as translator between guitarist and another musician, explaining that “3rd fret barred shape, to this one” is a GM7 to CM progression. Not because it was impossible to figure out, but having a common language made it that much faster to get to work.

I can sit down at a piano and immediately play any melody I know by ear, with reasonable-sounding chords, unless there’s some very surprising harmony involved. I’m not sure how sight reading would improve on that speed.

I’m also not sure why you assume I’ve never had a teacher, just because I haven’t bothered to learn to read notes well.

All the benefits you talk about in sheet music, you can also get from listening to and making recordings, and playing with other people.

I'm curious, how does this approach fare with classical pieces where learning by sheet is the norm? The melody and chord progression is not hard to for a lot pf pieces (e.g. pop and jazz if you have a trained ear for complex chords), but learning Listz's Un Sospiro by ear is going to be really hard no?

I learned that piece by its 6 page sheet then memorized it, would be amazing if that could have been done by ear, if possible.

I think the people who don’t want to master reading from sheet music are not interested in learning long piano pieces like that.

It would be like learning Romeo and Juliet by ear. You could in theory do it, but it's far easier if you know how to read. And far easier to refresh your memory down the road if you need to perform it again.

I guess it would take a lot of patience, but it should be doable, maybe even more feasible than some contemporary music that is heavy on sound fx.

I don't find this as rewarding as trying to understand how a piece "works" though, or creating something new.

I think sheet music is a great way to see how a piece works since things are laid out spatially, and complex or subtle patterns can be teased out (e.g. voices in a Bach fugue). I often follow the sheet music as I listen as well.

Yes, excellent point. Notation can be a great aid to understanding. I should give this a try more often.

That type of music really isn’t a part of my life, but I can’t see why not. Even classical music tends to boil down to a melody + harmony. You wouldn’t get the exact fingerings right, but I don’t really find that interesting anyway.

I can also do what you describe. It’s just now happening after practicing way more in lockdown (1hr/day vs 20min before).

Had lessons as a kid but never practiced. Picked it back up 4 years ago.

I have a Beatles fake book. I play the melody and chords in the right hand, some simple metronome like bass line with the left.

This year I mastered every song and I found that I could play any other fake book song the same way on sight. Then I tried doing it without the book and was shocked to find it pretty passable.

The music (real chords and melody) is much more fun for me. It gives me a beautiful structure to play around with.

I get your point that it's easier to learn music faster by learning to read sheet music. But there's also no such thing as doing it right without context. If someone wants to just jam & relax by playing the piano, learning to read sheet music may not be doing it right.

Yes, much like a non-programmer can relax and hack up an Excel script to do a bunch of automation stuff. In certain contexts there's nothing wrong with that. But on the other hand, the farther you go, the harder it becomes, and the more the equation shifts towards "it would have been better to start with the right fundamentals".

I’ve played piano for the last 20 years, starting off initially with a teacher and going through the usual sheet music pieces. I used to think as you do for the longest time - “The important thing is to have fun!”

I still believe that, but I find that I learn new techniques every time I read, learn and play an existing piece - this makes my improvisation and jam sessions all that much better! So would highly recommend learning to read sheet music.

Curious what your thoughts are on the second best way to learn Piano, then? Second best to an in-person/live teacher.

Second best is to do it without the support and discipline of a teacher. It's just like any other thing that takes skill. You can either stand on the shoulders of giants and learn from their wisdom, or you can go it alone and make the mistakes and form the bad habits they could have warned you about. And when it comes to a musical instrument, it's all about the FEEL and POSTURE as you play, which only in-person teachers can show you.

Anyone can take a hammer and saw and build something that resembles a table, but the one who learned from a craftsman (even for just a little while) will be able to produce far better work with far less effort and mistakes. Knowing how to draft and read plans will also go a LONG way towards getting good results.

Yea, right now i'm looking for resources that at least attempt to describe and teach posture, hand patterns, etc. I may at some point hire a virtual teacher, but that seems difficult to setup. I imagine they'd need cameras, my hands, body, etc - and it sounds like work. So i'm going to pursue some non-live methods i can find, if any.

There's an app for VR on Oculus Quest [1] that the dev is working on live instructor collaboration as well as set learning pieces. He has upgraded the hand tracking for physical keyboards a few times I am not sure it's current state but this maybe of interest to you guys.

[1] Grand Reality https://sidequestvr.com/app/476/grand-reality-the-future-of-...

It's just not the same. At one point I moved away from the city where my teacher lived. We tried it over skype, with me changing the camera angle a bunch so he could observe properly, but it was slow, frustrating, and he missed so many things that came to light when I went to visit for an in-person lesson.

Music teachers are cheap to hire. Even 1 hour a week for 6 months would do WONDERS, and not cost much. Plus, your teacher will likely be a student as well, or attempting to supplement their music career. Taking lessons is supporting the arts directly.

Fingering patterns are 'work in progress', we have some interesting ideas about this.

This is very much true. I've played saxophone with lots of pleasure for many years without being able to read notes properly (spell notes would be more accurate). But I've found that being able to read notes is a useful skill and once I've decided on something like that I tend to plod away at it until I get it, I'm a pretty slow learner but I have good stamina which is what usually gets me to the end of the race. As long as I'm enjoying myself it will work out fine.

The main takeaway from your comment is that whatever you do with music should be fun, to make sure you don't destroy your motivation. And just like there is fun in being able to improvise there can also be fun in being able to play some piece perfectly (which, as you can see from the linked video I still had a long way to go with on that piece when I made the recording, it is actually much better now :) ).

Incidentally, the biggest consumer of the software in my house is my son Luca, who has taught himself a whole raft of pieces that he likes, he learns far faster than I do and his confidence is impressive, huge jumps from one end of the keyboard to the other without ever looking down, and all that with nothing but the software to guide him. He tends to come to me with some piece he wants to play, we find a youtube video, I extract the mp3, turn it into a score, we polish the score until it looks and sounds just right and then he's off to the races. Floors me every time how fast he will master something and how confident he is when playing.

"The main takeaway from your comment is that whatever you do with music should be fun, to make sure you don't destroy your motivation.”

Absolutely. As a child, I encountered both music and maths in a way that destroyed any idea that they could be fun. I took piano lessons for eight years and got to a fairly advanced amateur level, but the emphasis on theory and having to play exactly what was written on the page led to me eventually hating it, and I haven’t touched a piano since.

I later took up guitar, and discovered that I just like making stuff up, and seldom play anything exactly the same way twice. I think that now a little more understanding of the underpinnings might give me more to play with, but I still have a rather visceral reaction when I see written music...

This very much mirrors my own experience as a child (violin, piano), the later on saxophone was a lot of fun and now piano again, but this time without guidance just enjoying myself figuring it all out and using my software skills to help me.

That’s great! I certainly don’t mean to say that there’s something wrong with playing specific pieces. The fact that you’re creating the scores yourself sounds great, as it should make it clear to your son that there isn’t a “right” or “wrong” way to play. If your son continues to be interested in playing, maybe consider encouraging him to learn some melodies on his own by just listening to a recording.

Yes, absolutely, I show him how to make variations on the pieces he knows, adding ornaments, stripping it down to chord changes, harmonizing with it when playing back a recording and changing the timing and so on. Music is like paint for time, you can mix it and apply it any way you want.

I get the intent of what you're saying, but the reality is there are definitely "righter" and "wronger" ways to play. Music is incredibly mathematical, and the piano even more so. Written music provides input and queues to things like phrasing, fingering and the patterns that are often harder to decipher by ear. The true beauty to me is that we can use rules and technique to produce something that sounds so organic and pure. That's probably what also drives my deep love of computers and software.

I took classical lessons on cello and played in a community orchestra while also learning to play jazz on the bass. Today I'm mostly a jazz bassist but I also play with folk musicians due to my family's musical interests. Among folk musicians, the one thing more offensive than playing the bass is playing the bass and not actually bringing it. So I bring my bass. ;-)

So I live in both the "reading" and "ear" worlds, and in fact my jazz band requires both, since we're a 19 piece "big band" and play from written charts. The bass parts in those arrangements range from being written note-for-note, to being fully improvised.

In my view the main reason for learning to read is if you're interested in one or more of the musical genres that revolve around written repertoire. That's going to be "classical" (which extends beyond the classical era in both directions) and some jazz. You can get lost in that repertoire, and it's a blast to play, by yourself or with friends. There's so much of it that you will never run out of "new" material. Even with "ear" music such as much of jazz, reading helps you function in a band if you happen to know some but not all of the tunes being called.

And you're never completely removed from playing by ear. If anything, written material forces you to improve your ear because in most cases the notes are coming at you too fast to play without some mental processing that involves hearing it in your head.

There are two secondary reasons:

1. Access to material that stretches your physical technique and ear by design (etudes etc) or just due to being difficult. It's hard for improvisers and ear players to get beyond the plateau of playing what their hands and ears are already accustomed to.

2. Commercial work. But even there, reading puts you into a category of employable musicians, that is already overcrowded with musicians.

But if it doesn't interest you, or is an insurmountable obstacle, then leave it behind and don't look back.

I'm classically trained in piano (13+ years) and reading sheet music fluently was integral to practice, but I also have been getting into improvisation and playing by ear over the last few years. I think knowing how to read sheet music and play by ear are both valuable, sheet music provides a visual medium on which to notate time, pitch and dynamics (how precisely to follow the markings of course depends on the genre). It'd be a nightmare to learn a Bach fugue by ear though.

Personally I think it's great fun to be able to read music and play a piece you've never heard before, and bring your own life into it just as it is fun to sporadically create the unwritten.

There are absolutely folks - usually parents without musical experience who want their kid to have it (often for status, as a "good" extracurricular) - who push the sort of perspective you're countering. Onerous practice routines and robotization of expression are indeed antithetical to joy, and often result in the kids quitting sooner or later.

But just because it is commonly misapplied and misperceived doesn't mean musical literacy is a bad thing. It has many benefits, just as regular literacy does - but it doesn't have a monopoly on expression or storytelling any more than prose, and indeed there can be bad writing and excessive concern over grammar as well. And, just as anyone who does live poetry readings often memorizes the words, actual musical performance should not heavily lean on the written copy - even in an orchestra, players should know the music well enough to also keep the conductor in their vision.

Lots of benefits of musical literacy are pretty similar to regular reading and writing - you can explore ideas from past creators, serialize and share your own ideas more broadly, and more consistently track something that you're making subtle changes to over time (ink on paper doesn't shift or falter as our memory does). But one non-obvious benefit - it's also critical to coordination for larger ensembles.

Musical expression is a joy, and a very individualistic thing. But the creations of an orchestra or similar ensemble require intense coordination - I believe this doesn't rob them of value, but rather adds another dimension to them. It's not unlike the difference between making a solo or small group project versus trying to build something as a company with more employees. People have to align on the basics so, as a group, they can achieve larger things.

A mantra I have for this is "Play the Indicated Pitches at the Indicated Rhythms", which I explain more here - https://gallant.dev/posts/play-the-indicated-pitches-at-the-...

> you can explore ideas from past creators, serialize and share your own ideas more broadly, and more consistently track something that you're making subtle changes to over time (ink on paper doesn't shift or falter as our memory does).

Recording does it even better!

> But the creations of an orchestra or similar ensemble require intense coordination - I believe this doesn't rob them of value, but rather adds another dimension to them.

Again, recording is an amazing tool for this. In modern music protection it’s not uncommon to coordinate many hundreds of tracks into a single song, without the involvement of sheet music.

Recordings are great, and powerful (and I'm very familiar with multitracking - it works for a studio setting, but not so much live ensemble performance which is what I was referring to). But, to someone versed in both sheet music and improvisation, the written form can be freer (leaves more up to you), yet also more precise (knowing the specific harmonies desired rather than whatever happened in the recorded take). It can also be more convenient and efficient for focused practice. You can also take in more visually in a score and see the overall form of something in a glance, whereas with a recording you have to experience it over time and store the model fully in your head.

This is really a "yes and" situation - improvisation and "playing by ear" are great, and have always been part of music (the original "classical" musicians improvised, a tradition we've sadly mostly lost). Improvisation is even more dependent on theory than written music (many folks who "read music" don't actually understand the theory behind it). But being able to read and write is just a super convenient tool, and it addresses use cases that other tools (including recordings) don't.

As with regular writing, it lets you give persistent form and structure to your thoughts. This enables sharing, reviewing, and coordinating in a way categorically different than recordings (books still have value despite the existence of podcasts). This doesn't mean you're "not a musician" if you can't read sheet music, any more than a classical musician who doesn't improvise "isn't a musician" - I'm not trying to gatekeep in any fashion. I'm just saying that both of these dimensions are valuable, and ultimately, complementary.

Recordings are the 'binary' form of music, sheetmusic is the source code and you are much freer to interpret that sourcecode than the binary form, which can only be listened to, it is as if all the meta information got flattened and there are only two layers of data left. (Assuming stereo...).

Things like pedal markings, subtle timing hints and so on are given to the interpreter as a way to encode the composers expression, a recording can have errors in it and will lose a lot of those markings. Even 'note release' can be very hard to pull out of a recording (heck, even 'note struck' can be hard).

Yes you can talk without being able to read. Nevertheless is reading a very useful skill.

Learning to read music just gives you some formal and consistent tools with which to learn and share music. If you are only playing by ear and always independently creating the music, you definitely don't need to learn how to read sheet music. I think this probably matches well with the big shift in general learning from textbooks to resources like YouTube. It is limited though; could you imagine a symphony orchestra working with a composer if no one knew how to read music?

I'm not sure why you equate the more formal techniques of music with a lack of joy. This seems a false distinction like ranking oral storytelling traditions over the written word. Sometimes the most passionate lovers of a topic seek to understand how it works, which typically requires deep mastery of the theory and foundational concepts. You're welcome to improvise but that's a very different approach. Ironically the best improvisors often have the deepest technical competence; art still has rules and some things work better than others.

Learning to play by ear and learning to play from a score are both useful and reinforce each other.

In addition, I would encourage learning to use music notation software like Musescore. These days I transcribe a lot of pieces because I have preferences about how they’re laid out on the page. (I prefer lead sheets and dislike turning pages.) Also, for some pieces I want to play, the sheet music isn’t available. Listening to a recording to figure out how it works can take a while, so why not write it down? You don’t want to forget and have to do it again.

Having a three ring binder of music laid out how you like it is very nice. It makes it considerably easier to go back to a piece if you haven’t played it in a while. You can also change it whenever you want and print it out again, making it more your own arrangement.

I don't disagree with your points, however not having a reasonable level of sight reading inhibits you from learning some of the more dense pieces (e.g. Bach Inventions and Sifonias) in a timely manner.

Developing that skill allows you to get pieces under your hands (and in time packed into your lizard brain), so you can then focus on the musical expression.

You should think of Jacques software as a type of 'touch typing program' for the piano. Instant feedback allows you to build up the 'this means that' in terms of what you see on the sheet music and what your hands are supposed to do.

Learning to sight read, even to a very basic degree, always seems to be a big hit and miss for many of the people I've known over the years learning piano.

For me, being able to expedite the process of developing that skill so I can practice effectively, in order to be able to focus on the actual music is a huge boon! Less time trudging through the snow and more time learning and playing pieces!

When you see a pro performer crank out 2 hours of Chopin from memory, they always had to start with the music. It's a core competency for certain styles, and Jacques software would build that competency quicker due to the hyper short feedback loop (like the red/green/refactor loop).

Yet, there are plenty of real world examples of performers and composers who can't read music, there's nothing wrong with that, it just depends on what you want to do (and I want to play Bach Inventions, again!).

EDIT: I'd just like to add that my previous piano teacher was both very highly skilled in both classical and jazz disciplines, and he was of the strong opinion that it's not an either/or proposition, but commented that to not have experience in both camps hindered ability if you were a typical working musician.

That said, it would be interesting to design a piece of software that helped you learn to play by ear! Maybe something that plays a short phrase or chord progression and asks you to repeat it. Improvisation is tougher because there’s no good way for software to say if you’re making progress or not...

The Suzuki method has been around forever. It typically targets children who don't yet know how to read period, but there's no reason an adult can't use it. It doesn't work as well with improvisation because you need to understand that improvisation is not just "different", it's different within a set of constraints. You can find this by accident but it's much easier to first understand the underlying structure, then experiment.

Rick Beato has an ear training course. He’s big on YouTube.

Would there be a way to grade or allow someone to orient themselves on multiple spectrums for different skills related to the instrument or music in general? Grading inherently isn't bad, it's the gradient on a spectrum - however the way the word has been used as all-or-nothing pass-fail with the perception that your future access to education riding on it is of course terrible. But I would like to be able to orient myself somehow and I can imagine some insights and direction could be gained by being able to input the output of your music/sound generated into a system could be useful, so long as the output isn't presented in a harmful way.

Although of course not necessary but as a personal advice, I would pick up site reading early on. It teaches you a lot about music and it's a formal way of persisting your musical thoughts.

Also, finding a good teacher and taking regular lessons helps. I would be suspicious of a teacher who wouldn't recommend learning to read the score.

Progress in the beginning might be slower, but long term trajectory and possibilities are greater when a student knows how to read/write also.

But of course everyone is free to use an instrument and music how they get most joy out of and can express themselves the way they want.

I’m not qualified to have an opinion here, but interesting to note that Thom Yorke of Radiohead has never learned to read sheet music (and they have a decent number of piano-based tunes)

Fellow pianist here who also feels like sometimes too much emphasis is placed on score reading!

One interesting counterpoint that was brought up to me by my teacher in uni was that for certain pieces, mainly old old ones (think way before recording), is that sometimes the score is the _only_ thing that we have left from the composer to base our interpretation on!

I hadn't really considered this before, and it did make me appreciate score reading more (altho I still mostly improvise these days :D)

Classically trained pianist here, but never learned to sight read at good speeds... instead, I read enough of the piece to memorise it. To be honest, I think this was really bad and essentially locked me into pieces I memorised rather than allow me to learn others at a faster pace. Now, 21 years since I even touched a piano, I'm going back to basics (learning to sight read is my main aim).

Nice job OP.

> people think the way you learn to play an instrument is by learning to play a score. No! These things are about a different as learning to program and learning to type in a program from a magazine.

In the time of home computers, before Internet emerged, just about everyone learned to program like that.

(Arguably, with StackExchange seen as a magazine, many still do.)

I feel the same and am also an improviser on multiple instruments but a few times I regretted not being able to cursively read sheet music. However, all that effort and energy it would take, I’d rather spend creatively and enjoying myself just improvising.

I'm a big fan of videos like this[1]. Feels like guitar tab for piano.

[1] - https://www.youtube.com/watch?v=W3FYU72jwEA

Really? I guess everyone is different but I get absolutely nothing from those types of videos and I've tried a few times. How do you do actually use these videos in practice?

They provide no key signature, timing, chord, fingering or expression information at all? It's just pitch and duration on a fast scrolling marquee. It feels like like trying to read a book when someone else who reads at a different pace keeps turning the pages.

My preference is the standard youtube "piano lesson" format - someone sitting at the keys who plays it once and then breaks the piece down into digestible chunks, explains a bit of theory/musical context, provides fingering advice, goes over the harder parts, etc. If not that, then traditional sheet even though I'm useless at sight reading but at least, given time, I can work it out.

I've been playing long enough I can usually pick up a melody by ear, but sometimes it is transposed. Even without a key signature the videos help knowing what key it's in. Timing and expression are in the actual audio, so for me it's not needed in the display. I'm trying to play it how I hear it, not how I see it.

I also have an ok ear for melody and given a bit of time can usually come up with some sort of harmonisation so I get learning by ear.

But for me that process is about listening to the music or replaying it mentally.

I don't get what this seemingly popular falling note/guitar hero style display brings?

You mention that you use it to tell the scales used in a piece? To me, the "falling note" representation is a poor and indirect way to communicate that information.

For me there is nothing indirect about it. The falling notes point directly at which key on the piano to press. I can hear it's a I, IV, V, VI progression, but I might not recognize it's in the key of A#. With the falling notes, I see the first chord is A#. I might get that from sheet music too, but it's a lot harder to decipher.

Classical guitar and piano, classically trained on both.

While I admire those who can just grab an instrument and have a great time, musical notation isn't just a way to real and play something, it's the foundation of learning technique, growth, exploration and developing as a musician.

The idea that practicing correctly makes perfect (no, practice alone does not make perfect) is at the core of this. Musical notation (along with a teacher) provides the guidance and structure necessary to learn and grow.

There's also the ability to communicate about music in general. A simple example of this is a book I have with practice exercises for the piano. Once I taught my kids to read musical notation and use a metronome (super important) I could simply ask them to learn and practice the scale on page 42 and that was that. On the guitar, Scott Tennant communicated his practice exercises to the worth through "Pumping Nylon".

It's easy to say "Yeah, but I can just listen to someone playing the scale and copy it by ear". Well, that's missing the point. None of this material would have survived decades or centuries if it were not for the evolution of domain-specific notation as a means of communication in the art. That's the other aspect of musical notation, it's a means for making works of art survive for centuries, something that is impossible without being able to write things down.

To go into CS for a moment, the power of notation got driven into my head when my Physics professor in the early 80's convinced me to not take a FORTRAN class and sign-up for an APL class he was teaching instead. It was absolutely amazing. At the time it was like being from the future. In just a few characters I could do what took heaps of code in COBOL or FORTRAN, and the power of communicating such ideas through notation was just unbelievable. Sometimes I wonder if I took to APL (which I used professionally for ten years afterwards) because I was fluent in musical notation already.

All of that said, I think it is great to "just play". Nothing wrong with that. I'll also add that it took me a long time to be able to pull away from sheet music and "just play". And so I do have a level of admiration for people who are able to do that and came at it without any form of formal training. At the end of the day, if your goals in music are not to be a concert pianist/guitarist, frankly, if you are having fun, go for it. Just be conscious of the fact that once bad habits are learned it is extremely difficult to unlearn them. That's were a formal and traditional beginning in music tends to be useful and important.

It's no longer musical notation, but audio recordings that are the "text," or primary source for popular music. This principle applies even only from a practical perspective, as the pedagogy of popular music has a heavy emphasis on listening to and replicating or transcribing audio recordings.

But more fundamentally, you're approaching from an outdated common practice-period perspective what should be understood from a (post)modernist, electronic framework. Transcribing Stockhausen or Xenakis into notation would be a worthy endeavor, but the output of such an effort is secondary to the recording itself. Not just in terms of importance, but in that the recording represents the ultimate creative decisions and expressions made by the musician, whereas the notation is a mere reduction produced for convenience. This was certainly not the case for classical music, but music has changed since then.

And so the same goes for jazz, rock, or any other popular genre. Our music has evolved in such a way that those musical elements that do not lend themselves easily to being written down in standard Western musical notation have become central to the expression and stylistic idiom thereof.

Finally, if you're interested in what I've had to say, look up a concept called "notational centricity" by musicologist Richard Middleton, and a book called "Everyday Tonality" by Philip Tagg.

Pure speculation, but does this impose an upper limit on the complexity of music you can play? Personally I cannot fathom someone play Liszt complete my by ear.

Exactly my thought and contention.

I took piano lessons for years, but was never good at reading music. I excelled at playing by ear. But when I wanted to learn something truly technical and challenging, there's no way I could have done it without sheet music.

Case in point: Imagine trying to learn this [0] by ear.

- [0] https://www.youtube.com/watch?v=KJbg9V2KnD8

I tried the webpage, and I got

> WebMIDI is supported in this browser

Do I need a MIDI wire to connect the piano or the page can heard the sound? (Sound recognitions looks very difficult.)

It would be nice to add some example with a graphic in the paragraph about labeling the scores for non musicians. (My wife plays the piano and guitar, but not professionally. I understand the that D♭ is somewhat equivalent to C♯ in a piano, but using the wrong one in a score is as bad as an unmatched parenthesis. But don't ask me the details.

I was going to ask if you support DoReMiFaSolLaSi, but it looks like another rabbit hole https://en.wikipedia.org/wiki/Musical_note#12-tone_chromatic...

Unfortunately, yes, you - still - need to have a midi out on your piano, so any electronic piano or pianos with a silent option can be used, ditto with almost every synth made after the 1980's.

That score labeling is actually in there, you'll find it at the top and bottom of the score for right hand and left hand respectively.

The DoRe-etc was considered but that's not so simple.

I was suggesting to add to the blog post an screenshot of the problem and the fixed version.

Okay, done!

Scroll all the way to the bottom after pressing F5 and it should appear. You can clearly make out the labels.

For those interested, I built a "falling notes" style web interface that works with a midi keyboard. You can mark a split in the keyboard and just practice one side or the other. Most of the controls I've baked in use the keyboards other buttons (Novation Launchkey 61).

It's open source so feel free to adapt as needed.


I've found it works well for my wife and I playing simple songs together, she never learned sheet music (and I'm not great at it) so this format is very easy to follow.

Looks good. Sometimes I play this with friends https://multiplayerpiano.com/

You can play online at the same time with multiple people


The digital pianos I've owned support MIDI over a USB Type-B port, so I invested in a Type-B to Type-C cable. Works like a dream with a Chromebook and websites like Flowkey (and Pianojacq, I'm assuming!) No adapters needed either. :)

Here's the one I got: https://www.amazon.com/gp/product/B00YQFPX8Y?th=1

Edit- I have the same piano as the OP, a Yamaha P-515. Best purchase I ever made!

The P-515 is very good for what it costs. I tried a whole bunch of them before deciding, Nord Stage 3 wasn't as good at almost 3 times the price. And what it lacks in gimmicks you can easily add in software.

I get so much entertainment out of just switching between the default Yamaha CFX voice and the Bosendorfer. They make the same song sound so different!

I'm excited to give Pianojacq a try soon! Maybe I'll even try my hand at custom user CSS. :) Have you given any thought to a theming engine?

No, a theming engine is not even on the list of 'possibles', but if you want to add it there will be a point when it would be a good moment, first the UI needs to stabilize.

Note that it works perfectly if you download it so you could download it and patch the CSS if you want to do this more permanently (you'd have to rebase periodically if you want to keep up with my development).

Is this like Synthesia? https://synthesiagame.com/

This has a pretty powerful 'auto' mode where it grades your practice and steers you towards practicing the bits that you have problems with, it doesn't require you to subscribe to any service or pay (the data is yours and it stays on your computer).

It tracks your progress in a very detailed way and remembers how you played a piece before to help you play it better in the future. It aims at making you independent of the program and to teach you to play well. It still is no substitute for a teacher, but it is certainly better than nothing at all. I have some plans to incorporate all of the Mayron Cole course into it but that will take a long time (and I currently do not have a whole lot of time, but that will change soon).

> the data is yours and it stays on your computer

Music to my ears! I hope more developers go this route and prioritize making quality software over squeezing every available drop of data from users.

I really like this but I can't seem to get my Nanokey2 to work with it.

Edit: Never mind, it was the settings, it set the output to the nanokey, but didn't set the input.

I have synthesia and no, it’s not the same. This one requires you to know sheet music, a must have skill I you want to learn a piece quicker than rote memorization in synthesia. I used to refuse reading sheet music. But, thank goodnesses, I knew I was mistaking and since then was able to learn more complicated pieces like the Aria in Goldberg or couple of preludes in the WTC.

You can actually hide the falling notes and switch to sheet only in Synthesia, this was a much requested feature update made years ago.

This seems more like Flowkey.


This inspired me to hook up the piano I once bought but then never touched.

Two questions:

1) Can you make it more sensitive to slow keypresses? I have to press quite fast / with force to get the keypress recognized (for comparison pianobooster does recognize my slow (quiet) press).

2) Can you recommend midi files that are well supported? Most midi-files I found on my PC don't work at all. Those who do look different in pianobooster (for example pianobooster has notes on both hands but pianojacq only one one).

Thank you.

As for 1) yes, I can do that, the reason it is set where it is right now is because very soft keypresses on real pianos with sensorbars installed are typically fingers brushing keys on the way to other keys and these false triggers leave a lot of errors that aren't really errors. I'll make that setting configurable.

2) yes, if you look in the 'midi' directory on the gitlab site ( https://gitlab.com/jmattheij/pianojacq/-/tree/master/midi , but also linked from the application) there are whole bunch of them that all should work well

If you have problematic midi files you can send them to me and I can try to figure out what the problem is and why they will not import the way they should.

Great, I'd appreciate that setting very much. And also thanks for the midi files. I missed visiting the gitlab link, do you mean there should be a direct link to the midi-directory? I can't find that.

The midi files are right on the page linked in the grandparent comment!

Another reply to the same comment, bad form, but there you have it :)

Can you have a look what level you are outputting when the notes are missed? Maybe there is some kind of happy compromise here that would do away with a setting that most people would not understand.

Level? Sorry I don't know midi at all. I sent you the file (it also plays very slow). In this case I don't even need the missing notes. That would just be more complicated :)

Reminds me of the Miracle Piano Teaching System for Mac, PC, Amiga, Nintendo Entertainment System, and other platforms in the early 1990s.


Interesting, I never even heard of it.

Is there a sync issue on your YT video? It seems like the audio and yellow line are not moving together. I'd also consider cropping the videos beginning, better lighting, and better audio quality.

The cursor is moving at indicated tempo, I'm a bit ahead of that but whenever I hesitate it - rapidly - catches up with me. Having the cursor advance at speeds higher than indicated tempo is re-inforcing bad behavior so I have so far not implemented that, though technically it is possible.

Better lighting and better audio are noted, this was just a quick & dirty demo of the program in action with the tools at hand.

My observation is that sheet music doesn't scroll. I spend most of my time sight reading looking ahead for patterns, for breaks in patterns, accidentals, for chords I have to prepare for, and so on. By the time you have to press the notes it's too late to be reading it. My eyes can't keep up with the notes as they stream by.

This may be a very naive question as although I have a good musical ear, I have never learned sheet music (I would like to, but it's difficult to find time).

I'm pretty sure that in the Prelude in C, all those notes have the same length. So then why are the notes at the bottom displayed as whites and those at the top semiquavers?

That's because for the left hand, the white notes are meant to be held for 2 beats. What you are hearing when you say the notes are the same length is that they start every semiquaver. If you see a player's left hand you'll see they get held for the corresponding denoted time.

OK, that makes perfect sense, thanks. But then, a second question: from looking at the sheet, how do you know when the second note in each measure should start? It needs to start in the second semiquaver, but where is that information encoded and how do you know you don't need to wait for the white to finish?

They're sort of represented as voices. The top voice has a rest for half a beat, the bottom voice starts immediately and the middle voice starts after a quarter of a beat.

I see, I understand it better here, where the rests that you are mentioned are pictured explicitly: https://www.gmajormusictheory.org/Freebies/Intermediate/Bach...

In this software they are not shown, I suppose with experience one can omit things from the notation.

Oh that's interesting. Rests are usually are not omitted in sheet music though, what does it look like on your software?

Al-Khwarizmi is right, the rests are not there - yet. The rests are very tricky (see other comment) to get exactly right. I've figured out most of the note timings to be precise enough to render the score accurately but the rests do not have any representation in a midi file, so you have to make them up as you go.

There are other problems like that, such as trills and other ornamentation, which show like a bunch of note on/off pairs in a midi file but as a single note with a decorator in the score. Reversing those is non-trivial, as are grace notes.

If you go to the software in the link (https://pianojacq.com/), "settings", "repertoire", "Bach: Prelude C", you will see it.

Rests are a non-trivial problem, even though they seem to be very easy to solve. The problem stems from the fact that rests have no representation in the midi file, so you need to figure them out. Because midi files can be quite messy if not done perfectly you end up with all kind of spurious rests. So I decided to leave them out for now, but they will be added as soon as I've figured out how to do them well enough that they are not a distraction or teaching people really bad habits. The spacing of the notes should be correct.

This is the single biggest item on my todo list right now, and I wished I had more time to dedicate to this project.

Anyway this is very cool. It made me want to have a MIDI piano here to try it fully. Great work and I hope you find the time to keep improving it!

I'm very much short on time at the moment, in fact, this weekend is the first time in a month that I have some time for myself but soon that will hopefully change and then I will be able to devote much more time to this project and some others that I'm tinkering with.

Does it require a real piano? I'm looking forward to buying a midi keyboard so I could practice with headphones, without neighbors hearing. I still have no idea what actual software to use for this though.

Midi keyboard is enough, no real piano required.

I highly recommend midi keyboard + synthesia

I’m using Brave Browser (which may not support WebMidi), but I get sent to the /Firefox page which tells me about Firefox. Should all chromium derivatives support webmidi?

Yours apparently doesn't, or the test is buggy. Chromium itself works fine. Maybe something you need to enable separately?

Can you try https://sightreading.training/

see if that one works, if it does the problem is on my end.

That's a bummer about Firefox. I've started using Edge and it feels so much faster that I'm starting to wonder if it isn't time to re-evaluate my browser again.

Outstanding, do you know this would work with an Android tablet. I've been able to get midi input via a USB C hub, but it always felt a bit off.

I could always buy a used Surface tablet

When I tested out my web MIDI piano software it worked on a Samsung tablet I tested. Just needed a USB OTG cable to connect it.


I do not know, no Android tablet here. Let me know if it does/doesn't though, assuming the Chrome implementation on Android is close to the regular browser on non-mobile OS's I see no reason why it would not work.

Your listening for Web midi events right ?

I really wanted to create a full midi instrument in the browser , as in I capture camera input and then translate that to midi output. Appears this isn't possible in chrome unless you install software to allow act as a sort of bridge.

Was a real bummer to run into this limitation

Yes, exactly, WebMidi event stream is what drives it, it also generates Midi output to drive the notes that the student isn't playing.

Thanks for making this!

Started playing (or trying to) a few weeks ago and spending almost my entire evening practicing fur Elise.

Neat! Enjoy, and let me know if I can help.

Let’s turn that around: Let me know if I can help! ;)

Well, the source is on gitlab so you can start with digging around in it if you're interested.

I am a blind piano player. Would this be useful in anyway?

Hm, that's a setup that I never even considered, but it poses interesting problems, how could I help to make the program work for you? The 'labels' could be turned into speech probably, but the visuals would be a lot harder.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact