Arnold Schoenberg observed the deep connection between music, perception, and cognition; anticipated composers applying timbral control as is now commonly done with synthesizers; and apparently did not anticipate non-subtlety of the effect in EDM:
"I think the tone becomes perceptible by virtue of tone color [timbre], of which one dimension is pitch. ... Pitch is nothing else but tone color measured in one direction. Now, if it is possible to create patterns out of tone colors that are differentiated according to pitch, patterns we call 'melodies', progressions, whose coherence evokes an effect analogous to thought processes, then it must also be possible to make such progressions out of the tone colors of the other dimension, out of that which we call simply 'tone color', progressions whose relations with one another work with a kind of logic entirely equivalent to that logic which satisfies us in the melody of pitches. That has the appearance of a futuristic fantasy and is probably just that. But it is one which, I firmly believe, will be realized.
...
Tone-color melodies! How acute the senses that would be able to perceive them! How high the development of spirit that could find pleasure in such subtle things!"
-- Arnold Schoenberg, Theory of Harmony, 1911, toward the end of chapter 22.
Personally, I anticipate that if humanlike AGI is achieved, then it will understand the illusion of music. Not merely because music understanding is an attribute of humanlike, but because there must be something intrinsic in the process of human cognition that causes the illusion of music occur. Music understanding could be viewed as an essential waypoint on the path to AGI.
I had a simple thought a couple years ago that still fascinates me. If you do something as simple as "clap clap clap" (with equal measures of time), you can naturally relate it to any other set of sounds with that simple pattern (ABABA). moreover, there is a 1 to 1 relation of this abstraction in space, where one can represent the sounds as marks on a screen (as I have done here for example, as "clap clap clap, X X X, - - -, or whatever).
Despite the different sensory systems (sound, or visual) and the difference in actual data (I don't know how many photons or audio waves entering your brain), we can abstract them down and understand/relate them with a simple pattern.
It begins by talking about this, maybe that's a good start! It documents that they are able to transmit complex messages like jokes and proverbs entirely through rhythm!
Yes, but its not just music. Our senses are very much linked in that we can build a one to one relation of ideas in visual space to auditory space and vice versa. There is a common abstract model we understand so to speak.
Our model of perception is very finely tuned with that of space-time.
Pop music really does do this as he described. There are lots of songs that play with expectations of timbre in the same way that normal music plays with expectations of pitch.
When I get mentally crystallized, I hear music as proof-of-work in the vein of Bitcoin hashes.
I very much appreciate your comment, and at the same time am interested to challenge this idea of "the illusion of music". What about music is an illusion? Maybe this was just an unfortunate phrasing, and yet it might touch on the heart of the matter. Music is a language of relationships, indeed it is nothing but relationships, in time, pitch, timbre, volume, voice, number and so on, non-symbolic and changing relationships of sound carrying aesthetic and emotional feeling and meaning. It is both objective and subjective -- objective in the sense that "4 + 4 = 8" is an undeniably more beautiful statement than "4 + 4 = 6", and subjective in the sense that you and I may have different favorite numbers and also we may react differently to various medications or prefer different foods. Music is external and internal, social and individual. What it is not ever is an illusion as then it would not exist. (It may express illusion, or become more like a creator of illusions, for example when the orchestra tries to conjure up the illusion of a thunder storm.) Now, one might be tempted to counter that the kind of "meaning" music carries is itself an illusion, an epiphenomenon maybe, but then why should one care about anything at all, including AGI, since the kind of meaning that music carries is the same as the meaning of love, of hate, of good and bad, pains and pleasures, and hopes and regrets, tragedies and successes?
Maybe a reason for calling music an illusion is to try to point out that it's all in our heads. Sure, the physical instruments are over there, but they are just some atoms and molecules rattling about creating vibrations that, like everything ultimately, share the meaninglessness of the physical universe of just more material stuff (ignoring the quantum and the unknown). In which case though life seems also to look more like an illusion. Ah but the meaningless vibrations enter our ears and are processed and become music, so like color there is none but in our heads, thus illusion? But these notions also fall apart. The redness of apples isn't all in our heads; it requires a certain relationship to light frequencies that we see as redness. It's not an illusion it's just what red looks like through a living human body. If not through our senses and brain, and then mind, where should we experience it?
Music is only ever meaningless if it is unheard, as apples can only not be red when they are unseen (or are Granny Smiths).
So music is only meaningless if there is nobody -- no body -- to listen. And maybe that's where all of this leads. Your AGI needs to be embodied and alive.
My use of illusion was not intended to diminish the significance of music. To the contrary, I meant to distinguish it as an exceptional perceptual phenomenon which reveals important aspects of the inner workings of human experience.
Somehow, the physical situations of our bodies become perceptual experiences. Usually, those perceptions are mere descriptions of the physical world, for physically navigating it. Some exceptions exist, such as music and optical illusions. The translation from air pressure fluctuations to a tree branch falling is one type of experience, the link to love or hope is another. Rather than being insignificant, my sense is that music understanding will prove to be an emergent capability available in whatever architecture might produce humanlike AGI, because it connects human perception and cognition too thoroughly to not be baked-in. Going further, building models to understand music would be an excellent step toward embodied AI with humanlike cognition.
This is something I've been thinking about the last 3-4 years and desperately what I am trying to do.
I have what I think some interesting ideas and implementations, but no one quite grasps my thought process and I am stuck between believing Im onto something and that I am furiously chasing nothing.
My experience in pursuing this is that I applied to about a dozen on-campus CS grad programs, over the course of two years, noting this research direction, but those applications were all rejected. You would think with the emergence of generative AI that there could be some interest. I suspect the application was of interest, but that universities have a shortage of experts in generative AI, and that a multidisciplinary project like this might be misunderstood within a CS department or would have to be accepted by a very specific advisor who may not exist. The best chance might be to convince investors to help assemble the right team and to provide a viable R&D budget.
Oh wow thats nuts. Did you end up attending grad school for something else? Ive been thinking about applying but I haven't even able to find any departments that do research in this area. There is a huge craze around ML as its producing tangible results and it seems the school of thought is that anything else is wasted time.
The closest thing Ive found to my own is Wolframs writings.
Seriously I know I seem like a delusional nut but this train of thought has really grabbed me. Ive written hundreds of pages of (mostly intelligible) notes/figures and spend countless nights trying to implement what I can so clearly see in my head.
Unfortunately I dont have anything concrete yet that would warrant building a team, and its hard to even find people I can discuss my ideas with.
I think there ought to be interest in this area in mathematics, philosophy or even linguists though.
"I think the tone becomes perceptible by virtue of tone color [timbre], of which one dimension is pitch. ... Pitch is nothing else but tone color measured in one direction. Now, if it is possible to create patterns out of tone colors that are differentiated according to pitch, patterns we call 'melodies', progressions, whose coherence evokes an effect analogous to thought processes, then it must also be possible to make such progressions out of the tone colors of the other dimension, out of that which we call simply 'tone color', progressions whose relations with one another work with a kind of logic entirely equivalent to that logic which satisfies us in the melody of pitches. That has the appearance of a futuristic fantasy and is probably just that. But it is one which, I firmly believe, will be realized.
...
Tone-color melodies! How acute the senses that would be able to perceive them! How high the development of spirit that could find pleasure in such subtle things!"
-- Arnold Schoenberg, Theory of Harmony, 1911, toward the end of chapter 22.
Personally, I anticipate that if humanlike AGI is achieved, then it will understand the illusion of music. Not merely because music understanding is an attribute of humanlike, but because there must be something intrinsic in the process of human cognition that causes the illusion of music occur. Music understanding could be viewed as an essential waypoint on the path to AGI.