Thanks, yeah I remember when that made the rounds a couple weeks ago. Although what I'm proposing is a little different than what's covered there: the AI designing a language for a particular task, writing the runtime to implement the language, and then solving the task in the language it designed. The blog rather is about how an AI performs with languages designed by people for general purposes.
I've got two experiments running now and I'm going to do a third soon. The first area is linear algebra, where there's a readily available notation for the AI to operationalize. In this area it's very easy for the AI to one-shot write correct algorithms that are shorter than typical languages because they are already written down, and the notation is very compact, so all it has to do is a direct translation from a textbook. I'm now working on comparing with algorithms it doesn't already know.
The second area I'm working on now doesn't have a readily available notation, which is state machines. Here, the AI can concoct a very terse state machine representation and write very complex state machines that can be statically analyzed so it has a better time than writing in a plain language without that capability. Now I'm trying to test how it fares against other state machine DSLs.
The next area I will move to after this I think is music, which also has a readily available notation that AI can operationalize. No numbers to report yet but I'll publish my research when it's done.
