Hi HN!
Testing the limits of current models that in theory are better coders than us (faster for sure), looks like they had a hard time understanding the machine code that will build their future AI brains!
Looks they can’t build “simple” good executable bytes (pure machine code), tested so far for windows 64, will test for linux & mac and mb uefi too…
The idea is simple generate the bytes for an exe file that displays “hello world” and so far all of them failed, I tested OpenAI GPT o1 PRO, GPT 4.5, GPT o3-mini-hi, Grok 3 (think) Claude 3.7 sonnet (extended), Gemini 2.0 pro, fancy new Inception Labs Mercury Coder, Deepseek r1 670b, just released Qwen QWQ 32b, llama 3.3, Mixtral LeChat, and the ones that create the exe file, none executes.
This is the prompt: (the last part is to avoid endless hex 00s all over)
“Create the bytes necessary to build the smallest exe file that could run on a windows pc 64 bits, and display the text "hello world", and embed the bytes in python function that creates the exe file, do it with the minimum possible lines of code so in python generate the bytes wisely for example 100 ceros is wrong you should write a loop with 1 cero outputted 100 times and so on. you can't use any other language, no c, assembly or anything only python”
Any suggestions?
Can you build a better (short) prompt that works?
Maybe it is another benchmark to add? (go to the real "source" beyond "simpler" languages)
This is btw part of the daily artificiology research to measure limits and real achievements of Ais for the E-AGI 144 Barometer metrics.
reply