Could you give an example of an apples-to-apples comparison between (symbolic) program synthesis and LLMs that the latter wins?
The reason I am asking is because, in my experience, LLMs never match the 100% precision (relative to the specification that is synthesised against), which program synthesis effortlessly matches.
On the apples-to-apples comparisons, LLMs often only obtain 70% - 80% precision, which means they are effectively solving a simpler synthesis problem. (Yes they have other benefits, like being able to extract knowledge from natural language, which conventional synthesis can't do, but that's offen not needed.)
The reason I am asking is because, in my experience, LLMs never match the 100% precision (relative to the specification that is synthesised against), which program synthesis effortlessly matches.
On the apples-to-apples comparisons, LLMs often only obtain 70% - 80% precision, which means they are effectively solving a simpler synthesis problem. (Yes they have other benefits, like being able to extract knowledge from natural language, which conventional synthesis can't do, but that's offen not needed.)