If you have to conflate programming language theory with linguistics to make an argument, it's not a good argument.
Because you can strawman all you want, but you can't change the fact that there's no well defined behavior regarding what happens when you instruct LLMs to make a program that calculates 2 + 2. What's stopping it from creating index.html with 5 in it as a response?
Are you claiming that the natural language of the LLM output (e.g., English, Chinese) does not have semantics?? Someone should tell all the people cited at https://en.wikipedia.org/wiki/Formal_semantics_(natural_lang...