In the article, I make the case that they can reason but it is in many ways distinct from the way we do it. It is also learned from very different fundamentals. Our reasoning fundamentals come from sensing, and experiencing. LLMs reasoning fundamentals are pieces of words.
We roughly go from sensing, to object permanence, to algebra.
They go from tokens, to grammar, to abstract ideas.
Human senses, inertia, gravity, and other things that are elementary to us, are complex abstract phenomena to them. They can only think of them in very hypothetical terms.
There are some clear examples of reasoning mentioned in the article, but I believe that a large part of the perception of them lacking it, stems from them having unnaturally good writing skills in relation to their reasoning skills.
That is, no human that can write this well is this bad at reasoning.
We roughly go from sensing, to object permanence, to algebra.
They go from tokens, to grammar, to abstract ideas.
Human senses, inertia, gravity, and other things that are elementary to us, are complex abstract phenomena to them. They can only think of them in very hypothetical terms.
There are some clear examples of reasoning mentioned in the article, but I believe that a large part of the perception of them lacking it, stems from them having unnaturally good writing skills in relation to their reasoning skills. That is, no human that can write this well is this bad at reasoning.