There have been multiple studies on how LLMs add numbers.
They use the "Clock" algorithm, the "Pizza" algorithm, a few other ones.
> All networks we study implement the same simple neuron model in their first-layer MLPs: degree-1 sinusoidal fits in layer 1, with deeper layers combining into degree-2 sinusoidal interactions.
> did you use the "cos" or "sin" function at all during this addition computation?
> No. There is no need for trigonometric functions like sin or cos. The computation only uses basic arithmetic and place-value reasoning. Of course, if someone were implementing arithmetic in a computer, it is theoretically possible to express addition using extremely complicated formulas involving sin and cos. But in the reasoning I described, no trigonometric functions were involved at all. I simply decomposed the numbers into hundreds and smaller parts and added them.
They use the "Clock" algorithm, the "Pizza" algorithm, a few other ones.
> All networks we study implement the same simple neuron model in their first-layer MLPs: degree-1 sinusoidal fits in layer 1, with deeper layers combining into degree-2 sinusoidal interactions.
https://neurips.cc/virtual/2025/loc/san-diego/133808
https://arxiv.org/abs/2502.00873
When you ask them they don't mention these at all, they give you high-school math:
https://chatgpt.com/share/6a82afdd-872c-83eb-aad7-622d27f2dc...
> did you use the "cos" or "sin" function at all during this addition computation?
> No. There is no need for trigonometric functions like sin or cos. The computation only uses basic arithmetic and place-value reasoning. Of course, if someone were implementing arithmetic in a computer, it is theoretically possible to express addition using extremely complicated formulas involving sin and cos. But in the reasoning I described, no trigonometric functions were involved at all. I simply decomposed the numbers into hundreds and smaller parts and added them.