On: Claude: System Prompts

[TZubiri]

Cached.

they are the first part of the input and it contains no user dependent variables, so the model is in a known state that it can reuse across all users, it does not need to recompute all that inference

[cubefox → TZubiri]

Unless they are using a linear architecture, the compute cost still scales O(n²) for n tokens, and nemory cost scales O(n).

[TZubiri → cubefox]

>the compute cost still scales O(n²) for n tokens,

That is never the cost, it's a common misconception.

Cost scales linearly per tokens. Unless you are sending one token at a time and avoiding using the same machine or cache.

Just look at api charges, they are charged by token, not by token squared.

[cubefox → TZubiri]

Which seems to contradict the usual consensus that purely linear architectures are not sufficiently capable and unsuited for frontier models.