Yes. Suppose you ask me what the sqrt(4) is and I tell you 2. Accurate and correct, right?
Does it matter if I answer every question with either 1 or 2 and flip a coin each time to decide which?
Deterministic means that if it is accurate/correct once, it will continue to be in future runs (unless the correct answer changes; a stopped clock is deterministic).
I think the analogy breaks down here. The elided bit "time indicator" implied at the end makes that statement is false. A stopped clock is not a deterministic time indicator.
If the correct answer changes, a (correct and accurate) deterministic model either gets new input and changes the answer accordingly, or is not correct to begin with.
Determinism is unrelated to correctness. Deterministic means the output depends only on the state you consider to be relevant, and not other factors. A stopped clock is deterministic: no matter what you do, it gives you the same output. A working, accurate clock is deterministic if you consider the current time to be a relevant piece of state, but not if you don't. Consider how "deterministic builds" need to avoid timestamping their build products, because determinism in that context is assumed to mean that you can run it at a different time and get the same result.
LLMs can be deterministic if you run them with a temperature of 0 or a fixed random seed, and your kernel is built to be deterministic, but they're not typically used that way, and will produce different output for identical input.
I never said it is. That's why I qualified my example with the word correct.
> no matter what you do, it gives you the same output
This is not deterministic. This is determined. I think this is the confusion I was pointing out.
>> Deterministic means that if it is accurate/correct once, it will continue to be in future runs (unless the correct answer changes; a stopped clock is deterministic).
The bit in the parenthesis, I am trying to argue, is nonsense. If the correct answer changes, the system is not accurate or correct to begin with so the point is moot. Correcting the system will make it accurate. A stopped clock is not deterministic, it's determined. As a time indicator, a stopped clock is not a correct, accurate or deterministic model at all under any possible interpretation.
You pretty clearly think determinism and correctness are related, otherwise why wouldn't a stopped clock be deterministic?
Determinism is about the behavior of a system. Correctness is also about the purpose of a system. A system can have deterministic behavior while being completely unfit for its purpose. And depending on its purpose, it can be fit for purpose while being nondeterministic.
You still seem to see correctness as a prerequisite for determinstic. I’m open to that idea but I really don’t think it’s the case.
I build a box. It has an LCD display. It has a button labeled “what time is it”. You push the button and it always shows “10:43am”. This is a deterministic system.
That depends. If the problem has been solved before and the answer is known and it is in the corpus, then it can give you the correct answer without actually executing any code.
Is it not generally true? If the information (i.e. problem and its answer) exists in the model's training corpus, then LLMs can provide the correct answer without directly executing anything.
Ask it what the capital of France is, and it will tell you it is Paris. Same with "how do I reverse a string in Python", or whatever problem you have at hand that needs solving (sans searching capability, which makes things more complicated).
So does not the problem need to be unique if you want to be able to claim with certainty it indeed has been executed? I am not sure how you account for the searching capability, and I am not excluding the possibility of having access to execution tools, pretty sure they do.
