It’s a process of transition between two distinct states. Say a bit flips from 0 to 1; there is a short time period during which that bit is physically “in transition”, but it would be incorrect to say that the bit has a value of, say, 0.5 during the transition.
Also remember that quantum mechanics is a way to make statistical predictions of outcomes of certain experiments; it does not claim to explain what is actually happening underneath.
I strongly disagree with your second paragraph. Quantum theories do, we think, correspond to what is actually happening underneath. They might not be perfect correspondences, in that our theories are incomplete or approximate, in the same way that Newtonian gravity is an approximation of General Relativity in the weak-gravity regime.
Bell's inequality places a strong constraint on what sort of physical theories can explain quantum phenomena. If you believe in locality (the universe has no global variables, and information propagates outwards through local interactions), then the wavefunction is a real thing, and the actual state of the universe.
The formalisms of quantum mechanics are widely accepted, but as you know, there is considerable disagreement about what may actually be happening underneath:
Yes, there is disagreement about what is happening underneath, but whatever your interpretation, it is highly constrained by Bell's theorem. Quantum mechanics has to be more than just a statistical description of some underlying deterministic process, unless you are willing to throw out locality.
I'm highly partial to the Many Worlds Interpretation, as I think it's the only interpretation that takes quantum mechanics seriously. If you assume that quantum mechanics describes both the system you're studying and the measurement apparatus (including the scientist taking the measurements, and the rest of the universe), then you're led inexorably to the Many Worlds Interpretation.
So this is what I do not understand about Bell's Theorem: it seems to dismiss in the assumptions the possibility of determinism (I don't understand where the super comes from, as there is no distinction) and then goes on to conclude "Hey, there must be fundamental randomness". Didn't we just assume the conclusion?
It does not assume from the outset that there is no determinism. It sets bounds on the types of results that you can get with deterministic local variables. Those bounds are violated in the real world. Hence, the universe is either quantum mechanical, or there exist hidden nonlocal variables that give the illusion of quantum behavior.
There is a way to escape the inference of superluminal speeds and spooky action at a distance. But it involves absolute determinism in the universe, the complete absence of free will. Suppose the world is super-deterministic, with not just inanimate nature running on behind-the-scenes clockwork, but with our behavior, including our belief that we are free to choose to do one experiment rather than another, absolutely predetermined, including the "decision" by the experimenter to carry out one set of measurements rather than another, the difficulty disappears. There is no need for a faster than light signal to tell particle A what measurement has been carried out on particle B, because the universe, including particle A, already "knows" what that measurement, and its outcome, will be.
I think it’s useful to be clear about the dividing line between demonstrated scientific results and our scientific intuitions, even when those intuitions are driven by observed patterns that have been reliable in the past. Intuitions are excellent drivers for formulating new theories and experiments, but it is epistemically dangerous to conflate beliefs and knowledge in our minds.
> Suppose the world is super-deterministic, with not just inanimate nature running on behind-the-scenes clockwork, but with our behavior, including our belief that we are free to choose to do one experiment rather than another, absolutely predetermined, including the "decision" by the experimenter to carry out one set of measurements rather than another, the difficulty disappears.
You see, I think Bell is kind of obfuscating here: this is just normal determinism and it seems to me like Lagrange would have been perfectly fine with this. Causality only, even for the atoms that happen to reside in a human brain.
Edit: what do you mean with your last paragraph? Because I interpret it like this: the belief that interferes is that we have free will and can choose and basically change the past (counterfactuals). But I have a hunch that you mean it like: QM is weird, be careful.
My understanding is that the “super” in superdeterminism is just making clear that we are considering the experimenter completely determined as well, but yes, they are essentially the same thing.
Last paragraph I mean what you interpret — our everyday experience of “free will” may blind us to certain possibilities that we have no strong evidence for or against, just as our everyday experience of a classical world makes QM seem “weird” and unintuitive, even when the experimental evidence is well-established.
There’s a particular curiosity when the possibility of a world without free will calls into question the extent of the power of science itself. Are there other techniques that can provide satisfying arguments about the potential nature of reality in a world where we cannot rely on the power of experiment to reveal this nature?
I think it does call into question the power of science, but also: assuming non-determinism does not give us this power back! If materials (atoms / humans) can arbitrarily disregard the laws of nature by making choices (however that would be implemented), what does an experiment even mean?
Edit:
This is a relevant theorem, that nicely complements Bell:
"it would be incorrect to say that the bit has a value of, say, 0.5 during the transition."
In what sense? It has a <z>=0, meaning there's a statistically 50%/50% chance of measuring 1 or 0 but the system is in principle coherent.
This means that half way through the jump, if you rotated it 90 degrees, you'd measure a _definite_ z value of -1 or 1, or you'd measure a bit value of 0 or 1, depending on whether you twisted it clockwise or counterclockwise.
In the sense that it ignores that coherence. Any state on the disc of the Bloch sphere's equator has <z> = 0, and 50:50 chance of measuring 1 or 0. If you want to assign a real value of 0.5 to a bit, then the only one that makes sense is the center, completely decoherent mixture.
With your second paragraph firmly in the back of my mind:
In the sister comment thread, two photon excitation seems to indicate that this intermediate state (0.5 in your coin example) is not invalid. Can you help me understand better?
Edit: Also, I think your coin "in transition" implicitly assumes that the system has more degrees of freedom than 1 bit (You can only flip a coin in 3 dimensions).
Also remember that quantum mechanics is a way to make statistical predictions of outcomes of certain experiments; it does not claim to explain what is actually happening underneath.