Different threats require separate considerations for sure.
To be crystal clear. I'm still for secret voting, and being able to lie about your vote if you want to. But without an ability for the voter to verify their vote, you must trust the entities themselves that are holding the election. Both that they are acting in good faith. AND that they managed to secure the election against outside tampering. The very people that you are worried might compel you to declare your vote are the ones running the election systems themselves in many situations.
We need to be able to operate with less trust here, not more.
> If each one has a secret passphrase, nobody can verify that the total is calculated correctly.
It's possible we are misunderstanding each other. There would be a verifiable ledger. With opaque tokens for each vote. The total can be verified by counting. Just like normal. We could use our signature method of choice to sign and verify the integrity of each vote and all the votes. The body holding the election would be able to verify the total counts are correct and not tampered with.
For a specific vote, an opaque identifier that nobody except the voter can resolve, provides a mechanism for the voter to self verify their vote was counted in the way they expected.
A passphrase was just one idea to avoid printing the token on your vote receipt. But if we really want to go down the rabbit whole of having cover. There are many other ways to provide plausible deniability. You could opt to not get a print out of your token. And your deniability would be you don't have it, and you can simply lie about which vote is yours (even though you know the one that is yours).
> It was a long fight to get secret votes
I'm still saying we keep voting secret. What we are discussing is the ability for a voter to verify their vote was counted. But it's still meant to be secret. In fact, something analogous to the 5th amendment to the constitution could help enshrine the right to a private secret vote as a fundamental right.
> Just imagine that during McCarthyism people that were requested testify in the committee has to first say their national ID, token and passphrase to be sure that they didn't vote for the Communist party.
We make it a constitutional right to have your vote be secret. Make this clearly illegal. If you are worried about the central government not obeying laws, then nothing really helps you. The central government ultimately wields the final say in all matters here. They can put you to death if they like. A verifiable election system is meant to help ensure we never devolve to a government that does what you are worried about.
Ultimately all of your examples about being forced to declare things apply also to your credentials to your personal devices and online account. All of which contain more less enough information to both figure out what your vote was, and much more.