It's actually how you decrypt an anonymous OpenPGP message (e.g. in GnuPG, you can create one using --throw-keyids or --hidden-recipient) - Normally an OpenPGP message records its intended recipients in the header, so GPG knows which key to use right away. But when the message is anonymous, GPG must try all the private keys in the keyring one by one, until it sees a valid solution or fails. If you have multiple private keys, you'll go through many passphrase popups (and smartcard/USB swapping), the struggle is real!
Trial decryption. This is a potential server behaviour for Encrypted Client Hello (the current iteration of the work to encrypt SNI in TLS traffic) too
ECH will be GREASEd. To prevent those who might want the capability to stop ECH in the future from getting a head start while it's uncommon, implementations would always pretend to be doing it anyway.
So talking to any TLS server, even one that has no idea about ECH, the client says basically "Hi, here is a normal unencrypted TLS 1.3 Hello message for this.server.example, also, here's an Encrypted Client Hello message". If the server actually does offer ECH, there could be a real Client Hello, perhaps addressed to another.server.example, encrypted inside the Encrypted Client Hello, but if not there's just random noise. An eavesdropper doesn't have the key, so they don't can't tell which is the case.
Obviously if your server can't do ECH, the Encrypted Client Hello is just a mysterious unintelligible extension with noise inside it, no further inspection needed.
And in some setups the server knows how to tell easily which key would have been used for any valid ECH, so if that key doesn't work then it was just noise, and can be ignored.
But in other cases the server knows two or more keys that might be valid, yet the client either can't or has chosen not to be open about which (if any) was used, so the server has to try them all until it finds out.
Simple, you just check whether the header, data structure, metadata, checksum, etc. in the trial-decrypted data in valid, no hack is needed. For example, something as simple as a 16-byte magic number can ensure a false positive rate of 1/2^128. Since any arbitrary binary data can be encrypted, "this is mostly ASCII" is unusable.
In GPG:
/* if KeyID is empty... */
if (!k->keyid[0] && !k->keyid[1])
{
log_info (_("anonymous recipient; trying secret key %s ...\n"), keystr (keyid));
}
err = get_it (ctrl, k, dek, sk, keyid);
k->result = err;
if (!err)
{
/* If get_it() succeeds */
if (!opt.quiet && !k->keyid[0] && !k->keyid[1])
{
log_info (_("okay, we are the anonymous recipient.\n"));
}
}
And in get_it()...
if (sk->pubkey_algo == PUBKEY_ALGO_ECDH)
{
/* Now the frame are the bytes decrypted but padded session key. */
if (!nframe || nframe <= 8
|| frame[nframe-1] > nframe)
{
err = gpg_error (GPG_ERR_WRONG_SECKEY);
goto leave;
}
}
else
{
if (padding)
{
if (n + 7 > nframe)
{
err = gpg_error (GPG_ERR_WRONG_SECKEY);
goto leave;
}
}
if (n + 4 > nframe)
{
err = gpg_error (GPG_ERR_WRONG_SECKEY);
goto leave;
}
if (dek->keylen != openpgp_cipher_get_algo_keylen (dek->algo))
{
err = gpg_error (GPG_ERR_WRONG_SECKEY);
goto leave;
}
/* Copy the key to DEK and compare the checksum. */
csum = buf16_to_u16 (frame+nframe-2);
memcpy (dek->key, frame + n, dek->keylen);
for (csum2 = 0, n = 0; n < dek->keylen; n++)
csum2 += dek->key[n];
if (csum != csum2)
{
err = gpg_error (GPG_ERR_WRONG_SECKEY);
goto leave;
}
}
You get the idea.
BTW, did I just "break GPG encryption and deanonymize users" according to Cellebrite because I found the correct function in the code? And no, I didn't "review dozens of code classes", I just grepped the word "anonymous"... /s
