Is there any way to know if my Gmail message will be protected by StartTLS before I press 'Send'? That is, is there a Google API that takes the recipient email address and tells me if StartTLS will be used?
It's not possible to know if STARTTLS will be used before you send a message. Because the sending system will only use STARTTLS if it's advertised during delivery.
Unless both the sending and receiving system support DNSSEC+DANE. GMail doesn't support DANE.
A while back I documented the steps necessary to encrypt GMail using Thunderbird on Windows. Does anyone have a working list of instructions for encrypting e-mail that are significantly simpler?
In other words, how can the number of steps to achieve desktop e-mail encryption be reduced to the point where the majority of users can accomplish this task autonomously--and what are those steps?
There's only a passing mention of end-to-end encryption. I'm concerned that email content encryption would break basically all anti-spam technologies... Here's some previous discussion: https://news.ycombinator.com/item?id=8275970
Full end-to-end encryption means you can validate the identity of the sender. Once you can do that blacklisting bad actors or whitelisting known-good actors is trivial.
Even hijacking accounts in this model for the purpose of sending spam doesn't work unless you can also gain access to the private key, which itself would be non-trivial. Encrypt that private key with a strong passphrase and you're in even better shape.
Encrypting and authenticating are two separate acts. Mutt with GPG/PGP support makes this clear -- your options are to sign (authenticate + integrity) and/or encrypt a message (you can do either, both, or none).
It's possible that you'll still be able to do header checks, though if people move increasingly to onion routing their Internet traffic, a lot of source-based validation (IP checks) will fail.
A great deal of spam detection works on noting the origin and contents of messages, and yes, absolutely, encryption makes that harder.
It does, however, also increase the costs for the spammer. And there might be other ways to improve protocols. But it's a challenge.
I'm quite aware that encryption and authentication are separable elements in a PKI cryptosystem. But I don't believe that invalidates my point.
Assuming a PKI cryptosystem, I can (and I contend should) greylist all emails that aren't signed by senders in my whitelist.
Again, I accept I could be missing something here. Maybe that model doesn't scale effectively or something. I don't know, I admit I haven't thought the use cases through that thoroughly. It certainly adds a burden on the receiver side, but it works (unlike today where whitelists can be defeated by forging the sender address in the email header).
Of course, it's true that the original comment is fundamentally correct: encryption basically obsoletes existing anti-spam technologies if the payload is encrypted. It just seems to me that e2e crypto provides the tools for a whole different range of anti-spam options that aren't possible today.
End-to-end encryption does not invalidate most existing anti-spam technologies. However, these technologies will need to run after decrypting the content, which may mean on the end user's machine(s). Less processing horsepower will be available than with server-side techniques, but the same techniques can still apply. If your mobile device and desktop PC both possess your decryption key, then your desktop PC could act as the anti-spam engine and transmit a list of good messages to your mobile device.
A bigger problem would be ISPs who wish to keep their anti-spam techniques confidential and evaluate them only on their own servers. Allowing the end-users to download the scanning logic and run it locally would create a risk of divulging their proprietary techniques. The ability to keep software truly confidential and under its creator's control is one of the motivating factors behind the rise of software as a service.
Whitelisting can also go a long way toward allowing only preapproved authorized senders to communicate with you using end-to-end encryption. The remaining senders might communicate with messages that are encrypted to your ISP instead. The DIME protocol anticipates this sort of layered encryption: https://news.ycombinator.com/item?id=8816806 When someone wishes to communicate with you using end-to-end encryption, they might first send an "introduction" message encrypted to your ISP, which can be scanned. Then the receiver can decide to authorize the sender for E2E or not.
⚫ It's expensive. Far cheaper to block in TCP or SMTP sessions.
⚫ If you cannot make and SMTP-time determination, you're going to end up with a nondelivery status message rather than an SMTP bounce. If you remember the days when these were common, the principle problem is joe-jobbing users with spoofed sender addresses. One option is not to send the NDS, which means that email becomes less reliable (than it already is).
⚫ TCP/SMTP time adjudication means you can leverage information in realtime. Waiting until keys are available means that the spammer has more time to spam before being detected (and possibly reported to spam-detection systems).
⚫ Receiver needs to spool potentially large volumes of spam.
None of these are killers, but they do change the game.
I'm leaning toward systems with 1) pervasive whitelisting, or 2) hold-and-notify, or 3) ticket-based permissions. Several of these would require extensions to or replacement of bits of present SMTP.
Pervasive whitelisting means that you've got an approved sender list, and non-whitelisted mail is either rejected or deferred. Deferral might well be sufficient to avert much spam.
Hold-and-notify would be a radical change. A system sending mail would effectively poll receiving servers "hey, I've got a message for you" (possibly with metadata), and the _receiving_ system would then poll the sender to fetch the message. This allows receiving systems to pre-fetch any verification / validation data it needs to assess message quality. It also puts the burden on the sending system to both 1) hold outgoing messages for retrieval and 2) persist online (many spam boxes are booted fairly quickly). As a bonus, nondelivery will be easier to assess (if your message isn't requested, it's not delivered). I don't see this being compatible with SMTP at all.
Ticket-based permissions would require a sending system to request a ticket -- essentially an allowance to be able to deliver mail -- from the receiving host. It's a form of host-based whitelisting. This allows receiving hosts to allocate specific access levels / quality of service to specific hosts. Tickets could (and likely should) be PKI-based (to prevent forging / transfer), and would allow for a specified number of messages to be delivered within a given timeframe, after which a new ticket would have to be requested. No tickey, no laundry.
This sounds in principle similar to DIME.
Other concepts include wrapping or encrypting various parts of a message to different levels. Headers could be encrypted to the MX key of the receiving host, so that they would not be readable except to the server (and ultimate recipient). This is effectively what SSL accomplishes, though in a live-on-the-wire rather than static encryption model.
It makes receiving more expensive, but it also makes sending more expensive, and it's the people doing so in bulk that are going to feel that the hardest. I don't see how this makes things worse.
I should have clarified my response above somewhat: an encrypted message is not necessarily verifiable as to sender, it is only if it's signed. Which encrypted email frequently is. How it's verifiable itself affects the security of the message -- an externally visible signature hash is stronger metadata than even the usual headers on email. Though that wasn't the thrust of your comment.
The thing about spam is its volume, and the cost (time, compute resources) required to defeat it.
Rejecting message at connect time (simply based on IP) or during SMTP negotiation imposes the lowest costs. DNSBL lookups are sufficiently expensive that they're typically deferred until _other_ checks are made -- running a half-dozen or dozen IP lookups takes time, though if you can manage that in parallel with other messages you're OK. Content-based checks are even more expensive, depending on how you run them.
And if your email's encrypted, you can't run them at all.
Signatures can be checked, but that costs some time (I get 0.006s for a 'gpg --verify' vs. 0.001 for 'cat' on a file consisting of "Hello, world.") -- and you need the signature present to be able to verify it. Searching and retrieving sigs can take several seconds or more.
If only a small quantity of email is encrypted, it's likely a good signal of non-spam mail, and the costs of processing it are low. If encryption becomes common, and spammers make use of it, that could change.
Though the upside to encryption is that it increases spammer's costs as well. They've got to find and use the right key for each recipient. Where this is worthwhile it might happen, though it likely often won't.
While I've generally defended SMTP in the past, I'm starting to think it's time to really explore alternatives. Open alternatives, not the "here's my proprietary solution to the email / messaging problem" crud that's been popping up every six months for the past fifteen years.
There's a thorough overview of current techniques and how they would be invalidated by encryption in the post I linked to. I agree that e2e crypto would enable new techniques, but that wasn't my point.
I think I was caught up by the word "all" in the phrase "would break basically all anti-spam technologies". I read that as implying that e2e encryption makes managing spam impossible.
Seems clear that wasn't the intent, so... yeah, have a happy new year!
> I'm concerned that email content encryption would break basically all anti-spam technologies
I've self-hosted mail for years and only received 2 or 3 spam messages in all this time. I ignore content analysis entirely, instead focusing on:
* DNSBLs such as Spamhaus, and
* DROP and eDROP (from Spamhaus), and fullbogons (from Team Cymru), and
* FCrDNS + whitelisting for legit senders lacking it, and
* block certain "blackhat" DNS providers, and
* do "nolisting" (see: nolisting.org), and
* (either) greylisting, such as OpenBSD's spamd,
* (or) greet delay + whitelisting, if not using greylisting
Requiring FCrDNS exposes a large number of incompetent senders, making it an "extreme" method that entails some babysitting. Requiring rDNS is good, and far less likely to cause issues.
Employing the above methods reduces spam so dramatically that the benefits of E2E encryption FAR outweigh the drawback of lacking content analysis.
