I've noticed that "if it's not plaintext, it gets deleted without being read" seems to be a pretty common rule among Germans on the Internet, who also have a tendency to like specifying very exactly what they want of email to them. Here's a few examples:
I feel like I'm making this comment once a week on HN but I host my own email and I haven't had any major issue so far with "big email".
The main caveat is that you will have a very hard time getting your email accepted if it comes from a home connection IP range instead of some host provider but if you do have a dedicated server and follow the guidelines (SMTPS, DKIM, SPF etc...) it just works, at least in my experience.
Hosting one's own e-mail server is a totally opaque random crapshot. You may
not have any trouble, but some other dude or gal will get their e-mail marked
as spam without any way to tell what exactly is wrong and what to change.
I've been running a personal mail server for twice as long with simply
following those rules and my e-mail is still tagged as spam in Gmail when
it's me who initiates contact (once the other party sends me an e-mail, reply
or otherwise, I no longer get tagged as spam).
I'm in a similar boat. I've moved ISPs a few times throughout the years, and each move it takes a few months to finally get things settled. I'm in the US, and had the best luck for deliverability when I was on a Comcast Business account.
My current ISP, a local fiber provider, was not great getting going. Most of the IPs that they have are in at least 1 spam database, and it took a while for the ISP to reach out to the database maintainers themselves. Even then, since they're a small ISP, the IPs are still blacklisted. The ISP wasn't even a company when the IPs were added to these blacklists.
After a few months they were able to assign me an IP that wasn't in a blacklist somewhere. I still randomly have issues with the big providers though - gmail is probably the most annoying. Like dozzie, my SPF, DKIM, and DMARC setups are all valid.
Overall, I really enjoying running my own mail server. Every now and then there are a few annoyances, but it's worth it in the long run.
Hmm well, I've found it to be pretty straightforward, I'm curious why this is happening to you though and there must be a solution...
I mean.. If you get a solid result from dkimvalidator but google is still shitlisting you then I'd definitely consider moving to another host/dc/isp at least.
Depending on your size, it might be best just to make this someone else's problem (if you can) -- like google, o365, etc..
The real problem is Google is opaque in dropping such mail in recipients spam folder. Hotmail can sometimes be annoying in this regard too. As far as I've managed neither does the sane thing and add a header that report on how they've divided that "this mail is spam" - so the recipient doesn't have anything to go on either, other than hoping "mark this as bot spam/add to address book/send email/reply to address" help Google/Ms treat it as "not spam".
Maybe it's worse for non-English mail (maybe the statistical models are biased against "not English" even for people whom don't have English as a native language).
But I'm quite convinced google's (and to a lesser extent Hotmail/ms') "magic" spamfiltering is subtly (but annoyingly) broken.
I'm definitely not giving to somebody else my /var/log/mail.log, IMAP/Mutt
access, sieve rules, nor ad hoc dedicated aliases for every website account
I create. And on top of that, I would land on some US-based server for ease of
illegal spying and my mail would be harvested for some advertising crap, all
to solve something that is not a problem for me. Not happening.
How do you do that? The vast majority of blocklists I've interacted with have been unwilling to deal with questions, instead responding only that if you fix "something" (not always specified) automated measures will remove it from the list eventually.
In my experience it has also been extremely difficult to deal with people using blocklists. It's easy to find a bunch of people using .tor.dan.me.uk rather than .torexit.dan.me.uk "just to be safe". Frankly, I'm not sure why the former list exists in the first place other than to be an arse? What threats do entry/relay nodes pose to you?
I worked for two very large ESPs and never really had many problems with blacklists from a deliverability perspective.
Whenever one of our mailer IPs was blacklisted by one of the big targets (hotmail, gmail, etc) it would only be for 24hours after which I'd put it back into the pools (although, at least initially perhaps for our more reputable clients to warm it back up before letting the more dodgy stuff through it again). If you host your own NS's for the sender domains flipping SPF ips/ranges is not too hard (it's all automated for us, anyway).
The big boys work that way at least in my experience. I'm sure sometimes you'd hit a 'somedomain.foo' domain which is using a blocklist style thing that their (usually inexperienced) sysadmins think prevents them from receiving spam; but they're not worth arguing with. If you're doing email at volume shifting sends to another range for that client is usually 'enough' to get them through if they care that much about that one segment to ask you to do it.
If it's not then we'd usually refund instead of trying to negotiate with such admins.
Honestly though, blacklists have never been an issue for me and I've sent a ridiculous amount of email over the last 10 years...
If I ran an evil email monopoly, I would systematically drop all email from say _half_ of the email servers out there. This way, nobody would be able to make a case that we're abusing our monopoly position, but at the same time running an email server is hard and the internet is filled with wildly varying experience reports (which is the best FUD there is).
This way, fewer people start email servers and, therefore, fewer potential competitors grow up.
Obviously I have no way to tell whether Google really does this, but if they would, the result would look exactly like this HN thread.
I'm probably totally wrong here, but note that there is absolutely no incentive for the large email providers to try to fix this mess and make email the free distributed network it once was.
I've gotten DKIM, SPF, DMARC, etc all verified on my Digital Ocean droplet (and removed from various greylists) but still get spam checked by AT&T / SBC Global, who is the main offender of keeping a private list.
I've been removed for now from their lists... but they tend to re-add IPs from previous ranges for no reason.
This is the most German thing I've seen since Octoberfest 2016.
It seems weird to me to have to find out the preferred way of communication to this level of detail, I already loathe that one cient of mine that only uses Facebook messenger for all communication.
I would love to only use plain text e-mail. However, how does this work in practice when you are not a well known person (albeit in a niche) that decides the rules?
In work I have to accept whatever my colleagues, clients and bosses will send me. For personal communication I use mail with two people. And services that let you choose whether they'll send you plain text are practically inexistent.
Many (most?) emails can be read in plain-text format even if they were written in HTML. The email often comes encoded with an "alternative" plain text version; of course you need a client that will show you that version instead. In my experience most personal email (i.e. not automated/form mails) I receive has a plain text alternative version that works fine.
For emails sent by an evil client that doesn't provide a plain text version, you can consider a tool that attempts to convert HTML to plain text (though I don't have a suggestion for this yet). Fall back on reading the HTML version only if the above fail.
- experience from working on / using my own mutt-like terminal email client.
> For emails sent by an evil client that doesn't provide a plain text version, you can consider a tool that attempts to convert HTML to plain text (though I don't have a suggestion for this yet).
I use emacs to read my email, and w3m to render HTML email as text. It does a good job. Newer emacs includes its own web browser (eww) but I have not tried it for HTML email rendering since I'm happy with w3m.
Replies always go out in plain text (my preference) but if you want to send HTML there are ways to e.g. write your email in org-mode markup and have it converted to HTML when you send. But IMO if you want to send "rich" email then just use a client that does that natively.
How much work would you propose is sufficient that grandma doesn't mind but spammers will be severely hampered in their sending and in a manner that doesn't require the receiver to store too much state?
Only a small amount is necessary. There is a protocol called hashcash (used by Bitcoin but generally applicable) that can be easily used with email [1]. You basically get a header like this:
You can choose how much work you want to do and the recepient can specify thresholds for minimum work required. This header is all they have to store, and it's easily stored through existing email infrastructure. Your mail server can do it without your client's help, too. If you spent a second on a proof I'm sure grandma wouldn't notice, but it would be very difficult for spammers to do the same.
For one, it'll only work if both sender and receiver use it. Which means for 99.99% of mail traffic right now, it is utterly meaningless.
edit: Plus you also literally wasted everyone's time and energy.
I'd rather favor some kind of mechanism like grey-lists that don't require sender opt-in otherwise it'll be a dead technology just like GPG. (You can probably count the number of GPG emails within 1000 average mails on one hand)
We make incremental change. Adoption starts slow, then some middle players pick it up, one big player grabs it and then it proliferates. I've been working on a mail client for some time and I have plans to write a mail server, and both will have first class, opt-out support for hashcash.
There's a certain zen to going back to basics and using plaintext. It's always my default choice whenever I'm given the option.
I'd argue in most cases you really don't need any fancy styles and markup. Although upon writing this I'm now wondering if unstyled HTML might provide improved accessibility over plaintext. What are people's experiences on the matter?
Although I respect that some people may find greater value in carefully styled marketing emails, to me it just feels a bit overengineered at times. How many man-hours have been spent trying to get marketing emails to look the same across all major clients? Are slight variations really such a horrible thing?
As we're on the subject, I'll add the following suggestion to gmail users: go to Settings and changing the Images option to "Ask before displaying external images". External images are regularly abused to track if the email has been viewed, which I consider creepy.
If nothing else, styling emails serves an economic purpose through facilitating signaling. The more a company expects that consumers will value it in the long run the more incentive it has to signal that fact, and a signal is only as good as it is expensive in appearance to the people being signaled to.
In this case, things like a marketing email or simply the follow-up email to signing up to some service provide that signal since the user can see how much effort was invested in making the communication look attractive.
As a recipient of those emails -- not really. Clients block remote content by default and there's almost never a reason to load them, especially when they're used for tracking. Those 'pretty' emails will almost always look like a soup of image placeholders interspersed with text.
> External images are regularly abused to track if the email has been viewed, which I consider creepy.
My understanding was that external images are automatically fetched and cached on their servers by Google, so they can't be reliably used to track message views [1]. Has this changed?
AFAIK tracking images are usually sent via a recipient-unique URL, which allows services like Mailchimp to provide open stats for individuals. Since the URLs are unique, the host servers will get hit at least once per recipient that opens the email, as Google downloads and caches the image.
That depends on how the fetching and caching is implemented. If it's done when you open your email, the tracking would still work. If the trigger is unrelated to anything correlating with your opening of the email it wouldn't.
It will still prevent the sender from setting cookies for you, or learning your IP and user agent.
> Although upon writing this I'm now wondering if unstyled HTML might provide improved accessibility over plaintext.
Yep. Email formatted for 65 to 78 monospaced characters with hard carriage returns (as per *nix mailing lists) look much worse on devices that are narrower or wider than this. HTML paragraphs don't have the problem.
This is silly. The authors establish that phishing is a serious problem (duh), and that this problem is caused by the absence of reliable authentication of messages (a worthwhile observation, albeit one that the industry is already aware of and doing its best to patch over), but they fail to establish that text-only email solves this problem in any meaningful way.
Text-only emails can and will still contain links, which users will still click on. Misleading domain names will work just as well in the email body as they do in the address bar. Even if (as this article seems to imply should be done) mail clients don't make the links clickable, users will still copy-paste them. (Not to mention that the usability benefits of making links clickable are significant enough that mail clients won't forgo them just for a speculative hypothetical security benefit.)
The authors seem to think that inserting a "speed bump" here will cause users to pay closer attention and not be fooled. This is not how humans work, especially very busy humans who get too much email and just want to get through it as quickly as possible.
Also, the reference to JavaScript in email leads me to question whether the authors have any idea what they're talking about. Mail clients don't execute JavaScript.
> but they fail to establish that text-only email solves this problem in any meaningful way
The quote from US-CERT isn't "meaningful"?
FWIW, I'm the first to bitch about security experts that sacrifice usability in the name of security any day, but for once I completely agree with them.
> Also, the reference to JavaScript in email leads me to question whether the authors have any idea what they're talking about. Mail clients don't execute JavaScript.
And that's only until someone finds a way to make them execute Javascript anyway. I don't think it ever actually happened, but not using an HTML engine drastically reduces the attack surface for sure.
Text-only emails can and will still contain links, which users will still click on. Misleading domain names will work just as well in the email body as they do in the address bar.
The 0/O/i/1/l/I distinction is a classic one that can be somewhat mitigated by good font choice (plaintext emails are often presented in monospace, so that helps a little), but some of the other sets of Unicode characters are designed to look identical to ones in the ASCII range, often using the exact same glyphs.
For those who use non-English characters exclusively, it is not possible to use ASCII only emails, but non-Unicode encodings are still helpful in this situation --- e.g. the Cyrillic characters most known for IDN homograph attacks aren't even encodable in 8859-1 or Shift-JIS.
Speaking of JavaScript in e-mail, gmail doesn't allow you to send or receive .js files (even tarred up), which is somewhat inconvenient, and I'm not really sure what attack that prevents. Maybe there is a mail client out there that will happily execute attached js?
I got a phising SMS today (please login to verify unusual account activity) with a link to mybank.online-eauth.co.uk. I knew from before even seeing that it was probably not legit, but I did check the domain whois as UK banks are rather arcane, and I wouldn't be surprised if they had a completely different domain for something like this. I know when I buy something online, my bank has a different domain for Verified by Visa.
Also, I gather that non-text emails make it possible to disguise the link target when you hover over it to see where the link goes. Whether that is using css or js, I'm not sure.
You can't falsify the browser indicators of link targets (status bar on desktop, modal dialog from holding down the link on mobile) without JavaScript.
The real URL will appear in the browser address bar anyway before the user gets the chance to disclose any information. I don't know exactly what proportion of users will notice a well-disguised phishing URL in the email body but not in the address bar, but I bet it's not that high.
The attack prevented is simply having the user open the attachment, allowing the sender to execute arbitrary JavaScript on their machine in the file:// context. (Modern browsers have made the security consequences of this somewhat less dire than they once were, but it's still not something you want to do if you can help it.)
That involves the rather large and utterly baseless assumption that users look at the address bar at all. You probably do. Does your somewhat less-savvy next-door neighbour?
The threat model here is phishing, not drive-by downloads. Browsers have a much greater ability to mitigate those. Also, a drive-by download email doesn't have to impersonate any particular sender, it just has to look like something that a user might want to click on.
> The threat model here is phishing, not drive-by downloads.
I think you missed my point. What if it isn't a phishing attack? Or even, what if it isn't just a phishing attack?
Your suggestion leaves users vulnerable by encouraging them to open suspicious looking links on the off chance it is, at most, a phishing attack.
> Browsers have a much greater ability to mitigate those.
Except when they don't. For what it's worth, I've also seen mobiles fall victim to drive-by download attacks.
> Also, a drive-by download email doesn't have to impersonate any particular sender, it just has to look like something that a user might want to click on.
E-mail worms are spread by the trust relationship between people known to each other (ie a user opening an attachment because it's from a recipient they know). I don't see why drive-by download attacks couldn't exploit the same human condition (ie "Hey bob, check out this link. It's awesome").
In fact I have seen that kind of malware in the wild, now I think about it.
I don't know what you mean by "encouraging". I am taking it as a given that users cannot be reliably prevented from clicking on links in emails. The security benefit of forcing the URL to be displayed before the click is extremely minimal.
A small, but not insignificant, proportion of malware and phishing emails I receive do not show any links at all in their plaintext part. Some of the links are probably easy to skim over and miss the fact they're suspect, but some really do look very dodgy indeed (coupled with very poor plaintext formatting), facts that I would expect a non-expert would notice in some cases. Certainly enough to make it worthwhile.
Every time I use some kind of graphical/web email client, I'm appalled by how hard it is for me to find the "show me all the headers" feature. In mutt when I have doubts about something I just hit "h".
Admittedly this requires a level of knowledge that the average user may be missing. But I find it really helps inform my opinion of any borderline-suspicious emails.
I recently switched to mutt and using plain-text email as well. So far there have been very few times that I've felt the need to jump back to the webmail to send something (e.g. inline images). Overall, I've thoroughly enjoyed the simplicity of plain text email using my text editor of choice.
Can you elaborate why?
I sometimes abuse this option by including screenshots in the body of e-mail, but I could as well add it as an attachment. Other than that I see no reason to use HTML in e-mail conversations at work.
Intra-corporate e-mail is often used as an ad hoc document collaboration tool.
"See my comments in blue", strikethroughs, inline diagrams, bullet points, big red font for emphasis.
Of course that should all be done in a dedicated application, but who is going to provision and authorise users for that versus just adding Bob to the cc line and giving him implicit editing capabilities?
The functional overloading of corporate e-mail is a user-driven reaction to the awfulness of most " collaborative' software.
I've always had my work email client set to text only. Never been a problem for me (unless you consider missing out on a lot of unnecessary smilies, fonts, and colors a problem).
I've always thought native handling of markdown for email would be pretty cool.
The MUA could strip out any HTML embedded in the actual markdown, render to HTML locally (for display) and you have nice formatting without the risks or cruft of email HTML.
And of course if you choose to view in text only mode, you don't actually lose any semantic meaning.
Sure, it could be used elsewhere. OTOH 7763 calls out email as a use case three times and in section 5 Examples "email attachment" is the only example.
This might be true, but I think that ship has sailed. Email for 99% of internet users is html. Thinking that some large fraction of news letters, outlook emails will ever be plaintext is just naive. I use html emails in outlook simply because I don't want my emails within the corporation to appear differnet from anyone elses. I certainly don't want to return something that looks different from what the sender wrote,.
The mail client is a web browser. In many cases it's an actual web app in an actual web browser. The solution has to be to make them safer for the user. Maybe a subset of html can be whitelisted, maybe link targets should be rendered more clearly, maybe something else.
Over 99.5% of the email I receive is either plaintext or renders perfectly legible in plain. This suggests that a large fraction of newsletters are already plaintext. The mail client is primarily a program to display and send email. The web browser is a web browser.
If the user wants safety, switching to plaintext is a very wide step forward.
> Over 99.5% of the email I receive is either plaintext or renders perfectly legible in plain.
Is that representative? Where do you get email from? Did you change preferences on things like mailing lists in order to get to that percentage?
Also: what kind of client do you use? iOS mail does send plain text emails as text/plain (which is fantastic) but if you look at e.g. inbox (gmail) it doesn't even allow sending plain text emails as far as I'm aware.
Emails can send in both formats for a single message, so it's possible and even likely that most mailing lists, etc. he receives send in both formats. In my experience, even most marketing e-mails are at least somewhat good about this. I'm 99% sure that Gmail will still send a plaintext e-mail inferred from your HTML content whenever you send, so it's not quite accurate to say it doesn't send in plaintext.
> I'm 99% sure that Gmail will still send a plaintext e-mail inferred from your HTML content whenever you send, so it's not quite accurate to say it doesn't send in plaintext.
I think that might be possible in the "old" gmail, I was referring to their newer "Inbox by gmail" client. I believe it never sends a text/plain email regardless of content
(And it being their "newer" revamped client kind of shows what googles take on this is...)
...and if you're PayPal or eBay, you offer the customer a choice of receiving plain text emails. Then you send them multipart emails with a blank text/plain part.
I run two email addresses: one business, one personal. As you may imagine, the emails I get between the two are quite different. I'm using Thunderbird and always, when given the choice, opt for plain text email. I find Thunderbird does a very good job of rendering HTML email legible (easy to read) and functional (links work, etc).
Perhaps once a month, maybe less, I have to use the toolbar button "Show HTML Temp" to do a one-off rendering of an email as basic HTML. Almost always the same offenders.
I find it such a success that I recommend it to my customers - none of which are computer experts, and most are not what I'd call "computer savvy". A process has to meet a high bar for me to recommend something like this to my customers.
Shame that when I explain how it can help in improving one's security that a vanishingly-small number of customers take me up on the offer!
I think the first reason people don't care for installing e.g. Thunderbird is that they check mail on a) mobile, b) work (which is usually outlook, or at least mandated by someone else).
If I asked 100 friends "did you use email today" 99 would say yes. If I said "did you do it on a computer" then 50 would say yes (those that work at an office). If I said "was it your own computer" then maybe 1 would say yes.
I think for us that sit at our own computers regularly, it's hard to imagine just how extremely rare it is for people to actually use desktop/laptops for personal use these days.
I'm not sure the ship has sailed. If HSBC switched to only mailing out text-only emails with URLs written out in full, after a while HSBC users would get used to only receiving text correspondence from their bank. I think that would be a step towards reducing phishing attempts, though certainly not a complete answer.
Disclosure: I work for a fintech company that utilizes HTML emails almost exclusively for customer communications.
I think one thing that is not being considered is that for most customers, branding and the consistency thereof are key indicators of trustworthiness - especially when dealing with financial information. HN users are rare creatures, they have technical context the average end user does not have. The rise of phishing has lead users to pay a great amount of attention to subtle hints of impropriety, like being taken from one sort of visual experience to a vastly different one. We saw vast improvement across all meaningful metrics when we switched from plain text to HTML emails that utilized branding consistent with our website.
As with everything that humans deal with, there are tradeoffs here. And I'm extremely concerned that this position taken to it's logical extreme would lead to the web being transformed into something that is "safer" but much less useful and dynamic. One outcome of this could be the slow death of the open web in favor of siloed networks and platforms serving actually functional content in "safe" ways.
That would require HSBC to value some kind of improved security so much that they'd accept not having the HSBC logo in the email. That's what I think is out of the question.
You could maybe see banks having plaintext communication as an optional, but I doubt they'd make it default (allowing users to switch to html).
Isn't this problem already solved with certificates online? Shouldn't this be solvable the same way? E.g. a bank sends an email containing a link to the content with some special attribute. The web browser displays the content if and only if the sender domain of the email (e.g. hsbc.com) is also the domain from which the content will be downloaded.
The solution assumed mail clients would be adapted to enforce this. So if you send me a forged email claiming to be from hsbc, the mail client would allow showing html content only from a https connection to somewhere on hsbc.
Kind of like the same-origin policy but where the origin is the domain the email claims it came from.
I use plaintext mail heavily. My only complaint is clients that don't support format=flowed[1] wrap text weirdly and make it look bad. The biggest ones are Outlook and the Windows 10 Mail app, though some webmail handles it poorly too. It gets bad when conversations start accumulating nested quotes.
I've found a quick tell if someone uses Outlook is if I send them a text/plain message, they'll send one back and there's no format=flowed. At that point I'll usually send them HTML mail.
I was expecting to read about some brilliant new way to solve all of the normalisation problems with Unicode, and make plain text safe again. What a let down!
Annoyingly, the Gmail Android app sends things multipart, even though I'm not sure it's really possible to have any formatting (except maybe links?). I wish there wan option to send text only, it might even make a (small) difference for people with limited data.
I think this is entirely dependent on the context of that particular E-mail. Email has evolved quite a lot.
A good example is a newsletter from say, Quora or Medium; Newsletters usually have links to a story or a news feed article. If this was done using plain text, the link would be one long mashup of characters, because they usually include an authentication token or something like that. In this case, using an html link or button is clearly the better option.
> A good example is a newsletter from say, Quora or Medium; Newsletters usually have links to a story or a news feed article. If this was done using plain text, the link would be one long mashup of characters, because they usually include an authentication token or something like that. In this case, using an html link or button is clearly the better option.
Well, that's a yet more hostile thing that one has to put up with. I follow news through newsletters, and I'd prefer they were in plain text and with normal links. My reader knows how to wrap lines and make link-like things in plain text clickable. But unfortunately if I wanted to enforce that I'd have to avoid news...
Yah,consequently. But last week someone posted a link here, a text only version of CNN, I looked it up, here it is; http://lite.cnn.io/en. We could have more of these soon, which is a good thing.
But even there you've got to be careful. For example if you're using GNU Emacs to read your email you could have been vulnerable to arbitrary code execution for the past few years:
Some client software hides the From and Reply-To addresses, only showing the name by default. A friend's accountant got hit because the From had my friend's name, but the address itself was bogus but hidden, so he opened the attachment.
So keep the main headers text-only as well (which most sane software does anyway).
I never click on any link in an email I didn't ask for. If I get a notification email from a party I have business with, I go to their website manually and check for messages.
and to get there, we need email clients with usable UIs. Is there a way to make outlook or apple mail or the gmail web app grab the equivalent of `document.innerText` on every piece of email? Who can tell, without spending hours hunting through increasingly obscure menu forests?
https://www-user.tu-chemnitz.de/~heha/email.en.htm
http://problemkaputt.de/email.htm
https://www.gaertner.de/~neitzel/email-to-mn.html
http://www.karo-electronics.de/448.html
...and of course there's this:
http://arc.pasp.de/