It's hard to believe people don't understand that 'shortcomings' are sometimes 'advantages'. Email is perfect for what it is, it's a textual, non-realtime medium of communication. You take the time to compose a long, coherent message using written language. Mail has existed for a long time, and hopefully it will never go away.
It's great if you want to invent something more instant, more automated, more visual, but it's not going to be an improvement on email, it will be something else that fills some niche.
Agreed. SMTP is such a lasting protocol that it is unlikely e-mail will be replaced in the long term. I certainly think improvements can happen on the server side, but the protocol has good failover, and for the most part, the reliability is just as good if not better than physical mail delivery.
Yes, the protocol seems dated, but it is well-understood and implemented by thousands of vendors. It's a miracle that we were able to agree on a protocol, and on top of that, agree on a protocol that is relatively decentralized (although it does depend on DNS if you prefer not to use an IP to email). It isn't like Twitter or FB where a single vendor controls the platform.
> It's a miracle that we were able to agree on a protocol
It wasn't that miraculous, actually. Remember that it was invented before HTTP even existed. It was developed at a time where the only contender was Usenet. Internet was not yet used for any commercial purposes; there was no concept of platform. Internet, and thus email, was the medium between universities that we know of. Sendmail was included by default in each UNIX from that moment on; when you connected to the internet, you could already send emails.
The conditions were very different, it doesn't make a lot of sense to compare it to what we have today.
Absolutely. All these attempts at reinventing email into a new instant messaging service are fundamentally flawed, even if they only intend to use email as bootstrapping. I /don't/ want enter to send the message -- I want enter to create a new line. I /don't/ want you to hide the reply field for noreply@ emails, because sometimes I use draft responses as short memos to a specific email.
No, he's arguing to not change major parts of what makes e-mail useful in order to try to shoehorn e-mail into an entirely different messaging pattern.
The only thing that makes email useful is "everybody has this thing that accepts stuff and they get notified when new stuff shows up." That's all.
Email is the single biggest vector for "hacks" and stealing money. Email is the biggest vector for spawning botnets. Heck, in half the email clients I see, there's no easy way to verify if <a href="phishingsite.com">your bank login</a> is a valid link to your actual bank or not.
Plus privacy issues, auto loading images (tracking pixels), accidentally reply-all, the ad-hoc way people think adding 50+ people to a CC list is a "mailing list," the way people think reply-all "+1 Adding Charlie" to the 50 people to add one new person to ad hoc email chain from hell is acceptable...
It's not good. It's just everywhere and people have accepted productivity killing social constructs around it so they can attempt to communicate at about a 60% effectiveness rate through such a broken medium.
The only thing that makes email useful is "everybody has this thing that accepts stuff and they get notified when new stuff shows up." That's all.
Also notes, contacts, file transfers, reminders and more.
Spam? Spam is everywhere and if it is not now, is because it's not popular. Once it gets to the level of "email", they will make sure you are getting spammed.
What bothers me is that people look for alternatives to make us more "locked" aka "safe" instead of trying to educate. We, devs/sysadmins, have to find ways to make sure that people know how to use and protect themselves not to hijack the tools they use/work/need just because we like to be EDGY.
> The only thing that makes email useful is "everybody has this thing that accepts stuff and they get notified when new stuff shows up."
And that's also what makes the alternatives utterly useless. It takes a huge advancement to unseat things like email. Even a new technology like Dropbox has powerful incumbent "defenses" against superior alternatives like Syncthing and Bittorrent Sync. Who am I supposed to use these new replacements with? Literally nobody I know uses them. I can't even use rsync to transfer data to and from my clients (not in the software industry).
> Email is the single biggest vector for "hacks" and stealing money.
If this were less vague and thus actually falsifiable, you could also say that the single biggest vector for hacks and stealing money is the internet. The reason that email is the main vector on the internet is its ubiquity.
It would be great if email had been originally designed with signing and encryption capabilities, but it wasn't. The reason it is still better than some new messaging service that includes these capabilities is that is federated and decentralized and very tolerant of network failures.
> communicate at about a 60% effectiveness rate
Honestly, if humans could communicate 60% effectively, our experience as a species would be completely different. I don't think that email is the bottleneck here...
To me what is important about email as infrastructure is SMTP and the principles of delay tolerant networking and federation. SMTPs store and forward reliability model is a bedrock of communication on the Internet, when all else fails email won't or it'll die trying.
Anything vying to replace it needs to capture these features. I won't trade reliability for real time. There are other real time channels I can use.
If the e-mail is simply stored persistently until it's fetched succesfully (over HTTP), isn't that delay resistant networking? Are persisting proxies really important?
To me the problems with email have more to do with spam and privacy than threads and real time.
We shouldn't have a unique address but rather give a unique random token to anyone asking for our email address which could be severed (or bound to receive from a single address, ie that counterparty can not transmit that token to anyone as my server wouldn't receive mail from another address).
We should have systematic guarantees that the sender is the mailbox it pretends it is.
Encryption in smtp should be mandatory.
We should have an easy to use content encryption (pgp style).
Ideally you would like to decorrelate the address from the service provider (lots of people are prisoners of gmail, hotmail or their ISP).
I am sure we can make the chistmas list much longer but to me these are the major issues with the current system.
Well, most of the items in my list exist in one form or another, but all are optional and/or not widely used.
As you mention we can use aliases for unique emails (which is what I do, but I had to create a software on the side of my mail server to manage them, as my mail server never intended to use aliases that way, and for instance makes the list of aliases visible to everyone on the server, and does not allow to assign a keyword to an alias).
It is possible to have guarantees on senders by using spf and dkim, but not many server actually use them and you currently can't make it a hard requirement for your incoming traffic, so pretty much useless.
Smtp encryption has taken a step back with TLS, as with STARTTLS we are now forced to use optional encryption that can be downgraded by a MITM, whereas before one could elect to only have an SSL port open.
Content encryption is possible using PGP but clunky and almost no one uses it.
And people can buy their own domains if they know it is possible and how to configure a DNS server, needless to say not many people do.
What I suggest is that we would need a protocol that does all that by default.
As others have said, the shortcomings of SMTP can look a lot like advantages, at least in terms of its simplicity and durability. When a protocol has survived and continued in use for so long there are usually deep-seated reasons why it is a fit for certain use cases. I don't want to go off on a chat vs. mail rant, but the following ...
>> You operate on a level of the entire messages flow. Forums, chats and web 2.0 networks are trying to make conversations feel as lightweight and barrier-free as possible.
... caught my eye. I manage a team of younger developers for whom real time chat is simply the way they grew up interacting. Everyone is signed into hangouts and "operating on a level of the entire messages flow" (sic) all day long. A large part of the interaction is off topic, vague, and ineffective. Here's an example from yesterday...
Dev 1: the repo url is wrong, I'm getting a 404.
Dev 1: wait, might be a typo.
Dev 1: my bad.
Everyone on the team received this message and presumably stopped their current mental task at least long enough to read it, as I did. In the end it had no impact whatsoever. The developer clearly should have thought through his problem before notifying the entire team that he had one. But this is how chat is used.
I've tried very often to get them to be more deliberate in chat communications, without any success. I have also tried to encourage complicated topics to move quickly to voice hangouts, with even less success. Streams of real time chat messages remains the primary way the group interacts, to the point where I have backed off my efforts and come to the conclusion that I'm likely the one who has to change.
That said, there will always be a need for deliberate communications within organizations. Email grew up as an analog for letters and memos, which served as deliberate, archived, time-stamped records of an important communication. They were considered, usually minimal (because they took effort to create), and got to the point. They were events, not streams of ephemera. I'm not sure that this constant buzz of chats does a good job of replacing that. So for that reason the sentence "You operate on a level of the entire messages flow" doesn't make me smile.
I think we should look at the proposed idea as two-fold:
1) giving E-Mails more inherent structure that is managed centrally (streams) instead of by each user (filters, folders).
2) making E-Mail more synchronous instead of asynchronous.
As far as I understand, you're arguing against (2). I agree with everything you wrote, however I still find (1) a very good idea. It is also an idea that IMO could be implemented entirely without changing the protocol - it's just a question of where you put this additional information. Streams could also simply be represented as a group address in the CC field that is managed differently by their client/server architecture, but otherwise fully downwards compatible.
Every now and then someone decides we need to replace email, or the web, or Craigslist, and they think the reason it hasn't happened yet is that no one has really thought about the details.
It's not.
The reason those things haven't been dethroned is they are he lowest common denominator. You can't dethrone them by making something better. You can only dethrone them by making something that is both better and more common.
By definition the lowest common denominator is crap. By definition it is some legacy junk that was already on every computer 10 years ago. By definition it's a hundred competing standards from multiple companies because how else will a technology work on every device ever?
You can try to replace it with another tech, but it's going to have he same kind of characteristics, and it's arc of adoption is going to look like email's. Which means maybe 10 years from now it starts to look like a real contender. And by then how much crap has been jury-rigged into the spec?
It's the long tail. It's janky, but it's a common standard that does one job like a freaking champ. The long tail will never be engineered away. It ebbs and flows. Only in the last 10 years is snail mail losing its status as a reliable way to reach people.
To me this seems so different from email that it rather invites a comparison with RSS feeds or Slack-like services.
What if it was a no-brainer to publish to and subscribe to feeds, there was some kind of feed exchange and access control built-in, and everyone was using them to communicate? What if Slack was open and decentralized, and everyone was using it?
> What if Slack was open and decentralized, and everyone was using it?
We tried that; it was called IRC. As far as I can tell, Slack is still little more than a hosted IRC service with its own web and mobile clients. Virtually none of the functionality they provide couldn't be replicated this way; it's more a matter of branding than anything else that they eschew the label "IRC".
Slack even provides an IRC gateway so you can access it using IRC clients. I don't know if their web and mobile clients are using the IRC protocol, but from what I can tell, they might as well be, which is the whole point.
What we need though, is something federated. Where everyone can interact with anybody, just like email (or like XMPP did).
IRC does not have this. I've to connect to your server, or you to mine. This doesn't scale as an email/IM replacement; it only works for rooms with a certain usage/target.
The huge problem with any replacement (aside from federation) is the network effect. No matter how good your solution is, you need critical mass for it to pick up, and no huge company is going to invest into getting users into a federated network which they won't control. There's no profit for the company.
> IRC does not have this. I've to connect to your server, or you to mine
IRC most certainly does have this. All larger IRC networks are exactly that: Networks of federated servers. It's been like that since the early 90's, when it was an absolute necessity as none of the ircds could handle all that many individual users.
Nope. IRC model is closer to clustering than federation. The key difference is that all connected IRC servers have nearly full control of the whole network which is why you can not allow untrusted servers in a IRC network. Also in IRC all messages are routed to all servers. Contrast that to something like SMTP or XMPP where every server is mostly independent and generally handles only messages that belongs to them, and where federation does require very little trust.
> The huge problem with any replacement (aside from federation) is the network effect.
Indeed, it’s the main catch. If at certain point ago critical fraction of people (‘everyone’) all were using X for what we currently use email for, be X IRC or anything else, then X’d probably be the today’s email.
Services such as the aforementioned Slack seem to be going maybe the only pragmatic ‘email disruption’ route. In corporate world you can have the central authority say “everyone use this or you’re fired!” and thus bypass the network effect barrier.
> Virtually none of the functionality they provide couldn't be replicated this way
And as always, there is a huge gap between "X is just A+B+C" and "Here is an actual X". Slack may have not done anything on the transport side, but they did make the client better. AFAIK the only IRC client with full text search on the whole history, with support for files and images, is irccloud.com. That's not a lot. I wouldn't say IRC does everything Slack does.
Is this not just folksomomies for taging email, it will suffer the same problem that tags do, consitancy of the tags, how will it know that kitties, cats, felines and pussys are all the same stream. How will people know what to name the content in the stream. How is this different to adding [CATS] to the front of the subject line, and having the software partition on that.
That is exactly what I thought. "Stream" is the subject line. You mail to a group on some sort of listserve, which gives you a long-term history and a URL to give new people.
This exists in rather decent form in Google Groups already -- there is Google Groups for Business (with a Google Apps account) that gives you a great web interface as well as email interface all just for your company.
We go through this a few times a year. "Email is broken, we must replace it", only to realise a few short seconds later that email is not broken, and does not need replacing. Its long lived for a reason - its simple, its textual, it delegates all the fancy stuff to the inbox provider, e.g., videos in emails are usually a link the email provider, say Gmail, shows as a video.
If I had 10 pence for every "email is broken" post, I could have had a free pint down the pub.
I read this and this that it's a lot closer to what RSS intends to solve, especially the read-only streams. Two-way streams is mainly providing a method to post back into that feed. It's a little more real time (rss feeds are typically cached and refreshed), and a real-time implementation would e more of an collection of entries that are pushed to the server presenting the stream and ultimately to the client.
What I think
With that in mind, I feel this concept is very close to a lot of the p2p or decentralized content distribution systems being tested out these days, like aether[1] or matrix.org[2].
The backend implementation that matrix.org uses is rather ideal. It's Federated JSON over HTTP(S). The clients out today are heavily focused on chat, but it is data agnostic. You could have client that writes rich media content and a client that displays this as a blog. Going back to the streams concept, you could have a client subscribed to multiple streams (rooms in matrix vernacular) and as people post their messages - rich media or simple "how are you" comments, it would get federated out to each home server, which can present a web client interface like in tonsky's mock-ups.
For the concept of amazon providing sensitive order information as a stream/matrix room, there would have to be an add-on concept built into the client of some sort of security token. I'm not talking about a losable gpg keypair, but something amazon generates and provides to you (like they do with their AWS services already) Amazon would then create a personalized stream for you, using that security token to encrypt. Since amazon provides the content and the key, they can replace your key at any time.
The problem I have with all of this is it seems like the XKCD where for some reason, the latest person / thing seems to somehow not think to google if there's anything like what they're doing, so we have yet another program / protocol / community that 5 people use and then dies. EVERYONE uses E-mail.
And now with Aether - has no one heard of freenet and i2p and Tor? No, we'll re-do all the sort of starting work on anonymity and security and get to re-make for the thousandth time the mistakes where it really isn't that secure or anonymous rather than building on anything that has at least some of the actual battle testing and bug squashing already done.
What is Aether but Frost with less testing? Hell, what is it but TOR access to Usenet? Maybe I don't understand what's so special that it can't use any of the long worked on infrastructure level work that's gone into the above projects for a decade or so, and it actually does make sense to start over from day one - but if so, they certainly don't explain it.
And I'm not talking about UI - I'm talking about the anonymity equivalent of deciding TCP isn't good enough so I'm going to re-invent IPX or something.
Yes, this could work. I like the idea of streams being an url with a path that might be a unique hash for you. The url's domain controller decides who is authorized to post on their streams. You decide which domains to trust.
Most social services we use today could be streams. Facebook, Twitter, even WhatApp, Instagram (I guess not snapchat). If they'd all implement some common stream spec. Unfortunately they won't, Twitter has demonstrated that they don't like alternative client implementations.
If it could replace e-mail is another big question. Nikita hilights the amount of cruft that's around e-mails, but the cruft is social protocol, and we humans love social protocol. Or perhaps that's just old fashion speaking.
The posting is quite old already (Oct 2014), so this isn't some big hype building and it doesn't look like anyone is working on a big implementation.
When you read the post, you know that XMPP has all the technical parts to fill the role here. There are some people who build things on top of XMPP, but I haven't seen something as advanced as buddycloud (http://buddycloud.com/) here. They have a concept of "channels" (https://xmpp.org/extensions/inbox/buddycloud-channels.html) which can have multiple types with multiple participants, each having a definite role. All of the use cases in the article are (theoretically) possible with buddycloud.
Technically, the standard XMPP PubSub protocol with its PEP companion could get you very far already, unfortunately there's not many clients that handle them today.
This sounds extremely exciting. But there's one question: how do I tell someone to subscribe to a stream? Do I need to email them a link to my stream? The killer feature of email is that it allows anyone to contact anyone else.
Also, issue tracking is something that could be replaced with a stream...
Neat concept. One challenge is imagining the transition from email to stream - how can I start using streams internally with my company gradually, but still use email when needed? Not another inbox...
I think you'd configure your client to pull e-mail as a stream or a collection of streams (with filters). You can then in your client fuse or split the streams into the rest of your streams. It could be a pretty fluent transition in my imagination.
I dislike how half of this blog post is just ranting about email. For example, I don't get this:
"electronic messages are not real-time, they’re hard to manage in huge amounts, they work best for person-to-person dialogs"
These are "limitations" according to the author. But IMO they're the reason I like email:
- I like to be able to reply when I want, without the sender having the expectation of everything happening in real time, I have other things to do than answering real time to everything.
- Anything is hard to manage in huge amounts, which is why we hide less important stuff. Email has folders where you can put stuff in. You can archive emails. Or maybe this management issue is technical, as in "storing petabytes of emails is a difficult task" ? In that case, I have no real answer.
- I actually like the concept of mailing lists: they work where I already am (ie my mailbox) and I can stay up to date just like in a forum. And person-to-person dialogs do indeed work very well with email. I don't see that as a limitation.
---
Also, about this:
"Stream is a group of threads. Simple as that."
That does not sound simple. In fact, just from that sentence, I didn't understand at all what it meant. I had to read on to get the idea. But I do see the point of "streams", and I agree that filters are usually complex to setup and maintain.
---
The part that I don't understand is this one:
"With proper user/access management Streams could replace not just email, chats and group chats but also forums, blogs, mail lists, RSS and most of web 2.0 communities."
It sounds like what you need is RSS everywhere. RSS has a pull model too. You can setup ACLs on an RSS feed if you have some kind of authentication in place. It can replace mail lists, except that you'd still need to be able to push messages to it, which I don't see anything about in the blog post.
---
Finally, regarding this:
"This fixes ambiguity around do-not-reply@ addresses"
I get the point of ambiguity and I have replied to no-reply addresses before myself, with an error message coming back. But I think ambiguity could be fixed with a header here, instead of throwing away the whole email infrastructure.
Imagine a "X-No-Reply: true" header. That way, email clients could disallow replying to such an email address. This has the advantage of being backwards compatible with everything that exists.
---
Overall, I do think that you can try something new, trying to "replace" email. But realistically, Google has tried several times and is struggling. Email has solid tooling and knowledge built around it, and I think trying to push for its self-improvement is worth a look.
Slack is sometimes said to be an "email-killer", but the fact is it replaces only one use of email. At my last company, we used Slack for real time stuff, notifications, quick questions, but not for everything. And that's great. Because email sucks for real time stuff right now (although initiatives like JMAP may make things better).
Email seems to carry the notion of importance, which is not true of chat messages I think. The reasoning behind this is, I think, an incoming email stands alone, whereas a chat message is quickly surrounded by other messages.
Also, emails have a clear receiver, the receiving email address. This means that if you get a message to "devs@project.org" and you are a dev, you are de-facto concerned by this email, which is a reason to read it thouroughly. This is not true of chat messages which do not have a "receiver" but are instead posted to the more liberal notion of "channel".
> Email seems to carry the notion of importance, which is not true of chat messages I think. The reasoning behind this is, I think, an incoming email stands alone, whereas a chat message is quickly surrounded by other messages.
This is critical. Understandably, "chat" user interfaces don't make individual messages stand out by themselves. This means that it is easy to miss a message, or even a conversation. It also means that you can't delete everything but the important stuff you need to reply to, which is trivial in email clients.
Would that be a bad thing though? What's the use case for needing it? Any sender initiated contact assumes that the receiver is in acceptance of your need to contact them. This is the failing property of email that allows for and implies that SPAM will always exist.
I buy something from Amazon. I'm initiating the contact, so I provide the means for replying / providing an invoice etc. (currently an email address etc.)
This is an awesome article, and has some similar ideas to what we are working on at Nylas. https://nylas.com/
I don't mean to steal this thread, but if anyone is interested in building the future of email (including this) I'd love to talk with you. mg@nylas.com
I hate to sound synical but Ive both invented this ui and seen others invent it a few times over. Even so I do commend the effort and thought process that went in to the post.
Edit: To see a fully functional version check out http://plugg.io
It is possible to make Streams work based on existing email, by saving email into a Streams server whenever the receiver opt-in to get that email (well, it is filters again) as a Stream.
Also, let people publish Streams as Streams and non-Streams users can subscribe to those Streams as email.
e-mail at its best retains something unique to the written letter, a tendency towards long, frank, thoughtful compositions that are sent with a lot of aforethought and intention. i think if we fully move towards communication regimes where the importance of individual messages is lessened (a chat / forum thread type thing) we might eventually lose this style of writing and the enjoyment of communicating this way.
It's hard to believe people don't understand that 'shortcomings' are sometimes 'advantages'. Email is perfect for what it is, it's a textual, non-realtime medium of communication. You take the time to compose a long, coherent message using written language. Mail has existed for a long time, and hopefully it will never go away.
It's great if you want to invent something more instant, more automated, more visual, but it's not going to be an improvement on email, it will be something else that fills some niche.