
Streams: Mail 3.0 concept - scapbi
http://tonsky.me/blog/streams/
======
Demiurge
> It’s hard to believe people still use email.

It's hard to believe people don't understand that 'shortcomings' are sometimes
'advantages'. Email is perfect for what it is, it's a textual, non-realtime
medium of communication. You take the time to compose a long, coherent message
using written language. Mail has existed for a long time, and hopefully it
will never go away.

It's great if you want to invent something more instant, more automated, more
visual, but it's not going to be an improvement on email, it will be something
else that fills some niche.

~~~
jasonjei
Agreed. SMTP is such a lasting protocol that it is unlikely e-mail will be
replaced in the long term. I certainly think improvements can happen on the
server side, but the protocol has good failover, and for the most part, the
reliability is just as good if not better than physical mail delivery.

Yes, the protocol seems dated, but it is well-understood and implemented by
thousands of vendors. It's a miracle that we were able to agree on a protocol,
and on top of that, agree on a protocol that is relatively decentralized
(although it does depend on DNS if you prefer not to use an IP to email). It
isn't like Twitter or FB where a single vendor controls the platform.

~~~
rakoo
> It's a miracle that we were able to agree on a protocol

It wasn't _that_ miraculous, actually. Remember that it was invented before
HTTP even existed. It was developed at a time where the only contender was
Usenet. Internet was not yet used for any commercial purposes; there was no
concept of platform. Internet, and thus email, was the medium between
universities that we know of. Sendmail was included by default in each UNIX
from that moment on; when you connected to the internet, you could _already_
send emails.

The conditions were very different, it doesn't make a lot of sense to compare
it to what we have today.

------
cromwellian
To me what is important about email as infrastructure is SMTP and the
principles of delay tolerant networking and federation. SMTPs store and
forward reliability model is a bedrock of communication on the Internet, when
all else fails email won't or it'll die trying.

Anything vying to replace it needs to capture these features. I won't trade
reliability for real time. There are other real time channels I can use.

~~~
zx2c4
Check out DJB's smtp replacement, InternetMail2000:

[http://cr.yp.to/im2000.html](http://cr.yp.to/im2000.html)

~~~
mondoshawan
That seems less a replacement and more notes on a random idea with lots of
questions. Some of his ideas in there are questionable, too.

Seems like he's increasing the bandwidth usage for an MX in a big way.

~~~
seiji
_more notes on a random idea with lots of questions._

That's djb's greatest strength and greatest weakness. He just does whatever he
wants and doesn't care how it integrates with the rest of the world.

------
cm2187
To me the problems with email have more to do with spam and privacy than
threads and real time.

We shouldn't have a unique address but rather give a unique random token to
anyone asking for our email address which could be severed (or bound to
receive from a single address, ie that counterparty can not transmit that
token to anyone as my server wouldn't receive mail from another address).

We should have systematic guarantees that the sender is the mailbox it
pretends it is.

Encryption in smtp should be mandatory.

We should have an easy to use content encryption (pgp style).

Ideally you would like to decorrelate the address from the service provider
(lots of people are prisoners of gmail, hotmail or their ISP).

I am sure we can make the chistmas list much longer but to me these are the
major issues with the current system.

~~~
HappyTypist
This is easily solved by an email service that makes creating aliases take 5
seconds, and offer a _very_ large maximum number of aliases.

Address exhaustion is a real issue though, so you will need to acquire many
email-worthy domains for this service.

~~~
cm2187
Well, most of the items in my list exist in one form or another, but all are
optional and/or not widely used.

As you mention we can use aliases for unique emails (which is what I do, but I
had to create a software on the side of my mail server to manage them, as my
mail server never intended to use aliases that way, and for instance makes the
list of aliases visible to everyone on the server, and does not allow to
assign a keyword to an alias).

It is possible to have guarantees on senders by using spf and dkim, but not
many server actually use them and you currently can't make it a hard
requirement for your incoming traffic, so pretty much useless.

Smtp encryption has taken a step back with TLS, as with STARTTLS we are now
forced to use optional encryption that can be downgraded by a MITM, whereas
before one could elect to only have an SSL port open.

Content encryption is possible using PGP but clunky and almost no one uses it.

And people can buy their own domains if they know it is possible and how to
configure a DNS server, needless to say not many people do.

What I suggest is that we would need a protocol that does all that by default.

------
markbnj
As others have said, the shortcomings of SMTP can look a lot like advantages,
at least in terms of its simplicity and durability. When a protocol has
survived and continued in use for so long there are usually deep-seated
reasons why it is a fit for certain use cases. I don't want to go off on a
chat vs. mail rant, but the following ...

>> You operate on a level of the entire messages flow. Forums, chats and web
2.0 networks are trying to make conversations feel as lightweight and barrier-
free as possible.

... caught my eye. I manage a team of younger developers for whom real time
chat is simply the way they grew up interacting. Everyone is signed into
hangouts and "operating on a level of the entire messages flow" (sic) all day
long. A large part of the interaction is off topic, vague, and ineffective.
Here's an example from yesterday...

Dev 1: the repo url is wrong, I'm getting a 404.

Dev 1: wait, might be a typo.

Dev 1: my bad.

Everyone on the team received this message and presumably stopped their
current mental task at least long enough to read it, as I did. In the end it
had no impact whatsoever. The developer clearly should have thought through
his problem before notifying the entire team that he had one. But this is how
chat is used.

I've tried very often to get them to be more deliberate in chat
communications, without any success. I have also tried to encourage
complicated topics to move quickly to voice hangouts, with even less success.
Streams of real time chat messages remains the primary way the group
interacts, to the point where I have backed off my efforts and come to the
conclusion that I'm likely the one who has to change.

That said, there will always be a need for deliberate communications within
organizations. Email grew up as an analog for letters and memos, which served
as deliberate, archived, time-stamped records of an important communication.
They were considered, usually minimal (because they took effort to create),
and got to the point. They were events, not streams of ephemera. I'm not sure
that this constant buzz of chats does a good job of replacing that. So for
that reason the sentence "You operate on a level of the entire messages flow"
doesn't make me smile.

~~~
m_mueller
I think we should look at the proposed idea as two-fold:

1) giving E-Mails more inherent structure that is managed centrally (streams)
instead of by each user (filters, folders).

2) making E-Mail more synchronous instead of asynchronous.

As far as I understand, you're arguing against (2). I agree with everything
you wrote, however I still find (1) a very good idea. It is also an idea that
IMO could be implemented entirely without changing the protocol - it's just a
question of where you put this additional information. Streams could also
simply be represented as a group address in the CC field that is managed
differently by their client/server architecture, but otherwise fully downwards
compatible.

------
erikpukinskis
Every now and then someone decides we need to replace email, or the web, or
Craigslist, and they think the reason it hasn't happened yet is that no one
has really thought about the details.

It's not.

The reason those things haven't been dethroned is they are he lowest common
denominator. You can't dethrone them by making something better. You can only
dethrone them by making something that is both better _and more common_.

By definition the lowest common denominator is crap. By definition it is some
legacy junk that was already on every computer 10 years ago. By definition
it's a hundred competing standards from multiple companies because how else
will a technology work on every device ever?

You can try to replace it with another tech, but it's going to have he same
kind of characteristics, and it's arc of adoption is going to look like
email's. Which means maybe 10 years from now it starts to look like a real
contender. And by then how much crap has been jury-rigged into the spec?

It's the long tail. It's janky, but it's a common standard that does one job
like a freaking champ. The long tail will never be engineered away. It ebbs
and flows. Only in the last 10 years is _snail_ mail losing its status as a
reliable way to reach people.

------
goblin89
To me this seems so different from email that it rather invites a comparison
with RSS feeds or Slack-like services.

What if it was a no-brainer to publish to and subscribe to feeds, there was
some kind of feed exchange and access control built-in, and everyone was using
them to communicate? What if Slack was open and decentralized, and everyone
was using it?

~~~
chimeracoder
> What if Slack was open and decentralized, and everyone was using it?

We tried that; it was called IRC. As far as I can tell, Slack is still little
more than a hosted IRC service with its own web and mobile clients. Virtually
none of the functionality they provide couldn't be replicated this way; it's
more a matter of branding than anything else that they eschew the label "IRC".

Slack even provides an IRC gateway so you can access it using IRC clients. I
don't know if their web and mobile clients are using the IRC protocol, but
from what I can tell, they might as well be, which is the whole point.

~~~
hobarrera
What we need though, is something federated. Where everyone can interact with
anybody, just like email (or like XMPP did).

IRC does not have this. I've to connect to your server, or you to mine. This
doesn't scale as an email/IM replacement; it only works for rooms with a
certain usage/target.

The _huge_ problem with any replacement (aside from federation) is the network
effect. No matter how good your solution is, you need critical mass for it to
pick up, and no huge company is going to invest into getting users into a
federated network which they won't control. There's no profit for the company.

~~~
vidarh
> IRC does not have this. I've to connect to your server, or you to mine

IRC most certainly _does_ have this. All larger IRC networks are exactly that:
Networks of federated servers. It's been like that since the early 90's, when
it was an absolute necessity as none of the ircds could handle all that many
individual users.

~~~
zokier
Nope. IRC model is closer to clustering than federation. The key difference is
that all connected IRC servers have nearly full control of the _whole_ network
which is why you can not allow untrusted servers in a IRC network. Also in IRC
all messages are routed to all servers. Contrast that to something like SMTP
or XMPP where every server is mostly independent and generally handles only
messages that belongs to them, and where federation does require very little
trust.

------
thawkins
Is this not just folksomomies for taging email, it will suffer the same
problem that tags do, consitancy of the tags, how will it know that kitties,
cats, felines and pussys are all the same stream. How will people know what to
name the content in the stream. How is this different to adding [CATS] to the
front of the subject line, and having the software partition on that.

------
falcolas
OK, silly question time. What does this concept of streams have which is
different from a mailing list?

mailto:(code-review|off-topic|blog)@tonsky.me?subject=[un]subscribe

Sounds more like he is suggesting improvements to mailing list interfaces than
email itself.

~~~
MetaCosm
That is exactly what I thought. "Stream" is the subject line. You mail to a
group on some sort of listserve, which gives you a long-term history and a URL
to give new people.

This exists in rather decent form in Google Groups already -- there is Google
Groups for Business (with a Google Apps account) that gives you a great web
interface as well as email interface all just for your company.

------
FreakyT
This concept seems quite reminiscent of Google Wave, if anyone remembers that:

[http://www.renepedersen.com/wp-
content/uploads/2010/01/googl...](http://www.renepedersen.com/wp-
content/uploads/2010/01/google_wave.png)

~~~
aroman
...the author spends a good amount of time in the article explicitly comparing
and contrasting his idea to Google Wave.

------
harel
We go through this a few times a year. "Email is broken, we must replace it",
only to realise a few short seconds later that email is not broken, and does
not need replacing. Its long lived for a reason - its simple, its textual, it
delegates all the fancy stuff to the inbox provider, e.g., videos in emails
are usually a link the email provider, say Gmail, shows as a video.

If I had 10 pence for every "email is broken" post, I could have had a free
pint down the pub.

------
mgkimsal
"Transferred to the internet, email inherited these limitations: electronic
messages are not real-time, they’re hard to manage in huge amounts..."

Anything is hard to manage in huge amounts, and anything _real time_ is harder
to manage even in small amounts.

------
zrail
Sounds like Usenet re-invented, where stream == newsgroup.

------
ytjohn
I read this and this that it's a lot closer to what RSS intends to solve,
especially the read-only streams. Two-way streams is mainly providing a method
to post back into that feed. It's a little more real time (rss feeds are
typically cached and refreshed), and a real-time implementation would e more
of an collection of entries that are pushed to the server presenting the
stream and ultimately to the client.

What I think With that in mind, I feel this concept is very close to a lot of
the p2p or decentralized content distribution systems being tested out these
days, like aether[1] or matrix.org[2].

The backend implementation that matrix.org uses is rather ideal. It's
Federated JSON over HTTP(S). The clients out today are heavily focused on
chat, but it is data agnostic. You could have client that writes rich media
content and a client that displays this as a blog. Going back to the streams
concept, you could have a client subscribed to multiple streams (rooms in
matrix vernacular) and as people post their messages - rich media or simple
"how are you" comments, it would get federated out to each home server, which
can present a web client interface like in tonsky's mock-ups.

For the concept of amazon providing sensitive order information as a
stream/matrix room, there would have to be an add-on concept built into the
client of some sort of security token. I'm not talking about a losable gpg
keypair, but something amazon generates and provides to you (like they do with
their AWS services already) Amazon would then create a personalized stream for
you, using that security token to encrypt. Since amazon provides the content
and the key, they can replace your key at any time.

[1]: [http://getaether.net/](http://getaether.net/) [2]:
[http://www.matrix.org/](http://www.matrix.org/)

~~~
jp10558
The problem I have with all of this is it seems like the XKCD where for some
reason, the latest person / thing seems to somehow not think to google if
there's anything like what they're doing, so we have yet another program /
protocol / community that 5 people use and then dies. EVERYONE uses E-mail.

And now with Aether - has no one heard of freenet and i2p and Tor? No, we'll
re-do all the sort of starting work on anonymity and security and get to re-
make for the thousandth time the mistakes where it really isn't that secure or
anonymous rather than building on anything that has at least some of the
actual battle testing and bug squashing already done.

What is Aether but Frost with less testing? Hell, what is it but TOR access to
Usenet? Maybe I don't understand what's so special that it can't use any of
the long worked on infrastructure level work that's gone into the above
projects for a decade or so, and it actually does make sense to start over
from day one - but if so, they certainly don't explain it.

And I'm not talking about UI - I'm talking about the anonymity equivalent of
deciding TCP isn't good enough so I'm going to re-invent IPX or something.

~~~
anonbanker
is there anyone doing an Aether-equivalent over I2P or Freenet yet?

------
tinco
Yes, this could work. I like the idea of streams being an url with a path that
might be a unique hash for you. The url's domain controller decides who is
authorized to post on their streams. You decide which domains to trust.

Most social services we use today could be streams. Facebook, Twitter, even
WhatApp, Instagram (I guess not snapchat). If they'd all implement some common
stream spec. Unfortunately they won't, Twitter has demonstrated that they
don't like alternative client implementations.

If it could replace e-mail is another big question. Nikita hilights the amount
of cruft that's around e-mails, but the cruft is social protocol, and we
humans love social protocol. Or perhaps that's just old fashion speaking.

The posting is quite old already (Oct 2014), so this isn't some big hype
building and it doesn't look like anyone is working on a big implementation.

~~~
rakoo
When you read the post, you know that XMPP has all the technical parts to fill
the role here. There are some people who build things on top of XMPP, but I
haven't seen something as advanced as buddycloud
([http://buddycloud.com/](http://buddycloud.com/)) here. They have a concept
of "channels" ([https://xmpp.org/extensions/inbox/buddycloud-
channels.html](https://xmpp.org/extensions/inbox/buddycloud-channels.html))
which can have multiple types with multiple participants, each having a
definite role. All of the use cases in the article are (theoretically)
possible with buddycloud.

Technically, the standard XMPP PubSub protocol with its PEP companion could
get you very far already, unfortunately there's not many clients that handle
them today.

~~~
anonbanker
jitsi does. who uses anything else for xmpp nowadays?

------
amelius
> It’s hard to believe people still use email. It has so many flaws yet still
> remains die-hard to replace.

Well, at least it doesn't live inside a silo, like most other messaging
protocols.

------
jakobegger
This sounds extremely exciting. But there's one question: how do I tell
someone to subscribe to a stream? Do I need to email them a link to my stream?
The killer feature of email is that it allows anyone to contact anyone else.

Also, issue tracking is something that could be replaced with a stream...

------
AlexeyMK
Neat concept. One challenge is imagining the transition from email to stream -
how can I start using streams internally with my company gradually, but still
use email when needed? Not another inbox...

~~~
tinco
I think you'd configure your client to pull e-mail as a stream or a collection
of streams (with filters). You can then in your client fuse or split the
streams into the rest of your streams. It could be a pretty fluent transition
in my imagination.

------
conradk
I dislike how half of this blog post is just ranting about email. For example,
I don't get this: "electronic messages are not real-time, they’re hard to
manage in huge amounts, they work best for person-to-person dialogs"

These are "limitations" according to the author. But IMO they're the reason I
like email:

\- I like to be able to reply when I want, without the sender having the
expectation of everything happening in real time, I have other things to do
than answering real time to everything.

\- Anything is hard to manage in huge amounts, which is why we hide less
important stuff. Email has folders where you can put stuff in. You can archive
emails. Or maybe this management issue is technical, as in "storing petabytes
of emails is a difficult task" ? In that case, I have no real answer.

\- I actually like the concept of mailing lists: they work where I already am
(ie my mailbox) and I can stay up to date just like in a forum. And person-to-
person dialogs do indeed work very well with email. I don't see that as a
limitation.

\---

Also, about this: "Stream is a group of threads. Simple as that."

That does not sound simple. In fact, just from that sentence, I didn't
understand at all what it meant. I had to read on to get the idea. But I do
see the point of "streams", and I agree that filters are usually complex to
setup and maintain.

\---

The part that I don't understand is this one: "With proper user/access
management Streams could replace not just email, chats and group chats but
also forums, blogs, mail lists, RSS and most of web 2.0 communities."

It sounds like what you need is RSS everywhere. RSS has a pull model too. You
can setup ACLs on an RSS feed if you have some kind of authentication in
place. It can replace mail lists, except that you'd still need to be able to
push messages to it, which I don't see anything about in the blog post.

\---

Finally, regarding this: "This fixes ambiguity around do-not-reply@ addresses"

I get the point of ambiguity and I have replied to no-reply addresses before
myself, with an error message coming back. But I think ambiguity could be
fixed with a header here, instead of throwing away the whole email
infrastructure.

Imagine a "X-No-Reply: true" header. That way, email clients could disallow
replying to such an email address. This has the advantage of being backwards
compatible with everything that exists.

\---

Overall, I do think that you can try something new, trying to "replace" email.
But realistically, Google has tried several times and is struggling. Email has
solid tooling and knowledge built around it, and I think trying to push for
its self-improvement is worth a look.

Slack is sometimes said to be an "email-killer", but the fact is it replaces
only one use of email. At my last company, we used Slack for real time stuff,
notifications, quick questions, but not for everything. And that's great.
Because email sucks for real time stuff right now (although initiatives like
JMAP may make things better).

Email seems to carry the notion of importance, which is not true of chat
messages I think. The reasoning behind this is, I think, an incoming email
stands alone, whereas a chat message is quickly surrounded by other messages.

Also, emails have a clear receiver, the receiving email address. This means
that if you get a message to "devs@project.org" and you are a dev, you are de-
facto concerned by this email, which is a reason to read it thouroughly. This
is not true of chat messages which do not have a "receiver" but are instead
posted to the more liberal notion of "channel".

~~~
glesica
> Email seems to carry the notion of importance, which is not true of chat
> messages I think. The reasoning behind this is, I think, an incoming email
> stands alone, whereas a chat message is quickly surrounded by other
> messages.

This is critical. Understandably, "chat" user interfaces don't make individual
messages stand out by themselves. This means that it is easy to miss a
message, or even a conversation. It also means that you can't delete
everything but the important stuff you need to reply to, which is trivial in
email clients.

------
fiatjaf
The only problem is that if email was replaced by Streams every form of
sender-initiated contact would be gone from the internet.

~~~
joshka
Would that be a bad thing though? What's the use case for needing it? Any
sender initiated contact assumes that the receiver is in acceptance of your
need to contact them. This is the failing property of email that allows for
and implies that SPAM will always exist. I buy something from Amazon. I'm
initiating the contact, so I provide the means for replying / providing an
invoice etc. (currently an email address etc.)

------
grinich
This is an awesome article, and has some similar ideas to what we are working
on at Nylas. [https://nylas.com/](https://nylas.com/)

I don't mean to steal this thread, but if anyone is interested in building the
future of email (including this) I'd love to talk with you. mg@nylas.com

------
robinhood
I don't understand why people want to get rid of emails. It's the best
asynchronous way of communication.

------
jv22222
I hate to sound synical but Ive both invented this ui and seen others invent
it a few times over. Even so I do commend the effort and thought process that
went in to the post.

Edit: To see a fully functional version check out
[http://plugg.io](http://plugg.io)

~~~
g8oz
Offtopic: Thank you for ezSQL

------
fiatjaf
It is possible to make Streams work based on existing email, by saving email
into a Streams server whenever the receiver opt-in to get that email (well, it
is filters again) as a Stream.

Also, let people publish Streams as Streams and non-Streams users can
subscribe to those Streams as email.

------
miguelrochefort
Hopefully that person doesn't think he's the first to think about any of this.

------
amagumori
e-mail at its best retains something unique to the written letter, a tendency
towards long, frank, thoughtful compositions that are sent with a lot of
aforethought and intention. i think if we fully move towards communication
regimes where the importance of individual messages is lessened (a chat /
forum thread type thing) we might eventually lose this style of writing and
the enjoyment of communicating this way.

------
jrochkind1
> When you start new thread, you don’t just start it by filling To: and
> Subject:, you also have to put it to some stream.

And right there, is why it won't catch on.

------
owenwil
All I saw here was what Google tried to do with Wave

