
Scaling email transparency - koopajah
https://stripe.com/blog/scaling-email-transparency
======
rattray
For some reason I was under the impression they used Slack at Stripe. Was I
incorrect about this?

In either case, to what degree would Slack allow Stripe to replace email
altogether? It seems set up to solve many of the problems their email system
was created for.

~~~
gdb
We do use Slack. It's great for real-time communication, and has made us way
happier than any other chat system we've tried. (In fact, a year prior we'd
tried out lots of chat systems, each time feeling like the switching cost to
try was higher than whatever incremental value we were getting. But when we
came across Slack, it was so obviously better we decided to try again. We
ended up pretty painlessly switching within a day.)

However, email hasn't gone anywhere. As soon as you want to interface with the
outside world, it's a hard requirement. It's also pretty hard to beat a lot of
the properties of email: arbitrary-sized blobs of text and attachments that
are fully searchable, taggable, organizable, and have read/archive states.

I think everyone would love to see an email killer come along (and I'd love if
that were indeed Slack), but thus far I haven't seen anything come close.

~~~
chimeracoder
> I think everyone would love to see an email killer come along

Can you explain this?

I've heard this sentiment before and never really understood it. The common
complaints I hear about email are usually inherent in the fact that it's a
universal communication mechanism[0], or a symptom of inadequate clients[1].

I can understand the need for better email clients (that's what Gmail
originally was, back in the day[2]). But I still haven't been able to find a
compelling reason that email itself is fundamentally broken in a way that
requires a new system altogether.

 _EDIT_ : I should acknowledge that privacy and security are obviously a major
issue, and email is by design incompatible with security (Silent Circle tried
this and failed, which is why they are creating an alternative protocol). But
I generally hear this complaint in other contexts, so I don't think that's the
reason.

[0] e.g. "I get too much of it" \- thirty years ago, people complained about
getting too much physical mail, simply because that's the way all important
work was conducted then (dead trees).

[1] Having email clients tailored towards email power users can make up for
problems like overwhelming volume - Google's "Inbox" app is an example of
this.

[2] Along with providing massive amounts of space (which was itself a hard
requirement in order to provide the many client-side improvements it brought).

~~~
grey-area
_But I still haven 't been able to find a compelling reason that email itself
is fundamentally broken in a way that requires a new system altogether._

I'd love to use something very much like email which had a few things like
this:

Verified identity

Public key encryption as standard (of content at least, possibly of most
headers too)

TLS everywhere

UTF-8 everywhere

Metadata for social presence so that twitter/fb/github/intranet profiles could
be referenced in mails and used for things like identity (see verified
identity above)

Standardised globally unique message ids (uris perhaps)?

Attachments uploaded to a server by the sender instead of clogging up
mailboxes, not fetched unless required

Maybe even mail uploaded to a server and not sent across the wire unless
actually requested - why do we need to send messages when I might be able to
infer from metadata that I don't want to read it 50% of the time? An API for
email clients which pull data as they wish would be nice, instead of the
current broadcast all the data model. This plus identity would make it easier
to block spam.

HTML with inline CSS for styling (clients could strip to plain text as
required, not send the message twice) - no JS for obvious reasons. We pretty
much have this already, but it'd be nice if it were just the standard.

Email is a great tool, but it really is showing its age - it was defined in a
different age where there was trust by default of network users and servers,
and it's been hugely exploited as a result. If it were proposed now it would
never be adopted. You could shoehorn a few of the above points into client
changes, but some things would be easier with a new protocol.

~~~
derefr
> Maybe even mail uploaded to a server and not sent across the wire unless
> actually requested - why do we need to send messages when I might be able to
> infer from metadata that I don't want to read it 50% of the time?

If you go there, you're basically talking about something that's more like
blogs with RSS feeds, rather than email. To me, half the point of email is
that it is "pushed": this allows to send messages to people you don't already
have pre-existing relationships with, simply by finding out their contact
details.

~~~
grey-area
You could push the fact there is a message at date n from x with subject y to
the recipient servers without wasting bandwidth on the message itself along
with attachments before you know it will be read. This would also have other
effects (possibly delete/modify after send, invert control from the recipient
to the sender etc), so it certainly would change the way email is used -
probably not everyone would agree with this particular point.

~~~
derefr
I would think that if you're already going to do the SMTP transaction, the
average (plaintext) email message body is small enough to slide into the same
packet as the end of the transaction headers. It's the same as keeping the
data of small files inside their directory-entry structures on disk.

On the other hand, it would indeed be interesting if email was simply a
"unicast presence notification" channel to allow accounts to publish knowledge
of a (private, ephemeral, encrypted, authenticated) message channel to other
accounts, and then messages flowed via automated subscription-based pull.
You'd still want your MUA to be a web service with active background polling,
though (like cloud RSS reader services), since otherwise a user could send a
message and then "retract" it from their feed before your own client had
retrieved it.

But this would solve at the very least the spam problem: you could simply
unfollow a channel that's sending you email, and receive no future messages on
it. It'd be like every (sender, receiver) pair having its own dedicated inbox,
that the true receiver consumes only voluntarily.

------
rnbrady
This is awesome. Being a payment company, how does this influence your ISO
27001 / PCI DSS compliance efforts?

I encourage lists at work but the first objection is often around our "need to
know" security policy, thanks to ISO.

------
dsl
I think this level of transparency is great while you are a startup. However I
have concerns over the "early discussions" and "sensitive partnerships." In my
experience that is where things have always managed to go the worst for the
employee base as a whole, not in the minor day to day operations of the
business.

Can you speak to how you bring transparency to things like this?

~~~
gdb
Email transparency is one tool for spreading information within an
organization. It excels at some use-cases (namely, freeing email that would
normally be locked up in someone's inbox). However, it's poor at other cases:
for example, it has no story for making that information more digestible (most
teams address this by sending periodic state update emails to the company).
Said another way, just lobbing an email to a list isn't sufficient for
transparency.

One of the biggest problems we've found with using email transparency for
"early discussions" is that it's often actively harmful (totally orthogonal to
the content of the conversations). For example, we once were considering doing
a particular acquisition. We approached it with full email transparency
internally, sending on-list emails before we'd really figured out the
parameters of the potential deal.

Immediately, people started to express strong opinions in favor or against it.
We had a full-company sitdown to talk about it, but it was really weird
because it was all just hypotheticals. Ultimately, we burned a lot of time on
it, and the deal was kind of dead in the water due to this approach.

The list of exceptions are the cases (either found empirically or through
inspection) where "absolute transparency" can be harmful. It's kind of like
the "shouting fire in a closed theater" free speech idea — you should make
sure your transparency mechanisms don't cause unnecessary emotional turmoil.

On the other hand, there tend to be good alternative mechanisms to make these
cases transparent, such as talking about them at your all-hands meeting, where
you can provide additional context and discuss in realtime.

~~~
derefr
> Immediately, people started to express strong opinions in favor or against
> it. We had a full-company sitdown to talk about it, but it was really weird
> because it was all just hypotheticals. Ultimately, we burned a lot of time
> on it, and the deal was kind of dead in the water due to this approach.

This sounds like the sort of problem you solve with a policy, not by breaking
the enabling technology. What about just telling your employees that nascent
partnerships—like nascent microservices—start out "owned" by the people
exploring them, and everyone can observe their development, but nobody has an
implicit right to have input on them until they get to the "release-candidate"
phase?

~~~
gdb
It's kinda (and in some cases maybe even irresponsible) unfair to drop major
things on people without providing more context. The point of email
transparency isn't to make everything transparent; it's to increase
efficiency. We don't need to cover _everything_.

Generally with similar initiatives these days we'll talk about them at all-
hands, or if they're particularly interesting have a dedicated gathering to
discuss them. We also find ways of increasing transparency once the need for
secrecy has been removed, such as opening up the list archives post-hoc.

------
sneak
> Early stage discussions about topics that will affect Stripes personally
> (e.g. changing our approach to compensation).

Why is this excluded from transparency?

------
fishnchips
I have too much email of my own to go around and read someone else's, email
transparency or not. In all fairness I think this is just creating a false
sense of openness at a cost of potential severe information overload (or
underfiltering).

~~~
gdb
Don't read it then! But it can still be useful to have a lot of auto-archived
email in your inbox -- for example, searching for a name or topic will often
pull up relevant threads you otherwise wouldn't have been discovered.

~~~
fishnchips
That's what mailing lists are for and it's hardly an innovation.

~~~
gdb
Yep. Email transparency gracefully degrades into just a plain old mailing
list.

------
horses
Can I ask what internal wiki software you guys use at Stripe?

