"In the aftermath of Google's announcement, some members of Congress are reviving a bill banning U.S. tech companies from working with governments that digitally spy on their citizens. Presumably, those legislators don't understand that their own government is on the list."
Some people on this discussion mention that they simply run their own mail servers -- that's dandy for incoming mail. But how do you guys make sure your outgoing mail is not blacklisted/ignored/considered spam? That's been a non trivial problem for me in the past when I was running a mail server (and spam was not such a big problem back then).
Also, is there any mail server you can run/recommend that has gmail-speedy searches and tagging? (And maildir support would be a super-extra-plus?)
I use a mail server stack that includes Postfix + Dovecot + RoundCube; RoundCube's search function isn't as good as Gmail's, but the ability to filter messages by literally any part of the message content just blows Gmail's tagging out of the water, IMO. (You need managesieve + a plugin for RoundCube to do this.)
I so far haven't had the mail server blacklisted even once since I started using it 18 months ago. I have a number of other customers on it, including one that sends out a 500-odd subscriber newsletter, using software I developed. I have taken a couple of precautions against spam: I use SPF records for as many domains as possible, the mail server itself is locked down tighter'n a flea's bunghole (including some proactive security measures), I have remote monitoring that keeps an eye on the server constantly, and (so far) I've had the pleasure of only doing business with people I trust.
It requires a lot of effort to build a good mail server, and there isn't a single "perfect" tutorial on the web for it. So, for a lot of people, it might not be worth it. For me though, I haven't touched my Gmail account in a very long time, and I really do love the fact that I have complete control over every aspect of my email (and my customers').
Hear hear! I use a similar set up, and it truly is great. It's even caused me to seriously consider a service that sets up a Linode for people that wants something as awesome.
Yeah, I've been sorely tempted to make a Stackscript for this setup and make it public.
There are two downsides: it's at least $20 a month to do it, and having your own mail server really isn't quite a set-it-and-forget-it deal. It requires an amount of attention that wouldn't make sense for a lot of people (or businesses).
Still, there might be considerable interest in at least some tutorial blog posts ;-) Consider this a subtle request :-)
I've been toying with this idea for a while, but haven't really started looking into it seriously, mainly because, as you said, at least at a cursory glance there are not a lot of in-depth and accurate resources for setting up a mail server with security in mind.
Keeping most of the software reasonably up-to-date isn't too hard; on Debian, for example, just regularly use apt-get update && apt-get upgrade. (Hopefully you have backups ready in case it goes sideways.)
Probably my number one time sink on the mail server is, "Where is the email from X that I've been expecting?" A close number two is, "I'm getting too much spam." If you're paying for hosted email from someone else, you can just give them a call and make it their headache. If you're doing your own, it's your own headache, and it's a headache that can easily consume a few hours at a time.
You also should regularly check in with various system and software logs, to make sure that someone isn't trying a novel new attack against the mail server, or that a process isn't struggling for some reason.
Most of the software does a pretty good job of being rock-solid, so I don't find myself having to tune things all the time or nudge dead processes or anything. But, I did find that if I left the mail server alone for a while, then when something did go wrong, it took me a lot longer to resolve it (I think mostly because I'd forgotten where everything was), which is not what I want in a downtime situation.
I'm surprised by how many businesses (even small ones) don't run their own mail server, or at least have a proper outsourcing arrangement rather than just using a public service. We run our own (well, I run our own...) because we work with banks and all contracts we sign with them have clauses regarding where information from them gets stored and who could possibly have access to it - this is to protect their data in instances where we might be sent (intentionally or otherwise) information about some of their employees or customers. I'm guessing a great many businesses work with clients who have similar concerns so those clauses will be present in contracts that have signed too, so using a "public" service like gmail or hotmail just isn't compatible with them - we can't make any demands to Google about who can access what on their servers or audit them.
No business should use a public service like that and leave the mail on it. This isn't a dig at the public services like gmail, as they provide a valuable resource for those the resource is not wrong for, but they can not provide the accountability I would expect to be able to provide my clients as a business. Businesses should (IMO, and in order of preference) run their own server, use a service that has some contractually enforced security guarantee, or pull down the mail to local systems rather then leaving it on a public server - otherwise they can have no hope at all of controlling who can access their (potentially confidential and sensitive) mail.
Aside from the data security issue there are other potential problems that should be concerned about. If a public services goes down there is nothing you can do to help a fast recovery and you will not be their priority: your services will be available again when it is available again. Also you need to implement a good backup system no matter what you chose - you should not (as many people do) rely on a single service for both your live mail handling and backups.
The $20/month is nothing to a business (or should be), but you are right in that a mail server should never be considered a set-and-forget system so there will be technical resource cost involved with running a mail server and dealing with possible issues like "friendly fire blacklisting" so a reputable outsourcing arrangement would be more cost-/manpower- effective for many small businesses.
FYI: we currently use Zimbra's "community edition", though at some point I'd like to convince the powers that be that using the paid edition would be worth it for the support (there has never been an issue I can't resolve, and there is never likely to be, but I'm not here 24/7 and don't have someone with the right skills to delegate the job to when I'm not around). It is more resource hungry than Postfix+Dovecot+RoundCube though so needs significantly more the linode's $20 VM product to be usable, but I recommend people give it a try as its single install removes the need for you to perform any integration work putting a stack together and the feature set aside from email is no unattractive either.
You have a good point. We just recently deployed an in-house mail server for a client with security concerns; they wanted all of their intra-office email to not leave the building. Our mail server does backup duty for their mail server, in case their connection drops or anything goes haywire, and we monitor their server and keep it healthy for a really low monthly amount.
I had only sorta-kinda considered trying to offer that to a wider audience, but I didn't really think the market for it was that big. I might be wrong.
I think something like this could be valuable to a lot of small to medium companies who atm can't resist subscribing to Google Apps (50,- per user you could have instead). I think with a service like that, the main goal would be to stay as simple as possible while still offering the "Exchange" integration with people's current infrastructure.
I would do the user side web frontend last and concentrate on a simple but empowering foundation plus administration interface first.
Clearly with the PSN breach and all the bad press Google, Apple & Co. are getting with location/mobile I can see a shift in people's minds towards more privacy and security similar to when people started to get used to anti-virus/firewall software.
A server in another data center does differentials of the entire mail server every hour on the hour, and weekly fulls. Differentials are stored for 14 days and weeklies are stored for 120 days. If a customer accidentally deletes an individual message, we can probably restore it; if a customer nukes their entire mailbox, we can restore it.
> And what happens if it goes down for a few hours? Would you not get that mail?
SMTP servers by convention are supposed to retry for at least a day if the destination server is down. A few miscreants (ahem Washington Mutual...) don't retry at all though.
Worst-case scenario, I can have another mail server up and running in 20 to 30 minutes, with most of the mailboxes restored to it. Since I also admin our own name servers, and I keep a short TTL, we could update the DNS for the mail server and have most mail services back online for most customers pretty quickly.
Unfortunately, true fail-over mail services is a challenging nut to crack, especially if you want duplication between distant data centers.
I run my own mailserver too for private use. I make use of a DNS provider who also provides backup SMTP service.
Basically it adds another address as alternative for delivery for your domain. That server is configured to forward the mail to your primary server or hold it until it comes up again.
Works great, and you could even use it as a spam filtering option, since legitimate servers should only try the first server, but a lot of spammers use the second one thinking that it probably has less spamfiltering or something.
Wouldn't whatever party you're communicating with also need to run their own mailservers? Otherwise the government or some other intruder could just read your conversations from the other end...
I too have been running my own mail server for a couple of years now using Postfix (SMTP) and Courier (IMAP). I never got around to setting up webmail.
Despite not doing much beyond SPF, I have never had any problems dealing with my outgoing mail being blacklisted/ignored/considered spam.
Downtime has not been a serious issue for me as my host (Linode) has generally been very reliable. Most of the downtime my mail server has experienced has been due to me. If having 100% mail uptime was really important to me, I'd set up a secondary server at a different data center.
For backups, even though Linode offers backups, I use Amanda to back up everything to my local backup server.
For security, I allow TLS connections, but relatively few servers eagerly try to encrypt connections. I also use full disk encryption on my server. Ultimately, I think that if you have sensitive information to e-mail, you cannot rely on the mail system to protect it. You should rely on something like GPG to encrypt the contents of the e-mail.
As far as searches and tagging go, that's really a mail user agent problem. So that needs to be solved either in your webmail or mail client software. I do run maildrop to do some preliminary sorting on a lot of my e-mail. It's a program similar to procmail, but it works my virtual mailboxes.
In the end, running my own mail server has been a good experience. It took some time to get it properly configured, but running it day-to-day hasn't been much of a problem.
I use PostfixAdmin to manage things like adding new mail aliases. It's pretty easy to use. My wife, who is tech-savvy but doesn't know anything about running a mail server, uses it to manage her own mailboxes
I do run my own mail server of which I use SMTP fully; incoming email OTOH goes straight to a gmail account. I don't have problems being flagged as spammer, as far as I know.
The main risk running your own server is downtime/screwing things up/intrusions and you need to work a bit on that but that's part of the fun for me.
"The rumor that China used a system Google put in place to enable lawful intercepts, which I used as a news hook for this essay, has not been confirmed. At this point, I doubt that it's true."
all this evil isn't done by some special evil people. It is you or people like you who does it.
people like you write the laws mandating backdoors, people like you force companies to implement it, people like you actually implement it. After all that, you dare to express displeasure with the thing you done pretty much by yourself to yourself. Man up and take responsibility for your actions. Next time you're groped by a TSA agent, you can find a relief in the thought that you (or your friend working at Google) groped the agent's Gmail account. Tit-for-tat.
Damn I wish more people would realize this. Our brains "need" a clear antagonist, someone to pin the blame on. But that's not how it works, rather it's an agglomeration of many small attitudes that result in an emergent phenomenon known as culture- every time you say "hmm we need this or that!" - or more likely, every time you don't question authority and instead just roll over and accept it- you are feeding into the fucked up system that results in evil. It's not special evil people doing it. It's us.
That's simply not true. Even in Nazi Germany there were pockets of resistance and people like Sophie Scholl who gave their lives because of their ideals. I for one am doing my best to resist tyranny in all its forms. I certainly won't write a backdoor or vote for anyone in favor of Patriot Act-style tyranny.
>Even in Nazi Germany there were pockets of resistance and people like Sophie Scholl who gave their lives because of their ideals.
that's exactly the point. Once you really start to not being part of the evil, you live gets really impacted, to put it mildly.
>I for one am doing my best to resist tyranny in all its forms.
so, how do you refuse to take your shoes off and quietly wait for the TSA agent to complete groping a child ahead of you? You just don't understand what you're talking about. You still have Internet access, and it sounds like you aren't even arrested. You do your best "armchair warrior" style. Have respect and don't mention people who gave their lives in the same paragraph with "your best"
>I certainly won't write a backdoor or vote for anyone in favor of Patriot Act-style tyranny.
you may have already wrote it, yet you'd not be able to disclose any details about the NSA letter that instructed your so and to avoid committing the crime you must to continue saying "i didn't write a backdoor" :)
So far, as far as I know they haven't executed anyone in this country for speaking out again tyranny. I know "my best" pales in comparison to people like Sophie Scholl, but that statement shows my optimism about the current state of affairs. We are on a decline, but still far from outright Fascism.
> So far, as far as I know they haven't executed anyone in this country for speaking out again tyranny.
I'm sure you know that Julian Assange is facing possible extradition to the US, and several top official are shamefully talking about dusting off the Espionage Act as a way to seek the death penalty in this case. The fact that he is not American hasn't seemed to deter them. Nor has the fact that other journalists and media outlets like the NY Times printed the same documents.
What is so scary is you have other "journalists" basically joining in to criticize WikiLeaks and JA, not realizing that they are setting the stage for future silencing of other members of the press.
>So far, as far as I know they haven't executed anyone in this country for speaking out again tyranny.
and the point is? there is no evil happens? or there is no need to bother until executions start? In Stalin's Soviet Union millions were executed and nobody bothered even then. Anyway, executions are so 20th century. The modern autocracies', like modern Russia for example, have found a way of doing pretty much the same thing without mess of executions. Modern citizens are so tender, they value their comfort and good job so much...
>my optimism about the current state of affairs. We are on a decline, but still far from outright Fascism.
You could pgp encrypt anything that you really don't to be read in transit. Short of that, you might as well assume that your emails are being read and stored by third parties.
Even if you personally use the most secure email server in the world, it doesn't matter because everyone that you send email to or from is likely using hosted services like gmail, verizon, hotmail, etc.
One big point here is that PGP only works when both sender and receiver and doing it. Try sending Bank of America a PGP encrypted email about your account. See what happens.
Edit: Re-reading my response... I don't mean this in a condescending way and I agree that PGP is a good way to handle email privacy. I only meant to point out that the majority of people don't use it.
All of your incoming email has travelled through so many insecure servers on it's way to you that encrypting it on receipt is like closing the barn door after the horse has bolted.
Try sending Bank of America a PGP encrypted email about your account.
Well, you have to use their web form anyway, which is encrypted. The reason they require a web form instead of email is for exactly the reasons PGP exists: they need to know that you are sending the email, and they need to know that someone else isn't reading their reply to you. Webs of trust are hard, a text box on their SSL website is easy.
If you use a hosted service, then unless it is specifically set up to prevent it (as with tarsnap for backups, which requires key management on the end-user's part), the people hosting it will have access to your data.
If that's a major issue, then your only recourse is to host it yourself.
Running a mail server isn't terribly hard, but it does mean that when the server dies you're on the hook for fixing it. If you're not a sysadmin, then the pragmatic choice is probably to accept that Google doesn't care about the details of your email and will only be spying on you in aggregate to determine what adverts to display.
i host my own server, but the best i found - about a year ago when i looked into this - was a norwegian company called runbox http://www.runbox.com/
their support was also friendly, helpful, and technically clued up when i discussed some issues (i didn't go with them in the end because i've hosted my own email before, know how to do it, and decided it wasn't worth paying anyone else).
[edit: to be clear, not only do i have more faith in the impartiality and openness of a scandinavian govt than the usa, there's an advantage in simply not using a host in your own country, imho.
also, running your own server isn't hard. with opensuse, for example, it's pretty much just a case of (1) clicking the right options in yast and (2) configuring getmail to pull your email from your isp. although to get something anywhere close to gmail in functionality you also need to understand (and use) procmail and mairix. then you need a client - i use mutt over ssh, but you can also install a webmail soln like squirrelmail if you want.]
We could ask Google. They've tested a PGP feature in Gmail back in Feb. 2009, so I'm sure they would (try to) release it publicly if we ask for it loud enough.
Under U.S. laws, providers are often required to give information to the government if they have a warrant. If you want (more) security, host it yourself.
True, however some providers make attempts to thwart this by making it impossible for even themselves to access mail by way of encryption. I haven't looked at it in a long while, but lavabit.com was one such company, and it's a pretty good raw POP/IMAP service (at last check the web interface and spam measures were somewhat lacking).
I like what lavabit has done, but at the end of the day, if they are forced to, they can modify their server side software to log your password when you log in, and then use that password to decrypt all of your email on the server side.
It also doesn't secure the email on the client side. If your IMAP client stores the email on disk, then you need to make sure it is encrypting it in a secure fashion first.
Lavabit should offer an extra layer of encryption whereby they allow you to upload a public pgp key which they encrypt all your incoming email with using PGP/MIME.
I don't think even what you said is sufficient -- as long as they are within the jurisdiction of the US court system, I think a judge can order them to start saving copies of incoming mail before encrypting, etc.
I think fundamentally you can't circumvent the law with technical measures. You need to change the law to require warrants.
Sure. The method I stated would protect historical email, but it wouldn't prevent the capture of new email after an order has been made to start saving it.
It's not necessarily 'spying.' Many large service providers have such as system for logging in as a user to aid in troubleshooting. Yahoo's is known as AMT for example and lets customer service (or anyone with high enough access clearance) 'be' that user, to see first hand the issues the customer might be dealing with.
For everyone who thinks they're an email pro, or never come across problems, there's thousands of users calling any support number they can to try to understand why their junk mail folder won't empty.
For me, I use TrulyMail. If another TrulyMail user sends me a message, it doesn't go through any email server (just TrulyMail's server). Everything is automatically encrypted /decrypted on the client so even if someone access their servers, there is nothing readable. They also have email encryption. They are also priced right: free.
PGP is great and I used to use it but, as many posters have said, if the other party doesn't use it (or if they use it and you don't know they use it) it doesn't do much good. This is really true of all encryption systems.
A good bet might be a Swiss hosted e-mail service such as Neomailbox which is what I use. Lavabit is another one which might be what you're looking for.
Living in Switzerland and knowing the laws, I would strongly recommend against that: since 2000 we have a law that forces all email providers to hand out communication logs to the authorities and to retain them for at least 6 months.
Granted: that's not the message contents, but it's bad enough. Also, from my experience I can tell you that the authorities do make use of this law even if it's just to track down a student anonymously badmouthing their teacher.
No. Stay away from Switzerland if you want your correspondence to be hidden.
That data-retention law is for the entire European Union.
6 months is the EU minimum, but for some reason the Netherlands decided to require triple that. I bet some idiot politician thought it must be "extra secure" ... :-/
I had a sdf.lonestar.org account. Once, when their mail servers appeared to me to be down for a day, I asked on the discussion forums if I was the only one experiencing problems. The site administrator posted a response right away on the forums.
He told me to stop whining, then explained to everyone that I received too many emails per day. (I was receiving <100 emails per day, and I was nowhere near my disk quota) He then named the most common sender of the emails I received. He did all of this in a public forum.
Not only would I not trust SDF not to leak the content of your emails to unauthorized parties, I would not trust SDF not to simply read your emails for shits and giggles, or leak their contents in order to teach you a lesson for asking rude, nosy questions like, "Hey, is anyone else having problems with the mail server?"
So let's go with a rational assumption, which is that your email provider has the capability to read your email (scenarios: warrant, hacker, bored sysadmin).
You, being a good geek, encrypt your personal systems out the wazzoo.
Then you want to take the next step: encrypted communications (examples: legal, business).
This now makes your request of everyone you deal with to dink with public/private keys and - likely - some sort of infrastructure.
What's the best real-world (i.e., non-propellerhead) solution to this?
What's the best real-world (i.e., non-propellerhead) solution to this?
There isn't one. I've looked at creating one, and while its a sizable engineering job all the pieces are available. What it isn't is monetizable. Not like 'make me a gazillion dollars' monetizable, but like 'pay me a living wage to work on it' monetizable.
The key (and its a horrible pun) is the key. You can build zero knowledge proof [1] key exchangers now (patent expired :-) and a relatively inexpensive 'key' based on either USB or bluetooh communications (see Yubikey [2] as an example). Such that email to a new third party could be done in an encrypted way such that the message could only be read when that exact party was reading the email in a reader that could get the keys to unlock it from the physical key.
Like most such systems it only 'makes sense' if everyone (or at least a large fraction of everyone) has one.
To get initial adoption it needs support in 'free' tools which means it needs to be open in the sense that folks can trust what it does, and implement a compatible protocol without paying you anything.
To earn a spot in your pocket/purse/pack it needs to be flexible enough to accomodate other uses. To get those other uses the folks who provide them need to be able to support it for 'free' since their customers won't be paying them to put it in.
So a large investment in propellor heads to make it usable by the rest of the world and achieve critical mass for adoption. Oh, and if you do start getting traction the governments of the world are going to want to disappear you (which was one of Bruce's points)
If you have stuff that you need to keep secret from your government, only discuss it face to face.
If you have so much secret stuff to talk about that its not practical to do it all face to face, then there's probably enough at stake that setting up some simple key exchanges and keeping a tight protocol will be the least of your worries.
Well, there's a scale of "likelihood to be intercepted" on the left and "practicality" on the right. I'd imagine that whispering something in someone's ear with the shower running in the background would be in the far left end, and email would be the far right.
As you move to the right, I imagine that OTR for IM is a very good compromise, followed by PGP, to a lesser degree. Those two should be good enough for most communication, and then there are some encrypted VoIP solutions that I don't know about.
Not that I think this would truly help (they probably replicate all inbound/outbound mail to some vast pool for statistical analysis), but is there an automated way to delete all mail from one's Gmail account? I use fetchmail to remove mail from my inbox and manage it on my workstation, but must periodically log on via the web interface and go to my "All Mail" folder and do a manual select and delete.
Mail removed from the inbox (or most other "folders" on gmail) works as expected. However, they still remain in the "All Mail" folder, which doesn't appear to be accessible to POP/IMAP clients.
Unless you host your own email, I assume that even without this specific backdoor built in google would have little trouble getting at the email they host.
This is not necessarily bothering to me, or unexpected for hosted services.
If Google can access my data anyway(even though its a pain in the ass), they can still comply with US subpoena's. I would imagine its easier to just automate the process(with a backdoor of sorts) then have to mess around doing this all the time.
I don't see any details on the actual backdoor in the article, so I hesitate to jump to conclusions.
SSL is for end-point to end-point security. SSL does not attempt to solve the problem of either end being compromised. This means that SSL won't help you if your computer is infected with rogue software or if the server you're talking to is compromised.
SSL does, however, help to prevent eavesdropping between point A and point B.
This is true but not relevant to this discussion. Any commercial provider would have to comply if the government asked information about their customers.
Gave me a good laugh this morning...