Hacker News new | past | comments | ask | show | jobs | submit login
Solving My Email Problem (nawaz.org)
179 points by BeetleB on Sept 29, 2018 | hide | past | favorite | 125 comments

My inbox is at 0 and other than gmail's auto-filtering tabs I don't have too much extra filtering going on (most of it is related to Google Alerts).

For the last 20 years my rule has been "If I check my email and inbox > 1 then deal with the email right now". I usually check my email every few hours.

I'm not Mr. popular, but I'm also not anonymous. I do personal support for 30,000+ people who have taken one of my online courses so I get a fair amount of email and other things. I tend to write massive responses from scratch for a lot of them as well.

The key is not to let it build up. Then you'll have 99 problems but email won't be one of them.

Also I'm not sure if this is related but I'm one of those people who think it's unthinkably rude to ignore someone. I would never in a million years read an email and not respond to it (unless it clearly wasn't meant to be responded to). Likewise, if I see a > 1 inbox I feel compelled to handle it ASAP because I don't want to keep that person waiting.

(Of course there's exceptions, like I'm not responding to pure spam, but I do get about 50 emails a week where people ask to post a guest post on my blog and I always manually respond nicely to them with a "no thanks but good luck" type of message along with some context around their niche).

I can't really relate to this -- my relationship with email is pretty different. Lots of my emails represent an ongoing todo item that requires some hours of work, or just some time to reflect, so I can't deal with it immediately.

I could turn each one into a TODO item tracked somewhere else and archive the email, then later search for the email and reply or take action. But it makes more sense to me to categorize the emails and use an organized inbox itself as my todo list.

This is the system I use. I've tried lots of different methods but this is what works for me. The best thing about it is that it's not based on heavy process (Note: YMMV because this was written before the recent Gmail redesign). https://xph.us/2013/01/22/inbox-zero-for-life.html

I like the ideas behind the post. My approach is similar except I do triage into a few different tags. A nice effect is that filters on incoming mail can do a lot of this for you, like almost all automated emails can be tagged as such and checked over as convenient. (I'm not using gmail for this but I think you can do all this in gmail.)

This is a wonderful, but unfortunately very naive, approach to email.

That you firmly believe this approach just means you haven't worked in a govt. dept. or any kind of position where your response represents the response of the organization (or a sub-set thereof). In such circumstances, emails that are open-ended (responses risk being misunderstood), ambiguously phrased (responses will be twisted to server the author's needs) or controversial (any response paints the organization in a negative light) are often best ignored. If your question is ambiguous, a busy boss or colleague does not owe you an answer or vice versa.

Sometimes people respond out of anger or frustration. You could respond in kind, and escalate the battle, or simply wait and let the individual calm down. Not responding is a very useful technique when conflicts arise between team members or in the management chain.

I would rather ignore an angry abusive email from an upset colleague who is having a rough time in life, than respond in kind and escalate it into an HR battle. Of course if the behaviour continues, there would be no choice, but most people are rather contrite and apologetic after a brief outburst and it's much better for everyone to just pretend that it never happened. People make mistakes after all.

Sweeping stuff under the rug helps in the short term but not the long term in my opinion. Compassionately responding to an abusive Email (even just saying, "I can understand why you were so angry...") might be the very response that person needs to dissipate their anger or re-evaluate how they communicate with people and that helps everyone they interact with after you. Sure, things can get messy too and some discussions are easier offline, but I think there's merit to completing conversations even if they're difficult.

Sometimes yes and sometimes no.

> This is a wonderful, but unfortunately very naive, approach to email.

I haven't worked in a govt. dept but I work in a situation where my response represents me being in business or not.

I'm dealing with 30,000+ people who email me all sorts of questions, with varying degrees of niceness. Some people are cool, others are over the top passive aggressive. I get very open ended questions, tons of "it doesn't work" with no details, etc.. My job is to guide those emails into a resolution.

If I treat anyone poorly (saying bad things or ignoring them) then I'll quickly be out of business because I don't have a monopoly. I take customer support very seriously. It's not even what I do full time either (although I'm always on call), it's just a side effect of running an online course business.

You could say instead of having 1 colleague I have 30,000+ all of which who have their own personalities and struggles, none of which were filtered through HR. I once had someone get pretty rude for no reason and then after a few emails we fixed the problem. The next morning he wrote a ~1,000 word essay to apologize because his father died recently and he drank a 12 pack of beer when he initially contacted me.

I always go down the kindness route and after answering over 5,000+ emails and questions I've honestly never had a negative experience where either party left the conversation upset.

I understand your premise. Customer support through email is much like customer support in person. I spent six years doing in-person customer support while working through university. But email is a lot more than just a medium for handling customer support, I do hope you realize that. For instance, you can hire or fire a person over email, sign or terminate a contract, promote or demote, change organizational structure, among others. It's a powerful tool. Certainly when responding to clients, early is best, even if only to say "I'll look into it right away."

This is fantastic emotional intelligence email advice.

Am I so weird that I don't have problem with my e-mail box? I don't receive spam (gmail, but recently all mail providers got really great at it and even if I do receive some spam is quite sporadic). I don't receive newsletters because (shock!) I don't sign-up for them (and even if I do once every blue moon I immediately click 'unsubscribe').

Yes, there is some small problem of having older messages "waiting for the right time" but this is not the problem of the e-mail itself but rather mental block to dedicate 1-3h to write response (you would spend same amount of time writing regular mail).

No, this is just good email discipline.

Almost everyone I know who gets way too many emails and complains about 100s or 1000s of unreads has put themselves in that situation by signing up for everything. Spam is already handled well and there's just not that many sources of email if you take away newsletters and notifications.

You’re not the only one! I unsubscribed from mostly everything and now my inbox gets anywhere from 1 to 2 new spam messages per week. Still annoying when it does happen, but a big improvement from before when I was getting 40+ emails per day, in which most were newsletters I never opened or just spam

>I unsubscribed from mostly everything and now my inbox gets anywhere from 1 to 2 new spam messages per week.

That would only work for a while for me. 6 months later new stuff has crept in and I would have to go through another round of unsubscribing.

> That would only work for a while for me. 6 months later new stuff has crept in and I would have to go through another round of unsubscribing.

OK, but why wait for 6 months? Be proactive - you see something you don't want the either use unsubscribe or filter them out (if it comes from the same sender, but most of the time they give you unsubscribe link which just works…). One annoying thing with unsubscribing is that often they require you to go throu a couple of steps (including providing the e-mail itself) and confirming it so sometimes only clicking on unsubscribe doesn't work with one click.

And to make e-mail a nice tool tune what you receive - if you get gazzilion of notifications that you don't read then just unsubscribe…

>Be proactive - you see something you don't want the either use unsubscribe or filter them out (if it comes from the same sender, but most of the time they give you unsubscribe link which just works…).

I addressed this in another comment:


Consider the two methods:

1. Just have a whitelist that I've described

2. Be proactive in unsubscribing. If you buy something from someone online, be proactive at the time you buy in finding the box which says you will not receive pointless emails from them (not all vendors will have that option). Ditto if you or an app you want to run requires your email address.

My question to you: If I go the proactive route, what do I gain compared to the whitelist approach? Yes, I could be proactive and actively unsubscribe. That is time lost - even if it's not too much time for some people - what is gained by doing it?

>And to make e-mail a nice tool tune what you receive - if you get gazzilion of notifications that you don't read then just unsubscribe…

I think the difference between me and many of the commenters is this: I want my inbox (and my email usage in general) to be for personal correspondence. A few exceptions are OK, but I want the majority of the emails in my inbox to be personal emails. So it's not about a gazillion notifications. I probably get less than one personal email per day. If I get more than one non-personal email/notification/confirmation receipt etc, then my inbox is more noise than value.

So - instead of doing a little work on your part you are forcing others (in most cases people that you DO WANT to communicate with) to jump through the hoops so they can communicate with your? In most likely event they will give up on the attempt and marketoids will simply find a way to circumvent the protection.

Again - you make it seem like it's gigantic work to keep your inbox tidy. Last time I had to click unsubscribe was months ago and it happens like 1-2 times a year - is that too much work?

I've addressed your points in more detail in other comments. A summary:

1. If someone can't be bothered to spend a minute in their whole life to be able to communicate with me, it's a pretty strong signal. I've had plenty of people who imply they want to communicate with me, but then are very poor responders to my emails. A hoop filters those people out.

2. No one has complained. If anyone is annoyed, they just ignore it and I'll whitelist them anyway when I see their email.

3. Marketers will not invent a way to circumvent it, because it's too much trouble for them to. I'm not that important.

4. Keep in mind the system has been working for over a year. This isn't a proof of concept or something in my imagination. Furthermore, if you read the other comments, people are paying money to have this feature.

5. Just because you don't get annoying emails doesn't mean others don't. Roughly half of them are unsolicited (I did not request to be on those lists). I think more likely is that the automated emails you do receive are ones you want. Whereas I don't want any in my Inbox.

My final point: Given where I am now, what possible reason would I have for switching to your workflow? Your stance comes from one who has a "default" position: the status quo. Look at it from my side - my system is working, and is very low maintenance.

If you unsubscribe every time you see a new e-mail that you’re not interested in it’ll help keep it in check.

For spammers who don’t respect unsubscribes and you’re using Gmail, you can create a filter which immediately trashes the message based on the “from” address

>If you unsubscribe every time you see a new e-mail that you’re not interested in it’ll help keep it in check.

That statement gets to the crux of the whole situation.

Why should I spend time tending an inbox, continually scanning for the unsubscribe link in any new email I receive? The sender put in almost no effort to send me that email, whereas I the recipient have to expend time getting off their list. I don't want to do that for the rest of my life. I'd rather just remove that email from my whitelist (one keystroke).

>For spammers who don’t respect unsubscribes and you’re using Gmail, you can create a filter which immediately trashes the message based on the “from” address

Same problem as above. I tried it and in the end found maintaining a blacklist (which is what you're essentially suggesting) burdensome. It's continual work for the rest of my life adding people to a blacklist. I inverted the whole thing. Let the sender work to get on my whitelist.

Makes sense. But I’m not sure inverting is really what you’re doing. It seems to me that the people most effected by this will be people who actually want to reach you - Friends, family, work associates. They’ll have to jump through an extra hoop until they’re on your whitelist. On the other hand the spammers don’t really care if their message hits your particular inbox or not, as you’re just one of very many people they’re sending to. Still a cool solution though, and kudos to you for devising a system that works for you.

>It seems to me that the people most effected by this will be people who actually want to reach you - Friends, family, work associates. They’ll have to jump through an extra hoop until they’re on your whitelist.

Fair point. In practice it has not been a problem. If I really want to get emails from them, I can always add them myself if they do not want to go through the trouble. On the flip side, If someone I know isn't willing to spend a minute once in their life to get access to me, it does send a strong signal on how much they value knowing me.

Im in the same boat, never had a problem managing emails. I get dumped on marketing emails but I make it a commitment to unsubscribe and it doesnt take terribly long. Most marketing spam I get is legit marketing with legit unsubscribe services. For work I just use thunderbirds bayensian junk filter, and spamassassin on mailhost and it works great. Filter out emails for important clients and for automatic system updates from vendors

Gmail does a lot of filtering for me though already. I write a few filter rules if I intentionally want spammy emails, e.g. watching github changes, status on shipments, etc.

My rule of thumb for managing email is thusly

- if the email is read, its done

- if theres 100 spam mail, and the top link is marked read, all ones beneath it are too spammy to delete or remove out of sheer laziness

-if the email is starred, I need to refer back to it, but its not actionable (e.g. welcome email for hosting sites)

- if the email is marked important, defer it for later

I spend very little time on email per day following these rules. I have many filter rules set up so nothing but important emails get to me in designated sections

My rule of thumb for email is thusly

- only look at email 3 or 4 times at most. Once in morning, afternoon, evening.

- send emails asap if it can be done in 5 minutes or tell receipient about a delayed response later for longer actions.

-modify subject titles to reflect status on email thread

- DRY, dont repeat yourself. Send a link instead. Use autoresponders if your on vacation etc

- every month or two, go through all your spam, unsubscribe to everything

- use autotext expanders to write email signatutes if its necessary

-use ctrl + enter to send

- limit yourself to zero to two mouse left clicks per email. If you are doing anymore, you are doing it wrong. E.g. dont organize emails its a waste of time learn to use search and make rich subject email titles.

Im 100% on the same boat. I’m not too strict in regards to handing out my email, but my inbox (Gmail) is completely kept free of spam. Is it really just gmail keeping me safe?

My solution is to have my own domain, and a little tool to create and delete email aliases. When a company asks for my email, I just give them a new unique alias. I receive spam on this alias, 1) I know who leaked, 2) I delete the alias, stopping the spam. It is transparent to my correspondents (I can reply from the alias), cuts the spam pretty much to zero, and is low maintenance. It also has a small security benefit: even if I were to share a password between websites (which I rarely do), the login (email, or login part of the email) would be different/non guessable.

I do something similar, except rather than creating aliases, I just have a catchall that sends everything to my inbox. Then, if one address starts receiving spam, I create a rule to move all emails sent to that address to spam.

As far as responding from aliases, I haven't figured out a way to do that in my mail client (Mail.app), though I am able to accomplish it in my provider's web interface.

I'm doing something similar with an Office 365 business essentials subscription (~$5/month) my main email address only has the most important stuff while everything else I setup a shared mailbox for. then every shared mailboxes forwards anything received to an office 365 group called Catchall which i have available on the main address and i just need to switch mailboxes if I want to reply from it (you should be able to do this through mail.app afaik).

this allows me tighter control on a case by case basis and I get all the benefits of Office365 business.

Gmail has this capability built in because you can just add +whatever to the end of the name and it's an alias to your email account, and you can block the yourname+whatever@gmail.com it as needed. Only problem is many sites mistakenly think + is an invalid char.

No overhead of creating addresses, you have an infinite address space for your account to play with freely.

More mail providers should offer such a workflow.

If you send me a name+whatever@gmail.com then I automatically know what your real address is and can strip the +whatever and now you're stuck.

With aliases, I can drop the alias and you can not easily guess my real email address.

Just depends on what you want, or what you use it for.

This is true, but most spam email is automated.

You're still correct though, your solution is more secure (and also what I use). It's always possible the form you submit automatically strips the +whatever from your email silently. I believe facebook does this (at least when checking to see if the email you're using has already been registered).

Not to mention that FM also allows plus addressing as well as a different style name@whatever.domain.com

how would that work ? don't you need to set up the MX records for that particular subdomain?

You set up wildcard MX records, such as for *.yourdomain.tld.

With Fastmail sets this up automatically if you let them handle your DNS, though you can (obviously) disable it.

you could set up a wildcard CNAME

Please don't. Per RFC, an MX RR cannot point to a CNAME.

That's not what I'm suggesting though.

The one downside I've found for this is that many email forms prohibit "+" in an address. They shouldn't as it's a perfectly valid character for an email address, but you know...devs.

I used to use the + system but ran into too many issues of it not being accepted and it seems some are smart enough to strip off the + and everything after.

I moved to wildcard forwarder on my own domain so that *@mydomain.com get forwarded to my main email account. This gives me all the tracking I want and I can ban addresses if needed.

That still gets me the zero overhead and practically infinite addresses with none of the downsides of gmail--and I can easily change my main email account as an added bonus.

I do the same thing. I also have multiple domains. I will create email-forwards to my main email addresses to use mostly for website logins and such. I don't think that level of paranoia is really necessary, but I do it anyway.

For my alias management, I wrote a script to add and delete aliases and reload the MTA. Makes it super easy to do. Long time ago I would have to go through the process of creating a fake hotmail/gmail/yahoo/etc email address if I wanted to do something similar. This way is much nicer.

I do the same thing with Gandi.net . Their email aliases solution let's you achieve the same result, while doing away with the hassle of running your own email stack.

This was my first idea, scaled up a bit. I wanted a unique email address per person - not just for when I sign up. This way if someone harvested my email address through someone else, I could just delete the alias and give that person a new alias.

Way too much maintenance. This solution is easier.

What little tool might that be? I'm looking to do this myself with a domain.

spamgourmet.com is not a bad solution similar to this.

The email aliases will forward a set number of emails, after which they go to /dev/null. You can white-list senders who you want to continue seeing emails from.

I did this with procmail 20 years ago: http://angel.net/~nic/spam-x/procmailrc.txt – and then stopped because lots of spam comes with random people's real e-mail addresses as the From, and my challenge autoreplies were quite rightly being reported as spam by their innocent recipients. Now I just use FastMail and let their excellent Bayesian filtering do its job. (Oh, and I filter all mail with "unsubscribe" in the body into a "Bulk Mail" folder".)

Between then and now, that problem was solved – you send autoreplies to the Return-Path header, which is protected by SPF.

Mmm. I have transactional emails in my boxes that still have unsubscribe in the body.

SMTP-time rejection, with a mechanism for specifying workaround ... doesn't exist, but if it did could address this failure mode of challenge-response.

It kinda does, it's called greylisting and already works today: the recipient basically says "can't right now, please come back later". A spammer will not bother even reading this and will hit other targets. A proper smtp sender that dutifully respects the specs (the legit ones do) will try again up until a few days later, where the recipient can whitelist the sender.

It's all automatic so it kind of doesn't solve OP's issue, but I feel like there should be a proper way to handle the challenge-response part.

Greylisting (e.g., greymilter) has long existed, but lacks a strong and reliable notification / remedy mechanism. It seems reasonably effective and non-noxious.

In France we have a software vendor called MailInBlack.

Their solution is exactly the one proposed in this article, where the sender has to solve a “challenge” to get in your mailbox.

If I think this approach is really effective, it also creates a huge pain for a lot of tools relying on emails such as WebEx invites or when you want to contact someone for sales.

As a result I think that this approach might be better if it was for instance triggered only on emails with an “Unsubscribe” link, or on emails with specific keywords.

This is super-annoying for legitimate, infrequent correspondents. Somebody asks me to email them something, and I need to take the time to notice and read the automated response, click to access an unknown website, and then try to solve the captcha, which sometimes requires multiple attempts. You asked for my recipe, don't hassle me when I'm trying to help you.

When I set it up, I knew it could be annoying, which is why I automatically whitelisted everyone I knew. I was seriously concerned people would rather just not be able to send me emails and will willfully ignore my emails, but as I pointed out, that's the exception rather than the rule.

And I am proactive. If I see an email in the quarantine folder from someone I want to be on the whitelist, I press a button and he's in. This isn't a hard line, binary, black and white situation. I can and do place people in the whitelist.

If I've requested something from someone, I don't expect them to respond to the email, and I usually reply back to them with an apology about the email. So far, no one has complained.

And I don't use captchas. There's no need for them. All they have to do is click on the link and type in their email address.

If you still check the quarantine folder once a week, do you even need the challenge email? At most it will take a few days longer to reply to a valid email and then you can whitelist the sender.

Think of it as "if they respond to the challenge email it'll get in front of my eyes quicker".

Also, keep in mind when I wrote it, I was getting 20-50 spam a day (not counting unsubscribe emails). And part of the motivation was that legitimate emails were simply getting lost (I wouldn't notice them in the sea of noise). Checking once a week would mean scanning hundreds of emails for the few legitimate ones. The challenge-response is more reliable.

(Today I get almost no spam - someone fixed a broken pipe on the Internet).

Are you speaking from experience or theorizing?

My thought is that if i asked you for an email, and I was running a system like this, i wouldn't be a dick and make you jump through that hoop. I'd proactively add your email to the whitelist before you even sent me something.

In practice this is not as easy as you think. More often than not, you don't know the email address they will use. I often tell them up front that they'll get a spam checking email that they are free to ignore. I whitelist once I get their email.

Who is to say that you have their email address? Even if you do, who is to say that the mail you're asking is going to come from that email address?

Exactly! I meet someone and they ask for info. I offer to email them with the info. I get their card and they don’t ask for my email address. I’m speaking from experience.

Even if they have my address, how soon is it whitelisted? Like, they run home and update their whitelist?

In at least one case, I suspect this set up is narcissistic. ‘People have to kow-tow to communicate with me.’

Next time, write your email address on their card and give it back to them. If they really care about the information, they will follow up.

You shouldn't bother writing unsolicited emails with your advice to people who may not care. It's a waste of your valuable time.

>If I think this approach is really effective, it also creates a huge pain for a lot of tools relying on emails such as WebEx invites or when you want to contact someone for sales.

I haven't used WebEx, but what is the problem exactly? Are you concerned you won't see the invite or that the sender will get an annoying automatic response?

Incidentally, this is all just a Python script. If you can construct a reasonable enough pattern for the email address, you can always have custom rules for them - it's fairly trivial. Just like I have a list where emails go straight to quarantine without producing an annoying email.

>or when you want to contact someone for sales.

My original design was to automatically whitelist anyone I send email to - in the end I didn't go that far, but it will likely alleviate this problem. My solution is mostly for my own personal email, though. If you plan to conduct a lot of business where you expect/want random people you don't know to email you, then this scheme won't work well.

>As a result I think that this approach might be better if it was for instance triggered only on emails with an “Unsubscribe” link, or on emails with specific keywords.

I just did a query. Fully one third of the quarantined emails do not have the word "unsubscribe".

> I haven't used WebEx, but what is the problem exactly? Are you concerned you won't see the invite or that the sender will get an annoying automatic response?

Basically, WebEx sends invites from their own email address. If your customer has not white-listed the WebEx domain, they will not receive the WebEx invitation. The only solution we found for our sales team is to "double" the invitation with a manual email sent separately, with the link to the WebEx invitation...

Also, I just realized that I was not very clear in my comment: my company does NOT use MailInBlack :). However, a lot of our customers do, and this has been a nightmare for our WebEx invitations process.

>Basically, WebEx sends invites from their own email address. If your customer has not white-listed the WebEx domain, they will not receive the WebEx invitation.

That's rather obvious. If they want to receive emails from WebEx, they should whitelist it. Keep in mind that the quarantined emails are like emails in any folder. You can still check them to see if there are important emails there.

Now I don't know much about WebEx, but ... are you sending WebEx invites to people who are not expecting an invite? If so, I would say my filter is doing its job! That's exactly the type of email I'm trying to cut down on. If, OTOH, I am expecting an invite from you, I will check the quarantine folder for it. And as I said earlier, if WebEx becomes big enough that I expect many people will use it to contact me, I'll just whitelist and put it in the low priority folder.

Fundamentally, the problem is that we've overloaded emails. Emails (for most people) were a means of communicating between individuals. Then people started using it as a TODO list. Then as an advertising platform. Then as a way to manage receipts. Then as a calendar system. And on it goes. One of my goals is to separate the personal emails from everything else. I may still use it for other things, and set up scripts to handle those other things, but I need a way to separate out personal emails from everything else.

Now if WebEx also starts sending me unimportant emails, they're out of the whitelist. Kind of like LinkedIn. It uses messages-noreply@linkedin.com for all its emails - whether it is to notify me that someone sent me a message or to let me know that "Hey, if you're willing to become a Premium member, everyone will want to hire you." - They are not in my whitelist.

And BTW, there is only one customer: Me :-)

Use the WebEx Productivity Tools extension for Outlook. Whenever I used WebEx with a client I supported, I would open up a new meeting in my calendar, type them a personal message, then click an "Add to WebEx" button and it'd automatically put all the necessary call information at the bottom of the invite. It'd be from the company email and I'd receive replies/notices of acceptance directly.

> The whole thing probably took a few hours to write.

Did you know about TMDA before you wrote it?


I think someone (you?) told me about this, but I couldn't remember the name and it's hard to construct a search query to find it.

Also, almost all their links (Documentation, etc) are not working - I think they're having server issues.

Mailinblack does this for more than ten years now, mostly in France. This technique is called a "challenge-response" filtering. It does work pretty well with other antispam techniques. Their solution send a daily digest to the user (x per day) of quarantined emails, allowing them to liberate legitimate emails of lazy senders who do not respond to the challenge-response email. They also use outgoing emails from users to populate their personal whitelist.

Their site is not shinny but the product does the job. https://www.mailinblack.com/en/

From my experience, it seems that the daily digest is not enough :). And they still seem to blacklist WebEx invitations, which is really weird as it is definitely a "standard" in web-conferences.

This is know as Challenge-Response and ... it doesn’t work. What happens when an automated system sends you a message? How do you handle the backscatter problem (ie when spammers spoof the sender address causing your helpful challenge messages to be sent to someone random who then complains..).

Laudable effort, but it’s not a solution.

Hard to say something is not a solution when it's been working for over a year.

>What happens when an automated system sends you a message?

I'm not sure I see the problem. I don't want automated emails in my Inbox. Period.

For the few ones that you'd think I want (e.g. overdue notices), they are in my "other" list - those that don't go into either my inbox or my quarantine - so they don't get an autoresponse email. But frankly, there are very few automated emails that I want.

As an example, I've set up Zillow alerts that tell me if a house in my neighborhood is on sale, or has been sold. In one sense, I do want these emails, but I simply don't want them in my inbox. They remain quarantined, and I'll see them whenever I check my quarantine pile. Most of the sites I intentionally signed up for because they occasionally will send me something useful - they all still go to my quarantine pile. I don't need them in my inbox intermingling with more important emails.

>How do you handle the backscatter problem (ie when spammers spoof the sender address causing your helpful challenge messages to be sent to someone random who then complains..).

If the spoofed email is in my whitelist, they don't get an email. If not, what exactly is the problem? That some random stranger got an email from me? That's just spam for them - I doubt they'll even notice. At best, they may realize someone is using their email address as a From in a spam email.

At least in the 1+ year of running this, none of these has been problems.

Funnily you've kinda inverted the mailboxes purpose: your real inbox is called "quarantine", and your actual emails are in your "inbox". Sounds like you could use inbox as inbox and then "archive" when it passes validation.

Anyway if it works for you _and_ is simple to put in place and use (as you've shown), then kudos to you!

This is interesting. I was initially tempted to replicate something similar to try it, but then took a moment to think about what my real gripes with email actually are. Then I went to work on solving them, one by one.

First, I have a tendency to keep emails in my inbox as well, thinking I will do something with them at some point. I can't say I've ever waited a year to respond; however, I have probably waited several months before deleting (archiving, since I use Gmail) an email. This often comes after realizing that I am not, despite my earlier assessment, going to do anything about this particular email. Second, the endless notifications are abhorrent.

Today, I finally got around to adding some little used account information into my KeePass database and archived two emails. Then I realized I wasn't going to do anything with the other 20 emails in my inbox, so I archived them, too. Then I turned off notifications on my phone. I turned off Outlook's toaster at work years ago after I was frustrated with the amount of useless junk my colleagues or other work groups keep sending me. Unable to remove myself, I just decided to deal with the junk and all the other emails I get on my time, not anyone else's. I took that practice to my personal accounts, after deciding that if I won't waste my time even when I'm getting paid, I should definitely not waste my time when I'm not.

Now if I could only talk my employer into letting me disable my email account when I go on vacation, a la Daimler: http://time.com/3116424/daimler-vacation-email-out-of-office....

I use my own simple email scripts with simple rules and it also works! Over the time, my system has become even simpler. I used to have Procmail as mda, now simply a 5 line Perl script. I used to have spamassissin as pre-filter, now I skipped it altogether. I used to have a database for whitelists, blacklists, and filter lists. Now I simply have hand-coded rules (several) and hand edited whitelist/inbox lists. There are three areas for my emails: inbox (as well as a dozen categorized ones, important boxes also sends notification to my phone), unsorted (handful every day), and spam box (hundreds every day). I do check my unsorted box about daily; I only grep my spam box if I am expecting something and something didn't come.

Every time I got a spam leaked (into unsorted) and I am annoyed or I want certain mail go to certain inbox, I simply edit my scripts (which I know exactly where and how). It is easier than -- e.g. -- reading procmail's man page.

Specific, simple, personalized approach is surprisingly effective than any generalized approach. The latter is a hard problem, but the former often isn't. The former I can choose my own compromises.

E-mail is currently useless for me. people have been talking about going through and unsubscribing from things.

NOT counting the spam folder i have 52,541 unread emails.

I am not going through all of that to click a bajillion unsubscribes considering i never actually chose to receive email from 99% of the people / companies sending me email.

I'd forgotten about the whitelist strategy. I'd written it off as obnoxious to the sender, but as i am reminded of this years later i'm thinking it's more obnoxious for me to never respond to people, partially because i miss their emails, and partially i don't bother looking most of the time because there's too damn much to deal with.

Mark all old mail read and start over today? Just unsubscribe going forward.

the strategy of continuing to unsubscribe from things is based on the flawed idea that i want to opt-in to an ongoing maintenance task for the rest of my life. I do not. We shouldn't just _accept_ the idea that other people / companies can assign us ongoing unpaid work for the rest of our lives.

>We shouldn't just _accept_ the idea that other people / companies can assign us ongoing unpaid work for the rest of our lives.

This is exactly my perspective as well - I even said in another comment "I don't want to do this for the rest of my life". Reading the comments, it is clear not everyone has assigned the same roles to email as I have (which is understandable). For some people, opt in as a default seems to be acceptable.

I don't know - perhaps because I had email for some years without getting all these emails, I may simply have a different reference point than many who grew up at a time when communicating individually over email was the rare use case (as opposed to being notified of sales, meetings, etc).

(Note: Definitely not complaining or being sarcastic here - to each their own).

I have a different solution, while I unsubscribe from mailers that I know I signed up for, when I have messages that I don't trust to verify my email address with a response (potentially getting more spam) OR I'm feeling lazy (even with services I've signed up for), I just hit the spam button in Gmail. That's a cheap and easy unsubscribe.

I very rarely check my spam box, only when I'm job hunting as that's very critical I don't miss anything.

Okay sure I don't like the status quo, but I was just telling you something that's actionable. If you have a strategy to change the email ecosystem, by all means work to do so. But until then what I told you to do is at least something you can start doing today. It'll also solve your problem more effectively than complaining about it on HN.

It's a public bi-directional communication tool that can be initiated from either side. If you don't want that then don't use email.

My personal mail solution is composed of:

- a good taxonomy on a maildir in my home and on my IMAP (thanks mbsync+cron+a small script to watch inbox with IMAP Idle)

- a good automatic filtering/refiling (imapfilter, for now)

- a good spam blocker (spamassassin)

- a good MUA which support both physical taxonomy and a search-based one

The workflow it's roughly simple: I have some saved search in notmuch-emacs mostly 'unread', 'important', 'inbox', 'live', 'todo'. When I have little or no time I look only at unread+important, when I have time I go through the rest of unread. Live stuff are thing in progress that I have to watch but no action or todo, todo are...todo's...

What I’d love to see is an email-like system where each message is accompanied by a small sum of money (1¢ or so). If the message is not useful to the recipient, they are free to take the 1¢, otherwise, if the message is useful, they wouldn’t (and it will return to the sender either when the email is replied to, or, say, in a week). This would very quickly kill all spam-type business models, and would be unlikely to cost anything for legitimate users.

I’ve made something like that 15 years ago.

It didn’t worked. People receiving the quarantine mails often misunderstood it or dismissed it without reading it.

We tried with CAPTCHA, without, nothing worked.

Our users had to spend a lot of time checking for false positives.

In the end we gave up.

Oh, and if you want to do something like that, add a reasonable delay for your response as some Spam list check for automatic responses and consider them as SPAM and will ironically block you for that ! Been there...

> 2. Spamassassin. This worked for many years, but then a few years ago, the accuracy went down like crazy.

I switched to Rspamd and haven’t looked back. It’s much better.

Bogofilter also works quite well if you have enough training material.

Sure, but bogofilter has no pre-made rules, so it makes a bad first impression, and it’s tricky (as in, I wouldn’t know offhand how to do it) to implement site-wide. Rspamd wins on both these points (in fact, its pre-made rules are subjectively better than those of SpamAssassin), and it still has bayesian learning.

True. I use bogofilter locally just for myself, and I had a set of ~100k ham and spam e-mails for initial training.

For single user mail folder/server and local use, it's very simple to set up, compared to anything server side. You basically use 3 commands after initial training and don't need to worry about anything else.

I don't want to keep an e-mail archive server side (I use pop3), and that would include a spam filter's database, there are not many other filtering options with that requirements.

Anyway, I agree it would not make a good first impression without training data, at all. Though if you train it with what you actually receive, it's quite awesome. I get > 99% accuracy for both false positives/negatives.

It's good to keep spam for training.

This. And it has a nice webui.

I just want to be able to classify email by “sent by a human” and “not sent by a human.” My inbox isn’t a (bad) todo list, it’s a (bad) notification center.

I filter anything with the word "unsubscribe" into a separate folder, then add a few filters to handle the exceptions.

What is generally left in my inbox is stuff directly mailed to me that deserves attention. 90% of the stuff in the unsubscribe folder get deleted without looking beyond the subject line.

Ah, like many people, I'm also one of those who let thousands of messages build up and respond them maybe a year after they were sent.

But my reason for such slow response, I believe, is a little bit different than most. I have social anxiety, and reading email is extremely painful to me. Not replying, but reading. I almost never open my inbox. The last email I sent was six months ago.

Sigh. Anyone with similar issues?

Ahhh, don't I ever. Too scared to have an engage with most online communities, let alone e-mails directed at me. I probably won't read any replies but I wanted to enjoy this brief connection with someone with a similar experience.

So your system automatically sends back messages to the non-whitelisted addresses. Those messages will eventually be classified as spam by big operators such as gmail. How do you prevent that from reducing your "email reputation" and ending up on spam email blacklists?

To be frank, this is a concern to me, and I think I'm already on some spam lists. Recently two folks using Gmail told me my (regular, written by me) email showed up in their Junk folder. I checked others who use Gmail and it showed up fine for them.

I may end up using a separate domain for those emails. I'm pretty sure that's a weak solution.

Hmmmmm, the spam list downside is a non starter for me.

It sounds like you've invented a captcha for email. If that ever took off and became widely used, it would be rather easy for the spammers to write a script to comply with the scheme. Followed of course by the usual escalation to adding an image-based captcha, and then efforts to beat the captcha, and so on.

As an aside, you could do it entirely in email as well by sending a response that requires a specific reply to join the whitelist. That is you could eliminate the webserver and the mildly-questionable-looking URL.

On my own domain where I've set up a mail server, I've never received that much spam. I have the impression that people are more civilized now than in the late 2000s with respect to emails. In my case at least, unsubscribe links work reasonably well.

The risk is some dodgy website (small shop, forum, etc) that has been pwned and which leaks your detail to spammers, not so much the company you give the email to.

Not sure how spam is the problem. Mostly solved by good filters. The problem is the amount of legitimate personal emails that one gets, creating the TODO list problem Paul Graham was talking about in his quote at the top of the blog post here.

This is exactly the problem: Some emails you can’t respond because they require taking actions, so they stay in the inbox to remind you. And then it’s the broken window principle... one cool thing I recently found is that on Trello you can have board specific addresses that create a new card when an email is sent to them. This allows me to move the TODO problem outside the inbox.

Unfortunately, in the Enterprise space, this setup is likely to be a zero sum game, as the challenge email would get dinged 90% of the time for spammy wording like "quarantine" and links to unknown domains (and/or blocked entirely, because inbound filtering != outbound filtering).

Large-scale email in its current form sucks, even with the Office 365 ATP tools or similar equivalents.

I have used Hotmail for many years and I really don't get much spam. Most of the junk that I do get ends up in the junk folder, essentially anything that doesn't come from someone in my address book as far as I can see. I check my junk folder whenever I expect someone new to send an email to me or about once a week.

I have had a hotmail account since almost the beginning. Spam hasn't been a serious problem for me in a long time, though occasionally I get a large amount.

I've always gotten more spam in Hotmail than in Gmail (not a lot more). But I've had more false positives in Gmail than in Hotmail. Always seems like there's a tradeoff.

A Challenging Response to Challenge-Response MAY 19, 2003 BY ED FELTEN


Given that it was written in 2003, and here we are 15 years later with CR working, I don't see much value to the post. The author there is hypothesizing, whereas in reality we have functioning CR systems.

Now I'll grant there is a critical mass beyond which CR won't work. Looking at the comments here, the likelihood of hitting that critical mass is almost zero. Most people don't like the idea. So CR will continue to work.

C-R has been very thinly used.

> No - I check the quarantine folder regularly - usually daily.

So what's the point? The whole exercise is intended to cut down on looking through non-personal./junk email, but you have to look through non-personal/junk email daily anyway.

Google Inbox was a nice way of setting up emails as tasks, but unfortunately Google is killing it, and Gmail doesn't come close to it.

I use Google Tasks as a workaround (link a task to an email conversation) but it's still clunkier than Inbox.

Thankfully I don't use e-mail much anymore. Slowly but surely all of my communication have shifted to better, but unfortunately proprietary, systems. As authentication gets more universal this will continue to happen.

Same here.

I don’t have an email problem anymore. I have a WhatsApp, Slack, Facebook Messenger, SMS, and Skype problem now.

I just wish we'd have OS level contact management and app integration for this, like a shell for messaging.

"Jeff is contacting you" over what? I don't care, I have a message from Jeff, and I can respond to Jeff over any protocol I choose, and the shell will show me a unified history of my IMs, chats, phonecalls, SMSes, Deathmatches, emails, chess moves, teledildonics sessions, whatever with Jeff.

That was my favorite feature about the n900 (predecessor to n9) - sms, skype, hangouts, whatever? They were all accessible in one place in a merged thread. Really enjoyed it.

App spam. It gets annoying working out which app suits the message. A bit “old man shakes fist at sky” but having masses of communication methods increases the noise for me too.

The thing is, I use my inbox for receipts of purchases, confirmations of online accounts and all sorts of other stuff where I am interacting with, or storing information from an automated agent.

>The thing is, I use my inbox for receipts of purchases,

Stuff like Amazon.com emails are whitelisted. They go into the "low priority" folder, not my inbox. For one-off vendors, I simply "save" their emails without adding them to the whitelist. It's easier to understand if you've used notmuch. I simply remove the quarantine tag from them and they're stored in the email database. One keystroke.

I'm not any different from you. If I purchase anything on the Internet and get a confirmation email, I aim to keep it.

I still check the quarantine folder from time to time.

>confirmations of online accounts

I'm not sure I see the problem. I still can see the quarantined emails. If I'm expecting such an email, I check it within a few minutes. I click on the link to confirm my email address - I merely do not add the sender to the whitelist.

>all sorts of other stuff where I am interacting with, or storing information from an automated agent.

An example would help. Some use cases from me:

- Calendar reminder emails - Whitelisted: They go to the low priority folder (as counterintuitive as the phrase sounds). They do not go into my inbox.

- Library overdue emails - same as above

- Emails from my medical provider (e.g. "new test results") - same as above

If you get automated emails for which you have automated actions, those can go right into the Python script and handled before they are quarantined.

> I'm not sure I see the problem. I still can see the quarantined emails.

I completely misunderstood that the quarantine is just a folder next to the inbox. It's the obvious solution, from both a technical and usability point of view, and takes care of all my points.

Thanks for clearing it up.

I think I'm going to add this approach to https://forwardemail.net when I implement the DNSBL.

Congratulations. He just re-invented SpamArrest.

Yep – this is exactly how SpamArrest works.

Only problem this wont work for any automated e-mail. But you could white-list countries where sending spam is illegal.

> "In a spammer’s economic model, spending even five or 10 seconds per message could be prohibitively expensive"

So, how about a proof of work system like blockchains? Like, if you've never sent me email before, my server returns some work to complete. If I've ever sent you an email, or not marked you mail as spam etc, your MTA won't be challenged etc.

21 years ago... http://www.hashcash.org/

Ahh thank you. I'm not a bitcoin fanatic but it does strike me as a workable idea.

This also only works until it’s a popular method and spammers automate a solution.

Any mutt users tried notmuch? Is it any good? Only for emacs users? :)

What tool do you use for this?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact