That's kind of a biggie. What happens when someone sends an attachment? Does it bounce? Are they warned? Do I get a notification? Is it silently dropped?
From reading the code it seems like it just doesn't include the attachment and then deletes it from S3?
Yeah, this is an issue I'm planning to work on. The reason I chose to release it before supporting attachments is that for my use case for receiving marketing emails (this project is primarily for my personal use, rather than as public software), there are almost never attachments.
Yeah and seems like the big providers don't like self hosted emails. I know someone who runs a hosting company using cPanel and his clients email go to spam a bunch even though not spammy emails, his answer is to just pay extra for Google Apps and that running a email server is too complicated. Not sure how true that is... For my own future projects I plan to run everything in containers, Node, etc so not PHP/MySQL so still need to host the email somewhere, kinda hate the idea of paying a third party and wonder how they would handle shared inboxes(like piping email to a script like you can with a cPanel server). Probably other email solutions though to look into though but haven't looked into it too much yet but kinda hate how we have to rely on large providers it seems for email.
Then sounds like people running WordPress or support help desk scripts with contact forms forwarding to their Gmail, etc is a problem too... Those services think your server is the source of spam.
But maybe it's worth paying for a third party email server for your staff and also a service with APIs for programmable sending/receiving since you don't want emails to customers like password resets, receipts, etc going to spam. I noticed even when I mark someones emails as not spam they keep going to spam, I don't check my spam daily but sometimes they have to send me a IM to let me know they emailed them...
I feel like might be forced paying though instead of self handling email. Kinda feels like giving the mafia some extortion money for protection though in away. Plus email is a bit broken in the first place, open and all is great but opens itself up to abuse to spammers and scammers.
Hosting company cPanel servers are a bit of a special case though. I've run these before and once customers start installing WordPress it can be a matter of hours before a bad plugin gets turned into a spam bot. It's at a point where blocking port 25 outbound is a responsibility imo.
WordPress contact forms are routinely abused - I regularly see forms where enter a victim as "my address" and then it helpfully copies the message to the spam recipient. There's always a web designer who wants it this way for UX reasons, even when I shown how it's abused.
Yep, contact forms in WordPress but even non WordPress sites too. Then of course if you got a plugin to detect brute forces, seem like WordPress sites get ton of them! Probably because you can detect if a site runs WordPress or not, so bots use that.
Sounds like the only solution to contact form spam might be rate limiting and/or captchas but even bots can bypass captchas too unless you use one like Google reCaptcha maybe but sucks your system has to rely on third party services then.
Oh wow, I was under the assumption that Google Apps(Now called G-Suite but I always remember it as the original) kinda gave you a guarantee if sending to other Gmail people and probably other big providers like Microsoft, Yahoo, etc would trust Google more too.
I'm far from shipping from what I'm working on though, but before going live want to get email setup for the main company website and product website for me and future staff, then want to be able to send email's for notifications, password resets, etc and then also have scripts receive and parse emails for replies to put into a database for the support portal part of the product domain but getting ahead of myself. Not sure if the product site would have users to people or just be all scripted sending/receiving while the main site is mostly emails hosted for humans.
I think one strategy would just run a mail server and have a script download the emails to the database and delete them, since doubt many solutions support a more of a webhoook type setup.
This is a really cool project, so I don't mean to be overly negative, but personally this workflow feels quite a bit more laborious than just having a catch-all email address. Before signing up for a service, I need to email myself to get an address to use for the service?
I have all emails for my domain route to me, so when I use a service I just do [service-name]@my-domain.com. If a bad actor gets a hold of it I set up an inbox filter or black hole the email address at the service level. The big advantage of this project seems to be that you can reply, but I've found that a huge proportion of these email aliases are inbound only for me.
Totally agreed, I do not want to manage my own workflow in a manner like this. I actually just put a doc together[1] last month for anyone looking to do the same through providers that support this workflow.
The one killer feature that is missing in Fastmail right now is the ability to reply to email as the same alias it was received as. I.e. if Airbnb support emails me at airbnb@mydomain.com, it'll default replay as me@mydomain.com unless I go manually setup the airbnb alias.
I remember thinking the same thing last time I looked into Fastmail, but earlier when I was checking before posting, it appears that's gotten much, much better. From the Fastmail docs[1]:
> Creating an [wildcard] alias also creates a matching sending identity. This means you can also send mail using this wildcard alias. When sending from a wildcard alias, you'll be able to manually type the From address (to sales@mydomain.com or accounts@mydomain.com, etc) when writing a message.
So, yes, it appears you still have to manually set the from address, but at least you don't have to actually go do anything outside of the composition interface anymore!
I am using Exchange Online, and I cannot reply using aliases that I've created. It also always defaults to the primary username. Anyone figure out a way to make this work in Exchange Online?
You're going to get an astronomical amount of junk mail with a catchall email address. Google is great at filtering spam, but not perfect. If you're running your own mail server, you're going to have a hard time dealing with it, even if you use SpamAssassin and other tools. It's also going to get worse over time, because every address that accepts delivery is going to get added to a database for future spamming.
I use Postfix and ViMbAdmin to manage my whitelist via a simple web UI, and I don't find it to be onerous. I don't sign up for new services every day, and it takes about ten seconds to add or delete a service-specific alias.
I disagree with the "astronomical amount of junk mail" statement. I've been using FastMail with a catchall email address for years, and get very little spam; most of which is correctly classified as such (I did have to 'train' FastMail for a while with custom spam/no spam folders though).
Yeah, I've also been using this approach for years now and see significantly less spam on this personal, catch-all account than I do on my work email (both are Gsuite). I will say I was honestly surprised at just how little email I see coming into random email addresses via the catch-all; over years I'd say (anecdotally) that number basically rounds down to 0 versus my "legitimate" email.
Most of the time I do see an influx of spam it's because of one of the aliases clearly having made its way on a list, so I'd say the system is working as intended.
No worries, thanks for the feedback (I appreciate all feedback whether positive or negative).
A catch-all email address is actually less complex to build and test, but there are a couple of reasons I chose to use this approach:
1. It seems that in the workflow you described, there is some work to be done in setting up an inbox filter/black-holing the email address too. In a way, my workflow simply shifts this work to the start?
2. I don't think it's laborious because I don't really need new email addresses that often. It's also really easy to generate a new email address imo - no logins or portals required, just email generate@mydomain.com and you get a reply with an email address within seconds.
3. You're welcome to fork it and tweak it for personal use too if you want! :) I'd love that.
1. I don't really think your workflow shifts any work over the catch-all workflow, it simply adds another step. In my workflow I essentially already have that alias you create in your first step. Then, if that alias gets abused, both systems require shutting off that alias.
2. Yeah, that's definitely just a different usage pattern. I'm one of those folks that enjoys testing out different onboarding workflows, etc, so I'm constantly signing up for services, etc.
3. I'm quite happy with the alias approach, but best of luck with the project :)
Email aliases ([user-name]+[service-name]@my-domain.com) isn't the best solution when spammers can remove the alias part (+[service-name]) and you can't know who leaked it.
At first I was confused and trying to figure out what AWS services Heimdall uses to work, and this was the section that explained it:
Infrastructure
I’m using AWS’s Simple Email Service (SES) to send and receive emails, S3 for storage, and Lambda functions for serverless computing. Here’s how it works:
All received emails trigger SES to store the email as a file in a S3 bucket, which triggers a Lambda function. Depending on the email, one of several things could happen:
1. The email gets forwarded to your personal email address
2. The email gets forwarded to the original sender (when you reply)
3. A command is invoked by you (e.g. to generate a new alias)
4. Nothing happens (when someone emails an invalid/disabled alias)
I chose to use AWS for practical reasons: I’m totally new to cloud computing, and AWS being the most popular cloud computing service means it is easier to find guides and resources online.
> I’m using AWS’s Simple Email Service (SES) to send and receive emails, S3 for storage, and Lambda functions for serverless computing. Here’s how it works:
> All received emails trigger SES to store the email as a file in a S3 bucket, which triggers a Lambda function. Depending on the email, one of several things could happen:
> 1. The email gets forwarded to your personal email address
> 2. The email gets forwarded to the original sender (when you reply)
> 3. A command is invoked by you (e.g. to generate a new alias)
> 4. Nothing happens (when someone emails an invalid/disabled alias)
> I chose to use AWS for practical reasons: I’m totally new to cloud computing, and AWS being the most popular cloud computing service means it is easier to find guides and resources online.
Thanks for linking! I would have linked directly to the blog post but I believe it's against the rules of Show HN, so I chose to include a link in the readme instead!
It integrates into Apache as a CGI program serving up a web UI for managing your aliases.
It works by managing the content of an alias file read by your mail server.
Authentication of the webUI is via IMAP or SASL.
Each throw-away alias is associated with a memo in which you can have text and URL's (that get rendered into links), and a creation time. You can regex search through the aliases, edit the memo fields, rearrange their order and delete them.
Not sure how this is relevant to OP’s post aside as an attempted hijack. You don’t even address the differences between the product you’re pushing and OP’s.
If you want to Show HN something, make your own post; this is just rude.
I like the idea but I do not understand why someone would want something self-managed/hosted and then use AMZN SES to send/receive mail and S3 to save mail/attachments.
1. AMZN has access to your mail (inc. your contacts) so you could just use any other service you do not trust.
2. Why would I process mail just so save it on another machine and not do both on the same server?
Probably it has to do with "serverless" (you have to use at least 2 "servers" now, don't you?) but maybe I am just missing the point.
The trust reason is theoretical - in practice, I would trust most decently large services especially for unimportant marketing emails (main use case).
My primary motivation in doing this was to learn to use AWS and Serverless framework and also because I really enjoy working on pet projects :)
Could you explain why email isn't trusted? It's encrypted (vs SMS) so I'd imagine it's a far more secure way of communicating sensitive information (e.g. bank statements or one time passwords).
> With Heimdall, you completely own and manage your data and the service. No feature limitations or having to trust a third-party company with your data.
> Pre-requisites: You need to own a domain and have an AWS account. For reasonable use cases, you should not exceed AWS's free tier (which is very generous).
Unless you physically own a server you have to put your data somewhere, whether that somewhere is Digital Ocean, some shared hosting provider, AWS, or whatever else. You're still more in control of your data than you would be using a closed third-party product.
Sure it would be nice to have other options in addition to AWS, but I don't think those two statements are contradictory.
Also, I don't know if it is, but the data stored on AWS could be encrypted by the app, in which case you're really not trusting AWS.
My charitable reading of GP's comment is as a reaction to the fact that the project's been designed in such a way (Serverless :tm:) that AWS is required; a pre-requisite, not an example.
From the first quote you might think that it was the deployer's choice where to put it, including on one's own hardware.
I don't know, however, what you'd do instead of SES.
> Unless you physically own a server you have to put your data somewhere, whether that somewhere is Digital Ocean, some shared hosting provider, AWS, or whatever else. You're still more in control of your data than you would be using a closed third-party product.
The claim was not "you are more in control than with a closed third-party product". The claim was "No [...] having to trust a third-party company with your data.". When you have to use AWS, then you evidently have to trust a third-party company with your data, unless you happen to be AWS. And not only do you have to trust a third party, you even have to trust one particular third party with no alternative if they misbehave somehow. That's pretty close to using a closed third-party product, if you ask me. I mean, really, you are using a closed third-party product--it just happens to be the infrastructure that you build on.
> Sure it would be nice to have other options in addition to AWS, but I don't think those two statements are contradictory.
So, AWS is either not a third party or could not access your data, no matter how much they wanted to? Or what other alternative do you see to make those statements not contradictory?
> Also, I don't know if it is, but the data stored on AWS could be encrypted by the app, in which case you're really not trusting AWS.
Wut? Am I just completely misunderstanding what this does? This uses SES, a service by AWS that handles your emails, right? As in: That speaks SMTP for you, and thus sees the plain text of the emails, right? And then, somewhere there is code that handles those emails that runs on machines that AWS has physical access to, right? As in: Code that AWS can trace and modify however they like, right? As in: Code where AWS trivially could extract any possible encryption keys from, right?
Unless I am completely misunderstanding this ... what would possibly stop AWS from reading all your emails if they wanted to?
No, you're not misunderstanding. I guess I made that statement with the implicit trust of AWS so I didn't think to qualify it.
You're absolutely right that if AWS is a bad actor it has access to all the information, but I'm working on the assumption that it's more profitable to AWS to be a good actor than bad.
> No, you're not misunderstanding. I guess I made that statement with the implicit trust of AWS so I didn't think to qualify it.
But that doesn't change that you are trusting a third party?! I mean, if that were to count as "you don't have to trust a third party", then anything does. Use gmail, so you don't have to trust a third party (except for the implicitly trusted Google)! Use Facebook, so you don't have to trust a third party (except for the implicitly trusted Facebook)!
There is nothing necessarily wrong with trusting any one of those. But then you don't get to claim "no trust in third parties required!"
> You're absolutely right that if AWS is a bad actor it has access to all the information, but I'm working on the assumption that it's more profitable to AWS to be a good actor than bad.
Well, for one, see above. But also: is it? Is it really more profitable to keep your data safe than to give the NSA access and in return get some of the good government contracts, say? Plus, trust isn't just about them not screwing you over intentionally, it's also about incompetence.
I'm working on the assumption that they have a contract with the CIA for hundreds of millions, and likely wouldn't get caught skirting the rules if it suits them - or can buy their way out if they did.
> Currently, attachments are not supported.
That's kind of a biggie. What happens when someone sends an attachment? Does it bounce? Are they warned? Do I get a notification? Is it silently dropped?
From reading the code it seems like it just doesn't include the attachment and then deletes it from S3?