

Postmark introduces easy inbound email processing for all accounts - alexknowshtml
http://blog.postmarkapp.com/post/15687406657/introducing-postmark-inbound-easily-parse-replies-other

======
jazzychad
Some feedback: This is a nice addition to a mail-provider API service, but the
way it works is way too complicated. All email I want to parse needs to be
forwarded to my special postmark inbound email address? This might be nice for
people that have their email setup through google apps (which kindly forwards
on all headers, etc when an email is auto-forwarded with a filter), but for
other mail systems they might not play so nice with forwarding headers, etc (I
know this from running my own email servers for Notifo and dealing with
incoming and outgoing email on my own).

With this solution I still have to setup an intermediate mail server solution
to first receive the emails from users, and then forward it along. Can't you
take that step away from me?

Other providers (mailgun, sendgrid, etc) let me do that and let users send
emails directly to my inbound mailboxes (which are then POSTed to my
endpoint).

Also, this excerpt from the documentation lowers my confidence that you know
what you're doing:

"Rather than leaving this information in plaintext, we recommend using SHA1 or
MD5 to encrypt the information after the + sign and include the encrypted
value after the +. This will avoid unintentionally exposing any sensitive
information."

~~~
alexknowshtml
Thanks for the feedback, Chad.

Regarding forwarding: that step is already being taken away - it's in beta for
a number of our customers, and we'll be adjusting workflow then releasing to
everyone.

There's lots of people who don't have access to their DNS records but still
want a tool like this. We want this utility to be simple for more them, too!

As far as SHA1/MD5 encryption - we know better than to use weak encryption on
any TRULY sensitive information. In the context of the walkthrough, we're
talking about internal IDs and other things that you might want to be obvious
to your customers, not things that pose a security risk to you or them. If you
even thought about using something like that as part of a unique identifier in
your email replies - I think there would be bigger problems at hand than
choosing SHA1 or MD5 to obfuscate that information. :)

~~~
jazzychad
Thank you for the reply!

Regarding the md5/sha1 stuff: The documentation as it stands is both wrong and
misleading. MD5 and SHA1 are _hash_ functions (not _encryption_ functions), so
while trying to suggest that people could encrypt information to put after the
+ sign is a good idea, you will utterly confuse newbies by giving examples of
hash functions. Of course you could use a unique hash value after the +, but
then you need a reverse hash lookup table in your app to correlate the hash
and the internal app state/data you need (but that is a whole other beast and
not nearly as simple as "just encrypt it in the to address, decrypt on
receipt, et voilà!").

In the end it is confusing for beginners and outright wrong for people that
know the difference.

Now for the constructive part of the criticism: Please change or improve that
portion of the docs, since docs are arguably the second most valuable asset to
an API-provider (right behind the service itself).

------
buro9
Wouldn't it be great if there was a standard JSON representation of an email,
and that multiple providers (SendGrid, AWS SES, MailJet, ElasticEmail, etc)
all then provided such an interface.

So not only could you choose (swap out very very quickly) your outbound
provider, but also your inbound provider.

After the thread the other day on outbound email, I ended up choosing MailJet
who have been wonderful with the questions I had when I hit their new member
ceiling within the first few hours.

The things that convinced me on MailJet were real-time outbound stats and
meaningful data (recipient + subject), API of status notifications (who
bounced), and the fact that they do campaign emails at a cost so low that
MailChimp ruled themselves out (one of my email lists has over 23,000
members).

I need inbound handling (currently using /etc/aliases and piping), but would
love all the reporting and info which centralisation from an external provider
could deliver.

My only concern would be resilience. Email is really good at re-delivery when
your end is down or something prevents delivery, would a HTTP REST interface
offer the same level of re-try and timeouts to ensure it got through
eventually. Would the provider queue for me?

~~~
old-gregg
Everything you're asking for (and much more) is handled by Mailgun (full
disclaimer - I work there):

<http://blog.mailgun.net/day/2011/11/07>

Moreover, our incoming mail parsing doesn't stop at simple MIME parsing, we're
dig deeper into the meaning of those characters and extract user signatures,
quoted parts, etc.

Moreover, we retry inbound POST, allow you to filter incoming traffic on the
server and (very important) we're the only incoming mail API processor that
generates proper bounces for 3rd party servers (spammers) trying to hammer you
with non-existent addresses.

~~~
cypriss
We use Mailgun at UserVoice.com for incoming mail.

Of particular help to us:

\- All emails are converted to UTF-8

\- There's always a 'plain' (text only) version of the email sent to us, even
if the original email only included HTML.

\- They offer a 'stripped-text' field, which lets us easily see the new part
of an email thread.

\- Pretty great support

------
brador
$1.50/1000 seems steep and could hurt should a spam attack occur...Someone
needs to package charged outgoing, with free incoming. Kinda like the cell
messaging systems they have in the UK...

Remember, you can add value by combining valuable services. Sell one as a loss
leader to bring in the users and the other is your revenue stream.

~~~
citricsquid
You could always use Postmarks spam checker service to monitor emails and if
you get say 2 or 3 from the same email address that flag up as spam block the
email?

<http://spamcheck.postmarkapp.com/>

~~~
alexknowshtml
We actually include spamscore information in the parsed emails.

[http://developer.postmarkapp.com/developer-inbound-
parse.htm...](http://developer.postmarkapp.com/developer-inbound-
parse.html#spam)

Also, if you use Gmail forwarding to foward emails into Postmark, you get the
benefits of their spam filtering before it even hits our system.

That said - we watch for spam-like activity across our entire system.

~~~
peterwwillis
Assuming I could craft messages which did not look like spam (to your system),
and I could do it really really fast, do you have any throttling options to
allow your customer to prevent an excess of mail from adding up too quickly or
an alert based on a threshold? (Maybe i'm overthinking it, but I like services
that put me in control of how much money i'm spending instead of being at the
mercy of botnets)

~~~
alexknowshtml
"Unusual" send activity is pretty easy to spot, especially if it's going to
put our customer at risk in any way. We're extremely proactive about making
sure that you're aware of anything that might be considered a "surprise" to
your service or your wallet.

------
amix
Lamson makes email processing easy - it is a very modern framework for writing
email applications. I have used it for a while now (almost a year) and have
not had huge problems. I have written about it here (including code samples):
<http://amix.dk/blog/post/19608>

~~~
old-gregg
Lamson is a great project but it has a number of issues with handling
international emails, try handling mails that have subjects in two different
languages, or have To/Cc fields with non-ASCII names in them. Basically if
you're using it you'll have to guard it with some additional code.

------
kogir
I read though the docs and couldn't tell: is delivery reliable? If my app is
down and I miss some POSTs, will I be able to tell? Will they be re-delivered?

I'm currently using Exchange to get real-time mail notifications and nice
parsing, but am looking for other options.

Also, encoding attachements as base64 is an interesting choice. I'd prefer a
URL where they can be downloaded instead.

~~~
alexknowshtml
Your activity feed will show that there was an HTTP error.

We're working on the workflow for retries, it's coming soon.

Thanks for the feedback about the download URLs. Any particular reason for
this preference that I can share with the team?

~~~
kogir
Photos and other large content. Most web stacks don't nicely handle large
requests like that, and to the best of my knowledge streaming JSON parsers are
both young and rare.

~~~
alexknowshtml
Totally fair - thanks!

------
siavosh
This is awesome, will play with it when I get a chance.

As an aside, I've begun to think that email is/should become a platform for
app development. Some sites are already using incoming email for active user
engagement (posterous, idonethis, etc)

I think it's a great avenue to fight app/user fatigue. Most people constantly
check their email no matter what, but few are making it past the app download
to regular usage. The assumptions of emails limitations should be rethought.

------
bryanh
Parsing email is a real pain point, kudos to Postmark for trying to solve it.
However, I _need_ this functionality in a more abstracted way, not locked to a
email address. I'm thinking about taking a swing at it myself and GPL'ing it.
(Basically full fledged IMAP->JSON.)

For a real world example, at my startup <https://zapier.com/> we have "new
email received" or "new email tagged in Gmail" as one of the many, many inputs
which you can map to any other write (IE: Tweet something, add a note to
Basecamp, add a contact to Highrise, etc...). However, this means I need to
log into a IMAP account, not receive an email at a predetermined address.
Parsing that raw email is an absolute bear (even with Python's built in IMAP
and parse libraries).

Anyone heard of a service or library like that?

~~~
delano
<http://lamsonproject.org/>

~~~
bryanh
Looks interested, without having the chance to give the docs a one over, does
it do IMAP as well? Looks more focused on SMTP/sendmail.

~~~
dougzor
At it's core Lamson is just a really sweet wrapper around smtpd
(<http://docs.python.org/library/smtpd.html>) using asyncore for asynchronous
socket IO.

When a mail is received you can optionally process it and when done processing
you can drop it on the floor (e.g. you're done with it) or you can send it to
a 'relay' which means a IMAP or POP server (or really anything) if you so
choose.

I would generally say that yes, lamson is much more focused on receiving email
via SMTP and doing [smart] processing on it.

~~~
bryanh
I would imagine that its parsing functionality could be a boon to someone
wanting to do something higher level with IMAP. Thanks for the response.

------
alexknowshtml
We've got more in store for this feature set, but we're really excited about
this as a start - we use it ourselves at beanstalkapp.com and I know that a
number of people HN were using the beta.

I'm happy to answer any questions, or field any ideas you might have.

~~~
anderspetersson
Hey, finally it launched, will definitely come in handy.

I found a bug in the docs: On [http://developer.postmarkapp.com/developer-
inbound-parse.htm...](http://developer.postmarkapp.com/developer-inbound-
parse.html) the link named 'API developer email list' links to
'file://localhost/Users/alexhillman/Sites/wildbit/postmark-
design/docs/groups.google.com/group/postmark-api-developers/'

~~~
alexknowshtml
Good thing I don't keep docs in my pr0n folder. That would've been
embarrassing.

------
siculars
Fantastic! I'm very excited to try this out. I've been using postmark to
quickly build out my MVP for a new site I'm working on. This is just the other
side of the equation and I am very eager to start playing with it. Integrating
with their nodejs library was literally as simple as putting the library in
the right place, adding the api key and calling the function. Could not be
simpler.

------
boolean
As much as I like the simplicity of Postmark, their pricing makes it very hard
for me to justify using their service. Sendgrid and Amazon SES are charging
$0.10 per thousand. Postgrid is 15 times more expensive.

~~~
dazbradbury
Also, MailGun is $1 per thousand. Which seems to offer a similar proposition.

~~~
alexknowshtml
I'll point out that we have no "account minimums", only the $1.50/1000 email
credits for both inbound and outbound that never expire.

Also, we're home to many happy customers that are sending very high volume -
we offer them bulk-credit purchase rates as low as $0.50 per 1000 when buying
2MM credits at a time or more.

------
kallena
This is great! Perfect timing too. I've been using postmark for over a year,
and have recently been looking to other providers for an inbound service....
soooo glad they did this!

------
kirubakaran
Try EmailYak. It is simple and elegant <http://www.emailyak.com/> It is my
friend's but I'd still recommend it otherwise. See:
<http://docs.emailyak.com/quick-start-guide.html>

~~~
ceejayoz
It's also quite pricey - same price as Postmark at its __cheapest __, and
$5/thousand at the first pricing tier.

------
paisible
There are a subset of applications that imo could rely exclusively on email as
the platform.

A few months ago I built such an app, a small side-project called
betabrokers(email trade@betabrokers.com with subject:about to signup and learn
more). The simplicity of the signup process for this type of app (simply
sending an email) is a huge benefit.

Email-based apps especially make sense in a mobile context, which is somewhat
ironic because the transactional nature of applications built on them would
suggest that they are slower, therefore less likely to be used. In our case
however, users keep coming back to it, probably because the email client is
the single most used app on their phone, and receiving replies to your actions
via email becomes addictive.

------
jeremybell
We've been using Postmark for quite a while now (maybe 2 years now?) and it's
fantastic. We recently began using the inbound processing while it was in
beta, and it too is fantastic. The API is great and it's absolutely painless
to setup and begin using quickly...

~~~
alexknowshtml
Thanks for the kudos, Jeremy. If you can share where you're using the inbound
API, I'd love to see it in action!

------
cullenking
This is neat, but it's not too difficult to handle incoming mail yourself with
postfix and a simple ruby/python script to process it. The learning curve is
slightly steep due to postfix and some _really_ bad suggestions, but the
actual work is straight forward.

I setup a simple ruby program to pipe the postfix email into a resque queue
(redis backed ruby jobs system) to be handled by my work pool...a couple easy
regexes determine what the email is: bounces get auto-delisted after 2x,
unsubscribes delist immediately, replies get shoved into the correct in-site
comment thread or conversation, etc. This is a day of work....

~~~
orenmazor
You are correct, of course. Nothing stops you from doing this yourself. The
basic functionality in SaaS is always easy to accomplish (hosting, email
sending, dropbox, etc). Its the little things that get you and eventually
drive you to the bottle.

------
brianbreslin
Interestingly we just had to build this type of processing into one of our
applications, and I would have much rather sent this through a simple API.

Alex are there high volume discounts?

~~~
alexknowshtml
Yup, the same discounts apply for volume credit purchases.

Our credits work for inbound and outbound and never expire.

* 500,000+ - $1/thousand ($500) * 1MM+ - $0.75/thousand ($750) * 2MM+ - $0.50/thousand ($1,000)

Sending even more? Email me - alex@wildbit.com and we can talk.

------
citricsquid
I've been using this amazing service <http://mailnuggets.com> for a while now
(6 months or so) and the guy is absolutely fantastic (I emailed some ideas and
he implemented them :D) buuuut I use Postmark for outbound email, it would be
nice to consolidate everything into one account. Will give this a try. I love
postmark :-D

Google apps + configured filters + inbound email processing = awesome
possibilities.

------
epicviking
This got me thinking... Are there any startups that are doing something
similar with physical mail? I can picture a large centralized mail room with a
bunch of industrial sized scanners, mail gets opened, scanned, and uploaded.
Customers could access pdfs of their daily mail through a Web interface and
could search through archives. Through in some super OCR and I think you'd
really have something...

~~~
orenmazor
You know, when I started working on Postmark I thought that would make a
fantastic April Fools joke.

then I discovered there really is a company that tries to do that. I can't
remember the name though…

~~~
bengl
Earth Class Mail?

<http://www.earthclassmail.com/>

------
bthomas
I'd love an email API that you could forward an email thread to, which would
be parsed into a set of linked emails with contacts attached. Email to JSON is
great, but not when the body is the text of 20 forwarded emails. Seems like a
gimme for a CRM app -- does anybody provide this?

~~~
alexknowshtml
This is a cool idea, we'll keep it in mind!

------
petenixey
How well does this deal with the cruft of a "Replied to" email

Are there particular requirements/restrictions for emails to be robustly
parsed? Even Facebook had problems early on with their email-response parsing.

------
pbreit
I can see how this would be helpful but it seems like the best way to do this
would be with a POP/IMAP client in your code. But I'm guessing those libraries
are probably not as user-friendly as one would hope?

------
necro
We use google for our domain email. In order to handle uploads via email on
this same domain we simply create a new account and send emails to this
account. You then can have a hook (imap push) in google imap to detect when an
email is received per account/box and process is right away. Or you can just
run a cron every minute the check if new emails have been received. We then
fire a script that pulls the email via imap, and then processes the
emails/images into json format for use.

This allows us to keep everything on google and we do not need move our MX
somewhere else. Also we keep our "upload" email on the same domain.

Pretty simple solution with no additional costs.

