
Free email address validation API for web forms - old-gregg
http://blog.mailgun.com/post/free-email-validation-api-for-web-forms/
======
Zikes
If it doesn't actually send a validation email to the address, I don't see the
value over a simple m/.+\@.+\\..+/

They say up front that email validation is hard, and yes there are tons of
edge cases and obscure tricks and rules and probably there's no guarantee that
even they managed to get it right with this service, but ultimately the
customer either puts in the correct address or they don't. If they're going to
make a typo then it's far more likely that it would be a legal typo, and if
they're going to intentionally enter a false address then it's likely it'd be
a simple asdf@asdf.com.

Edit: This was a bit of a knee-jerk reaction to what I at first saw as a
redundant overcomplication, however as russjones points out below it has
already proven its value in reducing bounce rates by a significant percentage.

So, it might not fit my own limited use cases, but it certainly can't be ruled
out entirely. Best of luck to the Mailgun team and I hope people smarter than
I am can put this service to good use.

~~~
russjones
Hi Zikes, I'm the developer of this service at Mailgun. Nice to meet you.

We've been using this service at Mailgun during testing and we've reduced
bounce rates by 5%. That might not seem like a signficant number, and it may
not be for a personal blog, but it can be significant number for a larger
ecommerce website.

For someone like us, a service that sends billions of emails, it's huge and we
wanted to provide additional value for our customer to help them reduce their
bounce rates and improve conversion.

Plus we don't correct typos on local-parts, just domains. So we won't correct
Jooohn@gmail.com, but would suggest a correction for john@gmaill.com.

~~~
airnomad
Could you confirm you won't store or otherwise log email addresses submited
for check for any time longer then neccessary?

~~~
russjones
Hi airnomad, I'm the developer of the validation service here at Mailgun.

I can confirm that we do not store email addresses. The parser runs completely
in memory and no address is persisted after the request is complete.

~~~
lucb1e
So tomorrow a national security letter shows up. Whaddyado?

~~~
mappu
If national security letters are part of your threat model, nothing they can
say can make the service acceptable to you.

You should also stop using EC2 or linode, stop injecting the google analytics
JS into your pages, stop taking payments via Stripe, and so on.

------
pudo
This may be very German of me, but the privacy implications of sending an
email address to a third-party service before form submission appear murky to
me.

Also, we need to find a name for "give me some personal data in return for a
minimal value add service" offerings.

~~~
baudehlo
Would you be interested in something open source that does this? I would be
willing to release
[https://www.emailitin.com/email_validator](https://www.emailitin.com/email_validator)
to github. It does all the same checks as this except for the "valid user part
for Yahoo.com" stuff.

~~~
jontas
I would definitely appreciate an open source option, though to be honest I am
more interested in the correction feature than I am in validation.

I have been using the same regex[1] for years it gets the job done adequately
--at least, I've never received any complaints from users or clients.

I've used the regex on some relatively high profile sites--the kind where if
someone was unable to signup with a valid email address we would've definitely
heard complaints.

1\. [http://www.regular-expressions.info/email.html](http://www.regular-
expressions.info/email.html)

~~~
baudehlo
It does the correction feature too. Try it :)

------
peterwwillis
I'm going to come up with a free password validation API for web forms. Just
call my API with your username and password and the service it's used for and
i'll return 200 OK status if it's a secure password.

~~~
jasonlotito
You joke. Heck, I was going to build it as an April 1 hackathon project.

Someone actually built it. I forget the name, but yeah. User accounts as a
service.

------
lucb1e
Unfortunately this is broken just like all other attempts I've ever seen.

If this is true: [http://i.snag.gy/RSwiG.jpg](http://i.snag.gy/RSwiG.jpg)

Then explain why the e-mail is arriving:
[http://i.snag.gy/gWPn5.jpg](http://i.snag.gy/gWPn5.jpg)

Sure this is an extreme edge-case, but this was my second test. Who knows what
else it rejects. Actually, why do we validate email addresses anyway? Whynot
just try and send that validation email that you're going to send anyway? And
on top of that, why would I ever trust a _free_ third party service to check
all my user's addresses?

~~~
alexk
Hi lucb1e, thanks for finding this issues. It actually gets validated
correctly by our parser (which will soon be open source) but hits a bug in our
API. We are working on a fix right now.

~~~
dsmithn
@gmailcom is invalid but doesn't suggest @gmail.com, might be another
improvement.

~~~
alexk
Thanks!

------
alexk
For the folks with privacy concerns: we are actually planning on open sourcing
the entire service as well as our MIME handling library. So if you have
privacy concerns, you'll be able to run it locally.

~~~
dougk16
Great to hear! Maybe it was mentioned and I missed it, but what language(s)
will it be implemented in? And is there a way I can be notified when it's
released?

~~~
alexk
It's all python, you can follow us on twitter here: @mail_gun

------
dudus
Guys after weeks of works I think I found the best way to validate emails.
It's the regex below.

/@/

But most of the times it's just overkill and you shouldn't care.

It's free and you don't need mailgun or anything else.

You are welcome

~~~
aytekin
There is great wisdom in your joke.

We have a web form builder service. (Jotform) We serve 3 million forms and
process hundreds of thousands of submissions daily. Every form pretty much has
an email question and most people enable validation feature.

We first started with a very very smart and long email validation check that
was going to be perfect. Every time users reported a case where our regular
expression didn't envision we had to reduce it. During the years we had to
change it so many times, I am pretty sure we have left with something like
this: Does it have an @? Does it have a period? Great! You are validated!

~~~
georgemcbay
"Does it have an @? Does it have a period? Great! You are validated!"

Technically a period is not required, you can have an email address directly
on a top-level domain. Though, of course, that is a rare enough use case that
I'm sure anyone who has one has long since given up on expecting it to be
validated correctly by most webforms.

------
Narkov
"We know that gmail.com is a valid MX host while gmali.com is not."

They've failed in their own example. gmali.com has valid MX records and
accepts mail. Just because they don't accept billions of email per month, why
should this service block any mail?

~~~
abirkill
When I enter fred@gmali.com on their validation demo, I get a 'Did you mean'
message, but the icon is a warning indicator (yellow exclamation point),
rather than the error indicator (red cross).

When I enter fred@gmal.com, I get a 'Did you mean' message, with the red cross
error icon.

I presume this means that it's detecting that gmali.com is a valid domain and
can receive e-mail, but for most people it's not what they actually meant,
whereas gmal.com is both a probable typo and a domain that cannot receive
e-mail, and therefore invalid.

In other words, I think it's doing the right thing, in that it's detecting
that fred@gmali.com is a valid address, but warning you that it's not the
correct one. I think there is definitely a usability improvement though, as
it's easy to miss the yellow warning icon, and assume that it's actually
telling you that you can't use that address.

------
graylights
"3\. Mail Exchanger existence checks

Again, due to the robustness principle, just because a host does not define MX
records does not mean they can’t accept mail. Mail servers will often fall-
back to A records to try and deliver mail. That’s why we go one step further
than just a DNS query, we ping the Mail Exchanger to make sure that it
actually exists."

Plenty of boxes don't respond to pings (icmp). Can I assume you're doing a tcp
scan on mail ports?

~~~
old-gregg
> Can I assume you're doing a tcp scan on mail ports?

Nope. We're an ESP ourselves. Mailgun has delivered (and accepted) many
billions of emails since our launch a few years ago. We have a lot of data on
99.99% other ESPs in the world and the most common misspellings for them. We
also have ESP-specific data on what kind of _RFC-compliant_ email addresses
they do not allow.

------
Sephr
The demo failed on Ian Goldberg's real email address, n@ai. TLDs work just
like every other domain name in DNS.

~~~
baudehlo
Works on my validator (which is also free, and does the same checks as this
one with the exception of knowing the Yahoo trick):
[https://www.emailitin.com/email_validator](https://www.emailitin.com/email_validator)

~~~
claudius
Which also stumbles over a#\ b.\@c@[IPv6:2001:4dd0:fc8c::1] :)

------
swampthing
This is super cool - can't wait to implement it. Do you have any stats on how
long calls usually take (mostly asking because of the DNS check)?

~~~
russjones
I only have performance numbers for the parser right now, and on average it
takes about 0.07 milliseconds to parse and validate an address. That's the
pure RFC syntax part.

The longest part of validation service is DNS checks, but once the DNS server
warms up and starts caching lookups the roundtrip time is going to be the
longest part of the request.

We're still collecting reliable statistics and once we have them we can follow
up with you.

------
knes
An other alternative is using Mailcheck.js

[https://github.com/Kicksend/mailcheck](https://github.com/Kicksend/mailcheck)

At least you are not sending email addresses to a 3rd party.

~~~
sleepyhead
Not the same though. It just hints that you have a typo.

------
DenisM
This is great, thanks for your effort. Couple more things I'd love to see:

1\. Parse a chunk of text and salvage any email addresses from it that you can
find. Use case: my users upload spreadhseets with email of their other team
members, but email field would often contain more than one email (separated by
slash or space or coma or god knows what), or other stuff like Skype account
etc.

2\. Actual validation service. I'd pay for it at standard mailgun rates, it
would be easier for me than rolling my own as I do now.

Thanks again!

~~~
russjones
Check out the /parse endpoint for our API, it does exactly what you want:
[https://api.mailgun.net/v2/address](https://api.mailgun.net/v2/address)

~~~
DenisM
But it doesn't. It expects the text to be coma or semicolon separated, which
in my case it often is not.

------
cleverjake
just a few example emails that are listed as invalid but aren't

"email with spaces"@gmail.com

"very.unusual.@.unusual.com"@mail.yahoo.com

"very.(),:;<>[]\".VERY.\"very@\\\ \"very\".unusual"@strange.example.com

n@ai

Another server-side alternative is isemail.info, which does validate all of
the above.

~~~
russjones
Hi cleverjake, I'm the developer of this service at Mailgun.

You can't actually register "email with spaces"@gmail.com or
"very.unusual.@.unusual.com"@mail.yahoo.com with Google or Yahoo.

Both addresses pass pure syntax checks but then the validator kills it when it
notices that Google or Yahoo won't let you register addresses like that.

~~~
Zikes
So it would be valid if I entered "very.unusual.@.unusual.com"@mail.zikes.me,
assuming I created that address on my mailserver?

~~~
russjones
Hi Zikes, that's actually a bug, thanks for finding it. I'll fix it, and yes,
that will soon be marked as valid.

------
jlgaddis
The best "email address validation" that ever existed was SMTP's VRFY verb
(and EXPN was quite useful too). Unfortunately, the spammers killed that real
quick.

~~~
alexk
Yep, we miss this command too

------
grey-area
There are a few concerns about privacy and response times using this approach,
but you can get a very similar effect just by writing/hosting a little bit of
js locally which consults a list of common errors, does a simple syntax check,
and shows a warning beside the email entry box asking if the user is sure
about the mail address they are submitting. That's enough to make users look
hard for any mistake, and can be done in about 10 lines of js, automatically
whenever the email field loses focus. I've found even a very simple script
checking common errors/domains has a significant effect on typos and bounce
rates, and it has the advantage of not sharing the email and being a lot
quicker. If you're only ever warning the user to take a second look, you don't
have to worry about false positives.

I'm not sure that anything other than basic checks adds a lot of value, and
I'd worry about sending off users' email addresses to an API on a third party
website before they've even agreed to terms - I don't think most users would
be happy to find out that was happening.

It would be interesting to experiment with different levels of checks, and see
which ones provide the most value though.

------
kfury
Oh the edge cases will be so angry. There are a very few engineers at Google
who have Gmail account names shorter than 6 characters. It's not the norm, but
they exist. Their addresses can't be validated.

I'm sure there are special cases all over the place. It would be nice if
Mailgun differentiated between 'this address is just malformed' and 'from what
I know of [ISP], this address oughtn't exist'.

~~~
alexk
Well, you are not forced to reject the email, you can just stick to the
unobtrusive spellchecker, that just suggests in case if something suspicious.
So the end decision is up to your app.

~~~
kfury
The spellchecker doesn't actually kick in all that often. The better solution
is to give flags back differentiating between 'unlikely' and 'impossible'
email addresses.

------
hopeless
To all those that think this is no better than a regex:

Yesterday a company invited 7 new users to their account using their email
addresses. 3 of those addresses had typos in the domain names which this
service would have caught. As it was, this error was only discovered when the
service tried to send invitation emails to the new users and that's not a
great UX.

Validation emails, particularly those with a confirmation link, are a horrible
horrible solution. They interrupt the user's process flow, taking them away
from their web browser, possible delaying the process, and you'll also get
users searching through their emails and clicking that link just to access
their account (yep, really).

I think I'm going to implement something like this Mailgun service plus
sending a welcome email (with no confirmation link). If the welcome email
bounces then I can handle that case but it should happen less often with the
Mailgun live-validation.

------
harryzhang
Ev and Taylor, this is awesome and free is icing on the cake. We'll be putting
this to good use.

------
jlgaddis
This looks pretty awesome and my first thought is it would be sweet if it was
integrated into common webmail software.

I work for an ISP and we, of course, provide e-mail access via webmail. Right
this moment, I can see dozens and dozens of e-mail messages queued up on our
outbound relays that will never be delivered because the user typo'd the
recipient's e-mail address.

An amazingly high number of messages bounced back to our users (the original
senders) are due to typo's like this. Some people, despite not being "techies"
can skim over a bounce message and realize they misspelled "live.com" and will
resend. Others, well, they call support wanting to know why they suddenly
can't e-mail Aunt Sally.

------
lebek
Just for fun: as an AngularJS directive/validator
[https://github.com/lebek/angular-guardpost](https://github.com/lebek/angular-
guardpost)

------
nmridul
Here an easier option for those concerned with privacy of their users. 1)
Validate the email address first on your side (using regex). 2) Then send the
domain name part to the service for validating and correction. Maybe append a
fake username before the @.

So if a user enters someone@yahoo.cm , I validate it first then send
someotherperson@yahoo.cm to the the mailgun. Now your real user is protected.

Now you don't send them the real user name but get most of the benefits.

------
wvh
It doesn't seem to accept IDN addresses, though it's able to resolve the
punycode equivalent just fine.

I was just implementing email and name validation checks for a project myself.
Luckily email addresses can at least be validated by a confirmation email;
it's the real name field I have no clue what to do with.

It's funny that after all these years, we still don't seem to have cracked
these basic problems.

------
darkhorn
Why just don't use <input type="email" name="email"> and then send it to
validate?

------
rcsorensen
This is a beautiful thing you've put out. Thank you.

The ASCII guardpost in the API docs is also pure gold.

------
lelf
вася@мгу.рф (valid by rfc6531)

------
jitnut
This is cool! One thing- as a mailgun user, can i use it to check the valid
email addresses when one of my client sends an email to his customers? That
would save some email credit him and reduce bounce rate.

------
nandemo
> _Formal grammars (and specifically in our case a context free grammar) are a
> set of rules that define the structure of a string._

Note that a regular expression is a formal grammar too.

------
andrewcooke
in general, this sounds good (particularly the big data approach - gmail.com v
gmali.com based on stats).

but i am concerned that they don't mention RFC 3696 when describing their
grammar. it's all there (and implemented in - the now unsupported - lepl).

[http://www.faqs.org/rfcs/rfc3696.html](http://www.faqs.org/rfcs/rfc3696.html)

[http://www.acooke.org/lepl/rfc3696.html](http://www.acooke.org/lepl/rfc3696.html)

------
Sektor
As long as it accepts the + symbol in the Gmail address I'll be entering I'm
happy for sites to use whatever they want.

------
hk__2
It rejects addresses that contain a @ in the local part, even if these are
valid ones:

"foo@"@…, "me@google.com"@google.com

------
jonursenbach
Any plans on making this an actual library instead of an API?

------
tn13
They just want to get access to email address of people!

~~~
alexk
As a dev@mailgun I can confirm that we don't have any passion for accessing
email addresses of people.

~~~
himal
As a user@internet i trust you.

------
fintler
It doesn't seem to work with my extremely common email address of:

"me@example.com"@example.com

Complete fail. /s

Seriously tho, I like the design. It's refreshing to see someone avoiding
another ugly perl-style regex.

~~~
graylights
Not a valid email address since example.com doesn't have a valid mx record.
It's a reserved domain.

------
claudius
a#\ b.\@c@[IPv6:2001:4dd0:fc8c::1] also fails.

------
mattbarrie
lolz all your emailz are belong to us!!

------
cabirum
ehew46ujwhtaeg@w4tg.com is valid.

A few more ideas: String Concatenation API, Number Addition API, String Length
API...

