
Mailcheck.js - How we decreased sign up confirmation email bounces by 50% - skyfallsin
http://blog.kicksend.com/how-we-decreased-sign-up-confirmation-email-b
======
yarone
I think there's a general purpose need for "optimal" form components. Let me
elaborate:

\- Fields for typing-in a credit card

\- Fields for typing-in an e-mail address

\- Fields for typing-in a U.S. street address

There are widely-known techniques for optimizing data entry for these fields,
yet these techniques aren't widely adopted, and further yet they're known to
increase conversation rates.

Someone should build a (subscription) service where you can embed a bunch of
fields (and labels) onto a form with a single line of Javascript.

Then, the fields would render on the page. The performance of the fields
(their effect on conversion rates) could be measured continuously. New
variants of the labels and fields could be A/B tested continuously as well.
That is, the performance of the fields would improve over time.

If there's interest I'll elaborate in a blog post (with mockups).

 _made a few edits_

~~~
lamby
Embedding a credit card field using third-party Javascript sounds scary.

~~~
gauravk92
Stripe does this really well using a js lib. The sensitive data thus never
hits your servers and it's a much better experience for devs who don't
want/have to deal with PCI compliance.

~~~
ceejayoz
I'm not convinced using Stripe or similar services gets you out of PCI
compliance.

The vast majority of PCI compliance requirements - those relating to server
security - would seem to still apply. If someone hacks your server, they could
easily add an additional bit of JavaScript that sends the CC field to a
second, malicious server.

~~~
gauravk92
That's a worst case scenario and things could be worse if they hacked your
production code base.

All data should be 256bit ssl encrypted for point to point security and asset
tampering protection. After that, i doubt stripes js lib is much of a problem,
it communicates in a secure tunnel from the client to stripe.

They as well say you don't have to worry about PCI compliance then because you
are never handling financially sensitive data directly, only indirectly.

------
asuth
It's a cool idea, but I think they're doing it wrong.

We experimented with doing something like this on Quizlet, but didn't actually
launch anything. We first looked at a lot of the data and doing based on
string distance is the wrong approach.

For example, if you type hotmail.de into that checker, it suggests hotmail.fr.
Another is ymail.com --> gmail.com. The more valid domains you add, the more
(correct) permutations get marked as invalid. We have 20k users with ymail
accounts.

I think a blacklist approach is much more solid than a whitelist approach, I
just haven't gotten around to building it.

~~~
cnspcs-cmplr
It sounds like you have the email addresses of your users stored in plaintext.
In that case, you should be able to extract all of the domains of verified
email addresses from your existing information, thereby covering all of the
likely domain names of your userbase and future users.

~~~
apgwoz
> It sounds like you have the email addresses of your users stored in
> plaintext

Just curious, but are you suggesting that plain text is the wrong way to store
an email address? Your comment makes me draw that conclusion, which of course
seems rather silly.

~~~
cnspcs-cmplr
It can be viewed as a security vulnerability, as many folks use the same
password everywhere. As such, if somebody compromises your user database, they
now potentially have a recoverable password and a plain text email address to
go with it. This potentially compromise all users' email accounts, as well as
other services that use email as username, such as PayPal accounts.

If email addresses are obfuscated in some way, the difficulty for an attacker
is increased.

The tradeoff in convenience is that you force a user who has forgotten his
password to remember what email address she signed up with in order to recover
it via email.

~~~
apgwoz
Obfuscated in some way implies that it's reversible, which simply means that
it's just going to take a little bit of time to unobfuscate the database--in
other words, it's probably not worth it.

Hashing an email address would be pointless because the the email address is
no longer usable to do things like, you know, send email to that person. As
such, the only real option is to store it in plain text--and that makes the
most sense.

------
fookyong
Cool until I checked the source and realised that the developer has to hard-
code a list of domains they want to check against (there are none included).

Off-the-shelf usefulness would be improved a lot if the plugin contained a
list of say 100 or so of the most commonly-found email domains.

My 2c.

~~~
mleonhard
Here is the list they're using: <http://kicksend.com/assets/splash.js>

------
meow
I really like the idea. My only nitpick is that these days most projects seem
to be ending up as jQuery plugins even when they don't really use much of its
functionality. I think a standalone version would save a bit of effort for
some of us bound to other frameworks (or not using any framework).

~~~
lukeschlather
I'm not going to lose sleep over adding a JQuery dependency. The fact is it
gets you code that targets 95% of browsers with no hacks. I don't want to code
explicitly for IE6-8, and JQuery gets me a lot of functionality on those
browsers for free. Sure, there may be other frameworks, and sure, JQuery has
its faults, and sure, a framework seems like overkill. But I feel a lot more
sure about the functionality of my code when I'm relying on a broadly tested
cross-browser suite.

~~~
jarek
An option would be nice for those who did decide, for whatever reason, to not
use jquery.

~~~
MicahWedemeyer
The source is available and the license allows for derivatives. Would you be
interested in forking the code and putting together a plain-javascript
version?

~~~
jarek
No, sorry, I'm not much of an Javascript programmer, neither am I that
interested. I realize my post kind of looks like standard wankery-won't-work-
himself; I was replying specifically to lukeschlather only regarding whether a
jquery dependency is advisable.

------
twakefield
Great work and thanks for the mention. We've been thinking about doing
something similar but going a little further and actually checking mx records,
with a response if they don't exist. This would help with the long tail
domains.

Edit: Another good idea from hinathan.

~~~
dko
No problem! The Mailgun webhooks helped us collect and analyze the delivery
failures, so that was _really_ helpful. Thanks for that.

A mx checking service sounds awesome. Keep us posted if you guys do it.

~~~
asuth
Just doing DNS checks might not be enough:

`$ dig hotnail.com MX` yields results but is still invalid.

~~~
Zarel
Well, technically, it's valid; it's just unlikely to be what you meant to
type. That sort of case should probably be handled separately.

~~~
jtheory
Well worth noting: auto-correcting the email addresses would be a very bad
idea; ditto for stopping them from submitting the form if this is triggered.

But just showing a little note "are you sure you didn't mean
username@gmail.com", then _letting them submit anyway_ , isn't generally
annoying even for misfires.

Because there actually _are_ going to be occasional users who mistype
"ymail.com" when they mean gmail. Hey, thanks for the warning; the y is right
near the g.

------
pbreit
I can understand email validation for a service like PayPal or Yammer where
money or access is truly tied to an email address. But it seems like there are
a lot of service that validate email addresses unnecessarily (and could thus
improve rates by 100%). If Kicksend only sends documents to email accounts
then it would be one of the latter. If it actually makes docs available in an
account, then, yes, it would need to verify.

------
hinathan
So how about keeping the user on-site for a while and notifying them
immediately if you detect their confirmation email has bounced? If you push
the temporary session dowbstream enough to correlate seasion with bounce tou
can talk back to the user. You could probably account for a healthy chunk of
those bounces which this js doesn't catch.

~~~
skyfallsin
This is being worked on.

------
carlos
Awesome. One issue found, a hotmail.es was suggesting a hotmail.fr domain

------
jazzychad
This is seriously awesome work. I know a couple places where I want to use
this already. Thanks for sharing!

~~~
dko
Thanks! Glad to share it.

------
richthegeek
Just a quick test of doing HTTP-based checking as opposed to a string-distance
checking:

<http://richthegeek.co.uk/ui/input/email.html>

------
moozeek
We've been doing the simpler version of this for years now, basically stolen
the idea from a 2007 MarketingSherpa article [1]: we just raise a modal window
for every subscriber address and ask something like "Please check again: is
this your email address? - Yes / No (I want to correct it) (with the email
address in big letters). Has worked wonders ever since ;)

[1] <http://www.marketingsherpa.com/content/?q=node/2223>

~~~
alexchamberlain
That would really annoy me!

------
adjwilli
This is really cool. It would definitely be useful on my sites.

One thing we do differently on <http://www.queondaspanish.com/> though is
allow users who haven't confirmed there email use the logged in features but
with limitations. They can keep track of their lesson progress for example,
but not send messages to other users. They can also change misspelled email
addresses, which I think would help in your case.

------
jstsch
Nice little gimmick, thanks! We put it in our sign-up script as well. Since
most of our clients are Dutch, we added the most popular Dutch e-mail
providers (+ a big German + Belgian):

casema.nl, chello.nl, hetnet.nl, home.nl, kpnmail.nl, kpnplanet.nl, live.nl,
online.nl, planet.nl, quicknet.nl, schuttelaar.nl, skynet.be, t-online.de,
tiscali.nl, upcmail.nl, wanadoo.nl, wxs.nl, xs4all.nl, zeelandnet.nl,
ziggo.nl, zonnet.nl.

------
kree10
This is good for sanity checking the right side of the '@' but detectable
things go wrong on the left side sometimes, too. One phenomenon I've been
seeing for years is the erroneous "www." prefix, often tacked on to @aol.com
and @yahoo.com addresses. I don't think I've seen one of these that _hasn't_
bounced.

------
papaf
Another option is checking server side using whatever is your language
equivalent of: [http://search.cpan.org/~rjbs/Email-
Valid-0.188/lib/Email/Val...](http://search.cpan.org/~rjbs/Email-
Valid-0.188/lib/Email/Valid.pm)

This checks email addresses using regexps and DNS.

~~~
ceejayoz
I'd imagine domain squatters have snapped up all the common misspellings of
popular mail providers, so a DNS lookup probably isn't as beneficial as you
might imagine.

------
tlrobinson
Now if only this could prevent people from accidentally using my email address
instead of their own...

~~~
jtheory
Ha! Yup, in the past I had a catch-all on "tellmelater.com", and there were a
bunch of people who apparently use that domain as part of their "I'm not gong
to give you an email address" address.

------
revorad
If you can predict the correction with good accuracy, why wouldn't you just
fix the address on the backend? The only reason I can think of is to avoid
spamming someone else in case of a wrong guess. But for popular domain name
spelling errors, that should be almost never.

~~~
jeff18
"Almost never" is not good enough when you silently send someone's sensitive
files to the wrong person.

------
wanderr
So awesome that this exists, and the timing is serendipitous -- I was about to
ask someone on my team to make just such a thing after I noticed that the vast
majority of our bounces are really obvious typos of popular domains, like
gmial.com.

------
mshafrir
Out of curiosity, is there a significance or reason for using 2 as the
threshold value?

------
gbaygon
Thank you very much, i just implemented it on a site i'm working on, in 5
minutes!

------
jebblue
I'm not a huge JS fan but wow this is a brilliant solution. I had no idea how
many people mistype their email addresses. Thanks!

------
goronbjorn
This is awesome–a trained DB of common misspellings for email addresses would
be so handy.

------
xspence
Does this check all possible/known domains or just the most common domains
amongst users?

------
jakejake
Very cool but how is this going to affect my hatmail.com address...?!

------
lewisgodowski
Awesome, we'll be implementing this ASAP! (:

------
Coeyman
Great resource. Will be implementing this.

------
micaeked
50% from what?

------
cinquemb
good idea! i think im going to make something like this for my website but use
php instead.

------
jacoblyles
Great! Now can you stop video recording everybody that walks by your office on
Castro st? What's up with that anyways?

------
flashmob
email != identity There are better alternatives to login these days such as
OAuth and OpenId, or FB for that matter. Nerveless, this script could be
useful - well done!

~~~
kami8845
who uses OpenID? I honestly don't know anyone that does, not even hackers

~~~
flashmob
Do you have a Google Account? Your Google Account is an OpenId account. So is
Yahoo. Many hackers use it without knowing...

------
edgesrazor
This is a great idea, thank you.

------
funkah
I bought a license for an app recently and didn't get my license key because I
entered gmail.con or something. I was really pissed at them for about a day
until I pestered them and they sorted it out. After it was over I was still
kind of annoyed with them, really for no good reason, even though I was the
one who goofed up.

~~~
simonbrown
A good solution for them would be doing an email confirmation before charging
someone's card.

~~~
funkah
Well, in this case they use Paypal for payments, and I used my Paypal account,
so I got the receipt just fine, but never got the license key. So the
possibility of "I gave them the wrong email address" never entered my mind for
me to double-check.

