
Why input validation could cost you a lot - thierryrietsch
https://medium.com/because-we-love-software/1b9dcc107a27
======
nraynaud
One thing I did on my website was being quite lenient in what I accepted and
then re-formulating what I understood to the user
[http://i.imgur.com/NiPtDEX.png](http://i.imgur.com/NiPtDEX.png) I'm still not
sure it's a good idea since I've never seen it anywhere else.

~~~
Ziomislaw
This is an amazing idea, thanks. It is also one of the best ways to check if
you are being understood IRL - just ask your interlocutor to explain in
his/her words what you just told.

I've seen something similar (well, now I know it's simmilar, it didn't occur
to me until I saw your example) - though I assumed it would be unhandy. It was
a way to show your webshop cart as a text saying something like: You have
bought X, We will send it to (Addres here). It should arrive @ (date). Will
cost (some)$. (it didn't look as awkward as my example though).

~~~
nraynaud
frankly I haven't done any research into that (the way to present it or the
effectiveness of the idea), I might have found the idea somewhere else, like
in the "About Face" book or in one of D.Norman's books, but I don't remember.
I strongly believe in showing the future in users before they click. They are
afraid to click and explore, that way they know what to expect. It's the same
with the date field, you can type a lot of stuff in it and it will tell you
what it decides to understand: the first letters of "Today", "Yesterday", a
number will be understood as the last day with this number(this month or last
month), a full date, etc.

------
InclinedPlane
I don't think this is a good argument against input validation, though it's a
good argument against _incorrect_ input validation, of course.

A few things that seem missing from this critique is the idea that shipping
and payment aren't just human mediated processes.

With most ordering systems payments and shipping are directly integrated using
external APIs, and if you pass garbage data out to those then you will fall
into the trap that professionals call "really fucking everything up", which is
undesirable. Processing payments typically requires a name and an address, as
does shipping. That's not much of an excuse for not doing proper input
validation or accepting non-ascii input but it does help to explain why the
problem isn't a trivial one. If users are just sending you cash (or, say, some
sort of gift-card code which might as well be digital cash) and you're just
hand labeling packages and taking them to the local post office that's one
thing, but that's not the way even very tiny businesses work today.

~~~
jordangsu
Agreed, especially when shipping, and invalid address can be expensive. We
were charged $$ for each address which was "invalid"...even as simple as
missing a "NE" on the street. An address lookup service was put in place, but
it was done non-obtrusively. The user was prompted if the address was
completely invalid, but they were allowed to ignore and continue anyway
instead of stopping the order. Bad-validation certainly doesn't justify no-
validation.

~~~
thierryrietsch
as long as one does not prevent the user from ordering, the process can be ok
but still it is not working in so many cases.

~~~
jordangsu
Really, what cases are these? We found the address lookups to be very
accurate, and saved money.

------
afandian
Last week I tried to fund something over KickStarter. The form wouldn't accept
my card's expiry date. The expiry month field went red. No explanation about
why, it just went red and I couldn't submit.

The expiry month is this month, but it expires at the end of the month, not
the start. Obviously an OBOE, right? So I wrote to them and explained.

They wrote back that the problem was that my card would have expired by the
end of the campaign (I'm still waiting for a new one because of the postal
service).

Form validation is normally syntactic rather than semantic, but KickStarter
mixed their validation types to something I didn't expect and then didn't tell
me why when it went wrong.

It's either surprisingly easy to get wrong or surprising that KickStarter got
it wrong. I'll be generous and let you choose which.

~~~
croisillon
I've seen the last month of expiry happen before. Although annoying it might
be a security for the seller. If you enter your card number on Sep 29th and
for some reason the bank only transfers the money on Oct 1st, it might not
work?

~~~
afandian
I don't care what the business rules are. I just want them to explain why my
card isn't being accepted.

------
afhof
Nope, you should still validate your input. The correct thing to do is the
validate that the input is semantically valid rather than syntactically valid.
If they enter an address, try looking it up. If they enter an email, try
sending a confirmation email. If they enter a phone number at least see if you
can find it in a phone number DB.

Customers and users do have an interest in getting their shipping details
correct. Help them get it right rather than telling them they're wrong.

~~~
danieldk
_Nope, you should still validate your input. The correct thing to do is the
validate that the input is semantically valid rather than syntactically valid.
If they enter an address, try looking it up._

We rented a newly constructed house. The old houses in that area were
demolished, but the same street names were kept. It was very annoying to have
web forms doing such validation telling me that the address was invalid (you
cannot have that house number in that street).

Another pet-peeve: I have an ë in my name. When I purchase some software
through a web form that accepts that character, please don't just remove it or
replace it with garbage in my license key ;).

~~~
Ziomislaw
this looks a bit different from the other side. When I worked at in a big
retail shop we had to have forms (ie. zip/city) validation. There was a time
when validation was as simple as proposed in the article (checking for
presence of data, not the meaning). You wouldn't believe the things people are
able to type into forms.

One of the most common errors was a bad combination of zip/city (ie. zip code
for a different country area) which required a live person on the phone asking
customer about proper data - it costed money.

The most unusual one was when a person managed to type gibberish in every
field avaliable and than add to it a proper adress, name, zip etc... all in a
three or four letter wide house-number box.

VALIDATE ALL THE THINGS!

~~~
thedufer
> a three or four letter wide house-number box

That sounds problematic to begin with. I grew up in a house with a 5-digit
street number.

------
hownottowrite
Trust but verify. The back end charges could end up eating all your profit
too.

[http://www.ups.com/content/br/en/shipping/cost/additional.ht...](http://www.ups.com/content/br/en/shipping/cost/additional.html#Address+Correction)

[http://www.fedex.com/us/2013rates/surcharges-and-
fees.html](http://www.fedex.com/us/2013rates/surcharges-and-fees.html)

------
cliveowen
I think this exposes a need for a service that does just that: localized input
validation. It knows the right ZIP code for the right country, it supports
every kind of name imaginable and lifts another weight from the shoulders of
thousands of developers.

~~~
wffurr
We use libphonenumber from Google to do just that. Have to stay on top of the
latest library version, though, to avoid pissing off customers when their
country's phone format changes, which happens surprisingly often. I subscribed
to the ITU-T newsletter for a while, and there are phone number format changes
somewhere in the world every month. Kind of amazing.

------
kutakbash
Sometimes these arbitrary pseudo-technical restrictions are just infuriating.
Once I came across an online store that wouldn't let you ship to one country
and pay with credit card of another one (say, you try to order something to
the US and pay with Canadian credit card), like no one ever moves between
countries. Or Elance.com won't check your address in Belarus because they
don't 'support' Belarus (I guess it means they didn't want to find anyone who
can read Belarussian), preventing anyone from Belarus from using their
service.

It's great the author brought this problem to everyone's attention.

~~~
bigd
This craziness with CC is the norm rather than the exception whenever the
company has international breadth. It sadly due to international treaties,
weird taxation and bureaucracy rather than coding. three examples: Apple store
Amazon Playstation Store

I agree with you that this is ludicrous, but sadly there's not much you can
do. (I keep my french bank account on only to keep my ps games.)

~~~
kutakbash
Is it really a norm? Most of online stores don't attempt to 'validate' your CC
number in this way. Entire reshipping businesses are built on this premise.

~~~
bigd
for my experience with ps store and apple store: ITA CC FR Account: No ps
store, no apple; FR CC US Account: No ps store, no apple; US CC FR Account: No
ps store; other combinations not tested. I remember having issues with amazon,
but not for the CC, but for shipping outside the country. for what i saw, the
issue with foreigners is very bad in france and us. in france your country is
usually "constant" and even the gov.fr does not allow you to specify a new
residence outside france. US are even worst. someone with no SSN almost does
not exist. Or the eventuality of foreigners is not even considered possible.
It is sad that then we go around bragging about "globalization" when it is
impossible to keep something like the credit score if you move country.

------
Gigablah
Things to watch out for when validating people's names:
[http://www.kalzumeus.com/2010/06/17/falsehoods-
programmers-b...](http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-
believe-about-names/)

Even following a spec for email validation could be trouble:
[https://github.com/swiftmailer/swiftmailer/issues/71](https://github.com/swiftmailer/swiftmailer/issues/71)

------
lnanek2
One place I've seen a lot of web developers really screw up on this when they
try to write something for mobile, they make tons of fields required that they
really don't need. Meanwhile people really hate typing and entering stuff on
mobile and it is really tough on the users. So much so that many just ditch. I
guess novice developers just see they can mark things required in some library
trivially, so they do so without thinking through the consequences.

~~~
scarecrowbob
Well, while I am all for pushing back on bad design and I'm all for showing
folks how many people abandon their forms, it's probably not so much that
novice folks think it's okay to mark everything required because it's easy to
do with a library... what is easy for novice folks to do is mark everything
required because your boss/director/client demands that it be done.

------
mholt
I hope web developers/designers learn this lesson -- quickly. Perhaps we need
to re-think how we do validation. For example, SmartyStreets is trying to make
the validation of street addresses painless, allowing you to type it basically
however you want: [http://smartystreets.com/kb/liveaddress-api/website-
forms](http://smartystreets.com/kb/liveaddress-api/website-forms)

------
oakwhiz
There's nothing inherently wrong with input validation. The real problem here
is twofold: Validation rules that make no sense, and what action is being
taken by the system when input fails validation.

Instead of just rejecting the form submission outright, why not just ask the
user for explicit confirmation that their input is not mistaken?

------
AJ007
The example the author gives is of poorly executed form validation. His
solution is to to tear down the firehouse because a firetruck hit a single
pedestrian -- "Always try to avoid input validation wherever it is possible
and not required."

If you have received a large sample from, say, 1,000,000 users a certain % of
those entries will be invalid. The number of users is too large to manually
fix or attempt to contact the user to fix the entry. If the contact
information is invalid, you can not contact the user. If the contact
information is invalid but you think you may be able to correct it, e.g., add
a .com to the end of "user@gmail", you may not even legally have permission to
contact that person.

The goal is to reduce the invalid % to as low as possible. There is no 100%
perfect rate. You are going to sacrifice 1 user for 10.

------
ygra
I like what Amazon is doing in that regard. When entering an address they try
to verify it and offer you a looked-up address, asking you whether that's
probably better.

------
moron4hire
this is exactly why Ive started arguing with my clients for using a freeform
text field for the address. Why have separate fields for state and city and
street? I dont ever need that information separately. And the user will have a
far better idea of how to format their address than I will. So let them type
the whole thing into one textbox and I never have to worry about it.

------
croisillon
Germany has a five-digit postcode system yes, but it doesn't start at 10000.

------
mariuolo
I don't understand why it complains for ö but not é or ü. o.O

------
ape4
Error: you have an accent in your name

