But as it relates to this specific example, I don't see how to test it. Surely you're not going to just randomly remove fields from forms, to see if anything good happens. The only way I can see to deduce this particular problem would be if the site is storing all of the failed checkout entries. And I think that's the wrong thing to do in this case, because of the privacy implications
Who doesn't store failed checkouts ? - I can't see why you would consider that any more of a privacy violation than storing successful transactions.
Especially in the travel industry where credit card fraud is very high, storing failures is important for auditing/anti-fraud activities, not to mention customer support.
When a customer calls you to say "my transaction failed", and you respond "We don't have any record of your transaction" at best that makes you look unreliable and at worst the customer thinks you're calling them a liar.
Who doesn't store failed checkouts ? - I can't see why you would consider that any more of a privacy violation
It seems to me that a common failure mode would be entering a value into the wrong field, and that includes putting the credit card number into address or something. I want to do everything I can to avoid storing a credit card number (except for the one place it's saved, encrypted, at the customer's discretion).
The process doesn't have to be random. Nobody likes to fill out forms, so the goal is to make them as simple and short as possible. What value did "Company Name" add to this form? Someone thought it would be a good idea to collect this data, but I wouldn't be surprised to learn that it was never used in any way that contributed to the bottom line.
Well, the testing of that form might have been as simple as asking few people (ex. clients) to buy something and watching as they performed the task (and how they fill in the form). It could have been arranged for a few bucks less than that $12m. That magic approach is called usability testing.
But as it relates to this specific example, I don't see how to test it. Surely you're not going to just randomly remove fields from forms, to see if anything good happens. The only way I can see to deduce this particular problem would be if the site is storing all of the failed checkout entries. And I think that's the wrong thing to do in this case, because of the privacy implications