Claude Hopkins was doing A/B testing as early as 1910. He had people send out different versions of direct-mail ads, which they would correlate with sales.
I recommend this book. Apart from this and other techniques that he pioneered astonishingly early (we know that's true because the book came out in 1923), it's striking how many brands Hopkins created that are still household words today (Goodyear, Palmolive, Quaker). The book is also a plain good read.
Lester Wunderman, the father of direct marketing, spoke about split testing existing in the 1930s and earlier in his autobiography "Being Direct: Making Advertising Pay". I cannot recommend this book enough for anyone interesting in mass persuasion.
Exactly. It really means that he's a great copywriter who rarely gets it right in the first try. Which describes a lot of great fiction writers and programmers too.
Split testing headlines in the 1950s? I wonder how many others were doing this in that era.