
Marketing Lessons from A/B Testing a Homeless Man's Sign  - aresant
http://dailyconversions.com/all-posts/always-split-test-literal-bum-marketing-example/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+DailyConversions+%28Daily+Conversions%29)http://dailyconversions.com/all-posts/always-split-test-literal-bum-marketing-example/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+DailyConversions+%28Daily+Conversions%29
======
jsdalton
I enjoyed this, but the title and concept are a bit misleading.

I was expecting something where he made iterative tweaks to the creative (i.e.
the sign) and did continual comparisons of the success rate to see how
different changes affected the result.

All this guy did was take a homeless guy's sign and add a really cool
marketing idea to it, which increased the homeless guy's take. Yes, the
results of his changes were awesome, but it really had nothing to do with "A/B
testing" per se.

~~~
Splines
Would A/B testing encompass changes as large as what happened here? Having
never done A/B testing, it sounds like an iterative process. Would continuous
A/B testing ever converge on a solution like the one arrived here?

I guess what I'm asking is this: Is A/B testing an evolutionary approach to
developing a product? Are "revolutionary" leaps possible if you strictly
follow the A/B testing methodology? (My guess: probably not, which implies
that most people don't follow A/B exclusively, which sounds like common
sense).

~~~
inerte
Yes, but not encouraged. The literature, or most common applications of A/B
testing, is about little improvements. Phrases being changed, picture
alignment, that kinda of thing.

Of course, in theory, A/B as the name implies is about testing two different
approaches, so again, in theory, any stuff completely different can be seen as
an experiment for A/B. The problem is that humans have been trying different
ways to do things for a loooooooooong time, and the whole point of trying to
teach and learn A/B is to test things, measure, and walk towards your goal.

So yes, you can "A/B test" two completely different things, but it will be so.
much. harder to actually know what it worked. For example, you're A/B testing
a bicycle and a Ferrari Testarossa. D'uh... obviously they'll work
differently, but what caused? Price, color, speed, brand?

That's why when A/B testing is mentioned it's about small, isolable,
repeatable, incremental and evolutionary approach. Because it's a technique
with this purpose in mind.

------
edw519
Nice story, but without any practical application to A/B Testing.

Focus was changed from "This is about me" to "This is about you." No need for
A/B testing - this will almost always result in significant improvement.

~~~
charlesju
I agree that the blog post could have contained more information, perhaps some
pretty graphs on the conversions per hour with sign 1 vs sign 2. Maybe taking
2 different signs, one with a red cup, one with a blue cup and A/B testing.

But the underlying thesis is still valid. Testing multiple scenarios returns
positive changes, and further showing an interesting application that I have
never thought about -- helping the homeless with it.

------
dylanz
The hand sanitizer is an added cost that should definitely be factored into
the test. Also, this isn't a straight A/B, but an A/Incentivized-B.

------
andrewcooke
am i the only person that finds this creepy?

(yes, i understand that in some sense he's helping the homeless guy; no i
cannot yet fully articulate why this bothers me).

~~~
jonbishop
"Before you get all 'ugh, you are mocking a homeless man' on me, realize
this... This experiment improved this bums ROI"

I got the same feeling. It's because he doesn't treat the guy like a person;
he never calls the homeless man by his name, Keith, but keeps referring to him
as a bum.

~~~
potatolicious
I feel like he did more harm than good. Panhandling, unless extraordinarily
effective and coupled with a lack of substance abuse problems (all too common
with the homeless) will not pull this man out of poverty for good, or even in
the medium-term.

Giving this man more money, but without coupling that with professional
assistance, may do him a _lot_ more harm than good.

There's a reason why real academic studies have to pass through an ethics
review, and I take issue with how cavalier this guy is about his study.

If the author of the blog has experience working with the homeless, then
ignore all I've said above. But otherwise, he's operating _way_ outside his
field of expertise involving people and situations he doesn't understand, and
complexities he does not seem to be aware of. This is dangerous.

~~~
guelo
Giving poor people more money might hurt them? You're an asshole. I'm sure if
someone gives YOU more money it'll be put to excellent use, right?

~~~
potatolicious
> _"You're an asshole."_

And you can't seem to make your point without name-calling. Kudos.

This is where even a cursory familiarity with the issues surrounding
homelessness in your area would be greatly helpful. A huge portion of the
homeless population has severe substance abuse problems and/or mental health
issues. I highly doubt a bum is going to be spending his panhandling dollars
on hookers, but alcohol and blow is more than likely. Given what I know of the
proportions through volunteer work, it is entirely unsafe to by default assume
a homeless person does not have significant substance abuse problems. Sadly,
this is one part of the homeless stereotype that is rooted in reality.

This isn't to pass judgment, but rather to point out that if you really want
to help people get off the streets for good, and bootstrap themselves out of
poverty, you need to do a lot more than just give money. For a great many
homeless people the cause of their homelessness is a lot more than just being
down on their luck, it has to do with long-standing problems - substance
abuse, toxic environment, and other negative influences. Giving money without
resolving these other external issues will just exacerbate the problem.

And note that we're not talking about _poor people_ , we're talking about the
homeless. These are people who can't show up to an interview showered with a
clean t-shirt - they are not at all like the "merely" poor. Do these guys make
enough from panhandling to afford even a shitty room somewhere? Odds are, no,
and that's pretty much a bare minimum requirement if your goal is to get off
the streets, clean up, and get employed. I contribute to organizations that do
this for this very reason - my money is doing much more good (heck, it's at
least not doing bad) in the hands of professionals who are able to offer an
environment where people have access to the help they need, and the stability
to make lasting changes in their life.

And FYI, I'm contributing significant amounts of my money and time to this
particular issue, a lot more than change most people give to the homeless in
any given year. What are you doing about it?

------
cryptnoob
I was disappointed not to be informed of the experimental method. Did he
normalize the data for each time period? (Dollars/Person within 20 feet), or
is it unnormalized and we could just be looking at a situation where more
people came later. How long were the time periods of each trial? How much
better did he do at each trial, in terms of percentages?

Also, since he apparently has the data, I'm dieing to know how much one of
these guys does make per day.

------
kasterma
"The first thing we always do is make slight modifications to what we already
have."

rather contrasts with

"The next big difference is that we changed colors and went from cardboard to
white to spark the interest of people walking by instead of automatically
having negative associations that they have with cardboard and homeless
people."

The second can (and in the context of the article maybe should) be tested. The
small change is to make exactly the same text on the different material.

------
roboneal
This is hardly A-B testing...this is more barely readable sign with hidden cup
to VASTLY improved sign with prominent red cup and BRIBE!

------
00joe
Forget the whole a/b testing thing. The real value of this blog post is that
it shows a great way to utilize personal skills to help someone else. That's
the real genius, out of the box thinking here.

------
brianimmel
I often get too wrapped up in the idea of A/B testing as a digital concept.
After reading I'm starting to think of all the "everyday" items that can be
optimized with a little testing.

That said, I am now researching a way to add hand sanitizer to my website.

------
hhjj
One reason to do A/B testing is to collect data/facts. That article contains
only opinions ie : "The biggest difference, is that we are now introducing a
bribe" Trust data, not opinions.

------
aresant
What I took away from this:

\- always be testing, even (especially?) offline

\- repackaging your offer can = drastic result

\- incentives work

Agree that lack of full data was disappointing.

------
rythie
So basically:

A) Homeless man begging

B) Useful service with a variable price

------
edo
You will love this video about a businessman helping a homeless man by
changing his sign. <http://www.youtube.com/watch?v=zNLmrv7-6OY>

On another note: Hilarious idea to try and A/B test on a sign of a homeless
man ;-).

------
jasonlbaptiste
when did max change his domain name?

