
Meet the Obama campaign's $250 million fundraising platform - kylerush
http://kylerush.net/blog/meet-the-obama-campaigns-250-million-fundraising-platform/
======
patio11
Friendly advice: your rate just got another zero or two added to it. (This
goes to anyone who can write a similar case study, too.)

~~~
bhousel
That was kind of condescending, wasn't it?

~~~
ssharp
It's condescending for an authority on digital marketing consulting to think
your portfolio and talent is enough warrant charging 10 to 100 times more? I
think it came off more respectful than arrogant.

~~~
b0k
OP built an e-commerce platform that raised $1B in a matter of months. Top
comment sells bingo card software to school teachers. Who is the authority
again?

~~~
modarts
Why do people hate on patio11 so much?

------
zaidf
The wrong lesson from this would be to immediately try and get your sites'
performance to similar numbers. But what isn't explicitly stated in this post
is that at the root of it, these guys first figured out _which_ metrics matter
for their goals. Only then did the optimization pay back. In their case, a few
seconds can make decent impact because of volume. Someone with 100 uniques/day
may have a better metric than load time to optimize.

~~~
kevinconroy
Completely agree.

There are always a thousand things that you can prematurely optimize (page
load time, HTML, conversions, etc). The trick is figuring out what the highest
ROI thing is for you to optimize right now given your specific point in time.
If you get 100 visits/day set metrics around sign up, retention, or whatever
is going to build you up to 1K/day. Then figure out how to get to 10K/day, and
so on.

------
tomkin
I see a lot of concern for integration with API, hosting, etc. but what I
don't see is disgust. Disgust because to become president of the US, you
_must_ participate in this monumental financial circus of who spends what. Is
this what people want? To know the best candidate will need to raise over a
billion dollars to compete? Campaign costs should be in line with needs.
Remember when Color got $40 mil for doing basically what other startups manage
to do with a couple thousand dollars? Why would HN readers champion this kind
of spending? The reasons are probably in line with the SuperPAC reasoning,
which is "in order to change the game, you must play within the game's rules".
Sure. But when do you say _enough is enough_? In Canada, our candidates spend
on their campaign, but dollar-for-dollar, they are getting more done with less
money. Somehow.

~~~
brown9-2
Not to defend the current campaign status quo, but when comparing the US to
Canada it is worth pointing out the population of the US is ten times that of
Canada.

~~~
tomkin
True, but my problem is that democracy is supposed to be about everyone, and
when you start eliminating people based on their propensity to generate
millions of dollars you eliminate a lot of potentially great leaders.

~~~
snowwrestler
If a national candidate cannot be organized and inspirational enough to
generate donations, how can they hope to succeed as president? I would argue
that fundraising is much easier than, say, passing a major piece of
legislation.

~~~
rbanffy
In a rational society, yes. If you find one, let me know.

------
clarky07
At the risk of sounding like a troll, honest question here. Why wasn't the CVV
part of this page? Seems like a lot of headaches and bad press could have been
avoiding by just actually including it (and you know, less fraudulent
donations).

Great work by the way with the optimizations. As patio11 noted, add a 0 or two
to your rate and point people to this blog post.

~~~
ceejayoz
> Why wasn't the CVV part of this page?

Checking CVV isn't required, but may result in a higher transaction fee for
the merchant. The campaign likely made a choice of "fewer fields = better
conversion".

~~~
clarky07
It's 100% certain that it would be less conversion. It's also nearly certain
that they were accepting illegal foreign donations. In this case, conversion
probably shouldn't be the deciding factor... (Note I'm sure the OP had 0 say
in this, just wondering if there is an explanation other than better
conversions)

EDIT: To the downvotes, see my comment below
<http://news.ycombinator.com/item?id=4843963>

This isn't just about Obama, it's about all of the campaigns. Obama was just
the most visible and highest raising, with a higher likelihood of attracting
foreign money. It's also not just the CVV, but many many things that aren't
being done (that aren't required yet but should be).

~~~
ceejayoz
> It's also nearly certain that they were accepting illegal foreign donations.

A CVV wouldn't make a difference for that, and given that US citizens live all
over the world, there's realistically no way to prevent illegal foreign
donations.

What would you use? Name? American have all sorts of names. Credit card
billing address? Americans live abroad. IP? Americans live abroad. SSN?
Fakeable (ask a restaurant dishwasher in NYC), and I dunno if you can ask for
it.

~~~
clarky07
It is a tiny minority of people that live overseas. It's reasonable to make
them jump through an extra hoop or two to keep the rest of the money out of
our elections. Note this isn't just Obama's campaign, many others have the
same problems. The only thing partisan about this is that Obama is the most
visible and also the most likely person to be receiving these illegal
donations. As such, he should have been working harder to prevent them, not
being the most lax in the group.

[http://www.breitbart.com/Big-Government/2012/10/08/Report-
Il...](http://www.breitbart.com/Big-Government/2012/10/08/Report-Illegal-
Foreign-Campaign-Donations-Assault-on-American-Sovereignty)

The GAI report recommends election officials: -Integrate safeguards to limit
the solicitation of money from foreigners by requiring donors with foreign IP
addresses to provide proof of U.S. citizenship before they can proceed to the
donate page

-Immediately require campaigns to use industry-standard anti-fraud security technologies including, but not limited to, the Card Verification Value (CVV) and a rigorous Address Verification System (AVS)

-Immediately require all campaigns to retain and disclose identifying information on all online campaign contributions, including those falling under the $200 nondisclosure threshold currently allowed under federal law

-Address the threat of "Robo-Donations": The absence of industry-standard anti-fraud credit card security features render campaigns more vulnerable to so-called “robo-donations.” Robo-donations are large numbers of small, automated donations made through the Internet to evade FEC reporting requirements.

~~~
ceejayoz
[http://thinkprogress.org/justice/2012/10/09/977251/why-
the-o...](http://thinkprogress.org/justice/2012/10/09/977251/why-the-obama-
foreign-donation-scandal-is-pure-fiction/)

> Though not required by law, OFA requires a copy of a valid passport from any
> contributor who has been affirmed as eligible but donates with a mailing
> address outside the U.S. If they do not offer in one in a timely manner, the
> donation is returned.

The big question, as with voter fraud, is really "is this happening much at
all?" I suspect the answer is similar to that of voter fraud - "nope".

------
trotsky
When running numbers like the A/B conversion improvements, how do you model or
isolate external factors like "conversion rates increase as election draws
closer". Is it correct to assume that no one is willing to continue to run
original, unoptimized pages for some percent of visitors as a baseline?

~~~
patio11
Within any one particular A/B test, the changes in the true propensity to
convert over time don't matter, since you're randomly apportioning folks into
the alternatives, such that the only difference in the samples is whether
they're exposed to A or B. Over the course of an A/B testing regimen, this
doesn't apply, and you should be skeptical of results reported like "CR went
from 10% to 15%", _but_ a string of winning A/B tests is virtually
statistically certain to be driving true wins and clients love hearing the
change-over-time thing, so even if you're getting a tailwind from the
environment one tends to report the numbers anyhow.

It is a poor allocation of resources for most clients to continue to run
original, unoptimized pages just to be able to satisfy one's curiosity as to
what portion of an improvement observed over, say, 6 ~ 24 months is due to
one's own effort and what portion is extrinsic to the business. There's both a
direct financial cost to maintaining old code branches and a regret cost of
not giving them the best you're capable of serving. Round numbers: to add one
sentence of clarification to this blog post, Team Obama would have had to give
up ~$5 million in donations.

~~~
trotsky
I suppose it is a fair thing to say that when listing the sins of marketing,
reporting accurate increases in metrics that one cannot be sure they are
responsible for would appear rather low on the page.

------
luigi
Can you describe how you set Akamai to host the site? I'm familiar with using
S3 to host a static site, and using a CDN to host assets, but it's not clear
to me how a CDN can be set to host a full static site. Did
contribute.barackobama.com just mirror something like
contribute-s3.barackobama.com?

And why Akamai and not Amazon Cloudfront?

~~~
kylerush
Our Devops team configured Akamai so they would have specifics for you
(@scottvdp).

With most CDNs you have what's called a pull zone that points to your origin
server on a different domain. So when the user requests something like
cdndomain.com/whateverpage the CDN looks for the file at
pullzonedomain.com/whateverpage and then it serves what it gets back from the
pull zone.

Akamai has more advanced configuration options than Amazon Cloudfront. One
example of this is that our payment processors did not actually live on
contribute.barackobama.com, only the static files did. So we used Akamai to
setup a reverse proxy to POST the donations to a different domain. Without
this reverse proxy it wouldn't have been possible because the same origin
policy forbids cross domain, client-side POST requests.

~~~
jdotp
Akamai GTM is the feature you are asking about.

------
ccarpenterg
This is a very good example of UI optimization:
[http://cdn.kylerush.org/kr/images/contribute-before-after-
sc...](http://cdn.kylerush.org/kr/images/contribute-before-after-
screenshot.jpg)

~~~
justjimmy
Would love to know more about the reasoning and discussion behind going from a
1 page process to 4 steps. (ie: The typical purchasing mindset vs. donation
mind set)

~~~
ihodes
Four steps is better for all sorts of purchases. Each bit of informations
entered is a bit of effort invested in the process, making each additional
step more difficult to quit (sunk costs). Making the first steps easier (and,
especially, putting the payment info at the end) means the user is eased into
the process. People also like seeing progress, hence the 1-2-3-4 thing (though
that can also be daunting; seeing that there are many more steps to go: test
this). Of course, while I think this is solid general advice, test/whatever
for your own situation.

Big forms are scary. No one likes forms.

~~~
buddhaofdoubt
That's exactly what we did with voteforchange in 08. Scott Thomas and Chris
Hughes were the brilliant team behind that. Then they went back to a one page
form, like the only thing good about voteforchange was that you could make it
cutesy and brand it really easily for lots of different demographic groups and
not the shit tons of UI research and optimization that went into creating the
shortest possible path for the registrant, collecting partial data, saving
application state, etc.

~~~
buddhaofdoubt
My personal experience with local politics was that when the chips are down in
a local race people don't see the value of technology to organize. When you're
working on a local level you have to think like a local businessman/woman.
Politics at the local level has traditionally leveraged free work. Volunteer
work. Above all. Get the sap. When the sap proves useful, use connections to
get them a position up the food chain instead of paying for it. Save money.
Save money. Don't spend. You're going to be a one term shill. You won't make
it even if you make it. You'll be more in debt than you can imagine anyway.
Sell out to those with money. Don't pay for anything. Don't pay for anything.
Don't pay for anything. Use promises. Lie Lie Lie. Don't pay. Get it free.
They live next door to lexis nexus and they don't "get it". Its wasted. That's
what's down the pipe in 2014 and that's when your shit is gonna go down.
That's when you'll need someone to fix it. You know who's gonna be on the
phone? A volunteer programmer who's taking 1 class this semester because he
failed his fuck... no fuck that. You know who's gonna be on the phone? Liz
lemon. With no staff. There IS no fucking saturday night live. Liz has been
pretending there is this fucking awesome comedy show and there isn't. There
IS. NO. FUCKING. TECHNOLOGY. at the local level. Its not worth doing because
if the president pays fuck all then we're not going to bust our budget
getting, "data" pfff. Throw more bodies at it!

~~~
buddhaofdoubt
Harpo spelled nigger is harper.

~~~
buddhaofdoubt
Damn Nigga you crazy.

Maybe you got something to say.

Can't you Constrain your Cunt?!

After its all, just A-O-K.

~~~
buddhaofdoubt
Opening that fat pussy is easy. Lets get to a financial agreement we can all
live with. -Son of Baby Momma

I suppose that's kind of staid. If I send a message back on the tyson rocket
it would be, "beat your children now! While there's still time!"

~~~
buddhaofdoubt
selection committees enjoy abstruse spelling. My son will host a stageplay
regarding your annoying penchant for doubele consonants.

~~~
buddhaofdoubt
I enjoy sitting around feeling useless. That's why I choose to work in
politics at the highest levels. Thank you so much for helping me achieve my
dream of one day getting hired to sit around with my thumb up my ass for
months at a time while you have endless fucking meetings. This is my thank
you. Because I know how much you love offending people and nonsense. Thank
you.

------
akamaka
Thanks for posting this fantastic write-up! I'm looking forward to reading
more.

Does anyone know if other team members from the campaign have publicly
discussed how other aspects of the online campaign worked? (Such as analytics,
ads, CRM, social media, etc)

~~~
luigi
This Quora board has been compiling all the articles and interviews with
members of the Obama tech team:

<http://www.quora.com/obamatech>

------
dccoolgai
First of all, congratulations and thank you for doing the work you did to re-
elect the president. Your write-up there is really interesting. I have a
couple questions:

1\. According to FEC reports, Blue State Digital made millions of dollars
working with OFA... With that in mind, why is it that the campaign tech team
had to build the redundant donation API? You would think for millions of
dollars BSD probably should handle that, no? I only ask because in all the
post-election stories a big theme is the "we decided to do it in
house"...which makes sense...but begs the question - If you were "doing it all
in house" then why was this third party vendor making so much bank?

2\. Since we have you here on HN, can you tell us (very generally) what you
and the other OFA Chicago Digital/Tech/Analytics people are going to be doing
now? Going back into politics? Forming political/non-political startups?

~~~
jdotp
Blue State had multiple roles on the campaign, not just processing payments.

Many of us are joining startups, some are starting startups, and others are
yet to decide.

------
faisalchohan
I do not agree to it. Has any one seen the Obama campaign spending on software
and infrastructure. They spend half a million dollar on Microsoft software
(the highest spending), and then all the highlights of the campaign are coming
from Open source software.

What was the benefit of spending on Microsoft software. Why it is always that
money is spent on Microsoft and benefits are highlighted of using open source
software.

------
tterrace
Kyle, great post. You said the final donation was made via an XHR POST, did
you guys have a plan B for users without JavaScript or were the numbers small
enough to ignore?

~~~
kylerush
Yes. It was very simple. We just put the URL to the vendor hosted platform's
POST endpoint in the donation form's action attribute so that if you didn't
have JavaScript the form would POST like a normal form.

------
stupandaus
Reading this write-up after having read Nate Silver's post[1] today on how the
Republican party has a harder time getting Silicon Valley talent leads to an
interesting thought. Going forward the party that can successfully target the
younger, smarter base will have a SUBSTANTIAL advantage in developing better
systems. Assuming there is not some ceiling on how much better technology can
impact efficiency, these systems will translate to exponentially more
effective platforms for the party with the brightest youth.

[1]: [http://fivethirtyeight.blogs.nytimes.com/2012/11/28/in-
silic...](http://fivethirtyeight.blogs.nytimes.com/2012/11/28/in-silicon-
valley-technology-talent-gap-threatens-g-o-p-campaigns/)

~~~
buddhaofdoubt
You could also argue that Republicans will slowly peel off disaffected or
disenfranchised talent as technology in politics becomes less of a cottage
industry for seasonal fruit pickers and they have the lawyers to push against
the true ceiling you're talking about... the legal hurdles and the stagnation
of innovation that comes with winning and scaling.

------
sshconnection
Seems a little dubious to claim that the increase in donations was a direct
result of increased speed. The nature of the election cycle seems like more
donations would take place as campaigning ramped up and the election date drew
nearer. Were those kind of external factors accounted for in your analysis?

Still, very interesting writeup and congrats!

~~~
kylerush
The increase in performance that I mentioned is directly due to the
performance increase. We built identical pages on both platforms and tested
them against each other at the same moment in time.

The only difference between the two pages was the page load time. Because we
removed all other variables (time, appearance, functionality, etc.) we can say
that the 60% performance increase lifted conversions by 14%.

------
delano
It's pretty cool that Jekyll played a role in the campaign.

------
gideon_b
I love this: the $1.1B Obama campaign uses nearly the same stack as my single
page personal website.

------
brown9-2
This is fascinating, thank you for the writeup.

I have a question about the donation API though, I feel like I am missing
something:

 _We settled on a very simple solution of turning the hosted platform into a
REST API. The only big change was adding JSON as an output option instead of
HTML... We consumed the newly created donation API using JavaScript on static
HTML pages which were served by our CDN (Akamai)._

What operations did this API handle? Were donation submissions being sent from
the user's browser to this API?

By the way, I think there might be a bug with your PJAX code on kylerush.net -
when you click links to other parts of the site or to read other articles, the
page title displayed in the browser stays as the initial title from the
initial HTTP request.

~~~
kylerush
The API was quite simple. Essentially it just processed a donation. It had
some other functions to it that involved saving a users payment information to
their account as well.

Yes, the donation submission was an POST request executed with JavaScript
(AJAX) from the user's browser.

Thanks for noticing the bug! I'll have to check that out.

------
rynop
How much did the infrastructure cost - roughly? I know your not gonna give it
to me, but breakdown of CDN vs AWS costs would be sweet to know.

------
msinia
Great writeup, thanks for sharing.

I've some questions to further understand the optimization process.

\- In general, how did you measure improvements in conversion? What metric
were you using? Something like LTV or the % of conversions each variation got?

\- How exactly did you verify that the new version had a 14% larger conversion
rate? Did you run an A/B test with both?

------
naterosenberg
Kyle, amazing post! You mentioned that the 60% speed increase led to a 14%
donation increase. You used webpagetest.org to measure the 60% speed increase.
How did you measure the donation increase? Is there some sort of A/B tool for
CDNs to help quantify the impact of speed improvements on CR?

~~~
kylerush
We used Optimizely (<https://www.optimizely.com/>) to conduct most of our a/b
testing. Optimizely works on the client level (browser), not the server level
(CDN).

Optimizely is the best a/b testing software I've ever used. It's flexible and
easy to use. I suggest you check it out.

------
maximveksler
A tech* question:

How did you implement the fail-over to a different amazon region when one
fails? I'm interested in common approaches to this SPOF and what have been
done in this case.

*seeing the discussion that is going on around this post, it almost feels off-topic to post a technical question :)

------
benbunk
How many pages is Jekyll serving?

~~~
kylerush
We had roughly 500 donation pages generated by Jekyll.

------
tlear
I am curious where this will lead the next election, I think it is quite
possible that 2billion mark will be broken as everyone is looking at this an
will be getting ready for the next round in 4 years. It is quite depressing
actually

------
danso
Wow, all of that with Jekyll as the CMS generator. I've been using Jekyll to
generate static sites and have been proud that mass traffic rushes don't bring
them down...but I haven't used it for anything that generates real revenue

------
nckbz
Very nice write up! :) It's really nice to see A/B testing promoted, as well
as, a case for static sites integrated with web services. Those are the types
of technologies I'm really looking forward to seeing grow in our industry.

------
iblaine
Who do you have to know to go from CSU Fullerton to a job like that? Wow.

------
lindstorm
Before and After Optimization - [http://cdn.kylerush.org/kr/images/contribute-
before-after-sc...](http://cdn.kylerush.org/kr/images/contribute-before-after-
screenshot.jpg)

------
mldriggs
So what're you doing now that the campaign is over?!? :)

------
rbanffy
There were a couple comments about the legality of foreign donations, but I
wonder whether foreigners can volunteer for a campaign.

Some causes are worth the hassle.

------
ckluis
Simplifying the donation form seems like such a big, duh, but glad to see it
was tested.

Great results (speaking from the management/tech side)

------
sakri
Kyle is right up there with whoever filmed the 47% video. Thanks!

~~~
jdotp
Yeah, that dude is amazing. :) Come hang out with me Kyle. :)

~~~
kylerush
You tell me when and I'll be there:)

------
alvaromuir
Congrats on amazing work.

------
scottmcleod
Thanks for the writeup

------
tonetheman
Cool stuff

------
joering2
Thanks for the write-up.

Too bad that there were no checks in place to determine who is actually
contributing. For that reason, many non-citizens were able to contribute, some
from questionable places, and even Osama ibn Laden had posthumously donated,
receiving congratulation later from Barrack's wife thanking him for his
donation [1].

Since Romney was rejecting contributions that could not be confirmed as of
where they came from, this race in terms of fund-rising was skewed from a get-
go. Imagine a marathon where everyone runs on their feet and you -- against
the rules -- are using a bicycle. Guess who's gonna win?

[1] [http://www.wnd.com/2012/10/obama-accepts-osama-bin-laden-
don...](http://www.wnd.com/2012/10/obama-accepts-osama-bin-laden-donations/)

For those who hate googling:

[http://www.washingtonpost.com/wp-
dyn/content/article/2008/10...](http://www.washingtonpost.com/wp-
dyn/content/article/2008/10/28/AR2008102803413.html)

[http://townhall.com/tipsheet/katiepavlich/2012/10/08/exposin...](http://townhall.com/tipsheet/katiepavlich/2012/10/08/exposing_barack_obamas_illegal_foreign_campaign_money_loophole)

[http://www.thecuttingedgenews.com/index.php?article=76534...](http://www.thecuttingedgenews.com/index.php?article=76534&pageid=17&pagename=News)

~~~
ceejayoz
Nothing in those articles reliably supports (i.e. beyond a couple individual
anecdotes) the contention that a) the Obama campaign had "no checks" or b)
that Romney was rejecting unconfirmable contributions.

~~~
joering2
hello there, google hater!

how about this one? [http://joshuapundit.blogspot.com/2012/10/obama-campaign-
bust...](http://joshuapundit.blogspot.com/2012/10/obama-campaign-busted-for-
accepting.html)

~~~
Osmium
From the link:

"Walker said he used his actual street address in England but entered Arkansas
as his state with the Schenectady, NY, ZIP code of 12345."

In fairness, that's fraud on Walker's part not on the Obama campaign's part.
There's a limit to what kind of checks you can feasibly do. By the sounds of
it, requiring a billing address in the United States seems reasonable.

~~~
ceejayoz
Thanks, I didn't see that bit.

> There's a limit to what kind of checks you can feasibly do.

Actually, this one would've been easy to catch - we know which states have
which zip codes. The company I work for checks this in web forms, but I don't
find that such validation is common.

> By the sounds of it, requiring a billing address in the United States seems
> reasonable.

Problem is, it isn't. US citizens living abroad have every right to donate.

~~~
Osmium
Actually, I think that zip code is valid :) according to a quick Google search
at least.

But I admit I hadn't thought about citizens living abroad; that makes it more
understandable there would be these issues. It's certainly a case of (perhaps)
incompetence over malice anyway.

Edit: my mistake, I missed that he claimed the zip was in Arkansas.

~~~
ceejayoz
It's a valid zip, but not for the state of Arkansas. It says he entered
"Arkansas, 12345", which is something that could be automatically caught.

