
Gigabytes of user data from hack of Patreon donations site dumped online - prawn
http://arstechnica.com/security/2015/10/gigabytes-of-user-data-from-hack-of-patreon-donations-site-dumped-online/
======
steckerbrett
That's pretty devastating to anybody who gave up their data to support things
they enjoy. I would really like to see services getting hit with massive fines
so they actually "take security very seriously" before they get owned. It's
far too late to care about it now, there's a lot of compromising data in that
leak.

~~~
icebraining
How do you distinguish someone who was lax with their security from someone
who actually takes it seriously and still got hacked?

~~~
raesene4
You can't :)

There is a huge Market for Lemons
([https://en.wikipedia.org/wiki/The_Market_for_Lemons](https://en.wikipedia.org/wiki/The_Market_for_Lemons))
style scenario in IT systems with relation to security.

Everyone will say "we take security seriously", but there's no way for
ordinary consumers (or indeed most companies) to determine what the company
meant by their statement, and to evaluate the relative security of the systems
of two companies.

This could actually provides a market incentive for companies not to spend too
much on security, as those that do will have lower profits than those that
don't.. until they get breached, and even then companies with good security
can be breached...

~~~
icebraining
I don't think it's actually that bad; for example, Patreon said that they
don't store CCs and that they correctly hashed passwords, and as a consumer
myself, I did take that into account. Obviously less knowledgeable consumers
don't know what "bcrypt" is, but that's true of any product - you can't judge
what you don't know how to judge.

~~~
raesene4
so with two patreon style sites both of which say "we take security seriously"
, until they have a breach, how would you judge which one would take better
care of your data?

The information (AFAIK) about their security mechanisms only got released as a
result of the breach, so even assuming you knew what the terms were and how to
judge good security from bad, you wouldn't have the information until the site
got compromised.

This is a _very_ common problem, some more examples here
[http://raesene.github.io/blog/2014/06/08/finding-
security/](http://raesene.github.io/blog/2014/06/08/finding-security/)

~~~
icebraining
The information that they didn't store CC cards was available in FAQs
previously: [https://patreon.zendesk.com/hc/en-
us/articles/203913779-Do-y...](https://patreon.zendesk.com/hc/en-
us/articles/203913779-Do-you-store-my-credit-card-information-)

The password hashing algorithm wasn't, but then again an informed consumer
uses unique passwords for each site, so that's less relevant.

~~~
BurningFrog
Many informed customers - perhaps most - use the same passwords on many sites
because it's too hard to remember hundreds of passwords.

~~~
icebraining
I don't think they're particularly informed if they aren't aware of password
managers.

~~~
pjc50
I'm aware, I just can't be bothered. Every time I create an account I ask
myself "do I care if this gets compromised?". If the answer is no, then it
gets a standard password.

~~~
sinatra
As long as you understand that when that site gets compromised, all other
sites where you use your standard password get compromised for you as well.
Collectively, all those sites getting compromised for you may be enough of a
reason to consider password managers.

------
mwarkentin
Apparently they were compromised via a publicly exposed Werkzeug debugger:
[http://labs.detectify.com/post/130332638391/how-patreon-
got-...](http://labs.detectify.com/post/130332638391/how-patreon-got-hacked-
publicly-exposed-werkzeug)

~~~
theintern
There's almost no hack needed here, as the article says, they basically opened
up remote code execution to anyone. That's shockingly bad.

~~~
the_mitsuhiko
Makes me wonder what can be done to prevent this from happening without making
it a terrible experience from a user point of view. Maybe the solution would
be to store a password for the debugger and ask for it on first usage.

~~~
andy_ppp
It said the guys name in the subdomain that this was available on. I can only
imagine how that guy must feel, worst thing that I can imagine happening as a
dev.

We've all made stupid mistakes and not paid a price as high as this!

~~~
xorcist
Mistakes happens. But it should not be possible to put production data in
publicly accessible systems by mistake. Not without committing a criminal
offense, at the very least.

Your customer data is your customers' data. It's not yours to toy with as you
please.

~~~
mahouse
>Your customer data is your customers' data. It's not yours to toy with as you
please.

That's the European point of view. In the US, customer data is nothing more
than an asset.

------
clebio
This just got me thinking, which would be worse -- your account data from any
one site leaked, or _metadata_ about your internet activities leaked? Thinking
about the arguments about mass data collection and metadata, I'm pretty
certain I'd be most worried about metadata. Do metadata-collecting companies
tend to be more secure, or more security aware, or it just hasn't happened
yet?

What I mean is, if your private browsing session from your laptop, along with
non-private sessions, and data connections for IM conversations, were all
leaked, that would be intensely revealing.

~~~
nitrogen
AIUI most states don't require breaches to be announced unless they get some
combo of credit cards, passwords, email/name/phone, and govt' IDs. So metadata
spying companies like Adobe/Omniture might never report being hacked at all.

Frankly it bothers me how lackadaisically business types treat security
sometimes, caring more about company image than customer privacy, to the point
of never revealing massive breaches if they don't hit states' ridiculous
requirements for disclosure.

------
bariumbitmap
I was somewhat surprised to find my email on the list, since I don't have a
Patreon account.

It seems that when Subbable was acquired by Patreon, they got the user
information, and possibly password database, too. I can't log in to Patreon
with my Subbable password, but my email address is definitely in the breach.

Makes me glad I use a password manager.

It's also a sobering reminder of the permanence of user data. You have to
trust the competence of not only the service you use now, but any company that
owns that service down the road. (Not that Subbable was necessarily more
secure than Patreon.)

------
jhh
I had just been listening to the "Talk Python to Me" podcast episode with
Albert Sheu of Patreon ([http://talkpython.fm/episodes/show/14/moving-from-
php-to-pyt...](http://talkpython.fm/episodes/show/14/moving-from-php-to-
python-3-with-patreon)). Having heard the guy's voice and his enthusiasm for
his work makes me really feel sorry for the team at Patreon. This is pretty
much worst case and may end the company.

~~~
Axsuul
Why do you think it'll end the company? It's a marketplace with network
effects. And not to mention, many people depend on Patreon for their primary
source of income. It won't simply just go away.

------
facetube
Hey, guess what? To delete your account, you have to e-mail delete@patreon.com
"from your registered e-mail address". There is no way to do it over a secure
connection. What could go wrong?

~~~
ne0n
For anyone that doesn't know, email is not a "secure connection" and the
"from" address on an email can be spoofed.

------
methodover
This scares me. I'm the eng lead of a startup that's a bit smaller than
Patreon. And hey, we program in Python too.

We're not vulnerable to this particular problem. And I feel pretty confident
that both our stage and production environments are well-protected. However, I
can't help but wonder if I'm missing something. I'm sure Patreon felt
confident a month ago.

~~~
nullrouted
If you'd like some free help I don't mind spending some time looking at your
overall setup. Feel free to email me ross [at] ruselabs.com.

------
jtokoph
Has anyone checked to see if any Facebook access tokens were saved unencrypted
in the database? They mention that users who logged in with Facebook instead
of username/password would be completely safe, but if access tokens were
leaked, then many users could have their Facebook data mined.

~~~
florianletsch
Are Facebook access tokens vulnerable to such an attack?

If I remember correctly, at least Google tokens wouldn't be: The application
receives a token from Google. With that token, a new session token is created.
This session token expires and can only be renewed with the application token
and the correct redirect URL.

If Facebook uses a similar scheme, tokens would be useless without the running
application renewing session tokens.

~~~
methodover
Nothing like that happens with Facebook, if I recall correctly. Once you have
an access token you're good to go until it expires.

------
deanCommie
This is the first time I've been Pwned
([https://haveibeenpwned.com/](https://haveibeenpwned.com/))

Does anyone know a responsible way I can check WHICH of my data has leaked
short of downloading the entire archive and searching for myself?

~~~
lt
Through the API:
[https://haveibeenpwned.com/api/v2/breachedaccount/foo@bar.co...](https://haveibeenpwned.com/api/v2/breachedaccount/foo@bar.com)

I also couldn't find the details in the site anywhere.

~~~
sp332
[https://haveibeenpwned.com/PwnedWebsites#Patreon](https://haveibeenpwned.com/PwnedWebsites#Patreon)

------
callesgg
So what of importance is in their database that one could not find by
otherwise searching the web?

From the email i received from patreon i reckon the only actual private thing
is the exact sum i pay to who.

Some find addresses to be private, but at-least in my case any one can find
where i live by looking on the local yellow pages website.

~~~
cuckcuckspruce
Some GamerGate supporters have made claims that video game reviewers have
covered games and develelopers that they have backed financially through
Patreon without disclosing this to their audience.

This disclosuee would allow those making that claim to demonstrate if their
claims are accurate, and it allows those about whom the claim is made to
demonstrate that the claim is inaccurate.

~~~
mattbasta
Do GG accusations actually matter anymore? Does anyone still care besides the
people actually making the accusations?

------
Udo
There is a lot we can do as programmers to make data leaks of that scope a lot
harder to pull off. I think web apps of a certain magnitude should not be
talking directly to the database, they should be talking to their internal
data API that resides on a different server. The reason for this is that a
completely separate data layer is easier to lock down, and also way easier to
monitor for unusual activity. Ideally, that internal API server couldn't even
communicate with the internet directly.

I've been doing web programming for a long time, and while direct DB access is
both convenient and fast, I have to admit it's not easy to monitor these
databases for breaches, or to stop them from spilling their entire contents if
so requested.

~~~
icebraining
If the frontend layer needs to access the data anyway, how does that
separation help? And why is it easier to monitor for unusual activity?

~~~
Udo
_> If the frontend layer needs to access the data anyway, how does that
separation help?_

It helps in multiple ways. Since you define the data API yourself, you can
tailor it tightly to only the exact operations you need. This makes it more
difficult to flat-out drain everything (or even query everything, depending on
your app). It's also another layer of insulation. The fact that it's another,
non-standard access protocol also helps. If I'm an attacker, I'll probably
have a harder time figuring out how the custom data layer works; and when I'm
done with that I still need to actually siphon the data, which will ideally
require me to either issue millions of suspicious requests, or try to break
into the API server itself, which has a drastically reduced attack surface
compared to a typical web app. Which is where your second question comes in:

 _> And why is it easier to monitor for unusual activity?_

Because you know what the legitimate access patterns of your app look like,
you can raise flags programmatically. In fact, you can incorporate certain
checks right from the time you start writing your API. It puts you, as a
programmer, into a position of control and you can use the global insight you
have into your app's design to your advantage by defining what's normal and
what's not.

Compare that with a general-purpose DB interface, which is ...well, general
purpose. It doesn't _know_ when it's doing something implausible. When
something goes wrong with these databases, it's often the _admin /ops_ people
who notice it, if at all.

~~~
icebraining
_> Since you define the data API yourself, you can tailor it tightly to only
the exact operations you need. This makes it more difficult to flat-out drain
everything (or even query everything, depending on your app)._

Considering that the database was designed specifically to support the website
(so it's not supposed to have unrelated data), and that the frontend
intermediates all access to the data, wouldn't the sum of the operations
require access to all data? Why would you have data that was not needed by the
website?

 _> If I'm an attacker, I'll probably have a harder time figuring out how the
custom data layer works_

This is just security by obscurity, except if the attacker has access to the
client code, it's not even very obscure. Finding all calls to the backend API
is a trivial task nowadays.

 _> Because you know what the legitimate access patterns of your app look
like, you can raise flags programmatically._

Why can't you do that from the database logs? A query is just like an API call
- you can raise flags if the frontend makes queries outside of a known pattern
(ie, non-recognized queries or in an unexpected order).

~~~
Udo
_> wouldn't the sum of the operations require access to all data_

 _How_ you can access the data matters, and typically not all data (nor all
views to that data) are required to run the app. Some info should only ever
travel one way (typically from the web server to the backend, like passwords,
analytics, or payment data). There are whole categories of information that do
not _ever_ need to be queried in bulk by a legitimate user.

 _> This is just security by obscurity, except if the attacker has access to
the client code, it's not even very obscure._

It's not like this approach in any way _relies_ on that obscurity, but as a
measure to increase time and effort required by the hacker, I'll take it. Just
increasing the time spent figuring out how to query the backend gives site
operators additional opportunities to spot that something is wrong. Compare
this to the immeasurably small effort required to dump all SQL tables into a
text file.

 _> Why can't you do that from the database logs?_

Typically logs aren't analyzed in real time, and I'd hazard a guess in most
cases they're not even analyzed at all. Most databases don't even keep an
access log by default, and in many cases it's hard to implement one that is
useful.

 _> A query is just like an API call - you can raise flags if the frontend
makes queries outside of a known pattern_

Theoretically: yes, practically: no. In case of SQL databases, this would
practically mean communicating with the DB exclusively through stored
procedures (or event handlers/filters in case of document databases), which is
a poor man's version of what we're talking about in the first place. In
practice, almost nobody uses their database in this way, because it's really
painful and does not even offer that much protection.

------
cfontes
This is now so common I am starting to become numb to about it.

------
trhway
"we take security seriously" reminds me "we come in peace" from Iron Sky :)

~~~
chucky_z
Unless every single employee actually follows this clause, just one bad apple
can ruin every security procedure in the world. Layer 8 is awfully
vulnerable...

~~~
trhway
>Unless every single employee actually follows this clause, just one bad apple
can ruin every security procedure in the world.

the point of good security architecture is exactly that such thing can't
happen. Security by compliance isn't good security.

------
dublinclontarf
I'm wondering what their setup is?

Are they running on their own machines? AWS? Heroku?

From looking at their careers the use PostgreSQL/MySQL, Python, Scala, Ruby,
Node.

I am assuming because of the nature of the breach that they are running their
own servers (either on AWS or their own machines), it's a completely
compromised server which had access to everything.

~~~
justusw
Since their servers appear to run on AWS, and knowing that the breach was
possible because their development servers were exposed to the public with
production user data, it is safe to assume that they had their development
servers on AWS as well.

~~~
dublinclontarf
Ah, development servers with real data.

------
throwaypatreon
This could be quite a scary prospect. There are certain anonymous and
controversial artists who have solicited and received donations through
Patreon. Some of these artists remain anonymous because of the controversial
nature of their work. I think this is a scary prospect for both patrons and
artists.

------
Axsuul
Why don't more websites that are vulnerable to these types of breaches also
encrypt user email addresses as well any personally identifiable information?
I don't foresee any performance implications if everything is cached anyways.

~~~
raesene4
the problem is, if you intend to use the data within the application, you need
to be able to decrypt it. If you can decrypt it, so can anyone who compromises
the website :)

with e-mail addresses you need to use them in their unencrypted form (e.g. as
login names), so encrypting wouldn't do much for you against most attacks.

~~~
Axsuul
Depends on the attack. If the key used to encrypt the data is stored on
another server and loaded into memory when the application is initialized,
then the attackers technically wouldn't be able to get access to that key
unless they also hacked the server with the key.

~~~
steckerbrett
I don't see how that protects anything, if the information is around for the
webserver to use it's around for whoever has root on your box.

~~~
Axsuul
It would still protect against attacks that don't involve having root access
to the box (i.e. Ashley Madison data breach)

------
hellbanTHIS
On the bright side if they dump the source code we may finally get a public
API

~~~
21echoes
We have a public API in development. ping david at patreon dot com for more
info if you'd like.

------
paulmd
All the data breaches lately have demonstrated the need for some kind of
professional engineering license to ensure compliance with best practices.
Even if your app is meaningless in and of itself, a data breach can reveal
Personally Identifying Information or credentials for other sites and
accounts.

There's too much of a "code cowboy" mentality out there right now. As a
community we've become very feature-driven and security is usually the last
consideration. You need somebody whose name goes on the dotted line and
assumes responsibility for the overall integrity of the system, because
otherwise it's always Somebody Else's Problem. If you get breached and you
weren't following best practices, then your PE takes on the liability, just
like a building collapse or something.

This can be accomplished without significant burden to smalltime admins now
that we have stuff like AWS and Docker. You should be able to Chef yourself a
secure AWS instance and then Docker-Compose yourself a functional application
deployment. Most smalltime users probably want some combination of CMS,
storefront, or forum, and it should be pretty easy to provide these as
prefabs. Some customization will be necessary, but then it will fall on the
appdevs to force validation and security on the webdevs. If your defaults let
someone inject SQL or something then it falls on the appdev, if the webdev
uses an "unsafe" flag and overrides the defaults it rolls downhill onto the
webdev. If you don't follow unit/integration testing best-practices, then it
falls on you. If you don't pin your versions and a vulnerability opens up in
the future - falls on you. Etc.

These standards should apply whenever you are a commercial entity or store
Personally Identifying Information. If you are a legit smalltime user and
don't want to use professionally licensed software then go ahead and do
whatever, but you should be content with storing a username/password and
perhaps using a one-time SMS or email validation that doesn't get stored.
These then travel in a "viral" fashion like copyleft - if you are building a
commercial website you need to do it using a storefront that has a licensed
PE, that runs on a runtime that has a licensed PE. Once these are commonly
available then most users would probably follow suit anyway.

This idea isn't going to be very popular with startup coders from Silicon
Valley, but the reality is that the US is way out there on this stuff and
we're seeing an epidemic of data breaches as a result. The EU has much more
stringent data privacy laws. These don't have a whole lot of teeth at the
moment, but the principles are down on paper and this is what I think you
would need to implement them effectively.

To me HTML code is really a microcosm of the problem. People will write
whatever old crap for as long as you let them get away with it. You have to
make it as easy as possible to do the right thing, but a lot of people will
have to be dragged kicking and screaming by turning on XHTML and disallowing
standards non-compliance. Professional licensing is how you do that for
applications and systems engineering.

~~~
Erik816
If we are going to put collective political effort into stemming the tide of
data leaks and hacks, I would much rather put it into finding and punishing
the people who steal the data than the people who are trying to create a
productive service and get hacked through less than perfect security.

~~~
paulmd
That's pushing on a string, man. You will never be able to effectively track
down and punish every bored Russian teenager.

As Schneier puts it - security is a process. The process doesn't have to be
perfect, but there needs to be one. If your process is drastically
substandard, someone need to be liable for that. The need for a standards
group and a single point of liability follow logically from that, just as it
does with other engineered systems. Under HIPAA an individual must be
designated a "security officer", and that role should extend to other systems
that store Personally Identifying Information as well.

I think the real point of disagreement with many people is about the
significance of a data breach. In my opinion (and the standards of the EU)
anything that leaks Personally Identifying Information is significant. It
doesn't matter if it's leaking from an app that sends fart noises to your
friends, only that it can be tied to you or your other accounts. If you really
need to be storing PII then there needs to be a requirement to do it securely
(meaning in accordance with secure best-practices). Otherwise, again, nobody
does it until it's too late.

A good read:
[https://www.schneier.com/essays/archives/2000/04/the_process...](https://www.schneier.com/essays/archives/2000/04/the_process_of_secur.html)

------
anc84
The centralisation of services like this has to stop. Why should giving money
to creators be a centralised thing (many to many) instead of just a one to
many relationship between the creator and their fans?

~~~
dennisnedry
So content creators should use PayPal then for donations? That's also a
centralized service. How do you propose a transaction takes place without a
service to process the details?

~~~
anc84
Simple standard bank transaction? If you want to support the creator
regularly, you can easily set that up as well.

~~~
paulmd
I'm assuming you're a Euro - most places don't have giro transfers like you
guys do.

A long time ago the US and the European financial systems diverged. The US
banks optimized for check processing and credit cards, while the Euro banks
optimized for bank transfers. Domestic bank transfers typically cost around
$25-35 a pop and take several days to clear in the US. Checks usually post
instantly but take about as long to clear - the time between it posting and
clearing is basically a loan, and the check may still bounce. Until a few
years ago, many banks didn't even allow you to do bank transfers online.
Stores will sometimes use "electronic checks" where you provide a blank check
and they read the account numbers off it, but this is just a shortcut to avoid
handling the piece of paper. The funds are still moved through the checking
system. These are not available to regular account-holders either.

That's why we come up with all these systems like Paypal and Patreon to move
money around - the bank-level tools are cumbersome, slow, and expensive for
us.

------
6stringmerc
Wow, I avoided Patreon because I didn't trust that all the music being hosted
was following proper licensing and compensation rules (e.g. looked like >50%
of songs were covers), so this is kind of a double shock to me about how they
ran their operation.

Believe it or not, this re-inforces my commitment to SoundCloud, which isn't
monetized (yet), thereby keeping listeners' financial information off the
table for the present.

~~~
griffinmb
I believe you are thinking of a different platform. Patreon is a Kickstarter-
like site, but with a monthly pledge. It is not for music hosting.

~~~
6stringmerc
No, I'm not mistaken, at all. I'm very clear on what I understand about
Patreon.

If you want, you can probably get into the hacked data and see my email
warning them that by being a financial conduit they were not abiding by Safe
Harbors with respect to the real rights owners.

It may not have "hosted" the music, but if you click on the music page, what
do you see? Music videos. Then Patreon was the method to give those people
money. People who may or may not have secured the proper licenses and paid the
original artists.

Also, if I was an artist on the site, my personal information would be part of
that dump - SSN, etc - so my reluctance to engage with them was prudent.

Edit: Downvoting my observations? I guess there are more people here that
don't understand copyright than I figure, oh well.

------
chatmasta
I've never heard of Patreon until now. This might be the most press they've
ever gotten. At what point do we start asking questions like, did this company
"hack" themselves for the publicity? There is a great moral hazard in
providing so much free publicity to companies that get "hacked."

~~~
6stringmerc
Unlikely, due to the rumored personal information of all the artists being
compromised. That's not a rational avenue.

~~~
chatmasta
Unless nobody finds out...

