
Shopify has paid over $300k in security exploit bounties - mrusschen
https://hackerone.com/shopify-scripts
======
xal
This wasn't unexpected outside of the extend of the bounties.

What you have to realize is how important Security is to Shopify. We are a
trust based business to an extreme extend. We host the livelihoods of hundreds
of thousands of other businesses. If we are down or compromised all of them
can't make money ( as some of you saw during Black Friday, to the tune of
$300k+ a minute at times ).

One of the best ways for us to augment our internal security team is to work
with the white hat community. This was a pain before Hacker One but now is
significantly easier.

One challenge is that Shopify (still) hasn't really got the profile in the
tech industry that a lot of Silicon Valley local companies have. This is
totally fine by me, but it's means that if a top white hat sits down and
decide what to work on, we are not automatically top of mind.

So we decided to overspent as a kind of "marketing" investment. Hacker one is
a classical two sided market place. There is plenty of supply of skilled
researchers but also a lot of demand for their services. We want to be known
for being one of the most responsive companies and also pay top dollars for
top findings.

So the basic idea is that when we launch something new, we 10x the payouts to
bootstrap the process of familiarization. We also provide a very convenient
local environment for doing the work in. It should be more fun and more
lucrative to make Shopify related discoveries then other companies. After this
initial period we then reduce the payouts somewhere slightly above community
standards. Its all just business 101.

Internally we are actually thrilled how the shopify-scripts/mruby program
went. Most (all?) of what was found would have been caught by our sandboxing
but we don't want to rely on this. As everyone who does security knows - lots
of exploits, even if superficially contained, can sometimes combine into "the
big one".

~~~
raesene6
So I'd be interested to know your thoughts on bug bounties as against
"traditional" security reviews.

Have these areas of your application been through external reviews before
being opened up to bug bounty or did you decide to start there?

I was thinking that for the amount you've paid out in bounties you could've
engaged a reasonable team for several man-months, so was interested in what
led you more down the bug bounty line for this.

~~~
EiNSTeiN_
You seem to assume we set out to pay this amount to begin with. Indeed for
this amount we could have went other ways, but hindsight is 20/20.

No one expected to get so many valid sumbmissions in such a short time. We set
the payout amounts this high as a way to attract talent at the beginning of
the program, which worked quite well to bootstrap it.

~~~
codinghorror
I literally just got off the phone with Hacker One on Friday and they
recommend the exact OPPOSITE of what you did. Start with low or no bounty to
get the easy stuff off the plate and figure out what class of reports you want
-- then ramp up the bounty over time.

Which is what we'll be doing!

------
rando444
_we expect most vulnerabilities will no longer be exploitable without
additional bugs in the kernel or seccomp itself, and so we are lowering the
payout amounts for our program to 10% of previous levels._

I don't quite follow this logic. If bugs are now going to be more difficult to
find, one would think they would be more valuable, not less.. and that by
lowering the bounties they are lowering the incentive for people to search for
them.

~~~
jazoom
I suppose because the vulnerabilities would actually be vulnerabilities in
someone else's code they don't feel it should come under their umbrella?

~~~
zdkl
That's flawed reasoning IMO. Do they expect the underlying porject maintainers
to have the same resources they do to compensate third party vulnerability
research?

It really should be the other way around, public facing, revenue generating
projects should do all they can to subsidise vulnerability research and
upstream their findings. The alternative would be to start paying up more for
the code they use from third party and what are the odds of that.

~~~
Normal_gaussian
With those lower 'underlying project' bugs there are multiple actors who can
compensate for vulnerability research, so the market rate goes down.

It makes sense to either: lower the payout to reflect market rate or start a
seperate scheme for those projects that others can buy into. Unfortunately if
you use a seperate scheme you end up paying for bugs that don't affect you.

Personally I'd have split my own payouts into things from my own project
(100%) and things from other projects (10%).

The fact they haven't done this suggests to me that they consider the bounty
system too expensive - either in payouts or maintenance. By reducing payouts
you will likely reduce interest and increase signal to noise at the cost of
less signal.

~~~
zdkl
or send the signal to other less well intentioned parties who see the value of
owning vulns to popular underlying libs

~~~
Normal_gaussian
I don't know how strong that argument is, yet I suspect not very.

The problem with selling to 'less well intentioned parties' is that they are
hard to get a hold of, hard to trust, and time consuming to work with. I very
much doubt that many people who sell to them are not already close to them and
their ilk. I also see this much like the arms trade, where illegal trading is
an intrinsic property of the trader, not a function of the market.

------
rvdm
As someone who has built a company around working closely with the Shopify
platform, I'm very happy Shopify is taking these initiatives.

I like that Shopify isn't your typical Silicon Valley tech company. But coming
from a background as a tech and security consultant for Fortune 500 companies,
Shopify does feel like I'm back in the tech little leagues sometimes.

And this is an unfair image association problem Shopify has. Their tech is
quite amazing and a lot of very brilliant people work there.

It’s great to see Xal, the CEO of a publicly traded company with a close to
$4bn market cap, this active on HN. I’ve always considered him one of the most
brilliant engineers of our generation ever since the Active Merchant days. To
me these programs and the way they are being shared on HN really help bringing
his company the credit it deserves.

------
dorianm
The found vulnerabilities are mostly on mruby itself so it's pretty
interesting.

A lot of PoC are very simple:

    
    
        a = Decimal.new
        a.initialize a
    

[https://hackerone.com/reports/185775](https://hackerone.com/reports/185775)

    
    
        A ||= break while break
    

[https://hackerone.com/reports/183356](https://hackerone.com/reports/183356)

    
    
        a = Symbol.new
        a.inspect
    

[https://hackerone.com/reports/185957](https://hackerone.com/reports/185957)

etc.

------
antoinefink
I don't know if it's the right place, but does anyone has feedbacks regarding
the Hacker One platform? Especially for small SaaS (between 1-2M ARR)?

~~~
almost_usual
As someone who has used hackerone on both sides (managing and reporting bugs)
I'd suggest starting a private program first. Select a small group of
researches known to provide good reports and wait for them to start rolling
in. Use this as a pilot, if you see value in what's being reported keep it
open.

Keep in mind you're going to see a lot of reports in the beginning, it will
level off as you apply fixes. You'll need to prioritize these bug fixes in
your organization, if you do not fix them within a time period the researcher
has the ability to disclose the bug publicly.

I recommend you review your program guidelines with a lawyer before starting
it.

~~~
collingreene
+1 to starting a private program first which is recommended by all bounty
programs.

If helpful I wrote down my notes about starting a bounty program although my
experiences were formed by larger companies
[https://medium.com/@collingreene/bug-bounty-5-years-
in-c95cd...](https://medium.com/@collingreene/bug-bounty-5-years-
in-c95cda604365)

------
jedberg
Still cheaper than one good security engineer. :)

------
breuvertje
This bug bounty program was limited to MRuby and paid by Shopify. Does anyone
know where they use MRuby in their stack?

~~~
allover
From [1]:

>> The Script Editor app lets you create scripts that are run each time a
customer adds items to their cart. Shopify Scripts can have many uses, from
discounting products with specific tags to running promotions such as "buy 2,
get 1 free". Shopify Scripts are written with a Ruby API that allows a great
deal of control and flexibility.

The description on the bug bounty page says those 'Shopify Scripts' are
executed in an MRuby environment, which they are trying to keep sandboxed.

[1] [https://help.shopify.com/api/tutorials/shopify-
scripts](https://help.shopify.com/api/tutorials/shopify-scripts)

------
JoachimSchipper
Shopify seems to basically have given up on application-level sandboxing, and
now relies on process-level sandboxing (e.g. seccomp).

This is probably wise; the track record of in-language sandboxing is pretty
bad (see also: Java applets.)

~~~
thegeomaster
I was under the impression that JVM, CLR and probably e.g. V8 are reasonably
secure. I'd like to learn more about recent sandbox-escaping vulnerabilities
in these runtimes. Got any resources?

~~~
sanxiyn
For JVM, consider this update which happened in 2016:

"Multiple flaws were discovered in the Hotspot and Libraries components in
OpenJDK. An untrusted Java application or applet could use these flaws to
completely bypass Java sandbox restrictions. (CVE-2016-3606, CVE-2016-3587,
CVE-2016-3598, CVE-2016-3610)"

[https://access.redhat.com/errata/RHSA-2016:1458](https://access.redhat.com/errata/RHSA-2016:1458)

------
bagacrap
about a year's salary for a security-focused engineer. Did they get more or
less bang for their buck? I guess we need to ask haquaman how many hours he
spent in collecting that $49k (by my count)

~~~
raydot
I spent 3 days on it and collected $70k. Per hour that's near a top lawyer :P

~~~
SamHoustonCM
Were you recruited to work on the bounty?

~~~
raydot
no

------
codecamper
pft, the salary of one engineer & you get pure results from him. Who wouldn't
do that?

------
knodi
Honestly the highest bounty is $2000 only, seem low for remote code execution.

------
DavidWanjiru
The page won't load for me, it said, because my browser, Opera Mini 4.x, is
not supported. "But I'm browsing on a Nokia feature phone," I vehemently
object. "No exploits will run. Even ads don't."

------
WheelsAtLarge
I'm surprised it's so little. 300k is very little compared to the financial
burden that a security breach would bring. Talk to Target and Yahoo about
cost. If anything they might start looking into way of increasing it.

~~~
andersonmvd
It's very delicate to talk about financial burden, given the following
references:

"Two months after damaging data breach, Target stock has its best day in 5
years"
[http://blogs.marketwatch.com/behindthestorefront/2014/02/26/...](http://blogs.marketwatch.com/behindthestorefront/2014/02/26/two-
months-after-damaging-data-breach-target-stock-has-its-best-day-in-5-years/)

"Sad reality: It's cheaper to get hacked than build strong IT defenses"
[http://www.theregister.co.uk/2016/09/23/if_your_company_has_...](http://www.theregister.co.uk/2016/09/23/if_your_company_has_terrible_it_security_that_could_be_a_rational_business_decision/)

"The Cost of Cyberattacks Is Less than You Might Think"
[https://www.schneier.com/blog/archives/2016/09/the_cost_of_c...](https://www.schneier.com/blog/archives/2016/09/the_cost_of_cyb.html)

And my take on this topic

"Is it really cheaper to get hacked?" [https://dadario.com.br/is-it-really-
cheaper-to-get-hacked/](https://dadario.com.br/is-it-really-cheaper-to-get-
hacked/)

~~~
user5994461
So, the best way to monetize a breach is to play with the company's stock
while you disclose the breach. Interesting.

~~~
raesene6
Yeah that's already a thing [http://www.zdnet.com/article/cybercriminals-turn-
talents-to-...](http://www.zdnet.com/article/cybercriminals-turn-talents-to-
stock-market-manipulation/)

Also monetizing security vulnerabilities by place bets on stock when
disclosing has already happened [http://www.careersinfosecurity.com/st-jude-
medical-files-law...](http://www.careersinfosecurity.com/st-jude-medical-
files-lawsuit-over-device-security-report-a-9385) . It does carry the risk of
a lawsuit however...

------
laurentb
and about $150k is made of only 3 people being rewarded interestingly enough.

------
jwilk
"It looks like your JavaScript is disabled. To use Hacker One, enable
JavaScript in your browser and refresh this page."

Kinda ironic that a site that is supposedly for hackers wants you to expose
yourself to zillion browser vulnerabilities before you can see its content.

~~~
jasonlingx
'Cos a true hacker would be able to enable JavaScript without exposing
himself.

~~~
vemv
How?

~~~
cryptarch
Up-to-date browser in Virtualbox with uMatrix for manual whitelisting?

To go more tinfoil, a "trash" laptop on its own subnet.

~~~
a1a
Better keep your real laptop at a safe distance unless you want VM escape -->
Bluetooth propagation --> pwned.

~~~
tekromancr
Well, web Bluetooth is a thing now, so no need to escape VM in that scenario

------
FBSecuritySux
What's funny is Facebook -> has a publically faced image server that has NO
authentication required to see even private messages. When FB Security was
contacted ... they say it was not a "guessable" URL, ergo security through
obscurity was their "security method" of choice. This was two days ago.

If anyone wants to test this theory - setup 2 FB accounts, message an image
one FB account to the other. Click on the image with the second account (to
bring up the lightbox custom thingy they have). Drag that image into notepad
(to get the URL)... then try and logout of both accounts, clear your cache,
and you'll see the image is COMPLETELY public -> meaning no authentication is
required.

They refused to acknowledge this as a "security risk". I laughed, then was
really pissed that a PRIVATE image shared between two parties can be viewed
w/o authentication above it.

WTF?

~~~
innoying
Hi "FBSecuritySux",

I'm not a member of the Facebook security team, but I work in the industry and
your comment frustrates me. I can understand criticizing companies for poor
security decisions if they are legitimately bad decisions, but I don't think
that's the case here...

I just tested this between two Facebook accounts, and got a URL like this:
[https://scontent.fsnc1-1.fna.fbcdn.net/v/t35.0-12/12628848_1...](https://scontent.fsnc1-1.fna.fbcdn.net/v/t35.0-12/12628848_10210262841021024_251749007_o.jpg?oh=6ae8902a3a7ae3a78397cd7fa0864b45&oe=5857E4D3)

Let's imagine, for the sake of argument, that all those numbers in the URL are
predictable and 100% the security relies on the "oh" and "oe" parameters.
Taking a rather naive approach both of these appear to be exclusively hex
strings. Therefor "oh" is 16 bytes and "oe" is 4 bytes making the total
8*(16+4) = 160 bits

In other words, assuming both parameters are truly random, an attacker would
have to try (worst-case) this many combinations to view a victim's image:
2,135,987,035,920,910,082,395,021,706,169,552,114,602,704,522,356,652,769,947,041,607,822,219,725,780,640,550,022,962,086,936,576

------
vemv
I guess that all the money Shopify makes allows them to afford weak
reasoning/engineering.

Giving your users a ruby interpreter inside your infrastructure is a terrible
idea. They're just one unreported bug away from disaster!

One could think of a few alternatives, all of them involving decoupling
Shopify's servers from users' scripts.

It could be anything from Docker/k8s to AWS lambda to a custom DSL. I'm not
saying any option is easy - proper solutions tend to require effort.

