
Ruby Security Have You Not - vasinov
https://hakiri.io/blog/ruby-security-have-you-not
======
phillmv
Hey!

I'm one of the maintainers behind the Ruby advisory database:
[https://github.com/rubysec/ruby-advisory-db](https://github.com/rubysec/ruby-
advisory-db)

We're trying to build a common database for people building these tools; at
the moment, trawling through CVE disclosures and various mailing lists is a
largely manual process that we can reap economies of scale by pooling our
efforts together.

It's free and volunteer run. I would like to encourage you and anyone else
reading this who is interested in bolstering the security ecosystem to
consider using and contributing to the advisory database.

It's the database that powers [http://github.com/rubysec/bundler-
audit](http://github.com/rubysec/bundler-audit) and (disclaimer: I made this)
[https://gemcanary.com](https://gemcanary.com) and the more people
contributing the more we can all benefit from improving the ways we can notify
end users of their vulnerable dependencies.

Thanks,

~~~
vasinov
Do you know of any apps like [https://hakiri.io](https://hakiri.io) that
automate security scanning and serve as a security monitor for any future
disclosures?

Also, how would one use bundler-audit if a Rails app is not actively
maintained?

~~~
phillmv
Yeah, like I mentioned above the app I built
([https://gemcanary.com](https://gemcanary.com)) automates disclosure
notification, and I believe?
[https://codeclimate.com/](https://codeclimate.com/) may use the advisory
database as well, but I do not know for a fact.

There's a few others but they've slipped my mind at the moment.

I use bundler-audit when performing security audits for clients. You could
wrap it in a git commit hook and have it publish results, but to my knowledge
no one's done it that way.

------
chc
I can't help but feel that this could also be written as "How secure are Ruby
and Rails apps? About as secure as all other apps."

In my experience, vulnerabilities are pretty much endemic to software that has
not been hardened by experts.

~~~
danudey
I think the big thing here that threw me for a loop is just how many gems are
pulled in by projects. 133 seems like a ridiculous amount of dependencies for
a single project. Our entire company runs on Python in the backend and we
don't even have 50 (and that's including ones I use for command-line utilities
or corncobs, for example, which aren't in our public-facing software).

Even if the Python modules we have had the same number of vulnerabilities as
gems, our exposure for a given system would be 1/10th, just from not using so
many external libraries. Couple that with the fact that many of those gems are
outside of end-user control, of generally unknown quality, might only be
pulled in for one small piece of functionality, and might themselves depend on
other gems which you weren't necessarily aware of and haven't necessarily
vetted.

~~~
GlennS
It would be interesting to know why this is. Are each of these dependencies
really small on average? Or is there some big cascade of dependencies? Or is
there a particular framework that's to blame?

I've had a similar issue when using the Spring Framework in Java, where
including it in your project at all brings a whole host of other (often not
very good) libraries in. This is a problem in Java, which doesn't have any
built in capacity for loading multiple versions of the same library.

In Python, I suspect that I've been in a similar situation when using things
like Pyproj, Numpy or Scipy, because those all depend on components written in
C or Fortran. Perhaps next time I use any of those libraries I shall take some
time to read more of the code and try to trace out what things they may be
brining in.

~~~
ddebernardy
> Are each of these dependencies really small on average? Or is there some big
> cascade of dependencies? Or is there a particular framework that's to blame?

All of that, really. For better or worse, scores of gems have dependencies.
Rails and a few popular others add large dependencies that cascade to other
dependencies at that — some large, some not. It's extremely rare to run into
quality gems with very few if any dependencies. (Sequel comes to mind, and
that's just about the only 1st-class gem I can think of in pure Ruby.)

------
enraged_camel
This doesn't surprise me, and I'm not even an experienced Rails developer.

I think a big part of the problem is that there are a ton of gems that are
simply hobby projects that gained traction and became popular. They were
originally architected by enthusiasts, rather than experts. Some of these
projects may also have been abandoned by their authors, but still in use
because they may be the only way to accomplish a complex task or integration.
You can't expect stuff like that to be very secure. You just have to think
carefully about whether the risks of using them are worth the gains made from
not having to implement the functionality yourself.

~~~
hox
This can be said of any open source library, but it is unfortunately true that
the average Ruby gem seems to be more hobby-grade than enterprise-grade.

The fact of the matter is, if you are developing an application and are
integrating 3rd-party code, always pay attention to what that 3rd-party code
is, how well-maintained it is, and how the project has responded to bugs and
vulnerabilities in the past.

~~~
midas007
Yup. I've done enterprise Rails consulting, which is almost a contradiction in
terms. But it's possible and I've brought fixes like net ldap to work with AD.

Edit: Can't take all the credit, client hacked together the first version but
we managed to extract it for "the greater good." Also fun stuff like XML-RPC
(Xen API -> gem xenapi, VMware (gem rbvmomi)), which wasn't bad and worked
OOTB. Wished MS exposed their APIs as RESTful endpoints, because WinRM + gem
winrm just doesn't cut it with some products... generated powershell run by an
agent instead. For some products, even having the (.someextiforgot) files that
describe the API, there's no MS docs on them, so lots of trial-and-error in
PowerGUI to find the right objects and methods (Yuck).

~~~
hox
You sir are a saint. Dealing with net_ldap and AD was the bane of my existence
for a while.

~~~
midas007
Thanks. I'm the 😈, really. }:) <\-- just in case you're using an ascii
browser.

[https://github.com/steakknife/ruby-net-
ldap](https://github.com/steakknife/ruby-net-ldap)

Shameless self-promotion: Clients call for this devil if something's hard or
something's broken.

------
midas007
Generally: don't trust anything from the outside world or anything that can
transit untrusted infrastructure, that means check types and sanitize values
before passing along. Break loudly and quickly to get attention for a fix.
Keep the codebase as tiny as possible too.

Ruby: recompile with minimized OpenSSL 1.0.1+ (LibreSSL when possible) and
with patches that improve Ruby's default OpenSSL security.

[https://gist.github.com/steakknife/8228264](https://gist.github.com/steakknife/8228264)

[https://gist.github.com/steakknife/10092587](https://gist.github.com/steakknife/10092587)

[https://gist.github.com/steakknife/10096008](https://gist.github.com/steakknife/10096008)

For Rails apps: use brakeman as one part of security audit strategy

For gem authors, sign them (please!): I wrote waxseal to make it dead simple

    
    
        [sudo] gem cert --add <(curl -L https://gist.github.com/steakknife/5333881/raw/gem-public_cert.pem) # adds my cert (do once)
        [sudo] gem install waxseal --trust-policy HighSecurity
    

For gem users, find which aren't signed

    
    
        Add this to ~/.gemrc gem line:
    
        --trust-policy MediumSecurity
    
        or just if there's no gem: .... already: 
    
        gem: --trust-policy MediumSecurity
    

For anyone using git, sign your tags (git tag -s ...) and commits (git commit
-S ...) por favor

~~~
InAnEmergency
For what it's worth, --trust-policy has been broken since RubyGems 2.0.0 and
will not be fixed until 2.3:
[https://github.com/rubygems/rubygems/issues/859](https://github.com/rubygems/rubygems/issues/859)

------
jusob
More alarming than vulnerable gems is the the number oh high severity
vulnerability found in Rails itself:
[http://www.cvedetails.com/product/22568/Rubyonrails-Ruby-
On-...](http://www.cvedetails.com/product/22568/Rubyonrails-Ruby-On-
Rails.html?vendor_id=12043). Compare this to Django
([http://www.cvedetails.com/vulnerability-
list/vendor_id-10199...](http://www.cvedetails.com/vulnerability-
list/vendor_id-10199/product_id-18211/year-2013/Djangoproject-Django.html))
per year.

~~~
techpeace
number of reported vulns != number of existing vulns

------
molf
The graphs of Gem distribution and Vulnerability distribution are utter
nonsense. You can't just measure your mean and standard deviation and expect a
bell curve that fits those two parameters to correspond to your distribution.
_Especially_ if you cut off one of the tails.

~~~
vasinov
Do you mind sharing how you would present this data?

~~~
blt
A histogram.

~~~
vasinov
Why do you think it would be better/more meaningful?

~~~
teraflop
The current graph is actively misleading because it misrepresents the data as
following a (truncated) normal distribution when that's very unlikely to be
the case. A histogram would show the actual distribution.

The kind of graph being used here -- a "best fit" bell curve based on the
sample mean/s.d. -- can be useful when overlaid on a histogram, to illustrate
how closely the data can be approximated by a normal distribution. Without
that context, it's essentially meaningless.

------
unreal37
I am not a ruby developer, but it does seem shocking that 66% of all gemfiles
contain a known previously-disclosed 5+ exploit. Yes, most software is flawed,
but you would think once it's been reported, it would be fixed.

~~~
spatten
The gems have probably been updated by the developers of the gem.

The data used in the article contains a list of the versions being used in
production by real app, not the latest versions released by the gem
developers.

~~~
klochner
Risk also depends on how you're using the gem.

If it's a bug in a YAML parser but you're not loading YAML from untrusted
sources, then it would be a false positive.

------
steveklabnik
It's possible some of these are false positives, at least with regards to
Rails, as most of the recent vulnerabilities I can remember come with
workarounds and patch files. Some people need to stay on a specific Rails
version, but may have fixed the problem.

I'm not ready to speculate which is more probable.

~~~
vasinov
Yep, totally agree, that's why I added a post scriptum :)

~~~
steveklabnik
Ah! Missed it! My bad.

------
thesp0nge
Hi there I'm the man behind the codesake-dawn security scanner for ruby code.
It will be great having some of you comparing the results obtainw with haikiri
or other scanner and mine. Just for sake to reach an enterprise level tool.

Regard paolo@armoredcode.com

------
Glyptodon
This article kind of confuses me - what does it mean for a gem do be 'secure'
? The idea of many gems being 'secure' or 'non-secure' is a non-sequitur.
Obviously for some projects, like Rails, it's fairly clear what is meant when
someone talks about a vulnerability - that you can hacked somehow. However, if
someone wrote a gem to wrap eval into a command line tool so they can use some
random ruby commands or libraries from their shell, well, secure would be a
non-sequitur, but also beside the point.

------
crazydoggers
Unpatched Rails apps are found vulnerable until patched... news at 11.

~~~
vasinov
I think the interesting part is how many unpatched, hence vulnerable, apps
there are.

~~~
crazydoggers
But this article does not analyze that. It's only scanning CVE's and most have
already been patched when the CVE is issued. Of the 53 mentioned for Rails for
instance, all have been patched.

This article is incredibly misleading.

~~~
vasinov
I don't understand... The article first analyzes _apps_ that use a certain
number of vulnerable gems. Sure, most of them are probably patched, but it
doesn't change the fact that certain apps use old unpatched versions of these
gems.

The second part focuses on historical vulnerability data for Ruby gems.

In what way any of it is misleading?

~~~
crazydoggers
> Sure, most of them are probably patched, but it doesn't change the fact that
> certain apps use old unpatched versions of these gems

Why is that news?? That is true across every and all software platforms.

> In what way any of it is misleading?

Because the article assumes or implies the conclusion that there are a huge
number of vulnerable Rails apps. The only way to discover that is to not
simply count how many CVEs have been issued for each gem, but how many systems
in production have not been patched.

From the comments here, many people seem to assume this is an analysis of the
number of unpatched vulnerable systems in production, which it most definitely
is not.

~~~
vasinov
> Why is that news??

It's not exactly news but it's nice to see it broken down in a cohesive way.

> Because the article assumes or implies the conclusion that there are a huge
> number of vulnerable Rails apps.

I think the article quite clearly indicates where all data points came from
(Hakiri Facets) and then analyzes them. There is a post scriptum at the end
that encourages readers to not evaluate their or any other projects based on
the number of CVE vulnerabilities and goes into detail why it's not always a
great idea.

------
mratzloff
This doesn't surprise me in the least, but the increase in 2013 is obviously
due to increased security scrutiny, which is good so long as people act on
that information.

As a side note, I'd be interested to see a similar analysis of popular Java
projects.

------
tkellogg
It's neat (if not alarming) to see security metrics visualized like this. Nice
work!

------
fuj
STOP LYING! Everyone knows the only security issues nowadays are caused by
developing in PHP, that's why we bash it every day, right guys?!

------
hawleyal
Nice graphs. Meaningless, but nice.

