
Vulnerability in JSON Parser in Ruby on Rails 3.0 and 2.3 - elektronaut
https://groups.google.com/forum/?fromgroups=#!topic/rubyonrails-security/1h2DR63ViGo
======
veloper
If everyone is really going to take the route of "My X Framework is fine b/c
nothing's been reported" then I'd like to contribute these links showing
vulnerability break downs...

* Rails: [http://www.cvedetails.com/product/22568/Rubyonrails-Ruby-On-...](http://www.cvedetails.com/product/22568/Rubyonrails-Ruby-On-Rails.html?vendor_id=12043)

* Django: [http://www.cvedetails.com/product/18211/Djangoproject-Django...](http://www.cvedetails.com/product/18211/Djangoproject-Django.html?vendor_id=10199)

* CodeIgniter: [http://www.cvedetails.com/product/11625/Codeigniter-Codeigni...](http://www.cvedetails.com/product/11625/Codeigniter-Codeigniter.html?vendor_id=6918)

* Top 50 Products (Better stop using these too! /s): <http://www.cvedetails.com/top-50-products.php>

~~~
justsee
Interesting.

Rails: numerous code execution and SQL injection vulnerabilities reported over
the years.

Django: no code execution or SQL injection vulnerabilities reported.

~~~
sil3ntmac
(yet).

~~~
tedunangst
To be honest, rails does seem to be going out of its way to increase its
attack surface.

------
eggbrain
Would it ever be possible to have a self-updating framework?

As in, I use Rails to develop web applications. In the past months, I've had
to painstakingly go back to every single app I've ever worked on, and manually
update it in whatever minor way it needed updating. Now, I'm going to do that
again.

If you consider that I'm going to continue to build Rails applications, the
number of apps I will have to update every time a security vulnerability comes
out will be larger and larger. For a framework that prides itself on sane
defaults, it doesn't seem quite sane to have to worry about taking down,
updating, and then relaunching every app you ever had when one of these
vulnerabilities come out.

I don't mean updating a Rails 2.3 app to 3.2 automatically, just applying
these security patches automatically, or prompting the user to do so. Our
operating systems do it, our IDEs do it, our programs do it, why can't a
framework? I'm not saying it would be easy, but I'm sick of having to be
subscribed to these email groups just to start the manual process of fixing
everything.

~~~
densh
Stop installing latest shiniest things manually and start using default
package manager of your linux distribution. Then all the security maintenance
you will probably need is to do equivalent of "apt-get update; apt-get
upgrade" once in a while. Yes, it was that easy before all these people who
knew better came and decided that they really need latest ruby installed via
rvm, latest gems etc. All these ruby/rails deployment "best practices" stroke
me as advice given by people who had close to zero experience of maintaining a
mission-critical application for a large period of time.

~~~
tptacek
This is the worst possible option and I strongly recommend you minimize your
dependencies on your OS package manager; if possible, build your web server
from source too.

You need to be prepared at all times to deploy workarounds to newly disclosed
security problems. Package managers can and do delay fixes to accommodate the
lowest common denominator of users. You cannot run a high profile application
and be at the mercy of whoever is administrating your OS package manager.

~~~
stouset
While I usually agree with your advice, I think this approach, while
theoretically correct, is actually damaging to the majority of your audience
here. The closest analogy I can think of is that it's like requiring users to
change passwords every 30 days: great in theory, but in practice it's a
disaster.

The problem is that it is simply untenable for all but the highest-profile
sites. Finding good ops people, even in the bay area, is extremely hard. Most
sites are only going to realize that a security update has been released when
their package manager tells them it has, and updating that package is usually
a 30s process.

The companies I've seen have a hard enough time keeping track of security
updates _with_ package management. For a small to medium team with two, one,
or even _zero_ dedicated ops people, asking them to custom-compile (for
example) a webserver, ruby implementation, and other critical libraries
(openssl, glibc, etc.), subscribe to the relevant security mailing lists, and
follow along with updates and security patches is tantamount to having them
leave unpatched vulnerabilities on their systems for months or even years.

If you ask your users to change passwords every 30 days, there are inevitably
going to be a few who take it seriously and generate and remember secure
passwords every single time. But the vast majority are going to use _weaker_
passwords than they otherwise would have, and duplicate those passwords across
accounts as much as they can figure out how. Likewise, if you ask already
overworked ops guys to manually compile and keep track of security
vulnerabilities for their webserver and dozens of libraries, a few are
inevitably going to keep on top of things and release fixes minutes after
vulnerabilities are announced. But the vast majority are going to simply give
up after a month or two and be significantly worse off than if they just use
Ubuntu's automatic security package updates.

~~~
tptacek
I don't know what to tell you other than that this is something you actually
have to get good at if you're going to run a high-profile app or hold
sensitive information of any sort.

I'm not saying you need to be able to find your own nginx vulnerabilities or
even write your own patches. But reinstalling nginx or Apache from source
shouldn't be a science project for your team; you should know that you can get
your prod servers running on a from-source build.

Sink in the time to make sure you can do that _now_ , so you aren't caught
totally flat-footed when an emergency happens.

~~~
stouset
I agree with your intent. I do. If you're a high-profile site, or you're
storing extremely sensitive customer data, this is absolutely something you
need to be doing.

But your audience here is startups. Almost always, these startups are cash-
strapped, time-crunched, and have zero dedicated ops guys. Ideally, even these
types of businesses would prioritize security to the level you're asking.

In reality, as I said in my previous comment, almost none of the startups I've
worked at have had the ops capacity necessary to handle this. Even _with_
package management, servers go months without having critical security patches
applied. Asking these types of companies to do something that increases the
ops overhead necessary to apply patches is going to result in a worse outcome.
Keep in mind that it's not simply compiling from source and having
infrastructure to apply security patches across multiple boxes. It's also
keeping an eye out for reported vulnerabilities — and not all projects have
dedicated security mailing lists. Not all projects even report this
information via mailing lists.

I wish it were different. I understand where you're coming from. But the
incentives are set up ass-backwards, and until companies start having serious
liability for data breaches, protecting customer data simply isn't going to be
a priority. In the meantime, encouraging them to set up their infrastructure
in a way that requires even more ops effort when they're already struggling to
keep up is going to have an adverse outcome.

------
benmmurphy
I have remote code execution working when YAML syck parser is being used. I've
also got RCE working when psych is used (default in 1.9.x) using a similar
trick to syck.

~~~
vinhboy
I know you're not supposed to share these things, but I found it really useful
how last time there were public PoC we could use to test our patches. Any such
thing this time around?

~~~
postmodern_mod3
Here you go: <https://gist.github.com/4660248>

~~~
vinhboy
thanks

------
eric970
Great news. It's good that all of these bugs have been surfacing lately.

~~~
instakill
I don't know why this is being downvoted - I don't think this comment is
sardonic. It IS good that they're surfacing because that means we can patch
them. Better visible than invisible.

~~~
sandofsky
It isn't good. Similar projects with a similar user base, like Django, don't
have vulnerabilities of this severity with this frequency.

This points to a seriously broken process.

~~~
PetrolMan
It isn't a bad thing necessarily. Just because the vulnerabilities haven't
popped up in other frameworks doesn't mean they aren't there.

~~~
sandofsky
That's creationist logic.

~~~
navyrain
Hardly the same. Lets not kid ourselves into thinking any of the popular
software we use is bug-free, which is nigh impossible feat for any large
codebase. If this rails bug had not been discovered, it would still be there.

~~~
sandofsky
Who said software is bug free?

The issue is that similar projects, with similar success, with a similar
number of developers looking at them, have fewer severe vulnerabilities.

The simplest explanation is that one project has more vulnerabilities lying
dormant.

------
postmodern_mod3
Proof-of-Concept exploit: <https://gist.github.com/4660248>

~~~
patio11
This simple command line tool to execute arbitrary code on your server works,
kids. I'm also north of 90% probable that I could weaponize it to turn any
image tag on the Internet into "roots your local machine", and 100% certain I
could do so for any page I could coerce to execute JavaScript.

You need to patch. Now.

~~~
spohlenz
> I'm also north of 90% probable that I could weaponize it to turn any image
> tag on the Internet into "roots your local machine"

Definitely not saying you're wrong, but I'm not convinced this is doable.
Every exploit I've seen requires a request body -- how would you do that with
an IMG tag?

~~~
patio11
Go on a Rails security safari, armed with the knowledge that any YAML parsing
is victory, and pay _very_ careful attention to code paths involving
Rails/Rack route/parameter processing, especially anything which smells of
magic. To clarify: I haven't actually done the work yet.

I'm actually going on a Rails security safari later, though not particularly
looking to widen this/these vulnerabilities. I figure I've gotten enough out
of the community over the years to contribute part of a workweek and get one
more hole plugged.

~~~
postmodern_mod3
Not unless nginx/apache routes the request directly to public/. There will
definitely be more code-paths to YAML.load, but so far
ActionDispatch::Http::Parameters has been the entry point.

------
benmmurphy
it appears that some versions of rails parsed JSON by first converting it to
YAML then performing YAML.load

I think they wanted a json parser but they didn't want to write one. So they
basically just wrote:

    
    
        YAML.load(json.gsub(/awesomeregex/, "awesomereplace"))

~~~
molf
YAML is a superset of JSON. So this works just fine to parse JSON (with the
caveat it parses any YAML and is therefore not secure):

    
    
        YAML.load(json)

~~~
benmmurphy
ah. i wonder if there isn't any json stuff that yaml doesn't support because
the convert_json_to_yaml method does a bit of munging. if you take a previous
PoC and feed it raw without any modification it corrupts it and makes it
invalid yaml. for example convert_json_to_yaml will put spaces around the ':'
character.

~~~
molf
I think that kind of processing is just to work around bugs in the old YAML
parser (syck). See this post for example: <http://project.ioni.st/post/1067>

------
cschneid
Damnit, I'm busy fixing the devise issue!

~~~
benatkin
I missed that. Here's a link in case anyone else missed it:
[https://groups.google.com/forum/?fromgroups=#!topic/platafor...](https://groups.google.com/forum/?fromgroups=#!topic/plataformatec-
devise/hj6i26Dvshw)

------
jacobn
If I've disabled the JSON MIME type as input, a la the workaround to the XML
issue a couple of weeks ago, am I safe against this vulnerability?

~~~
cgcardona
It looks like this only affects 2.3.x and 3.0.x.

If you're on 3.1.x or 3.2.x then you should be good.

~~~
jacobn
Ah, sorry, should have been more clear: I'm on 2.3.x

------
instakill
Hooray for 3.1 and 3.2 apps.

------
hemancuso
The only meaningful take-away from these continued security vulnerabilities is
you shouldn't ever let a rails project you maintain ossify to the extent that
you can't easily/safely run "bundle update", commit, and deploy.

(Didn't expect to post this comment twice today, JFC)

~~~
PommeDeTerre
Well, there's also an issue of trust that I think is being overlooked.

We now need to ask ourselves, "Can we trust the Ruby community, and can we
trust software written in Ruby?"

Before these recent exploits, there were a lot of us who would have already
answered "no" to both parts of that question. Now there may be many more
people who answer them the same way.

The warning signs have been there for a long time. The general attitude of the
Ruby community is one of these warning signs. The smugness, the emphasis on
"best practices" (which usually aren't very good, in reality) and the drama
and semi-religious worship surrounding certain members of the community (DHH,
Zed, and _why) are what I'm talking about. This kind of attitude promotes an
environment where bugs can happen in the first place, then go undetected, and
in many cases also go unpatched once discovered.

At this point, I think it's necessary to scrutinize the Ruby community and
their software much more closely than has been done in the past. The
complacency of the past is not acceptable any longer, given what has happened
recently.

------
matthuggins
Can someone explain the security issue in more detail? Is it that I can supply
Symbols (and other Ruby objects) in my YAML, which normally can't be included
in JSON? That seems to be the basis of it, but I'm looking for more info if
available.

~~~
jmillikin
The Rails YAML parser allows execution of arbitrary Ruby code, by design. If
an attacker can figure out how to get input into the YAML parser, they win.

~~~
tveita
That's awful.

This seems to be something of a theme with RoR. XML parsers that parse YAML,
JSON parsers that parse YAML, YAML parsers that parse Ruby...

Why would they let that mess near user input? I thought Ruby included a proper
JSON parser now.

~~~
knowtheory
It does _now_. But it didn't in versions <3.1.

------
jballanc
Null terminated strings are simple to understand and convenient to work
with...until they can be exploited.

Non-bounds-checked arrays are fast and convenient to work with...until they
can be exploited.

Database queries using interpolated strings are flexible and convenient to
work with...until they can be exploited.

Serialization formats that can encode arbitrary objects are useful and
convenient to work with...until they can be exploited.

...and the collective wisdom of the programming world continues its
relentless, gradual, monotonic increase.

------
ollysb
There's been a lot of rails bugs coming up lately, why are so many being found
at this particular point in time? Who's finding them and what's spurred their
interest?

~~~
swang
This is related to YAML loading (I believe) so it is somewhat related to the
major bug from a couple of weeks ago. My guess is someone went through the
Rails code combing for any interaction with YAML that wasn't caught last time.

------
mjhoy
Quick & dirty find (for Rails apps 3.0.x)

    
    
      find ~/rails -name 'Gemfile' -exec grep -E "rails', '3.0" {} \; -print

------
jrochkind1
Hmm, I'm curious of the story behind this vulnerabilty NOT existing in 3.1 and
3.2. How the heck did it get fixed in 3.1 and 3.2, but still exist in 3.0 and
2.3? It was accidentally fixed in 3.1? It was on purpose fixed in 3.1, but it
didn't occur to the fixer to backport to 3.0 and 2.3? Eh?

~~~
jdminhbg
The JSON parser backend changed between 3.0 and 3.1, and that's what the
vulnerability was in.

------
jiggy2011
I wonder if it's worth someone putting together a kickstarter so that all of
these rails dependant startups can crowd finance hiring some penetration test
firms to do a thorough audit of the entire rails codebase?

------
jtchang
So who is hosting the rails security update party this time?

Is there some magic command I am missing to update all rails projects I've
ever worked on automatically?

"gem super-update-everything"

------
sil3ntmac
Here's the fix: <https://github.com/rails/rails/pull/8853/files#L1L35>

Happy hacking!

------
hayksaakian
This doesn't actually affect rails 3.1+

Awesome.

------
rurounijones
Why is it only Rails issues seem to be so popular on HN?

Is it the lovers wanting to spread the word for their rails colleagues using
it.

Or is it the haters enjoying the schadenfreude.

No other framework seems to get as much publicity on HN whenever something
goes wrong.

------
static_typed
The smell of late night coffee, having to update Ruby on Fails yet again, or
better, the colder more bitter coffee in the morning, when having to offline
and rebuild a compromised server due to this framework.

It started with such promise, and now, we are looking to migrate all Rails
apps off to alternative frameworks at the first opportunity. It really is a
shame.

~~~
koide
The usual fallacy: "the more vulnerabilities found on a piece of software the
less secure it is."

That just doesn't logically follow: your chosen alternative framework just
might have not been thoroughly audited (yet.) Or not audited by the right
people, etc.

~~~
static_typed
Actually I see it more of indicative problem with the design of Rails. We keep
getting told we should like magic, convention over configuration, but maybe it
would be better to have a config file where we can turn the magic off in
Rails. But please, no more Yaml files - it seems parsing Yaml has already
caused enough holes in Rail this quarter.

~~~
koide
YAML (or JSON) parser bugs are not related to convention over configuration.

You are free to dislike Rails design, but you can't blame it for some vanilla
parser bug.

Parsers are fertile ground for bugs, because they are by definition exposed to
arbitrary input.

~~~
mikeash
My understanding is that YAML is _supposed_ to be highly general-purpose and
is _not_ supposed to be resilient to malicious input. These aren't YAML parser
bugs, these are framework bugs that pass malicious data to the YAML parser
when that parser should never be exposed to it. And _that_ in turn appears to
be entirely due to an extremely overly-helpful and magical nature of the
framework.

~~~
koide
Nope, the problem is not due to the magical nature of the framework.

They implemented the JSON decoder with a JSON to YAML converter, passing that
through the YAML decoder.

The fix involves using an actual JSON parser and skip the going through YAML
part. So it does qualify as a JSON parser bug, IMO (which is what I clumsily
attempted to imply with my "(or JSON)" clause above.)

