
Making ActiveRecord 2x faster - tenderlove
http://tenderlovemaking.com/2014/02/19/adequaterecord-pro-like-activerecord.html
======
cheald
It's good to see members of the Rails core team taking performance seriously.
I've been on a personal performance quest for the last couple of years, and
what I've discovered is that the Rails community as a whole (not necessarily
the core developers, but just people producing Rails-related software as a
whole) really seems to consider performance to be something of an
afterthought. There is a _lot_ of low-hanging fruit laying around that could
substantially improve quality of life for the entire Rails ecosystem if people
would put a little time into profiling their stuff and eliminating hotspots.

It's been very discouraging to me to see responses to performance issues that
range from lukewarm to plain "I don't care". The Rails community could benefit
a lot from a visible push by the core team to make performance a priority.

~~~
fat0wl
yea this attitude is a BIG part of the reason i switched to Java. If you learn
some common Java design patterns (ahem... program to interfaces) then all of a
sudden the magic of Ruby doesn't seem to add that much (in fact the data-
binding in Java usually simplifies form validation). So then with a huge boost
in performance for free, all of a sudden hosting costs are far lower and less
billable hours are put to testing cache expiration.

That said, I will still probably end up migrating toward Clojure/cljs and js-
focused apps...

EDIT: It's a waste to bleed a client if all your blood-money is going to
Heroku! XD

~~~
mdasen
This is a serious question, though a tad off-topic: what tools do you use in
the Java world?

I've tried to get into the Java web world, but it's always felt like the tools
weren't what I was looking for. Partly, it always feels like the Java world
builds so many ways to do things that it's hard to get a feeling on what is
canonical. Even with something like Play, should I use Java or Scala? If I use
Java, should I use Ebean or Hibernate? If I use Hibernate, should I use XML
or. . .

In some ways, it feels like Java advocacy is focused on non-coding managers.
Or maybe I just don't know where to look for developer-focused advocacy.

For example, in the C# world, Microsoft has done a decent job telling
developers how to get started and why they should care about .NET MVC. In the
Java world, it's hard to know where to get started: Spring, Play, Ninja,
Wicket, Dropwizard, Tapestry, Wicket, etc.

I guess I'd love to hear your recommendations and experience in moving to
Java.

~~~
angryasian
I think it really depends on your goals. You can look at JEE for enterprise
standards , JSF, JAX-RS, EJB, JPA or look to the spring stack which may be a
easier to work with such as spring mvc, spring and some mix of spring jdbc or
spring data. I think in general you can look at how a project like Play
integrates these various frameworks and start from there.

------
thibaut_barrere
Quoting @tenderlove: "I think we can make the next release of Rails (the
release after 4.1) the fastest version ever."

I'm glad to see the gradual progression since 3.2 stable!

------
midas007
Props for Aaron's continuous work on _mature_ optimization of Ruby & Rails. I
recall his yeoman's work on Ruby dtrace probes and Rails initialization time.

------
mrcwinn
I applaud anyone trying to improve performance. It is discouraging, though,
that it only helps with some of the fastest and most efficient queries: direct
SELECTs by ID without any joins or LIKE statements.

For those interested in good caching without bypassing parts of ActiveRecord,
Memcache + Dalli (gem) is a great solution. Caching is not always the right
solution, though. Sometimes the best choice is reversing how you get to that
data or optimizing your query directly using SQL.

~~~
jrochkind1
these particular optimizations aren't caching any data from the database.

They are only caching calculations internal to AR. So you don't need to worry
about using cached data when the actual external data has changed, is one
thing.

The caching here is only of things that can't possibly change, essentially the
output of a function for a given input.

Your memcache suggestions are a different sort of thing entirely, which
certainly may be helpful for some apps in some circumstances.

~~~
midas007
Exactly. Use both (with caveats), they sit at different levels.

It's like

    
    
       cloudflare/varnish/nginx -> rack -> rails -> app
                                              -> memcache
    
    

Anything you can use to take load off the DB, which in turn may hit some sort
of blocking I/O call (SSD or HDD) is a very Good Thing (TM). Or a sleeping
heroku dyno. ;)

~~~
jrochkind1
The optimizations being discussed in OP do not take any load off the db. They
only reduce CPU time for AR to calculate what it's going to ask the DB for.

Unless I'm misunderstanding.

~~~
midas007
Isn't the goal that the refactored AR code is basically cleaner, which allows
AR query caching to be more mergable?

Not asking for the same thing twice (or N-1 or N^2-1 extra times) if has to go
over the network is a good thing.

Databases often aren't properly configured OOTB, especially when it comes to
temporary tablespace and query caching.

~~~
jrochkind1
Seriously, have you read the OP? The optimizations in the OP do not prevent AR
from asking for the same thing more than once. They do not effect how many
times AR asks for things at all, as far as they can tell. It is _not_ caching
database responses. It is caching internal AR calculations only, and does not
reduce the number of times AR asks for things from the DB. Unless I am
misunderstanding.

------
jrochkind1
I am skeptical as to whether the calls speeded up by this optimization form a
sufficient portion of the actual call time of any actual real world apps
actions, such that this optimization will have any non-trivial effect on a
real world app.

It might. But I definitely wouldn't assume it does. Micro-optimization.

Of course, it doesn't hurt either way, unless it does in code complexity, or
opportunity cost.

------
ikawe
This seems to be but a first step - it's currently limited only to calls of
the form `Article.find(id)` and `Article.find_by_xxx(value)`.

e.g. `Article.where(xxx: value)` or `Article.where(xxx: value).last(3)` will
not benefit from this improvement yet.

~~~
tenderlove
It's possible to do that in AdequateRecord, but I've purposely left the
mechanism undocumented because the API is not stable. You can do something
like this:

    
    
      statement = undocumented_method_call { |params|
        Article.where(xxx: params[:xxx])
      }
      statement.execute xxx: "some value"
    

[https://github.com/rails/rails/blob/e5e440f477a0b5e06b008ee7...](https://github.com/rails/rails/blob/e5e440f477a0b5e06b008ee77e3c635049405957/activerecord/test/cases/statement_cache_test.rb#L18-L20)

The trick is that the cache only supports caching ActiveRecord::Relation
objects, and not everyone knows which calls will return a Relation object. For
now, I want to hide this API and make it as "automatic" as possible so that
people don't have to change app code, but still get performance boosts.

------
adamkittelson
Isn't the find_by_XXX style (soon to be?) deprecated?

~~~
chancancode
There has been a lot of confusion about this, but this is not true[1].
`find_by_xxx` and `find_by_xxx!` are not part of the deprecation.

1\.
[http://edgeguides.rubyonrails.org/upgrading_ruby_on_rails.ht...](http://edgeguides.rubyonrails.org/upgrading_ruby_on_rails.html#upgrading-
from-rails-3-2-to-rails-4-0-active-record)

------
sheff
Something else to look forward to in 2014 for Rails performance, along with
the planned arrival of JRuby 9k !

------
troyk
It appears the 2x speedup is from by-passing ActiveRecord::Relation

------
mikkelewis
Will this work for Rails 3.2?

