
Speeding Up Rendering Rails Pages with render_async - nikolalsvk
https://semaphoreci.com/blog/2017/06/08/speeding-up-rails-pages-with-render-async.html
======
matthewmacleod
That's actually a relatively nice, Rails-magic-style approach to solving this
sort of thing. Of course, if you were building a more interactive application,
you'd already have a JS framework in place that would negate these benefits –
but I'm still convinced there's a nice middle-ground for server-rendered Rails
apps that avoids the various problems of SPAs.

It would be nice if it worked without JavaScript though – an increasing pain-
in-the-arse about the web generally. If only it were possible to have
<iframe>s adjust to their content size, then we wouldn't need JavaScript at
all!

~~~
kawsper
I was the original author of this gem, so it is nice to see it here on HN, I
have since passed it on the the caring hands of Semaphore since they wanted to
maintain and improve it :-)

The software was built for a forum where the admin buttons was only for
specific users, but the rest of the frontend was the same, so we used
render_async to render some content for some users, and other content for
admins, and the rest of the page could be cached statically.

We later changed it to use edge side includes with Varnish. An example of this
is where you add the to your HTML:

    
    
        <esi:include src="http://example.com/1.html" alt="http://bak.example.com/2.html" onerror="continue"/>
    

This will make Varnish fetch the URL(s), assemble the page and present it to
the client, so your backend might see more requests, but the client only sees
one.

~~~
gingerlime
It's an interesting approach. It's only async from the client perspective
though. Right? The server still processes the action/render inline? (i.e. it's
not spawning a sidekiq process as far as I can tell).

We recently implemented something that's async on the server, using a
combination of pusher and sidekiq. This was particularly useful for cases
where we access an external API and don't want to hold the unicorn process for
too long. So when the response comes back we push it back to the client and
display it. Would be interesting to maybe combine this approach with some
server-push and background workers and what this gem is doing.

~~~
kawsper
Render-async lets the client render the page, and then request "partials" from
the server, the backend sees two different requests from the browser

The inline includes that Varnish does is synchronous for the client, but
asynchronous in your backend, your backend sees multiple requests.

I am interested in your pusher+sidekiq implementation, it is a good idea, but
it might be a lot of work :)

~~~
gingerlime
It's definitely more involved, and might be slightly more complex to
generalize, but in essence it's quite straight-forward:

1\. Client sends a request via Ajax to a rails controller and simultaneously
opens a private pusher channel to wait for a response.

2\. Controller returns a response immediately and fires-off a sidekiq job.

3\. The sidekiq job does the heavy lifting and pushes the results back via
pusher (it can be JSON data, or a rendered partial).

4\. Client receives the response via the pusher channel and shows it on the
page.

~~~
jalopy
How do you set up a "pusher channel" to the browser? Websocket?

Would love to see a write-up/notes/gist that demos this.

~~~
kornish
Pusher is based on websockets, yes.

[https://pusher.com/websockets](https://pusher.com/websockets)

------
jonathanhefner
I think there is a more Rails-y way to do this. Derek Prior gave an excellent
talk[1] at the recent RailsConf. The TLDW is "all-REST all-the-time." Applying
this principal to the example in the article, instead of adding a custom
`movie_rating` action, you would create a `MovieRatingsController` with a
`show` action.

Additionally, you might create an SJR[2] template which injects the rating
directly into the page, possibly avoiding the need for a separate gem.

[1] [https://www.youtube.com/watch?v=HctYHe-
YjnE](https://www.youtube.com/watch?v=HctYHe-YjnE)

[2] [https://signalvnoise.com/posts/3697-server-generated-
javascr...](https://signalvnoise.com/posts/3697-server-generated-javascript-
responses)

------
deedubaya
This could be done without a gem relatively easy if you're using Rails5+ &
ActionCable.

Basically generate a uuid to use as a channel identifier, then pass that uuid
to the front end to subscribe to. Pass the uuid to your ActionCable background
job to do the expensive rendering in a background queue and push the
information over the websocket channel. Have the JS insert it appropriately
(maybe the contents of the element with an id of the uuid).

This would have the benefit of displacing the expensive computation in a
background job, which wouldn't impact other web requests.

------
ewalk153
I would use a technique like this sparingly. It looks to be the "web request"
equivalent of an n+1 query.

Imagine you have a table of results and each one is taking time to render. You
add this for each row, but you only test it locally with a few rows. In
production, maybe that table has hundreds of rows.

You've just DOSed your server.

~~~
Colex
Browsers usually limit the number of requests to a certain domain (around 8
requests if I'm not mistaken, but it may vary). So I don't think it'd be the
cause of a self inflicted DOS. Also, every solution must be used carefully
where it makes sense, if it doesn't make sense, then a different solutions
must be thought of.

~~~
mtarnovan
That's incorrect. Browsers limit request to 8 req/domain _at the same time_.

~~~
mtarnovan
Don't know why I was downvoted. For reference:
[https://en.wikipedia.org/wiki/HTTP_persistent_connection#Use...](https://en.wikipedia.org/wiki/HTTP_persistent_connection#Use_in_web_browsers)

The limit applies only to persistent connections.

------
mendelk
If you're interested in these types of improvements, you'll love
intercooler[0]! It's basically this, with lots of more options and patterns.
It's been great for me in the (small-ish) projects I've used it.

[0] [http://intercoolerjs.org/](http://intercoolerjs.org/)

------
juliand
It's good to see useful libraries still being created for Rails. I plan to use
this on one of my projects. Thank you.

------
Dangeranger
Is it possible to configure render_async to make the request and DOM
manipulation without JQuery? Rails 5.1+ will not continue to bundle JQuery
with UJS anymore since UJS has been re-written in vanilla Javascript.

If it were possible, I'd find this feature more useful for smaller Rails apps
that are in the middle ground between server rendered HTML and full blown SPA.

------
nateberkopec
Does this work with HTTP caching?

i.e. if I use this to render a movie rating asynchronously, can I return a 304
not modified response and the client will insert a previously-delivered
response fragment?

~~~
nyargh
Yes - assuming the underlying request uses a GET verb, you can avoid a trip to
the server altogether if your cache headers were set aggressively enough on
the response to the initial request.

------
raman162
I think this with a combination of caching can be very powerful.

------
jbverschoor
why not use fibers or something to do db calls async and then join them at
render. Saves you all the network calls

------
JangoSteve
For their example, you could also just do the database lookups asynchronously
in the controller, which is good performance practice for queries that don't
absolutely need each other. For example, similarly to their example, let's say
you start with something like movies, which each have many movie ratings, and
your controller looks like this:

    
    
        @movie = Movie.find(params[:id])
        @ratings = @movie.movie_ratings # or equivalently, @ratings = MovieRating.where(movie_id: @movie.id)
        render
    

The second line above needs to do the movie lookup, in order to get its
ratings; and the second line basically constructs a SQL query that looks up
records from the movie_ratings table using the movie_id foreign key that joins
them. However, you already have the movie_id before you look up the movie,
because it's in params[:id]. So, you could instead do something like this:

    
    
        @movie = Movie.find(params[:id])
        @ratings = MovieRating.where(movie_id: params[:id[)
        render
    

However, Ruby being synchronous by default, the above will still do the same
thing and take the same amount of time, since line 2 will synchronously wait
for line 1 to finish. But now that you've disentangled the query for line 2
with the results from line 1, you can now explicitly do them asynchronously:

    
    
        [].tap do |threads|
          threads << Thread.new do
            @movie = Movie.find(params[:id])
          end
          threads << Thread.new do
            @ratings = MovieRating.where(movie_id: params[:id])
          end
        end.each(&:join)
        render
    

I've had projects where I abstracted the above logic to reusable methods like:

    
    
        def asynchronously(&block)
          @async_actions ||= []
          @async_actions << Thread.new do
            block.call
          end
        end
    
        def synchronize
          @async_actions.each(&:join)
        end
    

And then in my controller actions, the previous code becomes:

    
    
        asynchronously do
          @movie = Movie.find(params[:id])
        end
        asynchronously do
          @ratings = MovieRating.where(movie_id: params[:id])
        end
        
        synchronize
        render
    

Or if that still seems verbose, since each block above is a simple one-liner,
you could use alternate syntax like:

    
    
        asynchronously { @movie = Movie.find(params[:id]) }
        asynchronously { @ratings = MovieRating.where(movie_id: params[:id]) }
        synchronize
        render
    

The thing you have to be careful about when doing direct queries, like above
where both queries directly use params[:id], is when you have a situation
where authorization to access the data plays a role. For example, if you
replace movie with user, and ratings with purchases, now you can't just
blindly look up purchases by the user id the untrusted request passed in.
You'd want to first validate that the params[:id] being passed is valid for
the current user sending the request.

You also want to make sure you understand thread safety and the fact that
@movie and @ratings above are not contained to their threads (which we're
actually using to our advantage here). But in the above example, you replacing
a single thread where everything sees everything anyway, with multiple threads
where everything still sees everything, so you're not really changing that
aspect in the example above anyway.

~~~
lorenzk
Bur this would still only start rendering things for the user after the slow
lookup is done, so it would not help in this situation.

~~~
JangoSteve
Yup, I guess I should clarify. This solves half the problem without a gem,
which may or may not be good enough depending on one's situation.

------
voidlogic
So this is an alternative of designing your app FE to use AJAX or using ESI
(edge side includes) right? With the caveat this is super rails specific?

------
alttab
Why is this a gem? It's two view helpers and like 30 lines of Ruby code, if
that.

I've solved this problem the same way probably 5 times in rails apps and it's
super easy to build this interface in yourself without taking on another
dependency

~~~
gnaritas
> Why is this a gem?

> I've solved this problem the same way probably 5 times in rails apps

Because gems avoid such code duplication, they're the correct way to "share"
code between projects rather than reinventing the solution each time you need
it.

