Hacker News new | past | comments | ask | show | jobs | submit login
Learning from Twitter (ejohn.org)
209 points by flapjack on Jan 20, 2011 | hide | past | web | favorite | 36 comments

It's good to know that this situation will lead to improvements in jQuery.

However, I'm disappointed in how twitter, or more accurately, Dustin Diaz handled this situation. They identified some performance issues. Narrowed it down to a jQuery version but just left it there.

jQuery is open source so they could have done some investigating on their own and tried to narrow down what changed in that selector. Especially since this change was blogged about (as mentioned by John).

Not to mention various people helped out twitter by finding some pretty obvious flaws and made suggestions. Twitter has a lot of resources and I'm assuming a lot of talented Javascript engineers. This is a situation where they could have used their resources to contribute to an open source project instead of the other way around.

I'm on the jQuery core team. As you might guess, when a high-profile site seems to be having troubles with jQuery, we want to fix it whether it involves a bug fix or tutorial help.

It would really help us out if people could slip our pre-release versions into their test environments. There's a 1.5 beta available now (http://blog.jquery.com/2011/01/14/jquery-1-5-beta-1-released...). Don't wait several months to try it the way Twitter waited on 1.4.4, or we may ship 1.5 with some bug that impacts your site.

Deployed on production already (http://cdn.mybucket.co/static/js/jquery-1.5b1.mini.js). I minify it using Crockford's jsmin script.

We happened to use http://www.infinite-scroll.com, which also bind to window scroll event as well. Let's see how it goes.

Thank you so much for all the hard work that goes into jQuery.

I don't think it's fair yet to say that Twitter won't do these things. The first step was fixing the regression as fast as possible.

And the second step was writing a highly-visible blog post slagging the free code they use in their many-million-dollar website before they actually understood the problem?


It's not hard to pinpoint the exact line of code that is causing the slow down and come to a conclusion. This could have easily been fixed without reverting to jQuery 1.4.2 and risk breaking other things in the process (assuming they upgraded to 1.4.4 for a good reason).

On top of that it was their own code which led to the slowdown. Passing the blame to jQuery seems wrong when it was their own mistake to disregard best practices.

Here is where Twitter failed:

1) Need better testing when updating a version of the library.

2) Need to understand what the library you are using is doing and how to avoid performance bottlenecks.

Basic stuff.

Well there's not much else to do. It seems other people did most of the work for them.

I use a generic version of the waiting-for-pause trick:

    // execute callback only after a pause in user input; the function returned
    // can be used to handle an event type that tightly repeats (such as typing
    // or scrolling events); it will execute the callback only if the given timout
    // period has passed since the last time the same event fired
    function createOnPause(callback, timeout, _this) {
        return function(e) {
            var _that = this;
            if (arguments.callee.timer)
            arguments.callee.timer = setTimeout(function() { 
                callback.call(_this || _that, e);
            }, timeout);
Use it like this:

    document.addEventListener('scroll', createOnPause(function(e) {
        // do something interesting here
    }, 1500, this), false);
Edit: Oops! The version in the article runs some code if any scroll event occurred since it last fired, but this one doesn't execute the callback until the user stops scrolling for some amount of time. Anyway, maybe it's still useful for some folks.

For anyone that's interested, two related techniques are called "throttling" and "debouncing".

Both of which are available in the awesome Underscore.js library: http://documentcloud.github.com/underscore/

I've used that idiom too. It's a great trick. One potential issue is that if you continuously trigger the scroll/resize event, your callback will go an extended time without firing. That's usually only a problem if you're using it to adjust UI elements on scroll/resize, in which case you'd want to add a timer to force the callback if the delay has been too long despite the event still firing often.

Why isn't John a billionaire already?

Heh, while I assume that this was mostly rhetorical and largely tongue-in-cheek it is a question that I've been pondering lately. (Or, more accurately, thinking about what work excites me.)

I've been working on jQuery full-time now for about 1.5 years (sponsored by my employer, Mozilla). Prior to that I worked on jQuery in my spare time for about 4 years while doing other things (I was in college, then in Y Combinator - Summer 2006, and then worked for Mozilla as a JavaScript Evangelist).

I've managed to pigeon-hole myself into the position of a JavaScript framework maintainer rather solidly (for better or worse).

That being said I really enjoy what I do and have trouble thinking of something that I would rather do as my day job. Being able to work on a piece of code that I've largely written from scratch and promote it to a community of eager users is my dream. I've often thought about making a leap into something else but struggle to think of something that would hold my interest as deeply.

I'm largely not interested in starting or joining a business. For whatever reason I frequently find that attaching a profit motive to work largely impedes my ability to be happy and seems to run contrary to my overall goals (educating people, giving good tools to developers -- and providing both to anyone and everyone, no restriction).

Perhaps I'll find some interesting project some day that will open the financial floodgates (or perhaps I'll enter a situation -- such as having children -- that will necessitate me earning more than I do now) but I'm quite content now where I am, doing what I'm doing.

All this being said, I'm constantly looking at and exploring new things. At the moment though most of that is outside the realm of JavaScript, or even computers, and mostly relating to art. It's unclear to me if that's something that'll hold my long-term interest (although it has for the past few years) but we shall see.

In my life, discovering jQuery is what made front-end development fun again. I think we're incredibly fortunate to have people like you helping us build software, and if it was possible to measure the aggregate impact of your work it would be staggering..

Same. Besides being a master of javascript, Resig also has a unique talent for API design which separates jQuery from competitors.

I didn't know you were part of a YC company. Which one was it?

I think it was JumpChat, an app for mass messaging by combining text, email and IM.

That's correct! We started work on it in early 2006 and moved to Boston/Cambridge in June 2006 to participate in the Y Combinator program (which was a lot of fun, as many have said). We had a monumentally hard time trying to get people to fund our idea. I remember having a meeting with a potential angel investor (in Boston) that said, after our full deck was done, "Well, that sounds cool, but I'll have to ask my kids to see what they think." That was pretty typical of the Boston investing scene, with regards to mobile in 2006.

It was around that time that Twitter came out (which we poo-pooed due to its lack of features). It wasn't until a few months later (and after we had run out of money and had to get real jobs) that we realized that Twitter was a much better solution for what we had wanted to achieve.

It's not a loss, though. We had a great experience and all ended up getting great jobs doing things that we love. I went on to work at Mozilla. Julia went on to work at a few startups and is now starting a pretty cool video dating site: Inlu.st, and Josh is working at Blue State Digital (he did work for the Obama campaign).

So like wuph?

Pretty much, yeah! When that episode of The Office aired we all emailed each other - it was pretty hilarious. Of course ours was a bit more pragmatic, it would only send to the medium that would best get in contact with you - and you could use it to bulk send messages to your friends. A tool like that doesn't really need to exist in this day-and-age as everyone always has their cell phone and modern cell phones are super-capable.

For reference, the clip from The Office: http://www.youtube.com/watch?v=ytc9-wGCHW0

That's awesome. Facebook seems to be trying to do what you did with their new messaging features. I'm sure it's totally different in some fundamental way but interesting that they went there.

I'd love to hear more about what art you are involved in (besides jQuery of course!).

Unfortunately I've never had any formal training in art (either through technique or through study) - and it wasn't until after college that I realized that this was a passion of mine - so it's been pretty slow going. I've been taking a number of classes and practicing on my own, starting to become more familiar with oil painting and generally trying to improve my technique and composition.

However where my real passion lies is in art history. Specifically I've been spending a considerable amount of time studying, researching, and collecting Ukiyo-e (Edo-era Japanese woodblock prints). It's an incredible art form and to study it is to study the complete culture of Japan in the 17th to 19th century. I'm lucky as I live less than a mile away from the Harvard art library and have a friend that works as a librarian there - so I end up spending a lot of time checking out research material, studying, and learning. (For example, right now I'm reading a book on Kabuki theater and its relevance within the context of Ukiyo-e.)

I've started to write a little bit about the art form (as my time allows, which is kind of challenging) here: http://ukiyoe.tumblr.com/

I also run the Ukiyo-e sub-Reddit: http://www.reddit.com/r/ukiyoe/

I hope to do more research and writing in this area during the upcoming months, time permitting of course!

jQuery makes things almost too easy. As a result it gives confidence and power to a lot of people to write code that despite all sillines will still work. A professional JS developer would've debounced the scroll events and cached the selector results.

jQuery, while used & loved by my team, has made hiring vastly more difficult.

jQuery promotes bad practices. People assume $('something') is a cheap variable access, when in fact it could be pretty slowly executing several functions. So they don't bother caching anything.

I think by "promote bad practices" you might mean "abstracts complexity away from the developer".

This is true of just about any API. Nobody blames the API when their ORM layer generates a massive SQL query based on complex query input. Understanding the performance characteristics of the 'backend' is assumed.

  $FOO in php is cheap.
  $('FOO') in jquery is expensive.
I don't think using $ to signify "Run some load of jquery code" was a wise choice. Newcomers who don't know what is going on under the hood will regularly write loops with $('FOO') inside them, which is crappy code.

They're doing what they think is best, so I don't blame them. I blame jquery for making it look cheap.

Why on Earth you assume that someone starting to use jQuery will come from PHP background, and more over, why should that person think that $FOO in PHP is cheap? I guess anyone thinking in terms of performance cost will be smart enough to understand what $() does in jQuery, especially when advice to cache selectors is on every wall.

idle in #javascript for a while. It comes up all the time.

This sort of thing is one reason I hate websites which use a lot of Javascript when they don't need to. Another reason is that it breaks expectations of how websites should work.

To be fair, the expectation of how websites work is changing. Lots of small touches in applications aren't strictly necessary(animation, transparency), but they do make for an overall nicer experience. People have come to expect these things from modern websites.

> Lots of small touches in applications aren't strictly necessary(animation, transparency), but they do make for an overall nicer experience.

I can't offhand think of a single example where they've improved my experience. (With the exception of things like Google Maps where the JS is an important part of the user interface.) When, for example, a menu or picture animates open instead of appearing instantly, at best I feel mild irritation.

Maybe I'm just a cumudgeon.

>since scrolling itself didn't change the DOM

Scrolling did indirectly change the DOM as you load more tweets when you scroll. But the point here is that the actual scrolling is not changing the DOM, the append is, and that's probably when you should update your query cache.

I interpreted this as meaning that not all scrolling changed the DOM, only scrolling that reached the bottom of the window.

What in the world. I expected something interesting, but this is some major fail by twitter. Not caching selectors just boggles my mind. How can a million dollar web company do this?

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact