Our site is over 5 years old, and if you've followed Justin.tv at all, you know it's been pivot city. All of those pivots have left their mark on the aging code base. It also turns out that many of our assumptions about how to build and scale a high traffic web app are no longer relevant. Servers are 10x faster, bandwidth is dirt cheap, and SSD's have entirely changed the database world.
There is nothing wrong with Rails, but everything else at Justin.tv is written in Python, so I wanted to use that. Switching languages has the benefit of forcing us to remove stuff we don't need anymore and implement the minimum set of features needed; cut and paste doesn't work across programming languages the last time I checked, and that's a good thing in this case.
Django has come a long way as a framework, to the point where it no longer "gets in the way". But the main reason I like it is that it's simple and it's python. All of our backed systems and video libraries are in python, so it will be nice to be able to share more code and leverage the brainpower of the non-ruby programmers at Justin.tv. There are many.
One thing I'd like to make clear- this isn't one of those massive, paralyzing rewrite projects. Justin.tv's website alone is not terribly complex. The scope of this is limited to porting only the templates still in use, porting over the minimum set of features, and removing as much cruft as possible. Our API, chat, video, data stores, and 99% of the infrastructure are untouched.
If you want to help build what will quickly be one of the top 5 django sites in the world, I'm hiring. Team is just two developers right now (everyone else works on TwitchTV) and we're looking for a lead developer.
However, Django is just so radically different in its approach than Rails. It may have the admin console baked in, but other than that I felt that Rails was far more "batteries included". It's nice that it leverages the Python ecosystem in many cases, but I found it frustrating to have to make so many decisions on what to use when Rails typically provided most everything I needed (minus auth and file uploading...but two gems quickly solves that).
I'm sure if I had learned Django first instead of Rails it would be a different story. As it stands, if switch to anything in the future it's likely to be Play! (http://www.playframework.org).
For me, the Django way of doing (almost) everything makes sense. Rails feels backwards. I also feel like I get more baked-in with Django, even when not factoring-in the admin. And its distinction of "Apps" vs Rails' "Gems" makes me feel like I have a lot more available from the community in terms of third-party plug-and-play code. Being able to have a minimum viable product without writing a single line of my own Django code because of apps in the community just doesn't exist for Rails as far as I could tell.
I have never hosted/written a django app, do people typically run a request in a thread in django? I thought python had a GIL just like Ruby (1.9+).
I wouldn't recommend using something like rails to stream video, however there are tons of great asynchronous ways to do this and it seems like the justin.tv guys are familiar with this. I would be curious to hear why the rewrite?
* youtube: http://highscalability.com/youtube-architecture
* itunes: I have a friend on the itunes team and from what i can garnish from him their java servers are single threaded and their boxes hum around 10% cpu usage however they are bound on memory (similar to rails as they are not using java threads for each request). Akamai serves most of the content as I understood it.
Depends on the handler, mod_wsgi allows for m:n (mixing threads and processes) for instance.
> I thought python had a GIL just like Ruby (1.9+).
That's correct, but any C IO (such as DB queries) will generally release the GIL, which allows other threads to run.
(technically, CPython has a GIL, Python-the-language does not mandate it and neither Jython nor IronPython use a GIL. Pypy also has a gil currently, but Armin has started exploring STM to see if it would be a good replacement)
Ditto with Ruby too.
I doubt just reusing the twisted code inside the django app makes much sense for that reason (nor does the other way round).
All it takes is for a single person on the team to be aware of the risk and to warn his colleagues.
I would also argue that Justin.tv has enough experienced engineers to avoid the SSE.
SSE is a kind of trap that preys on ignorant and inexperienced.
Both are dynamically typed and tremendously slow compared to the right .net or JVM stack. That would be my main criteria for stack selection in a project of this magnitude.
Marking a bug as "wontfix" because the developers of the RoR platform are arrogant enough to say, and I quote, "[...] figure out what other software is putting that version in your environment and stop it from doing so" is now called mature?
The dialogue on this bug alone was enough for me to put RoR in my rear-view mirror and not look back.
Which would you rather use: a tool which demands it is the only tool on your system which can set environment variables (RoR), or a tool which plays well with others (Django)?
I don't think you understand the context here. The Rails dev is saying (correctly) that nobody should be polluting the environment with a variable as vaguely named as "VERSION" this includes Rails.
Rails uses the "VERSION" environment variable temporarily for migrations. They're not trying to "claim" that variable name for themselves only.
Django is very strong as a rapid-development tool (forms, generic views, admin), but I am more comfortable with Rails for a project with a reasonable timeline. Django class-based views are a far cry from Rails controllers.
They just need to rename the environment variable into something like RAILS_VERSION instead, as suggested by several commenters in that issue. Such a simple fix, I don't understand why they won't do that.
1) VERSION is a horrible name for a non-transient environment variable. It's fine if you're just using it on a per command basis (as rails does with 'rake db:migrate VERSION=123') but some app has stuck a ridiculously generic word into a global namespace.
2) (trivial) RAILS_VERSION would make no sense because it's the migration number you want to set your database too.
Any non-transient variable that you need to set should be specific enough to not conflict with other apps using the same namespace.
If anyone should go fixing things it's the fool who permanently stuck a variable called VERSION in their users global environment hash.
I agree that there is a problem and that it is affecting rails users. But, if for example some whacko was running around slapping all rails users with large fish would it be the rails dev teams responsibility to stop him? No.
No development team should have to tailor their app to compensate for the stupidity of one ignorant developer who couldn't think about the ecosystem that we all have to share. The only exception to this is when the idiot developer got a job building your OS...
Django maybe, purely because it doesn't depend on too many C Libs.
Oh! Does the Python world have something like Rack?
WSGI 1.0, PEP-333: http://www.python.org/dev/peps/pep-0333/
WSGI 1.0.1, PEP-3333: http://www.python.org/dev/peps/pep-3333/
hope sick office is a typo, must be slick office
You mean "turned to".
"turn on" is either "sexually attracted to" or "suddenly attack"
Oh, the English language...
"My old roommate really turned me on to the Wire. What a great show."
Doing it wrong.