
Python at Netflix - Lightning
http://techblog.netflix.com/2013/03/python-at-netflix.html
======
jedberg
As one of the main drivers of Python at Netflix, this one is near and dear to
my heart. Let me know if you have any questions, or come visit our booth!

~~~
tootie
I have to say that after doing a python project I totally don't get why people
love it so much. The syntax is slightly cleaner, but the lack of threading,
first-class functions and dependency management tools make it waste at least
as much time as it saves. I honestly don't believe in the concept of a
programming language being able to make code more or less readable. A Java
project with a POM file can be up and running on any OS in about 5 minutes.

~~~
frankwiles
Ummm, not sure where you're getting your information but Python has threads
(<http://docs.python.org/2/library/threading.html>), first-class functions
(<http://en.wikipedia.org/wiki/First-class_function>), and dependency
management tools (google for pip, easy_install, etc.)

~~~
tootie
Sorry, I should have said "lack of true threading in CPython" by which I mean
the GIL limitation. And lambda expressions are lame compared to what you can
do in JavaScript/Perl/Scala. And the build tools suck compared to Maven which
is saying a lot. Maven is cross-platform and install-free. It takes a long
time to makes head and tails of pip, easy_install and virtualenv and it still
does less than Maven.

~~~
SilasX
>And lambda expressions are lame compared to what you can do in
JavaScript/Perl/Scala.

Python lets you pass functions other ways than using lambdas, you know. If
you've defined a function with the standard "def foo" style, you can pass
"foo" as an argument.

Even the standard "1-line lambdas LOL" limitation can be circumvented by
defining the function elsewhere and making the lambda a call to it.

------
jkldotio
Python really does hit a sweet-spot for many types of operations. Devops
capacity in system scripting, crawling and scraping with excellent web
frameworks (Django through to Bottle). On top of all that there are great
machine learning libraries as well as good APIs to almost anything not written
in Python. My site gets huge amounts of NLP, machine learning and information
extraction done in just a few lines because of libraries available in python
that aren't in the languages that it's usually compared to.

I've also pretty much replaced bash with the ipython shell at this point, as
you can use all of the bash commands anyway and even seamlessly mix them with
python, file_list = !ls, it's a excellent environment for data analysis and
manipulation.

~~~
jedberg
You know, it never occurred to me to use iPython as my shell. I think that
_would_ make me more productive. I'll have to try that, thanks!

~~~
jkldotio
I have 'ipython --profile pysh' in my terminal's custom command, on servers
I'm a bit more cautious and have a 'pu' (power up, or python up) command which
is basically the first thing I type. You have to watch out for the shell
history not working quite the same and that newly installed programs won't be
seen until you do a '%rehashx'. Other than that it's pretty much all gravy.
Now I just need to get around to transitioning to that python tiling window
manager.

While I've got you, and seeing you are not longer at Reddit, what do you think
of jkl.io (launched today)?

------
netcraft
Re: Web applications in python - I am curious about the overall feeling for
them - Do you write web apps in python because rapid development makes up for
any performance problems and if you run into problems you would rewrite it in
something faster, or do you feel like a python web app can be just as good as
alternatives when it comes to speed?

Put another way - when you think of python for web applications / services -
do you think rapid development and speed, or just rapid development and you
would go somewhere else for speed?

~~~
jedberg
I like Python for the speed of development. Also, at least for these internal
apps, speed is generally of secondary concern, since these aren't typically
high load applications.

At reddit we solved the problem by writing the most often called parts in C
and using c extensions. There is definitely more work to do there, but overall
I think Python gets a bad rap as far as speed is concerned.

It's certainly not the quickest, but it isn't a dog either.

~~~
daeken
> At reddit we solved the problem by writing the most often called parts in C
> and using c extensions.

I'm wondering if anyone has solved these sorts of problems with Cython in the
real world. It looks like a great solution for a large set of problems in
terms of Python performance, since it _becomes_ a CPython extension, but I
never hear anyone talk about it. Can anyone chime in?

~~~
takluyver
I get the impression that Cython is used quite a bit in the scientific
computing side of Python, but not so much for web development. pandas & pyzmq
are two projects I know that make serious use of Cython.

~~~
jasongrout
Sage [1] (which you might say is the birthplace of Cython) makes a _very_
serious use of Cython as well.

[1] <http://sagemath.org>

------
voidlogic
As a Java/Python shop I think Netflix should evaluate Go after 1.1 RC comes
out in April. Go is very popular with people who have heavily used both Java
and Python (as well as C).

~~~
robertfw
Just curious, why would this be downvoted?

~~~
Afforess
If I had to guess, I would say because GP didn't actually give any real
reasons why adding complexity (a new language) would improve or solve a
problem. Advocating languages is fine, we all have our pet languages, but it
is important to remember they each are just one tool in the toolbox and that
what is more important is how the language solves the problem well, instead of
just blindly advocating one language over another.

~~~
voidlogic
Language comparison are so ridiculously multi-dimensional and my knowledge of
Netflix's internal applications are shallow enough I thought specifics would
be a waste of time. I see Go in general as a reaction to the overcomplexity of
Java/C++ and the performance issues of Python/Ruby. Since Go seems to be
popular with people who use Java/Python, and Netflix uses Java/Python, I was
curious if Go was on the table. I would hardly call my original post blind
advocacy...

~~~
sigzero
Some folks would consider this as an attempt to hijack the discussion. I am
not saying that is what you are trying to do but what some folks would think.
It gets tedious sometimes when the post is specifically about "Python at
whatever" and people post "Hey you should use this language instead."

Maybe that is why you were down voted. (FWIW, I did not down vote you.)

~~~
lclarkmichalek
I posted this: <http://news.ycombinator.com/item?id=5356913> today as a reply
to someone who commented about Go on a post about programming languages today.
It seems that (at least so far) I was correct.

~~~
belandrew
Besides Google using Go for some things like part of Youtube and
dl.google.com, CloudFlare, Heroku, Canonical, Torbit, SoundCloud, BBC, and
others are using Go for parts of their back-end infrastructures.

------
donutdan4114
Why does Netflix only allow 4-10 character passwords? My _weak_ password is 14
characters. There is no reason to have a length limit.

~~~
belandrew
My Netflix password is 20 characters. Where did you get that it only allows
4-10?

~~~
donutdan4114
Trying to update my password: <http://i.imgur.com/wLNpR0C.png>

------
mixmastamyk
Any remote jobs or must you live near Los Gatos?

~~~
jedberg
Generally we want people to be in Los Gatos, but there are exceptions, mostly
in the CDN team.

