As a former Perl hacker who started using Python in 2005, I saw Python ride several waves. (Numerical computation, data science, deep learning)
Perl was the leading tool for scripting and text parsing. Python didn’t really supplant it for a long time — until people started writing more complicated scripts that had to be maintained. Perl reads like line noise after 6 months whereas I can look at Python code from 20 years ago, prettify it with black, and understand it.
Python got picked up by the scientific computing community, which gave it some its earliest libraries like numpy, f2py, scipy. Some of us who were on MATLAB moved over.
Then data science happened. Pandas built off the scientific computation foundations and eventually libraries like scikit and matplotlib (mimicking matlab’s plotting) came along.
Then tensorflow came along and built on the foundation of numerical libraries. PyTorch followed.
Other systems like Django came and made python popular for building database backed websites.
Suddenly there was momentum and today almost all numerical software have a python API — this includes proprietary stuff like CPLEX and what have you.
Python was the glue language that had the lowest barrier of entry. For instance, Spark was written in Scala and has a performant Scala API but everyone uses PySpark because it’s much more accessible, despite the interop cost.
The counterfactual to all this was Ruby. It had much nicer syntax than Python but when I tried to use it in grad school I was quickly stymied by the lack of numerical libraries. Ruby never found a niche outside of Rails and config management.
Essentially Python — like Nvidia today — bet on linear algebra (and more broadly on data processing) and won.
I get why there’s hate for Python — it’s not a perfect language. Yet those of us pragmatists who use it understand the trade offs. You trade off on the metal performance for programmer performance. You trade off packaging difficulties for something that works. You trade off an imperfect syntax for getting things done.
I could have used Ruby — a much more beautiful lanaguage — in grad school and worked around its lacks, but I would have not graduated on time. Python was pragmatic choice for me and continues to be one for me today (outside of situations requiring raw performance)
I agree with you, and I'll put it slightly stronger. Ruby is a better language than Python in every way except the very most important two:
- Imports in Ruby seriously suck compared to Python. Everything requires into a global scope and an ecosystem like bundler which encourages centralizing all imports for your entire codebase into one file.
- Python has docstrings encouraging in code documentation.
Add common ecosystem things like the Ruby community encouraging generated methods, magical "do what I mean" parameters, and REPL poke-driven development, and this leads to the effect that Python codebases are almost always well documented and easy to understand. You can tell where every symbol comes from, and you can usually find a documentation entry for every single method. It's not uncommon for a Ruby library, even a popular one, to be documented solely through a scattering of sparsely-explained examples with literally no real API documentation. Inheriting a long-lived Ruby project can be a serious ordeal just to discover where all the code that's running is running, why it's running, where things are preloaded into a builtin class, and with Rails and Railties, a Gem can auto insert behavior and Middleware just by existing, without ever being explicitly mentioned in any code or configs other than the Gemfile. It's an absolute headache.
My dream language would be Ruby with Python-style imports and docstrings.
I think your comment needs to mention that Python has syntax for type annotations and two mature type checkers (mypy and pyright) with more under development. Python is thus very much part of the modern statically typed languages scene (moreso than Go) whereas Ruby isn't at all. Many people wouldn't touch Python today if it weren't for this.
I’ve never quite understood how this works. Surely a type system is absolutely fundamental to a language - how can you have multiple incompatible ones?
Do you need to choose a particular type checker for each project? Are you limited to only using third-party libraries that use the same type checker?
I think Python was successful because it started off without a type system and you can still choose not to use it. Duck typing is the big feature really.
It might float your boat to think about types but why would everyone have to want the same thing?
Look at JavaScript and typescript - Python’s typing is maybe halfway to that gold standard but there were other typed languages based on js. Python is special in that it provides type hinting syntax which is not used by the interpreter, so writing types doesn’t require the Byzantine build systems of js.
Hard agree on the global Ruby import issues. I remember inspecting large custom Rails or Capistrano codebases in pry and having thousands of names imported. That and monkey patching had me wishing for Python with imports only having module scope and being a lot more explicit.
It's a shame Python has a strong anti-FP stance with crippled lambdas. And an OO system that looks like it has been bolted in, compared to Ruby which is essentially a Smalltalk with Perl-like syntax and some Lisp influence.
These two issues would have been quite easy to fix and would have led to a completely different development experience. Python had a good implementation with a nice C FFI (CPython) right from the beginning, whereas Ruby MRI had lots of efficiency issues with long-running computations. IMHO this is one of the reasons why Python won. Building a numerics stack on top of MRI did not look very promising.
> an OO system that looks like it has been bolted in, compared to Ruby
I think the two languages just have different design philosophies. In Python, functions are fundamental and classes are built on top of them. In Ruby, objects are fundamental and functions (i.e. Procs etc) are themselves objects.
You could just as well claim that in Ruby, functions look like they have been bolted in. For example, you can’t call a Proc itself but need to call one of its methods.
I agree. Python was designed in 1989 and it looks like the OOP we were doing in C (without the ++) back at the time. Objects were a struct with data and function pointers and we were passing them around as pointers. Python has self, explicit in function definition and implicit in function calls, and that self is really like the pointer to the struct. By the way, OO languages from the 90s (e.g. Java and Ruby) were designed to always hide that self, both in method definition and method call. They use it when there is a need to tell the difference between instance attributes and local variables with the same name.
Maybe the explicit self was there to make C programmers feel at home. Functions as fundamental building blocks of the language also make C programmers feel at home. Developers got more familiar with OOP by mid 90s so the new languages could jump from functions-first to objects-first.
I think Python's OO is a bit suboptimal even if the goal was to have method-centric OOP like in C with classes. For example, mechanisms to hide information, a fundamental part of the OO paradigm, are hacky. You need to use name mangling.
Same applies to FP, a few things are weird and crippled. IMHO, the net result is that Python code tends to look longer and much more algorithmic than in Ruby, Smalltalk or various Lisps, where the language favors lots of little functions that call each other.
Things are changing a bit, though. For example, pattern matching (PEP 622) brings some conciseness. Fixing those other issues would be great.
Pragmatic use of $LANGUAGE is a telltale sign of the wizened programmer; one who understands the use-case and solution set well enough to know when the tool fits.
I wrote Ruby when I got started because it was the most accessible and the Rails learning content was top notch. Now I use python when I need more than a few `bash` pipes to accomplish anything, but if I were to solve a capital-P Problem, of course the tool often chooses the project after constraints.
There was also the major anti-wave of Python 3. But it has managed to pull through despite ending up with broken strings (RIP all old code that needs to deal with legacy-encoded data), probably because there was no viable replacement.
As someone that did the Perl to Python transition back in 2003, for UNIX scripting tasks, the way to do OOP with packages and blessed references was clunky, and having to always go back to the manuals for some clever programming tricks from team mates was tiresome, while Python provided something nicer, and I wasn't really into the sed/awk like features in Perl anyway.
However due to being a interpreted scripting language I never bothered to use Python for anything beyond OS scripting.
Perl was the leading tool for scripting and text parsing. Python didn’t really supplant it for a long time — until people started writing more complicated scripts that had to be maintained. Perl reads like line noise after 6 months whereas I can look at Python code from 20 years ago, prettify it with black, and understand it.
Python got picked up by the scientific computing community, which gave it some its earliest libraries like numpy, f2py, scipy. Some of us who were on MATLAB moved over.
Then data science happened. Pandas built off the scientific computation foundations and eventually libraries like scikit and matplotlib (mimicking matlab’s plotting) came along.
Then tensorflow came along and built on the foundation of numerical libraries. PyTorch followed.
Other systems like Django came and made python popular for building database backed websites.
Suddenly there was momentum and today almost all numerical software have a python API — this includes proprietary stuff like CPLEX and what have you.
Python was the glue language that had the lowest barrier of entry. For instance, Spark was written in Scala and has a performant Scala API but everyone uses PySpark because it’s much more accessible, despite the interop cost.
The counterfactual to all this was Ruby. It had much nicer syntax than Python but when I tried to use it in grad school I was quickly stymied by the lack of numerical libraries. Ruby never found a niche outside of Rails and config management.
Essentially Python — like Nvidia today — bet on linear algebra (and more broadly on data processing) and won.
I get why there’s hate for Python — it’s not a perfect language. Yet those of us pragmatists who use it understand the trade offs. You trade off on the metal performance for programmer performance. You trade off packaging difficulties for something that works. You trade off an imperfect syntax for getting things done.
I could have used Ruby — a much more beautiful lanaguage — in grad school and worked around its lacks, but I would have not graduated on time. Python was pragmatic choice for me and continues to be one for me today (outside of situations requiring raw performance)