Hacker News new | past | comments | ask | show | jobs | submit login
Python leads Tiobe Index for first time in 20 years (tiobe.com)
43 points by ericholscher on Oct 8, 2021 | hide | past | favorite | 88 comments



Turns out being No. 2 language for all domains leads to being No. 1

What is fascinating that Classic Visual Basic jumped from 19 to 11. TIOBE is like that

I've been teaching Python for the last few years and there has been a marked increase in using Python to "Automate the boring stuff".

Adult students come to tell me that Pascal or C++ taught in school was just not something they could use practically, however Python is very practical across many disciplines.

Of course if we could solve the deployment issue on a fresh computer...


Python feature and library bloat is at late 1990s and early 2000s levels Perl these days.

This has made Python very accessible and useful to Perl refugees.

This explains the pervasive use of Python to "Automate the boring stuff".


Remember folks, the Tiobe Index is a ranking of how many web searches are being done for a language. The farther you get from that metric the more you are speculating.


There is also:

https://pypl.github.io/PYPL.html

Which ranks languages based on number of tutorials searched for (via Google trends). It also has python at number 1.

I lucked out and learnt python 10 years ago so feel "ahead of the game". It's a language that feels so malleable...

Thre only negative is that type annotations aren't part of the core language. You have to run a separate tool (mypy) to check your types. Feels like it should be a flag on CPython or something.


Are we going to pretend that there are really more searches for C and Visual Basic than for Javascript?

I don’t know what they’re measuring exactly and how but on the surface it doesn’t pass the smell test.

edit: Here's the actual method - https://www.tiobe.com/tiobe-index/programming-languages-defi.... It doesn't count searches.


> Are we going to pretend that there are really more searches for C and Visual Basic than for Javascript?

Probably. A lot of things I would add the language name to while searching for another language I qiickly learned tagging MDN on instead gets the best JS info, and once on MDN, navigating between topics rather than doing a new search for a related topic os much more common than the haphazard set of resources that you end up with for many languages.

Of course, that's a minor reason among the vast number of reasons why number of web searches explicitly targeting the language is a horrible measure of language popularity or impact.


They don't even count searches (i.e. traffic), they count hits from searches in a number of popular engines - https://www.tiobe.com/tiobe-index/programming-languages-defi...

> The ratings are calculated by counting hits of the most popular search engines. The search query that is used is +"<language> programming". The number of hits determines the ratings of a language.


Whatever they do TIOBE Index methodology is flawed. MATLAB at #13 ahead of Ruby #16 and Swift #17 makes no sense. MATLAB is a niche language popular mostly among Electrical and Mechanical Engineers while Ruby and Swift are general purpose and have much larger user and project base.


I'm more inclined to believe Collabedit's stats as a measure of language use in indsutry since it's commonly used for online interviews.

http://collabedit.com/programming-language-stats


Isn't it how many results there are for searching "LANGUAGE_NAME programming"? Not how many people searched for it. Or maybe I am mixing it up...


It actually counts number of search results. I think the intent was to use number of web pages as a proxy to measure popularity of languages, but since large chunks of the web are behind walled gardens (e.g. social networks that are invisible to bots), this metric might not be as relevant as it used to be.


Tiobe is great. Whenever I see someone taking those rankings at face value, I get wary of their ability or general motivation to check their sources.


I find that it is a good proxy for how much each language is being used.


You may want to also consider other data points like the annual Stack Overflow Developer Survey [0]. Again, it's not necessarily the perfect stat reflecting actual real-world language use (since it's really more about languages used by visitors to SO), but I'd say it's a pretty decent proxy.

[0]: https://insights.stackoverflow.com/survey/2021#technology


How do you find that?


It seems clear that the more a language is used, the more a programmer will makes searches. Not a linear relation, but it is common sense.


It seems clear that the worse the documentation for a language is, the more a programmer will make searches. Not a linear relation, but it is common sense.

It seems clear that the harder a language is to understand, the more a programmer will make searches. Not a linear relation, but it is common sense.

It seems clear that [reason somebody would do a search] ...


So Javascript, SQL and PHP hover around at 2.5%. This index does not seem to be very meaningful. More akin to "confirmed by Netcraft" (a Slashdottism).


Python has one giant plus over pretty much every other popular language:

    The syntax
I was reluctant to to meaningful whitespace for a long time. But after doing some projects in Python I have to admit the code looks so much better than C style languages with all the {, } and ;s

And it has one giant plus over PHP:

    The module system
PHP went down the wrong road (of insanity) by deciding that the way to avoid naming conflicts is to use longer classnames. Because that is what namespaces in PHP are. Comfort tools to make your class names longer.

There is still one giant benefit of PHP over Python:

    Performance
Currently, PHP is about 6x faster than Python.

Let's hope that the Python devs will focus on performance so we get the best of all worlds at one point!


You can avoid curly braces and semicolons and also not have indentation determine the control flow. Julia, Fortran, Visual Basic, and other languages do this.


None of these 3 is in the top 10.

Personally, I would only consider languages very close in popularity to the leading one when deciding on the language for a new project.


Visual Basic and Classic Visual Basic are in spots 6 and 11. I am not against Python's use of indentation, but I don't think it's better than having keywords such as endif or next terminate blocks.


Wow, you are right.

Still, I don't think I would choose Visual Basic to build the next big thing though :)


Yes, and they use keywords, wich is basically another flavor of curly braces.


The only negative is that type annotations aren't part of the core language. You have to run a separate tool (mypy) to check your types. Feels like it should be a flag on CPython or something.


Hmm, PHP developers were already making long class names prior to namespaces existing, for example the "PSR-0" underscore-delimited naming convention.

The namespace feature is more of a comfort tool to make those long names shorter.


I don't know, I think the syntax is ok but it definitely has significant downsides, like making lambda functions stupidly hard.

Python's competition isn't PHP; it's Typescript.


Typescript is position #46 on the Tiobe index.


But anyone using typescript would just be Googling JavaScript syntax. Rarely is there anything Typescript-specific you need to look up while writing TS code.


Why do people care about so much lambda? If you want a function, use a real one. Python makes it easy to create throwaways everywhere. The only thing it enforces is proper indentation. What advantage should a lambda give you?


Naming things, especially functions that you tend to use in lambdas, is often hard. Imagine if you had to name a for loop to us it. That's how it feels.


Then don't do it. Use a generic name, like f or l2. Reusing names is possible anyway, so you can name all the onetime-functions the same.


I'm not saying it's impossible, I'm saying it's annoying and I prefer languages where it's not.


It lets you do functional style programming easily. This sort of thing:

  [1, 2, 3, 4].map(x => x * x).filter(x => x < 10)
Python's lambdas and list comprehensions are quite restrictive in comparison to functional style.


Huh? This seems like it works fine in python?

    filter(lambda x: x < 10, map(lambda x: x * x, [1, 2, 3, 4]))
But even for more complicated logic, it also doesn't seem like it's that burdensome to just throw a name on the lambda:

    def square(x):
        return x * x
    
    def less_than_ten(x):
        return x < 10
    
    filter(less_than_ten, map(square, [1, 2, 3, 4]))
Which I find to be more readable/maintainable.

That's probably the most underrated feature of python. Decisions about how the language should function like this which seem "restrictive" from one perspective, and yet force you down the avenue of good decision-making for readability/maintainability. Sure, it's possible to write terrible python, but you're generally cutting against the grain to do it.


Yes you can do simple examples like the one I gave, but when it gets more complex you have to do what you did and name the functions, which is pretty horrible when they're just one-offs. It also makes the code read in a weird order.


That looks like a Ruby one-liner which is very OO (calling methods with blocks on everything, everything also being an object is very Smalltalk-esque).

Python's problem isn't a lack of lambdas, it's a lack of consistency; for some things you use functions and for others methods.


I know what lambda is, I just don't see any usecase where the limitation is a problem. If your lambda is so complex that you need more than a oneliner, then what's big thing with using a nested function?


List comprehensions are actually set notation in disguise, so they’re quite powerful and compact. Haskell has them too.

You can express the above as:

[x * x for x in [1, 2, 3, 4] if x < 10]

Which in set notation is {x^2 : x in (1, 2, .., 4), x < 10}.


The list defined above is [1, 4, 9], but the one you just defined is [1, 4, 9, 16]. It's interesting how the shift in notation subtly confused the semantics of what's going on.


Good point. I missed that and made a mistake. It should have been:

[x * x for x in [1, 2, 3, 4] if x * x < 10]

There is a subtle shift in semantics. The functional form defines a sequence of function applications whereas the list comp notation is more of a declaration.


I think this would be the Python equivalent:

    x = [1, 2, 3, 4]
    x = map(lambda x: x*x, x)
    x = filter(lambda x: x < 10, x)
    x = list(x)
Or is there a more elegant way?


You can write this as a oneliner too. But the problem here is not lambda, but the lack of map/filter-methods on list. Disputable whether this is good or bad.


This, maybe

```py x = [xx for x in [1, 2, 3, 4] if xx < 10] ```


I thought what you wrote wasn't valid because xx isn't defined in the scope of the inequality... however I now realize that's because Hacker News deletes the asterisks. In any case, it's

[x [asterisk] x for x in [1,2,3,4] if x[asterisk] x < 10]

which is still a little annoying because the x [asterisk] x gets computed twice. This is one of those little details that really annoys mathematicians, as I saw with SageMath for decades (I started SageMath -- which uses Python -- and got no end of complaints about the above). In the Magma programming language there is a "where" version of comprehensions that eloquently solves this problem as explained here: http://magma.maths.usyd.edu.au/magma/handbook/text/10

For example,

{ xx : x in [1 .. 4] | xx lt 10 where xx is x*x}

which you can try at http://magma.maths.usyd.edu.au/calc/


> however I now realize that’s because Hacker News deletes the asterisks

You can escape asterisks with a backslash. \* => *


from functools import partial

from itertools import repeat

from operator import pow,gt

x = list(filter(partial(gt,10), map(pow, [1, 2, 3, 4], repeat(2))))

(please don't do this)


Capturing values, which isn't really obvious with functions.


what's so hard about

    >>> (lambda x: x+1 if \
    ... True \
    ... else x+x)\
    ... (10)
    11
:p


Now try including if statements, loops, structural pattern matches, context managers, etc.


If it's too complicated for a lambda, you shouldn't be using a lambda. Define a separate function if it takes you more than a line or it's harder to reason about.


The point is it's only complicated in Python. In other languages, like Rust, it's idiomatic.


People all know lambdas are expressions only, it's not a biggie really. In all honesty you rarely inline complex lambdas with all you mentioned, it would become hard to read or debug.


> Currently, PHP is about 6x faster than Python.

In a 100KLOC codebase, 99% of the performance issues are going to come down to 5 or 10 lines of code.

Don't get me wrong, I'm extremely happy that MS is funding a multi-year project to improve Python performance, and I think it will confer enormous benefits to the ecosystem. And frankly I think that it's absolutely needed.

But for me it would also be weird to choose languages based on some notion of average performance. Like sure, a line of Go might be faster than a line of Python on average, but if the entire bottleneck of your codebase revolves around one line of regex then what is doing a rewrite actually going to accomplish?


I don't think that holds true.

80/20 rule? Probably. Then 80% of CPU time is spent in 20K lines of code.

But 1%? I don't think so.

A web project of 100KLOC will be a shit ton of features and CPU time will be spent in hundreds of different places.

Another issue is what I would call "code depth". Even if you find a few lines deep down in a framework or library that makes your specific use case slow - what can you do? Patch the framework? That comes with a ton of issues on its own.


one can argue about how much of an exact science Tiobe or other indices are but Python definitely is everywhere and has continued to grow in a lot of domains. And I think deservedly so because to me it is by far the most pleasant and versatile general purpose language around. People have their legitimate gripes about performance but a language that is used for everything from scripting to gigantic industy scale applications (largest I'm aware of is a ~30 million LOC codebase at JPMorgan) has done some things right.


30MKLOC in Python? Largest I've seen is 500K and that was one overcooked codebase. So problematic. They were looking at a Java rewrite. I'd love to read a lengthy insider exposé on that JPM codebase.


I was curious about the LOC sizes of some well-known packages:

   % cd Python-3.9.1
   % wc -l `find . -name '*.py'` | tail -1
     843196 total
   % cd ~/clones/numpy
   % wc -l `find . -name '*.py'` | tail -1
     256900 total
   % cd ~/clones/scipy
   % wc -l `find . -name '*.py'` | tail -1
     397250 total
   % cd ~/clones/django
   % wc -l `find . -name '*.py'` | tail -1
     383167 total
That's not quite 2MLOC.


Assembly ahead of Ruby? That doesn't seem right.


And VB is ahead of Javascript. Tiobe is useless as a measurement of anything.


How do you know? Maybe there's an endless army of programmers supporting ancient one off VB programs written for various odd tasks across many organizations.


Tiobe's methodology sucks and their results are equally bad.


Not quite endless, but plenty of programmers out there grinding it out in 'unfashionable' languages.


The index reads more like a list of 'Programming languages that university students search for on Google'.

VB in the top 10? Matlab, R and Fortran so high?

Kotlin and Dart in the 30's? And JS not top 3?


This is nice, but not really much of a surprise.

With tons of ready-to-use libraries for string processing, machine learning, images, graphics, game development, web, etc., there is not much to dislike about it. It is my goto language for quick scripts -- works flawlessly on Windows/Linux/Mac.

The only downside is efficiency and being one-threaded, but if you are gluing together library calls, which you are most of the time, then it is good enough.


I disagree. Python's popularity is actually kind of tragic. The package ecosystem leads many to choose Python, which has both maintenance and performance scalability issues (not to mention the sorry state of packaging, distribution, and parallelism). So it causes plenty of headaches, despite its friendly syntax. In those senses, choosing Python is like a trap.


I absolutely agree, but this sort of tragedy is a bit unavoidable. What language would you choose to have that ecosystem instead? The next thing right now is probably JS which is even worse. Every language has plenty of issues. Python's worst IMO is how C extensions and other stuff are tied to an interpreter definitely unsuited for it's popularity. Parallelism shouldn't really creep in considering how most code is written. The only language I can think of being better is Lisp if somehow the lisp curse wasn't a thing.


JS has at least good package management and distribution compared to Python.


Kind of. Python definitely has more legacy cruft, but also has an npm equivalent PEP 582 with a working example, pdm. I don't see how JS is better unless you include web browsers as distribution. It's mostly people thinking bare `pip install` is a reasonable way to add various applications to your system that complain about python packaging.

I'm not saying Python packaging doesn't suck, it definitely does. But so does JS and the mess that the npm ecosystem is makes working with it it much more painful.


> Python definitely has more legacy cruft, but also has an npm equivalent PEP 582 with a working example, pdm

That's part of the problem. Python has lots of things that cover lots of things, but nothing does everything.

> It's mostly people thinking bare `pip install` is a reasonable way to add various applications to your system that complain about python packaging.

It is reasonable in JS, and I think it should be. Having to worry about virtual environments is a pain, and having to learn new tools to avoid that is a pain too.

> I'm not saying Python packaging doesn't suck, it definitely does. But so does JS and the mess that the npm ecosystem is makes working with it it much more painful.

I don't think that's a good point. The JS ecosystem adapted quickly to TypeScript, while Python is lagging behind. Most of the new JS features are reasonable, and the backwards compatibility is top-notch. Meanwhile, Python is spending brain power on pattern matching and :=. JS is missing something like Django, but on the other hand Python has no React equivalent. All of that without mentioning speed or the fact that JS has multiple implementations that works very well with almost all the ecosystem.


>It is reasonable in JS, and I think it should be

It's not. Even if npm install -g happened to work for you, it's insane to rely on it. These language package managers have a purpose in isolated environments but are incapable of building reliable, modifiable and updatable systems on their own. It's a good thing pip fails quickly so people don't attempt this.


Why is it "insane" to rely on it? It works well on widely used distributions. The only thing missing from node is a version manager like nvm.


I've always wished that Ruby had the place of Python. For all the love that Python syntax gets, Ruby syntax and the gem system are an absolute joy to work with.


Huh, I've always found dealing with gems/bundler to be a pain point.

I'm sure to some degree it's just that nobody likes whichever package system is the "one more" in their workflow.


python is built on solid fundamentals, ruby less so.

I used to love ruby, but their inner workings seem more ad hoc if you look under the hood (AST and broken operator logic)


> The package ecosystem leads many to choose Python, which has both maintenance and performance scalability issues

My guess is that easily 95% of Python projects will never hit these issues. In all my years of professional Python programming we didn't hit them.

Most projects simply do not need to scale, and will not benefit from it.


I think it definitely depends on where you are and what you're doing. In 3 of my 4 years of professional Python experience, we have run up against scaling, maintenance, and parallelism issues constantly.


For large projects, sure. But most uses are small one offs.


Well it's basically Visual Basic for adacemics and it excels at that. Aside from it's obvious problem: Python is a very practical, though almost generic language, with a rich collection of useful libraries -- so why not?


It's also the better shellscript for administrators. And user-friendly, like basic. It excels on many fronts. The surprise is more that it took so long to reach that point, considering how old and widely used it is.


That's a great point! It was invented somewhere in the late 90s, no? So maybe it was a confluence of alot of things that broke it out finally.


1990.


Wow, nice! I didn't know it went back THAT far. Pretty cool actually.


There is no large tech company heavily promoting it.


Nowadays, I guess they are promoting it the area of ML, with libraries such as TF and PyTorch.


That's its biggest strength.


Looks like they only started counting Visual Basic in 2010 which accounts for it's steep rise it seems. But it's obviously very popular otherwise.


even if tiobe is not solid, I sense a bit cover effect for python




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: