As a huge fan of Python, my go-to statement is that Python is the second or third best language for whatever you want to do.
Want to do web server development? Flask, Django, Tornado, etc... can do that. Probably not as concurrently as Go, but it'll do.
Scientific computing? Its no Matlab, but the syntax is easy and the time you lose in processing you make up for when developing.
Desktop computing? PyQT is pretty good. Electron may be faster to build with, C will be faster at runtime, but Python, even with Tkinter, is ok.
Functional programming? There are list comprehensions, map, reduce and filter. Not as good as Haskell or Lisp, but it'll do.
Python isn't the killer language for any of these really, but its good enough in almost all cases that you'll be very productive and can iterate to a good MVP quicker than with most other languages.
Any language with static types should be easier to make sense of, though. I get it's faster to write dynamically typed code, but your point is specifically about reading code, which I can't follow.
Of course Python nowadays has support for static types too.
I program in Python professionally, and I agree with your sentiment in general. Dynamic types hurt readability. Also, in general, Python is faster to develop with than most statically typed languages. There are two nits that I’d like to pick: the first and lesser is that Python’s static typing story is in an alpha stage. It’s not ready for prime time (but I’m cheering it on) and Python shouldn’t be considered as having a static type system yet.
The second is that I realized that Go is a statically typed language that is quite a lot easier to develop with than Python and other dynamic languages. It strikes a good balance between protecting you from mistakes while not making it hard to express your intent (most of the time, anyway). As I advance and become more concerned about my time, I find myself reaching for Go more often because the static type system has my back. I spend less time writing unit tests or iteratively running my code to make sure things still work. (People said this about Haskell and Rust and OCaml too, so I understand the skepticism dynamic language proponents have for claims like these). The deployment, dependency management, and performance stories are also quite a lot nicer in Go, and these are undervalued components of “developing velocity” as well, especially if you’re on a small team. It really changes the calculus for static vs dynamic language discussions.
> 35 years of programming and I've never met a language so well integrated with my mind.
That's a pretty good way to put it - and mirrors my feelings. I've been programming for 20+ years (with breaks here and there), and it took a while for me to find Python. But when I found it, it's like I finally found the language that my brain thinks in.
Personal anecdote as well, but I think you nailed one of the reasons I love Python.. It's the way the language was designed (PEP 20)
I had written some POC scientific code for my undergrad thesis, but the thesis was sidetracked for about a year or so due to real-life issues. When I went back, I could try and test things again in just a day or two, in comparison to much more detailed documented Matlab and C code.
- There should be one-- and preferably only one --obvious way to do it.
It's the killer language in scientific computing I'd say, having used both Python and Matlab. The open source nature/libraries and being a more fully-powered programming language (expressive object system) make it better.
As someone who does a lot of scientific computing in Python, I'd probably recommend learning Julia as the 'killer language' if scientific computing is the main reason you want to learn programming.
Pythons strength comes from being pretty good at scientific computing, and equally good at just about everything else.
Was going to say that Julia might be the long term best open source replacement for closed source scientific computing (Matlab, Mathematica...etc). In some ways it is already just as good, but in others, it has a long way to go. From a speed perspective, it is top notch.
Yea, PyCall makes it trivial to write a quick Julia wrapper around a python library. It's also really handy for porting python code to Julia. Start by wrapping all your existing functions with PyCall and then replace them one by one with Julia functions.
In the microcontroller it is good enough for hobby/proof-of-concept level things. Same as the various Javascript engines. Lua might actually be doable for some simple products.
But a lot of embedded these days is on Linux, and then Python is pretty great.
Languages have always come and gone; Python has always mattered, it's simply finally showing itself as a much more suited tool for certain tasks (such as CS education, lightweight scripting by data scientists, glueing together large frameworks, ...). While popular, it can't really hit every single spot required in software development; try to maintain > 10k LOC codebases in Python to understand what I mean.
The point that I'm trying to say is that, while we're living in a moment where python is at the center of everything, it won't last forever; it never does. Something better always shows up, given enough time.
Perhaps paradoxically it is easier to maintain a gigantic Python code base than it is to maintain one that is small or medium sized.
The reasoning, a gigantic code base means infrastructure and tooling investment will obviously pay off. It probably even means you get one or more dedicated resources of this sort. Whereas a small to medium sized codebase might bump into some of the same issues as a larger code base but not justify such investments.
If anyone else is looking: https://www.youtube.com/watch?v=ZYD9yyMh9Hk :) I'm definitely interested in this - I've seen a struggle to manage a far smaller codebase than that in Python and really interested to see what you're doing differently to make it work.
The most concise answer is 'tooling'. We have a DevTools team that builds the SDLC and automated testing infra, all tightly integrated into MS Visual Studio. That lets the thousands of developers focus on productive coding.
There isn't a ton of detailed public information on this kind of stuff. My talk focused on the general principles of our approach, rather than the myriad of features and optimizations necessary at such a large scale.
A good next step would be reading the 2016 ACM paper "Why Google Stores Billions of Lines of Code in a Single Repository" [1] then reading a recent HN discussion [2]. That has a ton of good links.
Is this made as a cluster of independent programs or is this actually a monolith, where you can access all functionality from some python method or class?
- Yes, it is a monolithic application architecture. There are distinct frameworks (e.g. GUIs, DBs, distributed compute, pubsub, lazily evaluated graph, reactive graphs, etc) designed to be used together in a unified way based on a global object store.
- A 150 person platform dev team writes these core frameworks and dev tools and runs the SDLC infra.
- Then the several thousand developers in business teams build applications for their particular needs.
- When the platform team sees businesses building their own solutions to a common need, it will productionize that as a new core framework for all business to use.
- In Python terms, there is a relative small number of core classes, and very many application-specific subclasses.
I used to work on the subsequent brain child of Kirat Singh et al at another big financial institution, and ditto. Admittedly we used a custom built source control system and release tooling, but also developed in Python.
The typical measure is “Source Lines of Code” (SLOC), which would be calculated on a per-language basis and doesn’t include comments or blank lines. You’d sum these together for your codebase overall. Not sure how you’d do it for HTML or mixed code. Presumably the same since it’s such a rough measure that that level of accuracy really doesn’t matter.
Not trying to split hairs, but we have no problem maintaining our 50K LOC python/django codebase.
What I like about Python is the ease of which you can layout a project to fit your needs.
Maintaining a large codebase is more about layout/architecture than the language itself.
Maybe other languages are simply better suited for programmers who lack the need of keeping their codebase clean and tidy :)
This is totally true. We are trying to maintain a codebase that is hundreds of thousands of lines of code, based on Drupal 5. It's a fucking nightmare.
I have worked on my fair share of Python monstrosities. To be fair I think I have seen worse in PHP, but I don't think the language itself is going to save you from shit architecture.
I used to use drupal and it was a nightmare. I switched to Django just because of python and it was the best decision I’ve ever made in regards to web development.
Sure, and that's true and if I were to set out to do a large service now I'd use something with typing. My last few net-new 10k+ loc projects were all typescript and I'll probably stick with that approach for a while.
The main issues with large codebases are often not feature-based but rather maintenance and operational overhead related: undiscoverability, config hell, dependency hell, and scaling issues - and typing helps a little bit with one of those.
With python, a good IDE should be able to find usages.
If you delete a method that has usages without searching for them, you're asking for trouble. Sure it's not a compilation error in python, but compilation can only show that the method is missing, not that changes are guaranteed to work.
When it comes to refactoring, there are so many automated, guaranteed safe refactors you can
do with statically typed language that you can’t do with dynamic language.
I can’t understand why developers would want to take a tool out of their tool belt like static checking.
I’m a fan of Python as a low overhead scripting language but for large systems with multiple developers, static type checking is a godsend.
Sorry for the bluntness of my question: I only have little experience in using Python, which I acquired while trying out Django, and I remember using scripts to generate certain parts of the site (things might have changed since then?).
To be fair, Django is pretty good: iirc, it induces good discipline for development, which is perhaps why it is so easy to handle a large codebase.
On the other hand, the business logic is outside the scope of Django (or any other framework), and there, if the devs do not follow good practices and develop the right tools, it might be more difficult to be productive.
The way I see it, people using dynamic languages have much more freedom in development, but also less safeguards, which means that they have to build them to make their code more robust. Having code generators is a way of getting back some safety, because you only have to make sure the generators are producing correct code.
Custom code, however, need to be thoroughly tested to be considered safe, and if the compiler doesn't do any type level consistency check, it's that many more tests that needs to be written, which adds up to the LOC count.
I frequently use code generation in python. why write thousands of lines of boilerplate code in any language when you can just generate it? Maybe I'm just lazy but the results are better and more consistent to get something done.
I utilize that too but why not generate new models with forms, cbvs, drf endpoints, serializers, etc? You can pick and choose what you need, add the mixins and move on.
Plus initial project using cookicutter to generate the original project in a much more fitting and featureful structure.
Honestly you are missing out, I've been developing in Django for years and can't imagine not generating code anymore. Generate and a bit of tweaking, the results are the same with a lot less time.
I'm sure you're right, but just for fun I'll make some counter-arguments.
All technologies follow an S-curve in their 'bang for buck' life cycle. Initial improvement and return on investment ramps up slowly, then it hits a middle era where all the low hanging fruit get picked for massive returns, then it enters a third stage where most of the easy benefits have been reaped and further improvement is incremental.
We see this with the internal combustion engine, jet engines, nuclear power, all sorts of thing. Eventually another technology might come along that supplants the old technology for many or most use cases.
The question I'm pondering is, are languages technologies in this respect, or are programming languages in general a technology? If the former then Python will top out in the benefits it provides and will almost certainly be supplanted by a successor. If the latter, maybe Python is near the top of the development curve (along with many other modern languages) and further improvements will be too incremental to lead to anything supplanting it?
But if that is the case, what technology might supplant programming languages?
Interesting observation! Throwing a couple ideas out there...
One possibility is visual languages that emphasize mistake proofing, interchangeable parts, and logic visualization over expressiveness. I’ve noticed an increasing number and use of visual languages (Blockly, Unreal Blueprints, NoFlo, WebGME). These are typically criticized as too limited for real programming, but seem to be climbing the S-curve.
A more likely near-term possibility is continued adoption of systems that emphasize package management, which values reuse of proven code over ability to solve problems from scratch. It seems like the language package managers are a big draw in Python & NodeJS. Maybe this trend moves even farther toward stuff like IFTT, where most people working with code are really chaining together applets.
We’re also seemingly headed toward the Star Trek TNG computer where humans query a computer with voice commands and work with it to solve problems.
I'm very happily maintaining a 15 kLoC code base (my file manager [1]). No issues at all. Though it must be said I'm alone, so know the code base inside out.
My point: Likely, there is a project size where Python's dynamic nature becomes a challenge. But 10 kLoC isn't it.
Writing code ≠ maintaining code someone else has written. Duck typed code relies in general in implicit behavioural assumption on how the values you're receiving should behave, and it's much better to be the one making choices than the once trying to understand them.
I'm not arguing that it's not possible to do it; I just think that it requires a bit of extra steps and care to document which assumptions your code takes and how it behaves that could be more meaningfully described by a statically typed language. But these are just my 2 cents.
It seems definitely promising; it depends on how much adoption it's going to take. Even the most interesting languages (like Nim, for instance) struggle these days without corporate backing.
Worth pointing out maybe that MyPy is not a language but a type checker that works with regular Python. I've been getting into the habit of using it on everything I do lately and find it extremely useful. You can even do runtime checking on the types with tools like Enforce [1], and the Molten framework that was posted on here last week [2] uses MyPy typing as a core part of the framework.
Nuitka is awesome. I used it to compile a Python command line tool recently and it worked fine. In fact, it worked so well that I added a Nuitka build script to that project... But Nuitka is written and maintained by a single person [0]. I think for such a huge task (with so many edge cases) it needs more attention/support in order to become mainstream.
horses for courses. I would prefer a 100k java LOC program over a 10k loc python program, any day.
The ease and safety you can make huge refactors is a huge win. Sure you can do it in python, but you will be writing x10 the amount of tests and final LOC counts including tests will probably have python come in closer to Java than the 10k > 100k loc indicates
I'm contributing to a >20k LOC open-source python/django project (counting .py files only). It's quite clean with an architecture that makes sense, and thus way easier to understand and refactor than an actually smaller Java project at work.
Admittedly, I have more of a Python background than a Java background.
I have personal projects that are bigger than 10KLOC and professional projects that are 10× that.
I'm not sure what you're getting at but code management in Python is really no harder than anything else. It can be clean and well delineated, or you can completely screw it up... But that's not Python's fault.
Like someone else said, I personally find that very large codebases written in dynamically typed languages tend to discourage refactoring. You may try to mitigate the issue by writing the whole application as standalone, independent modules, but we all know that a bit of interdependence always creeps inside even the best designed application.
I love Python; that's the first language I've learnt and I can't really live without its REPL. I only think that the industry has a tendency to excessively obsess towards certain languages, even when they are not the best tool for the job.
For instance, I strongly believe that the academia's morbid obsession with Java has crippled a whole generation of new programmers, which are now unable to think and code in no other way but the Java™ Way. I won't say that python is as bad as Java in this regard (it isn't even close), but it annoys me to death when it looks like we're heading once again towards a language monoculture; at least this time the language isn't utter crap.
At least some of academia agrees with you about Java. Our (large UK university) undergrads learn C, Python, Haskell and Java in their first year. We're aiming for them to get different things from each of these, and also to avoid them settling into a mindset where one language is how they think about programming. Currently I've got final year students doing projects in Rust, C and C# (each being appropriate in the appropriate domain). We didn't teach them Rust or C#, but they're not afraid of learning a new language while doing the project, which is as it should be.
I've seen short python programs (5 pages of code) mutilated with over architecture and boiler plate from a Java dev. The great thing about python is that short scripts of a few pages don't necessarily need a class if you're doing something super basic.
Agreed. An ex Java guy I was working with asked me how yo do a singleton in Python. I had to google it because I had never thought of the problem before. As the answer said a module with functions is essentially a singleton. No need to overcomplicate things.
As an iOS developer I so hated getting into Objective-C code written by Java developers. They really don't get the principle of dynamic languages and they are too fond of making shitloads of classes for everything.
In many Universities, C introductory courses are taught by people that hasn't touched C in at least two decades, or has close to zero experience to how programming modern C is. More often than not, this C is taught by people that have only used Java in years; I've seen horrible stuff, such as C89 still being thought like it's the only standard available, or gets() being used by professors that have no idea whatsoever of what's wrong with them.
Here in India, 90% of teachers haven't touched what they're teaching, like ever. But you still _have_ to study it because you need to give the exam...
Not like it solves anything -- just that there's this side of the argument too..
> I've seen horrible stuff, such as C89 still being thought like it's the only standard available, or gets()...
Youre absolutely right -- although one could argue that the curious will have the decency to look this stuff up (like myself)
The teaching system is still pretty rekt on the python side of things. Its not like you'll find people who actually code teaching at Universities anyway :(
Like now we're just going into how bad the teaching system is. The original argument that python is reaching is a monoculture seems weird to me, when I can't even run the damn thing on a browser..
Any comparison with the Indian system is futile. Basically people hop on to <insert the flavor of the times technology> every few years, mindlessly. I'm told there are now tens of thousands of people taking ML/AI, blockchain etc courses in IT services firms. A while back it was cloud computing, before that Grid(Hadoop/Pig/Oozie), before that Python...
At the end any new technology trend only has only 2 - 3 years, before people flood it and dilute both quality and wages.
The education system is another mess, which no one can clean up because the education system is designed to address literacy skills at scale(quantity), not quality.
Other big problem in the Indian system is endless optimization towards exams/interviews. Go to any OJ site on the internet and you see people even from Ivy Leagues(IITs etc) and NIT's flooding the site all day doing interview related algo work instead of focussing on their day jobs.
Its not just Python, tomorrow some thing else could be famous and you can see thousands, may be even tens of thousands of people learning it in India in a few months/years. This is regardless of the merit of the technology.
People really do need to be taught how to make out buzzwords from real tech; and the major responsibility falls in the hands of universities.
Maybe it's just because most people are just trying make their way out of poverty, and this kind of work _looks_ like a sure shot, get out of slum ticket.
Maybe it's because most people are led to believe that money is the only thing that matters..
Type systems generally are thought to help with large scale refactoring. Unless you have perfect path coverage with your unit tests, there can always be paths lurking after refactoring your duck objects to be geese.
In some ways and some ways not. Making a global change is much harder in a dynamic language. However static languages by their very nature tie the code together more tightly.
It is much easier to decouple something in a dynamic language. The re-factoring needs in a dynamic language is thus entirely different.
Yes, but these are two sides of the same coin: being able to inject code dynamically in a way that cannot be statically checked at compilation time results in code that will cause runtime errors if there are issues (even basic issues). Also, I would argue that your use of the words "tie the code together" implies something that isn't actually occurring. With a static type system, verifying that code is using the correct types for classes, methods, functions, etc. is not binding them together in any way. If you drop down to assembler, you can do anything you want with any of it, just as you would with a dynamic language. A static type system is simply a compile-time integrity check on the validity of your code.
Also, the de-coupling and modularization of functionality has been a thing long before dynamic languages became so popular. You simply need an FFI, and then the compiler makes sure that everything is straight before you get anywhere near deployment.
Personally, I think that refactoring Python is straightforward (and many refactorings you need to do are much simpler and easier to do without breaking existing APIs -- doing refactorings in Java usually needs changing much more code even for simple things).
I think that the real problem is not having a good test coverage, and I agree that bad programs written in Python may be harder to work with than bad programs written in a statically typed language (but in a good codebase, even if large, I'd rather be working with Python).
-- although note that in the latest incarnation of Python you can try to use Mypy (http://mypy-lang.org/) along with https://github.com/Instagram/MonkeyType or https://github.com/dropbox/pyannotate to add types to you and go from there if you really miss types too much and this could make a bad codebase bearable (I'm personally waiting for PEP 544 before adding any types though as I'm not a fan of the type system implemented in PEP 484 very much).
Personally, I think that refactoring Python is straightforward (and many refactorings you need to do are much simpler and easier to do without breaking existing APIs -- doing refactorings in Java usually needs changing much more code even for simple things).
Refactoring in a non statically typed language cant be as straightforward - many refactors in statically typed language can be automated and guaranteed safe.
Why do you think Java is so much harder to refactor? Java and python are not that much different, and Java has static types which aid refactoring as well as a really strong tooling ecosystem. I haven't worked with large python codebases, but it hasn't really been my experience that refactoring Java is unusually hard, except insofar as average code quality among projects written in Java is probably lower.
> refactoring a equivalent Java project is way more challenging. I'd argue.
Sorry but you are wrong. Just plain wrong. There are benefits for both languages. But to say python is easier to refactor is like saying Java is more concise. It isn't.
>Refactoring a Java project takes a whole department and process management crew.
Has nothing to do with the language, but the company. A small company will refactor and have an update out same day in Java. An enterprise using some python will be as you say
Java is absolute garbage, and it is definitely not the kind of language I've in mind, which is something like Go, Rust, C#, Kotlin, Swift, ocaml... Even C++ is perhaps easier to understand than Java, if not because it tends to have half boilerplate than a similarly sized Java project.
Java. C#, C++ will never be replaced. Want job security learn and code in these languages. Enterprises love them and are not going to incur cost to replace their code-bases with anything else.
Cobol had 10x fewer developers at its peakc and probably 100-1000x fewer lines out of code out there, in the wild. Plus it didn't have a huge Open Source community.
There were 10x fewer developers in general back then. And who's going to take over that community once all its current members retire?
The cool kids of the future ain't going to learn Java once companies move to a better technology. It may take decades, but eventually people will stop building new code in the language.
Exactly. Even-though my livelihood right now comes from Python, I was trained in Java in my university(along with C). I did some coding(web scrapping) in Python but that was it. I still can and willing to write in Java.
There are a lot of Cobol replacement projects underway. Yes, it'll still be around for awhile, but there's less of it in the wild every year. The sad thing is that many companies prefer the mainframe Cobol service as it has run reliably for decades and the Java replacement does not have that trait.
I actually believe that we can solve this problem for most cases.
Take dart for instance -- it doesn't require any typing as such, but has an excellent type system that supports generics too. The type checker is pretty robust, but flexible at the same time...
While I may personally prefer statically typed language for large code bases, I think people greatly exaggerate the problems of maintaining large code bases in dynamic languages. Python projects such has NumPy, SciPy and Pandas are several hundred thousand lines of Python. Pandas is 250k LOC in Python.
GNU Emacs is 1.5 million lines of LISP code, another dynamic language.
It is ultimately a question of sound software architecture. You don't have to care about how many lines of code is in the libraries you use daily or in the Operating System you use.
In similar fashion any software can be modularized so you only need to concern yourself with your part.
Chiming in anecdotally here: 50k ``real'' lines of Python and climbing. No problems maintaining it but I wrote it all for my dissertation. It's scientific/CAGD programming, -Not sure but it feels like that makes it easier. Pretty sure I need to re-write it to entice others in on the fun. Anecdotes aren't valid far from the experience, I am afraid.
I, along with one other guy, maintain a >500k fortran 77 code base (bit of C/C++ sprinkled in) at work. It's way cleaner than my home-brew python. Maintainable is about wise practices, not language choice.
Everyone keeps saying that, but my experience is rather the opposite. Static typing is great when the program is small and you can keep the model in your head, but the larger the program, the more you benefit from dynamic typing. Sure, static typing helps with tiny refactorings, but making non trivial changes that don't play well with the existing model proved to be too much effort.
I can't speak for Python but since the codebases were Perl and Common Lisp based, I believe it would be similar, should there be anything similar to the MOP for Python.
Static typing is great but not essential when the program is small and you can keep the model in your head, but the larger the program, the more you benefit from static typing. Static typing helps with tiny refactorings, but it truly shines when making non-trivial changes that don't play well with the existing model; it's as effortless as making the desired changes to the model, then mindlessly fixing all compile errors, and it will likely just work afterwards.
Dropbox has had several headaches from its reliance on python, first among all from the fact that Python is still bound on running once thread at once on a single core after all these years (thanks GIL), so it does not parallelize at all without abusing subprocess + IPC.
The things I do in Python - mostly straightforward simple automation scripts around AWS - I would hate to do in my go to language C#. The overhead and ceremony of C# is not worth it. There are also no issues with a mismatch between what you think should be running based on what is in source control and what is running.
Static typing doesn't have to mean overhead and ceremony. Do you think you would, theoretically, enjoy writing scripts in a language like Crystal [1], which provides both the benefits of static typing and a scripting-language-like syntax?
It’s not just the language being statically typed, it’s also about the tooling around the language. Modern IDEs around mature static languages are s powerful.
I took a quick look at them and this is just first impressions, so bear with me?
They still look kinda low level to me. Yes, they provide a lot of high-level constructs and data types like python -- But what they fail to do (IMO) is a consistent abstraction.
For example, all of these languages have a concept of pointers. All of them have a distinction between different integers (int32, int64 etc..)
I had similar feelings when I used dart or even java. They're high level, but not quite!
I don't mean to thrash on these languages in any way. I think they have some real value for programmers.
It's just that I've never seen a language just throw everything out of your way and provide you with this extremely high level and easy to reason interface.
Nim actually looked pretty good, but then I realized it was just the significant whitesapce xD
Maybe I'm too biased because Python was my first (proper) language.
typically languages like Haskell, OCaml are based on great abstractions (and are good to express new ones, especially from a type-system point of view).
If you're looking for languages which are similarly "easy to use", I can suggest to try Nim or Crystal.
Nim looks a lot like Python(indentation-based and similar constructs), it has a nice type system and very strong metaprogramming which helps to often express stuff in <= lines you would in Python (and also C and JS backends with excellent interop.
Crystal is inspired by Ruby, but it's also statically typed and mostly used for web development from my impression. Its windows support is still under development but it's also very expressive.
With such a rapidly growing user base and wide array of capabilities, Python might seem destined to become the lingua franca of coding, rendering all other competitors obsolete.
I realize it is weasel worded not to mean what it implies but statements like this is a surefire way to lose any credibility. Not that there was much underpinning besides some search engine popularity.
Nevertheless I'm baffled by python's popularity. With my admittedly limited exposure I would not reach for it again any time soon. When it is the right tool for the job?
Coming from a data science background, I wasn't surprised by this.
* From doing Engineering problem sets in Matlab and also using Java and C/C++ in CS classes, Java's inability to overload operators made it unbearable for anything math related.
* C/C++ wasn't bad to write small algorithms but as someone wanting to look at data, I really wasn't up for solving memory leaks.
* Then the big problem with languages like Matlab is they aren't general purpose programming languages. This meant things tooling around things like sending an email notification, creating a light web framework, etc. were limited, meanwhile, numpy's eco-system began offering many of the same benefits to the commercial data science libraries.
On the testing side of things it’s a really good option. It’s what I’ve mainly used and the Pytest test framework is incredibly flexible and powerful. I personally find it a much cleaner and more readable option when compared against Java. I’m migrating over to Websriver.io now only because I feel there is finally a good JS option that offers the flexibility needed for UI tests.
Python is in part a "quick and dirty" language that nevertheless has many advanced features, which means it's a great language to easily hack something together.
Whether to use it afterwards in production... it doesn't shine in the maintainable aspect (it allows for some killer one-liners close to perl level), or in the low level performance aspect, so there might be some better alternative for a particular case.
Still, Python and in particular Python 3.x is a decent compromise between hackiness and something more serious, so it's no surprise that many choose it as a starting point.
Writing powerful CLI tools. Take a look at AWS's Python API.
Data wrangling -- it has libraries for handling so many data formats.
Quick and dirty scripts. I've used it for many one-time scripts that need a bit more than regex.
Data science / analysis -- it has so many great libraries for math and data wrangling.
The biggest pain-point I have with it is around module handling. I'll often run into issues the first time I'm deploying a new pycharm project. The second biggest pain-point I have is around concurrency/multi-threading, but I think the concurrent.futures module might provide a decent solution.
In my very bubble, it kind of already is though, not because Python is so good in and of itself, but because you can express general coding/programming ideas in a readable and concise manner. The syntax provides a general way to get your ideas across to programmers with all kinds of backgrounds and to proove they also actually work, anyone can open a shell and see for themselves and tweak and play with your idea.
I understand it (and code a hell of a lot of Python) - but it's a shame. While it's a lot better than the many low-barrier-to-entry languages that came before it, it is such a heavily compromised, suboptimal language in so many ways. I feel like a whole generation of programmers is becoming stuck in a local optimum that will cost us very dearly later on.
While you're likely correct that FORTRAN, COBOL, C, Visual Basic, C++, Perl, ColdFusion, Java, Javascript, and Python, were sub-optimal choices, they're just tools whose popularity boils down to current programming pop culture.
That being said, it could be something much worse than Python.
Well, this is how lengthy arguments start about pros and cons of languages, as there are many many people who will argue these are not problems or can be worked around or are actually preferable for whatever reasons.
But my biggest problems with Python are:
- Lack of strong enough / useful / culture around typing
- Effective limitation to single core through GIL
- Lack of proper solution to closures
Really the first two are the ones that make it for me, a fundamentally crippled language. And they are exactly the things you don't notice when starting out, and the things that limit you eventually.
I can see the sudden jump in popularity for python alongside data science - it really is great for that.
My view is: python is great for offline tasks, like functional testing and training in machine learning - tasks that do not have huge, direct customer interaction. But to put something in production, please give me a strongly typed language with compiled binaries and no additional installations/VMs (and for me that's Go).
Python is strongly typed, insofar as the term has a meaningful definition at all.
You meant "give me a statically typed language". Which may not actually be what you want, since statically typed languages can be, and some historically have been considered, weakly typed (C, for example).
No, again you're thinking of "static", not "strong".
Strong versus weak is historically poorly defined, and there's no single universally-accepted definition, but you can get the gist of it from considering the following:
a = '1'
b = 2
c = a + b
In JavaScript, the above code results in the name 'c' being bound to the string value '12'. In Python, the above code raises TypeError. Generally, "weak" typing refers to the idea that the language may coerce the values of operands to other types in order to make an operation succeed (in the above, JavaScript coerced the value of 'b' to a string and interpreted the '+' operator as asking for string concatenation, hence the result is the string '12'). "Strong" typing generally means the language won't do that, and will error out on you if you attempt an operation on values of incompatible types.
"Static" versus "dynamic" typing is a completely different set of terms. There's also some debate over definitions, but broadly, consider the following expression:
a = 1
In a statically-typed language, both the name 'a' and the value 1 have types, and in order to bind names to values their types must be compatible. So in the above, it's likely that both 'a' and 1 are of type int, so the assignment expression would be legal. Later attempting to bind 'a' to a value of a type incompatible with int (say, 'a = "foo"') would fail. Statically-typed languages also come with an expectation that all expressions can be checked for compatible types without needing to execute the code, and that a tool will exist to do this checking for you.
In a dynamically-typed language, however, only values have types; names do not, so binding a name to a value of a particular type does not foreclose later re-binding to a value of a different type (i.e., the name 'a' might be bound to an int now, and get re-bound to a string later). This means there will be cases in a dynamically typed language where the only way to verify correctness of an expression is to execute the code (though you can usually check a high percentage without running the code).
Is it though? I mean, yes there is fragmentation and lots of old documentation but that's because Python is 25 years old and several approaches have been tried.
What's especially difficult about the following two steps?
Ubuntu 18.04, Miniconda 3.6 with Numpy etc. I tried Nuitka on my own project, it kept spinning for 20 minutes compiling Numpy dependencies. I eventually gave up and killed the process.
One issue with PyInstaller is that it doesn't create a proper standalone binary. It either gives you a full directory tree or, with `--onefile`, an archive which self-extracts into such a tree.
I usually shudder at duck typing and/or GC for anything bigger. Both have the ugly tendency to explode in your face at runtime. The first type of error doesn't occur in C++ (at runtime), and in the other instance I feel like having more control. But maybe my academic experience with the two is to blame (multi-person projects in python (multiplayer game using lecturers lib) and Java (SQL DBMS, GC killed performance while in C++ I would have written a specialized allocator)).
Having worked in Python as a way to pay for food nearing 7 years now, the biggest complaint I have at this point is that, at an enterprise level, it's severely under-equipped to enforce its touted "There's probably one right way to do things" mantra.
People are often introduced first to Python by way of scripting, but the tricks and tools learned while scripting are exactly the sorts of things I've found cause dirty code. Applying clean code principles to Python is hard, because it's so easy to write dirty code, and that's a byproduct of how easy everything is to do in Python. That ever-nagging "but it's working" continues to get in the way.
It's a weird problem. I dunno what to do about it, and currently end many of my debates with colleagues on topics related to the problem without agreement.
It was true recently enough ago that the people writing Python code professionally were introduced to Python from a scripting entrypoint.
For non-programmers, I think it's definitely still true that "scripting" is their primary use case for Python, and when those non-programmers shift into software development jobs because they have "3 years experience" with Python, they bring all of their bad habits with them.
Most programming languages suffers from lack of enforcing idiomatic coding style, don't you think? Perhaps python more than others? Scientists having learned that OOP is the one true methodology make horrendous code if a language has classes. I've also seen FP coders being a bit too extreme in Python, inlining everything and not using comprehensions.
It's fascinating to see the differences between the two. The OP is a bit shorter and a bit more awkwardly worded. It also inlines some names (for instance, referring to the package manager as "an online repository" instead of "the Cheese Shop").
I wonder if the OP was written for the website (with a lower quality threshold) and the "full article" was edited/expanded-upon for the more prestigious print edition.
Here's a nice combination of _Python_ and an _economist_: Paul Romer, this year's Nobel prize laureate in economy, moved from publishing recent research using Mathematica to Python and Jupyter notebooks.
StackOverflow has the data on this: Python for most purposes is entirely static. Python for data science is utterly dominant. That’s what’s driving the stats.
I found the data points coarse (x = 5 years) but useful to see similar trends of popularity between C, Javascript, and Python. Note: these metrics are defined by global search engine popularity.
Why not? C++ is still the de facto standard for desktop software (including games), last I checked, and C is still the de facto standard for systems and embedded programming. Python hasn't really displaced anything in either realm as far as I can tell (some C/C++ codebases use Python for user-customizable scripting or plugins, sure, but it probably ain't doing the heavy lifting).
This doesn't match what I see and hear, on Windows if you already have a C/C++ codebase, its likely that one will stick with that. Otherwise, legacy Visual Basic/Delphi programs seem to consistently be rewritten in C# (primarly due to Microsoft heavily pushing it, providing tons of documentation and easy examples).
This is the impression I get from a few proprietary software vendors I used to work with, and those that pop up at local events in my area. I can't name one actively migrating to C/C++, C# is big if your stuck on Windows.
That being said, I have been diligently avoiding proprietary software for the last few years. My life is much happier without it, ain't got no licensing worries :D
That entirely depends on how “popular” is defined. If you mean “preferred choice for greenfield consumer and B2B apps”, then yes, C/C++ would be ranked way lower. But as far as I understand this statistic is based in search results, so it takes into account the huge body of existing software that needs maintenance (after all, you need to stay up-to-date and also look things up again from time to time), as well as domains in which C and C++ are a good still dominating, such as programming embedded devices.
Python is really widespread, but at least in my company it's used for analysis and prototyping only. Then the prototype is ported either to Java, C# or C++.
I don't know if it's common practice, but we work like that and totally makes sense to us.
Depends on the organizations needs, for what we work on performance does not matter terribly much, thus a MVP in Flask is often cleaned up and put up on Gitlab (with a Github mirror) once its vaugely usable.
How do you compare the popularity of full featured languages like Python to languages like JavaScript that have numerous frameworks built upon. Were React, Angular, Vue, Typescript search results taken into account to draw this graph ?
Many people google "boost graph library" but are c++ developers or google "pandas how to drop columns", "numpy how to append to array" but are python developers.
Not really. TypeScript is just a JavaScript superset, and developer code JS with all its good and bad with some annotation types. TypeScript is not like Dart or reasonML for which your analogy would have been correct if someone said Dart count as JavaScript.
A few jobs ago was in Python. Joy to write, absolute terror to deploy.
Seared into my mind was a failure caused by delivering .pyc files (oops) to a server whose shared libs were compiled with different flags than our CI box.
To be fair, this was before docked (or at least before I’d heard of it). Maybe that’s improved things a bit.
Python is, despite its simple appearance, a much more complex language than JavaScript. A V8 equivalent for Python is certainly possible, but would require even more work than went into creating V8 itself - who would be motivated to spend that much effort on Python?
Dropbox? They seem to be the company driving Python nowadays, but gave up trying to speed up Python in favour of writing "performance-sensitive code in other languages" [1].
PyPy uses a meta-tracing approach in an attempt to reduce the amount of work required. Its speed is impressive, but I'm sure meta-tracing leaves some performance on the table.
Another way to reduce the amount of work required would be to support only a subset of the language - for example, a V8 equivalent for MicroPython rather than for full Python. I'm sure this could be made very fast, but it would have very poor compatibility with existing Python code, so what would be the point?
It’s interesting that by this metric, vb.net beats javascript. It’s remarkable for a language which maintainers have placed in maintenance only mode.
I think it is a language that people should consider for beginners. Rich and helpful IDE, easy and permissible syntax (english keywords rather than symbols, case insensitive, optional parenthesis). You can do pretty much anything with it (console, desktop, website). And it provides an upgrade path to business users which only programming skills are usually toying with VBA macros.
I was thinking the same thing. Their google search language comparison is probably incorrect. For example no one includes the keyword "javascript" when searching for nodejs or react. Instead they should have used Stackoverflow's developer survey results.
From an article that analyzes Stackoverflow's developer survey results[1]:
1. JavaScript
2. HTML
3. CSS
4. SQL
5. Java
6. Bash/Shell
7. Python
8. C#
9. PHP
10. C++
But nodejs and react aren’t programming languages. No one is including “c#” when searching for spreadsheetgear, asp.net or wpf even though they are .net libraries and frameworks.
Precisely the point. If you don't count searches for popular libraries and frameworks into their respective language buckets, the results are mostly bullshit, and at best probably reflect the undergrad curriculum.
Yeah, JavaScript also has too many names for the language itself. I mean, I search for “Django” or “Python 3” instead of “Python” half the time. But with JS, if I’m looking for info about the language itself it’s usually with the keyword “ES6” instead of “JavaScript”.
> Every device in the world shipping with at least one environment for JS
I doubt that's true, while it would be more true to say that most devices in the world is running something compiled from C or C++. Also, The JS environment (web or Node) is running on top of C++ code.
Well, this ranks them by search engine popularity. That metric certainly has its problems, but I have to assume it does correlate with people coding in those languages more than it correlates with people using things coded in that language.
When I have a React or Vue or Angular problem, I don't search for "JavaScript".
Only when I need docs on JavaScript's native features, for instance Array.sort(), I include "JavaScript". And even then I may use "JS" instead, but I assume Google knows that's a synonym for "JavaScript".
While each of those searches would be part of, say, a React project, I would definitely consider that part of the JavaScript space.
To be fair, all of the above would also go for Python vs Django, etc.
Never been a fan of Python , but all the comments about Java are legit.
I’m a big Java User mostly for Enterprise Apps & Services. A third of my code is boilerplate, Python really shine when it comes to making stuff concise but in the same way not having type is something that has always bothered me.
Dynamic typing is something I have accepted with JavaScript but for Python for some reason it has always bothered me...
Sorry, I misunderstood. I thought the OP’s reference to dynamic typing was what you were referring to, but they were actually the ones confusing static with weak typing.
Just because people are searching for it doesn't mean it's popular. It means people are trying to answer a question about it. I'd posit there is more of a correlation between searches and newusers/issues/problems with the tool/language than how much it's being used. Stack Overflow is a better source for these stats anyway.
I learned development with Python (after trying to grasp the concepts of C for a while) and I have to say that alongside .NET it is one of the easiest languages I had the pleasure to work with. So much so that after 10 years of mainly Javascript and Node.js work I have decided to go back to Python simply to make my life easier and be more productive.
Seems unlikely Python can be more popular than JavaScript/TypeScript since they are equivalent to Python and have the critical advantages of working on the web (and thus being known by all web developers) and having optimized engines in web browsers.
The computing world is a lot larger than the web, believe it or not. Python is huge in scientific computing, and is used for lots of server-side scripting. JS/TS don't have much of a presence in scientific computing beyond the web presentation layer, which can be generated from Python libraries. Python is used everywhere in science, and the ecosystem is incredibly rich.
I started programming in python in 2010. It has been a good tool for solving a variety of problems.
I do have to wonder though. Is python popular right now because so many more people who have never programmed before are now starting to program?
I came from a biology background and my hatred for matlab following one quarter of exposure was sufficient to push me to python because that was what was offered as an alternative (taught by a gradstudent rather than the guy who funded the lead dev for the Octave project). There was no other choice. Julia didn't exist etc.
In the 8 years since I have written a lot of bad code have come to very much dislike python (though I still write it daily). The reason is not because of any of the visible failings of the language. It is due to the deep hidden inconsistencies, and unexpected and unpredictable behavior of the runtime (try using list comprehension in class scope (What is class scope you ask? Well. Python is a language for newbies... so by the time you can ask this question you are too invested to realize that you might have made a bad decision)).
Python is kludge on top of kludge without the diversity of implementations to take another path when someone calls bullshit.
Python can get away with it because they have runtime hacks that let you sort of do what you want, but there will be a corner case
that doesn't work because there is some unspoken and undocumented assumption that you are violating and will be duly punished for.
None of this is the language's fault. It's the runtime's. The python runtime (don't even bother with discussion about the GIL, which is a red herring and a bad case of Stockholm syndrome at the same time) is a mess. The language is fine, even great. But the fact that different scopes have radically different meaning and that expressions only work in certain scopes [0] is a complete nightmare.
My main emotional interaction with python is that if I follow the BDFL approved path, everything will work. If I stray from the fascistic predetermined happy path, good luck to me, I was on the wrong side of history TM. 99% of the code that I write that doesn't start as a class I end up rewriting as a class (probably because python doesn't have closures ...). Python lures you in with promises of easy to write easy to read, but when you get to that point where your actions are no longer approved then then pain and sadness and rage are all you have left. I won't name the language that has almost the opposite interaction that I come to vastly prefer, but the emotional experience is radically different, I get extremely frustrated because I don't understand something in that language, and eventually come to realize it is because I did not understand something simple. In python I initially think that I am the one to blame, but after months or years, I finally realize that it is actually the language that is fundamentally broken and inconsistent and I have been mislead the entire time. I get frustrated because there no possible way for me to solve my problem by increasing my knowledge, I would literally have to fix a 4+ year old problem in the language (which I'm sure I couldn't do and that I'm sure has not been done for very good technical (debt) reasons), or find a library. There are a lot of libraries, but don't be fooled. Every library is a new language. Python is just the gatekeeping runtime.
Python will thrive for a large set of use cases, but when you hit a wall in python, the problem is not because you don't understand, it is because python is broken. That is not a happy realization. It is like growing up and realizing that people are actually evil, not that you were just misunderstanding them. Or more generously, that expecting people to be on their game 100% of the time is unrealistic. The difference is that python is that attractive person that is great at first impressions, but a disaster in the long term, and if I hadn't had the good fortune to meet a language that was super awkward for the first 6 months or so, but turned out to be a keeper, I'd believe that python was the only thing there is.
Python is and abusive language, and probably exists and is successful because things like perl also exist (and because it solves a whole bunch of practical problems). I'll keep using python for a long time, and the coming of age of pypy brings me great hope for the future,
but there are classes of problems where I now go elsewhere. There are problems everywhere, but in some places they seem to be rational rather than contingent.
>I do have to wonder though. Is python popular right now because so many more people who have never programmed before are now starting to program?
In recent years coding has become hip/cool. I'm pretty sure none of the 10,000 Bootcamps/online courses that popped up are teaching c++ at the moment, it's all the easy stuff like Python and web. This gets people to their 'aha' moments quicker and keeps them feeling like they're progressing, even though they're doing barely any thinking. I will admit though if there is one language for anybody to learn for general purpose, it's Python.
Unfortunately you have not provided a lot of examples to go along with your major complaints. I feel like at some level, every language will have its idiosyncrasies that create annoyances and issues. From what I can tell, you are describing concerns that would not void the use of Python entirely. These issues tend to increase with higher level languages as the complexity of the feature set expands and it become unreasonable to support every combination of every idea.
I don't know if you'll have better luck switching after making such a large investment in Python, but if you do find something to supplant it please document your experiences for other to learn from.
I only know about Python from what I've read and tinkered with, but IMO, Python = FoxPro (look it up): it is fairly easy to learn, immensely useful and you can do lots of cool things with it, but it a) relies very heavily on C or external libraries for any type of raw code performance, b) it's inconsistent and the inconsistencies are somewhat hidden and not obvious, and c) distribution is a nightmare, so developing anything professional that needs to be dropped on a target machine is a problem.
"a language that was super awkward for the first 6 months or so, but turned out to be a keeper"
I wonder, do you consider the language's awkwardness somehow related to its quality? Do you see it as a prerequisite to avoiding language shortcomings?
No. In this case that part of the analogy is actually not apt at all. To use another bad analogy I think my point is that the feeling awkwardness actually comes from the fact that I was doing the equivalent of holding the hammer by the head and trying to pound nails, only to later realize that I should have been holding the handle, and that there actually was a handle. In python it is like having a sledge hammer that you can't possibly hold by the head, but that no matter how hard you try you keep putting holes in the wall when you try to hang a picture.
Someone will say 'but why not use the tack hammer?' and, at that point I argue, I am no longer using python the language, but a whole new language that happens to look like python but that has a different runtime and a different set of assumptions. This even happens inside of the core language where `[d(a) for a in b if a == c]` can be valid in some scopes but not in others. The other language does the opposite, it says "you can't do that" and so I try to hold the hammer by its head and think that it is the language that is awkward because I don't know how to use it and that there is an fact a nail gun that is configurable along every dimension from nail diameter, to nail length, to insertion speed and pressure profile over the whole insertion timecourse, it is just not there in plain sight and I the first few times I used it I accidentally nailed my feet to the ground with railroad spikes.
I guess it is more that every time I learn more about python it is about what I have to NOT do in order to be safe. In the other case I learn what TO do in order to solve the problem the way I want to. Maybe the clearest example I can think of is lambda, where you have something that looks like a lambda, but that doesn't doesn't even work like the def (): that it copies due to the biases of the language designer. Honestly it would be better if Python just removed all the syntactic features that don't have generalized runtime support because that way you don't have to know which features you can use where. Sure, there is a use case for lambda in certain contexts, but if Python refuses to support lambda as a general abstraction that I can use everywhere, then it should be removed from the language, because including it leads to those hidden inconsistencies which require an enormous memory burden on the developer just to know what is not allowed. That is dead knowledge that in no way empowers and enriches the programming experience.
In essence the push for pythonic code is not just about maintainable code, it is about keeping people on the happy path without removing delimited abstractions, which is fine, except that the explicit solution to this problem would be to have `from __i_know_what_im_doing__ import lambda, comprehensions` so that the core language didn't misleadingly offer these delimited abstractions. Of course this means that many of my favourite features would be thrown out until some core technical challenges were resolved.
Want to do web server development? Flask, Django, Tornado, etc... can do that. Probably not as concurrently as Go, but it'll do.
Scientific computing? Its no Matlab, but the syntax is easy and the time you lose in processing you make up for when developing.
Desktop computing? PyQT is pretty good. Electron may be faster to build with, C will be faster at runtime, but Python, even with Tkinter, is ok.
Functional programming? There are list comprehensions, map, reduce and filter. Not as good as Haskell or Lisp, but it'll do.
Python isn't the killer language for any of these really, but its good enough in almost all cases that you'll be very productive and can iterate to a good MVP quicker than with most other languages.