Python has all the bells and whistles you need to build large code bases, but you do need to get organized, no language can really do that for you and attempts at forcing you to do it give rise to 'factory factory' nonsense. I'm no huge fan of Python, I use it when I have to but Rust is from a bells-and-whistles point of view a step down from Python, it is more a better 'C' than a better Python and Go has it's own share of issues that has kept me away from it so far.
If you want to crank out some research project or application quickly building on the libraries that others provide (especially in machine learning or other data intensive fields) then Python would be my first choice, and something like 'R' probably my second (though I see 'R' mostly as a data scientist's workbench that has a programming language bolted on).
1 - When working with a large enough code base that you don't remember exactly what every function you have written needs, or using third party libraries, function signatures lacking type specifications make it harder to know what is expected. if there is documentation that is good, you can of course load that up, but this slows you down. IDEs today can usually show you the function signature as you type it, but with dynamic types you get less information.
2 - When refactoring something in a statically typed language, changing a data structure or type name will cause the compiler to error every single place that you need to adjust for your change. With a dynamic language you need to rely on unit tests or you will be finding places you missed during runtime for a long time afterwards. If you do rely on unit tests for this kind of thing, then you have sort of built an effectively statically typed environment, at some large degree of effort, with worse runtime performance to show for it.
I understand some of the workarounds people use to deal with these problem but a lot of them seem to push you to the point that you might as well use static typing and get better runtime performance for it.
> Mypy mostly seems immature, buggy ...
That was my experience some time ago as well, but it's getting better from release to release. Look at the repo , it's continuously improving. I find Mypy deserves more recognition.
The same guys working on mypyc , which I think is very interesting too.
Either some weird support library hasn't been upgraded to 3.x yet or some expensive piece of software/hardware can't be accessed with Python 3.x
If your project is just _your_ project, then dynamic languages let you start faster and produce quicker. Apart from anything else, these languages attract people who value productivity, so clunky tools don't survive in those ecosystems.
Of course, almost no successful project stays in the hands of just one or two people: new developers come in, or it's Open Source and more people show up, as well as the "drive-by" contributions.
IME, Ruby on Rails projects scale to about three core developers before you start to feel pain. The strict conventions of the framework help mitigate the communications issues of keeping people in sync. Beyond that, the implicit knowledge that is not in the codebase is harder and harder to scale.
The one thing that stands out to me is that it's heavy on the unit tests and light on integration tests which would make an application like this (at least for me) more difficult to handle. I'd go heavier on integration testing and lighter on unit testing.
* dependency injection errors,
* template errors,
* magic annotation malfunctions, etc.
The last one in particular actually seems less safe than Python's decorator mechanism. Java splits it in two parts, so that the annotation classes used at compile time are separate from the library that will process them to actually do something. If the latter part is somehow broken, the code will just do nothing, so you don't even get a runtime error.
In Python, decorators are just functions that serve as annotations as well as doing the actual work. There is usually no separate processing part that can go missing. I've never encountered similar problems in Python.
\*More than one, of course. But I use only PyCharm :)
I support the notion that Python is unsuitable for larger codebases with all my heart.
I can write reflection based Java code that will make automatic renaming choke.
There are ifs and buts. They may be uncommon, but they're around.
While you can do reflection/code generation even in Java, for the non-framework code almost nobody does it.
That's why I feel lot safer with large codebases in Java, even though I love some dynamic languages too. I don't think they're as scalable for big programs though.
There's your problem then! You don't write a codebase that size in such a way that you need to keep it all in your head at once. The whole point of 'getting organized' is to reduce the amount of code that is in scope for you to have to understand. If it's more than a page or so then you will always end up with hard to debug problems. Reducing the amount of code in scope is one of the most powerful tools you have to deal with large codebases.
...you're doing it wrong.
It sounds like your loss of interest may be correlated with your level of discomfort with increased cognitive load.... the alternative explanation would be strange: you can only be interested in simpler problems/implementations. (Which I doubt)
I build things from scratch. By the time a project hits that size, I’m utterly bored with it. It’s been figured out and built. It’s been refactored. It’s been tuned in places that need performance improvement. It’s had new features added in. But fundamentally, the idea(s) behind the software have become completely boring. 5 years of working on the same thing through multiple iterations—including reducing LOC while expanding features. There’s a certain project size that I’ve found correlated with time ticking my internal interest clock to 0.
It’s a bit like how Professor Farnsworth feels when there are no more questions.
Edit: I’m not very good at describing my feelings. Delight was a poor word choice. For me, 15K is typically early in a project, when there is still delight to be found in figuring out the problem space. The unanswered questions are delightful, interesting, thought-provoking. Bringing Professor Farnsworth into it, i was reminded of him saying it was the questions that drive him.
This is almost satirical.
> Assembler has all the bells and whistles you need to build large code bases, but you do need to get organized, no language can really do that for you and attempts at forcing you to do it give rise to 'factory factory' nonsense.
I can definitely see someone replacing python with any X language.
Just because you can do something, doesn't mean you should. I personally like opinionated languages that have made some decisions for you based on best practices and research.
I didn't read it that way - but your retort reads as quite definitely snarky.
Either way -- could you condescend to explain to us what was so outright ludicrous in the sentence you quoted?
The parent I was replying to said this:
> Python has all the bells and whistles you need to build large code bases,
but you do need to get organized, no language can really do that for you and attempts at forcing you to do it give rise to 'factory factory' nonsense
I have a problem with this statement which I hoped to convey by satire (but obviously failed).
The argument being put forth by OP is that python's lack of static compile time typing makes it hard to build large code bases. This is a legitimate argument supported by several whitepaper research papers conducted by well known and trusted entities. The parent I replied to countered this with "but you do need to get organized", but neglected to specify any details about the said organization.
Obviously I have a problem with this because the statement is utterly meaningless.
You can make this type of statement about literally every language. I picked assembler to demonstrate this point. Also, it is low effort jab against Java programmers, as if they are clueless about organization.
That statement is equally applies here. Assembler obviously has all the bells and whistles you need to build a large code base, given that it's turing complete and that people have in fact built large code bases with it.
So in summary, the parent I was responding to:
1. Made a vague statement that can be applied to any language.
2. Made a low effort snarky remark about Java developers without providing any support/proof.
3. Said you need to get organized in order to write large code bases in python, provided no further details on the organization.
4. Contained no useful information at all that can actually be applied and used to write software in python or any other language.
5. Tried to counter a legitimate, research supported position with a vague meaningless statement.
I think given all that the satire was warranted.
IMO Python is still the best language to go with when you are writing anything "scripty":
- Stuff that's not expected to be long. Do one thing well, let some other program do another, compose the higher order functions from functions that work. Of course this isn't always possible.
- Stuff where the types are not going to confuse you. Often you just have a few types, and it's fairly obvious what you can do with a given object. If you get too many types, there's a good chance it will start to get confusing and then strong typing would help.
- Stuff where performance is not a problem. Don't write your HFT system in Python. You simulation though, might benefit from orchestration of some VMs.
The ultimate use case for Python is glue code. And guess what, a lot of code is just glue. A good glue language has a large ecosystem of libs, and some language features that make it easy to read. That's Python in a nutshell: whitespace formatting makes things easy to read, and you get built in list/dict syntax to keep everything short and sweet.
For me Rust and Go are interesting having come from C++. There's a lot of pitfalls in C++ that might be helped with Rust. At the moment I have a whole load of Valgrind suite tests that look for potential problems. Perhaps Rust can bake those into the language? I haven't had time to look at it yet.
Go seems to be interesting as a compromise between easy and fast, and by the sound of it things have turned out well. I have people on my team using it, and it seems to be the thing to do for web services that need to be faster than nodejs but you don't want to spend a load of time making sure a c++ version is correct.
Note that Python is actually a descendant of the ABC programming language developed to be human-friendly, not a shell replacement language per se.
I prefer it to Korn and Bourne because it's compatibility is so widespread.
For this reason, the safest thing to do is write code for either the original Bourne shell, or ksh. Programs written in ksh are the most portable across systems, moreso since other systems have it too. And to top it all off, ksh sets the standard for POSIX, especially ksh93.
"If you are writing a script that is more than 100 lines long, you should probably be writing it in Python instead. Bear in mind that scripts grow. Rewrite your script in another language early to avoid a time-consuming rewrite at a later date."
Python brings a ton of dependencies, thereby seriously limiting your deployment options. And, together with Python's not-so-great package management story, this spells a lot of trouble for as long as Python 2 and 3 need to coexist. Many useful Python 2 programs (such as Apple's old caldav server based on twisted) won't be ported over to 3, and the migration to Python 3 creates an unnecessary burden on apps that chose Python as user-scripting language, such as Blender. There might be reasons to use Python, but the aesthetics (or lack thereof) of a scripting language, or an artificial limit on the number of lines most definitely isn't one of them.
Shell portability issues are a thing. Also, why all the snark?
So he’s basically comparing 20+ years of Python usage with marketing material from Go and Rust. Right.
I wish people could be honest with their motivations, instead of making up excuses to justify this sort of change. Here it’s a classic case of “I got bored and I don’t like the new features, want new shiny”, wrapped in a bit of whining about problems that have well-known solutions (use a good IDE, use type hints or one of the many libs that enforce types...). I have no problem with that, just don’t try to sell it to me on technical grounds, because it looks very much like you have none.
Python is still one of the best ways to get stuff done, quickly and with a minimum of boilerplate. Its limitations are well-known and have not really increased for about 15 years - if anything they have shrunk in many, many areas. Adoption rates keep exploding, despite a lot of doom forecasted by some alpha geeks about 10 years ago: “There is no mobile story! We will all be using Ruby/Go/NodeJS tomorrow! Nobody will ever go to Py3!” Etc etc etc.
You got bored by it? That’s fine, it happens, no harm done. But you don’t need to convince anyone, including yourself, that you’re moving on because of this or that flimsy excuse.
Why do you assume motivations other than what the author claims? I very much started on Python, but as my career progressed, I got tired of being frustrated with poor library documentation and having no static types, making it extremely difficult to figure out even the parameter types to pass in, especially after it's been a while.
After switching to static typing, things have been a lot less brittle and even faster to develop. It is often said that Python is fast to develop in, but this is really only true for prototyping. Making a program and making a robust python program are entirely different, whereas they're much less separate in a statically typed language.
Also, statically typed does not imply FactoryFactory or no type inference, especially for languages like Rust, Swift and to a certain extent Go as well. The code written in Rust, Swift etc. could often be as elegant & short as one in Python, without loosing the benefits of static typing.
> use type hints or one of the many libs that enforce types...
As if that's pythonic. Heard of duck typing? And those are things you have to do yourself, as additional work and maintenance rather than getting it free because it's a feature of the language. And you don't always have full control over the stack.
Is there an IDE that will guarantee that no runtime errors will take place due to typing?
> Adoption rates keep exploding
Because python made its way into the rapidly growing machine learning market. Python is great for writing glue code/scripts because it's easy to extend with C. Everyone and their grandma getting into data science use python.
> I have no problem with that, just don’t try to sell it to me on technical grounds, because it looks very much like you have none.
Looks like you are just not willing to accept or even recognize any valid criticism of Python. This type of fanaticism towards the language is not the first time I am seeing it.
pycharm also uses the type hints that are PART OF THE LANGUAGE (aka pythonic) as of python 3.x and enforces them.
You cannot do with heuristics what you can do with static types.
Just goes to show you how incredibly messy the transition from 2 to 3 really is. Duck typing and typing hinting are two fundamentally different approaches of doing the same thing that rub against one another. That's one more thing adding onto the long list of things that make it a pain in the ass to migrate from python2 to python3.
It's possible to start using type hints without migrating to Python 3, and it's possible to migrate to Python 3 without using type hints. They're almost orthogonal.
It's weird to bring it up like this - do you have any experience migrating code to Python 3?
Coming back to Python in 2019, it seems almost mandatory to be on Python 3.4+. I can not imagine someone still using Python 2.7. Except for this one place I worked that was stuck in the last century, God bless their little souls.
So if you want people to be pythonic in py3, they have to make massive changes like replacing duck types with abstract classes.
I think it's weird that you don't question why vast majority of python code has no type hints. Do you have experience in Python?
I don't think I've seen anyone argue that Python 3 code without type hints is unpythonic. Except maybe the parent, but I think that's just unfortunate wording. It's explicitly using a non-standard definition of pythonic ("part of the language").
They're not at odds at all.
I find that on the internet, after proofs are given, more disagreement ensues. When one mind's is set, is set in stone. Proof or without proof.
Python's typing story is far from perfect and rather annoying compared to other languages. Python is good for interacting with the operating system, networking and other things.
But wanting a proper compiler is an honest motivation.
Personally I find type systems a lot more annoying because they lead to more verbose code and more cognitive load. There are studies that show bugs being proportional to the number of lines of code, regardless of language. And that effect seems to be more significant than the difference among dynamic and static type systems.
Defensive programming can be achieved through various means. Type checking is one of them and often can't replace Unit and Integration testing. Validation of Inputs or other data is another issue.
Of course they won’t replace unit or integration tests, but they will replace a lot of damn trivial tests you have to write to make sure you exercise all paths of the program, because there are no types to help you.
Also, at least when I think of defensive programming, I don’t think of how Haskell et al. allow me to more clearly express my intents in the type system. I think of the times where I write JS/TS and cannot rely on any data anywhere being in the format it says it’s in.
Have you forgotten the cognitive load of "what does this return and what can I call or access on it?"
> There are studies that show bugs being proportional to the number of lines of code, regardless of language
But that value isn't the same across all languages, only the trend.
Other traditional languages are adopting type inference, etc. We could be near an inflection point where we get statically type languages that are as convenient as Python.
I love Haskell computing model, but holy hell, why is so difficult using things like State Monad or even a trivial thing like a global counter  ? Why something so simple should be so difficult? Don't say it's because functional programming, Scheme/Racket is not that difficult!
Haskell and OCaml has their own problems, I agreed with you that both languages are more expressive but the code density is not necessary a good thing neither.
counter :: IORef Int
counter = unsafePerformIO (newIORef 0)
usesGlobalState :: IO ()
usesGlobalState = do
count <- readIORef counter
putStrLn (“the counter was “ ++ show count)
modifyIORef (+1) counter
Anecdotally, I find it incredibly rare that such a thing is desirable, when you could instead create it in an IO monad and pass it through to functions that need it. But, it’s there when necessary. And making this kind of anti pattern hard is a strength of these languages, not a weakness.
The popular statically typed languages which are actually Python's competitors are for example C++, C#, Java and the like.
Sure as hell they are more verbose. Even Typescript was extremely intimidating to me when I started to try and understand Typescript code bases.
And I am NOT going to try and teach Haskell or Ocaml as a first programming language...
For the 1% work that is actually engineering the model's deployment in production, it's not uncommon to use faster languages with the exported model.
If what you're doing is writing a vanilla CRUD server that gets thousands or hundreds of thousands of hits a second, Python is probably a bad choice.
Even a mere 1,000 req/sec adds up to 86.4M per day. That sounds like something at least as popular as Stack Overflow, for example. How many web apps are there, really, that have millions of users?
On the other hands, there's various APIs and tracking type of services that probably do handle well in excess of that, being run by not-large companies none of us have ever heard of.
If you need easy-to-use tools, R libraries cover significantly more ground than Python. You don't have to worry about 2/3 compatibility of niche libraries. Plus the functional language forces the libraries to have similar API's. You also get to use tidyverse and ggplot2 in all of their majestic beauty. At most you have to write a "%>% do() %>%" to get around weird library API choices.
However if you are in the frontier, you could avoid so much awkward notation with an array-first language like Matlab or Julia. You have "^" instead of "np.linalg.matrix_power", "/","\" instead of "np.linalg.lstsq", the . operator and so much more life-saving tools.
Ignore that reality if you want to, but it is a fact.
Big complicated python projects are seldom pure python, they are usually a friendly python frontend to a serious application written in something else.
It seems in no way remarkable that someone wanting to build a serious backend type piece of functionality would pick another language that was, just for example, multithreaded.
The point is that Python is the entry point for large majority of data scientists currently and its absolutely disingenuous to try to dispute that reality.
The key word here is "users". Python is fantastic for users. It allows users to get things done without having to worry about types or memory or the underlying hardware in any way.
I think Python made some really good choices with their types from UX that lead to you having to care more about your logic than the machine. Only one integer type, for example! For most users it’s perfect UX.
However, I think (possibly due to the early time when a Python was made) some of the decisions made practical but regrettable trade offs. Duck typing is ultimately just type inference with strong performance penalties and really weak static tooling, for example. _Many_ static type systems are still very far from the user’s actual domain, but the core techniques don’t have to be.
If you program in Python for performance-critical applications, as I do, then you will frequently hand-write C-level extension modules, write extension modules in Cython, use JIT tools like numba.
All of this stuff “is Python.”
R is a strang thing to me, though it has its moments (graphing). I still think pandas which gives python R like data frames is one of the great tools out there.
The sentiment that R is easier than python to install/manage is common among some R users but I disagree. With R I'm constantly facing dependency hell problems -- one package wants an old version of R, while another needs the newest. Conda/venv solves this problem very nicely in python.
Recently lI've been using Julia more and really like it. One nonstandard case where I've found it really shines is parsing large bioinformatics data, such as pileup files. Python is just so slowww here, but neither do I want to write a C program to do the text parsing. Julia is perfect in this case.
I can really understand his:
"Obnam has not turned out well, from a maintainability point of view. It seems that every time I try to fix something, I break something else. Usually what breaks is speed or memory use: Obnam gets slower or starts using even more memory."
Also from his current post:
"I could perhaps have been more diligent in how I used Python, and more careful in how I structured my code, but that's my point: a language like Python requires so much self-discipline that at some point it gets too much."
If the issue is one of fundamentals it will only follow them to the next project or language
Really, changing a Python project is not pretty, because speed and memory usage guarantees vary wildly between releases. One factor is that CPython is constantly being rewritten.
And that does not even cover projects that depend on 20 PyPI packages, which introduces practically guaranteed breakage.
Exactly. I have an impression that the programmers as the whole already spent significant time only to develop and maintain different solutions only to manage different Python versions and dependencies in production environments, which were maybe much less needed exactly with some better "discipline" in the Python development itself.
On another side, Python won many hearts over Perl 5 which is much more stable, but "appears to be" harder. Interestingly, Perl actually forces a programmer to "care more" (i.e. be more precise): the reference to something requires different notation than the direct use of something, the use of the array requires different notation than the use of the scalar etc... it actually has "some kind" of typing enforced by the language and explicit in every line. In my personal experience, I have much more "trust" in a 'biggish' Perl program than in a that big Python program to behave "exactly how I'd expect it" (e.g. in the sense that Perl doesn't produce as easy "exceptions" and whoever wrote it had to think more about the correctness than for Python, especially if "use strict" was used, which has stronger guarantees about the typos in variable names than Python). Going further, however, C-like (actually Algol-like) compiled languages simply allow much more than the scripting languages, regarding the smallness of what is eventually produced. E.g. Busybox ( https://busybox.net/ ) has to be written in C or something very close to C. Free Pascal http://wiki.freepascal.org/Platform_list is actually also very practical language for many use-cases.
I like Python well enough, have been using it since mid-2001 and was very much a hardcore Python enthusiast for a number of years, but nowadays I prefer languages that encourage immutability and functional programming styles, as I find that they make it much easier to avoid bugs. I've had plenty of Python experiences where bugs were caused because something somewhere else was mutating stuff behind my back, messing up my mostly-unrelated code. Usually this other code was also written by someone else, so I had no way of knowing about it. This kind of thing can be prevented in other languages, but requires a lot of discipline to avoid in Python (and other languages, I'm only picking on Python here since that's the topic of the article). These problems won't follow you to the next language, if the next language is more suited to avoid them.
I had read some stuff about Rust and it seemed like a interesting language, mature enough for my applications.
I tried it out and came to like it more than I would’ve expected. In fact I tend to use it over python for many things nowadays. One of these things is definitly how the module system works, but I agree with you: if you fuck up in Python with your code structure, the problem is definitly between chair and desk.
However: Rust won’t let you get away for cheap if you don’t think about structure, and this could certainly help some to adopt better practise (which in the end is good for all of us).
Interestingly enough my appreciation for and interest in languages like C and C++ grew a lot, after I had learned Rust. It gave me a good new perspective on C++ and certainly learnd me ton of good patterns and concepts that will definitly end up beeing useful in other languages. Rust certainly is well thought out and good ideas are never bad to look at.
This seems to be a rather shallow reading of the blogpost, and the way you phrased your reaction is not exactly polite either. If anything, the "adoption rates" of Python (like those of ECMAscript, a language of a similar vintage all-things-considered) are most easily explained by non-technical factors, so I have quite a bit of trouble making sense of your comment.
I've worked with the opposite who absolutely refuse to ever learn python/go/whatever, producing these awful thousand line bash scripts.
"our central finding is that both static type
systems find an important percentage of public bugs: both Flow 0.30 and TypeScript 2.0 successfully detect 15%!"
In fact there have been more than 1 study with the same conclusion.
So the argument is analogous to saying "how can I trust the types in this Java program are correct, if it has never been compiled".
Well, whether some "old code base" has them correct or not, just run the type checker and find out.
In my opinion optional type checking is not comparable to statically typed language guaranties. So if I do not need types I would use python. If I need/want type specification I would use statically typed language.
To me it sounds like circular reasoning, because in my subjective perception, dynamically typed code is a lot easier to understand to begin with. Also, tooling has become a lot better, with and without typecheckers/annotations.
"Despite their importance, whether static type systems influence human software devel- opment capabilities remains an open question. One frequently mentioned argument for static type systems is that they improve the maintainability of software systems—an often used claim for which there is no empirical evidence. This paper describes an experiment which tests whether static type systems improve the maintainability of software systems. The results show rigorous empirical evidence that static type are indeed beneficial to these activities, except for fixing semantic errors."
An empirical study on the impact of static typing on software maintainability
"We further conduct an exploratory analysis of the data in order to understand possible reasons for the effect of type systems on the three kinds of tasks used in this experiment. From the exploratory analysis, we conclude that developers using a dynamic type system tend to look at different files more frequently when doing programming tasks—which is a potential reason for the observed differences in time."
Note that I wrote "evidence", not "proof". Also, there is no robust evidence for safety improvements.
Are there potential confounders? You bet. Are there other aspects that can help? You bet.
I do find that types help me with older code, despite the fact that my preferred languages (Objective-C, Smalltalk), both have keyword syntax that helps tag arguments even if there's no types or documentation. Particularly protocols help me define the interactions that I expect in my system, which otherwise tend to be only defined implicitly through the objects. I'd prefer real connectors.
When I try to understand Smalltalk code, that's a lot of browsers open and digging etc. When I look at some of my older Objective-C frameworks, all the id types do leave me with a bit of head-scratching. A lot of the times the types are trivial to add, and just make things obvious at a glance.
However, I also totally agree that these languages help me write cleaner code, and that enforced type systems get in the way of useful architectural abstractions. Sometimes I need that id.
For me it's bit rot. My Python is approaching EOL on this version and needs upgrading to Python 3, and I have a lot of 3rd party code in here that made things simpler to create but over time they've changed enough or disappeared that I now have to take on work there too. Then Django and it's internals have changed a lot in the last 6 years.
My Go code written at the same time (6+ years ago) hasn't had any of it's libraries change so much that I've needed to do anything. Every now and then I pull the next major version of Go and recompile and it's done.
This is all side project stuff but a lot of people use it (monthly average visitors being +250k)... it's non-profit and I don't wish to have to maintain it much.
It worked once, if nothing has changed in the functionality it should keep working... but this isn't as true in non-compiled languages where you need to depend on so much of the environment, let alone build culture that doesn't fully vendor dependencies.
I like Python, but prefer to write code and move forward with other projects.
As to Rust vs Go. I code in both and only picked Go here because a lot of the code in another project I worked on can be used here. I'd probably try Rust if I was starting from scratch now. But I'd be a little concerned bit rot would affect Rust too, it's still quite early for the wider ecosystem.
And good look finding replacements for Python/Django packages in Go or Rust.
Don't get me wrong. You'll find a lot of libraries for those languages. But probably not everything...
They explicitly said they don't want to rewrite. It looks like they have some existing projects in Go that work in a self-contained way and upgrade with minimal effort.
9 years later, still works.
It's perfectly possible to build a python application that is extremely reliably tied to its dependencies, you just have to avoid the default python package "management" solutions that get shoved down everyone's throat (pip, virtualenv, pipfile, tox and the like). I've been building python projects on top of Nix for years and have managed to maintain stable and reliable results.
Go is mediocre as a a language, but it has one big edge - if you're doing web backend stuff, the libraries for that are the ones used internally by Google. So they're being executed billions of times per second on Google servers and tend to be well behaved. Python tends to have multiple libraries for basic functions like the database interface, all with different bugs. If you write web backend stuff in Go, it will probably work.
Having suffered through the Python 2 to 3 transition, I'm less of a Python fan than I used to be. At some point, they need to stop adding features. Currently, Python is acquiring unchecked "type hints", which look like a terrible idea. Optional typing, sure, but unchecked type hints? If the compiler enforced them, they might be worth something. (And no, running a separate type checking tool is not the answer. Especially since no such tool exists that's production grade and in sync with the language.)
Rust has no idea when to stop adding features. Rust started as an imperative language and then morphed into a semi-functional language. Mostly to get a sane error handling model. The borrow checker was a brilliant innovation, though. I used Rust for a while, but quit about two years ago in disgust. I never want to hear "that only compiles in nightly" again.
I don't understand how you can reconcile these beliefs. They stabilized too many features, but you also want more nightly-only features to be stabilized?
It's my experience as a Rust programmer since pre-1.0 that very few libraries require nightly; they tend to be niche things (e.g. Rocket) by authors that prefer to experiment on the bleeding edge of the language over having stability or many users.
All it takes is that one library you want to use requires Nightly. Since Rust adds features all the time, it's not uncommon for library authors to start using these features before they're in Stable. It's just the nature of a young language with an active community who likes being on the bleeding edge.
This also extends to the toolchain; it's not that long ago that RLS and Rustfmt were still Nightly-only, for example.
The presence of Nightly alone gives you a choice that usually doesn't exist with other languages. Not too long ago I went down the rabbit hole with some text stream stuff (Unicode is surprisingly poorly supported in places), and ended up with a temporary workaround because the std::io::Chars API wasn't in Stable yet and I just didn't want to deal with Nightly. But it was tempting. With other languages such as Go, the choice generally isn't there, even though you could theoretically download HEAD and go from there.
This is a classic case of our standards of success continually moving, and that some domains are better suited to stable Rust than others. For example, if you want ergonomic async/await, then you need to use nightly. But otherwise, the number of crates needing nightly has shrunk enormously over the last few years.
> it's not uncommon for library authors to start using these features before they're in Stable
I would say this is actually pretty uncommon.
So are Java and C++ programs at Google. golang is very mediocre for webdev, if anything, it can be used for small network tool kind of stuff. For anything that needs to be more complex, Java would be a better fit (e.g. golang doesn't have annotations that can be used for validations or automatic mapping between different internal and external types, which come in very handy and prevent bugs).
What does "production grade" mean w/r/t such a tool?
After working on a quite large application and having to use lots of assert(isinstance(arg, type) at the beginning of almost every function, I began to think that a strong type system is very much needed for large projects. I believe this was one of the reasons why Twitter moved from Ruby to Scala.
I still love to use Python for hacking something quick for myself. But I also look at some popular strongly-typed languages now and hope to get better at one of them soon.
You’re confusing static/dynamic and strong/weak. Python is a strongly typed language, but also is dynamicly typed language.
Need proof? Try doing this:
a = 3
b = “3”
c = a + b
> having to use lots of assert(isinstance(arg, type))
Python has type hinting now, try using a current version of Python and this isn’t needed.
For example, it doesn't have a built-in way to make a dictionary-whose-keys-must-be-integers (so with your example above if you wrote thedictionary.get(b) rather than thedictionary.get(a) you'd get an error rather than None).
I've often seen learners have trouble with this sort of thing (you read a number from a file, neglect to call int(), and get mysterious lookup failures).
Technically, dict.get() returns None if the item isn't found and you don't provide a different value for its default.
thedictionary[b] would throw a KeyError on the other hand.
One could always subclass dict if they wanted to ensure the keys were always integers or whatever.
> I've often seen learners have trouble with this sort of thing (you read a number from a file, neglect to call int(), and get mysterious lookup failures).
Not sure if you're arguing for a stronger type system or having the language promote the string to an int as (I've heard) other scripting languages do.
Though, once you figure out the language doesn't automagically parse files for you you're well on the way to finding more fun and exciting ways to get python to throw error messages at you.
Type hinting does not guarantee to catch everything, but for Python proponents 95% is apparently "good enough".
It's definitely a neat feature when writing an application from scratch that interfaces with few other modules or only the well-tested ones.
Python should not be written like this; assertions can be ignored at runtime: https://docs.python.org/3/using/cmdline.html#cmdoption-o
A much better solution is to use qualified try/except blocks to handle potential errors.
The most irritating thing I've found is trying to write a function that accepts either a path or the raw contents of a file.
Python 3 makes this easy, you can check against string. Python 2 of course treats everything as a byte string.
In the end I just check the length of the input if the interpret identifies as python 2. If it's a short byte string, I assume it's a file name. Otherwise the minimum expected file size is almost certainly larger than the maximum path length.
Another case is where there are two or more parameters that you want to be generic in this sense: if your only option is duplicating the function you can end up having to define inconveniently many variants.
Python's optional typing could also be used to get rid of a lot of your assertions.
For personal projects (and past consulting work) I have been loving Lisp languages since the 1980s but many languages have so many of the benefits of Lisp (thinking of Haskell and Julia) and there is definitely a ‘lisp tax’ on deploying and shipping products.
Anyway, for writing new books, writing open source projects, and perhaps some consulting, I would like to cut back to a single language, instead of the 4 or 5 I use now.
EDIT: I did some Go Lang codelabs when I worked at Google and have played with Rust (principally spending some evenings with an open source blockchain project) and neither language will be my home base language.
Elixir is perfect for servers and fault-tolerance. Also scales quite well to several tens of machines before needing any special tooling.
For a huge percent of most projects today Elixir would be ideal.
OCaml I want to use more for its really concise and strict (but extensible) syntax. And mostly for its strong typing and really well-optimized compiled native code.
Congratulations on retirement, relax and tinker at your own pace. ^_^
And enjoy your opportunity to work with whatever languages you like. The variety of tools available today is so much greater than it was in the 80s.
Anyone else agree with this view ? Programming languages should be choosen based on technical merits, rather than who is behind it. Is there a lesson from history that I am missing ?
Considering the motivations of the language owner is a valid concern. Many people, for example, jumped on to HHVM as a "faster PHP". Then it started branching away from PHP. Predictable, if you considered Facebook's reasons for having it.
Go is quite unique in that it produces very independent binaries. This might make software written in it quite future-proof.
Personally I like Python's philosophy and the way the community is run. I like the trajectory. I understand the trade-offs. And I am sure as hell not going to teach Go or Rust as a first programming language to people who aren't computer science students. And even then...
What do you mean by "it produces very independent binaries."? Do you mean that Go implements system calls itself, instead using corresponding wrappers from the OS's underlying C library?
I read something like that recently, but haven't looked into the point yet.
Absolutely not. Technical merits are only part of the considerations.
Big companies, for example, where the stakes are higher, never chose languages on their technical merits alone. The also look at the owner/stewardship of the language, and in many case, roll their own languages, to avoid being at their mercy.
So it’s not a completely fair comparison.
It got fully open sourced?
This can be both a blessing or a curse. Google seems to be on the good side, but I’ve seen plenty of languages which are frustrated by its steward.
To me the main advantage of it is that the small number of primitives and simple syntax forces people to in general be very explicit about what you’re doing. The code almost doesn’t need comments.
That has little to do with 'simple syntax', and more with the language itself (and/or its community) setting clear standards as to what "idiomatic" code in the language should look like. And Rust is fairly good at that, albeit with the inherent limitation of having to support the additional complexity of ownership/lifetime management. (Go can do without this since it uses GC instead. Swift just uses ARC everywhere.)
Rust is quite the opposite of Go in that regard - just like C or C++, it makes it really easy to write over-abstracted and clever code that is hard to understand, audit or review.
Rust is also an extremely powerful tool and a ray of hope for systems programming, but comparing it to Go is an apples and oranges comparison.
I've seen people copy protobuf definitions from repo to repo because they couldn't get them building consistently from a different directory, never mind issues with different versions of the go protoc plugin.
If you are writing code that must work, you should be either formally prove the system (if feasible) or have a huge test suite covering everything.
While true, it's a big benefit to eliminate classes of known bugs (e.g. memory safe languages eliminate undefined behaviors, and Rust eliminates data races). That allows the programmer to pay attention to other aspects of the code, since there's only so much energy one person can spend.
You are talking about systems that SHOULD work, but aren’t required to always work. Typically, safety-critical software is the latter, the rest of software is the former. But the latter are entirely different beasts, and is what we were discussing.
By the way, Rust does not eliminate all race conditions; and memory safe languages may still have undefined behavior unrelated to memory (e.g. Rust)
I find it a pleasure to organize code in python.
What am I missing here?
Obviously this doesn’t replace the need for functional tests but writing those tests is as much prone to user error as writing the code itself. So having a compiler check for stupidity up front is always going to be a bonus.
Also the static type system helps a lot too. Having dynamic types can be awfully convenient at times and I completely relate to why some people like them but static types means your structures and classes effectively have written specifications defined in the code itself which the compiler verifies the rest of your program against. So if you’re accidentally trying to write text to a numeric type, your compiler will catch that.
In my experience the extra up front overhead (in terms of development time) of static typed AOT compiled languages end up saving you development time in the long run when working against larger projects.
On other hand dynamic languages give you flexibility and speed when developing. In some cases that is worth a lot more then losses you suffer from them.
Having said that Rust is just terrible application language. It was built for a purpose use it for that purpose.
From typed perspective from what I have used recently Swift is probably my favourite.
Here comes the catch, libraries, community support all tramp the hell out of almost any language feature.
There is a reason we write metric ton of C++ for everything and basically everything Python has C++ and C under the hood. C++ has stuff to do it all, it has been tested polished, tested some more documented and works. Now even Pytorch has first class C++ API.. It’s about time!
Java traumatized me as a young engineer fresh out of school. I can’t bring myself to even look at it these days. I am sure it is much better now. I just can’t do it.
In my experience, if I'm writing relatively decent tests and sprinkling of sanity checks these types of errors get caught quickly and are very obvious.
I can sympathize with somebody who has largely worked with untested statically typed projects and has moved to untested dynamically typed projects though - this is where the benefit of compiler type checking might seem to be most valuable.
>In my experience the extra up front overhead (in terms of development time) of static typed AOT compiled languages end up saving you development time in the long run
Yeah, mine doesn't.
I do agree that tests are important; hence why I raised that point from the outset. But ultimately in Python you’re still having to add extra tests just to catch up with the default behaviour of Go and Rust. Which is the point I was raising. Go and Rust are better at catching stupid user error.
Clearly though you can write large projects in Python if you’re disciplined enough. Same could be said for C, BBC BASIC or even assembly. The question isn’t whether you can, it’s whether other languages makes it easier or not.
You don't though. The tests you'd write which would catch these "stupid user errors" are basic ones which you'd be remiss not to write in any language.
The fact that frequently developers do choose not to write any tests at all on large code bases does mean that they start to rely heavily on compiler catching 'stupid user errors' but it doesn't have to be that way.
>Go and Rust are better at catching stupid user error.
...if you don't have a basic level of tests, yes.
(although rust does provide certain type level protection against, say, data race bugs which python doesn't).
Positive being that the code does what you expect under normal operation. Those you’d obviously need in any language
Negative tests are, amongst other things, checking that your functions behave correctly when you put garbage in. If your compiler flat out refuses to send a string type to a function that is designed to accept an integer then you’re already covering a whole class of bugs without needing to write specific tests for it.
The straw man of my argument living in your head flip flopped. I did no such thing.
When I said "basic tests" I meant demonstrate the behavior under normal circumstances". If you're doing TDD on 60% of your code base you'll write enough of these.
>Negative tests are, amongst other things, checking that your functions behave correctly when you put garbage in.
This is a completely unsuitable use case for a test. If your function expects non-garbage it should raise an exception if it gets garbage with a sanity check.
I wrote one just a few hours ago like so:
The point of functional tests is they catch developer errors before your codebase hits production. Which you cannot guarantee if your error checking is only in the runtime.
Disclaimer: I write services used by millions of people (literally) each day. Some of those services are written in Python.
That’s a terribly unsatisfactory answer.
> In my experience on a half decent code base these sanity checks never trigger during production in normal code; they're designed to get triggered during tests. They're there to make it easy to track down the root cause of bugs.
There are a lot of conditionals attached to your statement. ;)
> The times when they did get triggered on production code I did it intentionally as a way to debug a train-wreck of an application in production.
If you hadnt relied on type checking in your runtime then you might not have needed to get to the stage where you were debugging something in production.
> I've been doing this for 10 years - on many projects that have had millions of users. You're not special.
I didn’t say I was special. I was just making a point that I have used Python on important services just in case I was coming off as a Python hater.
Btw ive Been doing this for 30 years - though most of that hasn’t been in Python but most of the last 20 years has been high availability systems. Which may explain why I’m less willing to push buggy code to production and rely on exceptions than you.
I honestly don't think you know the first thing about writing stable software so are now just lashing out at me.
Basically (and this goes for both of you as well as the rest of us), when you find yourself unable to keep from replying as irritation mounts, it's best to wrest oneself away from the internet for a bit.
I agree. I said this too
> and the ones you shouldn't be writing (if you do, simply switch to another language).
That was my point
Humans error. I’ve seen numerous occasions when stuff that should have been added into the tests wasn’t. Even from engineers who are normally thorough. I’ve also seen cases when competent engineers have written test cases that pass when they should have failed. Heck even I’ve done that and I’ve been writing software for 30 years.
The more a languages build tools can catch developer errors, the less you need to rely on developers remembering to check for them and thus the more time developers can focus on the project at hand.
If your devs are pushing code into production that doesn't pass the tests (or they aren't even running the test suite) then you're asking for trouble no matter what language you use.
As much as I love reading 20+ pages of errors c++ likes to spit out when you use a template wrong if I want to do something simple I just reach for python.
Clearly that’s not even remotely what I said. But I’ve posted the same thing twice already. I see no point posting it a 3rd time.
Sure, yeah, if you want to ensure type safety then use a type safe language...argument over.
...unless you want to do something like duck punch some function into someone else's library and you really don't want to rearchitech how the whole thing works because of that one special case which just doesn't quite work.
That said, I’ve been on projects debugging code that has had functions overridden like that and it wasn’t fun. Sometimes people can get too clever for their own good. But you can have that problem regardless of the language
This is because what you call "systems" is not the "systems" that the lack of GC would benefit from. For all intents and purposes Kubernetes could be written in Python or any other general purpose programming language (and for example OpenStack is in fact written in Python).
On the other hand, systems like "writing an operating system kernel" or "running on a microcontroller" are things where GC might indeed be problematic and/or impractical.
Also, writing self-referential data-structures is very difficult in Rust (but not impossible) due to the type system. I suspect that a lot of developers who just want to get work done get turned off by fighting with the typechecker, and choose GoLang instead.
And also: GoLang is a very accessible language, that can be mastered in a couple of days.
Not really true (at least, not nearly to the same extent) since Pin<> was stabilized. I suppose that only serves to prove your point about Rust still being "in research phase", though!
Other things like the situation with async/coroutines is still it great in rust.
Go is, syntactically, a much simpler language than rust.
It all depends on what you need, what you want, and where then community around it is (e.g. Docker came first and was written in Go).
You'd be surprised.
Kubernetes may as well have been written in Python. It's mainly IO bound, so the benefits of compilation as opposed to running on an interpreted VM (like Python) aren't that big.
Now I am not arguing one way or another. People can just have opinions or they can just write software or they can do both.