The author essentially states that he just wants to build a website and "be done with it" and never have to touch it again. He seems to acquiesce that security updates are necessary, and I suppose his greatest complaint is that old versions of software (eg python 2, django 1.11) are no longer receiving security updates, which necessitates him to update to python3 and django 2.
I feel that. But on the other hand, without trying to sound too mean: what exactly does he expect? Is he also complaining that "Windows's goals are not our goals" because Microsoft stopped supporting Windows XP? Does he complain that Ubuntu version 11 is no longer getting updates? Java SE 9?
For better or for worse, this is simply how software works. Things have to be updated. Unless you want to eschew security updates (which is not smart), there is never such a thing as a fully "finished" product. Software maintenance is part of the job responsibilities.
This post is particularly targeted at Django, but I feel that it applies to almost every web framework or piece of software, and I'm afraid this is unfairly casting Django in a bad light. I'm also afraid that we're going to see a lot more posts like this in the second half of 2019 as we move towards python 2's EOL date.
Yep. I was interested in the post as I've always felt Django is python software "done right". I stopped reading when they started complaining about python 3.
Python 3 has been out for over 10 years now. Did OP really think Django was going to use python 2 forever? Frankly that's a selfish outlook, and I'd go as far as saying it's a cancer of software development.
Backward compatibility needs to be a thing, but not forever. Things need to be deprecated eventually.
> Backward compatibility needs to be a thing, but not forever. Things need to be deprecated eventually.
This mentality is the true cancer of software development IMO. Can you imagine the utter insanity and chaos if any other form of engineering worked this way? Programming languages are like protocols and standards: they are a common platform that people build on; simply deprecating them with no plan for backwards compatibility requires everything to be rebuilt, which represents a tremendous cost. Sometimes these changes are worth it, but outside of software, these are usually once-in-a-lifetime changes.
I can't think of any fundamental technical change between Python 2 and Python 3 that make this scale of disruption worthwhile. If the maintainers of Python don't want to maintain Python 2 anymore, they are completely free to make that decision, but it's a decision worth criticizing and complaining about. It is not a good thing for all software developers to be forced to rebuild the same sand castles every time the tide goes back out.
But big changes do happen. In 1948, the west coast changed their power standard from 50Hz to 60Hz[1]. Or changes don't happen, Japan still has two standards[2]. Or, like metric in the US, it slowly changes but likely will never completely happen.
Sure, but the pace of these changes is significantly different. Clearly part of this is due to Moore's law; perhaps standards in aeronautical engineering shifted about as quickly during the transition from biplanes to supersonic jets as our software standards and frameworks are changing today.
But that's no reason to reflexively dismiss the very good and very interesting question of why one should have to largely rewrite a Django app in 2019 that does the same thing that it did in 2010, in a language that is virtually functionally equivalent in any respect that is relevant to that particular developer. Software does not inherently need to be as ephemeral as it is.
This is a scope problem. If thousands of people were constantly attacking your door's locks you better believe that locks would constantly be changing and that a house with older locks would need to have them upgraded.
Changes to physical items are costly and not affected by the same types of security scope that _any_ code connected to the internet is. The code in my decade old microwave has never needed to be upgraded and probably never will be upgraded, but it's also not under constant attack.
His Django app continued to run, he just couldn't make use of the most up-to-date features and security unless he upgraded.
I get the need for security updates. That’s why you use high level frameworks to write your application and update the rest of the stack that runs underneath. The issue is when parts of that stack are replaced with backwards-incompatible-but-functionally-equivalent parts, for reasons that have nothing to do with security.
But that's why you have to look at the maturity of the tech when first building it--sometimes you just don't know. Python 3 was first released in 2008. Python 2 EOL is 2020. If I built an app in 2010 using a Python 2 stack you have to know by 2019 you're going to have to rewrite it completely using a new stack or modify it heavily (assuming Django would move to Python 3 and be active 10 years later).
If you had built your page using static html there would be a lot less surface area of attack and a lot less maintenance. Even with Django, the amount of churn depends on which features you use.
You hear similar stories about people chasing the new Javascript framework treadmill. Some people stuck with Vanilla JS, others are still using jQuery.
I just don't see the real world being all that different. Blackboards still work just fine like they did 100 years ago. Some schools transitioned to marker boards, and some moved again to "smart boards." Many were forced along because repair/replacement supplies were harder and more expensive to find. I see the same thing with washing machines or people's favorite pen/pencil.
In fairness, I suspect the author would agree that it was a mistake to use Django in the first place. I think that's what he was trying to get at!
> Python 3 was first released in 2008. Python 2 EOL is 2020. If I built an app in 2010 using a Python 2 stack you have to know by 2019 you're going to have to rewrite it completely using a new stack or modify it heavily (assuming Django would move to Python 3 and be active 10 years later).
In 2010, the conventional wisdom was still, "nobody uses Python 3, there is no library support for Python 3 so just use Python 2". Django itself didn't support Python 3 until 2013.
In hindsight he can say it was a mistake, but it was glaringly obvious back then that Django, Python 2, or Python 3 wouldn't give you a situation where you would have minimal maintenance for the next 10 years because of the volatility already publicly discussed.
But maybe you're right that this isn't something people think enough about when choosing technologies and the author is trying to raise awareness.
> Backward compatibility needs to be a thing, but not forever. Things need to be deprecated eventually.
That might not need to be the case. I guess we'll see: one of Rust's tenets is perpetual backwards compatibility. New features (and potential breaking changes) are opt-in with a per project configuration setting.
The author acknowledges that some maintenance is necessary, especially for security updates. My reading of it was that his complaint is that in keeping up with the security upgrades, Django breaks other things.
This is true of a number of frameworks I've worked with. If you want security, you have to constantly stay on top of the other changes that the framework makers implement, often "fixing" things that aren't broken, removing features, or changing the way things work.
I don't know how practical it is, but it would be nice to see a framework where security updates and feature updates were separate things. One could keep one's site secure through updates without having to debug formerly working things every few months.
> I don't know how practical it is, but it would be nice to see a framework where security updates and feature updates were separate things. One could keep one's site secure through updates without having to debug formerly working things every few months.
This is more or less what an LTS release is. The problem as I see it is that, as the codebases diverge, it becomes more and more work for the dev teams to support what is now essentially two different products. And especially in the open source world, it seems futile to expect unpaid or low-paid contributors to continue to do extra work simply because someone else's dev team doesn't want to put in the work doing updates.
> My reading of it was that his complaint is that in keeping up with the security upgrades, Django breaks other things.
Django takes great pains to ensure that this isn't true. We don't backport anything other than bugfixes and security updates to releases. It's not like we suddenly introduce a new feature into a previous release that's backwards incompatible with your current code. If you're on the LTS release then you can be happy in upgrading Django to the latest point release with all the latest fixes.
And the Django project has finite resources. It cannot backport security fixes indefinitely which is why we have the LTS release in the first place. If you want something stable: sit on that for several years at a time, or spend your time backporting any security fixes yourselves.
Unfortunately software is a moving target, but Django tries to make it easy to upgrade from one LTS release to the next. The upgrading difficulty often comes from third party packages, which is hard to avoid in any framework or language.
Unfortunately it's not practical because the test matrix explodes. They would have to develop and test every security update on every previous version which is quite costly, especially if the codebase has diverged.
The best strategy is probably what many large distros like Ubuntu have adopted: designate only a few long term support (LTS) versions, and commit to supporting only those versions.
The only matrix i support now is latest release and master, not even LTS anymore because it quickly gets too old.
I consider not updating my software to be tech debt: something I must do before adding a new feature.
In return, my software is always up to date and ready for new features, except once in a year when I do the said upgrades.
There is a saying amongst growers of bonsai trees: A tree isn't "finished" until it is dead. In other words, it keeps growing and so you constantly have to keep working on it. You can't just think of a design, prune the tree and then say, "Oh, I'm done! Let it stay like this forever".
Software is exactly the same. Your needs may never changes, but the world around you does. You have no control of that and so you must constantly maintain your code -- even if there are no changes that you want.
It's not just Python or django. It's hasn't even possible to compile Ruby 1.8.7 with recent versions of GCC for at least a year (don't ask me how I know... :-P ). Starting a software project -- any project -- is a long term commitment. If you want it to survive, you need to care for it.
>I feel that. But on the other hand, without trying to sound too mean: what exactly does he expect?
It's not unreasonable to expect to be finished with software.
>For better or for worse, this is simply how software works.
This is false. The author has already fallen into this trap by using a programming language which is also subject to this, however.
>Things have to be updated. Unless you want to eschew security updates (which is not smart), there is never such a thing as a fully "finished" product.
There is such a thing as finished software, but I can understand why something such as a WWW framework wouldn't be. The WWW is ugly and misguided and inspires disgusting systems to accomodate it.
>Software maintenance is part of the job responsibilities.
When you have a project with no real design goals, apparently, I agree. Otherwise, you can remark on how a program achieves its goals and so is finished. That doesn't stop minor improvements or reworkings, but those would be entirely unnecessary.
>This post is particularly targeted at Django, but I feel that it applies to almost every web framework or piece of software
I agree, because, again, the WWW is a disgusting tangle of systems that encourages similarly disgusting systems.
One language I enjoy is Common Lisp. The standard hasn't changed in decades and there are many implementations. I can finish a library and then never need to touch it again. Python can't compare.
> For better or for worse, this is simply how software works.
This post is more or less a lament about that fact, not necessarily a complaint or gripe. Chris is a Sysadmin, and there’s a big difference in perspective between this and a developer of an app whose reason for being is THE APP. A Sysadmin needs to provide a bunch of services that are not the core revenue generator, and multiples of them, meaning that every internal app has this same kind of updating overhead. It becomes tiresome to have to keep all of them up to date all the time. We know that is a fact of life, but not all facts of life are fun and people like to blow off steam about that.
My favorite part of Backbone.js is that its development as a framework is essentially “complete” (it works as designed, functions reliably / predictably, and the maintainers aren’t running in circles trying to reinvent the framework with breaking changes)
That said, I feel like I’m losing “street cred” for admitting that we still use Backbone.
It would be great, at least in my opinion, to see more frameworks like Backbone.js (with respect to reaching a “framework is complete” state).
We did some client work up until about 2 years ago with Backbone, too. We loved it. The entire team knew it very well and it got the job done. At least a few developers at the client would also be familiar with it every time, so it was a win win.
The author writes that they have a particularly big piece of maintenance coming up.
> The latest change we need is an especially large amount of work, as we will have to move from Python 2 to Python 3.
This sounds like the transition from Django 1.11 to (probably) Django 2.2, if they're moving from LTS to LTS, and it definitely is a big piece of work if you also have to change python versions.
Django defined a release schedule DEP[0] a few years back to try to preference stability and reduce the amount of maintenance when moving between LTS versions, which is approximately every 3 years.
If your application runs on an LTS version without any deprecation warnings, then you should be able to move to the next LTS version without any changes. Which is nice in theory, but 3rd party packages in particular aren't always aligned in such a nice way. If the stars align though, it is 6 years (LTS + LTS) of no changes.
This is all to say, I think the author has a point, but that Django has recognised the maintenance burden as an issue, and has tried to help developers with that as best they can, without just shutting down all progress. I think the balance is mostly right.
Moving from 1.11 to 2.2 should be mostly painless. Unfortunately, moving from Python 2 to Python 3 is going to make that more painful. From 2.2 the maintenance burden should be much less than in previous years. Stick with Django if the features meet your needs.
Every once in a while someone opens an issue on a repo i maintain asking for "Django 1.11" support: 2 days ago, 20 days ago ...
For me, "LTS" is a mistake indeed and and goes against any Continuous Delivery / Agile philosophy that I know of, it serves as an excuse for users to not update their software which in reality accumulates tech debt.
I mean, if you're going to do very little customizations to a system then LTS might be fine, but if it serves as a base to something that's custom development from scratch, like Django, it really doesn't look like a choice that's going to pay off in the long term.
As a library author that’s well within your rights to determine what you’re willing to support or not.
Django has a very big user base. There are many users working on “very large sites” where the cost of “churn” is dramatic. There are threads on Django-developers right now complaining that the current model is too aggressive. Striking that balance is difficult for a heavily used framework.
One of the big sites that I maintain was built upon a framework that was fine for its purposes five years ago. But the developers of that framework have decided to take it in a direction that is incompatible with the site's goals.
Once I'm done with my current project, my next task is to rebuild that site without its current framework, and there's a 95% chance I'll go without any framework at all. Just plain HTML5 + PHP + MySQL + CSS3 + a very small amount of javascript.
I know it sounds primitive for a 700+ page site with interactive elements supporting English, Spanish, and Chinese; but earlier this year my boss gave me permission to de-framework a similarly-sized site, and it worked out really well.
Frameworks are great for many projects, but not all. It's important to weigh the risks of being at the mercy of someone else, and that organization may eventually adopt a different vision of the future. Their pivot will not make your life easier.
Would a microframework (no idea what's the equivalent of Flask or Sinatra is on PHP) help? I still have nightmares from my beginner days of doing everything in plain PHP.
Would a microframework (no idea what's the equivalent of Flask or Sinatra is on PHP) help?
I have no idea. Maybe. I've just been coding straight PHP. With the way IDEs are these days, once your environment is set up the way you like and you have your favorite macros bound, it's not a problem.
I'll look into Flask and Sinatra before I start the next site. Any other ones you recommend?
For PHP I'd recommend to take a look at libraries that adhere to the PSR (Php Standard Recommendations). It is very easy to build your own microframework with them and add only what you need and nothing else. For the libraries themselves look at the League of Extraordinary Packages: https://thephpleague.com/. Zend and Symfony also have PSR compatible libraries you can use. They are no longer the big monoliths of yesteryear.
Highly recommend you give CherryPy a look. Its older than most Python web frameworks mentioned (Flask and Django included) and was the most beautifully backwards compatible framework. In spirit of migrating to Python 3 they are obviously dropping old versions in a reasonable time frame though. I wouldnt recommend Python 2 for any new projects regardless.
I used it for 3 years nonstop. It is minimal so you include your own ORM of choice and template library.
Flask would be my personal preference but I have a Python background (you mentioned PHP so I assumed that was your main language).
Sinatra is in Ruby. The "hello world" looked very similar to Flask but I haven't explored it deeper so no idea.
For PHP maybe also check out Phalcon ? It's not as "light" as the aforementioned ones but maybe they're more conservative in terms of breaking changes?
This is endemic to the tech world. So much of it is invested in a reflexive forward momentum that is inimical to at least some of the products it facilitates. This is very much true for example of many pretty slow-moving small-scale line of business mobile & web apps. So many of them end up being a waste of money after initial launch as the platform outruns them, and further work becomes, as the author here suggests, pure overhead.
It's among the reasons I tend to be deflationary when being consulted with about extending web or mobile presence by small business. Often I just suggest they don't bother, or do something far more minimal than their first impulse.
This is a need I’d like to see addressed more broadly in software. The computing needs of many in my non-technical entourage have not changed meaningfully in the past 10 years - they crop photos, write letters, put together videos of family holidays, do basic spreadsheets, etc - and yet every few months I field support requests because their software has updated and some feature has gone missing, or the UI has been redesigned, or it’s not compatible with the new OS update, etc.
Bug/security fixes and new format support (eg a photo editor probably wants to support HEIC in the 202x) are fine, but we don’t really need the user facing part to change. This is a niche that seems open source could fill nicely.
(The big edge case in all this is the web browser)
Open source software has always had a major problem with user experience (as well as aesthetics).
Most people working in those projects come from a strong technical background, and in many cases users come from that background as well - so there's not much of a push for improvements in areas like ux, nor a business need to improve to boost sales.
There's a business opportunity here that Rails people already caught onto: you can start a 3rd-party "Django LTS" service that backports security fixes, and charge money for that.
I’ve previously suggested this as something the DSF could do as a money maker, but I don’t think the operational complexity makes sense for a non profit foundation.
I’m surprised no one else has picked this up as a business idea though. The actual technical work likely wouldn’t be very hard.
I think complexity here is migration to Python 3. Seems like the author picked Django and Python 2 in 2011. Python 3 was released in 2008. So by 2011 should have been clear that you might be facing a migration project sometime in future. Having made that choice getting 8 years out of Py 2 is pretty good. (Not saying he made the wrong choice. Would have made the same choice myself.)
I'm aware of the well-established movement to static websites for (mostly) static sites, but what work is happening right now to counter this problem in rich web applications? Are there frameworks that take a stance that favors stability and maintenance?
Frameworks aren't the design choice that favours stability. You take the inversion of control to get all the convention/plug-and-play/just-works features that come with it. Those are focused on developer productivity, which is a constantly shifting goal.
Avoid that design and using a "libraries over frameworks" approach solves this problem but has other tradeoffs obviously, although a good ecosystem of libraries that have designed for interop solves most of them. These library ecosystems aren't the norm though, and the lure of "I'm going to package these together in a framework, that does things the way I like, and then brand it and help my career" for the library devs is always there (see Flask in Python for example, it was the "unDjango" until it got popular and now it's just another framework)
I think the move toward micro-services (and serverless and whatever other buzzwords you want to use for a distributed loosely coupled design) is an answer to that problem, if not exactly the answer you're describing.
The idea being that each small piece is easier to leave alone for long periods of time, and, should it ever become impossible to leave it alone, whether for security reasons or something else, updating it or replacing it is much easier than doing the same for a big tightly coupled pile of components.
Different frameworks and language run-times move at different paces. It's very often the case that projects don't match their product pace and life-cycle to their toolset pace and life-cycle. "Standards" be it ABIs, APIs, data formats or markup languages all have different paces and life-cycles. Then there are the paces and life-cycles of the stacks underneath, the OS, the hardware, networks and data sources.
Then there's the security question. If this is a concern at all, ultimately the system may need to be air-gapped (e.g. like many systems depending on Windows XP are these days) to safely keep it running in lieu of fixes.
Another approach is common in the gaming scene with painstaking work on timing and bug-accurate emulation to keep old software running.
Ultimatley every system has a cost per annum, not a static capital cost. The end game being to maintain the hardware and "firewall" or update the software.
The author isn't considering an alternative strategy, which is to address security issues in-place (i.e., without upgrading Django). This means keeping up on them and substantively addressing them through configuration or possibly custom code, but for something they want to freeze in time with respect to a platform, it's a reasonable choice.
Every time I've had security issues to address, there's almost always a short term fix (close a port, disable a particular eval method). In the cost/benefit analysis of maintaining a legacy app, it's a legitimate option to simply rely on those.
Take a day or two to do your upgrades once in a year and you'll be just fine.
Start upgrading your own plugins as soon as possible, so that you can learn how to practice the upgrade steps.
Python 2 to 3 was hard for me for the first few codebases, but then piece of cake for the other dozens of codebases I have upgraded.
The author has opted to use Django and would like to get to a state where their app is essentially "completed" -- where it will work fine without having to touch it again or make major changes.
The conflict arises because Django has a stated mission to always be improving (like a perfectionist). This causes issues for the author when they have to re-write parts of their application in order to keep up with updates to the mainline framework.
The author would like be able to get things like security fixes and performance improvements without breaking changes -- whereas the Django team is willing to require breaking changes in pursuit of improvements.
Ultimately, the author may be better served by using tools/frameworks that are considered "completed" (examples might be things like jQuery) or have a maintenance philosophy that prioritizes backporting and compatibility over improving the features/experience.
To get security fixes they have to update to new versions, which requires them to change their code, but they don't want to have to change their code once it's done.
Vanilla PHP isn’t any “safer” in this regard. There’s lots of old PHP code out there that stopped working when PHP 7 removed a bunch of old, long-deprecated legacy cruft. That was good for the platform overall, but if you wrote a vanilla PHP app in 2008 and haven’t touched it since, odds are it complicated your life.
TFA is asking for a platform that never moves. The problem is that the only platforms that never move are ones that have died.
That's a good point. You need a language that doesn't move and always remains backwards compatiable. But I presume C is not a great choice as it's too low level.
I think JS will have this property for a long time, because of it's browser ties. And Haskell is another language I think won't change dramatically if you use the fundimentals. However in both cases you will want to roll your own web framework or at least pin to one you like, ideally something minimal.
However it does raise the problem of security patching, or lack of if you go down such a route.
The other mitigation, which you can do in any language is Inversion of Control. Have your code specify what sort of things it needs rather than use hard dependencies. E.g. "I need something that takes a string and a function and routes it" rather than "I need Express.js", with a layer that maps this up to actual implementations.
This way if frameworks change the mapping layer should be all that needs to change. Again pick a language like JS, Haskell, probably C#/Java etc. that is likely to be backwards compatiable for a few years to come.
I feel that. But on the other hand, without trying to sound too mean: what exactly does he expect? Is he also complaining that "Windows's goals are not our goals" because Microsoft stopped supporting Windows XP? Does he complain that Ubuntu version 11 is no longer getting updates? Java SE 9?
For better or for worse, this is simply how software works. Things have to be updated. Unless you want to eschew security updates (which is not smart), there is never such a thing as a fully "finished" product. Software maintenance is part of the job responsibilities.
This post is particularly targeted at Django, but I feel that it applies to almost every web framework or piece of software, and I'm afraid this is unfairly casting Django in a bad light. I'm also afraid that we're going to see a lot more posts like this in the second half of 2019 as we move towards python 2's EOL date.