This article is trying too hard to dress the author's limited experience up as absolute truth. For example:
> Have you got PostgreSQL in production, but you want to test with SQLite because it's easier to set up? If so, your tests are just a tick-box exercise: you are not testing anything remotely plausible and resembling your live system.
This is not completely wrong but it's just telling you that the author has never worked in an environment where it makes sense. Anyone who has knows that this approach can be used to catch a high fraction of errors locally, while not precluding a full slower CI build.
> Even if you added some form of manual auditing to each save() method, it will not capture changes made outside Django. It wouldn't be very secure either: Django uses a single user to access the database so if someone manages to hijack that user, they would be able to change the data in the database and alter the audit tables to hide their traces.
Similarly, while it is true that Django defaults using a single user for everything you can easily configure multiple connections (containing, say, an issue with your normal web code but not your management commands), you can grant access to only SELECT and INSERT on the audit log table but not UPDATE or DELETE, etc. There's nothing with the separate approach, of course, but it's not like it's some great discovery that frameworks are not intended to cover every possible use-case and sometimes you want to turn them off.
The extended “I hacked around because understanding the migrations module seemed like more work and then I was too far in to reconsider” section has the same problem. Being surprised that the migration system triggered an update when you renamed a field is, well, exactly what it's designed to do. That's exactly why there is an entire section in the documentation which tells you exactly what to do in that situation and how to avoid data loss — in that case, if the problem wasn't making foo.bar magic (which it almost always is) you'd create multiple migrations to add the new column, move (and usually convert) the data, and drop the old column. You can even run raw SQL if there's something like an efficiency hack which makes it worthwhile. This takes a couple of minutes and is reliable.
>This is not completely wrong but it's just telling you that the author has never worked in an environment where it makes sense. Anyone who has knows that this approach can be used to catch a high fraction of errors locally, while not precluding a full slower CI build.
Sqlite isn't faster though. The only reason I can think of why people do run tests with sqlite instead of postgres or whatever is because you don't have to install it.
This is an undeniable benefit but it does mean that instead of having the up front cost of installing postgres wherever you have tests you have the ongoing cost of "oh, it looks like my test failed because sqlite and postgres aren't actually alike".
It is faster (same process, memory tables) but that’s my point: you can say you always need to test on your target database, which is true, but not needing to install things is a benefit, and that can matter when you’re doing integration tests with lots of data. I have seen plenty of projects where fast local SQLite + CI Postgres with lots of test data is reasonable.
I had a project just a few weeks ago where somebody tried to do that and hit a brick wall nearly instantly because one table had an array column. SQLite instantly became useless.
IME unrealistic tests bite much harder than slightly slower tests or needing to install the odd bit of software. Realism is pretty much the only thing that gives you confidence you'll actually catch some damn bugs...
You’ll note that I said the same thing. The point was just that the set of bugs people write includes both those which will affect any database and those which depend on certain configuration and behaviors. There are enough people who find it useful to do both that I’m comfortable saying any position which pretends they don’t exist is incomplete.
As with the original author, the problem here is forgetting to add “On my projects” or “In my experience” rather than speaking in absolutes.
> This article is trying too hard to dress the author's limited experience up as absolute truth.
Sure you're not doing the same thing? As Vivekseth writes below, the author wrote psycopg2.
---
Allowing the DB user that the web app uses to have schema modification privileges is a massive security hole. If you're not hacked, someone will eventually drop the production database.
So, migrations in anything but SQL are a Bad Idea.
Yes, you’ll notice that I’m not saying his ideas have no place but that they’re not universally applicable. For example, the security “hole” is trivially avoidable by using a different settings file to do migrations on a separate server/container which doesn’t get normal web traffic.
If you want SQL, you can also generate it from your migrations and send it over to someone else to run. This is not uncommon in enterprise IT.
Limited does not mean he hasn’t accomplished anything, only that he’s not speaking for the entire community and wording his post that way doesn’t add anything to it.
He’s obviously very proficient, which is going to shape your perspective of what’s easy and how much control you want just like the scale of the projects you work on and the number and skill levels of your team.
There’s nothing wrong with his opinion - my objection is the overly broad framing. It would have been just as good as “here are some things which worked for us” and letting the reader decide whether they are in the same situation.
Maybe you have a lot of busy tests and you can get feedback twice as fast using less RAM on a SQLite memory database on your laptop, which is not a maxed-out current model, and you know that every commit you push will go through a CI build which uses the same database as production.
Maybe you are consciously trying to avoid locking your project into one database accidentally.
Maybe you work in an enterprise environment where you can’t run Docker or network services on your local system.
Maybe you know that your CI process will run against more data than fits on your laptop and so you’re not worried about missing something.
My point wasn’t that you shouldn’t test in the same environment but that it’s too quick to say that because you have never had this need nobody else does either.
> Maybe you are consciously trying to avoid locking your project into one database accidentally.
People keep bringing this up, but for applications (not libraries), how many times has this mattered (for FOSS DBs), and how many times has upholding agnosticness required weird hacks or limited something?
The reason people bring it up is that it's real for some people, even if it's not a universal requirement. Anyone who sells software has customers who have standardized on one database. Some places have senior management wanting to avoid lock-in (Oracle migrations will almost always be mentioned in this case) and may have pressure or outright requirements — and if you're a contractor working for them this will be written into their requirements. Some products have a demo/lite mode to make it as easy as possible to try – e.g. SonarQube can run in a single container (with prominent warnings that your database is not persistent) just to make it easy for someone to try it.
It's certainly not the most common and I would definitely seriously question it if you know you're going to benefit from a Postgres-specific feature (the last big Django project I started used JSONField and several other features heavily), but the world is a big place and there are many features which only some people use but it's critical for them. This discussion reminds me a lot of when people would question why you need a localization framework because they've never used one.
> Have you got PostgreSQL in production, but you want to test with SQLite because it's easier to set up? If so, your tests are just a tick-box exercise: you are not testing anything remotely plausible and resembling your live system.
This is not completely wrong but it's just telling you that the author has never worked in an environment where it makes sense. Anyone who has knows that this approach can be used to catch a high fraction of errors locally, while not precluding a full slower CI build.
> Even if you added some form of manual auditing to each save() method, it will not capture changes made outside Django. It wouldn't be very secure either: Django uses a single user to access the database so if someone manages to hijack that user, they would be able to change the data in the database and alter the audit tables to hide their traces.
Similarly, while it is true that Django defaults using a single user for everything you can easily configure multiple connections (containing, say, an issue with your normal web code but not your management commands), you can grant access to only SELECT and INSERT on the audit log table but not UPDATE or DELETE, etc. There's nothing with the separate approach, of course, but it's not like it's some great discovery that frameworks are not intended to cover every possible use-case and sometimes you want to turn them off.
The extended “I hacked around because understanding the migrations module seemed like more work and then I was too far in to reconsider” section has the same problem. Being surprised that the migration system triggered an update when you renamed a field is, well, exactly what it's designed to do. That's exactly why there is an entire section in the documentation which tells you exactly what to do in that situation and how to avoid data loss — in that case, if the problem wasn't making foo.bar magic (which it almost always is) you'd create multiple migrations to add the new column, move (and usually convert) the data, and drop the old column. You can even run raw SQL if there's something like an efficiency hack which makes it worthwhile. This takes a couple of minutes and is reliable.