You might think this is trivial -- but SERIAL creates an "owned" (by a certain user) sequence behind the scenes, and so you run into massive headaches if you try to move things around.
Identity columns don't, and avoid the issue altogether.
I'll stop short of giving a recommendation or using the word "should", but ill give encouragement to consider using uuid's for keys. I have used them in several systems and have never had any issues with them, and they solve so many issues. The ability to generate a key on the client or on the server or in the database is great for one. And the fact that keys are unique not only in a table but in the system (or many systems) is a huge advantage.
- Avoids people being able to just iterate through records, or to discover roughly how many records of a thing you have
- Allows you to generate the key before the row is saved
I think I default to auto-increment ID's due to:
- Familiarity bias
- They have a temporal aspect to them (IE, I know row with ID 225 was created before row with ID 392, and approximately when they might be created)
- Easier to read (when you have less than +1,000,000 rows in a table)
I agree and think you're right in that UUID's are probably a better default.
Though you can never find a "definitive" guide/rule online or in the docs unfortunately.
I had reviewed existing UUIDv7 implementations and many were incorrect or had subtle timing bugs.
We ended up implementing UUIDv7 in our ID generation library https://github.com/MatrixAI/js-id. And we have a number of tests ensuring that it is truly monotonic even across process restarts.
Came here to also hype ULID. It's just like INT, but 128bit, timestamps, every lang, and moving data around is trivial because no ID ever collide with anything else ever. Makes finding crap in my logs easier. (But UUID has that feature too) - it's the sorting that does it for me.
Something I always liked but have never done -- start auto-increment IDs at some number other than 1. I worked on a system where account IDs started with 1000000, invoice IDs started with 2000000, etc. That way, if you saw a random number laying around like "1001234", you knew it was an account ID. "2000123", an invoice! I don't remember how often it helped things, but it was more than 0 times, and it always "felt" good. I never loved the implicit upper bound, but it was also never a problem.
(And remember, the values are in different tables, so the only disadvantage is that your millionth user has ID 2000000. To the trained eye they look like an invoice; but there is no way for the system itself to treat that as an invoice, so it's only confusing to humans. If you use auto-increment keys that start at 1, you have the same problem. Account 1 and Invoice 1 are obviously different things.)
> but there is no way for the system itself to treat that as an invoice, so it's only confusing to humans.
I agree in principle, but then you have Windows skipping version 9 because of all of the `version.hasPrefix("9")` out in the world that were trying to cleverly handle both 95 and 98 at once. If a feature of data is exploitable, there's a strong chance it will be exploited.
I use this for internal items, as in: real customer IDs start at 10,000, so we have 9,999 possible users/invoices/whatever for our own usage in the database. That makes it easy to filter demo accounts of your sales employees, or production test accounts, in queries: just add an offset.
Re: the upper bound: if you reach a million customers, you have lots of other nice problems, like how to spend all the money they earn you :-)
I worked on an existing database once that stored geographic locations and used the signed bit for namespacing.
Positive IDs were home-grown, negative ones were Unique Feature Identifiers from some old GNS system import (from some ancient US Government/Military dataset).
Then when something had to be edited/replaced/created it would possibly change a previously negative ID to positive, fun times.
I’ve seen systems that had to track only two types of things - one started at half the value and went up; the other started just below that and went down. Think 500000+ vs 499999-
I prefer UUIDs as well but one other benefit of an integer type key is in the index. UUIDs, especially v4, make for bloated indexes. If there’s a very specific performance reason I’ll use int, otherwise uuid.
One of hidden pros is that UUID has no gaps visible to product owner or manager. More than once in my life I have try convince people, that the gap in sequence caused by transaction rollback is OK. Several times we were «fixing data» in db manually
This is what I do as well. Most schema structure is on tables that are not very large, and it's nice especially when dealing with multiple environments to prevent errors, or to produce schemas with some polymorphism or joins between several tables without a chain of id-resolving joins in-between.
There are UUID variants that can work well with indices, which shrinks the case for big-integers yet further, to micro-optimizing cases that are situational.
The person you're replying to didn't even mention those advantages though! The ones they did mention are more significant to me.
Of your disadvantages... if I wanted to know when rows were created, I'd just add a created timestamp column.
But "easier to read" is for real -- it's easy when debugging something to have notes referencing rows 12455 and 98923823 or in some cases even keep them in your head. But UUIDs are right out.
And you definitely don't want to put UUIDs in a user-facing URL -- which is actually how you get around "avoids people being able to just iterate through records", really whether you have UUIDs or numeric pks I think putting internal PK's in URLs or anything else publicly exposed is a bad idea, just keep your pks purely internal. Once you commit to that, the comparison between UUIDs vs numeric as pks changes again.
I mean, the links are going to be a template either way, so you can change the identifier part to change the item; it's just a question of whether the identifier portion is the internal rdbms PK, or something else.
It's considered undesirable because it's basically an "implementation detail", it's good to let the internal PK change without having to change the public-facing URLs, or sometimes vice versa, when there are business reasons to do so.
And then there are the at least possible security implications (mainly "enumeration attacks") and/or "business intelligence to your competitors" implications too.
But it's true that plenty of sites seem to expose the internal PK too, it's not a complete consensus. Just kind of a principle to separate internal implementation from public interface.
Random keys have the positive aspect of not creating any hotspots, and they have the negative aspect of not creating any hotspots.
So if you have concurrent updates which are correlated with row insertion time, random keys can be a win. On the other hand, if your lookups are correlated with row insertion time, then the relevant key index pages are less likely to be hot in memory, and depending on how large the table is, you may have thrashing of index pages (this problem would be worse with MySQL, where rows are stored in the PK index).
(UUID doesn't need to mean random any more though as pointed out below. My commentary is specific to random keys and is actually scar tissue from using MD5 as a lookup into multi-billion row tables.)
I generally use both - a serial for internal RDBMS storage to coerce writes to be in order, and a secondary GUID that is app-specific and generated. The app will never care about the serial, and the RDBMS doesn't care about the GUID other than secondary index look-ups (since they're rarely used for range scans).
Be wary when using guids for PK in clustered indices, in databases that support them, see for example https://stackoverflow.com/questions/11938044/what-are-the-be...
. Sadly, Postgresql doesn't have them maintained on-the-fly like Ms SQL does so it is less used feature.
That is true, but that has never been a bottleneck for me. We have a table with multiple hundreds of millions of rows, 20ish UUIDs indexes (yes it did get out of hand) and insert time isn't an issue.
Ofc, your app performance requirements might vary, and it is objectively true that UUIDs aren't ideal for btree indexes.
Then you are lucky. I once found an insert performance drop of sometimes 5x once a table reaches 500,000 elements or so with BTree's and Uuids in particular. Problem is: UUID's often scale well on the app side, especially with multiple writers so the app wants to use them.
Unless you are using a time sorted UUID, and you only do inserts into the table (never updates) avoid any feature that creates a BTree on those fields IMO. Given MVCC architecture of Postgres time sorted UUID's are often not enough if you do a lot of updates as these are really just inserts which again create randomness in the index. I've been in a project where to avoid a refactor (and given Postgres usage was convenient) they just decided to remove constraints and anything that creates a B-Tree index implicitly or explicitly.
It makes me wish Hash indexes could be used to create constraints. They often use less memory these days in my previous testing under new versions of Postgres, and scale a lot better despite less engineering effort in them. In other databases where updates happen in-place so as not to change order of rows (not Postgres MVCC) a BRIN like index on a time ordered UUID would be often fantastic for memory usage. ZHeap seems to have died.
Sadly this is something people should be aware of in advance. Otherwise it will probably bite you later when you have large write volumes, and therefore are most unable to enact changes to the DB when performance drastically goes down (e.g. large customer traffic). This is amplified because writes don't scale in Postgres/most SQL databases.
Some databases don't give you a choice, and your table data is actually stored in a clustered index sorted by primary key. This means random inserts when using a UUID (without embedded time), and more page splitting.
Technically there are many advantages, but operationally, I find it extremely useful in systems where an int PK is auto-generated to use it as a pretty good proxy for a relative created time with relation to other records, with built-in indexing that avoids having to index on an actual created datetime column for a lot of less precise lookup purposes. The backend frameworks I use around data access make the security benefits of UUIDs pretty moot.
What will really be nice is when UUID V7 has wide support, because in many ways it is the best of all possible worlds:
1. The UUIDs are generated in increasing order, so none of the b-tree issues others have mentioned with fully random UUIDs.
2. They're true UUIDs, so migrating between DBs is easy, ensuring uniqueness across DBs is easy, etc.
3. They also have the benefit of having a significant amount of randomness, so if you have a bug that doesn't do an appropriate access check somewhere they are more resistant to someone trying to guess the ID from a previous one.
A friend of mine recently recommended CUIDs, which I'd never heard of before. Any thoughts on those? Neither usecuid.org nor https://www.npmjs.com/package/cuid give me much confidence, though (the former is not https, and the latter has not been updated in 2 years... which might mean it's done, or might mean it's abandoned).
One huge factor for me: assuming you stick to v4, you can use UUIDs directly as identifiers in public contexts without leaking any business-related info and without having predictable keys.
"A collision is possible but the total number of unique keys generated is so large that the possibility of a collision is almost zero. As per Wikipedia, the number of UUIDs generated to have atleast 1 collision is 2.71 quintillion. This is equivalent to generating around 1 billion UUIDs per second for about 85 years."
part of the point of UUIDs is that you don't have to. Collision is unlikely enough that your time would be more wisely spent worrying about the sun exploding unless you generate an absolutely absurd amount of data.
Unless you are writing nuclear warhead management system, writing UUID collision handling is waste of time. Client can retry on top level if request failed.
my worries were more about access control, it is sort of fine if a costumer experiences data loss because an insert fails and the application doesn't retry, it is less fine if a collision causes a user's documents to be swapped with another user's docoment and they end up showing kinky porn on a live press conference.
Sort of the distinction between unspecified behaviour and undefined behaviour in C.
The application shouldn't report the insert as successful if it actually failed. That way, the user don't go around thinking the insert actually succeeded, and there is no data loss (if it didn't succeed it must be retried, by the app or manually)
You need some UX like an error message in a red rectangle or something.
there would be data loss if the insert/update would meet a collision and either decided to overwrite the record or to report the operation as already completed (the latter is what I imagine git would do for a collision scenario).
the solution might as well just be not to care about this case (no sarcasm).
But if we're talking about a database insert and there's a collision (and the uuid column has an unique constraint - as it should, since it's meant to be a primary key), then the insert will not be successful.
Your application might not care about treating this error, but the DB will report it.
Another favor you should do yourself is use bigint keys (unless using uuid as suggested elsewhere, or you're very, very sure you'll never chew through 2 billion ids).
I often move ID generation into the application layer (this also helps avoid things like enumeration attacks), and actually quite a lot of cool Postgres features blur that line a little bit. It's interesting to think sequences and other computational mechanisms in a DB, and whether they make architecting applications easier or harder. I don't have a strong opinion either way, but I'm interested in HN's opinion.
One often hears the counterargument 'but using DB-specific features makes your application less portable!' to which I like to argue: When was the last time you moved an application from SQL-db to SQL-db engine? Follow up question: When was it ever 'easy' if you did?
If you start from the basic premise that the database engine and the application are intertwined and are not loosely coupled, using Postgres-specific features feels much less icky from an architectural point of view. They are essentially part of your application given that you use something like Flyway for migrations and you don't manually run SQL against your production to install functions and triggers and such.
So, I did an easy migration of an analytics app from elasticsearch to ClickHouse: what made it easy was (a) we could replay an event stream to regenerate the database and (b) we had interfaces around the database defined in terms of the questions we needed the database to answer, and not in terms of ES-specific concepts.
But, what makes this sort of design so nice is that you can use DB-specific stuff behind the interface because you’re not trying to write all your queries in the minimally supported subset of SQL or something.
I've given up on believing that there is such a thing as portable SQL. If you use the lowest common denominator, you're likely to have queries that perform suboptimally. Hell, even temp tables aren't portable because of the different syntax between databases. I've worked with tons of queries that would would take minutes to run without temp tables while running in milliseconds with them. People may as well take advantage of all the nice things their database offers instead of fearing a situation that is unlikely to happen and which will be a tough change anyway.
Adding on, even when you are trying to make a portable application, you tend to want to make it work on either X or Y, and you can still exclude people from switching their existing data from one to the other.
I've used software that at least tried to be portable, so you could install it with whatever database you had available.
I did not cite reasons of portability for this reason, and generally I agree with you. Also some of the non-portable extensions of Postgres like PostGIS are practically indispensable for certain applications.
A more reasonable argument for application layer ID generation is flexibility. It's likely that I want some specific primary key generation (e.g. by email, or username, or username prepended with role, and so on).
Done it multiple times, most recently we got an MVP off the ground using only ElasticSearch as a backend, since the primary use case we wanted to validate and demonstrate was discoverability.
As we added more features, the lack of transactions and relational structures started to slow us down, so we dropped in Postgres as a backend, and having application-generated UUID4s as primary keys was a big part in making that move fairly painless
We're currently in the process of moving from MongoDB to CockroachDB. It's not been easy. Certain persistence needs to be duplicated for both of them. Certain queries need to be compared between the two, and we need to make a conscious decision when we pick from which DB.
Having said that, moving logic for id generation to the application because it's less portable otherwise is an odd reason.
> One often hears the counterargument 'but using DB-specific features makes your application less portable!'
OK, sorry for the probably stupid question: Isn't it just a matter of, for each sequence, selecting its current value and then creating the new one in the target database to start from there? Should be, if perhaps not easily, still reasonably scriptable... Or what am I missing?
For backends, I typically use ID generation in the application layer (the backend). And with ID generation I mean the HiLo generator from Hibernate (or NHibernate). By default the HiLo generator will still use a monotonic increasing number, possibly with gaps. (Either way, as long as you expose some form of monotonic increasing number in your api, enumeration attacks could work. The correct way is to simply not expose the original id if that is an issue: you could expose some kind of opaque token in the api instead of a literal id.) The advantage of the HiLo generator is that the ORM knows the generated id up-front and does not need a round-trip to the database to determine it. So it can generate more performant queries. Also, contrary to UUIDs it does not have the disadvantages regarding indices. The only reason I have ever used UUIDs was because of a requirement that external clients needed to generate the ids.
>You might think this is trivial -- but SERIAL creates an "owned" (by a certain user) sequence behind the scenes, and so you run into massive headaches if you try to move things around.
Maybe it is because I'm too old, but making id grow by sequence is the way how things 'ought' to be done in the old skool db admin ways. Sequences are great, it allows the db to to maintain two or more sets of incremental ids, comes in very handy when you keeping track of certain invoices that needs to have a certain incremental numbers. By exposing that in the CREATE statement of the table brings transparency, instead of some magical blackbox IDENTITY.
However, it is totally understandable from a developer's perspective that getting to know the sequences is just unneeded headache. ;-)
Why? Concrete ownership is a good thing. I prefer more rigid structures and I like designing my tables around that. It was the Wild West when I did database design around MySQL 15ish years ago. Of course I didn’t know nearly as much as I do now back then.
Regarding "Match Against Multiple Patterns", the examples are about finding the _suffixes_ of something, email domains in the example.
An attempt to find a suffix like that will not be able to use an index, whereas creating a functional index on the reverse and looking for the reversed suffix as a prefix will be:
# create table users (id int primary key, email text);
CREATE TABLE
# create unique index on users(lower(email));
CREATE INDEX
# set enable_seqscan to false;
SET
# insert into users values (1, 'foo@gmail.com'), (2, 'bar@gmail.com'), (3, 'foo@yahoo.com');
INSERT 0 3
# explain select * from users where email ~* '@(gmail.com|yahoo.com)$';
QUERY PLAN
--------------------------------------------------------------------------
Seq Scan on users (cost=10000000000.00..10000000025.88 rows=1 width=36)
Filter: (email ~* '@(gmail.com|yahoo.com)$'::text)
# create index on users(reverse(lower(email)) collate "C"); -- collate C explicitly to enable prefix lookups
CREATE INDEX
# explain select * from users where reverse(lower(email)) ~ '^(moc.liamg|moc.oohay)';
QUERY PLAN
----------------------------------------------------------------------------------------------------
Bitmap Heap Scan on users (cost=4.21..13.71 rows=1 width=36)
Filter: (reverse(lower(email)) ~ '^(moc.liamg|moc.oohay)'::text)
-> Bitmap Index Scan on users_reverse_idx (cost=0.00..4.21 rows=6 width=0)
Index Cond: ((reverse(lower(email)) >= 'm'::text) AND (reverse(lower(email)) < 'n'::text))
(Another approach could of course be to tokenize the email, but since it's about pattern matching in particular)
Another great hidden feature is the LISTEN/NOTIFY system. You can use it to watch for changes on a table, and then use that to e.g. update a view in the UI.
For example create a table with a trigger on insert "NOTIFY new_data". Then on query do
LISTEN new_data;
SELECT ...;
Now you'll get the results and any future updates.
Expanding on the regex trick: I often use [.] instead of \. to match a period.
They achieve the same thing, but [.] avoids the problem of your host language "helpfully" interpolating \. into . before sending the query to Postgres.
Two things that aren't exactly lesser-known, but that I wish more used continuously as part of development:
- generate_series(): While not the best to make _realistic_ test data for proper load testing, at least it's easy to make a lot of data. If you don't have a few million rows in your tables when you're developing, you probably don't know how things behave, because a full table/seq scan will be fast anyway - and you'll not spot the missing indexes (on e.g. reverse foreign keys, I see missing often enough)
- `EXPLAIN` and `EXPLAIN ANALYZE`. Don't save minutes of looking at your query plans during development by spending hours fixing performance problems in production. EXPLAIN all the things.
A significant percentage of production issues I've seen (and caused) are easily mitigated by those two.
By learning how to read and understand execution plans and how and why they change over time, you'll learn a lot more about databases too.
It's worth to note that earlier versions of PostgreSQL didn't include the "AS NOT MATERIALIZED" option when specifying CTE's. In our setup, this had huge hits to performance. If we were on a more recent version of PostgreSQL (I think 11 in this case), or if the query writer just used a sub-query instead of a CTE, we would have been fine.
On this first point — 'Get the Number of Updated and Inserted Rows in an Upsert' — unfortunately this `xmax` method is not always reliable and there does not seem to be any reliable alternative.
Atlassian gives us sales figures as a large CSV. Nothing better than loading the CSV as a Postgres view with a foreign data wrapper, and performing pivots/joins/anything on it. Replace the file with a more recent one, and you even have versioning!
one place they are great in production is as part of a data migration. if you have an old database system with application code on top that you want to move to pure postgres, fdws provide a way to incrementally rewrite the application logic without ever having to do a hard cutover.
I believe it's about the common case of two events sharing a single (infinitely small) point of time: meeting A from 3 to 4, followed by meeting B from 4 to 5.
Those two time periods do overlap mathematically, but not physically: they share the common point of "4", even if that point has duration 0.
I know the iCal standard goes into this for a bit.
The upsert strategy mentioned (using ON CONFLICT to update instead of insert) has a catch, if your PK is a sequence it will increment even though the insert has failed.
It's not a huge deal in most cases, is a bigger issue if you are running any sort of scheduled ETL to update (or insert new) things.
Careful with DISTINCT ON, it can be extremely slow, like 100x slower than a verbose and less readable alternative. But it might not matter in your use case.
This example didn't bring anything new to the table, only adds redundant extra chars:
SELECT *
FROM users
WHERE email ~ ANY('{@gmail\.com$|@yahoo\.com$}')
Perhaps they were intending something similar to the following example instead.
This one works but has a several potential lurking issues:
with connection.cursor() as cursor:
cursor.execute('''
SELECT *
FROM users
WHERE email ~ ANY(ARRAY%(patterns)s)
''' % {
'patterns': [
'@gmail\.com$',
'@yahoo\.com$',
],
})
The dictionary-style interpolation is unnecessary, the pattern strings should be raw strings (the escape is ignored only due to being a period), and this could be a SQL injection site if any of this is ever changed. I don't recommend this form as given, but it could be improved.
One of the hardest types of queries in a lot of DBs is the simple `min-by` or `max-by` queries - e.g. "find the most recent post for each user." Seems like Postgres has a solution - `DISTINCT ON` - though personally I've always been a fan of how BigQuery does it: `ARRAY_AGG`. e.g.
SELECT user_id, ARRAY_AGG(STRUCT(post_id, text, timestamp) ORDER BY timestamp DESC LIMIT 1)[SAFE_OFFSET(0)].*
FROM posts
GROUP BY user_id
`DISTINCT ON` feels like a hack and doesn't cleanly fit into the SQL execution framework (e.g. you can't run a window function on the result without using subqueries). This feels cleaner, but I'm not actually aware of any other DBMS that supports `ARRAY_AGG` with `LIMIT`.
I would use lateral join. Alternatively you can use window function instead of lateral join, buy from my experience lateral joins are usually faster for this kind of top-N queries in postgres.
I assume you mean `ROW_NUMBER() OVER (PARTITION BY ... ORDER BY ...) as row_number` with a `WHERE row_number = 1`. We ran into issues with memory usage (i.e. queries erroring because of too high memory usage).
The `ARRAY_AGG` approach uses a simple heap, so it's as efficient as MIN/MAX because it can be computed in parallel. ROW_NUMBER however needs all the rows on one machine to number them properly. ARRAY_AGG combination is associative whereas ROW_NUMBER combination isn't.
The upsert strategy mentioned (using ON CONFLICT to update instead of insert) is something we use internally quite a bit, esp in multi-insert transactions which were horrendous to handle without conflict resolution because any conflict would end up cancelling the entire transaction.
Though I wish there was an easy way to figure out how many times "CONFLICT" actually occured (i.e how many times did we insert vs update).
I read that blog post but I don't quite understand it.
I assumed it would work as a regular pub/sub pattern where you get notified when some event happens. However, in the attached example, they still poll the database every half second. I'm not sure I understand the idea.
Quite strange that this specific driver didn't support asynchronous notifications: if you have to poll the database anyway, there's no much difference between doing it without listen/notify support, I guess.
They're the ones that will be hardest to migrate to a different database, most likely to be deprecated, and least likely to be understood by the next engineer to fill your shoes.
While many of these are neat, good engineering practice is to make the simplest thing to get the job done.
Databases often outlive applications. Database features usually do what they do very quickly and reliably. Used well, a full-featured database can make things like in-place refactors or re-writes of your application layer far easier, make everyday operation much safer, as well as making it much safe & useful to allow multiple applications access to the same database, which can be very handy for all kinds of reasons.
> While many of these are neat, good engineering practice is to make the simplest thing to get the job done.
I'd rather have a non-portable SQL feature than an app-level "portable" implementation that almost certainly is slower and non-transactional.
Migrating to another DB will always take work, no matter how much you try to ignore flavor-specific features. Query planning, encoding and charsets, locking abilities tend to be very different. A query can run fine in MySQL and cause a deadlock in Postgres even though it's syntactically valid in both.
How often does migrating databases ever actually happen? I'm not saying it doesn't happen, but I've never experienced it or know anyone who has (I've asked!). I certainly would shy away from using features that make my app code cleaner and improves the all-around performance of the app on the slim-to-none chance that one day I might one day have to migrate off of postgres.
20-year career so far. Never seen a database swapped out. I've seen "apps" replaced on top of databases, or more programs added to access the same database. I've seen the app and database both get thrown out and replaced, because they were tightly coupled[0], as the parent advocates (Rails + ActiveRecord seems to be prime for this kind of "gotta throw it all out" situation). I've never seen the program stay the same while the DB is swapped out from under it.
[0] yes, that's actually tightly coupling them, because now your DB is too unsafe to use without the "app" you built on top, and doesn't provide enough functionality to make it worth trying to retrofit that safety onto it.
I've seen it firsthand, but we're talking about an early-stage startup moving from MySQL to Postgres. It's definitely not enough to convince me not to use database-specific features.
I’ve added an activrecord interface to an existing schema (on sql server no less). A lot of overriding of it’s assumptions and sometimes preventing writes via active record but it is doable.
20 years ago I ported an app from Microsoft SQL server to Oracle as a new requirement from the incoming CTO equivalent. I was told the whole place was standardizing on Oracle as they had in house expertise. Turns out they didn’t and while the port was completed and successful they had no end of trouble managing it.
I’ve since ported a few apps in early development or production between Microsoft SQL server, MySQL and Postgres (in various directions) but nothing in prod in over 10 years
I'm not sure why you are being downvoted. Your argument is reasonable and clearly presented, and "do the simplest thing that can work" is a tried and true design philosophy.
I just learned about OVERLAPS this week when reviewing a colleague’s code. We talked about the complexity of the conditions and found the overlaps operator and a ton of range operators. None of us knew they existed and they made our query a lot better.
The author's posts on Django are always worth reading. They're way more detailed than the run-of-the-mill rehash of the Django documentation that most blogs contain and there's always a nugget of "I never knew that" in them.
Wow, I've been writing a lot of PL/pgSQL recently and did not know this.
Would these be drop in replacements for PL/pgSQL? Are there any performance tradeoffs to using one over another? Any changes in functionality (aka functions you can call)?
I personally have never run into whatever the limitations might be of pl/python, but I suppose you could make a disaster by trying to dynamically load conflicting symbols or something like that. But I have used numpy in stored procedures, so at least that works.
pl/sh also works wonderfully if you want to run a complex procedure in an isolated subprocess, but pl/sh is not part of the main postgresql distribution.
This feature might be great for OLAP systems, but it's awful for OLTP systems. The last thing I want is debug permission errors bubbling from my database into the application layer. It may not be as efficient, but you should always manage your ACL rules in the application layer if you're building an OLTP app.
> If you know a good usecase for ACL in database for OLTP workflows, I'm all ears.
Here's one that's meaningful to me. We have a single database with five different applications that access it, each of them managed by a separate team. By enforcing access constraints in the database we guarantee the access constraints will be applied in all cases. It is difficult to ensure that in application code managed by separate teams.
(Just to be clear: I don't think your advice is poor, I just wanted to give an example of a case where it isn't universally applicable.)
Even then, it would be better phrased as a tradeoff. What's the downside of putting in the database? I'd guess it makes horizontally scaling harder, and it's less portable to other database vendors. Is there an upside, like you've got stronger data security guarantees?
An upside to using db to handle data permissions is the data has the same protections if you are accessing with direct SQL or using an app. Also, those permissions would persist with the data in restored backups.
I’m not advocating this and think for OLTP, building RBAC in the app layer is almost always a better idea, but these would be some benefits.
Good point. Phrasing it as a trade-off is the better approach. Agree with the downsides you listed, as well.
Better security guarantees, for sure, but if an unauthorized user gains direct access to your database, you might have bigger problems from a security perspective.
https://hakibenita.com/postgresql-unknown-features#prevent-s...
You might think this is trivial -- but SERIAL creates an "owned" (by a certain user) sequence behind the scenes, and so you run into massive headaches if you try to move things around.
Identity columns don't, and avoid the issue altogether.
https://www.2ndquadrant.com/en/blog/postgresql-10-identity-c...