
Looking Back at Postgres - craigkerstiens
https://arxiv.org/abs/1901.01973
======
lazyjones
As someone who bet his startup in 2000 on Postgres (with little experience at
the time) and registered on HN just to praise Tom Lane for being exceptionally
helpful and forthcoming on the mailing lists, I enjoyed this but don't think
it's very relevant to Postgres' success in the past 20-odd years - that was
entirely due to a few good people who had the right priorities and made good
decisions without drama and without being overly opinionated. A very notable
exception in the world of open source...

~~~
irjustin
> without being overly opinionated

My take is Postgres is extremely opinionated, but in such a way that it is
beneficial for the long run.

PG adheres to the SQL standard far stricter than MySQL (i.e. select w/o order,
ACID). From 2000-2010, it severely slowed the progress of features compared to
MySQL (clustering was big topic for a while).

But the problems of hacks, speed and probably a significant one,
management/owners, caught up with MySQL.

Feels like tortoise & the hare.

~~~
jeffdavis
"Postgres is extremely opinionated"

When I think of the term "opinionated" as it applies to software platforms, I
think of things that have intended purposes in mind and optimize for the ease
of those use cases, potentially to the detriment of people doing other things.
It also suggests a "one way to do it" approach.

If that is the definition, I don't think Postgres fits. Postgres tries pretty
hard to meet users where they are, and tackle uncommon or emerging use cases.
Examples include JSON support, support for writing functions in various
languages (javascript, python, perl, R, etc.), and C extensions that can hack
the engine in innumerable ways.

~~~
chousuke
I don't think "opinionated" in the context of a database system means refusing
to implement useful features. When I think of unopinionated software, I think
of software like Jenkins; it feels like a plugin glue framework that does
barely anything useful out-of-the-box and leaves it up to the user to create a
working configuration out of the tons of plugins available.

Aside from being immediately useful just as a basic relational data store, my
impression of PostgreSQL is that the "opinion" is apparent in the way that new
things get done. There seems to be a culture of either implementing things
_well_ or not at all, and to avoid rushing things just because people want the
new feature right now.

------
parhamn
> The highest-order lesson I draw comes from the fact that that Postgres
> defied Fred Brooks’ “Second System Effect” [Bro75]. Brooks argued that
> designers often follow up on a successful first system with a second system
> that fails due to being overburdened with features and ideas. Postgres was
> Stonebraker’s second system...

I really hope we pull off some of these software adages.

~~~
VectorLock
>Postgres was Stonebraker’s second system.

What was his first system?

~~~
fiddlerwoaroof
Ingres.
[https://en.m.wikipedia.org/wiki/Ingres_(database)](https://en.m.wikipedia.org/wiki/Ingres_\(database\))

However, if you look at the projects he’s worked on, he has a track record of
writing database engines that end up in delivered database systems.

------
sxcurry
One of the first C programs I wrote pulled data from an Ingres Database, all
running on a PDP 11/34\. I now use PostgreSQL heavily on AWS. Thanks Michael
Stonebraker!

~~~
fcatalan
I'm still developing web services connected to an Ingres instance running on a
rickety ~22 year old Sun server with lots of bits and pieces cannibalized from
its long dead siblings.

We have the replacement system almost ready but won't really be able to swap
the thing until the project owner retires (yay politics!).

It's important but not time-critical, so if it dies before he goes we can just
eat the downtime and take a couple of weeks to migrate.

------
boshomi
See also: "The Postgres and Illustra Codelines" by Wei Hong:

[https://dl.acm.org/doi/pdf/10.1145/3226595.3226623](https://dl.acm.org/doi/pdf/10.1145/3226595.3226623)

~~~
macdice
I have done some work on parallelism in PostgreSQL, and I was fascinated to
discover Wei Hong's work in this area from more than a quarter century ago:

[https://www2.eecs.berkeley.edu/Pubs/TechRpts/1992/ERL-92-3.p...](https://www2.eecs.berkeley.edu/Pubs/TechRpts/1992/ERL-92-3.pdf)

The POSTGRES 4.2 tarball includes a bunch of his parallel query work, which
you can find wrapped in #ifdef sequent. I _assume_ that this was a precursor
to XPRS, and it was ripped out of regular POSTGRES after that. I wonder if
anyone knows what happened to the XPRS sources...

------
felixge
I love PostgreSQL history, so this was tons of fun to read me for me.

While reading, I also ended up highlighting and sharing some of my favorite
parts on twitter:
[https://twitter.com/felixge/status/1272613965219139585](https://twitter.com/felixge/status/1272613965219139585)

------
rurban
I wish he described more the problems of their insular approach to avoid SQL.
It used something like postquel, nobody used. It only shined under a C/C++
API, but was not usable for users. Just for developers to map their objects
to. An opinionated OO database, which didn't follow the standards. And there
were much better OO - DB persistency mappings these days, like for Lisp. There
you had to write no glue code. Postgres was a turkey.

But everything changed when someone added SQL. Soon it became the very best
SQL standards implementation. That's when I picked it up again, and stayed
with it.

------
kcolford
Is this the same Postgres as the PostgreSQL database we all know and love here
on HN?

~~~
breakingcups
Yes, although originally it did not support SQL but QUEL.

see
[https://en.wikipedia.org/wiki/PostgreSQL#History](https://en.wikipedia.org/wiki/PostgreSQL#History)
and
[https://en.wikipedia.org/wiki/QUEL_query_languages](https://en.wikipedia.org/wiki/QUEL_query_languages)

~~~
takeda
I wish how things would be if it was QUEL that would became the standard. The
language looks closer to the access pattern that ORMs using.

------
jansan
Totally off-topic, but anybody who is into databases should at least have
heard the name dBase, which was the most successful database in the 80s.
However, they totally fucked up the release of version IV, which had serious
reliability issues (not something you want in a database, in case you did not
know). This lead to a dramatic decline and eventually the death of dBase. Who
knows how the database and software landscape would look like had they
released a version without major bugs.

~~~
joering2
> at least have heard the name dBase

Oh man, you just wet my eyes! What a trip into the past!

I wrote my first paid project in dBase in 1982 when noone knew what "computer
mouse" was. I answered the ad in newspaper and initially they wanted me to
help them organize text files. I told them about dBase and that was my first
project - a simple database to manage their parking lots. It was such early
days of programming, we had basically zero agreement! Just half piece of paper
that they pay me 10% in advance and short list of features to get remaining
90%. And funny thing is - no copyright agreement! So while I made few months
of rent for their project, I ended up paying off my first mortgage by just
offering the same piece of software to many other parking corps! It was
incredible feeling! Like creating money out of thin air - they sponsor my
plane/hotel and I just told them what PC (with DOS) I need to be ready. Then
in just few hours of installation and training, I was on my way home with nice
check. I frankly don't even remember how much it was back then...

Good times!

~~~
72deluxe
Amazing story. How did you get known to the other parking companies? Did you
ring them up or something?

I suppose the secret is quickly delivering software that does what someone
wants.

~~~
joering2
Yes. Back then cold calling was welcomed. Everyone picked up the phone with
excitement of who's calling. All I had to tell them is the name of the company
and how they can benefit for the software. Honestly conversion rate was close
to 100%, its just some locations had headquarters across the country and I
didn't feel like flying in winter time lol.

------
danielovichdk
I find Postgres a really good database but with really poor tools.

I don't want to stir up a religious battle here, but the tooling around MSSQL
is just that much better.

And the sql dialect is imo more to my taste.

~~~
kiwicopple
I also found this when I switched from MySQL. After a while you discover the
right tool for the job, and often some tolls that don’t exist for other
databases (things like PostgREST come to mind). Sometime I wish it was a bit
more idiomatic though.

We’re working on a lot of tooling at
[https://supabase.io](https://supabase.io). We’re open source too. If there is
anything in particular you’re missing, let me know

~~~
RNCTX
Since you guys are doing something I highly agree with and I replied to your
announcement post as well...

How about a UML/ERD diagram tool which can spit out schemas and, given a
schema to start with, migrations?

There are a couple dozen of these being offered as SAAS with varying levels of
usability and complexity. If you built one of those you'd soon have the
attention of everyone who is bitching about paying 20 dollars a month to see
their own database in a way that management can visualize for their charts and
graphs.

ERD diagrams have existed for decades. Even Oracle gives such a tool away. But
for Postgres, the easiest way to create a schema with migrations without a
credit card is still to write a Django model by hand, afaik.

~~~
kiwicopple
That's a great idea for the diagrams. I'm slowly working towards dumping
schemas/types for the database in various formats over here:
[https://supabase.github.io/pg-api/](https://supabase.github.io/pg-api/)

Once we've stabalized this we will start on git integrations - link up your
repo for to keep everything in code - your types, schema, tables, functions,
etc. We are also brainstorming ways that we can offer branching - since we
give a full database server, we can possibly "branch" the default "postgres"
database into various schemas which you can test with.

> ERD diagrams have existed for decades

If you know of a good open source tool which we should be supporting for this,
let me know. Otherwise we don't mind building from scratch!

~~~
RNCTX
Here are a couple of examples in React:

[https://github.com/projectstorm/react-
diagrams](https://github.com/projectstorm/react-diagrams) (diagram library)

[https://www.npmjs.com/package/react-database-
diagram](https://www.npmjs.com/package/react-database-diagram) (an example
using the above library)

There is a Node migration library that seems well received but as I hinted
above I have always trusted Django's migration system and still use it to bang
out schemas if I know the schema will change, so I can't vouch for this lib
personally...

[https://www.npmjs.com/package/db-migrate](https://www.npmjs.com/package/db-
migrate)

~~~
kiwicopple
> projectstorm/react-diagrams

woah. This looks powerful.

Thanks for the libraries - i'll research more and loop back here if we end up
using them.

~~~
RNCTX
If this is worth pitching in a meeting, I'd point out that the creator of
GraphQL is also the creator of Airtable, which seems to be a thriving startup.
The whole business model there is to give people visual tools to get into a
database and out of a spreadsheet.

The story in all of this is that the only thing keeping Joe Q. Public from
using a database instead of a spreadsheet is the very first step: designing a
schema. Database design isn't really an impossible task for a non-developer to
learn, if it can be visualized. If you start out at a command line and try to
explain many-to-many relationships to them in code, you just lost them. People
are visual creatures.

Since we're talking about giving people an API that takes care of CRUD and
basic view logic at the database itself, and returns the data and the
success/fail/delay messaging in a way that is standardized on the UI via
javascript objects and async browser functions, the only hurdle that's left is
that first one...

...designing a schema.

edit: on your topic of branching into test environments, I am in the process
of evaluating low-code platforms for non-profits with limited development
money but complex data needs at the moment, so I think we're thinking about
the same things.

Everyone and their brother is working on a low-code platform at the moment. A
big part of whether those platforms succeed or not is how they handle "last
mile" complex logic. By last mile I mean, you can feasibly deliver basic CRUD
and data analysis visualizations with drag and drop and query binding in
generic javascript UI components. But there's still going to be the 10%
complex business logic cases that don't fall within the 90% of an application
that the generic components and boilerplate don't cover.

Off the top of my head, you could guide a user through some form logic that
picks the first hundred or fifty or whatever rows from their existing tables
that match a certain subset of shared keys, and tells them to flag columns
which have sensitive data. Once the columns are flagged the tool could jumble
the data in those columns and spit out a sandbox without any identifiable
sensitive data that they could then safely hand to a contractor or freelancer
to work on for custom development.

~~~
yen223
> I'd point out that the creator of GraphQL is also the creator of Airtable,
> which seems to be a thriving startup.

This is news to me. Any sources for this?

~~~
RNCTX
I saw it in a conference talk, looking...

edit: [https://medium.com/@calebmer](https://medium.com/@calebmer),
[https://github.com/calebmer](https://github.com/calebmer),
[https://calebmer.com/](https://calebmer.com/)

I misspoke, not a founder of the company but an engineering hire there after
he left Facebook, and the creator of PostGraphile. The gist of what I was
getting at is there is a lot of activity swirling around putting business
logic back on databases, it seems.

We might look back at these days as the high water mark of "peak javascript."

------
RedShift1
A good read but man these PDF's are terrible to read on mobile.

~~~
gen220
Do submitters write academic papers in "native" LaTex these days? Or do they
write in some intermediary format like markdown?

If it's the latter, it'd be pretty trivial to generate a more mobile-friendly
html format. I wonder if arxiv supports something like this.

~~~
scott_s
As others said, yes, academic papers are still written in LaTex. However,
arXiv does not actually accept a pdf for submissions, but require the actual
LaTex sources and they generate their own pdf. As LaTex-to-HTML improves, they
could always generate it for all of the submissions for which they have the
source.

~~~
gen220
That would be fantastic! I hope it’s on their long term roadmap :). They
probably have the biggest corpus of LaTex source in the world, that’d be
pretty useful to the project maintainers.

