Hacker News new | comments | show | ask | jobs | submit login

It sounds like you're not so much in "the" industry but a subset of the industry that doesn't consider free open source software to be an option.

If you're fighting the attitude that the only viable option is one that costs a ton of money instead of a decision based on technical merit, additional facts about Postgres won't help.

Sometimes the decision gets made on a golf course and not after reading very long texts.

>Sometimes the decision gets made on a golf course and not after reading very long texts.

This is why I swore to only write software at software companies some years ago. For other industries software is just another expense like buildings or supplies. The people buying it have no idea what they're looking at so they buy the shiny thing that everyone else is buying.

When I worked in other sectors I regularly saw executives attempt to assemble software as if it was a large building. Materials were gathered, agreements were made to purchase or rent heavy equipment, and the whole project was planned out with exacting detail. When all the permits and contracts were in place construction would begin. This was to be done in a measured amount of code using the chosen architecture, and delivered complete on a certain date. Any deviations were seen as workarounds for not following the original plans, and overruns the result of incompetence. The whole thing was just a shitshow of incompetence at the higher levels.

You're correct that non-software companies will simply outsource but not quite correct that it must be closed source - software is simply something they don't have much expertise in and never will. The problems only show up when software becomes such an integral part of business operations that you really do need to actually do well at software or risk a lot of business. The reason that it tends to be closed source companies is that most of the relationships that the non-software companies have is when open source was really not very viable of a foundation for one's software.

Demanding and exercising the use of a throat to choke is a massive, massive part of how at least American business works due to prevailing leadership styles as well. It also shows how they tend to view their workers as well. "Blameless culture" is something that is a tiny, tiny minority that only seems to have worked out well among socialist-ish boutique software companies and until we start seeing companies that practice traditional Taylorist and authoritarian leader worship cultures fail very disproportionately, nothing will fundamentally change.

I think that is another reason why some companies outsource all aspects of IT, so they can blame someone. Who cares if it takes 10x the time compared to having the skillset in house when you can just point at the 3rd party and blame them. Doesn't matter if it is the same person selecting all the 3rd party providers who continually are horrible at their jobs, we are saving money!

I guess I already covered that in the culture of desiring parties to choke. Whether it's a vendor or your own employees, leaders that rule by fear like to find scapegoats to shift blame away from themselves and take credit for others' efforts. The people that know better are effectively kept away from any form of power or under-resourced, so they'll always be too small to succeed.

I'm familiar with a number of very large deals done basically between C-level to C-level where the scope of IT projects has nothing to do with technologies but entire about cost savings - literally "I will save you $n MM / yr in opex so you can get your bonuses" and other vendors get shut out. Sometimes these deals work out, other times they don't and the executive is basically ousted. Companies with bad politics and enormous cronyism may have worked fine for decades, but they just may not be doing as well anymore unless you're on Wall Street and you make so much money it doesn't matter how it's done.

well, yeah, obviously.

Software is not the end-goal itself. The point is not to make (or use) amazingly elegant software. The point is to make money.

If a supplier says "I will provide the same service as you are currently getting and cost you $X less" then that's a no-brainer regardless of what service they're providing. It's got nothing to do with technology, and technology doesn't change the nature of that decision.

"Having a throat to choke" is also a matter of insurance. You can't insure against your own incompetence, but you can sue a supplier for not fulfilling the terms of their contract. Executives would much rather negotiate what they think is a tough contract with a supplier than manage a complex project themselves. To that mindset, the removal of risk (because if anything goes wrong they can sue the supplier) is a huge bonus.

It's a totally different mindset from those of us who actually make things.

Nobody ever got fired by hiring IBM mindset I guess...

It's worse than that. You're not just choosing a reliable solution, or even a solution with a conservative reputation so that your butt is covered. You're choosing a solution which may actually be worse because when you fail to deliver for someone else, you have someone to scapegoat. If there is nobody else to blame, you might take the blame for having chosen open source with no specific party to blame (the hot potato stopped with you).

Is that not just plan driven development? (I.e. waterfall)

It's not about technical merit or cost - it's about service contracts.

I was a Sybase point person for years at a fortune 50 medical devices enterprise company (we had revenues of well over $1B annually on devices running Sybase dbs). There were dozens of bugs/issues I found that I pushed up the chain to an engineer and had special patches turned around within 48 hours, sometimes even hours.

Before I left that job I started playing around with Rails and mentioned MySQL and Postgres for a potential greenfield project. I was told it would be fine for internal use but no way, no how were they going to deploy any software without that kind of parachute based on economic leverage.

But you can pay for this level of service for PostgreSQL too, there are a bunch of companies which offer it, and I bet you can get it for a fraction of the price you would buy the same service for from Oracle or Microsoft. So this seems to me to mostly stem for a poor understanding of open source.

Some years ago - never mind how long precisely -I was one of the engineers at Sybase who would have produced that EBF patch for you. More recently I do the same sort of thing for clients running Postgresql. I have worked on the code of both db servers, been involved in bug hunting and fixing as well as support escalation for both, and interacted directly with users and developers of both. If my life depended on a medical device, with a choice of one running Sybase and one running Postgresql, I would choose Postgresql, in a heartbeat (so to speak).

FWIW, there's several firms providing such services for PostgreSQL too. At $previous_company others and I, as a PostgreSQL committers/contributors, provided escalation exactly for such cases. I know they, and others still provide such services. (Not naming names right here, so this doesn't come over as an ad)

Often enough you'll even get similar turn-around times from the community, if you can't (or don't want to) afford such a support contract.

> no way, no how were they going to deploy any software without that kind of parachute based on economic leverage.

Making software work at scale without that economic leverage could never work in theory. It only works in practice.

Those in practice fixed bugs in mysql themselves and had to make it more performance themselves. Which basically means larger development team and more time. The parachute is meant to avoid that expense.

Oh, and those who did it were above oracle in terms of data size making it rational decision too. The calculation is still different for majority of companies.

I've gotten better support out of the Postgres mailing list than I have our of any commercial contract.

I spent several years in securities and derivatives trading and the most frequently cited reason for avoiding open-source software I heard was that, in the event of a major foul up, there was no one to sue if you got sued yourself. It's not that difficult for an attorney to paint you as reckless for using "free" software.

I spent 13 years writing the core trading system for many of the well known exchanges. We used open source wherever possible because the software tended to be more reliable. That said, clients usually got to request the database and we used Sybase a lot. I have been using Postgres for the last eight years since. every day of the week and I really like it but the planner is quite a bit worse than Oracle, SQL Server's. The postgres planner is still way way better than MySQL's. It still has correlated subqueries explode into cartesian joins. Mysql is great as a data store but it's more of a replacement for noSQL than an advanced query engine.

MySQL's planner is predictably stupid; structure complex multi-table predicates as joins (nested if necessary) rather than subqueries and it's almost imperative. Postgres OTOH is very unpredictable; sometimes it does the right thing, and sometimes it does something amazingly asinine, where simply swapping a table between from vs join clause can result in 1000x speedup.

Specifically, I've seen pg take a query that looks like this:

  select ... from a join b join c join (select ...) d
where a has millions of rows and is an equijoin with d where d has 10 rows, and it decides to materialize a x b x c, only joining in d at the last step. But do it like this:

  select ... from (select ...) d join a join b join c
and it does the right thing! And analyze gets it right (i.e. the plan for the reordered joins is recognized as better) - never mind genetic optimization, it's lacking analytic optimization.

With the lack of hints, almost the only tool you have to control query plans effectively in postgres is parenthesized joins. Since it's more liable to rewrite the query, the language ends up being less imperative, and thus less predictable. And I like predictability in production.

SQL-level feature set is no comparison of course, pg wins easily.

There are settings for choosing between the exhaustive search planner and the genetic planner. The exhaustive planner is better, but can be slow for complex queries with a lot of paths. But, if your query is at all time consuming you probably want to increase geqo_threshold and geqo_effort as well as join_collapse_limit and from_collapse_limit.

I'd also suggest disabling nest_loop_entirely if you are having problems with bad cardinality estimates resulting in nestloop plans that run 100 times when the planner estimated once.

An interesting argument for the predictability of mysql. Great observation.

It is interesting to see how postgresql will often choose hashmap scan, even with very up to date statistics and much better paths available.

SQLServer's planner does an amazing job of digging right into joins/sub-selects to constrain preliminary results for joins.

It's a very hard job and MS and Oracle obviously have had some of the best people on the world paid well to work on this.

Show me an enterprise DB license that offers you better indemnity/liability options than open source. (I negotiated them on behalf of huge clients for many years.)

It's not so much the license per se. It's that the setup is done by a third party that can be blamed when something goes wrong. I saw one one contract that was specifically saying that they are insured for 1 mio. in damages.

The company for the longest time wouldn't even touch basic firewall rules without having the firewall contractor implement it.

This is so true, it hurts. I call it The Triangle Of CIO Turnover.

In my experience with couple of banks, it comes down to support. Lot of systems in banks are written for longevity. So they frown upon software which might be obsolete or people stop working on them 5 years down the line. A paid software, they reason, can at most release a new version while free might not provide enough incentive for people to work on it continuously.

Many also think looking up issues on stackoverflow, google or blogs as unreliable. Then there are times when issues might be specific to installs or data, in which case sharing the logs/sample data (even masked ones) can be risky. They feel comfortable sharing logs/masked data with for example Oracle because they believe it to be safe and locked under Oracle's security guidelines.

The 2nd refrain I hear is - security. In case of a major security issue being revealed, there is a general sense that FOSS will be slower to react in releasing a "stable" patch. Comparatively paid software take it as a reputation risk and work towards quickly releasing a "stable" patch.

If people have to use FOSS, then they try and search for the paid support flavor. Recently we were looking at MQ software. When we zeroed in on RabbitMQ we were asked to deploy only the paid Pivotal version and not the free version because "support".

Sure, these things might not be completely true but for many higher ups paying for something somehow makes them sleep better at night than a "free" alternative.

This is wrong on so many levels, I suppose you mostly understand it, but here are the counter arguments:

> Written for longevity

OSS is much better at longevity than proprietary. Even if the authors all die without will, it is possible to fix the little bugs that prevent you from using the software on [NewTechnologyHere]. I've done it countless times with Java software; If anything OSS is the guarantee that you own your future and that the system will exist in the legacy.

> Use paid flavor

It's good, but what's better is joining the golf club of a principal maintainer. He's key in paying him to fix the issue you're having quickly and merging upstream.

I think there is more to this than that, though. Software companies usually run R&D at about 10% of revenue. So, these support contracts are really returning only 10% of their cost. And the companies who are buying the support contracts are are usually big. The money they spend on 10 developers for support could pay for 100 developers once you add in license fees, etc, etc.

Long, long ago when I worked at Nortel (a now defunct, but then huge telecommunications company), they used to pay millions of dollars a year to Cygnus to support a particular embedded version of GCC. This, despite the fact that Nortel had more than 10k programmers on staff including a compiler team!

I think the real reason these support contracts exist is because companies (even large ones) don't want to dilute their focus maintaining projects that are peripheral to their core business. It's not so much a technical problem, or a money problem -- it's a management problem. They can't scale out to handle every little thing.

I think OSS is a red herring in this conversation. Most companies just don't care about that. They don't want to support it themselves (even if they are big enough to do so), and they need to have confidence in the company that provides the support. Build that company (hint: you need to be sales heavy!) and you could sell Postgresql just as easily as any other database. Of course breaking into an entrenched area in Enterprise software is always going to be difficult, so I'm not sure how successful you would be with this particular product, but you get my point, I think.

There are several companies which sell PostgreSQL like that with EnterpriseDB and 2ndQuadrant being the two largest ones. It seems like these companies are at least semi-successful since they hire more people all the time. So I agree with your idea, that you just need to convince the enterprise customers that you are a reliable partner.

Most enterprises don't want to own coders to fix/hack OSS. They want a solution, roadmap and 24x7 support.

Sure OSS has a longer lifecycle because a dev can lookup the source code and fix it. But companies don't want to spend twice. For example in case of a DB, they would rather want a DBA to manage the DB. They don't want to hire a developer and a DBA - that's how they view it. Sure if you can find someone who is good at both but they are few and far in between. It is much easier to have an Oracle DBA manage Postgres with paid support than find a developer with enough programming under his belt to ensure he can take care of Postgres issues.

As hindsightbias puts it they want solutions and 24*7 support.

and how often do securities firms sue oracle, microsoft, ibm, sybase and other current and previous database giants?

Nobody ever got fired for buying IBM.

I think is the better point or rewording of your point.

The state government of Queensland (Australia) has forbidden any further contracts going to IBM.


Some of the money gets funneled back into salespeople; that's how enterprise software gets bought. The salespeople target the CTO or another single point of contact.

Regarding "The industry". Okay.

Let me rather call it, The Real World.

I work in a Fortune 500 company, a consultancy of 400.000 people, on projects for other Fortune 500 companies.

My experience is from the real world.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact