In a short matter of years, they took a consulting services company, adopted some poorly designed abandonware from VMWare, and rebuilt it into a product that now serves some of the biggest companies in the world. Pretty much everything has been rewritten and almost all of it is available in github.
CloudFoundry is not sexy software. It's basically Heroku that big stodgy companies can run in-house. Startups will never use it; they can use actual-Heroku. The crazy thing is that Pivotal supports these huge enterprise software installations inside other people's data centers, often without direct access. It's really, really hard work.
I really hope this makes a bunch of my friends rich - they deserve it.
HN's audience is not really the same (broadly speaking). A landing page that shows you how simple it is to deploy a python function into production on Pivotal Foundry would probably be a better sell here.
What happened to it? Why did VMWare abandon it? As far as I remember it was written in Ruby / Event Machine. As I understand it, Heroku also has large parts in Ruby, so that isn't a problem by itself.
How did Pivotal get involved? Were they hired by VMWare, and then they took it over?
Why / how did they rewrite it?
I had some ex co-workers who consulted for Pivotal around 2005-2006, and I have seen their name pop up from time to time. Pivotal tracker seems somewhat popular, and then I remember their name popping up again around Cloud Foundry.
It's nice to see that they turned it into something. It does seem like a nice success story for "open source cloud". This was around the same time that OpenStack was getting going too, but it seems like the success of OpenStack has been more mixed.
* EMC bought VMWare
* EMC bought Pivotal
* EMC moved all the enterprisey software to Pivotal (including CF & Spring)
* Dell bought EMC
* Dell/EMC spun off Pivotal
When I say "pretty much everything has been rewritten" I mean it has evolved like that; they didn't sit down to rewrite CloudFoundry (Pivotal does not do rewrites as a cultural axiom). I mean that piecemeal pretty much every bit of code has been altered in significant ways. A lot of the Ruby has been swapped out for Go for performance and maintainability reasons. It was pretty funny to watch a large body of die-hard Rubyists get excited about static types (come to the light!).
As a consultant I was really a fringe player; I'm sure there are people reading this who are/were much more involved and could tell the story better.
The thing I found most interesting is that this project should have failed. A HUGE incredibly complicated body of enterprise software with near-100% team turnover? I would have bet against it ever working. But all that pair programming and rotation and writing stories and backfilling tests etc just eventually ground the problem down. It was expensive as hell and it took years but it looks like a success story now. I don't know of any other big takeover project like this that worked. It's a huge credit to the people working on it, and yes - to the "Pivotal process" that seems to irk so many people in this thread.
I agree with the philosophy of "grinding it down". Writing quality software is largely about doing the straightforward thing, and not regressing, for years on end... At a certain size, it's too big for any heroics to make a difference, and you have to rely on plain old "engineering" (tests, prioritization, etc.)
Pivotal Labs got bought by EMC in 2012, who also owned VMWare. A year or so later EMC spun out Pivotal Labs combined with VMWare and some other companies(Green Plum) in a new company under the Pivotal name.
Cloud Foundry is VERY mature nowadays, is completely opensource and is part of the Linux Foundation. You can view it as Linux, which is the vanilla opensource version and which has different distros (CentOS, Ubuntu, ...)
Cloud Foundry is the vanilla opensource version, and there are many (commercial) distros available, like Pivotal Cloud Foundry, IBM Bluemix, SAP Cloud Platform, etc...
I'm late to the game here; was it GSX Server a.k.a. VMware Server?
Spring has never been abandonware to the Java community. It is probably the most used framework for enterprise - I've seen maybe 10 jobs looking for Spring experience vs the JEE crap that it has mostly replaced.
I know it gets a bad view on HN - a lot of FactoryBuilderFactoryBeans and a rather large kitchen sink of APIs.
But the framework itself is well written, great docs, lots of resources.
I've been involved in a dozen different projects where the architects hated Spring and instead went to some newer hipper lang like Flask, Django, Rails, Finatra / Finagle. Spring has a learning curve, but once you get it, it is far more powerful and stable than any of these alternatives.
This is a really cool risk factor that I've never seen before in an S-1, exciting that more big public companies are willing to accept the risk reward that comes with contributing to open source:
If open-source software programmers, many of whom we do not employ, or our own internal programmers do not continue to use, contribute to and enhance the open-source technologies that we rely on, the market appeal of our offering may be reduced, which could harm our reputation, diminish our brand and result in decreased revenue. We also cannot predict whether further developments and enhancements to these open-source technologies will be available from reliable alternative sources. If the open-source technologies that we rely on become unavailable, we may need to invest in researching and developing alternative technologies.
"If open source software programmers, most of whom we do not employ, do not continue to develop and enhance open source technologies, we may be unable to develop new technologies, adequately enhance our existing technologies or meet customer requirements for innovation, quality and price.
We rely to a significant degree on a number of largely informal communities of independent open source software programmers to develop and enhance our enterprise technologies. For example, Linus Torvalds, a prominent open source software developer, and a relatively small group of software engineers, many of whom are not employed by us, are primarily responsible for the development and evolution of the Linux kernel, which is the heart of the Red Hat Enterprise Linux operating system. If these groups of programmers fail to adequately further develop and enhance open source technologies, we would have to rely on other parties to develop and enhance our offerings or we would need to develop and enhance our offerings with our own resources. We cannot predict whether further developments and enhancements to these technologies would be available from reliable alternative sources. In either event, our development expenses could be increased and our technology release and upgrade schedules could be delayed. Moreover, if third-party software programmers fail to adequately further develop and enhance open source technologies, the development and adoption of these technologies could be stifled and our offerings could become less competitive. Delays in developing, completing or delivering new or enhanced offerings could result in delayed or reduced revenue for those offerings and could also adversely affect customer acceptance of those offerings."
"If open source software programmers, most of whom we do not employ, do not continue to develop and enhance open source technologies, our development expenses could be increased and our product release and upgrade schedules could be delayed."
PCF is very solid. The only issues we ran into were some confusing configuration settings and your basic federal government bureaucracy nonsense. Once we got all the IaaS boxes ticked, it installed without issue and worked beautifully.
I'd never want to install or maintain open-source Cloud Foundry as it's a nightmare of complexity; however, PCF is a different animal because of its streamlined installer. I definitely recommend giving it a go if you're looking for on-premises PaaS and you've got the IaaS layer to support it. You can also demo PCF within a single VirtualBox VM.
That said, I just can't understand why you'd ever pick PCF over Kubernetes. Sure, I get one deploys application code, and the other deploys containers. But after using the buildpack system for a while, I really think containers are the better solution (assuming you have the CI/CD to keep your custom containers up-to-date with new base images).
So it'll be interesting to see what Pivotal does there, and how they position PCF, especially now that they have their own branded version of Kubernetes.
And having someone provide a more enterprisey K8S is helpful. The version churn right now is pretty bad, and not something F500 companies want to deal with.
The thing that bothered me with the model was it disincentives modern microservice architectures. One big monolith cost less than lots of smaller components.
It may have changed since I last saw the details.
I hope for Pivotal's sake that the eye-popping price is not crucial to their growth forecast. If it is, I think 2019-20 will be difficult for them. They just don't have the market share to force these prices down customer's throats.
The question is: are the $800k customers getting good value for that subscription, or are they looking at projected costs and thinking "we better switch to a competitor before this gets out of hand".
(Though in enterprise sales, even amazingly poor vendors often get to re-bill for several years, until the exec who signed off on the subscription moves on and it becomes politically possible for people internally to admit to each other it was a stupid decision to sign up in the first place... So perhaps 2 years growth here is only telling the "sales capability" side of the story, not the "ongoing value provided" side...)
And the other similar version: tightly fought procurement between two opposing vendors with different internal "sponsors". Winning vendor's ability to deliver is stymied at every opportunity by losing sponsor or their internal supporters. Then the few years of abysmal results happens in spite of winning vendors capability and efforts - same outcome plays out.
I think the thing is that in consulting, the equation is "margin = billable rate - salary". The higher the salary, the less profitable you are. Feels to me like Pivotal mostly hires mid-level people to hit the sweet spot of experience/skill/salary.
It's a bummer because I went on to work with a company that engaged Pivotal Labs and I loved working with the Pivots.
Not only are many of his claims dubious, unwarranted, or hyperbolic, but there are a few gems like this:
> Remember when the CEO globally emailed a picture of his ugly newborn and drugged up post-birth wife to the entire company because he thinks we all submit to his cult of personality?
It's not uncommon for that to happen. People share pictures of their new kids to their departments and whole companies all the time. Usually it's because new kids are a joyful experience, for, y'know, humans in general. CEOs do it too--for cynical reasons or not--and raging against that, to say nothing of the criticism of the "drugged-up" mother (epidural anesthesia is incredibly helpful and medically essential in tons of births), displays a somewhat shocking failure to grasp typical corporate-people behavior in America at best, and an axe-to-grind willingness to engage in ad-hominem slander of people's families at worst.
Even if any of his claims have merit, I have a ton of trouble believing anything from someone who talks like that.
Full disclosure: I am currently employed by another member of the Cloud Foundry Foundation.
Not sure how honest the metrics around success will actually be...
Almost every company I've been at has used abhorrent alternatives like Asana, Jira, etc. and they've been consistently awful experiences.
The one tool that I've been loving for the past year is http://clubhouse.io. Such a well-designed product and extremely flexible (I think it could fit most teams' requirements around Agile/Kanban/etc.)
It was the absolute __worst__ year of my life. I had to pair with people non-stop even when __fixing bugs__. There were zounds of stupid rules which did not make sense whatsoever.
Perfect example of forcing a methodology on a company where it will never work.
I really like Pivotal products though. RabbitMQ is superb, and although Spring Boot is really slow and has its problems I use it on projects where performance is not important.
After trying so many of these methodologies I've come to realize that people just work differently. It's self-defeating to try to push Extreme Programming or whatever on someone who prefers and is efficient at working alone.
Maybe adopt some of Scrum/XP/whatever that can be adopted at a high-level, but trying to push it through a team composed of people with different personalities and different ways of working just defeats the entire purpose.
It's hard to shoehorn it into an existing team where some portion of the team doesn't believe the premise. For example, some people (introverts, Asperger's spectrum, etc) will just never be comfortable with pairing.
Pivotal doesn't have this problem because developers that don't believe wouldn't ever apply for a job there.
But, that doesn't help Pivotal in their consulting practice. Very few of their clients would have the same advantage.
CF even can't even survive unplanned reboot of DC, after such incidents you literally will get parted cluster and no one know what to do.
Thank you Pivotal for such experience.
I did a couple consulting stints at Pivotal, working on the CF team. They have a very particular methodology that evolved from the earliest days of agile (Rob Mee was a friend of Kent Beck back when XP was being worked out). I walked in there as a very experienced engineer and a healthy amount of skepticism... and it turned out to be a fantastic learning experience. I've since taken the Pivotal process (yeah, even the weird stuff like pair programming) and used it effectively to manage subsequent companies.
I don't love everything about Pivotal (neither Ruby nor Go make my favorite-languages list) but then, not everyone at Pivotal does either. But overall they do agile right, and they have a lot to show for it. You can mock 9:06 but also note that people go home at 6pm and have lives. The schedule has its upsides.
I think it's pretty clear that these strict agile processes work well for some and poorly for others - some people do better in a highly structured environment and others do worse - it's a bit like remote work on the other end of the spectrum.
The criticism and pushback comes from the negative experiences devs have had when management pushes a one-size-fits-all approach.
I've been involved in more than 5 failed agile methodology process improvement initiatives. I've seen great engineers reduced to a tiny fraction of their former productivity and seen other "hopeless" engineers get a lot better.
So any positives can safely be attributed to the new process and any negatives are clearly just the new process shining a light on existing problems.
And the parent poster was wondering why people were so cynical.
> making your engineering team more stable
Can you expand upon what you mean by more stable?
One of the main selling points to management is it becomes much easier to switch developers between teams.
> product delivery more predictable.
In what sense? And through what mechanism?
It does reduce risk on short term deliverables but that's a very narrow interpretation of predictable.
It's certainly not the case as an external customer - trying to nail down an xp team to a fixed deadline more than a few weeks away is an exercise in frustration.
Specialization is highly discouraged, so the result is a mix of people who can kinda accomplish most things eventually, and a rush to find external resources when you need even moderately specialized knowledge about components in your stack.
I am no capitalism apologist. The interoperability of technicians diminishes our marketplace bargaining power, thus diminishing labor's leverage over management/capital. At this point in my life, I have accepted this trade-off in exchange for
1) mutual code-review,
2) nights off on-call,
3) opportunities to learn new tech,
4) a product roadmap that can be prioritized without a freaking gant-chart to slot e.g. language-dependent work into language-adherent technicians' backlogs.
While these and other factors make my role less stressful, and my team more effective, I do concede that knowledge dissemination diminishes tech workers' bargaining power.
I'm not defending them, but I know I'd love for my shop -- we do the same things as them -- to grow in this way.
If they're successful, you're left with a team who adheres to or even enjoys the rules and rituals laid down by Pivotal, and you end up with that team I described who can "kinda-sorta mostly get anything done... eventually... with the help of a few external vendors..." From a business perspective, this isn't a terrible place to be.
If they aren't successful, oops, your entire team churned during your engagement with Pivotal, but don't worry -- they're happy to help out with your hiring processes.
I'm sorry you had such a disheartening experience at Pivotal; it's uncommon — in general, Pivotal people tend to be very warm and kind, and not snobby at all.
When I joined Pivotal six (seven?) years ago, I assumed that I'd stay maybe two years, tops, but the people were nice, and that's what has kept me here.
Also, the comment you heard ("Oh I think we already know how [Database X] works") may have not intended to be snobby but rather a nod to one of the developers there: if the database company was InfluxDB, and the office you visited was the NYC office, then there's a good chance that they were referring to one of the Pivotal developers, John, who was one of the early developers of InfluxDB along with Paul Dix and Todd Persen. 
Best communicators I have ever worked with.
But, I do really like a lot of their processes. Pair programming isn't for everyone, but I think it's a better way of building software. Now that I'm not doing it full time, I constantly miss the benefits of it.
Their strict schedule is kinda nice for work life balance, but it also means working with them was a little strange. Our whole team would go out for happy hour at 5, but they would stay until 6, even with almost nobody else in the office.
The place sounds more like a cult than a company.
I'm glad I read this. Now I'm sure to never apply for this company
Standup at 9:06 is important, though. In order for pairing to work, you need to have everybody there at the same time.
The iron bargain is that you work a rigid eight hour schedule, but then you go home (or wherever) and don't think about work. No overtime or crunch mode, ever.
It's not for everybody, but many folks really really really like it.
However having flexible time is a more important asset to me than doing the occasional overtime
I didn't believe you for a moment, but wow:
I could find you 100s like this just in the Bay Area.
The joke was always, "You're white, why aren't you the manager?"
Besides consulting and software development (Labs), their main product is PCF (Pivotal Cloud Foundry). Basically it helps combine the best bits of cloud centric, automated, containerized tools - without locking you in and letting you be portable across AWS, Google, Azure or private clouds.
The big thing here is that there is a massive opportunity of large companies still managing large monolithic apps with slow, expensive horribly inefficient infrastructure and deployment practices. Pivotal helps (especially bigCo.'s) rebuild their whole app infrastructure as well as transform their culture of software development. The biggest thing, according to their case studies, is that both speed of development as well as the ratio of 'ops' headcount to 'developer' headcount can be significantly improved.
Pretty smart way to do it - and I can see how it would be really aggravating from a power-user/10x engineer perspective.
You can run whatever you like, even Windows stuff. You only need to have (a) build pack(s) (aka runtime layer) and a stemcell (base operating system).
CF is like Heroku on steroids or like Lamda+API Gateway+Services for more complex applications.
And Pivotal provides one product of Cloud Foundry (
Pivotal Cloud Foundry), that can be deployed to various Cloud Providers (such as AWS, Azure or OpenStack). There are other vendors, like IBM (Bluemix).
I guess, you mixed it up with the Spring ecosystem, which surely a big driver in the adoption of Cloud Foundry. Pivotal is the core maintainer of the ecosystem.
And no, you don't need to be a big enterprise to run a CF.
I also suspect that providers will not keep feature-competitive with one another. It seems like every month, AWS is releasing new tools, and there's no way that other providers are maintaining this pace. My suspicion is each provider will specialize in non-overlapping domains. So unless your company will always be running a lowest-common-denominator site, you'll eventually encounter a vendor lock-in feature.
This is the level PCF plays at; and the level they’re useful, but in my experience Pivotal doesn’t know how enterprise companies develop software. Their scrappy pair-programming Dojos don’t really build systems the way those kinds of companies need to design system requirements.
So in my mind, Pivotal’s consulting services are a bunch of startup guys coming in to the enterprise world telling enterprise developers how to build java services. They’re smart and know what they’re doing, but they’re totally out of touch with the kinds of internal controls, reporting metrics and architecture management that larger companies often require.
Its just extremely hard for the old "enterprise world" to adopt to change that is inevitable.
But again, this is really only useful if you’re big enough to have many application groups developing for a bunch of different business units. I have yet to see a good use case for PCF outside enterprise IT departments.
Large competent software companies like Facebook and Google have still not completely integrated Instagram and Waze respectively for example.
They are kind-of the OG multi-cloud vendor.
I also had to do work with Pivotal HD, which was their custom Hadoop distribution. Not sure the status of that these days, they may have nixed it - it's been a while since I worked on that platform.
We’re back to this model of “innovation” I guess.
But even actual businesses can get a nice boost by filing along with a successful related IPO.
Spring Cloud includes a lot of the Netflix OSS stuff though like Zuul and Archaius.