Initial experiences with Rackspace in late 1999 and early 2000. I was amazed that every time I called 'support' I got a real live person. I called at 3am once on a Saturday, got a live person answering, and was connected with a competent support engineer in about 90 seconds. I was impressed.
The server(s) we had cost a lot, but knowing you could connect with someone competent 24/7 was amazing.
Then... a few months later... we had to use it, and it was good, but they were about 2 minutes away from wiping our main drive. We'd requested a new beefier server and requested either the drive be moved or everything copied over to the new server.
Got a message new server was up - great. Logged in - nothing. Connect to old server? Nope - offline/disconnected. Ugh. Calling in got me a live person, and I stressed the urgency. We did get connected with someone, and they did find the right person, who was about to wipe the previous server. Crisis averted, but... immediately I start asking why the support ticket showed all requests as 'done' when... they weren't.
We stayed a while longer, but it felt like they were growing too quickly even then to keep the 'fanatical support' model that initially attracted us. FWIW, I had a couple of dedicated servers at a small place in NH, and they were... $70-$80/month on average. The rackspace servers typically started at $500/month. I think we were paying about $1800/month for the setup in question at the time of the crisis. My belief in 'you get what you pay for' was impacted that day.
I had another client years later (2010? 2012?) that used them and it felt like a completely different beast, far more generic and commodity like every other competitor, and I've never since looked at them again.
Similar experiences, they were amazing for the first couple years I was on a team that worked with them. They really provided amazing support and took a lot of the burden of managing infrastructure off our smaller teams. Then it changed, the writing was on the wall as public crowds grew but one of my last experience was of them wiping out one of my databases doing what should have been a non-destructive drive expansion. The kicker was they restored from backup, a backup from the previous evening and didn't bother to inform us. We found out when customers started calling asking where their previous days data was. Accidents happen but the whole cover up was unforgivable.
Any company, even a startup, should have procedures in place for what to do when data loss occurs. I would hope that any service I’m a paying customer of would proactively reach out to customers in the event of data loss. Blaming the cloud service provider isn’t enough because backups are ultimately your responsibility. But doing the “right” think is pretty tough if your cloud provider is not being honest with you.
Rackspace hosted Techcrunch after we we loved to death on the old host and every single person there that we interacted with what not just competent, but went out of their way to make sure we were live and thriving.
During one weekend the staff there split our wp instance between a separate web and db host + varnish without us even noticing because of increased traffic
The additional hosting cost (which they waived for us in exchange for a sponsorship slot) were more than worth it when you consider what consulting costs were/are. I realize this might be a biased experience because of the high profile of TC - but i referred many people there during that time and only had awesome ++ feedback which was similar to our own experience
I knew something was up when a lot of those smart people who were on the email chains suddenly responded with mailbox not found because they had left.
It made sense then that the model of having competent people managing servers for small-medium businesses doesn't really work since those same people can earn much more while you're only ever going to get ~hundreds to single digit thousands from customers for that level of service. The economics of hosting + what is consulting, and to clients who pay bills on credit cards, just doesn't add up
It made even more sense now after reading this article and knowing that the techs were pushed out by MBA's and finance people.
I had very mixed experiences as a customer in ~2009 - it was definitely possible to get through to someone very competent, but it was pretty difficult. We were trying to use their "cloud" VPS offerings (I believe it was openstack-based, and I liked the idea of supporting something more open than AWS), which were extremely buggy (for example, we had to default-retry every api operation multiple times in scripts because pretty much every type of call failed 40% of the time.) I generally had better success getting information about outages by reaching out employees I knew on freenode than I did through official support channels.
> He puts the blame for Rackspace’s deepening financial struggles — it’s posted a steady string of quarterly losses, and the value of its stock has fallen 80 percent in the past year — on its replacement of tech-oriented leadership with board members and managers “who don’t have any connection with the product.
This seems to be the natural cycle for the vast majority of business, especially when the original founders leave or get pushed out. They enter a long period of stagnancy before the weight builds up and it collapses, or more often gets absorbed into a larger firm seeking a monopoly.
The longer term Google/Meta stories are the rare exception. And that's not always a bad thing by any means.
I think this is just a story we (fellow tech-oriented types) like to tell ourselves and repeat, not because it's true, but because it sounds good.
If you look, there are actually plenty of examples of professional management or 'bean counter' types taking over companies and running them more successfully than their former tech-oriented management. And also, plenty of examples of tech-oriented management ruining a good thing. But those kinds of stories don't get very good play on Hacker News.
Its definitely not 100% true in either direction, but I think an "actual connection" to the product is important. Obviously YMMV on what an "actual connection" is, but to me that just means an understanding of what your product is / who your customers are / why your company has been successful in the past that goes beyond case study aphorisms like TAM / YoY Growth / etc. It's important because when things start going poorly, you need to understand where to lean in or divest.
There are definite cases of corporate looting by the bean counters, but there are certainly cases where a moribund company was saved by the counters of beans (and of course, some where the malaise was so deep nothing could have saved it).
Judged by earnings and general business success (not "do tech people like what they are doing?"):
- Steve Ballmer @ Microsoft
- Tim Cook @ Apple
- Sundar Pichai @ Google (MechE by education)
- Elon Musk @ Tesla (he is not an auto engineer, so functions as the monied management)
There are a lot more examples out there. However, recent vintages of tech unicorns have been slower to replace founders in part due to dual-class stock structures that can give founders final veto over shareholder actions.
Steve Ballmer work in Microsoft since 1980 years as employee 30th, arguably part of the founding team. I won't say he's a bean counter and couldn't relate to the products.
"Just" is doing some heavy lifting there, as if it's "just" easy to run a trillion-dollar company with tens of thousands of employees spread around the globe.
It's tech & tech changes fast. It would be relatively easy to generate a list of companies that had massive growth momentum that then stumbled when handed off to new management. All of those businesses have faced serious challenges (most recently: the pandemic & high inflation), and their managements have performed well nonetheless.
Except for Elon Musk, who is/was extremely passionate about the product, the names on the list are not the peak of these companies. Steve Balmer performed reasonably at Microsoft after Gates, Tim Cook arguably after Steve Jobs and Google is lately a hit or miss on many fronts.
I chose these leaders specifically because their tenures saw their companies earn tons more money than when the founders were in charge.
For example: Jobs was visionary, but Cook's tenure has seen the launch/expansion of the Wearables segment and the Services segment. Those two are "only" a minority of Apple's revenues, but together form a business larger than Comcast or Meta or Target. Tech people won't give Cook credit for that accomplishment, but that's the point of this thread. :-)
One of my favorite examples recently is Frank Slootman (sales/operations/general management background) taking over as CEO for Snowflake, over Bob Muglia (developer/technical product background). The company's market cap is up >10x since Slootman took over.
The time scale that success is defined on is important here.
The root of the issue seems to be that manager types don't view profit as the market's reward for making a good product; they view it as the product itself.
If a product's quality is sufficiently high to begin with this can be great.
Once quality dips to the point where it softens demand though, it can be the beginning of the death spiral [0] (tl;dr: when production chases revenue downwards and the company suffocates under its fixed costs), and market perception/demand lags cost-cutting product changes so the causes and effects can be very difficult to link through financials alone.
Delta Airlines has improved drastically, both financially and in terms of the product since their current CEO, Ed Bastien who is an accountant, took over.
Meanwhile, Boeing sold deadly products under leadership by their engineer CEO.
I see the "bean counter bad" narrative a lot here, but I don't think it's that black and white.
The most important bad decisions in the Boeing case were made in 2011-13, when they promised that the 737 Max would not require a new type certificate for someone with a 737 cert already, and that it would use the new, larger, and more efficient engines. At that point in time, the CEO of Boeing was James McNerney, who had a Harvard MBA, worked at P&G and McKinsey, then spent 20 years at GE (running their airplane engine business). When he lost the race to succeed Jack Welsh he went to 3M and then was picked to run Boeing.
At the time that these key bad decisions were made for the 737 Max, Dennis Muilenberg- the engineer you cite- was running the Boeing Integrated Defense side of the house and had no authority on the 737 Max. He does bear some responsibility for the Max- he was CEO when it first flew, got certified, was delivered to customers, and the first crash that was not investigated with appropriate speed- but the most important bad decision was made when a MBA was running the company.
International airliners and semiconductors are two areas where I've become convinced that engineering CEO's are required, because they both involve making 10 billion or more dollar investments that won't even enter the market for more than 5 years. So you need someone with a very good feel for technological possibility, one who understands that you can't bafflegab mother nature the way that Harvard's MBA school teaches you to.
It's possible I'm over-indexing on Boeing and Intel: Airbus' big recent fiasco- though since it didn't kill anyone it's not nearly as big as the 737 Max- the A380, was largely a commercial failure not an engineering one (by some estimates the A380 never even turned a profit on fly-away costs, totally ignoring development costs). So maybe having Harvard MBA's and market people would have saved them. On the other hand, when those key decisions were made (2000 was when the A3XX was approved) Airbus had a convoluted two CEO-two Chairman structure designed (to this American's jaded eye) to prevent any personal responsibility from leaching back to the bosses, so I can't tell you who was in charge when the wrong decisions were made for that program.
"When people say I changed the culture of Boeing, that was the intent, so that it’s run like a business rather than a great engineering firm. It is a great engineering firm, but people invest in a company because they want to make money." - Harry Stonecipher, Boeing executive and CEO until 2005
Reportedly at Boeing the culture at the executive level remains 'bean-counter focussed'. Does the opposite hold for Delta Airlines?
It seems to me like the disposition of the CEO matters a lot more than their formal background. If an engineer rose through the ranks by siding with the MBA/accountants/bean-counting-culture in a company, he is likely to run it in the (presumably correctly maligned) bean-counting ways. Whereas an accountant who understands the bean-counters but has seen the pitfalls of that kind of management could easily decide not to manage like a bean-counter.
One cool fact is that prior to running Red Hat, Jim Whitehurst was COO of Delta Airlines. He could absolutely talk the talk on open source software. I always thought that was cool.
I think it’s also a canard that the CEO is in charge. They’re responsible, but often the CFO or COO are in charge, or some rule class culture of middle management. My understanding from engineers at Boeing it has not had engineers in charge for a long time and it’s entirely run by accountants and business development.
> Boeing sold deadly products under leadership by their engineer CEO.
I remember hearing I think in a documentary that the merger with McDonall-Douglas made it have a more business-y culture that lead the the terrible failures.
McDonnell-Douglas is often blamed for changing the Boeing culture, but it was the Boeing CEO Philip Condit who led the deal, and more importantly helped engineer the cultural takeover by strategically placing McDonnell-Douglas executives in key positions. Condit was at one point a lead engineer at Boeing, but was deliberately chosen by the Boeing board (at least the majority faction) precisely because they knew what he wanted to do with the company.
The person who pushed back the hardest against the McDonnell-Douglas merger was the Boeing CFO Boyd Givan. He was quickly ousted not long after the deal was finalized, accused of being too financially conservative compared to McDonnell-Douglas' M.O.: https://archive.seattletimes.com/archive/?date=19980715&slug...
The blame ultimately rests with the Boeing board. There was a minority faction on the board that fought tooth-and-nail. They couldn't stop it, but IIRC their concession (to prevent the minority from creating a ruckus in public) was being able to keep Givan as CFO after the merger. But Condit ensured that that didn't last long by undermining Givan at every turn, which was probably the plan of the majority board faction all along.
Based on this and a few other anecdotes, and if we're making wild speculations, I'd say that whether a CFO makes a good CEO depends on the type of CFO. There's the traditional CFO--the "bean counter"--whose job is to manage the money and keep a low profile so the rest of the company can do their thing. Then there are the new generations of CFO, which started coming up the ranks in the 1970s and 1980s (see, e.g., McDonnell-Douglas), who see creative accounting (aka financial engineering) as a legitimate way to boost stockholder value and an end in itself, independent of a company's core competency. If a company has good bones, a traditional CFO can make a great steward at the very least. For the new type of CFO, they often couldn't care less about a company's core competency, and are more speculator than bean counter, at least in the traditional meanings of those terms.
The airline industry is more or less an oligopoly and perhaps even a commodity service, so growing market share kinda means not sucking. The basic experience of an airplane (getting from point A to point B quickly) hasn't changed much. I can see how a MBA / spreadsheet type at the top can add a lot of value here.
Rackspace on the other hand is in the middle of a pretty cut-throat market segment, with lots of competitors and a large need to keep up with them feature wise. You absolutely need a visionary / this is what our market will look like in 3 years person running the show here. Rackspace could have filled a need between Digital Ocean and AWS, but I don't think they ever saw it that way....
They're making money, that's for certain. They're not making new product categories and markets like they used to, though, just iterations on things they were already doing well enough (and sometimes those iterations take away more than give). Their established capital and position in the market are their strength more than any other particular vision or talent on display. That's enough to keep them strong for a long time on logistics. Not sure what happens if they face a well-funded vision challenger.
Find accountants are actually much better at C* positions than MBA’s.
It seems like there should be a lot of overlap, but I think accountants have a better grasp of the underlying business reality. Or at lest the idea that you can’t get something for nothing so saving money on X likely costs you elsewhere.
The Boeing engineer CEO who replaced the accountant CEO (one of the first in Boeing history) under which most of the preliminary design of the deadly product was performed.
iirc, Ed is a career DAL employee who bootstrapped to the CEO position, so while he's not an ops guy, per se, he still "gets" Delta and what makes it the company it is. That's a trait I suspect you'll find makes in every successful chief executive, irrespective of the industry.
It’s interesting that those are the people you chose. Were you unaware of the background of the engineers?
Tim Cook: degree in industrial engineering and by all accounts an operational wizard for supply chain management. That’s why he was COO of Apple for long time focused on how to get components into its iPhone.
Satya Nadella: electrical engineer with a CS degree and then an MBA and at Microsoft started as an engineer.
Can’t speak to Jamie Dimon as banking and finance is outside my wheelhouse, but certainly neither Cook nor Nadella are examples of MBAs who don’t understand their product line. Arguably Apple’s is a much simpler product line to understand whereas Nadella has many more B2B and B2C product lines, but I don’t think the only thing Cook is thinking about are cost downs.
I don't think my choices were particularly interesting, I'm was just loosely aware of their backgrounds and confirmed them as MBA graduates with a few searches.
The parent comment (and a common HN trope) was about how awful MBAs are at running companies. Those are 3 counterexamples to that trope. All three are highly successful in their CEO roles.
Given that Engineer -> MBA is a pretty common path, maybe we should just stop with the broad strokes?
Yes, and especially for people above a certain age, an MBA--whether a regular two-year program or an executive program--was often treated as table-stakes for a lot of roles at companies that weren't hands-on engineering roles. And, yes, in many cases, MBA grads who were looking to work at companies in computer-related fields had some sort of engineering undergrad degree.
I think that the broad strokes capture something real. Namely that an MBA does not qualify you to run a tech company.
As your examples show, it also doesn't disqualify you. In fact there are plenty of roles where it brings value.
But the model of "I have an MBA, I can run any kind of business" is dramatically false. You need to have useful theories about how the company actually works to run it. And the theories by which tech companies work usually make a lot more sense if you have a tech background.
There are exceptions - for example Meg Whitman did an excellent job of building eBay when she took over early on - but they are notable as exceptions. And I know of no exceptions among companies who are in one way or another primarily selling their tech expertise.
In my comment MBA is a state of being, not a conferred degree.
Jamie Dimon comes from financial services, which is a very different industry. Engineering minded firms in financial services were sponsored by finance folks that believed engineering / science / math / computer techniques towards finance would make them more competitive. Internally at JPMC Dimon invests heavily in tech and gives tech engineering leadership autonomy. So as CEO he has delegated a lot to the engineering culture and shielded it from the accountants and MBAs. (Again, MBA the state of being which can not be attributed solely to spending two years in a program)
Nah, just an error of description. The question is whether the person you put in charge is deeply knowledgeable about the product, capable of working on the product, and using that to make decisions. As opposed to someone making decisions in a trained-for-generic-business way.
MBA vs. not-MBA is an oversimplification. Especially since someone can have an engineering degree or whatever and an MBA. But that's not a no true scotsman. There is a pretty clear dividing line between the two groups.
Broadly speaking, anyone drawing broad conclusions on the basis of education is wrong. Anti-MBA, anti-engineering, anti-non-Ivy, anti-Ivy, et cetera. To the degree there is information in these conversations, it’s someone holding these broad-brush views being generally unqualified for higher leadership.
Are there good examples of people like this that then rose to the position of CEO and failed? There are plenty of CEOs who have failed, so is a disproportionate amount these "short-path" MBAs who have failed after climbing so high?
Microsoft has been "doing fine" financially, but look at their products and user feedback.
Windows Vista and 8 were widely seen as colossal failures, Windows Mobile got utterly destroyed by a series of dumbass decisions, Windows 10/11 is a mess UI-wise, where the fuck does one even want to begin when it serves ads in the start menu, vendors keep pushing bloatware crap because Microsoft allows them to, developer experience is a mess because there are just so many different UI and other frameworks (although tooling does have improved since the days you had to shell out thousands for Visual Studio), Microsoft software in general is riddled with bugs and security issues to a degree it puts even Adobe Flash to shame, its Office suite is a hotbed of UI issues, bugs and privacy issues...
Microsoft is only being kept alive because of business moat, the competition being too focused on fucking up each other in bike-shedding and purity contests (a general issue in anything Linux sans, to a degree, the kernel) and doesn't have the money to hire actual UI/UX experts or, in the case of Apple, is just too expensive.
Windows is no longer Microsoft's main focus, and hasn't been for quite a few years. It's about 12% of their revenue. The main income is now Azure, Office 365, and an assortment of other smaller (comparatively) things like Surface, X-box, GitHub, etc. If Windows revenue would drop off to 0 tomorrow it would be a blow, but Microsoft would survive just fine.
People keep thinking about Microsoft in 2001 terms but it's not the same company. This what Satya Nadella brought to Microsoft (don't think it would have played out like this under Balmer).
On one side, people recognizing when they have lost, the competition is inarguably better and converging to that as a new standard.
On the other side, projects aiming to replace sometimes decades-old infrastructure like init scripts also have to be more open towards community feedback.
Both sides need to relax, the effort lost on clearly doomed projects is absurd, and the sometimes extremely heated and rough, barely civilized "debates" continue to alienate new and existing developers.
People whine and complain all the time about "codes of conduct" - and sometimes for good reason, power tripping committees are just as bad as what they want to prevent - but it is clear that all the infighting is the major factor keeping open source software from widespread recognition.
I don't know of Mr Dimon, but tim cook headed apple while they pursued form over function, removing ports and creating the god awful butterfly keyboard for.... reasons?
Satya has overseen microsoft to be very profitable yeah, but they haven't exactly been the best company, buying a bunch of popular companies that most people probably would rather they not own, and investing as much as possible into making using windows a terrible, ad ridden, telemetry infused pain.
But they spend four whole dollars every other day buying coffee!!! If they save two and a half of those coffees they could buy a whole banana! Surely they will be a millionaire like me in no time right?
The general counter-point is, why would you want someone who doesn't understand the work done by the company to run that company.
Besides that, the polarization between management and 'the work floor' naturally leads the work floor to see someone who studied 'management' as the epitome of management, and thus as the epitome of 'the enemy'. Personally I believe the work-floor is right to be skeptical of the management style that is commonly taught. Because the style is not based on treating employees as humans but instead about extracting a much value from these human resources as possible.
Only if your curriculum is from the 1950s. Modern leadership & HR programs emphasise multiple benefits of happy workers:
* More productive
* Less likely to leave (which is expensive)
* More motivated to share cost saving ideas which can save companies millions of dollars annually.
Healthy companies can employ workers longer too.
A good manager doesn't need to know how to build the product. They need to understand their market, the company's strengths, make strategic decisions and create an inspiring culture.
> They need to understand their market, the company's strengths, make strategic decisions and create an inspiring culture.
I'd say that's largely orthogonal to studying business. You need to understand why customers buy from you and what kind of value you are offering. An MBA isn't going to help or hurt on that dimension.
Studying algorithms doesn't help directly with many coding tasks, but does provide people with a way of reasoning. It also makes people aware that eg some sorting algorithms are better than others.
Similarly, studying business makes one aware of a wider range of issues and tools, and case studies allow learning lessons from others.
If an MBA doesn't help with those things, you should ask for your money back.
That's true, but he got a degree in electrical engineering and a masters in computer science first, and picked up the MBA later in his career while at Microsoft. When people complain about MBAs they usually mean people who's primary qualification is an MBA.
Nadella got an engineering degree in India, followed up with a masters in the US - with that Booth MBA coming after he'd worked at Sun and Microsoft as an engineer.
Pretty much every "Wow this MBA does a good job of running a tech company!" person has a story similar to this where they trained and worked as an engineer before obtaining their MBA.
Go read “The HP Way” by Dave Packard. It was standard procedure for HP to hire promising engineers from Flyover State U and take the best and send them to Stanford for an MBA. This worked really, really well. Until it didn’t.
That was the intent of the MBA, as a supplement to some other core expertise so they could manage the business effectively. Then, MBA's became an entire profession. Many consulting firm MBA's go to business school twice before having any real world experience (undergrad then MBA). I did business school for undergrad and refuse to get an MBA for this reason, I seriously had MBA students sitting in the classes I took as an undergrad.
I had to maim the Windows Update process to keep it from freezing my computer whenever I tried to use it.
Clearly right when the user logs on is the correct time to install updates — pinning the HDD to 100% use and preventing the user from accomplishing their task. After all, Microsoft is more important than the user and their needs come first.
> I had to maim the Windows Update process to keep it from freezing my computer whenever I tried to use it.
Windows update for Windows XP got worse than that. It would churn your drive for hours, but not make progress. There was an update to install manually and then you were good.
But if you're running Windows 10 off a hard drive, you need to replace it with an SSD. Although, I guess you can probably blame that on Nadella, the mass layoff that included essentially ending the developer-in-test QA program was June 2014, and Nadella became CEO in Feb 2014. IMHO, quality dropped significantly since then, especially quality on hardware developers probably don't have or don't use (like hard drives and Windows Phone)
You can be passionate about product and customer service, and be an accountant.
I think it is rare to be passionate about product, when you are an MBA. Because your passion in seeking an MBA is yourself ( ladder climbing, status seeking ), and that path is not forged on sacrifices for customer service.
> Once you put an accountant or MBA in charge of any company the company is dead. You can’t cost manage your way to growth.
What about all the techie founders who have run companies into the ground? Most startups I know are run by techies. Only a small number actually succeed.
> The longer term Google/Meta stories are the rare exception.
I think that is more that making money as an advertising monopoly is easy, and can not only support constant Google/Meta failures in many lines of business, but even the gradual deterioration of their core offerings.
> The longer term Google/Meta stories are the rare exception. And that's not always a bad thing by any means.
Seemingly there are two types of companies: "Killers" and "To Be Killed". In order for you to grow like Google/Facebook, you need to "Kill" (acquire) other companies, who will be the ones who die, eventually.
Rackspace went to way of being acquired, so eventually to be killed. Google/Facebook went the way of acquiring other companies.
The only way to win is to not be either. Just a company that provides enough profits to cover the expenses and pay out bonuses whenever, but not acquire others nor be acquired by others.
Why does this happen? It seems to be self inflicted fatal wounding that businesses would try to avoid.
Is it just principal/agent problem by those who are in a position to extract funds from the company? Do people think "it will be different this time!"?
"Death by a thousand cuts" doesn't show up in a spreadsheet because each cut is hidden in a different cell.
We've all worked in the industry long enough. We know that if you approach the PM and say, "man, I watched my girlfriend try to use the site on her phone this weekend. She gave up because every page tried forcing her to use the app," the PM would respond, "we did some testing and not many people stop using the site when this happens, and people who use the app are 3x more engaged, so this isn't true in practice." But those tests don't show what happens 3 months from now when your girlfriend uses a competitor that doesn't have any of that shit, and she thinks "wow this is so much better" and just never goes back.
I feel as if there needs to be a modification to the phrase "Data Driven". Data, generally, will not help you predict the future. It is however, fantastic at finding bias, inefficient existing processes and tracking the results of your changes.
When applied blindly you end up with scenario you described. Product direction that says something isn't necessary because the question was never asked to begin with.
"Data driven companies" aren't, because none of the people involved in making decisions ever took anything more than stats 101 and have never experienced a real world data situation where things are messy and if you aren't careful about how you measure and collect data you make it useless for decision making and also apparently because they don't teach goodharts law in business school?
At some point, they decide growth from technical innovation has mostly played out. (Did Rackspace have a path to building a sufficiently disruptive product to grow another 10x?)
At this point, the MBAs try innovating in finances / marketing, as that's likely to 10x profits over time. This is why all the big Detroit automakers were essentially banks that gave you a middling car for taking out a loan when the lending crisis almost wiped them out. That strategy actually worked great for their investors (until it didn't).
It's not something they can avoid. These companies were often dead before any of those changes were made; and none of those changes would have been made if the company was growing and healthy.
Let's say you're a late 90s company that rents servers to people (yes, Rackspace). There are lots of these companies, so to differentiate yourself, you hire some smart people to provide tech support and managed servers. You charge quite a bit of money (IIRC, they would charge like 400-500/mo for their cheapest server). Core technical ability is: managing data center costs/infrastructure and server administration (whatever is in RHEL certification). Companies love you, and give you lots of money.
You operate this way for the next decade. Then one day a company comes along and says, we'll let people rent servers hourly for a couple of dollars. No support. Some of your customers move a few workloads over, sales get a tiny bit harder... but your core customer base stays, because you sell fully managed servers and AWS doesn't. Your customers arent AWS customers, for the most part.
This is the foothold in the market that will eventually destroy your business... AWS will keep adding services, and eventually support contracts (which most people wont buy, meaning in general, AWS will be cheaper than rackspace). Each time AWS becomes more competitive, your position deteriorates slightly.
At some point, you recognize the threat, and decide to do something. You have two choices, which may be only one, depending on how much cash/resources you have available: 1) Start building features and services to duplicate your competitor. These will be expensive and becomes more difficult if the core competency of the two businesses isnt the same (like if you admin servers, and need to produce lots of software). This will burn through your capital and shorten your lifespan if it fails; and/or 2) You can control costs to ensure the business remains profitable. This will increase your life in the short term but ensure it fails in the long term.
The options that many lay people think of like decreasing prices, or increasing sales (somehow??) arent real options. A product has an entire structure behind it to deliver it, and just decreasing prices will not solve the competitive position, while increasing losses. So shortened life span, and no solution. Increasing sales isnt really an option, because if you could sell more you would... when customers have another option they prefer, selling to them becomes virtually impossible. If you run ads, for example, you'll see your metrics deteriorate, and this will limit your ad reach.
Now, rackspace did both of these things... they developed openstack... which was terrible (software from a non-software company, likely contributed to that)... and when that failed they went the cost control route.
My point is: No decision was made to put the company in this position... it's a position they found themselves in... and unless the leader can copy or innovate rapidly to save the company, or move the company into another segment... they'll find themselves in this poor position and likely eventually die. This bar is incredibly difficult to overcome: even if the founder was innovative, finding a new innovative product on short notice on demand while the source of funding for that declines.... most people will fail. This is why microsoft, facebook, google put an emphasis on innovating, even when they do not need to... easier to innovate when your position is strong, than in the final days of your core business.
it was so strange to see Rackspace left in the dust by AWS, despite Rackspace seemingly going full steam ahead with OpenStack.
yeah, OpenStack was a stinking pile of python shit cobbled together with RabbitMQ, mysql, an ungodly amount of hope and iptables rules...
but AWS was also literally just a lot of Perl scripts under the hood, with a UI and UX that made DIY silicon plus Linux from Scratch appealing.
yet AWS is now an undisputed king (with the same steak knives in my eyes UI/UX), and Rackspace is dying. but OpenStack lives on, and during this time k8s appeared from nothing and completely turned many things upside down.
it seems the hard truth is that "fanatical support" was great, but it was simply not pushed far enough, and it took someone like Jeff Bezos to commit to it. (see "consumer obsession"). and probably the lynchpin behind "why AWS grew so big" is that the whole profit of Amazon was pumped into it. AWS has more PoPs, more services, more knobs, ... than Rackspace and OpenStack combined ever could have. and it has all these to be able to corner the market and completely extinguish competition. (as it did with books initially.)
Most businesses started by passionate technical founders fail as well, there is a selection bias in the big companies that are in the news. A lot is just luck, having a product that the market needs that can be sold at a profit without being undercut by competitors.
There was a popular business book in the 80s called In Search of Excellence. Basically the authors distilled their findings down to some common principles like "Stick to your knitting."
However, as our professor observed, for pretty much every one of the principles you could find counterexamples of companies that succeeded doing the exact opposite thing. And also, over long enough time horizons, you could certainly find companies where a principle like "stick to your knitting" worked--until it didn't.
Industries evolve. In this case Cloud probably killed them. They failed to adapt when a new innovation came along.
This can happen for multiple reasons, but mainly because incumbents are at a disadvantage in the face of radical innovation since they not only have to learn the new tech, but unlearn the old.
Some companies also double down in the face of new tech and keep improving their obviously inferior product. E.g. when refrigeration was invented, companies that shipped ice from the arctic focussed on shipping it faster instead of accepting the inevitable and pivoting while they had resources to do so.
And screw the employees who have bills to pay, families to feed and may have been able to keep a business viable without this incessent need for quarterly growth.
Pay the rents and wages and everything else is gravy.
Sure the current management team is full of bean counters that aren't doing great at understanding product, but don't give the old management much credit.
They completely missed the transition to cloud, they missed AWS, and they laughed when I told them that we (DigitalOcean) wanted $100MM if they were serious about any acquisition talks. This was in 2013 so startup economics were different back then.
They were early to a massive market that is growing and they basically botched their lead repeatedly and every transition since then has been a comedy of errors.
They would have been better off following Equinix's model of just acquiring and managing datacenter space instead they ran their own DCs, didn't have great support, couldn't build product, and ultimately were acquired by PE and saddled with debt along with no road towards innovation.
I wanted better CloudFront pricing and AWS referred me to Onica, which later got acquired by RS. Frankly I don't really use it, I just get better bandwidth pricing and they charge me for AWS support (I've only ever messaged AWS directly, not Onica/RS). Honestly I don't know what they get out of the deal except maybe a kickback on the support I'm required to pay for, but I'd have paid that anyway if AWS asks.
If they die off, I guess I won't miss it, but I can't say it's been a problem.
They've pretty much guaranteed I won't return as a customer due to one extremely stupid thing they have done with accounts (at least as of the time I moved to another provider...I don't know if they have since changed this): deleting all your servers, storage, name server configuration and all other things you have set up does not reduce your monthly bill to $0. To reduce your monthly bill to $0 you must delete your account. An account using no services still gets hit with a $5/month fee for support [1].
That means that if I ever want something they offer and they have a slightly better deal than someplace I have an account with, their deal with have to be sufficiently better to be worth the hassle of creating a new account. That's fairly unlikely so I probably won't be back.
[1] That fee was why I left. I just had one instance of the smallest VM they offered, plus storage for a couple backups, plus name service for my domain. The VM was used for a very low traffic website and hosting my email. When they added that $5/month support fee it raised my costs about 30%. I switched to Amazon Lightsail for my VM and switched from hosting my own email on my VM to hosting it with Fastmail. That comes to $8.32/month.
It wasn't OpenStack, that was just the attempt to recover.
Rackspace just lost. They failed to grow into a market leader before their industry commoditized. When you're in the early rush, growth is all that matters. But now if you want to compete with AWS, you need to be cheaper to operate than they are, and that's extremely hard if you're not at their scale. GCP and Azure can do it, little companies can't.
The one caveat here is that little companies can compete in particular niches in which they can provide a more tailored solution.
For instance, Lambda Labs is "AWS for ML". They have GPU cloud instances with pre-configured ML packages and a network attached filesystem. That's all you need.
The other day, I wanted to run Stable Diffusion, but I don't have a GPU. I went to spin up a GPU instance in AWS, but got hit with a quota limit. By default, AWS accounts have a quota limit of 0 for GPU instances. Presumably to prevent fraudulent crypto mining. You have to file a support ticket that is manually looked at by humans to get this limit raised. In my case, it took 4 days for them to raise it.
I had an A100 instance running in minutes on Lambda Labs. It's significantly cheaper than AWS, too.
I'll be using Lambda Labs for all my future GPU needs, significantly better experience.
Niches like this are often ephemeral and poor uses of capital in the long run. Early on they prove a need exists and can be profitably satiated with the right solution. As long as the market stays minuscule the company introducing the solution has a moat to be a lifestyle business. Once the market is sizable enough though a big guy just offers the same service and consumes the market share.
I get your point about small companies occupying a niche as their distinct competitive advantage, but in this case, one has to ask the obvious question: why doesn't Lambda Labs have a problem with fraudulent crypto mining, if the bar for entry is so low and it's so much easier to be up and running in minutes?
And whatever the answer is, one hopes that there is some particular reason why Lambda Labs doesn't think AWS could replicate it... otherwise, it wouldn't be a durable enough moat to defend their business model long-term.
That's quite easy to explain. They don't have a problem because the people that contract them do it explicitly to run GPUs. They don't have to protect a massive number of people that don't exactly know what they are hiring.
That makes sense under a certain set of assumptions I suppose, namely that you might think AWS's problem arises mostly from normal customers not realizing that someone has hijacked their account to run GPU crypto mining without their knowledge, because GPU instances were not their intended use case. So "people that don't know what they are hiring" is the failure mode you're concerned with.
But I wonder more about, say a fraudster using stolen credit cards or gift card abuse, to open an account, or abuse a free trial, and if Lambda Labs makes it super easy and low friction to start up a GPU, then the fraudster can open an account, mine crypto immediately, rack up as much crypto as they can for days or a month until billing fails or trial expires, and just try to squeeze Lambda for as much computing power as they can subject to Lambda’s ability to detect fraud.
Low friction for customers, usually translates directly to higher risk of abuse. AWS adds its limits and friction for a reason, not because they enjoy making life harder (esp. since, easier for customers == more business/revenue).
There's Vercel, there's Netlify, there are more. They all have their niche offerings, which sometimes beat what AWS does. However, they're not a _generic_ cloud hosting provider as Rackspace is. Seems like specialisation is the only way out for them.
I think the ability to compete with AWS and Azure has crossed a threshold. If I tell a customer we host their data on a platform that is not AWS or Azure, we will have a lot of explaining to do before the answer is accepted. If I tell them it's on AWS they will simply ask a set of particular questions about patterns and practices.
Not really. I mean it in both interpretations. Amazon basically invested years and years of profits into AWS, Rackspace did nothing like that. (Nor did they really attempted to do it. And if OpenStack somehow "won" just like k8s won, Amazon would just sell that too.)
Sure, but public cloud exploded a thousand-fold, while private clouds are just a niche. People standardized on Terraform, there are no big OpenStack hosting providers, there's not even a big group of small providers, etc.
I have some clients who host at Rackspace. They have old-school monolithic web-app architectures (single web server that also runs the application code, and a database server). They're never going to rebuild these legacy application -- the codebases are decades old, and comprise hundreds of thousands of line of code. The support at Rackspace has really cratered, and it's hard to have much faith in their technical capabilities anymore either.
I'm no sysadmin, so I don't know if it's realistic to port this kind of architecture to AWS or Azure. Also, some are contractually obligated to host at a facility that offers "managed hosting" (although I suppose the exact wording could be weaseled) with uptime guarantees and staffing levels.
I'm trying to figure out where I should recommend they move. It doesn't seem like a common use-case anymore.
Assuming the application code is mostly stateless (it pulls all the data it needs from the database on every request), this kind of app is usually not a big deal to containerize and put on some cloud platform (AWS, GCP, Azure should all be about equivalent) and even get more 9s and scalability almost for free. This is essentially the Wordpress architecture, which every infra engineer (née sysadmin) in the industry has been asked to move from on-prem to cloud sometime in the past ten years.
If the application code has non-disposable state in-memory then it's bit harder but can usually be done, especially if it's low-traffic enough to comfortably live on single Rackspace dedi now.
Edit: I should also mention that if you're looking for somebody to do this kind of work for you, there are contact details in my profile. ;)
I'll have to look into the web-server customizations. One of them has a custom Apache module for talking to a 3rd party mainframe, but that should theoretically be movable too (if it's the correct platform architecture).
Of course, an Apache module's just C code, so how hard could it be to port to another architecture? But in all seriousness, I suspect it's a relatively simple UDP socket communication interface, so it is probably portable.
Thanks for the mention of Wordpress. It is a similar architecture. I'll read up on how to move a Wordpress install, and use that as a model for testing one of these apps.
Linode, DigitalOcean, Vultr, EC2/Lightsail and many many other companies lease dedicated servers and VPS. Lease an app server, lease a DB server, move the data and port the code.
Yes, no doubt there's some other 'devil in the details', but if it's already "old school legacy" architecture, why not just replicate that some place else that offers better service (and likely better value for money)?
All good recommendations. I think the difference is in the contracts (I'm reporting second-hand, since I don't have access to the actuals). With Linode, anyway, I do a lot of hosting in VPSes, but these clients have contracts stipulating the hosting company manages/maintains/monitors the OS.
I guess if I were smart, I'd get the actual contractual requirements from the clients. That'll be my next step.
Cloudways is a provider that does 'managed' on top of various cloud providers, and "Regular Security Patching" is one of their checklist items. There are no doubt others.
> This is not a company that’s on a trajectory of growth. They’re on a trajectory of death.
Maybe Rackspace is on a trajectory of death, but this is a false dichotomy. It isn't (or at least shouldn't) be necessary to grow for a company to survive. If you are profitable, staying the same size is perfectly fine.
The phrase usually used is a "leaky bucket". Sure a company can operate in a neutral space, but it's unlikely that overtime their revenue growth will match their customer churn.
At the same time, a lack of growth limits how much can be invested on staying relevant in terms of R&D which further separates them from competitors with a positive growth trajectory.
Its also incredibly hard to kickstart a growth engine once it has slowed down. The net result is a flywheel but in the opposite direction.
PE buys companies like this to slow that negative growth flywheel down attempting to stay neutral until the investment is paid off at which point profit can be made.
The long tail is massive attrition as well, because few people want to work at a sinking ship or rather a ship unwilling to invest internally.
If you're in a growing industry but you're not growing, then you're becoming less and less relevant to your potential and actual customers. In a business like their scale is crucial. It's not a stable situation to be in.
Having said that they have been growing, their revenue is up, but so are their losses. They've just not managed to stay competitive.
Are there any examples of companies with flat profits that continue to exist years later, where the non-growth isn't caused by mistake or incompetence?
The downfall of Rackspace started even before the Apollo acquisition. It was obvious back then that the company no longer had anything unique to offer the market.
I've migrated multiple companies away from RS, usually to AWS. Very easy workloads and RS was charging them way too much without any useful support. This isn't surprising.
I was a Racker from 2010 to 2014, I left shortly after Lanham did, I was also located in San Antonio near Castle until 2022. It's a very sad thing that Rackspace is dying, but I must concur with the assessment. Apollo, like most private equity firms, is a bunch of bean-counters who are focused on financializing acquisitions, using their assets as leverage to extract money, and then offloading it to bag holders in the market who ride it down into the abyss. When I left Rackspace they had no debt, a massive war chest, and a reasonably competitive product offering to AWS that could have been capitalized on. As soon as Lanham left and they put an MBA sales guy in as CEO (Taylor Rhodes), and I heard rumors they'd turned down an acquisition from a major tech company but were considering a sell-out, I left to go do startups, and so did many other top-tier Rackers.
It used to be commonplace in South Texas (San Antonio / Austin) tech scene to meet new Rackers and talk to them about the old days, and just generally see Rackers having a huge presence in startups and tech. You still see a lot of former Rackers around, but they're all so far removed from those days now that this is not how we identify when we introduce ourselves. Not much new blood. Apollo decided it would be more financially sound to get rid of the core values, end "fanatical support", delete the fanatiguy logo and branding, and then lay off every single non-management American worker to replace them with 2-3 people working overseas from India. Rackspace support has been a boiler room bottom-barrel call center since at least 2019, and the pandemic massively accelerated what was already on the horizon.
The company has been culturally dead and financially doomed far before the ransomware attack. The ransomware attack is just the final nail in the coffin. Apollo took a ~$5B company with no debt, loaded it up to do multiple acquisitions theoretically worth $2B ($7B if you're following) and made it worth less than $1B on market. Real geniuses there, who managed to completely destroy a solid American company that helped create a burgeoning tech scene outside of the Bay Area, so they could pocket the money personally as they drove the company into bankruptcy.
It's unfortunately a tale as old as time, or at least as old as the 80s. Private equity is a bunch of sharks doing leveraged buyouts to con retail investors into holding the bag. Hasn't changed since I've been alive, and isn't likely to change before I die. The moment Rackspace sold to Apollo, they were on the trajectory of death. It just took awhile for reality to set in and consequences to catch up.
I don't always agree with Yoo and I don't agree with everything he said in this article, but he's absolutely right about bean counters ruining tech companies.
More anecdata: I'm working with a customer right now who will be moving all their apps off Rackspace to one of the big cloud providers later this year (work is underway currently).
Recently I went to sign up for Rackspace and their systems flagged me as 'Suspicious'. I tried registering without a VPN in a brand new session, and again: 'Suspicious'. I provided all the right details for them, correct contact and address, etc I wonder what their churn rate for auto-flagging away potential customers is? They are in the business of taking people's money in exchange for services right?
I've had what you describe happen recently with some wireless vendors. They flat out won't even acknowledge my own domains a valid email domains despite being so and following every possible best practice. No VPN and using a static IP no less. I ended up using a reseller that apparently has not gotten big enough to use the over-zealous tools just yet.
One of my theories is that the older internet businesses probably lost most of their original developers, downsized their staff too much and had to shim something simplistic into their customer on-boarding flows and then found a turn-key solution that had a maximum protection setting that does not give insightful or accurate statistics. They are probably not even aware of all the lost financial opportunities.
At least their object storage service has super old ssl, and breaks under recent ubuntu lts versions, or really any ssl config that's reasonably secure.
Rackspace was taken private, saddled with a bunch of debt while the stuffing cash into Apollo's pocket, and then re-IPO'd so Apollo could fleece the market.
Major IPOs go into 401ks, Wall street wins, the average American is left holding the bag.
Rackspace currently has $5B in liabilities! Any fund manager that bough Rackspace after it re-IPO's should be in prison for fraud. The banks that underwrote the IPO should be in prison for fraud.
Rackspace is a zombie company at this point. They lost a SAN during some maintenance, no backups, remote volumes lost. And first tier support can't debug Linux VM liveness issues. They have to talk file a ticket and get a Linux SME on the case.
The article is paywalled but I would imagine DigitalOcean is screwed for many of the same reasons.
As things do in our industry, I fully expect cloud computing to die out in lieu of an “on-prem renaissance” where companies realize they can floor on-prem servers 24/7 for a one-time* cost instead of AWS nickel-and-diming for every little thing. It’s clear AWS (and Azure) are anticipating this because all their new stuff is very AWS-specific or very Azure-specific. God it would be basically impossible for my company to move away from AWS to another cloud, nevermind back to on-prem. Even AWS’s ElasticSearch aaS (OpenSearch) is diverging from ElasticSearch proper. AWS could double prices overnight and my company would pay it.
well, back to basics
people are blaming the MBA for ruining this company,
but I have to disagree, I think there is good MBAs and bad MBAs
since the 90s, MBA are thought about change and agility
rackspace failure is basic, they failed to adapt to change, and were not agile
this is a basic , basic scenario
they had a bad MBA, a good MBA would have seen this long before it happened and would have made necessary changes , companies change and adapt all the time led by good MBAs
The server(s) we had cost a lot, but knowing you could connect with someone competent 24/7 was amazing.
Then... a few months later... we had to use it, and it was good, but they were about 2 minutes away from wiping our main drive. We'd requested a new beefier server and requested either the drive be moved or everything copied over to the new server.
Got a message new server was up - great. Logged in - nothing. Connect to old server? Nope - offline/disconnected. Ugh. Calling in got me a live person, and I stressed the urgency. We did get connected with someone, and they did find the right person, who was about to wipe the previous server. Crisis averted, but... immediately I start asking why the support ticket showed all requests as 'done' when... they weren't.
We stayed a while longer, but it felt like they were growing too quickly even then to keep the 'fanatical support' model that initially attracted us. FWIW, I had a couple of dedicated servers at a small place in NH, and they were... $70-$80/month on average. The rackspace servers typically started at $500/month. I think we were paying about $1800/month for the setup in question at the time of the crisis. My belief in 'you get what you pay for' was impacted that day.
I had another client years later (2010? 2012?) that used them and it felt like a completely different beast, far more generic and commodity like every other competitor, and I've never since looked at them again.