Hacker News new | past | comments | ask | show | jobs | submit login
Uber lays off 435 people (techcrunch.com)
976 points by coloneltcb 10 months ago | hide | past | favorite | 579 comments

Hello Uber Engineer here!

Obviously everything I say is personal experience and opinion. I think the layoffs were long overdue and should've happened sooner. There was a general understanding between my coworkers and I that Uber definitely over hired in 2016. Thanks to that a lot of engineers ran out of things to do which led to political infighting over roadmap, ridiculous redundant resourcing - every single team had mobile/web, severe shortage on infra teams since head count was already taken by main Uber teams. We even started building and maintaining our own chat app. I disagree that all the cool eng project that we put out and share here is a waste of time or resources. Those project was initiated by real needs on Uber and only the best make it out into the rest of the world. Engineers that shipped those projects are still here and we would've done it regardless of whether the 435 people that were hired in the first spot. I fully realize that it's an insensitive thing to say but I think it's important that it's expressed somewhere.

I think the lesson here is to grow responsibily, and it's real people's lives that are being affected. Don't hire people to alleviate your current engineer's burnout and stress. Learn to plan better and to prioritize aggressively instead of just hiring and getting everything done.

> We even started building and maintaining our own chat app

God I love the Not Invented Here disease. Somebody, somewhere, thought "hey, slack is way to expensive, we don't want to get 'locked in', and 'mumble mumble privacy cloud evil stallman' so lets write our own chat application.... how hard can it be?"

Boom, now the company is straddled by a shitty, homebrew chat application that is maintained by nobody because the original author left for greener pastures. Nobody dares replace it though because that would be sacrilege. That chat app is part of the company, after all!

Arg.... I hate, hate, hate when engineers with a bad understanding of business and too much time on their hands lock their organisations into homebrew crap that has nothing to do with the value delivered by the business.


> Somebody, somewhere, thought "hey, slack is way to expensive, we don't want to get 'locked in', and 'mumble mumble privacy cloud evil stallman' [...]

Like I completely agree with your overall point about reckless overengineering, but you also just listed three true and really good reasons to avoid integrating Slack.

I setup an enterprise-grade Mattermost server in a weekend. The hardest part wasn't even Mattermost itself, but getting SMTP working reliably in the age of DKIM/SPF/DMARC policies. This could be sidestepped by using any of the well-established email providers.

No reason to use Slack when such options exist, but also no reason to write your own chat app, either.

Uber have talked about their internal chat app, uChat.

When OP says they built their own internal chat app, that's not entirely accurate - uChat is built upon Mattermost.


That kinda sucks the air out of this whole thread. "that's not entirely accurate" is a bit of an understatement. But, don't let facts get in the way of a good rant!

Mattermost has been rock solid for the last couple of years for my team BTW.

> The hardest part wasn't even Mattermost itself, but getting SMTP working reliably in the age of DKIM/SPF/DMARC policies. This could be sidestepped by using any of the well-established email providers.

This is valid not just for Mattermost but many other apps that support sending e-mails. And that's why it's important that instead of giving up and outsourcing it, we all try to set it up and have first-hand experience in challenges we have to face in the process. Because if everybody just uses Mailgun & SES, in a decade or two it won't be be possible to set up your e-mail server anymore (of course you will be able to, but your e-mails won't reach most customers).

> of course you will be able to, but your e-mails won't reach most customers

Unfortunately, I found this to already be somewhat of a reality. Even with all the best practices I went to, many MTAs simply don't trust my IP address. Most "sane" MTAs seem to greylist me rather than blacklist outright, so it just adds a bit of delay to users receiving their emails. Thankfully it seems to be gradually getting better, but gone are the days of simply pointing your apps at sendmail letting them go.

What's interesting is setting up a handler for DMARC aggregated feedback reports. I determined that it's not worth doing yourself, even Microsoft do not handle their own DMARC reports.

I ended up outsourcing DMARC report processing, if I wanted to do it in-house I'd have to setup and maintain an Elastic stack[1] which seems like huge overkill.

[1] https://domainaware.github.io/parsedmarc/

Let's not pretend that hosting an OSS chat server is a walk in the park for every enterprise just because you personally got one up and running in a weekend.

Let's not pretend that building one from scratch isn't at least an order of magnitude harder, either.

I'm not necessarily in the 'build one from scratch' camp either, I'm just pointing out that there's a trilemma in enterprise systems. How fast you're able to get something up and running is just one factor and not always the most important one.

Looking at this from Spain, you did do this over the weekend. I also work 6 days a week—but my hope is that this didn’t take the place of building a family, having a relationship, finding a relationship going for a hike in the mountains, citizen science, or making art. Your weekend is valuable!

You're totally right that the weekend is valuable, but some people enjoy doing this sort of thing with their free time. It's not anyone's place to decide what anyone else does for entertainment or self improvement. That is to say - what is not valuable to you might be valuable to someone else.


No, that pretty much falls under "None of your business".

Why? If you're not a friend or family member, it's none of your business.

My comment isn't concerned with using 'personal' time for such endeavors. My concern is with the implicit conclusion that how fast you can get something up and running (a 'weekend' in this case) is an argument that you can and should deploy a new stack in any given enterprise.

You'll find a lot of times the champion of the new hotness moves on and leaves someone else to maintain the old hotness at costs that were never factored into the original deployment.

As I noted in another comment, it's a trilemma where you have to pick and choose your battles. The initial deployment time is rarely the most important factor when introducing a new stack within an organization.

When large MMO guilds have had a mostly trouble free chat client that supports a few thousand concurrent users on the free time and good will of admins I'm loathe to believe that this shit is impossible

I'm not sure how you read from my comment that I'm saying 'this shit is impossible'. I'm simply saying in many organizations 'how fast the intern got it up and running' is not always a valid indicator of how ready it is to be deployed to a few thousand concurrent users.

For example, large MMO guilds have a quite different regulatory environment from, say, a large healthcare organization in the US governed by HIPAA. It's an extreme example, sure, but a homegrown, dumbed down solution, might be the only way to ensure regulatory compliance.

It doesn't take a lot of imagination to come up with other scenarios. I can't defend Uber doing it because I'm not involved, but large MMO guilds are not archetypal of enterprise systems.

The funny thing about your comment is that Slack was born out of a MMO game.

If the company relies on Mattermost, I would assume they'd need at least one maintainer per ~100 users, just to make sure that the SMTP keeps working reliably, there are backups etc. One person to maintain Mattermost would cost a company at least 150000 USD in the bay area. That's $100+ per month per user. Slack is way cheaper than that.

Really, 50 maintainers for a 5000 user system? I'd think it scales much better, probably 2-3 people for the whole company. And in that case, it can quickly be cheaper than slack

Exactly. No way does it take 50 people to scales mattermost to 5k because of smtp. This op made that all up.

I think you are really underestimating the bureaucratic overheads in a large company. It is not the same as a 3-person startup serving 5000 customers. Of course I pulled that number out of the air, but the point is that it can make sense. I think I am right around the ballpark of needing a 5 people team to serve 500 users - you'd need a minimum of 3 people just to make sure there is enough redundancy if one person is sick or goes on vacation. You'd need 24x7 on-calls. The scaling factor probably changes after that - may be it is 10 people for 5000 users instead of 50.

Adding to other large company problems, these people will need management layers, the team will need a charter for growth. Who wants to be the maintainer of the Mattermost servers in a large company? Add all of this up and then the slack deal starts to look reasonable.

Edit: Just to add some real numbers, slack costs $12.50 per user per month [1] - that is ~$750k per year for 5000 users = 5 engineers. (More like 3 really in the Bay area)

[1] https://slack.com/pricing

Your general point is solid, but one quibble with the numbers - even the pricing page has an "enterprise" tier "for very large businesses".

There's no way a company with 5000 Slack users is paying the sticker price per head - they'll call Slack's enterprise sales team and negotiate some volume discount with whatever shiny enterprise-grade features, SLAs, support guarantees, etc. they need.

Or you have your existing corporate IT team manage it the same way they're managing other internal apps.

There's no way it suddenly needs a new team of dedicated full time resources.

This is a more realistic scenario. I own/operate shared services for thousands of users on a team of 3.

If it takes you that many engineers you are either using the wrong software or using it incorrectly.

At Uber scale, you probably already have your own SMTP servers used by all your employees, so that's not really a problem.

Moreover, email is hard and does require a dedicated team, but the order of magnitude is wrong here. 5 people is plenty.

Lol, where did you get the idea that smtp servers need more maintenance as users increase? What if I told you I’ve witnessed a team of 3 manage a mail system of a university of over 100k active email accounts?

We are a company with around 100-150 employees, using Mattermost in a real Enterprise setting for years now (with SSO, CI and mail integration and whatnot) and that server is being maintained by our admin team basically "on the side" - nobody was hired specifically for this job, as it does take less than 20% of one person's time according to what the team says.

These Bay Area tech people must be either incredibly inefficient (especially when comparing to what they get paid), or your estimate must be waaaaay off.

You're making a good underlying point but contriving way too many details in your argument for it to be taken seriously.

Sure but like, run Mattermost or Lync or an XMPP server with apps pre-built for it... don't build your own system. Just a waste of effort.

Ex-uber engineer here. Uber's chat app is based on Mattermost, not built from scratch.

From my experience that kinda makes it worse because constantly people will say "X has this now this add that" and it's normally a case of "Well that's a ton of work to do because we did Y".

Does the fork stay current with upstream?

They were a bunch of bored engineers... they weren’t thinking straight!

Ok, so now those s/SWEs/operations/ folks had nothing better to do than reinvent the wheel.

It takes 1 person a few percent of their time to manage something like that for internal use only. And you still get the gains of keeping secrets internal and cost savings on licensing costs which might actually pay for a good chunk of the effort in the case of running an existing service.

Compare with a small development team for a significant percentage of their time to build a halfway decent chat tool.

Miles of difference.

Slack is not too expensive for a company like Uber. So maybe two good reasons...

Heh, this is the story of how Slack got built. Slack was built as an internal tool while building Glitch, an online game. From Wikipedia:

> Slack began as an internal tool for Stewart Butterfield's company Tiny Speck during the development of Glitch, a defunct online game.[13][14] "Slack" is an acronym for "Searchable Log of All Conversation and Knowledge.

Some day, Stewart Butterfield will launch a gaming company that is successful in the gaming space instead of becoming the inspiration for an entirely unrelated startup.

Butterfield needs to fail at an enterprise app so his side game blows up.

Fun fact: Tiny Speck was actually Stewart Butterfield's second game company.

His first game company also didn't do well, so they pivoted to release Flickr.

I simultaneously wishes for him to continue trying to make video games, and to continue failing to do so. Seems to be working pretty well.

"Enterprise" slack is $15 per head per month. This would equate to an recurring annual expenditure of nearly $4.86million for Uber's 27000 employees.

Given that Slack can be seen as IRC with some pretty decoration glued up to an Elasticsearch instance, and that Uber already has a highly trained tech org in place, if it wants to make a Slack "clone" for less than, say, $25million (cost of Slack to Uber for 5 years), it could actually be a pretty sensible decision- especially when you take into account the tax and stock-price benefits of capital expenditure.

Unless NIH/buy are truly the only options. `docker run whereverthatactuallyis/mattermost` looks like a cheaper option that doesn't require building from scratch.

Funnily enough, mattermost seems to list Uber as their client on their home page. https://mattermost.com

Maybe they realised it was a superior solution to building their own app.

> mattermost seems to list Uber as their client

Not directly related to mattermost and Uber, but companies tend to list clients even if the product is not widely used or was even dumped.

You are a small team in a big company, you evaluate a product and find that it satisfies some of your basic needs but want to work with it for a while, so you purchase a couple of licenses for you team but stop using it after a year or two.

voilà ! you are a client listed on their web site

> Maybe they realised it was a superior solution to building their own app.

Uber's internal chat tool is built on top of Mattermost.

That's unfortunate - they could have made Mattermost on par with Slack instead.

+1 for mattermost. Works out of box for most applications. But I think people will find it surprising how quickly one starts to run out of space. People can be very chatty.

You think a client of that size would’ve been billed using pricing designed for mere mortals?

As other replies have said, with a high user count, they've likely negotiated that per head figure WAY down.

But even if they didn't, it's not as simple either as "can those engineers clone this software for less than X", it's whether they can afford to take the opportunity cost of not having engineers work on something core to Uber's business.

I take it you mean Uber? I find it hard to believe Slack has 27000 employees.

Anyway, as another commenter pointed out, on an enterprise scale I'm sure you can get a good deal. I can also imagine however that at that scale you'll want multiple Slack servers / instances - which would add cost given how slack charges per person, per server.

So there's a few alternatives then; self-hosted (some parties will offer enterprise customers the option to host an instance of their application on the customers' internal hardware), or rebuild it yourself. The latter one however takes a bit of math - how much is it going to cost to build, maintain and run it? It's easily underestimated.

And yet, it's also easy for people working in an engineering culture with nearly limitless engineering capacity to just build it themselves - motivations being "how hard can it be", and "I need something more challenging than tweaking this button for my employer to earn 0.1% more money on this particular feature". I've seen it happen far too often.

Zulip is already a thing. Not to mention other floss alternatives.

That's assuming that, with 27,000 employees (why does Uber have so many?), monthly licensing fees are not negotiable. Which I would doubt.

they're about to get a LOT more ...

Slack was evaluated a few times. The reason it was initially rejected when we first considered it is because it wasn't able to handle our scale at the time and the team over at Slack was not interested in prioritizing scaling for us over other things on their product roadmap.

An organization involved in as much sketchy drama as Uber has probably doesn't want its internal communications on someone else's server.

On top of that they 100% can read your private messages

Interesting. IBM adopted Slack just fine, back in 2016. I believe they've got at least 100k employees in their Slack workspace. Not sure if it's Slack or IBM doing the hosting/scaling for it, though.

How many Uber drivers are there? Also, don’t forget that slack has an API so write loads can be very different regardless of user count.

> How many Uber drivers are there?

None, they're all independent contractors.

That’s still an Uber driver. They’re literally being contracted to be so.

Was this chat app for external contractors as well? Highly doubt it.

There is a cultural difference between Uber and IBM which would explain why Slack didn’t shown its limits at IBM: Slack people belong to a startup which typically wouldn’t mind get everything done in a chat; whereas IBM is a 100+ years old company where everyone has to fill a paper form and managers type with two fingers (Source of this specific example: I’ve worked with IBM consultants with one of their products). Chat apps do not play the same role in each.

There's no way this is true. There are companies far, far larger than Uber that run slack with no problem.

Even the public kubernetes workspace has almost 75k people in a single channel.

This has only recently been possible. For a long time Slack was capped at 5k per workspace, then later a similar number per channel.

What about all the other chat apps? Why wasn't the whole thing dumped into a pile of hot lava the second slack could scale to an org the size of uber?

What what does scale mean anyway? Just employees using it or employees + contractors?

It's easy to armchair quarterback on this stuff, but I find it hilarious how tech companies that employee supposedly smart people justify writing this kind of stuff. I'm sure some junior engineer did a cost/benefit that completely neglected the total cost of ownership for writing a chat app and focused on the sticker shock of having to pay tens of thousands a month for a chat app.

another issue was data domiciling and retention policies around 2016 when uber was evaluating slack.

Were these actual concerns that were raised by the actual legal council of the company, or were they issues invented by engineers who are not lawyers but suffer from engineers disease and think that knowing to code means they are experts in everything else? 'Cause I know plenty of those!

I've worked in plenty of companies with engineers who think they are legal, privacy and security experts. If we made business decisions based on these folks, we'd never do anything. Hell, every damn text change we makes invokes a few who worry about the legal ramifications ("should we file a JIRA ticket with legal?").

Sadly without somebody stepping in, people actually listen to them instead of the actual experts employed by the company... thus you wind up wasting tons of engineering bandwidth building a feature / platform / product that has no business existing all (or worse, not building some feature / platform / product) because nobody bothered to check with the legal / privacy / security team that some concern by a engineer was actually a genuine concern.

And even then, in any good organization legal / privacy / security teams exist only to point out all the risks in any business decision... sometimes it is worth the risk despite what legal / privacy / security say. What appetite the company has for the risk depends on a lot of things that are well outside of legal/privacy/security.

> Were these actual concerns that were raised by the actual legal council of the company, or were they issues invented by engineers who are not lawyers but suffer from engineers disease and think that knowing to code means they are experts in everything else? 'Cause I know plenty of those!

the general counsel and thus the entire legal org required for any company approved comms system a way to automatically by default destroy chat logs after 7 days (and to turn that off for folks who were on legal holds). slack in 2016 did not have that capability.

The fact that that number was "7 days" tells you everything you need to know about Uber's internal culture.

Not sure why the parent is down voted. I have worked in financial services and had the same experience. IT typically held a view on the interpretation of rules and regulations that was far more literal and restrictive than the actual compliance officers.

it was the privacy/legal/security types driving this strategic decision, I can't really talk about it in more depth. A lot has changed since the decision was made, and if I personally was to reassess the situation now I believe I would go with a 3rd party SAAS solution, but back then building a chat app was on balance the correct decision to make.

> it was the privacy/legal/security types driving this strategic decision

Booo on legal.... like I said earlier, it's easy to armchair quarterback. Sometimes I wonder how the fuck humanity even manages to make progress with so much waste and boneheaded decision making.

Lyft is no better, spending $8M per month on AWS to process 1M transactions per day.

AWS is the opposite of homebrew. Lyft has no business maintaining its own data centers and operational overhead. There is a huge opportunity cost involved with running your own infrastructure. AWS and the like free your own teams from worrying about how to provision new systems, upgrading DB server versions, etc. Running your own data center is hard.

Running your own DC is the same as building your own chat app. It is almost always more expensive and almost all the people who claim they can do it cheaper inhouse aren't factoring in all the costs--both costs that are easy to measure and those that are almost impossible to measure

Examples of hard-to-quantify costs for running your own infrastructure include:

- What is the cost to the company of being three versions behind on mongo because IT doesn't have the bandwidth to upgrade? How many work-arounds do teams have to do because they couldn't use the latest features of mongo?

- What is the cost to the company when developers cannot quickly spin up new environments like they could using a service like heroku?

- How much innovation and new business (read $$$$) is not happening because the in-house IT simply doesn't have the toolset so a dev can quickly riff on an idea?

- How many engineers have good ideas and don't bother building them because spinning up an environment is too much trouble?

I'd argue by not using AWS you are holding your product teams back and stifle innovation. Unless of course you want to waste company money to build out your own half-assed provisioning tools that are lousy knock-offs of AWS's tools. But then, you might as well just use AWS...

I should have been more clear. AWS is absolutely the right choice.

What I was trying to point out is someone engineered a system that costs 8M/mo to run yet only handles 1M rides per day.

So they're paying 26 cents per ride on all compute resources across their entire company?

That doesn't really sound that bad, considering that rides can easily cost dozens of dollars. Just about any other non-tech business would kill for such low overhead.

> 26 cents per ride That's terrible.

Other non-tech businesses, hell EVERY business that I've ever worked in that used AWS did better per transaction - healthcare, digital advertising, virtual office tooling, gaming. If you can't do better than a credit card processing fee per paid customer interaction across the company on AWS, you might as well be selling retail goods...and depending on age, your company might be soon.

Even if it is terrible, is the business such than a 26 cent overhead on rides is going to make or break it?

Companies like Lyft are doing a lot more than updating a few fields for each ride. They would be processing massive amounts of location data and they are building out models to eventually use for self driving cars.

Thank you, my point exactly. Their infrastructure costs are terrible. Especially considering how easy their data is to shard etc (buyer and seller belong are in the same geo physically).

I don't see the analogy here. This is Lyft's core business. Of course they'll go all in. On the other hand chat is not Uber's core business. They didn't have to reinvent the wheel for an internal tool.

Is 8M/mo they outrageous a number here? Assuming Lyft paying 200k/year on average to its employees, 8M is about hiring 500 of them.

It is sizable, not ridiculous. And optimizing your cloud cost is definitely your own responsibility and deserves every bit of attention.

Running your own data center is at one extreme. Most companies lease space in a datacenter, and all of the overhead of running the data center is on a different company. You'd have an infrastructure team racking and stacking, and supporting the deployments.

What? You are crazy. At a certain scale, it is cheaper to run your own DC. But that scale is reached by a very limited few.

But past a certain point AWS is very expensive compared to renting a rack in a couple of DC's

Uber didn't make their own chat app. They just use Mattermost. It was a comical sham internally perpetuated by the team that rolled it out that it was "built in-house".

I heard about that. That the team responsible tried to pass it off as their own. However, according to the CEO of mattermost, Uber has made extensive alterations.

Having used Mattermost a lot for the last few years, you'd really _want_ to make extensive modifications to at least the mobile clients if you want things to work properly.

Contrarian view: not your company or resources. Management was enabling it until now.

Since when did wanting to be a software developer mean there’s “one true way” to do software dev?

Why are developers supposed to be budget babysitters for a huge company with tons of people watching budgets?

I don’t owe my employer being an expert in all contexts to the benefit of their business. I am not their servant. I write code to serve contexts they want served.

While you're not wrong about most of this. uChat is actually built by a pretty large team


> uChat is actually built by a pretty large team

Which is even more scary. Why isn't uber laying all these people off and paying cash money for a real chat application? What competitive advantage does uber have maintaining a chat application?

They might have to pivot into a chat company

Only profitable if they can ever find a way to make the chat self-driving.

Group chats all ready are:

You can take your hands off completely, even go and do something else, read a book, write an email, book some flights, and the chat will drive itself ... somewhere.

With more and more chat and email apps offering auto-generated/suggested replies (e.g. what Gmail offers) we're on our way there.

I might actually hire an Uber for the express purpose of having someone to talk to. In Germany, taxi drivers were often former "German studies" or Philosophy students who did not find a job, so it was kind of guaranteed that you'd have a competent, honest communication partner (albeit left leaning).

I think that's brilliant. I'm curious as to why it is not regarded as proper job. I don't think it's unique to Germany though. I think it is as honorable an occupation as any. And for the philosophical types it might actually be quite a good occupation as it may not be as taxing on the mind (apart from the time they spent thinking on their favorite subjects). That frees them up to study, research and write in their spare time. Secondly I'm curious as to why they tend to be left leaning...

"Uber Chat -- because that's one thing you didn't know you needed!"

Why doesn't Uber spend less on Engg and pay more to their drivers- which may actually help its business.

If they fired 1000 engineers, at $250k/year each, that would come to about $1.2/week more for each monthly active driver.

Preventing engineers who are working on it from working at the competition?

That's pretty stupid. If this was truly their strategy, they're just raising everyone's costs. They can't out-compete Google at this game.

Google has a chat app?

What's it called this week?

How is Slack not suing Uber for uChat? The images on the link look like an exact copy.

On what grounds would they sue? Not only is it completely legal to copy a competitor's design (in most cases), Uber isn't selling this to other companies.

I don’t understand why the above user is getting downvoted to oblivion by people who don’t understand legalities. Trade dress[1] is a form of intellectual property. This is what prevents a company from copying a competitor down to the pixel. Apple and Samsung fought an infamous and long legal battle over this. [2]

[1] https://en.wikipedia.org/wiki/Trade_dress

[2] https://revisionlegal.com/trademarks/lessons-trademarking-tr...

It only matters for product you sell (hence the reference to consumer confusion). Ubers chat sounds like it's solely for internal use.

uChat is a fork of Mattermost, which looks like an exact copy too.


You think Slack came up with that design? Hah.

Check out some screenshots of Discord, that basically looks like "Slack with a dark theme."

And they all look like any generic IRC client with fatter margins and some other 2010+ fluff.

Would need to sue Discord too.

The post says they used an open source chat core called Mattermost.

Only companies with mega deep pockets (to pay the real winners ie the lawyers) would engage in these type of tom foolery.

I am talking about Samsung and Apple here..

Constrained solution space > all chat apps look the same.

Except Snapchat, I still can't work out what that's about.

Wow...that is pretty wasteful

Decision to build a chat app might seem wrong now in hindsight but this could have been a strategic move (as hinted in original comment).

Look at their competitors in Asia (all-in-one services like WeChat, GoJek etc. where search, chat, ride hailing, banking etc. are all built in one app) and it would make sense for them to have an engineering team building a chat app.

I love how the solution to this is always: "Let's build our own thing!" instead of: "Let's just use something free and/or open source."

Open Source chat tools are sadly almost universally disappointing. I believe I tried just about every one of them before giving up and just using Slack. I detailed the reasons here:


I have the impression that Slack has gotten worse. We're again having problems of notifications not showing up.

If you were willing to use Slack after all, then I guess you were willing to consider non-FOSS alternatives? In which case, one service you don't seem to have considered: Discord.

Yes, really, for business. Look past the theming.

Discord has a much more solid tech base (and engineering team) behind it than Slack, IMHO. A better API, too; and—unlike Slack—real support for third-party clients!

I built my own niche team-collaboration app back in 2012, and I tried out a lot of things, but ended up building my whole own stack. That was back then. If I was doing it again today? I'd just make it an alternative Discord frontend.

(There are also offerings specifically designed for the "build your own custom frontend for our chat backend" use-case, like https://pusher.com/chatkit, but IMHO Discord even has these beat.)

Right, let's just do all our business on an app that doesn't have SSO and instead anyone with the link who can sneak in has access. I'm sure that's not sketchy at all and your IT department will be thrilled.

> and instead anyone with the link who can sneak in has access

You don't have to make those links (and you can turn off, per server, the ability to create those links.) You can invite specific users. The user already has to have a Discord account, though.

And that's the thing; Discord has a different accounts model. Which is part of what I was referring to when I said they had a more solid tech base: when you're signed into a dozen Slack workspaces, the Slack client has to keep a dozen websockets open, because the open connection is for a given Slack account's view of the world, and Slack accounts are a sub-resource of Slack teams. This isn't a scalable model, and when the Slack app is using 1.2GB of memory and two full CPU cores, this is much of the reason why. When you're signed into a Discord account, on the other hand, you just have one connection to the server—and one state-sync session that includes all your open channels in all those servers—because your profile on a given Discord server is a sub-resource of your global Discord account.

It's the difference between how an email client connects to N accounts through IMAP, and how the Gmail mobile app is connected to N GSuite accounts.

And, as such, it wouldn't really make sense to have Discord SSO at the account level, because a Discord account is associated with the individual, not with a particular employee record of a particular Enterprise. It's not like an email address; it's more like a Keybase identity or a Facebook profile. (You could certainly have particular profiles under the Discord account that did SSO to a particular server, but this wouldn't be traditional SSO-as-we-know-it; it'd be more of a "connector" flow, like when an app wants to use your Github profile.)

Github is a good comparison, actually. Most corporations just use Github, not Github EE. You don't Enterprise-SSO to Github.com. Github.com is an SSO provider, in fact, that other sites (e.g. Asana) can take auth from!

I don’t think this is an advantage? Work account can be accessed by your IT department, locked out when you leave, investigated for any reason. I certainly don’t want to have any links to my personal stuff at all!

As for github.com, our official IT recommendation that people don’t use their personal accounts, but create one just for work. This way there is no worries about personal data there.

(This is at a subsidiary of a large international company. I can believe small startups use a simpler methods)

> I certainly don’t want to have any links to my personal stuff at all!

Keep in mind, putting SSO into Discord’s account system (if that ever happened) wouldn’t be like putting Enterprise MDM on your personal phone. It would be more like having Chrome Sync signed into both your personal Gmail account and your GSuite account, as separate Chrome “People” (or whatever those are called.)

With Chrome Sync, the profiles are still isolated from one another; but updates to them ride in the same sync carrier connection, because that connection is going to Google either way. In this sense, the Chrome installation itself, and its “installation-specific sync client ID”, is like a Discord account. It’s not a personal account, or an enterprise account; it’s a nothing in-and-of-itself, that has those types of objects under it.

I always thought of Electron as a way to kick out a minimum viable product quickly before writing native versions. But nope, I guess give a $16 billion company 4 years and 1600 employees and that's still not feasible.

Electron gets a lot of hate on HN. I get it. But IMHO the existence of VSC (Visual Studio Code) -- as an extraordinarily useful, powerful, compelling cross-platform application -- serves as the only counterexample needed to puncture the "it sucks bc it's Electron" trope.

It can absolutely still be useful, and it would make sense for an early stage startup or someone's side project to use it so they're not duplicating work porting it to .NET, Cocoa and Qt or whatever. And sure, having an identical look and feel everywhere is nice. But with all the resources that Slack and Microsoft have, they need to publish a lengthy guide for the users to deal with their performance problems [1]? They take 22 seconds to load a 6 MB XML file [2]? Ehhh... it's not asking that much of a multi-billion dollar company. You deserve better!

[1] https://github.com/Microsoft/vscode/wiki/Performance-Issues

[2] https://github.com/jhallen/joes-sandbox/tree/master/editor-p...

"Visual Studio Code jumps to the end of the file quickly, but then takes many seconds for the highlighting to complete."

I'm quite fine with that, frankly. Highlighting correctly requires actually parsing the file. I use vim and the colors get wonky way too many times, because it tries to "keep it fast".

A dozen web sockets are not the reason for high resource usage. Web sockets take very little to maintain.

It’s not the dozen websockets in a literal sense that are the problem; they’re more an especially-visible consequence of their design choices that is easily noticeable during a design audit—a “canary” that shows the flaws of their model.

Slack has decided that each team workspace must be firewalled off from the others: such that it needs its own websocket, its own client-side sync session, its own open channels, etc. This is because Slack was first-and-foremost a web-app, and in the Slack web-app, you’d just have one browser tab per Slack team. Slack built up its infra assuming this “one tab per workspace” model, and it crept into their backend API design, and now they’re kind of stuck with it. So now, even in the native mobile app (let alone the desktop Electron app), when you have 12 Slack teams open, you have basically 12 copies of the app’s view-controllers open and idling in the background, 12 background sync controllers, etc. In the case of the Electron client, that’s also 12 renderer processes, just as if you had 12 literal browser tabs open!

You might not notice this at first, because Slack won’t initialize all those resource for a workspace’s “tab”-equivalent until you actually focus the given workspace at least once. But if you just skim through all those workspaces once in the morning, and leave Slack open, it’ll be hogging those resources all day.

(And, on mobile, that’s especially a concern, because despite what you say, web-sockets themselves have heartbeats, and 12x the heartbeats is 12x the reasons for your phone’s modem to wake up. If you’re ever wondering why the part of your phone around the antenna is getting hot even when you’re not actively using data—it’s probably that you have several active Slack workspaces in a live Slack app-container.)

>because despite what you say, web-sockets themselves have heartbeats, and 12x the heartbeats is 12x the reasons for your phone’s modem to wake up.

Don’t put words in my mouth. A heartbeat is not high resource usage.

Your wall of text doesn’t negate the fact that multiple websockets can be maintained with trivial resource consumption. It absolutely cannot be generalized to assume that multiple websockets means copies of everything. I’ve seen multiple clients supporting multiple websocket connections to distinct servers while re-using all of the same UI components.

Everyone in the Open Source community with the skills to write a good chat app is perfectly happy running IRC in a terminal. Good Open Source only happens when someone needs to scratch their own itch.

We've been using mattermost internally for a year or so and his has worked flawlessly. I can't tell the difference from slack.

At least a year and a half ago, Android notifications weren't at all reliable, even when on wi-fi. You'd have to start the app most of the time to get them. I can't remember if it was Rocket.Chat or Mattermost that also was taking 15% of the CPU while idling on macOS, but I think it was Mattermost.

Have you tried out wire? I like it, but never used it in a work environment.

Matrix has been a better fit at my company, though it was a bit rough a year ago.

Bridging in Slack, Gitter, IRC and other platforms makes working with external teams a breeze, everything is in one place.

Uber's chat platform is based on the open source mattermost

Instilling a culture of relentlessly outsourcing to open source is insanely hard. I’ve been trying for years.

Engineers like to build haha

I mean, it probably wasn't in the right programming language and was missing one little feature the org thought it needed and hell.... how hard is a chat application, right?

At big companies, this is often the preferable alternative. Making and releasing quick bug fixes on internal projects is way faster than open source and public projects. Ownership and control is critical.

Big companies can't fork?

I recently worked with a ex-uber engineer, hired by a ex-uber engineer manager. We were piloting use of launchdarkly, and he suggested we just build it ourselves as he has already done it few times and would take him only few months to build.

Your rant is misinformed, Uber uses Mattermost: https://mattermost.com/customers/uber/

I work at Uber and we do use an internally built chat app.

The story I heard was that the team in charge of building uchat whitelabeled mattermost and tried to pass it off as a custom job.

Perhaps the memo didn't make it around to everyone.

Which is built on Mattermost.

So nothing related to Mattermost?

It cannot be shittier than slack. My slack desktop app consumes like 20% of my macbook's ram and phone app misses important notifications

I have found the over-consumption problem with electron based apps in general. However I didn't really miss notifications.

I hate, hate, hate when engineers with a bad understanding of business and too much time on their hands

The equal and opposite problem is managers who don't understand the sunk cost fallacy and refuse to throw out home grown solutions. Sometimes there might have been a good idea to homebrew a thing years ago because nothing good existed. Now several better solutions exist but the company refuses to adopt them because doing so would mean "throwing away" years of work, and the managers who signed off on that work are afraid they'll look bad.

Back in 1994 I worked at British telecom on one of the first high profile web projects for our intranet.

Apparently there was a serious suggestion that this new www was shoddy and we needed to write our own version of http - this got stamped on by some senior people at Martlesham (UK version of bell labs)

> because the original author left for greener pastures

Well that's the incentive for that kind of behaviour, pad your resume and get another job.

The part I don't get is how

> hey, slack is way to expensive, we don't want to get 'locked in', and 'mumble mumble privacy cloud evil stallman'

gets followed up immediately (almost inevitably, in my experience) with

> so lets write our own chat application.... how hard can it be?"

...rather than, say, "hey, I googled and there are a few FOSS alternatives for 'group chat with history': Zulip, for one; Mattermost, for another. Let's see if either of those fit our needs. If they don't quite, though, let's also see if either one of them has a solid, customizable codebase to fix up!"

But... they did, uChat is a fork of Mattermost, and they contributed the changes back to the open source project.

They did. Which makes them not an example of the GP's comment. Most companies do, in fact, end up literally attempting to "write our own chat application" from the ground up. (After all, that's where Slack itself came from—NIHing an MMO-game's chat middleware!)

This is exactly what they did


they wrote that fancy blog post but couldn't scale ejabberd to 50k users? Whatsapp managed to scale ejabberd into whatsapp so..

Is there any "chat app" that is really better than github/gitlab and a good IRC or XMPP server? I am constantly disappointed by slack. I wish google waves was still around...

I think this could be one of the reasons for Uber pulling out Twilio. Someone must have said 'hey we will build our own telephony suite .. '. I wonder how that has panned out.

You aren't wrong, but if someone paid me to write a chat application I probably wouldn't fight it. It sounds like a lot of fun.

> God I love the Not Invented Here disease. Somebody, somewhere, thought "hey, slack is way to expensive, we don't want to get 'locked in', and 'mumble mumble privacy cloud evil stallman' so lets write our own chat application.... how hard can it be?"

And even then they could have just used Matrix, which ticks all of those boxes.

How do you explain Amazon, then? They invented AWS to sell books on the Internet.

The real issue here is a lack of vision at the top. The current CEO needs to step down for the health of the company. He has a clear lack of vision for the company, as evidenced by his statement of "do your divisions look like what they should if we had to start over?" Who says that? Something so vague that is basically grasping at straws and shows a complete lack of strategy. Would Jeff Bezos say something like that? I highly doubt it, because he has complete control over his company and a vision that has worked at micro and macro levels.

You can do things that aren't directly related to your core business and these things should generate value both internally and externally. If you can't do this, eventually, your company will stop growing.

What does AWS have to do with an internal chat application? Is Uber going to roll out another Slack clone because they thought of something marginally better?

This where understanding business matters. AWS makes sense as a business and it benefits mostly from economies of scale which is something unique that a massive company like Amazon can provide the service which few other companies could.

NIH stuff is also rarely externalized outside of the business. Even as open source. Which as the OP pointed out needs proper investment of human capital. Not some side project of a bored engineer or two.

Most innovation happens outside of mega companies by small teams for a good reason. Even the Skunkworks type approach is only useful for certain types of work like Googles X projects. Some B2B SaaS programs roll out of parent organiations but typically they are started by teams who quit the bigger company to solve the problem they had.

You're correct that Amazon does NIH correctly and most companies do not. Amazon's pipelines and web (especially frontend) tech at scale is leagues ahead of the 'market'. However, very few people are able to appreciate this, so it's best to not even mention it on a forum like HN.

> How do you explain Amazon, then? They invented AWS to sell books on the Internet.

The first AWS service was launched over 15 years after Amazon.com started selling books on the Internet. It wasn’t even a separate brand until after Amazon S3, SQS and EC2 were launched.

> The first AWS service was launched over 15 years after Amazon.com started selling books on the Internet.

Close. It was roughly 11 years. I've been using Amazon AWS since early 2007; I believe it launched to the public in the form we know today in early 2006. They started selling books online to the public in mid 1995.

If you go back to the very first AWS services - 2002 - it's closer to being only seven years.

Sorry, typo on my end :-)

Amazon wasn't even using AWS at the beginning. AWS was a new product category for them. Albeit, they were able to leverage their institutional knowledge of managing data centers and probably the extra capacity of some of their servers.

In Amazon’s defense, there wasn’t anything as useful to their needs as AWS before AWS.

If a different company had dominated the on-demand computing space before Amazon’s peak load issues became too big a problem, they very may as well have gone with that and not made AWS.

Almost nobody needs to write their own chat app. Slack is fine.

That line about do they look as they should is just first line of the marketing spin they put on their layoffs. Obviously there was a clear order to cut a lot of staff to reduce payroll costs. And people here are saying there were way more staff than necessary.

Yeah and why would anyone ever buy a tailor made suit when you can get them off the rack for cheaper?


> 'mumble mumble privacy cloud evil stallman'

RMS was always right especially with regards to privacy.

Now imagine they did it like 4 times. Am I talking about Apple or Google?

Dam. I could have saved them a fortune by recommending Mattermost. Maybe even got Uber to make some e2e encryption commits also :)

uChat is actually built on top of Mattermost.


Interestingly, your comment reminded me that I wanted to look into Mattermost and they have the Uber logo listed on their front page...

Yes and no. I would love to use Slack over uChat, don't get me wrong, but uChat is a skin/integration of an enterprise chat app not a greenfield build.

Look familiar? https://mattermost.com/

>Uber definitely over hired in 2016

That's interesting, I interviewed with Uber for an engineering management role in 2016, and was blown away when my potential boss said he had a "hiring target" for next year of close to 100 people. This was a relatively small product offshoot of Uber that is now closed. I thought that was absolutely nuts, not just the magnitude of the number but that getting bodies in seats was a metric that he/I would be measured against, but thought maybe that I just didn't understand how hypergrowth startups work and it would be a good way to gain a new perspective!

Full Disclosure: I did not get an offer, even though I walked out of that interview feeling I had never nailed an interview as much as I had that day.

Not to doubt your own experiences, but if that was honestly true then Uber should also be scaling back hiring and not be opening new major offices with large hiring sprees.

There was some earlier discussion here on why trust between employees and employers is so low now, and this is a great example. Employees being laid off while new positions open up for that exact same role so that they can hire new developers at a lower cost or avoid having to promote other employees.

At companies I have worked at, the employees being let go are often given the opportunity to apply for other available positions. Maybe the case here?

This is golden. Grow responsibily. Don't overhire. Consider overhiring neglegent and treat it as such. Dissmiss managers responsible for neglegence, before or with overhires. This will also alleviate the costs of overhiring.

This guy thinks he’s safe.

He isn't?

Over hiring is a typical mistake after raising a new round of funding. Good to hear that you agree with the layoffs. Like the old saying goes; you can't put 9 women in a room and make a baby in a month.

The arrogance in this post is mind blowing!

> Learn to plan better and prioritize aggressively instead of just hiring and getting everything done.

I often wonder how accurate the roadmap prioritization can be. I do agree it’s pretty silly to build a lot of infrastructure that will never effect your clients.

If you have for instance a billing process that needs to be automated, custom infrastructure is needed. Manual approvals of all transactions is a lot even for banks. Operations can always benefit from something that cuts their overhead time down a significant chunk. Operations approval on transactions seems like an operations problem but clients having to wait for approval means now the problem is shifted.

Hey you're still at an order of magnitude fewer layoffs than Sprint ION after they blew off a $2B loss in the early 00s. Maybe get your leadership to look at that case study.

>Obviously everything I say is personal experience and opinion. I think the layoffs were long overdue and should've happened sooner.

I assume you're not on the side that got laid off. I wonder what their opinion is...

>I think the lesson here is to grow responsibily, and it's real people's lives that are being affected. Don't hire people to alleviate your current engineer's burnout and stress.

Because those are trivial matters, and they should just suffer through them?

A new chat app or mattermost with customization?

> severe shortage on infra teams

Seems like infrastructure/devops always gets short-staffed, presumably due to being considered a cost center. Interesting to hear someone observe this even as they're saying a company "over hired" and "engineers ran out of things to do".

Can you tell me if the M3 (time series db) team is still healthy? I was planning to evaluate it soon.

Any open source project without a healthy ecosystem is always at jeopardy. Use something that doesn't require you to question NDA information from some company.

Chronosphere (https://chronosphere.io/) was recently started by ex-Uber engineers to build a commercial ecosystem around M3. M3 is Uber's metrics platform and isn't going anywhere. M3DB (the TSDB built to support M3) is becoming fairly integral to Uber in its own right. I'd say the trajectory is positive.

Disclaimer: I work on the M3DB team.

Really interesting, it sounds like you overhired in engineering and underhired in product management.

Probably some reality shaking out. Uber's engineering team puts out some impressive stuff, often as OSS. Their engineering blogs are regularly on HN. I've been genuinely surprised that they churn out some of these things and release them for free given their relatively extreme financial situation.

In contrast to companies like Google, Apple, Microsoft, Amazon etc. that have mountains of their own money to burn (rather than investors') on research and OSS side-projects, it always seemed to me that Uber was trying to play the same game, but far too early. Paying lavish SF engineer salaries to generate cool, but not revenue generating, software is probably excellent for morale, culture and recruiting, but a dubious use of resources when you are losing money seemingly faster than it would be logistically possible to literally burn it.

Saying they're ~ "culling the low performers" can be entirely true, but it is also a Silicon Valley, meritocracy-culture-friendly way of saying "we're losing far too much money to pay bloated growth-stage poaching-game salaries to engineers, so if you're not working on something that generates revenue, glhf"

> that have mountains of their own money to burn (rather than investors')

I totally agree with everything you're saying. But I'm going to quibble with your phrasing. Apple's cash reserves belong to the shareholders just as much as Uber's funding rounds.

Too many CEOs operate under the mistaken belief that retained earnings is "play money" in the way that paid-in-capital is not. For investors, retained earnings are subject to the same opportunity cost of capital as funds raised by equity or debt.

Its management's responsibility to deliver returns exceeding the firm's weighted-average cost of capital. If they can't do that, then they should return capital to the shareholders, who can then use it an alternative higher-returning venture.

Good point. I didn't have that perspective in mind so the wording was off.

My sentiment was that if a company is losing money consistently and egregiously, they are on borrowed time and a borrowed dime in very real terms as the trajectory is towards 0 - but the context is largely psychological. To your point, waste is waste. I agree wasting cash generated from profits is equivalent to wasting it from earnings.

I'd insist there is some practically relevant difference in there though.

Wasting money during a trajectory to bankruptcy creates a narrative of negligence that accelerates failure, while wasting it during a consistently profitable trajectory seems like sub-optimal management. The kind of thing that is theoretically identical, but in the real world of behavioral economics, the former seems more certain due to how easily the trajectory to failure can be estimated. The latter creates a weak narrative because of hidden information - nobody will ever know what "could have been" and so can never quantify how sub-optimal the management was.

E.g. nobody is dragging GE executives out of retirement/the grave to answer for long-term effects of sub-optimal management, and further nobody could prove at the time it was sub-optimal, only hypothesize. On the other hand, everything Elon Musk does at Tesla is torn apart and front page news, because they have a trajectory towards failure a high school student could easily calculate.

So "management's responsibility" to optimally allocate capital is sound in theory, but in the real world of imperfect and outright unknowable information, nobody ever really knows what optimal is. Sub-optimal comes to be expected as normal, but accelerating a trend towards failure is a powerful defining narrative. Somehow this matters.

Why would someone whose been tasked with a job that is involved with gathering as much money as possible, be willing and able to just turn that part of their personality off and start giving money to someone else? Why would they do this just because its investor's money rather than customer's money?

Presumably because the board (which represents the investors) would be displeased.

I get that this is the normal ideological description of firms post-70s neoliberal whatever, but if that’s true, like why not just say firms suck, we need communism? Like your description makes Pikkety look like an optimist. “The purpose of a firm is to help rich people get money faster than other rich people” logically implies that eventually a small group of rich people will have all the wealth. That’s feudalism. If that’s the goal, let’s start a revolution instead.

share holders longer term don't have to be rich

I agree, but the theory "firms exist to maximize shareholder value" ensures that they will be.

Put it this way: gravity turns space dust into supernovas. Dust is just ever so slightly attracted to other dust, so it accumulates and accumulates, and eventually it becomes so massive that it forms a star.

The theory that firms should beat the market makes money gravitational. A shareholder who beats the market will get more money than other shareholders. Now they have more money that they can use to invest in other market beating schemes, etc. If whether a firm beats the market is random, then some investors will win and some will lose but it all balances out. But if beating the market is not random (and how could it be totally random?) then those with the most money can invest in the best firms faster and more easily than smaller investors and crowd them out. Remember that companies only have a finite number of shares, so not everyone can invest in a winner. If there's even a slight bias towards having more money making it easier to beat the market, then eventually you will get a supernova.

We all understand this on some level. Why is insider trading illegal? Because it makes it trivial to beat the market!

Put it this way, if the standard theory of why capitalism beat Soviet communism is that competition created better products etc. none of that requires that the goal of firms is to create better than market average returns to shareholders rather than the goal of firms being a) earn sufficient profit to continue to exist b) benefit consumers c) benefit employees d) benefit shareholders. Yes shareholder would prefer to invest in companies that prioritize them over all else, but why should we the public not say “that’s not an acceptable charter. We don’t want feudalism to happen again, so we are not allowing firms to prioritize shareholders over consumers.”

If the only defense of capitalism is competition, why not market socialism? As in, markets, competition, and entrepreneurship still exist, but all firms must be worker-owned.

Because then you would be forcing everyone to be an equity partner, even if they would rather take a higher salary than share in the risk. Doesn't sound like freedom to me.

The "freedom" to monopolize capital is not a meaningful freedom, any more than the freedom to starve in a ditch is a real freedom.

I imagine independent contracting would be a way to avoid that.

Quibble: I don't really see worker-owned firms as a form of socialism, which I understand as the government ownership of the means of production. Worker-owned firms I associate more with distributism.

Worker-owned firms are a great idea, but if that's the only ownership model then you lose some ability to diversify your investments. Everyone's 401k goes poof.

Thinking through this, what are the alternatives? Simple cash reserves are out (because most societies can't seem to shake the tendency towards inflation). Bank savings accounts are a good idea, although impractical in our current society because interest rates are so low.

It is too fragile.

Did I miss when Denmark elected Trump because the people completely distrust the elites and wanted someone to burn it all down?

Alternatively, management could be inclined to not give piles of wealth to people who had zero hand in its creation (shareholders, modulo some employees who also own shares [rounding error]) and use it to pay engineers to do interesting things.

You need 4 things to run a company, in small companies some roles may overlap. In no particular order Workers, customers, management and shareholders/capital. When one gets too much power the company goes rotten. Usually but not always, it’s management.

Shareholders pick the board, board picks CEO. If CEO wants to spend money that his choice until the board gets rid of him. It's not some sacred pile of cash that 'belongs to investors/shareholders'

> Saying they're ~ "culling the low performers" can be entirely true

Even if it isn't they have just branded everybody with 'Uber' on their CV that is on the market a low performer. Think before you speak.

It's especially a dick move when everybody already knows they're hurting for cash, and the official position could be "we just can't afford all these people" without the company losing any face. They aren't fooling anybody by pretending it's not about the cost.

If you're going to lay off people in a cross cutting manner, who would you choose?

If you could chose low performers, lay them off, and be more focused and higher productivity with the remaining people...

_Why haven't you already done that?_

Nobody likes low-performers, even when you're flush with cash. Fixing their mistakes is demoralizing for your high-performers. The word gets around that you have low standards, and it's hard to recruit.

The only way to get your money's worth from them is to put them in death marches and grind them down until they burn out and quit.

I am always extremely suspicious of claims that a company is going to lay off its poor performers and magically be better. If they're telling the truth, they are actually saying that their existing managers are incompetent.

There's a reason that all tech companies try to brag that they only hire the top performers.

>If you could chose low performers, lay them off ... _Why haven't you already done that?_

There's a reason that all tech companies try to brag that they only hire the top performers.

Performance of an engineer is relative to his environment. I was a top performer on some teams/projects and a low performer on others.

Even if they ace your hiring process, you still can't know how they'll fit with the team long term.

Fine, but laying off people who don’t work out is an ongoing process in all companies. If people haven’t already been let go as a part of the company’s existing management review process, what changed suddenly that the company can suddenly lay off a bunch of people in one go and “improve?”

Another way to think about it is that everyone has some productivity, some contribution to the company’s net progress. If someone’s contribution is negative, they should already have been let go.

If their contribution is less than others, maybe “bottom 10%,” but it is still positive, you may be letting go a relatively poor performer in a round of layoffs, but you’re still shedding a person with a positive contribution.

You are going to be worse off, no matter how you sugar-coat it.

> Nobody likes low-performers, even when you're flush with cash. Fixing their mistakes is demoralizing for your high-performers.

If you define performance as change over time then this would seem to have nothing to do with mistakes. From what I've read about the space shuttle programmers, they would have been classified as low performance (low change over time) but who also made with very few mistakes (as I understand their process). At the other end you could have high performing programmers (lots of change) with high defects that they're always fixing (which could in turn qualify as still more change).

> _Why haven't you already done that?_

With layoffs you want to apply “cut once” approach or at least as rarely as possible, in batches.

Constant trickle of layoffs is very bad for morale, no matter who is laid off. It is also bad for an external image of management.

Well, the colloquial understanding of a “layoff” is that it is not based on underperformers, but based on changing business conditions.

For example, closing a plant, or getting out of a line of business. If Uber decides not to have anything more to do with self-driving vehicles, they might lay off everyone in its division.

That would have nothing to do with poor performance on the part of individuals.

On the other hand, there is “These people are underperformers,” which is part of Uber’s allegation as they throw their former employees under the bus rather than take responsibility for their management choices.

I contend that if people are underperformers, a constant trickle of letting such people go is not bad for morale. It’s perfectly normal.

“Did you hear they let Dave go?” “Yeah. What took them so long?” That’s the usual talk.

Whereas, “Did you hear that they shut down ML?” “Yeah, and it was half the database tuning group last month, who’ll be next?” “I dunno, but I’m not hanging around to find out...” is the thing you are describing.

Long story short, I agree that a trickle of layoffs is not good, but I suggest that this is true when the people being let go are not thought of as holding the company back.

I disagree that it doesn’t matter who it is. If they are underperforming in the sense of being bottom 10% but still carrying their costs, I agree with you about trickle, but then we can’t claim that letting them go makes the company stronger.

But if they are underperforming to the extent that letting them go makes the engineering group more effective, then management should have identified them earlier, done everything in its power to make them perform, and let them go if they didn’t improve.

Ignoring net negative employees, or being blind to whether they are net negative, or keeping them around even though they are known to be net negative is bad for the company’s bottom line and bad for its morale.

Hello Reginald, I am a fan and a fellow Torontonian!

I agree with your points, but optics might depend on a company. In a startup or smaller company being aware that underperforming employees are let go might actually improve morale. For bigger companies it is a typical situation that you notice or get to know that people from other teams are gone, but you may not be aware of their performance.

True, but then again you probably don't notice so much. Every month there is a new face here and a new face there, and one old face has... Quit? Retired? Been fired? Who knows...

I like reading your writing so I would appreciate your input/feedback either here or you can email me at gmail, whichever is more convenient.

> Nobody likes low-performers, even when you're flush with cash. Fixing their mistakes is demoralizing for your high-performers. The word gets around that you have low standards, and it's hard to recruit.

Fixing mistakes is part of development. I am yet to be part of a team that never made a mistake. Mistakes are how we learn and get better.

If the same mistakes repeat, THEN you have a problem but if a single individual is to blame everytime, then it's not the individual's problem - you have a bad team. Your team has failed at teamwork - the primary focus of any team.

If you have a team of dozen engineers making a car, the car does not drive until all the parts are not only in place but they all work well together.

The way to make all parts work well together is to have the designers of the parts communicate and work together, well.

If the engine guy is not talking to the intake and exhaust guys, don't be surprised if there are leaks at best or the engine blows up at worst.

I AM assuming each team member went through an interview process. If the going was tough where you could not interview and had to let just any person in, hopefully this person helped you through the tough times.

What value does the interview process provide if you can't figure out if your candidate is a high-performer or not?

Why did your interview process select a low-performer?

Also, performance is the output of a team - a single person should not be expected to provide high-performance day in and day out.

That's impractical to expect out of a human being. A machine and a person have vastly different characteristics.

If my team is working well, we will have a high performance team. There is no other way to have high performance sustained over a long term.

Each team member needs to actually looking forward to working with each other.

The other extreme is programmers work in these silos and come up with solutions that barely work well with each other and then there's blame game being played.

Why do that nonsense? Save money? No - of course not!

Just work as a team!

> The only way to get your money's worth from them is to put them in death marches and grind them down until they burn out and quit.

I don't follow.

Death marches?

First of all WHY do this at all?

What's wrong with the alternative "Hey, it looks like it's not working out. We will be letting you go and support you in looking for a job elsewhere over the next X days"

At one point in time in the past, you and your team interviewed this person and found them useful - what changed that they are no longer useful?

Was an error in hiring made?

Own up, take responsibility and move on. Make the interview process better.

Did the person expect to get a larger bump come bonus/review time that did not happen and now they are demoralized and upset?

Own up, take responsibility and move on. Clarify better what bumps and bonus would be like and how much effort that would entail next time.

> I am always extremely suspicious of claims that a company is going to lay off its poor performers and magically be better. If they're telling the truth, they are actually saying that their existing managers are incompetent.

I also don't follow this and want to hear more about this perspective.

What happens to the people working in mining, transporting and delivering ice from Iceland to California when refrigerators get invented?

Are those people inherently low performers or has the industry they work for shifted beneath them?

We (developers) work in an area of high specialization and the market values us for it (to the tune of six figures).

I have worked at both product and service companies before where entire departments were laid off because a contract did not get renewed or the market changed.

Most employees AND manager involved there were and continue to be competent. We even hire and refer each other wherever we might happen to be to this date even though we met decades ago.

I'd cut at the team or department level. I'd choose teams/departments based on KPIs and necessity.

People who aren't critical to the core product, or whose projects aren't big revenue generators. They might be high performers while being technically unprofitable.

> They might be high performers while being technically unprofitable.

Why would you do that vs moving the high performers to the business critical projects?

Not necessarily a bad idea, but disrupting those business critical teams by just swapping out people could actually slow down execution and hurt revenue.

Because they might not be high performers there? There is no such thing as an objective "high performer" in any area.

So you're saying software devs are just cogs that can be easily replaced?


I'm implying that rather than laying off top performers you find them a different role in your organization.

Granted if the skill set is incredibly niche--such as hand optimizing HC12 assembly and you've moved to ARM then perhaps not.

It's hard to imagine a scenario where an entire project teams skillset is so niche that they couldn't find a home for top developers in other parts of the org.


You can't just swap out members of a team, or add more members to a team, and expect the result to be faster execution (in short run) even if the new team members are high performers.

In the first case, you're losing team members with tribal knowledge of the system and its requirements. In the second case, you're exponentially increasing the pathways of communication. In either case, you still have the ramp-up time for new team members.

Any number of reasons. They might be high performers in skill sets not needed in those business critical projects. You might be leery of the time needed to get up to speed on something completely different.

Uber's stock has been going down the drain. Why would they give more reason's for investors to cash out?

You can just say something like "we're restructuring to focus on our core competencies". If you fire a bunch of non-critical staff, you just told investors "our revenue to cost ratio is about to get a lot better" and so if anything investors who have been watching the decline are going to recognize you are doing something about the problem.

In most situations market reacts positively to layofss unless there is a BK looming.

Indeed, HP (or HPE and HP Inc.) has done many layoffs in the past few decades. I just narrowly dodged one a number of years ago. Pretty much always the goal is to simply reduce costs without reducing revenue (as much). Investors don't care how much staff you have, they just care if you are making money.

Layoffs are always a failure of leadership, but leadership will always choose to dodge blame if given an out. Hence, "low performers."

> Layoffs are always a failure of leadership

I completely disagree. Which would you rather have:

(a) Business is booming, but leadership holds off on hiring because they believe the boom won't last forever.

(b) Business is booming, so leadership hires to support the boom. The boom eventually subsides, so leadership decides to lay off as the business is no longer there.

Fortunately, even as an employee, I'd much rather have (b) (assuming the timeframe was relatively decent, i.e. I didn't get hired and laid off in 3 months).

I've said this before, but in growing industries, I do not believe lay offs are something to fear. I mean, given the desire for experienced engineers and product people, I have no doubt these Uber folks will get snatched up extremely quickly. (and to emphasize, I certainly do NOT believe this is the case with shrinking industries)

My issue is this framing of it as "we're cutting low performers."

(c) Robot cars were a mistake.

You can't be a unicorn and not be able to afford the people. This is not a story for people in the tech who know about the industry and the six figure salaries engineers at unicorns make.

This is a story for retail that will be buying stock. "We cannot afford X" can be an stock buster.

So? They could lie (which is far worse) or say nothing and let people speculate.

A job candidate with solid skills will be able to show their value to employers, and if an employer puts more weight on this quote over a candidate's qualifications then it was a bullet dodged.

> if an employer puts more weight on this quote over a candidate's qualifications then it was a bullet dodged

I generally agree with this, except when the candidate in question needs the money paid from a job more than they need a good job.

Good point - didn't think about that at all. Did you mean think before I speak, or Uber?

In any case, yeah, in that respect it is a massively shitty and unnecessary thing to do to employees. They could have easily left it as "re-structuring" without a risk of tainting the reputation of former employees.

I don't understand where people are getting that they said they are "culling low performers". I didn't see that anywhere in the article.

Yeah, they should be firing low performers. A lay-off is when you lose business, with the prospects of being re-hired in an up-turn.

Certainly that should be part of it, but the goal is to reduce costs without reducing revenue. Low performers are certainly a cost, but there's only an indirect relationship between job performance rating and revenue.

They did some impressive stuff, but internally the eng org was bloated as hell, thanks to Thuan's leadership. Ironicaly, Thuan recently asked his directors to tell him which departments are bloated so he could decide who to cut. So much for the "amazing leadership".

>> In contrast to companies like Google, Apple, Microsoft, Amazon etc. that have mountains of their own money to burn (rather than investors') on research and OSS side-projects

You are assuming that these companies are letting engineers to do side-projects that are unrelated to their day to day work?

As far as I know the OSS projects these companies put out are in line what they use internally for day to day work and it is absolutely core to their business.


- Amazon: https://firecracker-microvm.github.io - Google: https://github.com/google/guava - Microsoft: https://github.com/microsoft/vscode - Apple: https://github.com/apple/foundationdb

Am I missing the point? Are OSS projects from these companies that are side-projects?

While on the subject, I am not sure about Uber's contribution to OSS. Their projects tend to be outside of my purview.

>> Paying lavish SF engineer salaries to generate cool, but not revenue generating, software

Nobody is forcing Uber to employ people in California.

A side-project isn't by definition irrelevant, and of course by sheer probability the engineers at those companies are, on average, going to create things that are within their area of expertise which is usually related to what the company they work at does :).

The motivations for making software OSS are varied and often strategic. For example software is often open sourced to:

* deliberately commoditize it - to eliminate competition and differentiation in that area / stack layer * leverage additional (free) testing and development. * Drive ecosystem / platform adoption * Literally free up customer budgets to be spent with them, rather than on licensing 3rd party software * Intangible benefits like community image, employee satisfaction, etc. * Some combination of above when the software is useful internally, but simply not in a market that the company wants to compete in, or a market worth their time

In any case, the point was that they are "side-projects" from a business perspective in that they cost money but don't generate revenue - the benefits are hard to quantify. Successful, stable companies have much more breathing room for strategic and the "hard to quantify" ROIs than does a company on a trajectory towards bankruptcy. It is kind of like an individual out-of-work engineer with a month's expenses left in the bank choosing to contribute to OSS instead of seeking out paid work. It is not that contributing to OSS is wrong, it is that the priorities in that situation are backwards.

RE: "Nobody is forcing Uber to employ people in California." That is entirely true, but I'm not sure what your point is. My comment is a post-mortem on what led to the layoffs. It is a priori that nobody has forced Uber to do anything that it did...

I have also been impressed by everything they are producing and I'm actually really wondering why do they need all of those new OSS projects. They need to scale but to an order of magnitude lower than most other webscales. None of their application is data heavy

I have said this before but they seem to have a strong "must be built here" culture. You typically see this in highly political environments where engineers are trying to create projects as a career highlight and as a justification for promotion.

> None of their application is data heavy

Really?? They have O(thousands) of drivers in O(hundreds) of cities with the app open, sending and receiving data approximately all day.

What companies in your mind are data heavy?

While not a "tiny amount of data" Uber's problem seems like one of those embarrassingly parallel ones where partitioning by location is trivial, and accuracy of all the data isn't as important as knowing everyone's location right now. I'd guess any relatively popular MMO deals with similar things under much tighter latency constraints.

Data heavy to me means terabytes of data with potentially multiple dimensions, like Google or Facebook.

Not to say that Uber's problem isn't challenging, but I don't expect that the amount of data is necessarily the problem.

Can't speak for OP, but I'd imagine most of the data in the system is metadata - i.e. very tiny. Sure they're moving a lot of packets, but an entire ride's data could probably fit in 100KB. If they have a million drivers active, what data do they actually need on them? Profile, GPS location, type? Few KB. Keeping the log of all rides probably takes up the most, but still, relatively small. In any case they essentially deal with small, highly compressible metadata.

Think of that vs. a company like Instragram where people are uploading probably terabytes of media content every day, and they have to maintain/host all that content to be served on-demand basically forever. This is probably a low-key reason for pushing "stories" - they get a reprieve by only having to store that for 24 hours. In any case, you're looking at orders of magnitude larger data usage in all aspects.

Or YouTube, with 1000s of hours of video uploaded per minute.

FWIW an Instagram pic, 1080x1080 is around 100kb.

Uber has to do a lot more processing on it's data. I'm sure there are real challenges and the OSS projects from Uber I've seen/remembered on HN seem to be the type one builds because existing tools don't solve the problem well enough for their cases.

Uber eats, for example, gives a guess of when your order arrives based on past data.

Uber gives estimates of when your ride will show up.

They do lots of stuff in fairly real time with that data and a lot of it is probably compute intense.

Not at all comparable to Instagram.

FWIW, Instagram stores Stories, privately to the user, in its "Archive" feature indefinitely.


Not to mention the hundreds of thousands of passengers waiting for and riding in vehicles.

The "data heavy" businesses often have the ability to cache their data in CDNs. Uber's data is realtime and dynamic, you scale that the hard way.

83 countries, 858 cities, 75 million riders, 3 million drivers.

5 billion rides in 2017. That's 14 million a day on average. I would guesstimate peak days have at least double the average.

That is data heavy to me. (-;


In the grand scheme of things it just isn't. 14 million rides a day, giving them a generous 1MB of data per ride, is "only" 14TB, 420TB/month. The data relevant to a ride for the clients like GPS location and ETA can fit into a few bytes...

A single video doorbell can use 200GB / month. With only 10,000 users that is 2 Petabytes of data traffic per month. Not to mention recordings that have to be stored and hosted for streaming later on.

Everything is relative (-;

Maybe, but the problem is that in a restructure like this the high performers often leave in protest to the culture problems that a restructure creates and austerity measures.

Often though they don't cull low performers, they give each department a budget or a head count they have to hit. A department made up of high performers will have to cull a high performer to hit budget, a department which is overloaded will become more overloaded and people will quit in response.

In a company the size of Uber a restructure is often a pretty blunt instrument.

Airbnb seems to do similar things, but has a better business model to support it. I kind of get it from a recruiting standpoint and they probably did need to build some customized software given the scale they have. My guess is they spent far more money trying to conquer rideshare in multiple countries, self driving, and side businesses. At the same time, they did look like they were also developing some pretty low level software that was probably not necessary.

Can someone explain to me what are the technical challenges at Airbnb? They need to handle less requests than the top airlines or hotel booking websites.

Airbnb is a business innovation with a shiny website frontend. But I really don't get all the hype around Airbnb engineering.

You don't need immensely challenging technical problems to make hiring good engineers worthwhile.

It doesn't really cost anything to open source something though. These are all projects which are developed for a reason: the commercial version might be prohibitively expensive at Uber scale, or not technically capable of operating at Uber scale, or just plain doesn't exist yet. At that point you can engage with the tech community and offer it as OSS (not to mention make your engineers happy), or you can keep it to yourself and eventually lose that opportunity to somebody else.

> It doesn't really cost anything...

Depending on context, sure this could be true. Like if you're comparing the cost of releasing some small OSS module of Uber's to Uber's annual revenue, sure, it's going to be a negligible cost. But if you compare against the cost of developing the module in the first place, it can easily be an equal expense. For example I spent several months on a BLE library for Android (https://github.com/iDevicesInc/SweetBlue) and asked my company to open source it. It took several more months to get the library to a point that was suitable for a proper OSS release. So cleaning up code, making sure no sensitive info, swear words and such, documentation, lawyery stuff, icons, PR copy, blog post, basic website, wiki, and much more nitty gritty I'm glossing over.

I mean a company can just throw code over the wall into GitHub and call it OSS, but I associate a basic level of polish with a proper OSS release that does indeed take a good amount of effort. And that's just initial effort! If it becomes at all popular then you have further ongoing overhead.

I'd say a general rule is that open-sourcing something is at least 50% of overall cost of the project.

> So cleaning up code, making sure no sensitive info, swear words and such, documentation, lawyery stuff, icons, PR copy, blog post, basic website, wiki, and much more nitty gritty I'm glossing over.

I'm the manager of one of Uber's OSS projects, and all that the things you listed here ring true. I just know that in our case at least, OSS prep absolutely pales in comparison to the feature/operational work put in by the team.

To be clear, when I added "really" to that first sentence it was meant to communicate that there is some cost, but in the sense that it's negligible to Uber's losses, as you point out. I was responding to OP's "dubious use of resources" comment. But I can see that wasn't clear.

Yup I gotcha, I wasn't really disagreeing with you, just adding the obligatory "well actually it depends" comment. ;)

>I'm the manager of one of Uber's OSS projects

Do you expect to still be doing this same job (or one step higher) in 6 months?

Yeah absolutely! It's a successful project and pretty integral to Uber, I love what I do and have a lot of faith in my manager/director. I don't anticipate any reason for our situation to change.

> If it becomes at all popular then you have further ongoing overhead.

THIS is the one that comes into play. All that stuff you listed before is a slog to go through when you go to open-source, but unless you're parking the thing as a proof-of-concept/end-of-life, you're now signed up for maintaining the repo. That means triaging issues, pull requests, helping out when contributors don't understand why their build is breaking, etc.

And there's the PM aspect of it: you don't just want to develop a bunch of features without talking to your community, so communicating (in a public friendly way) what you're planning to work on, when folks might reasonably expect that, how they might be able to help out, which of their contributed features you might be able to take depending on where you're at in your lifecycle, and RESPONDING TO COMMENTS all takes way more time than just "building [closed-source] product" in a team of 5-20.

And of course, one of the hardest of all: telling people "no, we can't take that change" when they've spent hours and hours doing work for your project for free. In that regard, we're still very much iterating on a transparent design process that allows for consensus BEFORE too much work has been done (though as we all know, building prototypes is often one of the best ways to find out if a design works right or not).

If you're doing it all right, everyone involved in the project should be doing some amount of all of this every single day. There's no compartmentalizing an engineer on an OSS project as "someone who just writes feature code" vs. "someone who does the repo stuff".

So going back to OP's point: no, it doesn't literally "cost anything" (or very much) to do the basic act of open-sourcing, doing it the "right way" at scale where you're truly engaging and working with the bazaar is very expensive.

Full disclosure: I'm a PM at Microsoft working on PowerShell and was heavily involved in it being open-sourced and ported to macOS/Linux.

Tangent: Could you explain why I might want PowerShell on MacOS or Linux, if I use those as my primary OS? (I usually only open my windows VM for checking that my websites work in IE 11).

Linux is actually half of our usage on PS Core[1], so it's a great question. A lot of folks use PowerShell inside of CI/CD pipelines, especially for cross-platform app development of .NET Core applications (having a single build/test/deploy script is often cited as vastly preferable to trying to maintain a split of .ps1/.cmd/.bat and .sh/.py/.rb).

But also, lots of folks prefer an object-oriented pipeline. Many of those folks (our primary use case for 10+ years has been IT management) are used to PowerShell on Windows and starting to learn or be exposed to other environments.

We've also got lots zsh-style optimizations in PSReadline. In some cases, we've got some catching up to do, but there's also lots of unique interactive optimizations hiding around[2].

It's also great for interactively exploring REST APIs and building scripts via that experimentation. Try running

Invoke-RestMethod [or irm] https://api.github.com/.

Store that as a variable:

$a = irm https://api.github.com

And then look at all the properties you can explore:

$a | Get-Member [or gm] $a.<tab><tab><tab>

Oh, and of course, one of the big reasons is "so you don't have to open a Windows VM to remote into your Windows Server boxes and manage them from a Macbook".

This is obviously not an exhaustive list, but it'd take a lot more time to write about every benefit and scenario here. In any case, it turns out that our user base is pretty spread out among different scenarios, but between our repo and usage numbers, we've been really happy with how excited that such a diverse group of folks really love PowerShell.

And feel free to reach out to me on Twitter @joeyaiello if you ever want to talk more about your experience (or just hop into our GitHub repo). :)

[1] https://aka.ms/PSGitHubBI [2] https://docs.microsoft.com/en-us/powershell/module/psreadlin...

This was far more detailed than I expected. You seem very passionate for PowerShell. The rest method exploration looks very cool, I knew I might learn something cool by asking. Thanks!

100% agreed, I started doing this for a small project my company has. We don't really use it anymore, but it would be helpful to many hobby projects however we are now sitting here saying 3 weeks work to just give this code away? ick.

A lot of this work is self made at companies although.

That's not true. It costs quite a bit to open source a product if you're a commercial entity.. It needs to be reviewed (by legal, security, engineering, etc) and approved..

You're not just flipping the public/private flag on the repo.

Also, the doc, blog posts, feedback from the community, evangelization at conferences, thinking out the API design so it answers enough cases, not just yours, all take engineering time, that's not free.

We open sourced my team's project a few years ago ( https://nomulus.foo ) and it was a substantial amount of work. There's everything you're saying plus a fair amount of engineering to get the darn thing working externally; we were built/hosted within the Google monorepo which is obviously not externalizable. And you end up spending lots of time writing documentation that you hadn't ever gotten around to when it was just a project being worked on by a handful of engineers all sharing adjacent desks.

Not disagreeing but elaborating: The costs are higher when it's an already existing product of which parts the company doesn't fully own enough rights of for publishing. Even higher when it's not known which parts are fully owned by the company and which parts are of foreign origin. The costs are lower when it's meant to be open sourced from the start and it's been a project constraint since before the first line was written.

Of course. But compare that cost to (say) six full time engineers pumping out code for a couple of years. It's not the dominant cost by any means. Not to mention you're paying those other folks a salary regardless.

If it helps you attract better talent, well - there's a financial incentive there through increased efficiency which I'm sure on its own is more than equal to the review cost.

Why would you give your open source code a tighter security and engineering review than your in-house code? Are you hoping for security through obscurity? And quality through obscurity?

If you are going somewhere where you have to get changed in public, say the gym, do you make sure you are wearing your good underwear?

I'm pretty sure I didn't say anything like that?

> It doesn't really cost anything to open source something though.

Yes, it costs next to nothing to just open source a project if you mean literally just putting the source on some hosting site.

However, open sourcing something properly, where you respond to issues, document things properly, and provide build/test infrastructure to external developers is much more work. Not only that but you have to be much more careful when creating APIs and be much more vigilant about keeping backwards compatibility.

There was the fixed cost of developing it, and then the potential lost revenue of not selling it, similar to an opportunity cost.

If the presumption is that the software would be useful ("developed for a reason") economics suggests there is some price people would be willing to pay for it, and you forfeit that when you make it OSS. That absolutely costs something.

Sure you may not want to be in the business of commercial software (support, maintenance, ultimately just responsibility) which is a completely sane thing to avoid. However when you are bleeding money at the rate of a small country's GDP per year, avoiding opportunities to generate profit based on intangibles like community engagement can come off as ill-advised. In fact, if your company is declared bloated, it stands to reason that you should have the capacity to take on the additional overhead of selling the software. It is not that contributing to OSS is wrong, or that your points are wrong, just that there is a time and situation for this kind of strategy.

Contrast with Amazon, whose entire development of AWS was for the reasons you point out. Instead of open-sourcing it, to the extent that would be possible, they made it into a product and almost in a single move turned consistent overall loss into a massively profitable company.

So this is not at all to argue against the merits of having healthy OSS contribution at a company, but more to say that a company can't contribute to OSS if they are bankrupt - so avoiding the latter should be prioritized over the former.

Maybe the 435 people should get together and start a Uber competitor?

Or they can start driving for Uber!

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact