Google and to a lesser extent Facebook invest heavily in internal tools. IMO this is because they hire for the company, permit easy transferring, and programmers sure love to scratch their own itch.
Google has a tool where interviewers can provide feedback and a numerical rating for candidates. But it also has a whole data mining workflow: you can visualize an interviewer's past ratings on a histogram, compare their hire ratio against average hire ratios, etc. It's cool and interesting, and sees a lot more love than many of their user-facing products.
Facebook has Bootcamp, their ~8 week onboarding process. There's a zoo of purpose-built tools just for Bootcamp, like the bot that sends you a question whose answer is written on the whiteboard just to confirm that you attended some Bootcamp class, or the Bootcamp Mentor rating machinery.
Some of that is excessive but Apple's underinvestment sin is far worse. The SWE org learned to run lean during the dotcom bust, and never unlearned it. Radar is your lifeblood, make it best in the world! Please!
Netflix is indeed becoming quite the sensation for its internal policies— the general intelligence apparently prevailing there. Seeing tools as first-grade citizens seems like a very sane approach to me— empower each other and watch people create marvels.
Papermill¹ notably is one incredibly seducing tool IMHO.
Yes, Facebook invests heavily in developer infrastructure. On my former team there were many staff+ engineers who were extremely capable. More speculatively I’d go so far as saying quality of talent is actually stronger than many groups working on public facing products.
Management of the group was also strong in my opinion.
Does Facebook maintain an internal fork of Phabricator or is it relatively close to mainline? It's such a great product but parts of it are quite clunky - I sometimes wonder if there's some internal secret sauce to smooth these out.
Developer infrastructure does not seem to be the right area to compare with here just like Google's internal engineering products aren't, more like - how polished are the tools used by the ad sales people, analysts, moderators etc.
It's an interesting question. And if the answer is no, I guess the logical conclusion is that internal tools make very little difference to the success of consumer product companies.
That doesn't seem like an intuitive conclusion, but it's a hard circle to square. As someone who mostly builds internal tools (though in a very different context), it certainly seems like good systems are an Archimedean lever that acts as a force multiplier across the entire org. Yet, if good internal tooling was important, you think at least one company would have been able to achieve success partially on this basis.
The consumer product industry is extremely competitive, so the counter-hypothesis of entrenched sclerotic incumbents refusing to pick up $20 bills off the sidewalk doesn't really seem likely.
> And if the answer is no, I guess the logical conclusion is that internal tools make very little difference to the success of consumer product companies.
That is on the implicit assumption that businesses are reasonable, and accurately know how to manage tradeoffs. Your counter-hypothesis seems rather likely to me, if we look at the internal behavior of a company. When negotiating, the more dots that you need to connect before reaching "and then we get more money", the weaker of a negotiating position you have. It's the same reason why sales teams get bonuses for making sales, but developers don't get bonuses for enabling sales.
> And if the answer is no, I guess the logical conclusion is that internal tools make very little difference to the success of consumer product companies.
Or perhaps no one has realized the power of good internal tools, and so crappy tools are the status quo and no one needs to do better.
But as soon as one company realizes the value of internal tools and it becomes a competitive advantage, other companies in that area will need to follow suit.
Long feedback chains take longer to optimize so this is possible. Also one feedback would be to have the company fail. Another possibility is that it really is a tradeoff of where to focus.
For example, we are now paying for our lack of preparation and sclerotic government with the economic and lives lost to covid-19. It just took between 3-10 years to pay the piper.
Internal tools are used by employees who unlike consumers will tolerate more usability friction, non-critical bugs and feature delays than consumers and will compensate for them with their own effort so the tools can still be critical in their core functionality but not polished or well-maintained.
Ironically and anecdotally, WeWork tried to build a strong engineering org for their internal systems before it blew up.
To the contrary, some of the most successful software companies have strong cultures of internal tools (think FAANG companies). How much of this is cause and effect I'm not sure though. To some extent it's easier to make investments in internal tooling if you're profitable.
Facebook in particular seems to wildly overinvest in internal tools, and if you ask them why they don't use external stuff they tell you it's because "it doesn't scale". We get things like Phabricator and HHVM out of it, but they could've just not written those, and I'm pretty sure their business would be fine.
They do it because it gets them promoted, and because FB and Google rely on hiring every engineer ever and giving them make-work so they won't leave and start a competitor.
Interesting! This isn't how I remember things at all. Do you know where you got this sense of things from, specifically?
In particular, do you remember what gave you the impression that Facebook "overinvested" a large amount of effort into Phabricator, that I developed and open sourced Phabricator primarily to get promoted, that I built Phabricator because of scaling concerns, or that the primary value I provided to Facebook during my employment there was just in not starting a competitor?
Some larger video game production companies have started to do a lot more on their internal tooling, as it has a huge impact on their content production.
Most of them learned the hard way though. And some learned the hard way and still have awful tooling despite some efforts to fix them (Bungie...).
The company i work for is not big or consumer-facing, so this is not an answer to your question.
But we do have a dynamic that i find interesting. The company is organised into many small business units - some very small, fewer than ten people. Each is accountable for its own profit and loss. They hire their own programmers to build the actual line-of-business software they need. But there are also internal tools and systems, shared across units or used by their developers, that are built by programmers hired directly by the top level of the company - their reporting line goes straight up to the CIO and CEO, rather than to a business unit manager. So, in a way, our company does the exact opposite of deprioritizing internal tools and systems! The programmers who develop the internal tools and systems are the elite!
I expect those tech giants which offer B2B solutions to have better situations. e.g. Google, Amazon, Facebook, Microsoft, etc. Many of their internal products eventually have made the positions in their cloud portfolios.
Googler here, but opinion is my own. Google's problem is that we have too many powerful internal engineering tools, many of which overlap in functionality, and so often its hard to know at first which one to use to solve a problem.
Thankfully, there is nearly always an overlap period when one solution gets deprecated and the replacement is launched.
The COVID19 office shutdown has really focused attention on our internal IT tools (most of which are also offered externally). So far, they've worked pretty great, in no small part because things like BeyondCorp [1] were designed from the start for distributed workforces in zero-trust security environments.
It wasn't always like this though. Corp-Eng, as it's called at Google was for a long time seen as a dead end career-wise - disconnected from Google's marquee products. However, a number of developments, including the rise of the enterprise and cloud computing businesses, have bolstered its internal importance, and with that have brought excitement and talent to the organization. It's now an important source of product feature ideas that eventually make their way to customers.
That said, a lot of things not core to Google's problem spaces do get solved with outside vendors' software, and some of that software is great. For example, I just used a site licensed version of the Chrome ScreenCastify [2] extension to make a video demonstrating a feature I'm developing. It worked perfectly.
I've heard Google internal tools are very hit and miss. The engineering stuff is top-notch, but things like their applicant tracking system is archaic (or was a few years ago).
On average, a Google internal tool is better than a commercially equivalent tool elsewhere, especially if it's developer critical.
For example the code search, testing, code review tools are amazing and better than any commercial equivalent.
Other tools like the bug database are on par with the industry.
Others like the food, directory and interviewing tools are still pretty damn good and even if they are lacking some polish, are often far more functional than the average commercial equivalent.
And of course they are hit and miss - because there are so many of them. The ones that are commonly used are well funded and have lots of people working on them.
I no longer work at Google but still consider it a role model for how companies should take tooling seriously.
Google's internal corp tools designed for majority of its employees are pretty nice. Tools for Chromebook sales analysts might not be as great though, but it also doesn't make sense to put tons of investments into a tool used by just tens of sales people. (Of course, there's a number of general marketing tools for thousands of sales)
