
Engineering management at Facebook - sk_0919
http://algeri-wong.com/yishan/engineering-management.html
======
unoti
His thoughts on internal tools and support are very insightful. It's why I am
typically sad when a company outsources its support.

When a company does its own support with internal personnel, it has a vested
interest in making the customers happy and also serving them as efficiently as
possible.

When a company outsources its technical support, there is no real incentive
for the outsource partner to let the company know how the processes and tools
can be improved and automated. In fact, quite the opposite. If doubling the
business means double the support costs, that works out great for the
outsourcing partner.

Perhaps it's still possible to outsource the support work, but still care
deeply about the details of how that work is done, and how the process can be
improved/automated with systems. I haven't seen it work that way, however.

------
wooster
I spent a few years at Apple writing internal tools.

IMHO, this is the way to have an effect on a company completely
disproportionate to any other activity I know of. The tools I wrote at Apple
have enabled projects which, AFAIK, are completely unheard-of at any other
tech company.

That said, don't expect a payoff proportionate to the effect of the tool.

From a company owner's perspective, however, excellent tools can provide an
advantage that is difficult for competitors to match. That's worth an awful
lot.

~~~
leftnode
Yup, I've done this as well at current and past jobs. I'll usually just invent
a new project, it gets the boss's attention and they like it, so I get to work
on that rather than the normal boring stuff.

It's great to invent your own projects at work.

------
randfish
I like much of the advice, but the style and presentation of the message is,
like so many things I've seen associated with Facebook, lacking any sense of
humility. There's no "we did this because it worked for us and we're sharing
in hopes that it may help others." Instead it's "Other people think this. They
are wrong." Or "A commonly held belief is X. It is false."

This hubris has certainly been a powerful ally to Facebook's founder, but like
so many other powerful people, companies, governments and organizations that
came before them, I can't help but think it will ultimately lead to demise
(unless tempered).

~~~
Nitramp
While hubris can be a problem, I'm very much a fan of stating your opinion
clear and bold. One might be wrong, but astute readers should be able to judge
ideas for themselves. Speaking clearly will also make it much easier for
people to see if you're wrong.

I think giving clear statements of your opinion and being proven wrong is much
better than obscuring your point with "maybe" and "personally" and "IMHO".
That's depending on the context of course, as you don't want to unnecessarily
offend people (which is bad in general, but also counterproductive). But in a
general analysis like this, not attacking particular people's shortcomings? Go
for it.

~~~
joelhaus
A reminder to "think for yourself" can be helpful, and does not necessarily
imply a lack of clarity or boldness. Sometimes a story is told so well, that
you get emotionally wrapped up and forget to take a step back and make a fair-
minded analysis. Not everyone is an _astute reader_.

You also included some conditional statements with, " _depending on the
context_ " and " _But in a general analysis_ ". I see no problem with
acknowledging that things aren't always black & white.

~~~
derefr
> Not everyone is an _astute reader_.

I have to single this out: must we all prepare our rhetoric for the remedial
education of those who were never made self-sufficient in the processing of
external knowledge into internal beliefs?

~~~
joelhaus
No, but reading complaints when others do is just dull.

------
Isamu
So the problem of "hiring the best" is solved by making hiring your top
priority. Likewise the problems of development are solved by making tools your
top priority. Presumably along with everything else that is a top priority,
like making something insanely great.

I'm sorry, I just sensitive to how many times I've seen "top priority" in
somebody's management presentation, as if it solves something.

That said, I mostly agree with the gist of what he's saying here.

~~~
eitally
I know what you mean, but you may be surprised to know how many people feel
like their organization doesn't have _any_ top priorities, so overkill on the
hyperbole isn't necessarily a bad thing. One of the issues in large
organizations is trying to engage staff strongly enough so they really do feel
a personal responsibility for the success of the company. It's hard, and very
few large corporations do it well.

~~~
Isamu
I agree this is a real problem. I guess I've come from organizations where
everything is declared a top priority, therefore nothing is. And nothing can
be pushed aside for a true priority.

In particular, it is damaging for a leader to declare a top priority and then
go on the same as usual, not making extra efforts to invest in that priority,
because of course the reality is that so many things are important, and they
lack the will to focus at the expense of something else.

If something is that important, you'll direct significant resources to the
goal and I imagine the author of the article is sincere in doing that. Many
organizations however are resource-constrained and must make tough choices.
It's just mismanagement to declare lots of top priorities - heck, you don't
have to say anything, just show us how you are allocating the money and
people, and adjusting your expectations of everybody's schedules and
deadlines.

~~~
eitally
I read this with my head nodding the whole time. :(

------
sdizdar
With all due respect to Facebook and many great engineers at Facebook, if you
look at quality of Facebook API ( documentation, bugs, reliability,
compatibility), I don't think they followed "hiring the best" in the division
working on Facebook API.

~~~
dasil003
I disagree. The fact is that Facebook does not optimize for documentation,
reliability or compatibility. The reason is because their goal is to be as
agile as possible, and iterate extremely quickly. It's true that this makes
life terrible for developers, but that's not a problem because Facebook does
not have a shortage of developers. Over the past 3.5 years the Facebook
platform has evolved so quickly that a lot of bugs simply disappeared by
deprecation; if they had spent more time on solid architecture they would have
left a lot on the table business-wise.

Also, I'm not sure if you used the Facebook API when it was new, but I
remember in 2007 being distinctly blown away by how functional it was and how
far ahead of the curve they were on web APIs. You can argue that "the best"
engineers wouldn't churn out something so ephemeral, but that's subjective.

------
danielharan
"You will begin to get the (objectively) best candidates"

If "it's [everyone's] job to say no-hire" when they're "not sure" about a
candidate, I'd like to know what is done to avoid systemic bias in hiring
decisions.

Anyone here from FB able to comment? How diverse is the work force, especially
compared to applicants?

~~~
tgflynn
One of the things that strikes me about what I've heard about hiring practices
is that they all seem primarily focussed on keeping the wrong people out
rather than on finding the right people.

Does anyone consider the possibility that there might be really capable people
who don't fit the model that N developers have of what a good candidate should
be ?

~~~
aprrrr
Sure there are. The standard rationale is that when hiring, the cost of a
false negative is far lower than the cost of a false positive.

I guess it's also true that the cost of a false negative is unknown and not
felt. It would be a really extreme case to be forced to say, "Hey, remember
that woman we didn't hire two years ago? She went off and founded Company Y
that's now eating our lunch." With a false positive, you feel the pain of
cleaning up their messes until after they're gone.

Finally, sad to say, the majority of job applicants are not competent to do
serious engineering work. A negative bias is right most of the time.

~~~
tgflynn
_Finally, sad to say, the majority of job applicants are not competent to do
serious engineering work._

I've often heard this claim. I have no data for denying it.

However I have a suspicion that when people make this claim the observation
that it is actually based on is that "the majority of job applicants aren't
very good at answering programmer interview questions.". They are hence making
an implicit assumption that not being good at interview questions implies not
being competent to do serious engineering work. I think this assumption is
highly suspect, because of the extreme differences in stress levels, time
scales and general context between interviews and actual work situations.

~~~
aristus
That is a very good point, but what alternative do you suggest? Keep in mind
that hiring can consume a lot of existing employee time.

~~~
silverlake
Strongly encourage applicants to contribute to any open source project.
Reviewing this code will tell you more than any stupid programming question.
This takes less time and can be done asychronously. An interview can be used
to determine if the qualified applicant will fit in your culture.

And if they haven't contributed any open source code, then whack them with
stupid programming questions.

------
dacort
The processes section intrigues. As the "CTO" in a 4-person startup, I'm
constantly juggling between getting $hit done and documenting what the heck I
did to get $hit done. Seems to be a fine balance.

~~~
DavidMcLaughlin
You can say shit here.

------
spitfire
He's wrong. You should not focus on tools, focus on people and ideas. The
people will then make the tools (and throw them away when they cause too much
friction) as needed.

But Facebook is still young, they're still learning. Unfortunately they don't
seem to be learning from the past, which means they get to repeat everyone
else's mistakes.

~~~
jasonwatkinspdx
I think you're both in agreement. I believe the point about tools was not to
enshrine any particular tool, but rather, to create a culture where internal
tools are viewed as being the most important technical contribution.

This is backwards from many technology companies where internal tooling is
often given a lower priority and the work assigned to less skilled engineers.
This is often done with noble intent, like to try to create the best value
possible for customers via improving customer facing features. But I think
that proves a short term strategy rather than focusing on enabling your
organization as a whole. You'd be surprised how many resources become free for
feature work when your tools and infrastructure are so good that you don't
wast effort on what could be automated or prevented.

~~~
yishan
Hi, I'm the author.

This comment is correct. My intention is not to prioritize tools over thinking
about anything else. If you read the essay linked from it, I said "your most
talented engineers should be working on your tools." So when I say "highest
priority," I mean development priority, i.e. you shouldn't have your best
people working on features and your second-tier people doing tool work, like
many organizations do.

~~~
spitfire
You might like to short-circuit a lot of this learning. There was a fighter
pilot who studied this for 40 years named John Boyd. He architected the f15,
16, 18, A-10 and the first iraq invasion. His academic work short circuits
what all the dotcoms (including facebook) are slowly learning through trial
and error.

If you want a starting place Robert corams book on boyd is excellent, while
Frans Osinga's thesis Science, strategy and war is essential reading for the
hard subject matter.

Here's destruction and creation.
[http://www.goalsys.com/books/documents/DESTRUCTION_AND_CREAT...](http://www.goalsys.com/books/documents/DESTRUCTION_AND_CREATION.pdf)

------
Swannie
A number of good responses about writing good tools. That's a no brainer.

It's scary to how many project managers I've had to explain why I have
allocated 20% of my project planning to writing a new tool. They see it as
wasted time because they don't think that we will be redoing the task again...
when it's something sales try and sell with every project?!

The biggest thing that struck me were the strong statements about technical
managers. I think the sentiment is correct, technically experienced managers
are great. But it reads like you expect all technical managers to be up to
date on their coding? Or just be competent at writing pseudo-code? Hopefully
it is the latter!

------
gabaix
impressive insights.

