
Facebook first level interview for Linux admin position - bsdpunk
http://bsdpunk.blogspot.com/2013/09/facebook-first-level-interview-for.html
======
zoober
Dumb questions. Good sys admin interview questions shouldn't ask about
specifics like "what is the command that does x" or anything that can be
easily looked up in a manual. Rather they should quiz depth of knowledge of
the Linux (if that's what its for) ecosystem.

I had some good questions posed to me during a Google DevOps interview, though
I have trouble remembering what specifically they were about, I do remember
that they were of a much more "holistic" and general bent. Instead of asking
me to provide irrelevant details (i.e what are the switches for ls that list
time oldest to newest) that are easily forgotten, they would pose a problem
that seemed to have an easy and obvious solution and then make the problem
more and more esoteric and difficult to solve.

These were the sysadmin trouble shooting problems and I found them quite
interesting because they directly related to my experience and opinion that
sysadmin trouble shooting is unfortunately sometimes quite complex and
involves less an understanding of command syntax and more of a problem solving
mindset.

Another point to take away is that if the interview is about linux, it should
test fundamental linux things linux things such as testing the candidate's
understanding of linux process management and signals. Perhaps filesystems as
well.

My take away point is not that I think the FB interview was overly easy, but
rather that it asks useless questions. This interview isn't looking for the
sysadmin mindset at all.

~~~
ghshephard
This wasn't a FB interview - this was a phone screen before you got to the
interview. Every linux sysadmin under the sun can answer 90% of those
questions from "muscle memory." The purpose of this phone screen was to ensure
that the incredibly expensive time of the interviewers was only used to meet
potential candidates.

There are certain trivia or data that the entire community of linux sysadmins
know, just by having worked with the operating system - and, if you miss more
than about 20% of those questions - one can say with pretty much certainty
that you are not a linux sysadmin.

I thought the questions were roughly the ones I would give to a technical
recruiter to screen.

(PS: ls -lart - muscle memory, I type it 100+ times a day)

~~~
D9u

      >(PS: ls -lart - muscle memory, I type it 100+ times a day)
    

_alias lsl= 'ls -lart'_

Place that in your .bashrc, or whichever shell you may be using.

Reduce typing.

~~~
wiredfool
Then you'll fail some silly interview question.

------
js2
_6\. 3 Timestamps_ nix keeps on every file: created, modified, last access*

ctime is not when the file was created, it's when the inode (the file
metadata) was last changed, as compared to the mtime which is when the file's
data was last changed. Note that changing the file's data also updates the
ctime.

~~~
ciupicri
Just to clarify things a bit more, I'll quote from the stat(2) man page [1]:

    
    
        time_t    st_atime;   /* time of last access */
        time_t    st_mtime;   /* time of last modification */
        time_t    st_ctime;   /* time of last status change */
    
        The field st_atime is changed by file accesses, for example, by
        execve(2), mknod(2), pipe(2), utime(2) and read(2) (of more than zero
        bytes).  Other routines, like mmap(2), may or may not update
        st_atime.
    
        The field st_mtime is changed by file modifications, for example, by
        mknod(2), truncate(2), utime(2) and write(2) (of more than zero
        bytes).  Moreover, st_mtime of a directory is changed by the creation
        or deletion of files in that directory.  The st_mtime field is not
        changed for changes in owner, group, hard link count, or mode.
    
        The field st_ctime is changed by writing or by setting inode
        information (i.e., owner, group, link count, mode, etc.).
    

[1]: [http://man7.org/linux/man-
pages/man2/stat.2.html](http://man7.org/linux/man-pages/man2/stat.2.html)

~~~
forgeman
Kudos on calling out the man page in the spirit of instruction.

------
plg
I guess Facebook employees don't have access to google and so this is why they
want people to have this stuff memorized

;)

~~~
jvehent
When you've been a sysadmin for long enough, you just know that stuff by
heart. This is what they are looking for.

You don't have time to google anything when your system is down.

~~~
earless1
lies, lies, damn lies. you've never witnessed faster googling than when a
system is down. googling is usually much faster than trying to recall the
proper switches for all utilities. I find Google to be far better sometimes
than even man pages. I've only been a sysadmin for 5 or so years so maybe my
views will change

~~~
shuzchen
To me, it seems the questions were pretty basic and geared towards judging if
someone has had experience actually administering linux servers. How many
people (who actually competently deal with linux servers) need to use google
to know what the output of top is? or how to kill a process? If you don't
already know what MTU is about, how would you even consider an incorrect MTU
as being an issue when you're debugging an issue in production?

Likewise, I wouldn't hire a Senior Javascript developer who can't describe in
her own words what a closure is and what it's useful for, or talk about
execution context (what 'this' refers to) and how to control that. You just
can't have seriously written any javascript and not have encountered those
things enough that they've been burned into your mind (though maybe an intern
or a Junior developer that'd be okay)

------
druiid
Well, I have basically all of these correct... been doing this for very many
years now so I'd hope so ;). I do hate these kind of interview questions (even
first round interviews) though. They're not really useful at all in practice.
It's like an old manager of mine said and I still think it's a good statement:
There's knowledge you keep in your head and knowledge you keep on the web.

Essentially, more often than not if I'm doing something shell related I'll
need to turn to man pages or doc files for a specific method/flag to use on a
command unless it is the 'muscle memory' type commands like ls related flags,
etc. There's only so much that is important to have memorized, and only so
much you can memorize. The rest, look to the docs my friend!

------
thwarted
I hope that these questions were from the author's memory and not worded
exactly as typed. Because the questions used for first round filtering of
interview candidates should be clear cut and not open to interpretation.

Namely, question 14 is odd,

    
    
        What Signal does a child send to a parent when it is finished.
    

The child doesn't send signals in this event, the kernel sends signals to the
parent when the child's state changes, and one of those states is the child
exiting. Although, the default action of SIGCHLD is _ignore_ , and can a
signal that is ignored be considered to be sent at all?

~~~
johnpmayer
I'm sure if you just said SIGCHLD you demonstrated the knowledge they were
looking for. If you gave your extended answer, they would probably be
additionally impressed.

~~~
thwarted
It really depends on who is administering the first pass questionnaire and how
they have been prepared. When I was doing hiring, a dedicated recruiting team
was tasked with doing this first round, which we called "prescreen questions".
I spent a lot of time briefing the people talking to candidates for my team on
how to interpret the answers, and I encouraged them to keep detailed notes so
we could audit what candidates were saying and needed to adjust something. I
personally reviewed every prescreen result to make sure we didn't pass on
someone who could provide more than the straight forward answer. However, we
found that prescreens were most effective when the answers were straight
forward and there was only one way to interpret the question (that is, for
prescreens, non-essay answers are preferred).

------
spamizbad
fsck checks the consistency of the file system _and_ the kernel? Asking those
two together seems really weird. If that's how it was presented the best guess
I could make is sync (forces changed blocks to disk) and fsck.

~~~
bizarref00l
My first guess was running package manager checksum verify, like "rpm -V
kernel" , for rpm based distro. But i'm not sure if it was the question was
aiming for.

~~~
gcr
The question, as written, has no answer. fsck only checks the filesystem.
There is no way to check the consistency of the running kernel but a kernel
panic is a telltale sign of INconsistency :)

~~~
shabble
What happens if you try to fsck /proc or /sys?

I presume 'not much' or 'you get an error', but I don't have an instance I'm
prepared to sacrifice right now, just in case something hilariously
destructive happens.

~~~
gcr
fsck doesn't run on a directory, it runs on a block device. You can't run it
on /proc or /sys because those aren't backed by a block device. /proc only
looks like it has a file system because the kernel "pulls" the "files" out of
thin air when you try to access them.

------
dysoco
I'm not a sysadmin (though I run Linux and have worked in the Kernel) and I
can answer 4 or 5 questions out of those.

So my question is, is this all you need to be accepted at Facebook? or "First
level interview" means more difficult stuff comes after this?

~~~
migrantgeek
There's more to come. I've interviewed with Facebook for a similar position.

After a phone screen, they setup another call that was probably more intense.
They asked me to be near a computer with a net connection so I assumed I'd be
writing code or troubleshooting something.

I ended up turning it down because I received an offer from another .com that
seemed more promising but I believe if I passed the next call, I would have
been flown in.

~~~
dysoco
Oh I see, so probably just an interview to filter out candidates.

~~~
jfb
This is standard issue phone screen sort of stuff. You could have a technical
recruiter do this, which is what I would assume happened here.

------
tankenmate
The last question isn't answerable; it's kind of like the question "have you
stopped beating your wife?", the question presupposes a fallacy.

Actually a child almost never sends a signal to the parent process on
termination; the signal is sent by the kernel from the kernel context.

A child could manually send a SGIC(H)LD by using signal(2), but then any
wait(2) would block; this is why you should always use a non-blocking flag to
waitpid(2) or its ilk in case some idiot program sends your program a
SIGC(H)LD.

------
cjaredrun
suddenly i completely understand why you weren't invited back...

i'm taking the fact that this article is on the front page of HN as my sign
that i've outgrown HN.

~~~
jjsz
What better places have you outgrown to?

------
x0n
That's a good first level interview I guess. I'm a .NET programmer and I knew
all but one (but I am also a full stack dev, so it's probably normal that I
would know these things.)

~~~
scrabble
I guess it depends on the stack. I'm also a .NET full stack dev, but my stack
includes Windows servers instead of Linux servers.

It's been 5 or 6 years since I've done Linux sysadmin work, but I still knew a
bunch of these.

------
hrasyid
Didn't they ask you to sign a non-disclosure agreement at these kinds of
interviews?

------
ivanbrussik
ridiculously easy, and I'm probably the least experienced Linux person on HN.

~~~
judk
Presumably they are screening for competence. They don't need Doug McIlroy to
run their servers.

------
graycat
Good illustration of why in some ways it's actually easier to be a solo
founder of a startup than an employee of one!

I know next to nothing about Linux -- it's yet another multi-user, demand
paged, virtual memory operating system with an 'embedded' operating system and
a hierarchical file system, written mostly in C. Right? That's about all or a
little more than I know about it. There have been a lot of those, back to
Multics. So, Linux is yet another one. Okay.

But I'm developing on Windows. Even on Windows I don't know that much detail.
For such questions on Windows, I wouldn't know a good source (wish I did), and
when I need answers to such questions have to Google/Bing a lot, hit Stack
Overflow, etc., one silly little question at a time. Bummer, but the frequency
with which I have to do such scavenger hunting in muddy swamps has been
declining.

No doubt when I bring up Windows Server I'll have a lot more such detailed mud
wrestling to do; similarly for serious work with SQL Server. Bummer -- again
I'll be screaming at the lack of good documentation for such questions.

Still, so far my software development is going ahead well without such
detailed knowledge. Actually I've long suspected that once my company gets
serious with number of users arriving per hour and revenue getting up to the
useful level, I'll get answers to such questions by calling for paid, expert
support at Microsoft. It's their software, not mine! And while most such
questions are simple conceptually, digging out the answers often is not.

Indeed, one reason I selected Windows instead of Linux is the hope, maybe
false, that one way and another, say, from Microsoft or Stack Overflow, books,
Microsoft's MSDN Web site and the similar site for SQL Server, I'll have
better means of answers to such questions. Similarly, when I need a high end
LAN switch, I'll try to pick Cisco heavily due to a hope for good sources,
possibly from Cisco, for answers to such questions. LAN switching is their
darned business, not mine; they can do their business, and I'll be a paying
customer, and I'll do my business.

For hiring, there's a recent remark from Richard Branson that he hires for
personality and not skills since skills can be taught, he believes quickly,
while, implicitly, it appeared he was saying that personality could not be
taught quickly. Actually, for such detailed questions for Linux/Windows
Server, he is significantly correct. So, I'm hoping to hire bright people with
good personalities who are at least okay computer users (even if they have
never written a line of code in their lives) but are good at typing, reading,
writing, speaking, 'critical thinking', getting some okay understanding from
some horribly badly written material (i.e., nearly all computer industry
documentation), and analyzing ambiguous situations. Right: Sounds like I might
want to hire some good humanities majors! Nerd me!

Then have such employees start with simple things, say, plugging together a
server, working up a simple, quick and dirty parts inventory system (maybe
just a flat file with some keywords maintained with just a text editor maybe
with a few macros -- maybe even a, yuck, spreadsheet), pulling CAT 5 cables
and using a crimping tool for the ends, installing Windows Server, calling
Microsoft for technical details, documenting what they learned about how to
install, configure, and administer Windows Server, etc.

Then, teach them programming a few lines at a time, e.g., Visual Basic .NET
(apparently about the same as C# for getting to the CLR and .NET Framework
but, to my eyes, easier to read on the page than C# with its syntax from C
and, thus, deliberately _idiosyncratic_ ).

For basic computer and network architecture, I'll give them a 30 minute
overview lecture they can capture on video and use for other new hires. The
hope is that such people will be able to grow in 'skills' as fast as the
company grows. I.e., I'll show them what I already know, and as we need to
learn new things it's their job to dig them out and document them for others.
Did I mention I want them to know how to write (and _not_ learned in the
computer industry)?

Then for Facebook hiring, I would ask: Whatever the heck happened to the last
person in that position? And, why the heck does Facebook have to go recruiting
for such a person? You mean, Facebook doesn't have a backup for such a person?
Or, Facebook has 400 billion executing instances of Linux, may be compiling
some unique versions of Linux, and, still, has only one Linux system
administrator? Not good.

Broadly, if Facebook needs such 'skills', then they should grow the
capabilities internally. Or, I can do my startup _without_ knowing all that
stuff; if Facebook would want me to know that stuff, then they should pay me
to learn it.

This stuff about having to walk in with a lot of detailed knowledge, learned
on my dime and time, just as needed by Facebook for just an 'employee'
position in a big company, that is, an organization that in a few years might
just decide to layoff 20,000 employees, is getting to be a bad, old story. A
startup, even for grass mowing, literally, looks more promising.

~~~
throwaway1979
"This stuff about having to walk in with a lot of detailed knowledge, learned
on my dime and time, just as needed by Facebook for just an 'employee'
position in a big company, that is, an organization that in a few years might
just decide to layoff 20,000 employees, is getting to be a bad, old story. A
startup, even for grass mowing, literally, looks more promising."

Spot on! Another way of looking at it is that tech companies don't want to
spend $ training their employees (including a slightly longer ramp up time for
an employee to begin contributing). If they did, the hiring process would be:
you had a 4.0 in college. This shows you are good at learning new things. Lets
check for personality fit and some other things that a college transcript does
not capture. Do a one hour interview. Hire or No Hire based on this.

As a startup founder, you should be happy about this state of affairs. The
attitude of large companies makes it easier for you to hire :)

~~~
graycat
I'm getting happy: I'm 70 miles north of Wall Street and that means really
close to Vassar and not far from Princeton, Yale, Harvard, and more. So find
some bright, humanities major coeds who otherwise would have to start in
retail or go to law school or for an MBA and get them started.

So, they have great SAT scores, know how to study and learn, know what high
quality work is, are good at working with people, are really good at _reading
comprehension_ , can knock out an A+/A+ term paper between 10 PM and 7 AM with
just a quart of coffee, can give a good presentation, etc.

So, teach them about Volts, Amps, Watts, Ohm's law, DC, AC, resistors,
capacitors, inductors, basic electronic signals, the basic conceptual
principles of how a processor core works, a little on caching and main memory,
data representation 101, how a disk drive works, the basics of an NIC,
Ethernet, a LAN switch, and TCP/IP, and IP router, a little on memory
protection, an 'embedded' operating system, virtual memory, virtual machines,
a file system with locking and security, console sessions and command lines,
text files and a good text editor, the Internet from DNS, SMTP, POP3, to base
64, MIME, HTTP, HTML, and CSS, the basics of programming languages, e.g.,
Visual Basic .NET and its memory management, classes and instances, the .NET
Framework, relational database, SQL, SQL Server, how to use Microsoft's MSDN,
about Stack Overflow, HN, and how to Google/Bing to get answers to questions,
how to use our technical support account at Microsoft and Cisco, all in about
one afternoon, with some exercises, and then have a graduation party.

Then get to work on some basic stuff, say, plugging together another 20
servers, repairing a server with a busted hard disk, etc.

Clean, indoor work, no heavy lifting, and better than grass mowing or
retailing.

------
paulrademacher
It's rude to post their questions online.

How is this lost on people??

~~~
shmageggy
While I don't really have a problem with being "rude" to a corporation,
especially some evil behemoth like Facebook, it's worth noting that sharing
these questions probably violates an NDA that was signed/agreed to before the
interview.

~~~
e12e
I hope they don't waste everyone's time with having an NDA signed before the
_first_ round of interviews? What would it accomplish?

~~~
pjscott
Its main effect would be to filter out people who aren't desperate.

