

The Code Is Just the Symptom - gvb
https://medium.com/@rubyghetto/the-code-is-just-the-symptom-c77f43b29320

======
grandalf
It is amazing how culture problems often start with the CEO. Sometimes, non-
technical CEOs are very good at their own speciality (selling, motivating,
etc.) but extremely bad at systems thinking.

Other times, CEOs hire yes-men and yes-women below them and everyone is
afraid/unwilling to tell the CEO the truth.

Other times the company has raised a ton of money and attracts people who
would otherwise not work at a startup and whose major focus is resume building
and moving to the next job.

If the CEO is the only person meeting with investors or only trusts one or two
other people to meet with them that is another big concern. It means there is
likely no transparency into the actual team and what is actually going on,
which means the CEO is likely BSing the investors.

All of these are reasons why you should not work at a startup if you do not
100% believe in the CEO. Be skeptical b/c most startup CEOs really do not have
a track record of insight and integrity. If the CEO is aloof or seems out of
touch and not someone who you think would really enjoy talking to you about
your job and who respects the work you do, be wary.

~~~
monk_e_boy
Replace CEO with "owner of small business" and this is what I see endlessly.
The chops needed to build a business and sell a product are a long way along a
spectrum from how to grow that business and how to keep the codebase stable.

~~~
jnbiche
Agreed. Amazingly, some of them are even "technical", but have no concept of
technical debt, and zero respect for software engineering practices that don't
lead directly to new features. Either they never understood these things in
the first place (since this is usually not taught as part of a degree in CS or
EE) or they've long since forgotten them.

~~~
derefr
You can be "technical" without a need for a concept of technical debt.

\- Indie video game designer/developers, for example, live in a world where
everything they write is a one-off product that will ship, at most, a v1.0.4
six years after v1.0.0. To them, code is a medium for crafting an experience,
and whether the code is of quality is immaterial as long as it creates the
correct experience.

\- Data scientists (and bioinformatics pepole, and a few other similar groups)
live in a world where everything they write is effectively a script-as-
experimental-apparatus. If the script is producing output conforming to the
proposed experimental method, then it's done. They run it, then discard it.
It's not usually even under version control; only recently have such scripts
been considered worth including for purposes of peer review and replication.

And there are other examples. I'm not saying that these kind of people are
"software engineers"—engineering of any kind is never even a consideration for
them; whether the product is well-engineered has no impact on their
livelihoods.

But these people frequently _are_ lumped into the category of "programmers",
and that tends to confuse this sort of discussion a lot. The "software
industry" doesn't just contain engineers doing engineering (well or poorly);
it also contains prototypers and illusionists and tinkers and scientists, all
working to their own criteria of art, none of whom will have any reason to
pick up a single shred of engineering knowledge, even with decades spent in
their own industries. And then these people apply for "programming jobs", and
we wonder why they don't know what git is or how to properly modularize a
codebase for maintainability.

It's a communication problem, I think. We need to stop calling everything
we're doing "programming." Maybe we need an entirely new verb for doing
software engineering, of the kind you do when you're writing something like
car firmware, or Amazon S3, or the Googlebot. "Programming" is about as
precise in describing that as "doing math" is in describing the invention of
the lambda calculus.

~~~
nostrademons
I've toyed with the idea of having separate titles & job descriptions for
"Software Features Engineer" (those responsible for pumping out user-visible
features) and "Software Infrastructure Engineer" (those responsible for the
long-term health of the code). In my experience, they really are completely
different roles, down to the languages used (features engineers tend to use
Python, Ruby, JS, and other dynamic languages, while infrastructure engineers
tend to use C++ or Java), attitudes, personality types (infrastructure
engineers tend to be much more attentive to detail), priorities, and points in
the lifespan of the product when they are most useful. There's already
precedent for creating a new job title to solve an organizational problem, eg.
Site Reliability Engineer.

Many big companies have such a split in their org chart, with separate
"infrastructure" and "features" teams reporting up to different managers, but
IMHO that's the wrong split. Very frequently you want a cross-functional team
with both infrastructure & features engineers on it, but they remain different
_roles_ , just like you might embed a UX designer and UX researcher on a
product team even though their job descriptions are very different.

~~~
tajen
Sir nostrademons,

"Software Features Engineer" vs "Software Infrastructure Engineer" is an
excellent differenciation. Actually I'm an SFE and I have always felt that I
am not a real programmer. SFE will always write more-shiny less-durable
features, more often working under an unreasonable management. I feel like I'm
tainting the reputation of real SIEs.

Question is: I've always assumed I _had_ to evolve into an SIE or die (or
become a Product Manager if I'm lucky to be better than 90% of my colleagues
at understanding users). Is it a correct picture of the reality, or is it
possible to strive as a "good" SFE?

~~~
nostrademons
No, but it will take you down a career path that exposes you to more luck &
uncertainty than an SIE.

Features engineers _can_ have lucrative, long-lived careers. It usually means
they need to be the primary or lead engineer responsible for a significant
user-visible product. Think of Andy Hertzfeld for the Macintosh, John Carmack
for Doom/Quake, Anders Hejlsberg for Turbo Pascal and Delphi, or Aston Motes
at Dropbox. Some of them cashed out directly with F-U money from a startup;
others used past track records to get a startup of their own funded or a
lucrative job at a big company.

The downside is that you're very exposed to the risk of your project failing
and nobody caring about it. At the stage of a project where a SIE is most
useful, the project is already underway, people know it's a good idea, and
they just need to execute solidly. At the stage where a SFE is useful, the
whole point of the project is unknown - that's why you need an engineer that
specializes in rapidly testing things out. This applies in big companies as
well - my role at Google could best be described as a SFE (though they don't
have the title), and I had to prototype a number of features, a couple of
which worked out and a good many of which did not.

There's very significant crossover between the SFE role and that of a
technical cofounder - the main differences are risk tolerance, willingness to
go out and interact with users personally, and product vision. It's very
common for people who enjoy the prototyping aspect to end up starting their
own companies, as that gives the best alignment of incentives and ability to
capitalize on their own work. It's still possible to be a "professional early-
employee" or even make a prototyping/R&D role work in a big company, but you
need to demonstrate wins in your career.

~~~
tajen
Indeed, I have created my own company ;) It feels like a failure compared to
the steady money that my SIE ex-coworkers have, but I enjoy it much more. Your
insights give me good context to visualize my career, thank you.

------
vinceguidry
Our CEO hired a VP of e-commerce last year, before there was just an over-
worked director handing me down directives from on high. As the sole back-end
engineer inheriting an ancient codebase, I've been able to keep things
reasonable and clean and not build new crap on top of old crap. But it's taken
the careful exercise and maintenance of leverage.

Now we're gaining traction and pushing back on the on-high directives and
coming up with a solid long-term plan.

I shudder to think what it would have looked like if there were a whole team
of engineers with nobody to add their voice to the leadership.

Engineers don't help this problem when they treat management like idiot work.

~~~
protonfish
Engineers don't help the problem when they treat management like glorious god-
like perfect beings either.

When your management is bad, there is nothing an engineer can do. They can
attempt to bring their concerns to leadership, look for another job, or play
the fiddle while the product burns to the ground.

Speaking of bad management, I was very surprised when he claimed that the
situation where you are asked to "...make architectural decisions on the fly
without knowing anything about what their problems are" he had never
experienced before. In all of my experience, it was standard practice.

~~~
vinceguidry
> When your management is bad, there is nothing an engineer can do. They can
> attempt to bring their concerns to leadership, look for another job, or play
> the fiddle while the product burns to the ground.

Well, they could step up and start fixing problems in the organization
themselves. Improving communication flow between departments, building better
processes, discovering metrics, these are things that you can do to help make
your company better without having to get the leadership involved. Not every
improvement needs grand strategy, there's often lots of low-hanging fruit that
one person can own and push by themselves.

Just Friday I was sitting in on a planning meeting and realizing that while we
have a front-end guy, we don't really have a decent web designer that can draw
out ideas and come up with plans for implementing them. So now I'm thinking
about how to go about the task of managing that information flow.

~~~
x0rg
Unfortunately, it's not so easy. If the culture (i.e. tech is not really
important, business comes first) is wrong, your effort is not gonna fix a
thing.

~~~
unaligned
This applies to more than just start-ups. I work for a company that regularly
claims 'we are not a software company'. But... we're writing software. So yes,
we are a software company.

~~~
bpyne
It's amazing how often the "we're not a software company" phrase is bandied
about. My employer has been around for a while, the IT department has over a
30 year history. Yet, we have the phrase repeated to us whenever we discuss
using source control, developing libraries, having a web services strategy,
etc. It is like saying, "We're only a youth hockey team, not a professional
one, therefore we don't need to use the same equipment and strategies as the
pros."

Just because you're not planning to be [put your favorite professional
software company here] doesn't mean you should avoid learning their strategies
for dealing with everyday problems associated with writing software - unless
you don't write software at all.

In the end, the phrase is nothing more than an excuse for avoiding the risk of
cultural change.

~~~
Consultant32452
If you're working at a software shop that doesn't even use source control I
suggest you get out of there ASAP. It's one thing to be missing core tools and
methodologies when you're applying for your second job 1-2 years in, but if
you hit the 5 year mark and have zero familiarity with source control or web
services (for industries where this is valuable), etc. you're going to get
passed over for interviews. Staying there may be causing irreparable damage to
your career.

~~~
mooreds
Or, at the very least, spend some time outside work on a project that uses
modern software practices.

Things like the Joel Test (over a decade old)
[http://www.joelonsoftware.com/articles/fog0000000043.html](http://www.joelonsoftware.com/articles/fog0000000043.html)
and the 12 Factor App (over three years old)
[http://12factor.net/](http://12factor.net/) are good places to start.

------
mosburger
This was a very interesting read, but I was a little disappointed that it
didn't leave me with a sense of resolution. Did company B get "fixed?" How?
How have other companies resolved culture issues that stem from the leadership
at the top? Are companies with these kinds of problems just doomed to fail?

~~~
fromtheoutside
Usually companies like this can fail for a very long time. As long as someone
is willing to pay, you can live with the slow development speed, the high
number of bugs and the constant team changes.

~~~
monk_e_boy
_this_ exactly. I expect they are paying below average wages, have huge churn
of people looking for anything to do while the perfect job arises. Throw in
some people new to the area who need a job now, some old contractors who have
money and are looking for pocket money and more stability (who's CVs are
amazing, they have the gift of the gab but get bored really really quickly and
just don't care about the job) and that is a the instant recipe for churn and
crap code.

------
calinet6
Really great article.

This is what Deming (old school father of the concept of quality control) was
getting at with his view of organizations.
[http://en.wikipedia.org/wiki/W._Edwards_Deming](http://en.wikipedia.org/wiki/W._Edwards_Deming)

This topic is traditionally covered under the umbrella of Quality, with the
premise that output Quality is primarily a function of the systems, knowledge,
and understanding of the people at an organizational level—and is _not_ a
primarily technical pursuit, despite what most people think.

We need this discussion in the software world now more than ever. Really glad
to see it.

------
bkeroack
This is what happens when engineers subordinate themselves to non-technical
management in a business-driven (as opposed to engineering-driven)
organization.

The ironic/tragic thing is that the consultants who are brought in at much
expense (and who therefore have some credibility with "the business") are
often no better than--usually worse than--the in-house engineers they are
supposed to help. If you ever find yourself an engineer in a dynamic like
this, you should interpret it as a sign of mistrust--even outright disrespect
--of you and your peers by your bosses. Imagine if you were a physician and
hospital management thought it necessary to bring in outside physicians to
help you with your patients. Not many doctors I know would put up with that,
but engineers are culturally so submissive that they go along with it and deal
with the Dilbert scenario of the $500/hour consultant giving the same
recommendations the in-house engineers have been saying all along (which the
consultants get credit for, of course).

If you're ever in a position of authority and feel that it's necessary to
bring in consultants to "fix" your codebase, it would be worthwhile to reflect
for a few moments on how this situation came to be. Either you failed your
organization on the hiring front and have terrible engineers (unlikely), or
your otherwise talented engineers cannot do their jobs effectively due to the
environment you've created. Perhaps you've forced them to incur so much
technical debt by nonstop "sprints", unrealistic deadlines and multiple pivots
that any forward progress is all but impossible. Consultants probably won't
fix this.

~~~
tieTYT
> Either you failed your organization on the hiring front and have terrible
> engineers (unlikely)

What makes you say that is unlikely?

~~~
bkeroack
If you accept that engineering skill fits the normal distribution, it's less
likely that they are "terrible" than merely average.

Amazing 10X engineers could still pull off shipping a working product even
with all the organizational handicaps I mentioned in my earlier post. This
doesn't excuse the deficiencies of the organization, nor does it mean that
"average" engineers are undesirable or can't do good work. I agree with the
theme of the cited article--rather than scapegoat the engineers and pay
external consultants to "fix their mistakes", fix the organization and enable
the engineers to do quality work.

~~~
s73v3r
"Amazing 10X engineers could still pull off shipping a working product even
with all the organizational handicaps I mentioned in my earlier post."

They could. They're also good enough that they could easily say, "Fuck that
shit" and go work somewhere that doesn't have that toxicity. And unless you
have a significant ownership stake in the company, I don't see why you
wouldn't do just that. Life is too short to work shitty jobs if you have the
option not to.

------
beat
There must be a corollary to Conway's Law here... the dysfunction of the code
produced by an organization reflects the dysfunction of the communication
structures of that organization.

~~~
Confusion
Yeah, I think that's an interedting amendment to Conway's law: it's not just
about the structure of the organisation/communication, but also about the
feelings associated with the communication.

~~~
beat
It's kind of a consequence of Conway's Law. I don't know if that's best
expressed as a corollary or as something else.

------
andrewvc
It's amazing that we've known for decades that concentrating decision making
at the top leads to outcomes like this.

It's far better to push decisions as low as possible, and let higher ups
coordinate those decisions, and only overrule when absolutely necessary.

~~~
databass
In the case of an architecture diagram being handed down from on high, sure.
But in the case of product feature whack-a-mole, this doesn't seem like
something that bottom-up decision-making could fix.

~~~
andrewvc
And that's where the "as possible" part of "as low as possible" comes in. :)

------
Terr_
When I was younger, immersed in usability, databases, and big sites, watching
the initial rollout of Facebook etc, I thought that a good _technical_
solution could solve social ones.

Now I'm a lot more pessimistic and want to approach it the other way around.

~~~
ayuvar
One of the common themes at my last gig is that I would start conversations
and presentations with "There are no technical problems here. These are people
problems."

Eventually they tried to solve both problems by adopting a technical solution
so bad that the people left.

~~~
mooreds
Peopleware is a great book about this very concept.
[http://en.wikipedia.org/wiki/Peopleware:_Productive_Projects...](http://en.wikipedia.org/wiki/Peopleware:_Productive_Projects_and_Teams)

A book from 1987!

Nothing new under the sun.

------
jalopy
Love this article, but I'm not sure the origin of the sloppy practices
described there always originates with the VP/Eng, CTO or CEO. I've worked
with plenty of engineers who just don't want to write tests, or tighten up
code, or expend the effort to socialize their ideas/accept others. The higher
ups might not have the visibility, inclination (or time!) to focus on those
issues and encourage best practices, but I don't always detect acts of
commission from them to subvert best practices.

~~~
badlucklottery
I don't know if he was saying it always originates from the top but more that
it's always allowed to perpetuate from there.

If management doesn't have the visibility/inclination/time to peer into
problems and gain understanding, they are encouraging whatever has happened to
continue happening for better or worse.

------
metaphorm
I thought this was a good read and I'd be interested in reading a longer piece
on the topic. Hopefully the author will continue writing on this theme.

------
M8
Ruby in a large app and also untested. A recipe for disaster.

~~~
jalopy
Not sure how Ruby the language is unique here. Doesn't the rest of the
statement apply when you substitute any other language?

