
On Software Tooling - jugjug
https://www.marcel.is/sw-tooling/
======
safgasCVS
Earlier this morning I spent 45 minutes in a meeting discussing the
replacement for a $4 million big data system we bought 2 years ago because no
one was using it. The solution discussed wasnt to ask the analysts why no one
was using it but to spend $20k on POCs for two competing systems each costing
a further $5mil as their on-paper specs are marginally superior.

DAMMIT THEY WILL BE MADE TO READ THIS LINK

------
hangtwenty
My head is nodding at the intention here, but it's important to remember this
reactionary attitude can be damaging as well. That is missing from the post.

Specifically, reactionary conservatism about tools and services can be a
competitive disadvantage.

In several workplaces I have seen this attitude co-mingle with pride in
original code, and result in a counter-productive NIH (Not Invented Here)[0]
attitude (reinventing wheels). Macho/heroism makes it worse.

While trend-following can be inefficient, if the engineers are both excited
and humble and ready to weigh many trade-offs, this inefficiency might be
"cheaper" than reinventing wheels for all problems.

Cue platitude: it takes balance :)

[0]:
[https://en.wikipedia.org/wiki/Not_invented_here](https://en.wikipedia.org/wiki/Not_invented_here)

\---------------------------------------------------------

EDIT:

To clarify I'm talking especially about _uninteresting problems,_ for which
there may be an obvious off-the-shelf solution, such as when I've seen
engineers invent a build system from scratch (and then cost _lots_ over years
as that custom thing needs maintenance) ... when their needs could have been
met by the language's standard tooling and using its plugin architecture.

Or, when "just use sticky notes and a whiteboard" gets so stubborn that a
growing team stays in chaos for lack of organization.

On the other hand, it is quite bad when trend-following causes not just churn
(switching tools), but overkill that introduces tons of cost and risk. i.e.
distributed systems where you didn't need one. Relevant on this point: You Are
Not Google by Ozan Onay [https://blog.bradfieldcs.com/you-are-not-
google-84912cf44afb](https://blog.bradfieldcs.com/you-are-not-
google-84912cf44afb)

~~~
nine_k
The point of the text is that a need, and an understanding _precede_ the use
of any tools. The use of a tool without a need or an understanding does not
help at best, and is harmful pretty often.

You do not become a photographer by buying a lot of gear. You train your eye,
study, and maybe buy gear eventually when you know why you need it.

You do not become a software developer buy buying a bunch of supercomputers.
You study, you start with smallest things, and maybe you buy hardware
eventually, or deploy a lot of cloud instances, when you know why you need it.

You do not become a successful company by quickly raising tons of capital. You
study, learn about the markets and the needs of your prospective customers,
start with building rather simple things, and maybe eventually you take hefty
investments, when you know why you need it.

You can go on and on. The old adage is to use the right tool for the job. This
begins with understanding the job, and understanding the need of tools. Then
you can choose the tools. Starting with tools, especially expensive tools, and
believing that the use of these tools will automatically help achieve success,
is a mistake.

------
arnon
A nice poem, and it may be true in some cases, but not a universal truth.

When you're a telecom, and you have more than 40 million subscribers, a
whiteboard may be a bit limiting.

~~~
voidhorse
Good point. The piece is sound in the abstract and is good advice to recall
when you consider the essence of your endeavors, but tooling does count when
you consider the material situation—some approaches become untenable at
certain magnitudes. Funily enough, the best we can do is often trade a big
problem for a smaller one—« how do we serve X million people » becomes « how
do we teach x thousand employees to use the tool that enables us to serve X
million people «

------
teilo
Process first. Then Automation.

Put another way: Commitment and accountability to a process must precede the
tool that implements said process.

~~~
petermcneeley
Perhaps your experience is different but to me the emphasis on process is a
focus on symptoms.

What I have seen is process is the attempted cure to bad employees or
incoherent culture.

Good employees just do the right things.

~~~
Jtsummers
The thing is, every organization has processes. Most just don't write them
down (or what they wrote down isn't what they do).

When someone checks code into your master branch, do you let anyone do it? Do
you do peer reviews (of some sort)? Do you first run tests? Those are your
process. If you don't record them (via documentation or automation) then
they're learned by word-of-mouth and can easily be miscommunicated.

Do you have a process (and this is more critical for larger organizations) for
purchasing? A lot of tracking here has to do with legal and regulatory
compliance, not just tracking for the sake of tracking (at least, for the
company, the things being complied with may be worthless to you). Is this
written down anywhere? Is it automated in some fashion? Who would you contact
if you need to buy a new <something> for your office? What if you need more
office space, how would you go about getting it? How does your office track
reimbursement for work travel?

~~~
petermcneeley
I agree that there there is an informal process that we call the culture. The
smaller the org the less explicit process required. I dont know about
"purchasing" but I do know software so let me address that.

The good SEs that I have worked with knew when to get reviews and when to run
what tests. The good SEs tried their best not to harm others and knew when
they were exceeding their bounds of authority. This was communicated via
cultural transfer and mentorship.

The reason process tends to be bad is that tends to be cumbersome, inhuman,
and ridged. "The first value in the Agile Manifesto is “Individuals and
interactions over processes and tools.""

~~~
Jtsummers
There's process and Process. Big-P process is broken, it's too rigid and turns
into quicksand. Or it's not what you do.

But processes happen everywhere, and even good developers aren't going to be
uniform in what they do or have experienced. This is why I still find myself
teaching 20-year programming veterans about make or *nix, because they've
never used it in the past. With many of our more recent projects shifting to
git (versus SVN or TFS for our historical projects), we've also introduced new
workflows that people aren't used to. We can either have dozens of
inconistently styled branches on the main repo, or we can establish a
protocol/workflow. Once it's established we have to communicate it. The choice
is: by "culture", which is loose, ad hoc, and often ineffective as people move
in and out of teams; by process, document what that workflow is.

The follow up to that: If that process becomes Process (seemingly fixed for
all time), you've lost. You should reexamine your processes as time goes on to
make sure they're still correct, or that they actually accomplish what you
wanted. Maybe a part of the workflow was to deal with a particular
technological limitation that's since been removed, then change that workflow.
But letting it go by word-of-mouth is not going to scale beyond small teams
and small projects.

~~~
petermcneeley
I totally agree about process vs Process. And I dont know who would every
really be to opposed to process as guides or how tos. I am a huge fan of self-
process and having my own coherent Confluence pages on how to do things and
setups and FAQs.

The only thing I would say is that the word of mouth can work quite well still
even in large spaces due to hierarchy. If someone on your team doesnt know how
to setup a repo and there is no process It is likely the leads responsibility
to convey this information.

------
mateus1
I like this. I've gone really far with just Google Query or a simple SQL
query. Another idea I like is not to "fall in love" with your analysis, per
se. I've seen plenty of extrapolations and wonky charts that end up just being
a waste of time.

------
nlawalker
From a different perspective:

 _Important_ companies don't do data science with pen and paper.

 _Important_ data-science teams don't work on whiteboards.

 _Important_ studies aren't presented in notepad.

What counts is _perception_ , the focus on image, and the courage to spend big
to prove value.

~~~
s3m4j
It's the same as "resume driven development" but at enterprise scale ?

------
alangpierce
I think maybe there's some nuance here to how to think about tooling. A data
science team without any sort of data warehouse is probably going to be
ineffective.

Maybe one way to think about it: some tools try to model and streamline the
things you're already doing, and some tools expand the set of things you're
able to do. I think the first type is the one to be more skeptical of. Face-
to-face conversations, whiteboards, and Excel sheets are all extremely
flexible and powerful. It's possible for tools to improve on these, but it's a
high bar.

------
marcinzm
I agree in principle but I've first hand seen what happens when a company that
has grown doesn't get the tools it needs.

A ticketing system, for example, is how you create a unified view of what
matters and what should be worked on. Otherwise it becomes a game of politics
and social cunning down to the IC level which is not how I want to spend 8
hours of my day.

------
DesHacker
Pragmatism v/s Idealism. We live in the real world. Your 'picture-perfect life
on FB' is not perfect at all.

