
A Checklist for Evaluating New Technology - zdw
https://www.gustavwengel.dk/evaluating-new-technology
======
simonw
If a technology is replacing an existing technology that a company is already
using (Elasticsearch v.s Solr for example) the questions I ask are:

1\. How much work will it be to /completely/ replace old-tech with new-tech?

2\. Does new-tech give us advantages that outweigh that switching cost?

My rule of thumb is that new-tech can't be an incremental improvement on old-
tech - if it's going to have a decent ROI when it comes to switching cost (and
re-skilling cost and so on) it needs to be a HUGE improvement - either a
multiplier of developer productivity or enabling features that simply cannot
be built with old-tech.

The worst result - which I've seen far too many times - is that new-tech only
50-80% replaces old-tech - and now you have to maintain both.

~~~
andrekandre
> The worst result - which I've seen far too many times - is that new-tech
> only 50-80% replaces old-tech - and now you have to maintain both.

i’ve seen this too, (in apps) where there are 3 different “reactive” libraries
in use that all do the same thing smh..

my feeling is, if you are going to change out the technology, then to avoid
said problem of maintaining both, is the change needs to happen very very
quickly to avoid having to maintain the old one while getting side-tracked to
implementing new features and the transition gets sidelined... its very
difficult to pull off

~~~
tonyarkles
Sigh... just yesterday I found code in a project I inherited where an
Observable was returning Futures to the observer. It probably seemed like a
good idea to whoever at the time but damn does it make it hard to follow
what’s going on.

------
Hokusai
> Sometimes there’s a technology that solves a problem really well in your
> particular project, but which is really hard to understand or learn.

I have seen this problem too many times. Some team find a framework/language
combination that makes their lives slightly easier in the short term and want
to go for the new technology. But, they do not think about how we will scale
that knowledge or what will happen if the code is moved to another team.

A couple of times I have been in the situation to have to re-write an
application to the company common language (in one case Java, the other
Javascript) because the receiving teams did not liked the technology and did
not wanted to spend personal time learning something that was not valuable to
them.

When a new framework is adopted, it is a commitment that we will train the
developers and let them experiment, fail and learn until they take hold of the
technology. It takes tame and the decision cannot be taken lightly.

~~~
simonw
One of my favourite observations about Python is that it's the second best
language for basically everything - which makes it a fantastic default for an
organization.

~~~
gowld
For small teams and small projects, sure. But you'd do well to have a settlers
team behind your pioneers, rewriting in something more scalable.

~~~
simonw
Well designed Python scales just fine - ask Instagram or Pinterest.

~~~
shandor
Any good pointers to some of their technical blogs about this?

It's quite astounding that architectures their size are implemented with the
"slowest language around". It would be very interesting to read in-depth how
they manage to sidestep the performance problems (that are brought up almost
always when someone mentions Python).

------
lowdose
1st check: should start at Zapier integrations. No API integration with the
other 600+ SAAS means the technology does not yet scale.

2nd check is there a link on the website for developers.

3th Are they hiring?

~~~
statictype
> No API integration with the other 600+ SAAS means the technology does not
> yet scale

This is an extremely naive take.

I’m curious about how many people actually approach technology evaluations
with this mindset.

------
golover721
A big one for me is “What is the community like?”. Especially for open source
technologies. Do the contributors spend time answering questions? What is the
general tone in the community, welcoming? Confrontational?

All the best technology decisions I have made have come with a strong friendly
community.

------
rooam-dev
No 1 on my list is "how testable is the new technology?".

