
Ask HN: When does it make sense to roll your own X? - li4ick
Suppose you find a library that kinda does what you need, but not really. Do you adapt it to your needs or do you build a version from scratch that really suits your needs, but might take a bit of time to develop?
======
anderspitman
My rule of thumb when considering adding a dependency these days is to start
by trying to implement the functionality myself. But I don't let myself spend
too much time on it (maybe an hour or two. maybe a day). More often than not
I'll get bogged down pretty quickly and realize the problem is more
complicated than I assumed. This approach gives me:

1\. Greater appreciation for the dependency

2\. Better chance of modifying the dependency when it breaks or doesn't work
the way I want, because I'm at least somewhat familiar with the types of
tradeoffs you have to make to implement such a thing.

3\. A chance that the problem turns out to be simple enough to not require a
dependency and the prototype works just fine forever.

4\. Get to learn something new.

Obviously there are exceptions. I wouldn't try to implement a relational
database just because I need one. But in these situations I try to take a step
back and ask if I even need a full DB. Would flat files work fine for the task
at hand?

~~~
bruth
I second this approach which is what I do as well. My baseline is that I
prefer to not have any dependencies. This is usually not realistic nor
necessarily pragmatic in practice (i.e. reinventing a relational database),
but its a good baseline.

My general feeling towards dependencies (in programs or life) is "do I really
_want_ to depend on this thing?" Almost always the answer should be "no". But
the follow-up is "do I really _need_ to depend on this thing?" and sometimes
the answer is yes because I don't want to or can't re-implement the thing due
to time or because I don't have the expertise.

------
CoolGuySteve
When the packages and configuration files for the project are more complicated
than the python code it would take to do the one simple thing that you need. A
lot of frameworks tend to bloat to handle every use case and become overly
complicated because of it.

Things like passing protobufs/gRPC when C structs, JSON, or CSV files will do,
job scheduling systems that could be replaced by a postgres table, most Kafka
applications.

I once worked with a guy who would pattern match whatever problem you had to
the trendy Apache or Google library and then spend weeks/months setting it all
up. He was easily the least productive person I've ever worked with.

~~~
closeparen
>Things like passing protobufs/gRPC when C structs, JSON, or CSV files will do

These will always get the job done, but at great cost to future maintainers
and callers. Explicitly declared interfaces that give some thought to future
evolution aren't that hard, and can save a lot of pain down the road.

I vividly remember the outage I caused by changing a struct field name, only
to find out it was implicitly a critical part of the API "contract" when
serialized to JSON. Another where I relied on a certain field from a
dependency's response, only to find it out it was populated in some situations
(the examples I used to learn the interface) and not others (revealed by
production). Our Thrift services do not have these problems. Anyone can look
at a diff and tell a) that the IDL is changing, b) whether the change is
breaking, and c) whether a field is required or optional.

~~~
pjc50
GRPC is great as an API generator, but there's a _lot_ of machinery under the
hood in its network layer. It's not small - there was an embedded project
where we were considering using it to replace our roll-your-own, then we
discovered that the compiled library for GRPC was larger than the entire
application.

C structs are so brittle they should not be used for any kind of
serialisation. You're just setting yourself up for upgrade nightmares. I wish
BSON or the ASN.1 toolset were easier to use for this use case.

~~~
CoolGuySteve
For C structs, start with a uint32_t 4 char code identifier and a size field.
Then you can extend them as long as you only ever append, which is a
constraint on protobufs anyways. This is how most media containers have worked
since the Amiga days.

Meanwhile, their encoding/decoding is infinitely faster than most other
serialization formats, allowing you to serialize data back and forth to disk
or as IPC faster than anything else.

~~~
pjc50
.. as long as you never change endianness, structure packing, word alignment
requirements, and always use types with explicit sizes. And you have to keep
the data in the structure itself, so as soon as you hit a pointer you need to
write explicit serialisation.

~~~
CoolGuySteve
> .. as long as you never change endianness, structure packing, word alignment
> requirements, and always use types with explicit sizes.

#pragma pack(1) and byteswapping solve all this and neither are more work than
specifying some intermediate format and calling some external tool in your
build system. In C++ you can make types that swap automatically if required.

> And you have to keep the data in the structure itself, so as soon as you hit
> a pointer you need to write explicit serialisation.

I have yet to see a serialization format that does this well and quickly. Just
make a new message type for it that follows the first.

------
gtsteve
Depends on the quality of what currently exists and the license. Typically if
I find something liberally licensed that does 80-90% of what I want, I'll fork
the project, add the remainder and send a pull request back to the parent
project (which is normally ignored).

Other things I take into account are how important the X is to the business.
My company's product parses a very strange industry-specific file format.
There are some commercial parsers and some open-source ones which do most of
what I need but I opted to roll my own as working with this file format is
very important and I can't risk license concerns.

Also, it might not be very nice but I don't want to help my competitors.

~~~
jdmichal
I'll strongly second your comment regarding business importance. However, I
will slightly rephrase it:

By expending resources to build "X", your company is now _in the business of_
building "X". Is building "X" a business the company wants to be in?

The story behind Amazon building AWS for internal needs, and then selling it
externally, is an exemplar of this thought process. They were already in the
AWS business... They just decided to package and sell the product to others.

~~~
o-__-o
only maintain a separate codebase if the feature is accepted, that way there
is a finite timeline on having to be in the build business. As someone who
maintained a forked openstack distribution for 5 years as a company selling
openstack tooling... we wasted significant time tracking down integration bugs
rather than working with the community to integrate our changes upstream

------
caseymarquis
"If it’s a core business function — do it yourself, no matter what." was Joel
Spolsky's take.

He wrote an article on the topic:
[https://www.joelonsoftware.com/2001/10/14/in-defense-of-
not-...](https://www.joelonsoftware.com/2001/10/14/in-defense-of-not-invented-
here-syndrome/)

~~~
alberth
Where do you draw the line though on what’s _“core”_.

Let’s imagine you’re a software shop that sells a SaaS app. Something like
Slack as an example.

Should you roll your own dedicated compute for your SaaS app or use AWS?

Should you roll your own web framework, or use Rails/etc?

Should you use AWS for file storage of Slack media files that are uploaded, or
roll your own dedicated storage solution?

Edit:

I have the upmost respect for Joel. His company created their own Programming
Language because they felt it was so core to their business to do so [1].

As did Facebook did with Hack.

[1]
[https://www.joelonsoftware.com/2006/09/01/wasabi/](https://www.joelonsoftware.com/2006/09/01/wasabi/)

~~~
npo9
This really depends on what the company wants to adopt as “core”. They cannot
be great at everything, so they must be careful in picking what topics to be
experts in.

For example, maybe the Slack like app wants to be able to upload, distribute,
and archive huge computer files. Everyone has tried to upload a file to an
email, slack, or discord message and was told the file was too large. Our
platform is going to be the platform for sharing 1TB sized files for
collaboration purposes, with the goal of supporting 10TB in summer 2020 and
1PB before 2022.

The company should 100% make their own network protocol for transferring
files.

------
franciscop
Ah this is a very difficult, multi-variable question. I think this question
should be given with a lot more context. Of importance, out of the top of my
head, are:

\- Do you need the get results _now_ , or long-term maintenance is more
important?

\- Are you going to roll your own internal tool or publish it as open source?

\- How responsive are the developers of the original library? Did you attempt
to ask them for help/consulting/etc?

\- Can you wrap the library so that you get 80-90% of the work out of the way?
Can you fork it so that you get 50-70% of the work already done? As everyone
mentions here, licensing is important for this question.

\- Do you have people who have created and maintained libraries before? Do you
know what it takes to publish your own library?

For example, I've written hundreds of small libraries and projects in
Javascript, so I know the tools, common issues and details of creating a new
library. So for me, creating a new one is fairly easy. For someone who has
written the same amount of code in their career, but mainly focused on a
single codebase, the cost (both short and long-term) of creating a new library
is a lot higher.

~~~
kd5bjo
I heartily agree with all the concerns here, but there are also business and
professional development factors to consider, in addition to your present
situation:

\- How stable are your needs? Will your ideal setup be the same 6 months from
now?

\- Is this a core piece of the business or a peripheral concern?

\- Which is more valuable: developers that understand the underlying theory,
or developers that are familiar with the preexisting library?

\- Will the act of rolling your own library make your team more valuable? Will
it make them too expensive?

\- How will this choice affect developer retention? Your ability to hire new
developers?

~~~
candiodari
A problem I wonder about is that frameworks and doing things well are often
diametric opposites. For example, studies keep pointing out that for
e-commerce, low latency and ridiculously fast sites matter a lot. They get you
from zero to functional really fast, and then you're stuck.

You're not getting that with django, or ror, or ... And no, caching will not
get you there, especially not if you take every framework's approach to
caching: always set cache headers to never ever cache because it might be
dynamic.

The thing is, if you expect to really grow, you're going to have to roll your
own, as that's the only way you'll ever get it really fast, really doing what
you want.

~~~
kd5bjo
Take two nominally identical physical products (from IKEA, for example), one
that’s on the shelves today and the other produced several years ago. There’ll
be lots of subtle changes that got made as the manufacturer figured out how to
improve the product, by cutting costs or improving reliability.

The frameworks you mention are great prototyping tools but as a community,
we’re missing the knowledge of how to take a proven prototype and continue
improving the quality rather than bolting on questionable new features.

~~~
dragonwriter
> as a community, we’re missing the knowledge of how to take a proven
> prototype and continue improving the quality rather than bolting on
> questionable new features.

No, we aren't. We know _how_ to do that, and can do it when we want to. In
software dev we rarely want to, because (loaded use of the qualifier
“questionable” aside) product teams tend to perceive (not entirely
inaccurately, though sometimes the particulars are in error) market demand for
additional features. (Also, what physical products are very often optimizing
is unit production cost for equivalent products, not quality. But software
already has an essentially zero unit cost, so there's essentially no gains to
be had optimizing that.)

------
gwbas1c
It depends on how critical the X is to your risk.

For example, I lead a project where we wrote a driver instead of using an open
source one. Writing our own manged risk, because the open source ones were
unstable or had data integrity issues. Overall, the "cost" of "writing a
driver" was about 5-10% of the total effort, once we consider integration, QE,
installers, signing...

The thing with 3rd party frameworks is that they aren't built for _your_
requirements. So, if you don't "roll your own X," you need to make sure that
the X you choose will meet your _business critical_ requirements early in a
proof of concept.

Also, if you are going to "roll your own X," it's worth it to do a few POCs
with other Xs. This way you can learn how to make a better design, and what to
encapsulate in your X.

(In my case, the 3rd party drivers encapsulated poorly. Most of my effort
spent writing our driver was things I'd need to figure out anyway, because the
other drivers didn't encapsulate basic details that I didn't care about.)

\---

Other times, it makes sense to "roll your own X" to keep things simple. ORMs
are a great example. Their leaning curve can be higher than simple queries and
boring code. If you only have a few queries, why bother adding something
that's just going to get in the way?

Which gets to: Do not use frameworks in place of design patterns.

------
oneplane
"When it adds value", or so it is said. Define when it value is added and you
get the edge you need to cross when it becomes viable.

Say you are working on a FOSS project to replace something and the one of the
dependencies doesn't work out. If you have the project capacity, it might add
more value to roll a replacement.

Example: say you are writing a metrics collector, and you need something that
can run inside a container because you want to deploy it as a sidecar. A
dependency you use needs root network access for some reason, and you can't
really bypass it using CAP_'s because it's hardcoded to look for UID 0. You
can patch it if the code allows for it, but if it doesn't, you still need a
fix so you might as well roll your own.

On the other hand, say you are in a commercial setting, then value is probably
not added by rolling your own until it's either freeing up resources or
addressing a business need.

Example: you are running a bunch of APIs behind a gateway, and the gateway
exposes those APIs on a subdomain based on their name. But you kinda want them
on the path of a single domain instead. You could do an ugly patch with a
reverse proxy in front of it and rewrite all requests, but at that point you
might as well implement a replacement gateway that does the same thing. While
it might be a slightly bigger effort, it both frees up resources and gets a
business requirement resolved.

There really is no one-size-fits-all rule, even if you take licenses, quality,
maintenance, sourcing, cost etc. into account it still won't cover all cases.
Say you are a small company but your software is used on billions of devices.
Rewriting a part of a kernel would give you a few cycles less CPU time
required for a task that multiplied by billions saves a lot of computational
power, but you, as a small company, don't have the resources to do so. Even if
the license, quality and maintenance is right, you still wouldn't rewrite
something like that.

------
mratzloff
Every time you roll your own, you need to attach a future cost: bug fixes,
documentation, testing, training, and so on. Your future costs can quickly
spiral out of control when you implement your own solutions. Furthermore,
because these are so expensive to maintain, yours will be a bare-bones version
and start to show its age rather quickly. The cost of switching away from
something increases over time, so inertia will incentivize staying with your
minimal, custom solution while other, better solutions will become more full-
featured and robust.

Unless you pay the cost to keep up with the industry (or switch away from your
custom solution once others meet your needs), you will fall behind. But keep
in mind that others are most likely not paying this cost, so they're investing
in features, continuing to outpace you in areas that matter.

In short, only implement something yourself when absolutely necessary—for
example, when it's a core competency, or when no current solution exists. Even
a de facto solution that's suboptimal tends to be better than your own,
because it will be documented, new contributors won't have to train on it, and
it may improve over time. If you have to switch away from it at some point,
there may be tools to automate or simplify that transition.

The smartest decision for things outside your core competency is "do what
everyone else is doing." Focus on innovating on the things that matter
instead.

------
davismwfl
One of the key considerations I look at is license. If the 3rd party library
has a license I can work with, modify and use etc. And you have to consider
distribution too in that conversation. A lot of times this one fact takes away
all doubt on the right way forward. People use Apache licenses on a lot of
projects that it is questionable if not down right improper, and I am kinda
anal about that as for commercial products you don't want someone coming back
and saying you violated the terms/spirit of the license.

Outside of licensing. It is usually based upon the needs of the project, if I
can find a library that does 80% of what I need and I can add the remaining
20% and I am happy to live with that code then I'll definitely use a 3rd
party. In general I favor those libraries, but just so many times the
licensing trips the flag that it isn't worth the risk and adding a few extra
days or even weeks to a project is well worth the time.

One other point, I also am really big on logging/metric collection, if I need
to go in and instrument an entire library then it adds another level of work
so I start thinking about just creating our own. But not all libraries do need
detailed logging and metrics (although IMO most do).

~~~
philipkiely
The Apache 2.0 license very clearly allows commercial use. The sole major
difference between Apache 2.0 and, say, MIT is that Apache 2.0 prevents you
from using the trademark of the entity that created the original software that
you are modifying.

"This License does not grant permission to use the trade names, trademarks,
service marks, or product names of the Licensor, except as required for
reasonable and customary use in describing the origin of the Work and
reproducing the content of the NOTICE file." from
[https://www.apache.org/licenses/LICENSE-2.0](https://www.apache.org/licenses/LICENSE-2.0)

I'm not a lawyer and not your lawyer.

~~~
davismwfl
I typed Apache and was thinking GPL, totally my fault.

Over the years I have sought council and been advised by multiple different
attorney's that for a closed source commercial product to avoid certain
licenses, like GPL and others. Apache in fairness is _not_ one of them, my
bad. MIT and Apache are the two we have used and included in projects in
recent memory.

In the end I am not a lawyer either and my core point still is the same, a
major question for whether to use a 3rd party library still is around what
license the library is released under. And it is usually best to get advice
from an attorney for any of the more complex license models.

~~~
kyruzic
What is a situation where liscensing your software as GPL is "questionable if
not down right improper"?

The entire point of the GPL is to spread the adoption of free as in freedom
software. If you aren't willing to share your project with the community you
are not allowed to use the community's work.

~~~
davismwfl
I agree overall, nothing is wrong or improper with GPL if the product you are
creating also complies with the same license terms. In the context I said
questionable and improper is discussing commercial software. When I think of
commercial software it is a non open source product being sold, licensed etc.
Not that open source can't be commercialized but if you notice almost all
companies run a dual license for their commercial vs open source software,
e.g. QT, Mongo and others. They do this because it is the only way to monetize
the software and not have every competitor have the same proprietary add ons
they created and companies are willing to pay for.

An example I dealt with a few years ago, I was working on an embedded project
and the associated desktop software that the original developer utilized some
GPL licensed open source libraries, compiled into the binaries (both embedded
and desktop). When I did a license review and discussed it with the attorney
it was deemed that GPL licensed components could not be included due to the
licensing terms and the way that product was distributed along with the closed
nature.

In general I follow the rule, and have been told multiple times, that GPL (any
version) with rare exception is not to be used in commercial software
(including API's and server based products) unless you are strictly
dynamically linking to the library. But if you are compiling it into your
commercial product and especially if you are making alterations to it and
compiling it in you are in violation of the license terms (and definitely the
spirit of the license). Of course, if your product is open source, then that
is a totally different story.

Even more open licenses like Apache and MIT still have copyright notices you
need to comply with at minimum, so all that has to be taken into consideration
as it needs to be added to product manuals, about pages etc otherwise you are
in technical violation of the terms. Hence, the license terms of any library
should be a key consideration on whether you include it into your project.

~~~
kyruzic
So someone you work with broke (or attempted to break? It's unclear whether
the product you're referring to was released) the law by using GPL code in a
closed source commercial product? This is done right improper (and illegal)
for the developer who used the GPL software. Not the creator who choose the
GPL likely to specifically prevent what you described.

People tend to understand open source software as software that is free to use
however you want. This is not what GPL software is though. It's entended to be
an ecosystem of software that is open and free to use as long as you are also
willing to share the software you create using the ecosystem.

------
tomxor
My _default_ these days is to roll my own (at least within some predefined
context), this is because I view each new dependency as a significant cost, so
i'll invert my answer: For me to include an external dependency it must
provide very relevant and high value to the project to outweigh it's inherent
costs.. even if that is satisfied I feel like I should also trust the authors.

This attitude has basically eliminated unhappy surprises from external code
for me. This is probably a disposition earned through developing mostly in the
JavaScript community (I am now the polar opposite of the NPM duct tape
developer, and I prefer flat dependency trees).

~~~
giffarage
This is interesting to me because I often see rolling my own X as having a
higher cost than bringing in a dependency. Maybe this is due to a lack of
experience on my part, so I’m curious what you all consider when evaluating
the cost of a dependency?

~~~
talkingtab
My opinion: rolling your own does have a higher initial cost, but it can have
a much lower long term cost. Consider the case where you need a very small
modification to the dependency. In many cases that will require a much greater
investment because you will now have to a) understand the dependency, b) make
and test the change and c) support the change long term (unless it is folded
into the mainline). Also consider the case where some dependency or
specification changes (way too frequently) or there is a bug that you
desperately need fixed. What do you do then? Technical debt. For me then,
understanding the probability and consequences of events like that is
important.

~~~
jklm
One thing I’ve gotten better at over the years is identifying the viability of
off-the-shelf tools early on. If you minimize subpar tooling at the source,
you can mitigate long-term frustration down the line.

------
teamski
My brain says never, for everything there is some turn-key thing. Even your
entire project might be franksteined with some third-party products and Zapier
into something working. Same with libs, better a crappy lib than working days
for something you might not need.

My gut says always: most of the times turn-key stuff has weaknesses and good
libs with perfect apis, well maintained and with a great community are rare
but they are there of course.

Hard question since what the gut says is more fun while the alternative is
just about gluing libs together. But using libs and once they don't fulfill
your needs build yourself is the right but more boring way to go.

~~~
BigJono
Your brain is falling into the classic JS/NPM trap of thinking that
dependencies are zero cost. The time it takes to use something is not 0,
there's still a comparison to make. There's _tons_ of stuff out there with
APIs, documentation and community resources that are way more complex to
navigate than whipping out a text editor and building something that handles
your own use case.

~~~
teamski
You are nitpicking and TBH I don't get your message. My post was a general
advice and the tendency of a creative mind, still you need to look into each
case and decide if make or buy. And your NPM trap analogy is just blatantly
wrong and doesn't help OP. You could have mentioned any package manager btw.

~~~
BigJono
I'm not nitpicking, I just picked out the first part of your post because it's
the only bit I disagree with.

I'm saying it's dangerous to have this line of thought that the "correct" way
to build a system is with a mish mash of third party libraries and that doing
any non-glue coding yourself is only for fun. You're insinuating that the
decision OP is making is between efficiency and fun, whereas in reality it's
an optimisation problem for efficiency that a lot of people mess up because
they don't understand one side of the equation. (and if OP wants to have fun
and roll his own that's great but it's another case entirely, I'm assuming
he's not asking HN for permission to do that).

I use NPM as an example because for JS devs in the places I visit it's
becoming a cultural thing, which IME is not the case in other communities. I
don't see as many python devs entertain the thought of maintaining something
for years with "crappy libraries" in it for the sake of saving "days" of work.
But in JS land that attitude is all over the place and the phrasing of the
first bit of your post reminded me of it, so...

------
talkingtab
If the library is crucial to what you are making it is worthwhile to
_consider_ rolling your own. Not because it will be better but in the process
you will understand the problem in a way that you otherwise cannot.

Libraries also tend to be general purpose so up to half of the code in a
library may not be pertinent to your usage.

I wrote an app that used a lot of Oauth and ended up writing my own instead of
using the excellent Passport library. It would have been easier to use that,
but I gained an understanding of Oauth that I would not otherwise have, and my
code is small, understandable and easy for me to maintain and modify. Just my
3 cents worth.

------
akling
For personal projects, I will happily build everything from scratch myself.
It’s great fun and I learn so much, so why wouldn’t I? (Although it’s fricken
awesome when people show up to help out.)

For work projects, I like to look for the best available FOSS option and then
try to help out on that project so I can feel a bit invested in it. This helps
calm down my hacker urges to build my own thing from scratch :)

------
err4nt
I always makes sense to roll your own X when you're a leader in the field and
X doesn't exist yet until you (or somebody else) makes it and releases it.

I've had a lot of situations like that and been accused of NIH syndrome, but
what is the alternative…not doing something at all because nobody else has
built or released something that can be used? Every library or tool in
existence started out because somebody wasn't content with the status quo
enough to bother inventing a new thing. This is the only way the state of the
art can progress.

------
bullen
This is not a popular opinion, but I think you should always rewrite the
wheel. The reason is simple, the product you are building cannot be more
interesting than the technology that lies under it. If you rewrite that, you
will have to learn how things really work. And only when you do that, can you
improve the fundamentals.

~~~
wpietri
To me the flaw in this approach is that we have finite time and attention. I'm
all for rewriting something when that something has the potential to radically
improve what I'm building. But if we are busy rewriting everything else, we
may never get to the one thing that delivers the most value.

As an example, if I'm building some new piece of SaaS that has an API, I could
write my own JSON parser, my own HTTP server, my own network stack, my own OS.
Heck, I could eventually learn how to make my own servers, my own processors.
But I'd be comfortably dead before I learned how to do all those things as
well as off-the-shelf components deliver for a SaaS API.

Instead I'd rather spend my time on activities that are more valuable to my
users. If one of those things turns out to writing a custom JSON processor,
sure, I'll do it. But if the only purpose in me doing that is to learn
something, then I shouldn't be jamming my educational experiments into a
production system that doesn't need them. I've had to clean up too much of
other people's flavor-of-the-month experiments to want to inflict that on
whoever maintains my code next.

------
davesmith1983
A few scenarios I can think of are:

1) If it doesn't work or works poorly.

2) If it is trivial to implement.

3) Licensing problems.

4) If the implementation is overly complicated.

5) The (framework) vendor have a habit of updating and breaking your code.
This happens a lot with the Angular team. There is nothing like the annoyance
of an Angular update and they have broken some feature you are using and you
have to update code that was working absolutely fine. Since then I have
avoided Angular entirely.

Things I am less strict on:

1) The project is EOL. It may work perfectly fine. This happens to open-source
software as well. Typically this means there is a better alternative out
there.

2) Awkward syntax / api. You can normally just wrap this.

3) Maintainers are difficult to work with or don't fix bugs.

------
Const-me
Depends on the library. If it's a good one, makes sense to adopt, I sometimes
do: [https://github.com/Const-me/nanovg](https://github.com/Const-me/nanovg)

Some libraries have hundreds of megabytes of their own dependencies hard to
remove.

Some libraries have very low code quality, makes it expensive to modify.

Some libraries are very hard to build, e.g. I've never managed to build Skia.
When you can't build it you can't adopt it.

You should also pay attention to license, there're GPL licensed libraries out
there, if you'll try to use or adapt them you'll have to open source your
complete software under the same license.

------
maliker
Makes sense to do it when you can hack together a quick prototype that will
teach you more about the problem and will be fun.

We needed a sensor data capture platform at work. There were a lot of odd
requirements, like consuming data from embedded hardware, severe bandwidth
restrictions, and high-precision (GPS) timekeeping. We hacked something
together in python + flask in 2 weeks, and it was easy to experiment on the
analysis side by just writing another flask route and generating some html
with plot.ly and leaflet.

We're also evaluating thingsboard and the elastic stack, but setting those up,
experimenting, and extending them was slower, less fun, and harder to debug.
At some point we'll do a shootout between all the platforms and pick the
winner if this is something that gains significant users, in which case the
community and active support of the platforms will be a big advantage.

~~~
o-__-o
Why hack something instead of using gprmc which is designed to handle the
backend for exactly what you are looking for?

------
AYBABTME
If the subset of feature that I need can be written in the amount of time it
would take me to read thru and internalize the library, I'll just write it
from scratch. Then if there's a problem with it, my team and I will already
have a good understanding of the code, instead of discovering that the library
is in fact crap (which they often turn out to be).

Then if it's something that doesn't really matter and the library code is
pedestrian (a logging library), I'll use the library.

If it's something that matters a lot and it's expert knowledge and is a large
body of work (a complex math/compression/crypto library), I'll use the
library.

------
scotty79
You might want to roll your own prototype to discover what you actually need.
Then when you know that, it's much easier to pick a library that satisfies you
actual needs not the needs you thought you'll have.

------
JustFinishedBSG
Whenever X is closed/proprietary and doesn't have an open alternative. Never
rely on something you do not ( or at least cannot if you wanted ) control.
Never. You WILL regret it one day.

~~~
tfigment
I have a closed source proprietary database that I wish I did not have to
manage. I pushed for the open source but CTO didn't trust that there would not
be a long development tail (it probably would) since there were few good
options at the time so he bought it and I regret not fighting harder against
it nearly every day since it went live.

The core product works fine and has little maintenance which is nice but its
cost scales badly and takes every bit of my budget now and the fancy features
never work or can actually be used in production. If I had that budget instead
we would have something far more suited for the company but seems like few
managers will take that risk unless its the core function of the company.

Also the mostly relevant xkcd:
[https://m.xkcd.com/1205/](https://m.xkcd.com/1205/)

------
Too
Google for how to do X in language Y. If 99% of the top answers on
stackoverflow say use library Z then just do it. Usually there will be 3
different Z in which case you should take some time to decide on one of them.

If there is no Z, don't try to find the needle in the npm haystack of poorly
maintained libraries with tons of needless sub-dependencies. Unless the
problem domain is very beyond your capabilities, just roll your own.

------
ashleyn
Programming your own GUI in an "engineless" plain-OpenGL/DirectX graphical
project or game. I rolled my own GUI for my C++ game because all existing
examples were either very badly designed (think singletons everywhere),
obscenely expensive, or just outright abandoned. Writing something that uses a
hierarchial box model with style attributes isn't too bad.

------
jacquesm
Reasons for valid NIH:

\- you are on your own and don't care about the future maintenance cost of
your code

\- you want to learn

\- the available solutions are objectively crap

Invalid reasons:

\- job security

\- you're a control freak

\- you think all code you write is immediately better than other people's
battle tested solutions, aka inflated ego.

\- you want to be seen as the maintainer of some prestigious project rather
than as the contributor to someone else's prestigious project.

------
ciguy
I wrote an article that covers this from a DevOps perspective, among other
things. Perhaps you'll get some value from it:
[https://calebfornari.com/2019/07/11/devops-decision-
making/](https://calebfornari.com/2019/07/11/devops-decision-making/)

------
chvid
Why would you make your own leftpad when someone already has spent the time
and the energy perfecting and publishing leftpad?

------
taormina
Another good question to always ask is "If I'm getting X from Y, what do I do
if Y stops providing X the way I need it?"

An example might be analytics. It's good to know that you can drop in a third-
party library and just go, but you need to make sure to look at how you can
move your data off of that platform and use it yourself. You also need to
consider how much of a time investment switching off of Y would be. How much
effort will rolling your own be? How much effort will rolling your own after
using Y? How hard will it be to swap Y out for Z?

Another classic case here is web hosting. I use GCP. I've used AWS. I really
don't want to, but if Google or Amazon kicked me off/out/raised the prices too
much, I'll host from my own hardware, and go to Fry's if I need to get more
hardware than I have. I really don't want to, but I will.

------
bluedino
Can we go over _bad reasons_ to roll your own x?

1\. You don’t want to take the time to learn an existing product

2\. You’re not going to properly code and test your own solution

3\. You think your product is too unique to use an existing solution

4\. You’re going to leave features out that you would have gotten for free

~~~
Const-me
> You don’t want to take the time to learn an existing product

Existing libraries can be very complex. Spending time on them is not
guaranteed to pay off. When a library is marketed as a framework, it’s a red
flag, such libraries often come with a strong opinion how the software should
be designed.

> You’re not going to properly code and test your own solution

This article is old, but still good:
[https://www.joelonsoftware.com/2002/05/06/five-
worlds/](https://www.joelonsoftware.com/2002/05/06/five-worlds/) Different
software needs different tradeoffs for quality, budget, and the rest of them.
If you’re working on a tool you’ll run only once, grab the output data, and
forget about the tool, you don’t need neither proper coding, not much testing.

> You’re going to leave features out that you would have gotten for free

Are you going to need these left features? For many real-life libraries, like
[https://www.boost.org/](https://www.boost.org/), you only going to use small
subset of these libraries. The features you’re not using aren’t exactly free.
Inflated binary size, slow compilation, tons of dependencies, complicated
build setup, complicated deployment are quite common issues.

~~~
bluedino
Some context might help my comment.

A programmer who used to work here, wrote his own PHP micro-framework, instead
of using one of the popular, standard ones.

Instead of just using one that already existed, and taking the time to learn
it, he used his own. He may have done it partially for the experience or his
own desires (he was probably actually bored because we have a bad manager, but
that's another discussion)

We regularly find and fix bugs with this PHP framework. Had he chosen a
mature, community-supported product, this wouldn't happen.

There are incomplete features you would have gotten for free with other
frameworks. It has maybe, 20% of the features other frameworks have, even
though it has much of what we 'need'. We had to write our own password-reset
email code, for example.

~~~
Const-me
Other reasons are possible, too.

1\. When they started, pre-existing frameworks weren’t such popular nor
standard.

2\. When they started, the functional requirements they had were very
different from what the product grew eventually. Many software projects start
small and grow.

3\. Couple times in my career I worked in projects where management didn’t
believe in functional specs, the requirements changed dramatically in the
middle of development. With large libraries, some changes in architecture are
handled by replacing the library with another one. Quite expensive to do. An
in-house library can be more flexible, just because it’s less code.

I’m not saying making in-house libraries was justified in your case. Maybe it
wasn’t and that developer did it mostly for fun. Just the trade off is quite
complicated, and your reasons are not necessarily that bad.

In many cases, I think it’s technical management’s job to make the decision,
i.e. evaluate alternatives, maybe allocate resources to make a prototype.

Also, from the management standpoint, if a developer can write their own
frameworks, there’s very high probability they can use existing ones. Maybe it
makes them slightly less happy i.e. need to pay them more to compensate, but
they’ll do the job. The opposite is not true, there’re people skilled at
combining libraries and copy-pasting third-party code who can’t write their
own code.

------
nickjj
It really depends on the task at hand and what the alternatives are.

For example I would much rather write 200 lines of my own code to deal with
user authentication (leaning on my framework or a lower level library to deal
with the gory bits of setting and deleting sessions), vs. pulling in some
higher level authentication library that has 3,000 lines of code but tries to
be so generic that it's a jumbled mess which is difficult to customize.

Basically the more important the feature is for my application, the more
inclined I am to write my own code. Although I may at times draw inspiration
from other libraries that don't quite solve my problem but helps me get closer
to what I want to do.

------
nullwasamistake
Depends on the license. If it's permissive I'll pop open the code to see if
it's any good. Maybe 50/50 it's good enough.

From scratch only if I'm doing something truly unique. New algorithm/protocol.
Unpopular language I'm forced to use is missing something I need. Existing
solutions too slow or buggy. Maybe a couple times a year I get to make wheels.

NIH is a huge problem at most (all?) software companies. At my current place I
spend about 1/2 of my time dicking around with crap internal tools and
libraries.

It's rare I write anything that's not glue code. Makes everything boring but I
don't get paged often and we have relatively few bugs.

------
RantyDave
Almost never. If you have (for instance) scale that the canonical solution
doesn't handle, consider it. If you'll be completely shafted if the canonical
solution disappears, consider it.

Please don't make another build system. Please.

------
knbknb
Polyfilling is also some kind of "roll-your-own X".

[https://en.wikipedia.org/wiki/Polyfill_(programming)](https://en.wikipedia.org/wiki/Polyfill_\(programming\))

In web programming, it might sometimes be useful to write your own polyfill
when you need to add a single feature for backward-compatibility reasons.
Well, a third-party polyfill library that provides that missing feature
perfectly might be available. However, the library is too big (with respect to
file size) or adds too much complexity during the build process (by pulling in
lots of other dependencies).

------
dragonwriter
> Suppose you find a library that kinda does what you need, but not really. Do
> you adapt it to your needs or do you build a version from scratch that
> really suits your needs, but might take a bit of time to develop?

Which do you think will take more time to develop and maintain: the wrapper
layer that adapts the existing library to your needs, or the ground-up
homegrown implementation? Your estimate of the answer here (along with the
risk calculus of the external dependency and the reliability of it's
maintainer) are the key factors in answering the question

------
jugg1es
There are numerous articles out there that will tell you not to roll your own
rule engine and indeed there are lots of them out there. But unless you are
solving a well known problem, I've always found the open source rule engines
to be either opinionated or just too large/unwieldy. It's not THAT hard to
make your own. You only ever a handful of chances in your career to implement
a rule engine anyway - since they are usually the core of the software that a
company sells. Make it right!

~~~
twhitmore
Tend to agree. When I've seen Drools integrated in Java apps, it's usually a
bloated mess and a simple DSL would have done much better.

------
hyperpallium
> adapt it to your needs

If it's a feature that can be cleanly added into the architecture - i.e. it's
just filling in a gap.

But if you have to modify the whole thing in a way that doesn't quite fit...
it's a nightmare. Even with your own code, it's simpler to start from scratch
with a clean slate.

Also Alan Kay's extreme position: "those who _can_ create their own
[libraries], should"

------
coffeefirst
In the "kind of does what you need" case, I'd write my own but maybe borrow
ideas from the original.

The moment you fork something, you've created work for yourself when you want
to upgrade it.

That may be fine for a prototype. But I've seen it done on software that lasts
5-10 years and those are always the things that turn into maintenance
headaches and blockers to unrelated upgrades.

------
meddlepal
If you want to own governance and direction of an X.

------
cgrealy
As always, the answer is highly dependant on a number of factors: \- How
complex is X? \- What expertise do you have available for X? \- How critical
is X?

But in general, if X is not part of your core business, only as a last resort.

99% of the time, even if there is a suboptimal library, as long as its
actively maintainted, you're better off extending the functionality, than re-
implementing.

------
contingencies
When the total lifecycle cost to use existing solutions is higher than the
cost to build and maintain, when the future direction of development is
unlikely to match your requirements, when the existing solutions lack critical
functionality and modification or extension are non-viable, or when you want
to learn more about the implementation.

------
jkoudys
I frequently run across libs that are more complicated to configure and call
than implementing the features myself. I'll often roll my own simply because
the lib (even if it's really popular) takes more time to use than that "bit of
time to develop".

------
mikekchar
I come from a era where free and open source software was quite rare when
doing commercial ventures. Also I did a fair amount of embedded work in my
early career. As a result, my normal first reaction is build over "buy".

There are lots of very good reasons to prefer building something. First, the
design of the code will fit your use case more or less perfectly. Unless you
get it wrong, in which case you can change it relatively easily. If you use an
off the shelf library/framework, you may need to jump through hoops to use it.
Those hoops may result in other hoops, which result in other hoops, etc, etc.
In the end, you may introduce nearly as much code complexity adapting your
code to someone else's design than you save by using their library/framework.
(NB: You _could_ just use "convention over configuration" ala Rails, which is
another way of saying, "Do everything my way and you'll never have any
conflicts" ;-) ).

Second you have control over the code. If you use a library/framework, it may
change over time in ways that don't fit your project. You are stuck making the
choice of maintaining all of that change or eventually suffering from bitrot.
A good example is if your library has another dependency. It replaces that
dependency with a new one which isn't supported on your system.

Related to that, you may decide that you need new functionality that the
library/framework does not give you. If you were working with your own code,
it's easy to add that functionality. If you don't, you need to either maintain
a series of patches, or try to get your changes merged upstream. Some projects
are easier than others to work with.

When I've decided that I'd rather not roll my own, if possible, I look at the
alternatives. The first thing I do is read the code. Can I understand it? Will
I be able to fix bugs if I need to? How much of the code is related to what
I'm doing and how much is unrelated? How will the structure of the code affect
what I'm doing? Are there any controversial dependencies?

Then I look at the community. I look at open issues. I look at how discussions
progress. I look at closed issues. Do people get yelled at for asking
questions? Are suggestions for improvement valued? I look at open and closed
PRs. How easy is it for an outsider to make contributions?

After that, I make an estimate in my head of the maintenance cost of code I
wrote myself vs the maintenance code of using one of the alternatives that
I've researched. Usually if we are talking about 1 week of work or less
rolling your own wins out (there are exceptions, though). If I only need a
handful of lines of code, I'll frequently make my own derived library with
those lines of code (For example, I do a fair amount of Ruby code and there is
useful code in Rails... But I'm not going to grab ActiveSupport just because I
want stringify_keys or something similar).

Finally, if a library is important to my code, I'll usually make an adaptor
for it. Instead of calling the functions of the library directly, I'll make
another library that wraps it. That way if I run into problems with the
dependency I can swap it out without much difficulty. Of course, for
frameworks that doesn't make sense because part of what you are buying with
the framework is the design.

~~~
jrumbut
I think part of the jumping through hoops consideration is whether the project
developers as a group prefer jumping through their own hoops or someone
else's.

On some teams it is easier to say "the wise developers of leftPad think these
are the hoops we need to jump through" than it is to present your own design
that has strengths and weaknesses. On others they will walk miles through
flaming coals just to avoid the downsides of someone else's designs.

I honestly have no idea which group is right but devs seem to fall into one or
the other. I myself started in the first camp but am wandering toward the
second. Sounds like you went the other direction.

------
koolba
If you read through the code and determine the quality is beyond repair, or
the project model itself does not lend itself to repair, it’s a fine time to
chart your own course.

If you decide to open source it then kudos to you. If you don’t, I won’t think
less of you.

------
qrbLPHiKpiux
If X=crypto, then never.

~~~
squirrelicus
Whoever downvoted this deserves to have their license to code revoked. It's
malpractice to roll your own crypto.

~~~
segmondy
... and how do you know the person rolling their own crypto doesn't have the
skills? Isn't the crypto libraries out there written by humans? If you're
rolling your own crypto because you believe in security by obscurity then
you're probably not smart enough to roll your own. But if you're willing to
open it up to the world because you have a new trick up your sleeves then sure
if you have the background.

I still think that ties into the original question, when does it make sense to
roll your own? If it's not part of your core business and doesn't give you an
edge, then it never makes sense, unless you're doing it for the fun of it.

~~~
squirrelicus
If you have the skill, we already know who you are.

You have millions in the project and a dozen PhDs working with you, and you've
already authored public papers demonstrating the weaknesses of things
previously believed to be strong.

If you are not at least one of those things, it's malpractice for you to even
consider letting your employer pay for your crypto "invention"s. That's like a
nurse building her own OR and insisting the hospital let her cut people open
in it. No. Don't. Stop.

------
tuckerpo
When you feel you could better implement X for your niche needs.

------
joaofiliperocha
depending the time that needs to be spend of the fix ... max 4h

------
gok
Always roll your own.

------
lazyjones
Almost never. People should even consider using SWIG or similar to use
libraries for other languages instead of wasting time on duplicating efforts
and neglecting their core product/project.

------
superkuh
The X window system is being dropped from dev by Red Hat. So yeah, you'll
probably need to roll your own. As for the other X, non-institutional
chemistry is generally frowned upon by governments.

Generally X existing things are going to be way too complex and heavy.
Implementing your own allows you to avoid all that. But of course this only
applies for individuals. Anyone stuck as part of a company or institution
won't have the choice.

