
Ask HN: How do you measure risk with an open source project? - bazMVP
If you&#x27;ve found a new project on GitHub that you want to use, how do you quantify the associated risk of using it in a &quot;production&quot; application? For example: the project is only open for 6 months, or has many more open issues vs. closed issues, or has negative sentiment in commit messages. Of course it&#x27;s a case-by-case basis, I&#x27;m looking for examples of what factors are deemed important when making this type of decision.
======
kazinator
* I look at the code and determine, subjectively, whether this was written by first-rate developers or monkeys. I will consider this from various angles ranging from the overall program organization, to the details of how the programming language is used. If I spot bugs in this inspection, I will skip the project and look for something else. In particular anything that is a security flaw or could cause a crash is an instant deal-breaker. Not because everyone should be perfect and write error-free code the firs time, but because I was able to find it just by casually looking, whereas the maintainers have been working with that code for months and are blind to it---that erodes my confidence in the developers.

* I will look for a regression test suite: how extensive is it? If you don't see any tests, that's a big warning sign.

* If you don't see any documentation, another warning sign: the behavior is not specified and could change.

* Who _uses_ it? If the code is reasonably widely used, that de-risks it for you quite a bit, particularly if the existing uses resemble your intended use. They trod through the code paths before you and uncovered the bugs.

* Lastly: if need be, can I just maintain this myself? How easily forkable is the thing? This point can overcome some other issues. If some code is 95% of the way there, and is organized in a good way that I can take it the last 5%, I might just do it.

~~~
partisan
Keep in mind that the last 5% might mean a heck of a lot of time. There is
usually a reason why some things go unimplemented in open source projects and
the complexity and time required are usually high on the list.

~~~
kazinator
The last 5% is _my_ last 5%, not (necessarily) the project's 5%. I.e. the
difference between what is there and what I require, not between what is
there, and the project's goals.

------
Gratsby
If there's an IRC channel, mailing list, issue tracker, etc., I have a look at
it. Active communities are bonus points regardless of the bug list.

I also look at how friendly the project is to pull requests and outside
development. If a bug is important to me, I will spend the time to code a fix,
but if there's no hope for getting any changes made, that represents a large
risk to me. It's not a bad thing if the team in charge pushes back for higher
quality code, code style, or solutions practical for wider audiences.

If there's continuous integration in place with automated testing and static
code analysis that is fantastic. It's not a deal breaker if it's not, but
having it in place is a good sign.

Depending on the project, I may have a look at the source itself. I certainly
don't look the source of every application I use.

I have found that online recommendations in developer communities are not
always good. More than once I've tried out projects based on people
evangelizing them only to find out that they are pretty far from acceptable.

There are differences in how I judge things that will be customer facing or
that I'm going to have to support operationally. I'm a lot more critical at
that point, but my top 3 points are:

1\. Can I put it in a container. 2\. Are the developers committed to long term
support. 3\. Will they give me a t-shirt.

------
cweagans
I evaluate it the same way that I evaluate any other code that I'm bringing
into our project: is it robust, extensible, free of obvious errors, covered by
comprehensive unit/functional tests, etc.

I would say that "project is open 6 months" is a pretty poor metric, because 6
months is a _lot_ longer than the handful of days that your custom code will
have existed when you add it instead.

~~~
bazMVP
That makes sense, any time period is probably less valuable than looking at it
relative to commits or issue resolution rate.

~~~
cweagans
Even evaluating it on historical commits or issue resolution rate is not very
productive, IMO. They really aren't an indication of future activity unless
there are at least 10s of developers working on it constantly for a long
period of time (say, > 6 months). I've been burned too many times by getting
hopeful about what a project maintainer is going to do in the future, so my
criteria is basically taking the code in it's current state and evaluating it
like I would any other code. If it's a matter of opening a PR or two to get
things cleaned up, and PRs seem to be merged relatively quickly, I might
chance it, but I can't wait on an unresponsive maintainer to merge things. I
have deadlines to meet. Ain't nobody got time for that.

------
thealistra
0\. Is README in good shape and the API is sane? 1\. Last commit date in last
month 2\. Number of stars depending on the complexity of the lib

------
twunde
When was the last time the code was updated? How many people have contributed
to it?Really I'm looking to see whether the project is being actively
maintained.

