
We found over 10000 bugs in various open source projects - Tatyanazaxarova
http://www.viva64.com/en/b/0421/
======
kazinator
I just love this one:
[http://www.viva64.com/en/examples/V504/](http://www.viva64.com/en/examples/V504/)

    
    
      void GlyphComboBox::update(SPFont* spfont)
      {
        if (!spfont) return // <=
      //TODO: figure out why do we need to append("")
      // before clearing items properly...
    
      //Gtk is refusing to clear the combobox
      //when I comment out this line
        this->append("");
        this->remove_all();
      }
    

Gee, why do you need the dummy append? It's because you put in so many TODO
comments that you concealed the missing semicolon that you otherwise might
have found ...

This is arguably not a bug; the developer tested the code, massaged it into
working as expected, using what look like well-defined constructs, and
documented their surprise at certain aspects of the code, leaving this issue
easy to find with a grep for TODO.

On another note, what is surprising here is that they didn't catch it with
their compiler. Look, the function returns void. GCC has a diagnostic for
this, something like "return with expression in a function returning void". I
think that still kicks in even if that expression has void type.

~~~
BruiseLee
Gcc will not issue diagnostic if you return a "void expression" because doing
so is valid in C++.

~~~
kazinator
It could still be a warning, since doing so is dubious in C++ even if it is
allowed.

------
raverbashing
What I take from this: computer languages (C++ especially, but also systems
built in C) are too complex for the average mind

No, I'm not significantly smarter than the average programmer

That's what needs to be addressed.

While C++ facilitates things, it is complex. While C is very simple it makes
mistakes easier by requiring things to be done manually

IDEs don't help. Code completion is helpful but it is of limited help

And we keep repeating ourselves every time we write a for loop

~~~
n00b101
>computer languages (C++ especially, but also systems built in C) are too
complex for the average mind

I'm not necessarily disagreeing, but have you looked at the source code [1] of
the Apollo Guidance computer [2]? This software was shipped to production 50
years ago. It literally implements a "self-driving space rocket." It is
staggering that this was completed in such a short time and actually worked.

The code is written entirely in the "Apollo Guidance Computer (AGC) Assembly"
language. The hardware that it was written for had to be invented in at the
same time, and was the world's first computer to use Integrated Circuits
("chips"). They also implemented a Real-Time Operating System with a multi-
task priority scheduler, as well as a virtual machine and assembly interpreter
within the software.

This was mission and life safety critical code, the risk involved boggles the
mind. And yet the software was incredibly reliable. It was so reliable that it
even saved the Apollo 11 crew and mission during the lunar descent by famously
recovering from a peripheral hardware bug.

I honestly do not know how this feat was even humanly possible, but I do know
that these programmers did not rely on "code completion."

[1]
[https://github.com/chrislgarry/Apollo-11](https://github.com/chrislgarry/Apollo-11)
[2]
[https://en.wikipedia.org/wiki/Apollo_Guidance_Computer](https://en.wikipedia.org/wiki/Apollo_Guidance_Computer)

~~~
drzaiusapelord
>I honestly do not know how this feat was even humanly possible

The Apollo program, fixed for inflation, was something like $130 billion
dollars. That's buys you a lot of talent, testing, etc. Its also worth noting
that the program wasn't without its screw ups. The Apollo 1 deaths. The radar
control on Apollo 11 eating up all the CPU during lunar landing. Apollo 13's
famous O2 tank explosion.

I think you may be glamorizing something that was well done, but falls within
the realm of 'you get what you pay for' and the occasional human error, which
in the case of Apollo 1, turned out to be fatal. I wouldn't call a program
with three deaths and an aborted lunar insert to be infallible.

That said, NASA/JPL publish code standards, although I imagine most tech
companies would balk at giving up their toys and hot languages/frameworks to
do fairly tedious work like this.

[http://lars-lab.jpl.nasa.gov/](http://lars-lab.jpl.nasa.gov/)

~~~
eremeevp
>That said, NASA/JPL publish code standards, although I imagine most tech
companies would balk at giving up their toys and hot languages/frameworks to
do fairly tedious work like this.

For most companies developing non-critical software, applying such harsh
coding standards will not be cost effective after all. The cost of you browser
crashing is not nearly as high as the cost of a crashing rocket...

Besides, even NASA sometimes makes mistakes:
[http://www.viva64.com/en/b/0416/](http://www.viva64.com/en/b/0416/)

~~~
drzaiusapelord
Agreed, there are reasons why we do certain things a certain way. But in the
age of ransomware and endless cyberwars and nation state spoofing maybe the
idea that browsers and other internet facing applications should be barely-
shippable shit code is antiquated.

~~~
eremeevp
You can't protect everything, but I'm sure they have a secure browser at NSA )

~~~
nickpsecurity
NSA just uses isolation approach in air gaps or hypervisors like these:

The high-assurance community in academia and commercial sectors keep building
secure browsers, though. One, OP browser, was inspiration for Chrome security
model. They severely weakened it for performance. IBOS is probably most
interesting. There's links at the comment below.

[https://news.ycombinator.com/item?id=9969961](https://news.ycombinator.com/item?id=9969961)

Quark kernel is one that's not in that comment for some reason:

[http://goto.ucsd.edu/quark/](http://goto.ucsd.edu/quark/)

~~~
nickpsecurity
I just realized I forgot the link with their hypervisors. It doesn't matter
because most of them suck. You're better off not seeing it. :)

------
JoeAltmaier
Many of the errors are just coding irregularities. Half are really good
catches; but half are of the 'lint' sort. Still worth reviewing in case an
actual mistake was made.

~~~
pm215
Right, and so I've developed the view that the most interesting quality of a
static analysis tool is not "how many bugs does it find" but "how many of the
things it points out are false positives" and (if you want to be really tough)
"how many of the things it points out are likely to cause problems for a real
user or be actually exploitable by an attacker". Most software projects
already have a huge list of bugs they haven't fixed yet, so a tool that
identifies 10 easy to fix real bugs is great, but a tool that mixes those 10
real bugs in with a hundred false positives is much less useful, simply
because of the time it takes somebody to sift through the list...

~~~
nickpsecurity
Yes. I initially promoted all kinds of static analysis. These days, I'm firmly
convinced the friction aspect means the tool needs to be plug-n-play with
almost no false positives. Ideally no false positives at all. That maximizes
uptake and knocks out many bugs even if more remain.

Another thing I think advocates need to do with lists of such tools is always
mention that each one might see errors another won't. This needs to be a
refrain that instantly is in people's minds when they wonder about using such
tools. Reason is that many make the mistakes of thinking one tool not finding
anything means (a) tools don't find stuff in general or (b) there's nothing to
find. That implication is even on SQLlite testing page where they applied
Clang's analyzer then implied a weaker tool finding nothing in a good codebase
meant static analysis itself had nothing to offer (?!).

------
woah
Here are the classes of bugs:
[http://www.viva64.com/en/examples/](http://www.viva64.com/en/examples/)

I'm no expert, but it looks like most of these would be caught by the Rust
compiler at the time they were typed in.

~~~
stebalien
Yes. That's why they don't like rust:
[http://www.viva64.com/en/b/0324/](http://www.viva64.com/en/b/0324/)

~~~
jcranmer
That ... is pretty jarring. Their arguments boil down to:

a) No one wants to use a "minor" language, so it will never get traction

b) It's slow (citation: the Benchmarks Game)

c) Static and dynamic analysis can find anything anyways, so what's the point?

First off, anyone who is seriously citing the Benchmarks Game as to making a
definitive declaration of whether a language is slow or fast is intellectually
dishonest. It's a game (as its name illustrates), so it reflects more the
effort that people put in to try to optimize code rather than a comparison of
"idiomatic" code. A case in point: the fastest program in C uses SSE
intrinsics for its implementation.

Given that they're selling a static analyzer, I would expect them to make the
case for analysis. However, it's worth addressing the limitations. Almost no
project has 100% test coverage, and even having test coverage isn't
necessarily a cure for having no problems (IOC still found undefined overflows
in SQLite, which is the only open source project I know of to have 100% line
and branch coverage in their test suite). To my knowledge, there still exists
no tools that can identify strict-aliasing violations or ODR violations. It's
also worth minding Dijkstra's aphorism: testing cannot prove the absence of
bugs, only their presence. Static analysis in C/C++ has to face the additional
problem of the inability to write a precise pointer analysis--something that
Rust's type-safety would fix.

~~~
EvgeniyZh
> First off, anyone who is seriously citing the Benchmarks Game as to making a
> definitive declaration of whether a language is slow or fast is
> intellectually dishonest.

But Rust is actually significantly slower than C++, that's for sure. You're
welcome to test it by yourself.

~~~
steveklabnik

      >  You're welcome to test it by yourself.
    

If this is true, please file bugs. They're treated as such.

~~~
igouy
A Rust k-nucleotide program based on std::collections::hash_map would be
welcome.

Edit (2017-02-09):

Here are Rust k-nucleotide programs -

[http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...](http://benchmarksgame.alioth.debian.org/u64q/program.php?test=knucleotide&lang=rust&id=1)

[http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...](http://benchmarksgame.alioth.debian.org/u64q/program.php?test=knucleotide&lang=rust&id=2)

[http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...](http://benchmarksgame.alioth.debian.org/u64q/program.php?test=knucleotide&lang=rust&id=3)

~~~
steveklabnik
I don't know the history here, but

[https://github.com/TeXitoi/benchmarksgame-
rs/blob/master/src...](https://github.com/TeXitoi/benchmarksgame-
rs/blob/master/src/k_nucleotide.rs)

[https://github.com/TeXitoi/benchmarksgame-
rs/issues/34](https://github.com/TeXitoi/benchmarksgame-rs/issues/34) (though
some things have changed since these comments...)

~~~
igouy
k_nucleotide.rs implements it's own hash table, and that is no longer
accepted: k-nucleotide programs that do that have been removed.

Hence --

    
    
        k-nucleotide
    	
        source 	secs 	KB 	gz 	cpu 	cpu load
      	    No program 	contribute your program
    

[http://benchmarksgame.alioth.debian.org/u64q/rust.html](http://benchmarksgame.alioth.debian.org/u64q/rust.html)

~~~
steveklabnik
What about languages that do not include their own hash tables? C is allowed
to, but Rust is not?

~~~
igouy
No, the C programs are not allowed to implement their own hash table -- and as
a consequence no C k-nucleotide program was shown for a couple of months.

[http://benchmarksgame.alioth.debian.org/u64q/knucleotide-
des...](http://benchmarksgame.alioth.debian.org/u64q/knucleotide-
description.html#knucleotide)

Rust std::collections provides hash table implementations, it's just a matter
of using them.

~~~
steveklabnik

      > some use a third-party hash table library.
    

This is what I'm getting at. Since C doesn't provide a built-in hash table,
they get to choose one that's good on this benchmark, but other languages
can't. That's not great.

(I actually have no idea if the std::collections hashmap is deficient in some
way, I care about this in the abstract, not specifically.)

~~~
igouy
No, they don't "get to choose one that's good on this benchmark".

They get to use a third-party library that was _not invented for this
benchmark_ \-- that would be the point!

------
AdmiralAsshat
I don't know what tone the article title is going for. Is it meant to imply
that open source software is somehow buggier than closed source? Or simply
that they made open source software better by finding these bugs.

If the former, it's worth noting that these are 10,000 bugs that are visible
_because_ the software is FOSS. Closed source software uses code
analyzers/linters too. I worked on a piece of proprietary software where we
had a giant spreadsheet for the several hundred bugs and other minor issues
that flawfinder found on our software. Those will never get disclosed to the
public.

~~~
wolfgke
> I don't know what tone the article title is going for.

Did you really not? Then you have not read enough articles in the viva64 blog.
:-)

The tone is always "Static analysis with PVS-Studio is a great way to find
bugs in projects and it would be a good way to use PVS-Studio for your
projects, too".

~~~
wila
Of course this is marketing, that is obvious from the title alone. The open
source projects give them a means to post a high numbers of bugs found by
their magical product. For a high number they just have to scan more products,
they can even decide to omit the projects that didn't trigger much bugs. They
can't post the results from paying customers in a marketing stunt like this.

They mention that they let the developers of the open source projects know
about the issues they find. I am wondering how much time the developers get to
patch up before they post their results.

------
spacemanmatt
Sad but not surprising.

------
meira
Now I challenge you to find 10000 software contributors.

~~~
evgryz
You can find tons of tickets from PVS-Studio in many bug-trackers :-).

------
tgarma1234
In other news: people who do things in their spare time often make mistakes
and do slipshod work.

~~~
topspin
Projects like Chromium and ChakraCore are not being written in anyone's "spare
time." It would be nice if people would stop conflating Open Source with
unpaid hobby work. The projects analyzed in this story represent billions in
paid labor.

~~~
tgarma1234
In other news: people who have to focus on detail oriented tasks often make
mistakes.

Happy now? Sheesh.

~~~
topspin
_people who have to focus on detail oriented tasks often make mistakes_

Whether they're paid well or not.

------
freddref
People shouldn't be writing code at all. A lot of testing and small code
corrections should be automated at the very least. We have the iron, lets put
it to work.

~~~
jschwartzi
So who writes the automation, then?

~~~
jjnoakes
Isn't that like asking "So who builds the assembly lines, then?" way back
when?

