
Can We Trust the Libraries We Use? - AndreyKarpov
http://www.viva64.com/en/b/0271/
======
InclinedPlane
Of course not. You can't even trust the firmware on the storage your OS runs
on. We're in a very weird period of time computer security wise. There was the
first golden age, when computers weren't easily connected and harmful software
was generally not very impactful. Then there was a brief dark age when
computers became connected and the utter lack of concern for security was
revealed to be hugely exploitable on an unprecedented scale with an enormous
impact. A lot of the worst mistakes were corrected and a lot of the most
important software was tightened up, leading to a long "security cold war"
lasting from roughly the late '90s or early 2000s to today. Right now we're
sort of in a cuban missile crisis period in the cold war, or something
similar, the impact of computer security holes could result in events (of
several possible kinds) which might have as much global economic and
geopolitical impact as a major war. Whether that happens or not is as yet
unknown, but the potential is there, and will remain there for likely quite
sometime (several key vulnerabilities: core libraries such as openssl,
essentially every major operating system, firmware on usb devices and storage
devices in general, BGP, and SSL/TLS under the CA system).

I don't think most tech companies and governments are taking the risks and
threats seriously enough as of yet so I suspect some sort of calamity will
have to happen first before things are truly tightened up well enough for
anyone to be able to trust anything and not be deluding themselves in doing
so.

~~~
DanielBMarkham
My opinion is that technology development is going to have to bifurcate -- or
perhaps even split into three paths. No longer can you simply sit down and
code something. Instead, you'll need to code some set of "rules" (this system
can't use the internet, it should never access the GPS, and so on) at the same
time. Technology will "run" when all the separate systems agree that
everything is behaving correctly.

We've tried to do this with various XML/manifest declarative constructs, but
the problem is that these constructs just rely on subsystems which are
themselves insecure.

These systems will have to be baked into the metal, and they're going to need
to come from different vendors.

Unlikely to happen, but that looks like the way forward.

~~~
InclinedPlane
Well, yes. A big part of the problem is the lack of terminology and models.
"Software development" is a stupidly large scope, it encompasses activities
that are comparatively as varied as backyard astronomy and heart surgery.
There are many different kinds of developers and many different kinds of
software, but we lack the words to express the differences clearly. Which
makes it that much more difficult to improve processes and standards. It's
common for bromides and recommendations to be made as blanket statements
applying to all software development, when it's clear that no one set of
practices could possibly apply universally.

Worse, many devs aren't even aware this is a problem, and few people are
working on fixing it. We're deep in a morass that will take a long, long time
to get out of.

------
fit2rule
Code coverage and static analysis are such valuable tools, I can't believe
they are not used more and more frequently .. it really seems that a lot of
open source projects would benefit from a coverage/analysis phase on checkin.

~~~
seren
It seems that Coverity[0] is offering free static analysis to open source
projects. Even if it is a marketing move, it will benefit the open source
ecosystem in the long run. You'll waste some time analyzing false positive
defects, the handful of critical/security error found make it worth it in my
opinion. Notably the duplicate "goto fail" in one of the recent apple
vulnerability should have triggered the dead code detection.

[0] [https://scan.coverity.com/](https://scan.coverity.com/)

~~~
AndreyKarpov
The PVS-Studio team also does a great job:
[http://www.viva64.com/en/a/0084/](http://www.viva64.com/en/a/0084/)

------
zurn
The title doesn't really fit the article. The author just picked up an open
source C/C++ library and found some bugs in its dependencies using their
commercial static analysis tool.

This is hardly a surprising result, and does not tell us anything new about
how one might evaluate or build highly trusted software.

------
kabouseng
You can't guarantee trusted software even your code and the libraries are 100%
trusted / bug less, as the combined software creates emergent properties which
will also have to be properly tested. As such you can only trust a software
system if it has been properly tested.

If this is the case, then you can provide 100% trusted software even if it has
been built with insecure code bases / libraries, as long as all the external
interfaces has been tested for all possible inputs.

Off course this is extremely difficult / impossible. This is why we will never
arrive at a point where software is 100 trustworthy.

edit - spelling

~~~
glibgil
Your comment provides no value. The OP has specific examples of tools being
used to find bugs in software all without the need for testing. Testing is the
least efficient way to fix those bugs. Tooling, as demonstrated in the OP, and
other applications of programming language theory are far more efficient ways
to have correct software. First, let the computer find the bugs we don't know
about, then we can focus on higher goals.

~~~
onion2k
_Testing is the least efficient way to fix those bugs._

Testing isn't a way of fixing bugs. It's a way of making sure you don't
introduce bugs when things change. The important thing to remember that 'code
the first implementation' \- where you go from having no code to having some
code - is a change _just like any other_.

Writing code without bugs is a lot more efficient than writing code with bugs
and then fixing them.

~~~
richardwhiuk
It's a pretty rubbish way of checking whether you've introduced bugs when
things change as well.

Testing is good for doing one thing: Validating that the code meets a specific
quality target.

------
collyw
Depends where they come from. The average Python or Perl library seems better
written and more tested that something I would write myself. JavaScript seems
to have a lot less quality in my experience.

~~~
GrinningFool
I was thinking about that in response as well; then I began to wonder. Is it
that the quality is actual higher, or that I'm more likely to gravitate
towards established, well-tested libraries (of which there are fewer in
javascript)?

The more I think about it, the more I think it's the latter. There are tons of
crap libraries out there for any language - but the more mature
languages/ecosystems naturally tend to have a wider selection of more mature
libraries.

~~~
jmspring
Think about node modules. How often have you done an n install only to have
10+ other libraries brought in. Some are going to be decent, some not.

I've found oddities like improper implementations of http digest auth and what
not. When libraries are small you can sometimes catch these things easily.

------
nullc
> A variable is assigned its own value.

Self assignment is used to silence overly aggressive and incorrect
uninitialized warnings in GCC (esp prior to 4.5).

I'm not sure if thats what the code there is doing— not enough context, but
I'd guess it's likely. Odd that the author hasn't seen that before.

~~~
tokenrove
That seems unlikely to be the case; the variable being assigned is through a
pointer to a struct, which is already being accessed in the context. (Plus, of
course, __attribute__((unused)) or similar in a macro would be preferable for
silencing warnings) It's pretty likely that, had this occured at the top of a
function, and been a direct assignment, the author would not have flagged it
(indeed, perhaps PVS-Studio ignores that case; seems easy enough to
implement).

~~~
Mathiasdm
__attribute__((unused)) indeed would be the best option when using gcc. I
believe when using C++11, this can be replaced by [[gnu::unused]].

Another one that I've seen quite often is casting to void.

~~~
nullc
Not unused. Uninitialized.

E.g. in older GCC you would get int x; switch (!!foo) { case 0:x=0;break; case
1:x=1;break; / _no default_ / } print(..x)

Or many other cases and the compiler will give an erroneous uninitialized
warning. you can assign x to itself at the stop to eliminate the warning
without changing the generated code.

------
hrjet
Java has a very good sand-boxing mechanism (AcessController), which helps
limit the effect of bugs in libraries. Firstly, the memory model is highly
abstract; no pointer arithmetic and type-less references. To mess with the
memory, the thread needs to drop down to native code, and this where
sandboxing can trap this behavior.

The same mechanism can also help increase trust in a third-party library from
a security point of view. Every access to a file or network resource, for
example, is checked before being granted.

~~~
protonfish
Compare this with the worst library include language: JavaScript. There is no
sand-boxing at all - code is simply inlined. (PHP's require does this too.)
But for some reason web developers pile on the shady libraries when they
should be extremely discerning.

~~~
wldcordeiro
How many wheels have been rediscovered in the Javascript ecosystem? It seems
to be a symptom of the ecosystem as a whole.

------
kristoffer
Of course you can not trust them. Just look at OpenSSL. If you truly need
"safety critical" there are no shortcuts.

------
ASneakyFox
Typically you test your software to your needs. If there's a bug in a library
you use it would show up when testing your code that uses the library.

~~~
mkal_tsr
> If there's a bug in a library you use it would show up when testing your
> code that uses the library.

That's not true at all. You may catch some lib bugs, but not all, even through
testing of your own code that leverages the lib. Unless you're implying that
you'll pass every possible value to your tests which will then cover 100% of
any and all code in the lib.

~~~
jenscow
You, the user of the library, don't care about finding any bugs in parts of
the library that your application isn't using.

Your tests will be testing the parts of code in the library that your
application uses, which is the part that matters to you.

~~~
clarry
You can still have bugs in parts of code that you use (or would use) given the
right inputs. The problem is making your tests so diverse that they test every
possible input.

~~~
ASneakyFox
You could. But you'd miss those bugs if you write that code yourself any way.

------
ris
Another PVS Studio advert

~~~
franzb
Who cares. It's finding and reporting genuine bugs in widely used open source
libraries, and most of the times it's even suggesting a fix or a workaround.
Think about it: each article is worth half a dozen to a dozen high quality bug
reports.

Moreover, the guys behind PVS-Studio seem like great guys. The CTO, Andrey
Karpov, was interviewed in early 2014:
[http://www.viva64.com/en/b/0231/](http://www.viva64.com/en/b/0231/).

Selling software development tools isn't easy. PVS-Studio "adverts" are
interesting and useful to the community.

------
DanielBMarkham
Test the hell out of anything you use, both hardware, software, and
infrastructure -- and write the tests before you write any code. Double-down
on that for stuff like medical tech.

Now, is anybody doing that? Anybody? (Sound of crickets chirping)

~~~
glibgil
The article is about static analysis. Static analysis is the most efficient
way to catch a whole host of bugs. Testing would be a waste of everyone's time
for these types of bugs. Now, the tools in the article are commercial and
perform static analysis on C and C++. Other programming languages have static
analysis and code path coverage analysis built into their compilers. Many of
those languages are free. Writing tests to cover those types of bugs is time
wasted on a problem that is already solved.

~~~
pjmlp
Funny enough in a sad way, static analysis tooling was part of the original C
tooling, but when developers started creating C compilers for systems other
than UNIX, lint seldom came along.

It also did not help the way C compiler phases were split.

If anything, we have to thank LLVM guys for making static analysis part of
clang.

------
jbb555
Ugh more spam for this product.

------
bayesianhorse
Do you have a choice?

------
6d0debc071
Depends what your use case is.

~~~
glibgil
In this case it is bugs that can be discovered and fixed through static
analysis. What about this case?

~~~
6d0debc071
If your definition of trust is that it passes static analysis without throwing
any warnings, then you probably can't trust it. But this is hardly a new or
useful insight, almost all code has known bugs in it at the time it ships and
the additional value of discovering another bug is generally quite low. The
value of discovering bugs lies in understanding how they relate to an
undesired behaviour of a running system.

------
itchmasterflex
I hope so

------
Mikeb85
You have to, because there's no alternative. And even if it were possible to
assure 100% safety from 100% of the code that runs on your computer, how do
you know you can trust your hardware?

~~~
glibgil
Let's down-vote these types of comments that answer, vaguely, the question
proposed in the headline and reveal that they did not even skim the topics in
the OP. Totally worthless.

~~~
Mikeb85
Maybe the title shouldn't propose a question that is controversial (and
obviously click-bait), and then have the bulk of the article be about
something completely different.

You're right, I should have read more of the article to realize that it's
basically about nothing (someone looking for, and fixing bugs in a library he
uses, which everyone who cares does).

