
Clang vs. GCC – code coverage checking - rkday
http://rkd.me.uk/clang-vs-gcc-coverage.html
======
notanote
Am I reading this right? He says he is comparing clang 3.8, which is a month
old, to gcc 4.8, which branched over 3 years ago. I would rather see what gcc
6 does with this. That release should happen in a month or so.

~~~
bubuga
I wouldn't read too much into the article. It's just a blog post of a dude
reporting what he saw when he recompiled his code with a different compiler.

~~~
Sanddancer
Aye, but it shows something important. Testing your work with other toolchains
and toolings can help discover weaknesses not only in your code, but in the
rest of your tooling. Strength in numbers is useful.

------
drewcrawford
I need to ask a stupid question. What does one _do_ with code coverage data?

Like, I know what it _is_. What I'm asking is, on what occasion would the
presence of code coverage data be more useful than stepping through the code
in a debugger. What is something you do differently having this data than not?

Like, is there a code review gate that forbids the code coverage figure to go
below 80%? Or do developers consult the reports in a self-motivated way for
some reason? Regularly? What is the step between "collect code coverage
reports" and "profit"?

We recently got this feature in our compiler, and I guess it seems obvious to
everyone why to use it, but I must have missed that day in class. So
everybody's really excited to create the report, but I can't figure out what
they do with it afterward that explains the whole exercise.

~~~
rkday
We actually hook it into our build process. Basically:

* we have 100% code coverage except for some well-defined exclusions (e.g. // LCOV_EXCL_LINE comments to exclude code that is logically unhittable)

* on each checkin, our continuous integration server (Jenkins) runs the tests and checks that coverage is still 100%

* if not, it fails the build

This means that _every new checkin_ either has unit tests that exercise all
the code, or // LCOV_EXCL comments that make it obvious to a code reviewer
what isn't covered (and they can then sensibly judge the risk). It's a good
way to keep our code quality high, and avoid the possibility that we quietly
skimp on unit tests when under deadline pressure. Also, if new members of the
team don't realise how important good unit test coverage is, they'll find that
out on their first checkin when the build breaks, rather than several weeks
in.

~~~
overgard
With all-due respect to other philosophies than mine, that sounds like a
nightmare. And I like tests. But everything has a cost, and I can't imagine
ever getting anything significant done with such a straight-jacket, I'd spend
all day tracking down weird compiler quirks.

I do a lot of cross platform stuff, and just having warnings as errors (a good
idea) means a lot of build fixing because clang and MSVC disagree on things.
If I were writing code in MSVC and I had to deal with code coverage reports
from clang breaking the build, I would be paralyzed.

"Code quality" isn't a thing that can be trivially measured, and in my
experience people that fixate on certain metrics tend to miss the forest for
the trees.

~~~
rkday
I can see that it wouldn't always be ideal, but on this (admittedly relatively
niche) codebase - core telephone network infrastructure, which we only support
on Ubuntu and which we build with GCC - the kind of clang/MSVC
incompatibilities you talk about aren't a problem, and the reliability
requirements are high enough that it's worth having CI check for 100% test
coverage.

I think it's a question of field rather than philosophy - someone else in this
thread said they do the same thing in avionics.

~~~
Too
Claiming coverage has anything to do with reliability is just fooling
yourself, at least if it's done on line or branch level. Will your 100% line
coverage catch the divide by zero bug in this function?

    
    
        int foo(int x)
        {
            return 1 / (5 - x);
        }

------
bla2
"Build your code with more than one compiler" was good advice 20 years ago,
and is still good advice today :-)

------
feylikurds
Dear God, now I remember why I do not program in C++ anymore (Go and C# all
the way!). That pre-11 code looked awful. If networking was nicer, I might
actually start considering recoding some of my projects in C++.

~~~
dave2000
If you have £40 or so, and a few evenings free, get the C++11 version of the
Stroustrup programming language book and see what's changed. Modern C++ looks
entirely different to older C++. Templates still a bit scary, so learn them or
ignore them.

~~~
feylikurds
Yes, but are there still tons of boilerplate? Or is there a 11/14/17 library
that incredibly decreases the amount of code needed? For example a HTTP
listener with routing?

------
techprotocol
A good way to represent a branch coverage miss.

------
mattiemass
I love stuff like this. Short, understandable.

------
yonayona
please look [http://www.expcov.com](http://www.expcov.com)

