

Frama-C is a suite of tools dedicated to the analysis of software written in C - nkurz
http://frama-c.com/what_is.html

======
nullc
I like Frama-C a lot in theory, though in practice I found it hard to use
except on very small code segments.

When its unable to prove a range the feedback you get from the solvers is
inscrutable enough that its very hard to figure out what additional data it
would need to satisfy the analysis. (Sort of like parsing C++ compiler
template related errors)

I'd hoped that language features like the typestate stuff that used to be in
Rust would someday make the work required to use sound analysis tools in
production code smaller. I'm not sure if much thought has been given to what
kinds of accommodations languages could give to ease static analysis while
still being programmer friendly.

It seems that newer languages have actually moved away from analysis
friendliness in some respects, however. E.g. in C a signed overflow is always
a bug so if analysis can prove one is possible you have something to fix.
Several modern languages have defined signed operations to wrap and so that
obvious safety test is no longer available. (You could define in your own code
that it should never wrap, effectively writing in a subset of the language,
but as soon as you call into third party code you never know if an overflow
was intended and safe or not— not without extensive analysis)

~~~
pcwalton
> I'd hoped that language features like the typestate stuff that used to be in
> Rust would someday make the work required to use sound analysis tools in
> production code smaller. I'm not sure if much thought has been given to what
> kinds of accommodations languages could give to ease static analysis while
> still being programmer friendly.

Well, since the Rust borrow check is basically a static analysis that's always
on and is required to pass for the compiler to emit code, we've put a lot of
thought into how to make it as programmer friendly as possible. The final
trick that seemed to got it to fall into place was restricting data to only
one mutable reference at all times—this was a restriction that's easy enough
to understand and can be readily explained through error messages and
tutorials. There's still a learning curve, of course, but I think we've got
the system down to a reasonable level.

~~~
nullc
Right, the borrow checker does great things. But memory safety is only one
aspect of program correctness— though a universal and very important one.

If a program does anything important, if it handles secrets, if it enforces
invariants ("the robots arm must not be out of the safe area"), etc. then
other aspects of the programs logic may be just as safety critical as memory
safety. Frama-C can prove other things about program saftey: this calculation
cannot overflow, this error state can never be reached, this operation will
complete within X steps... even if the results of non-safety don't result in
anything to do with memory errors.

Some of the programmers expectations can be extracted from the program— signed
integers won't overflow, loops (unless otherwise annotated) will always
terminate, pointers will not be extended outside of their objects, a pointer
to an integer won't be called as a function pointer, behavior will not depend
on the order function arguments are evaluated in, etc. Some of the
expectations must be expressed as assertions.

Once as many of the programmers expectations are understood the analysis needs
to know the invariants that allow them to be true. Some of these can be
extracted, some must be asserted.

The gap that exists today is that a lot that could be extracted (e.g. by
another programmer reading in a context free way, with high probability of
success) can't be soundly extracted because there is a lot of technically
valid behavior which is just usually unlikely to be what anyone intended. This
means that anyone attempting to apply tools like frama-c has to spend a lot of
time making the implicit assumptions explicit with assertions. Thats why I was
saying that something like making singed overflow defined may be a step
backwards because it takes a whole class of "you couldn't possibly have
intended this behavior" back to "maybe you meant this".

~~~
pcwalton
Sure, I totally agree that there are many other invariants that you might like
to prove. Many of them require full-on dependent typing—not going there for
now is a choice we made to make the language approachable. (See your sibling
comment for how difficult it is just to get memory safety!) But I completely
agree with you that there is more to be done.

------
TorKlingberg
It would be good to hear other HN:ers experience with various static analysis
tools for C.

I have had good experiences with Flexelint (PC-Lint). It does not attempt to
deeply analyze control flow, more like compiler additional warnings. It flags
a lot of common mistakes and can basically turn C into a more strictly typed
language. I feel a lot more confident in C code if I know that it passes lint,
since it warns if you try to mix unsigned and signed ints, cast away const,
call functions with wrong types etc.

Like many static analyzers it takes some work to set it up, and tune which
warnings you actually car about. It is definitely business-priced and feel a
bit old (although command line tools age well.

The is a clear lack of good open source tools. I tried all i could find, but
Splint was the only one that would flag switch-cases without break. It was
last updated in 2010.

------
eliteraspberrie
One feature you can start using right now: prove an assertion with the WP
plugin. [1] It's especially useful to prove the absence of some undefined
behaviour, bounds on variables, or invariance conditions.

[1] [http://frama-c.com/wp.html](http://frama-c.com/wp.html)

------
xvilka
It is very handy -
[http://xvilka.me/frama-c.png](http://xvilka.me/frama-c.png) \- screenshot of
the integer overflow found.

~~~
z3phyr
I really like your config :)

------
rhth54656
The only option for windows is manual compilation using a POSIX library.

Any suggestions for a similar tool?

~~~
conductor
Cppcheck -
[http://cppcheck.sourceforge.net/](http://cppcheck.sourceforge.net/)

~~~
rhth54656
Thanks, I just tried it and it found an assignment in an assertion that the
compiler didn't warn because it wasn't an if statement.

I ran it over a couple of old finished projects and it found a problem in each
one. Awesome.

Unfortunately it misses a class of errors memcpy would do,( similar to
heartbleed ) , if copy size is determined at runtime.

------
glifchits
What would this say on the OpenSSL code that had the Heartbleed bug? (Edit)
this:
[https://news.ycombinator.com/item?id=7571506](https://news.ycombinator.com/item?id=7571506)

~~~
rurban
Frama-C does not say much, when do don't tell it what it should do. It needs
to reason over symbolic facts, C code is not enough. It's not just another
simple static code checker.

You can look at the Frama-C blog what it does with a major openssl competitor,
polarssl. [http://blog.frama-c.com/](http://blog.frama-c.com/)

