
27000 errors in the Tizen operating system - matthewwarren
https://www.viva64.com/en/b/0519/
======
edejong
This sounds so familiar. Some 6 years ago, I joined a team of engineers
developing a large (1.5M LOC) classic java/spring/hibernate application for
administration in higher education. The software was buggy and loads of time
was wasted on bug-fixing, so I introduced the Findbugs static analysis
package. Result: >6000 (suspected) bugs. It took a couple of days of blunt
perseverance of the whole team to clean up the mess and bring it back to
normal proportions, but the result was a stable, well-functioning system.

If you have static analysis tools available and you're writing a larger
application (more than 10k LOC), and intend to maintain it for longer than a
couple of months, it is definitely worth the initial investment. Better than
static analysis, is static typing in a language with an expressive type-
system, such as Scala. You simply cannot have NPEs with Option types.

tldr; Used static analysis on 1.5M LOC codebase to reduce bugginess
tremendously. Use of static analysis and statically typed languages is
adviced.

~~~
blub
The first time we ran static analysis on the C++ code base of a specific
project, it found ~20 minor issues and a couple of medium severity ones which
were mitigated by other checks. However, we also used 3rd-party open source C
libraries, where it found thousands of problems. I'm talking about serious
issues, memory leaks, invalid memory access, etc.

P.S: I hope Scala implemented optional types better than Java, where the
optional types themselves are nullable :)

~~~
edejong
Well, technically you could write `val opt: Option[Int] = null`, but I've
never seen it used and would seriously consider sending the author back to a
course Scala for Dummies.

It's interesting how many bugs can be found by static analysis. The problem
with C and C++ is that the domain of 'normal' operations is much larger than
Java, leading to more false positives in static analysis and/or less true
positives.

------
sliken
With Samsung's android record, why would anyone want more Samsung written code
on their phone?

I had a nexus 5, reliable, reasonably fast, snappy under most normal usage
patterns.

I wanted a SD card and wireless charging so I upgraded to a note 4. 50% more
ram, faster cpu, faster GPU etc. Felt way slower, and crashed fairly often.
Annoyingly the home button was quite sluggish. Even more annoying is despite
50% more ram it was very aggressive in killing background apps, making
multitasking almost useless. So whenever you switched apps it would have to
restart the app, MUCH more so than my nexus 5. I tracked it down and
apparently it's some benchmark optimization. Android updates (security or
otherwise) were quite rare and was about a year behind on the android
releases.

I switched the note 4 to cyanogen. The phone was much snappier, much more
consistent (no random lag), and stopped crashing. Oh and the home button was
much quicker as well.

~~~
matthberg
I had a similar experience with a Samsung Galaxy S4, which would run out of
battery life in under 5 hours on any given day, and which was dropped for
major updates after two versions. Switching to Cyanogenmod, even the _nightly_
version extended battery life to ~9 hours, and greatly improved performance.
It also allowed access to the next version of Android, and native skinning and
easy customization (with a literal toggle in the dev settings for root).

In short, I highly recommend upgrading any old hardware you have to Lineage OS
(Cyanogenmod continued).

------
geofft
Every time I see a PVS-Studio article I wonder how many errors are caught by
the compiler itself. In my experience, things like the same expression on both
sides of a less-than sign, returned pointers to local variables, are caught by
-Wall -Wextra in GCC or LLVM.

If they're not building with -Wall -Wextra -Werror, are they really going to
make use of PVS-Studio's output?

(Also, as a member of the Rust Evangelism Strike Force, I am obligated to
point out that a lot of errors PVS-Studio catches are just straight-up not
possible in Rust. In order to avoid getting too off-topic here, here's a
comment I made on Reddit on a similar post from PVS-Studio about two years
ago:
[https://www.reddit.com/comments/3aehq5//cscj2w5/](https://www.reddit.com/comments/3aehq5//cscj2w5/))

~~~
AndreyKarpov
> If they're not building with -Wall -Wextra -Werror, are they really going to
> make use of PVS-Studio's output?

This is very likely. Do not forget that the static analyzer is not just a
warning. This is also the infrastructure.

For example, PVS-Studio is:

\- Saving and loading analysis results allow doing overnight checks - during
the night the analyzer does the scanning and provides you with the results in
the morning.

\- Interactive filtering of the analysis results (the log file) in the PVS-
Studio window: by the diagnostic number, file name, the keyword in the text of
the diagnostic.

\- BlameNotifier utility. The tool allows you to send e-mail notifications to
the developers about bugs that PVS-Studio found during a night run.

\- Mass Suppression - ability to suppress all old messages raised for the
legacy code, so that the analyzer reports 0 warnings. You can always go back
to the suppressed messages later. This feature allows you to seamlessly
integrate PVS-Studio into your development process and focus on errors found
in new code only. Details:
[https://www.viva64.com/en/m/0032/](https://www.viva64.com/en/m/0032/)

\- Relative paths in report files to view them on different machines.

\- CLMonitoring feature allows analyzing the projects that have no Visual
Studio files (.sln/.vcxproj); in case the CLMonitoring functionality is not
enough, there is a possibility to integrate PVS-Studio in a Makefile-based
build system manually.

\- pvs-studio-analyzer - a utility similar to CLMonitoring, but working under
Linux.

\- Possibility to exclude files from the analysis by name, folder or mask; to
run the analysis on the files modified during the last N days.

\- Integration with SonarQube. It is an open source platform, designed for
continuous analysis and measurement of code quality.

\- and so on

~~~
virtualized
I don't remember reading about any of that even though I am a regular reader
of your "checked projects" series. All of the relevant information on your web
site is hidden in some wall of text. Please consider hiring someone who knows
how to sell something.

~~~
JdeBP
It isn't hidden, really. I followed the hyperlink at the top to the main page
of the site, and the bullet-point list that you are responding to is right
there, on that very first page. It is under a heading entitled _Main Features
of PVS-Studio_.

* [https://www.viva64.com/en/pvs-studio/](https://www.viva64.com/en/pvs-studio/)

~~~
virtualized
"Main Features of PVS-Studio" is one of those walls of text that I meant.

------
coldtea
Relevant:

[http://appleinsider.com/articles/17/04/05/researcher-
calls-s...](http://appleinsider.com/articles/17/04/05/researcher-calls-
samsungs-tizen-os-the-worst-code-ive-ever-seen)

[https://arstechnica.com/gadgets/2017/04/samsungs-tizen-is-
ri...](https://arstechnica.com/gadgets/2017/04/samsungs-tizen-is-riddled-with-
security-flaws-amateurishly-written/)

------
kronos29296
The number of errors seem like pylint giving me errors. Some of them were like
a is not a valid variable name. Maybe the numbers are like that. (Pylint that
is not configured sanely/with default config really gives you thousands of
errors and warnings for a fairly large code base and some of them were not
errors but coding style.)

The numbers here may also be something like that. Nice blog post, may even
make developers adopt something like cppcheck but most guys won't buy this.

~~~
loeg
It's a lot like that. PVS-Studio ran against FreeBSD and produced a ton of
false positives.

~~~
jotux
Running any sufficiently advanced static analyzer on any large codebase will
generate a ton of false positives.

~~~
loeg
So what? Qualitatively, it feels like PVS-Studio's false positive rate is
higher than Coverity. And far higher than a modern free compiler's -Wall.

To describe the 27,000 detections as errors is a little misleading, and I
think that's what the grandparent post is discussing.

~~~
AndreyKarpov
This is not the way you think and it is not nice to mislead people. It all
depends on the project.

I've heard people who said that Coverity gives many false positives and
Cppcheck gives few. I've heard the opposite, that it is impossible to use
Cppcheck because of the huge number of false positives, but Coverity is doing
great. I heard that Coverity gives more false positives than PVS-Studio. And
vice versa. And so on and so forth. What is the reason for such differences?

Projects have their certain styles of writing and different kinds of macros.
These are macros and peculiarities of style that become the main source of
false positives. This is why the first impression of using the analyzers of
code depends on luck, and not on the coolness of the analyzer. If the analyzer
doesn’t like a self-made my_assert() it will issue 10000 false positives.

So there is no point in talking abstractly about the number of false
positives. Yes, you can not get lucky and there can be a lot of false
positives. However, the static code analyzers allow you to configure them. In
articles I have showed many times that the simplest configuration of the
analyzer can greatly reduce the number of false positives. Example:
[https://www.viva64.com/en/b/0496/#ID0ENNAC](https://www.viva64.com/en/b/0496/#ID0ENNAC)

------
wyldfire
> As you can see, the calculations are absolutely fair and transparent.

...if you can grant them the premise that the 3.3% sampled represents the
project as a whole so that the ratio will project out to 27k overall.

~~~
ChrisLomont
Sampling 3.3% of lines in a 72 million line codebase, especially across so
many sub projects, is statistically significant enough to make claims like
this with decent certainty.

That's a few million sample lines tested.

~~~
ordu
Oh... We can ask about way sub projects were selected, and are they can be
called a representative sample.

Nevertheless I agree with you: there are no obvious way to do a better
estimation, so words "fair" and "transparent" seem right in place there. Of
course only if Karpov does not try to mislead his readers deliberately by
choosing sub projects by some criterion that correlates with error rate.

------
lvillani
The mere mention of EFL brought this article back to my mind:
[https://what.thedailywtf.com/topic/15001/enlightened](https://what.thedailywtf.com/topic/15001/enlightened)

The author makes no names but ends the article with "[...] did I mention EFL
is the basis of all applications on Tizen?".

I don't know whether EFL is still the base for all applications on Tizen (I
think they changed technologies several times, they are now on .NET?) but
that, plus the SDK churn and Samsung reputation of generally low software
quality and severe security issues don't really instill confidence in the
platform.

~~~
gue5t
I've read a lot of Tizen code, and EFL is by far one of the more professional
and well-engineered components. And this is realizing that in relation to many
open-source libs EFL is rather a goofy oddball.

95% of Tizen is a huge hairball of needless nested and overlapping
indirections (custom IPC layers, serialization formats, "management" daemons,
libraries to interface with them, etc.) implemented in varying (except for the
pervasive messiness and broken english) C styles; deep beneath all this you
have the open-source libraries doing the actual useful work. You could try to
make some similar claims about Android, but the percentage would be much lower
and the code quality is much higher. I wish I had some real metric to point at
(maybe this article suffices?) but the code in Tizen is truly awful.

Also they don't give a shit about their bug tracker; I've had system-breaking
bugs, with patches, in "new" status for over a year. Because I don't work at
Samsung, I can't contribute.

------
irundebian
I'm dreaming of a language which prevents bugs a priori so that there is no
need for static analysis. I know Rust is out there, but I'm not sure if all
state of the art knowledge is getting into the language specification. Is
there some programming language geek here which could confirm that this is the
case and all safety features (except these with non-optional run-time trade
offs) of Ada are included?

------
exabrial
Samsung doesn't have a brilliant track record for security...

~~~
tw04
And yet they want to be the go-to IOT company... kind of scary.

~~~
ethbro
Security is invisible.

------
AndreyKarpov
Continue. Exploring Microoptimizations Using Tizen Code as an Example -
[https://www.viva64.com/en/b/0520/](https://www.viva64.com/en/b/0520/)

------
chris_wot
Those alloca issues are a real concern! Especially given the fact that there's
no way of knowing with any certainty that it has failed, as if it fails then
it's not specified what behaviour will occur.

~~~
syncsynchalt
The first alloca (alloc in a loop) is in a test, which makes it less
concerning IME.

The second (free() of a stack pointer), oof. Depending on the malloc
implementation, you'll get a noop, a crash, an abort, a warning to stderr, or
worst of all silent munge of stack vars.

~~~
chris_wot
Yeah, but you also don't want to allocate too many of these and smash the
stack.

------
joshmarinacci
I will never buy a Samsung product that includes software written by them.
They simply don't care about quality.

~~~
chambo622
Do any Samsung products not contain software written by them?

------
AndreyKarpov
Tizen: Summing Up -
[https://www.viva64.com/en/b/0522/](https://www.viva64.com/en/b/0522/)

------
HillaryBriss
are there similar security analyses of android OS source?

------
based2
[https://sonarcloud.io/projects?sort=-analysis_date](https://sonarcloud.io/projects?sort=-analysis_date)

------
analogmemory
It's on my "smart TV" and there's a 50/50 chance that when I turn on the TV it
crashes. On the plus side it's helping me stop watching so much TV.

~~~
rsynnott
This sort of thing is exactly why I'm very unwilling to buy a smart tv. It's
getting harder to find decent dumb ones, though.

~~~
vanderZwan
Why not just get a big computer screen?

~~~
rsynnott
Nobody makes an acceptably priced 40-50" computer screen that I can put in my
living room.

~~~
hengheng
Dunno, Iiyama has a decent 40" 4k for 400-500€ that does the job for me.

~~~
ProAm
I need almost double that size unless I want to watch TV like a peasant....

------
adekok
We can PVS-Studio against FreeRADIUS. It found things which were missed by
Coverity, clang analyzer, and cppcheck. Sadly, they don't respond to my
queries about _paying_ for it.

[http://freeradius.org/security/pvs-
studio.html](http://freeradius.org/security/pvs-studio.html)

------
virtualized
Yes, PVS-Studio is very good and useful for avoiding bugs.

No, companies don't really care about avoiding bugs. No one at Samsung will
risk delaying the release of the next plastic crap product by fixing any
problems. Why would they? Defects will even encourage customers to buy a new
device because they are used to getting no bug fix updates. This is the worst
possible company for Viva64 to go to.

It seems like C++ developers either don't give a shit about code quality or
they do, and don't make many mistakes anymore. Either way there is little
demand for a static analyzer. It won't get easier for Viva64 in the future. A
few years ago, Compilers and IDEs started providing similar warnings as PVS-
Studio, although not quite as sophisticated yet. Code-level testing begins to
be a thing in proprietary software. Clang offers valgrind-like sanitizers for
different classes of bugs that even PVS-Studio cannot detect.

TIL that Viva64 has over 20 employees. The passive-aggressive blog posts
always made it look like one or two people running PVS-Studio as a side
project or so. I read those posts regularly, but I don't remember them
announcing any new features or improvements in PVS-Studio itself. Why do I
have to pay annually if they don't spend the money on improving the product?

I am not even sure if PVS-Studio is worth the money. They don't have prices on
the web site and defend their business decisions in the FAQ
([https://www.viva64.com/en/order-faq/](https://www.viva64.com/en/order-faq/))
in a very unprofessional way. After reading that it feels like I might get
ripped off.

~~~
AndreyKarpov
> They don't have prices on the web site

This is standard practice. PVS-Studio is a B2B solution. There are many
details to be discussed.

For individual developers we propose the following: "How to use PVS-Studio for
Free" \-
[https://www.viva64.com/en/b/0457/](https://www.viva64.com/en/b/0457/)

And "Handing out PVS-Studio Analyzer Licenses to Security Experts" \-
[https://www.viva64.com/en/b/0510/](https://www.viva64.com/en/b/0510/)

~~~
virtualized
"How to use PVS-Studio for Free" is ridiculous.

"Handing out PVS-Studio Analyzer Licenses to Security Experts" \- sorry, tl;dr

