
Looking at CPU/GPU Benchmark Optimizations in Galaxy S 4 - MBCook
http://www.anandtech.com/show/7187/looking-at-cpugpu-benchmark-optimizations-galaxy-s-4
======
enraged_camel
This is not the first time Samsung has been caught in a disgusting lie:

[http://www.techspot.com/news/52274-samsung-admits-to-
posting...](http://www.techspot.com/news/52274-samsung-admits-to-posting-fake-
user-reviews-on-the-web.html)

I don't understand why people keep buying this company's god damn products. I
mean, holy _shit_.

~~~
ajross
Pretty much all major hardware vendors selling into a performance space get
caught in this kind of game. This certainly isn't new behavior, though it's
the first time I've seen a handset vendor try it.

But as far as why people keep buying the products: sadly it's because they're
better. I'm looking at the GS4 (i9505, not the Exynos one) right now vs. the
HTC One and Xperia Z. On paper, they're all basically the same phone (2G quad
core snapdragons with 1920x1080 screens). But all the tiebreakers (really, all
of them) go to the Samsung phone. The GS4 is slightly smaller, thinner, and
lighter yet its screen is slightly larger. It's CPU is clocked higher and its
battery is larger. It's cameras are better (much more so than the HTC,
slightly more so than the Sony). It has a removable battery where the others
don't. It has a sdcard slot where the HTC doesn't. It supports the "T-Mobile"
HSPA+-on-AWS-band frequencies. It has a thermometer, hygrometer and barometer
built-in.

It's just a better phone on everything quantifiable. Not so much so as to kill
the competition, but to anyone paying attention and recommending hardware to
their friends, they're doing an awful lot right.

~~~
uchi
HTC One's camera isn't bad at all. It was built that way for a purpose. You're
doing the straight up Megapixel comparison between the phones.

[http://www.anandtech.com/show/6747/htc-one-
review/4](http://www.anandtech.com/show/6747/htc-one-review/4)

Every phone has its features. It's just at a point where a simple chart-by-
chart comparison won't work for most people anymore. Practically every
reviewer likes the One despite the fact that on paper it's a downgrade to the
S4 in every way. Basically, it is like you said, the devil is in the details,
but bigger is not necessarily better.

Personally, out of those three phones, I'd rather go with the HTC One only
because I like the aluminum body chassis (along with how the screen looks) and
the fact that their stock Android version is 50 dollars cheaper than
Samsung's. The decently sized front speakers are also a plus. I very much
dislike the look of the GS4 and TouchWiz which, in comparison with HTC's
Sense, has less cooler whistles and doodads IMO. But ultimately I like stock
Android the best.

The beats audio chip on the One is a con for me though
([http://www.youtube.com/watch?v=Cdbn_pmxFic](http://www.youtube.com/watch?v=Cdbn_pmxFic)).

I haven't given the Xperia Z much of a chance, but aside from the fact that it
looks aesthetically pleasing to me and I can use it while showering, I don't
see much of an advantage. The GS4 came out with a water and dust resistant
version, but it's slightly bigger and the audio jack is off so it won't fit
most S4 cases. The Z's overlay is much more stock Android (GS4 version is
TouchWiz only). So I'll pick it over Samsung because of that.

Basically, I really don't like TouchWiz. For the most part it just feels
cluttered and half baked and a pain to use.

~~~
ajross
All that is true. But "I like these other phones for personal taste reasons"
is hardly a refutation to "people buy Samsung phones because they have
features they want" either.

I'm not saying you couldn't possibly decide to buy an non-Samsung phone. I'm
responding to a poster who claimed incredulity that someone _could_.

~~~
diroussel
The poster you responded to said:

> I don't understand why people keep buying this company's god damn products.
> I mean, holy shit.

And you responeded that the SG4 had some better metrics than it's competition,
but did no consider what made a better phone. Benchmark apps are one of those
metrics that people consider, and has now shown to be misleading.

Even the higher CPU freq doesn't mean app performance is better, or even that
general user experience is better.

Stop it with the metrics!

------
cscheid
It's Quack3 all over again! [http://techreport.com/review/5226/further-nvidia-
optimizatio...](http://techreport.com/review/5226/further-nvidia-
optimizations-for-3dmark03)

~~~
bobbles
Man, can't believe this was 12 years ago.. History repeats itself

------
MBCook
It's so sad to see this kind of stuff. It's not like the Galaxy S4 is a slow
device. It reminds me of Nokia used a fake video [1] to sell the Lumia 920,
which was supposed to have a pretty good camera. It just ends up being an
unfortunate distraction that causes people to mistrust you.

Ars is right that this isn't new. I remember ATI being caught in the
Quake/Quack thing [2] a dozen years ago.

[1] - [http://www.theverge.com/2012/9/5/3294545/nokias-pureview-
ads...](http://www.theverge.com/2012/9/5/3294545/nokias-pureview-ads-are-
fraudulent)

[2] -
[http://www.hardocp.com/article/2001/10/23/optimizing_or_chea...](http://www.hardocp.com/article/2001/10/23/optimizing_or_cheating_radeon_8500_drivers)

~~~
ngoel36
Even worse - when Microsoft ran Xbox One demos using high end NVIDIA PC's...

~~~
spartango
At least those weren't benchmarks that were part of reviews, where people are
making decisions based on 1-1 comparisons...

------
DominikR
Samsung has responded that the higher GPU speeds are also available certain
other applications that run in full screen. (browser, camera app, video app -
which are all apps that are used frequently)

If that is the case, then I fail to see anything immoral. Bad engineering -
yes - but there's nothing immoral in arbitrarily boosting the GPU to run the
browser or the camera smoother. (and once you regularly do this for some
standard apps, it would also be appropriate to run benchmarks with the fastest
speed)

Link: [http://www.theverge.com/2013/7/31/4574210/samsung-
benchmark-...](http://www.theverge.com/2013/7/31/4574210/samsung-benchmark-
rigging-response)

Edit: I also don't understand why anyone would investigate with a hex editor.
(like Anandtech does)

Samsung is quite open with its Android devices, and regularly releases the
source code (Link:
[http://opensource.samsung.com/](http://opensource.samsung.com/)) It also
would be quite strange to cheat, and then release the source code so everyone
can see.

~~~
diroussel
> I also don't understand why anyone would investigate with a hex editor.
> (like Anandtech does)

The binaries were on the device, but the source code was not, so the binaries
are a natural place to begin the investigation.

Is it ever clear which source from
[http://opensource.samsung.com/](http://opensource.samsung.com/) is on the
device Brian was investigating?

~~~
DominikR
It is absolutely clear which source is on the device because you can look up
Kernel version, Baseband version and the Build number in Settings->About
Phone.

Anyways, what I would have liked to see in this article is if the Intent they
are referring to (the one that triggers the increase of the GPU clock) is only
used for cheating on benchmarks, or actually a feature that is used on many
apps - not just speculation/baseless accusation.

~~~
diroussel
> not just speculation/baseless accusation.

Having read the article, it is clear that it is not baseless speculation.

Two benchmarks running the same code, but with a different name. One runs
faster than the other. The one that runs faster has it's name hardcoded in the
samsung binaries.

You are right that the source code could show a fuller picture, but I'd say
this is an example of good reporting and not speculation.

------
mokus
If the higher clock speeds are truly never used for anything but benchmarks
then that's downright dishonest. But is that established to be the case?
Perhaps those speeds are available for extremely heavy loads under specific
circumstances (wired power, etc) that are rare but can actually occur. If they
are, then this does seem like a fairly reasonable thing to do. Still mildly
shady, but no more so than the norm when it comes to benchmark optimization.

~~~
alec
The AnandTech link in the article says they found the string
"BenchmarkBooster" in the apk alongside the class names of several well-known
benchmarks.

[http://anandtech.com/show/7187/looking-at-cpugpu-
benchmark-o...](http://anandtech.com/show/7187/looking-at-cpugpu-benchmark-
optimizations-galaxy-s-4)

~~~
mokus
The fact that they found a code path that ensures particular programs get the
full hardware speed does not mean that no other code path enables that
performance level. I'm asking whether it is established that there is no other
way that performance level is activated, as it does not seem clear to me from
the article that that is the case.

~~~
deveac
_> The fact that they found a code path that ensures particular programs get
the full hardware speed does not mean that no other code path enables that
performance level._

When those particular programs are benchmarks and the path is coded
specifically to boost performance on a unique set of said benchmarks in
contrast to the most common use cases, that's an issue regardless of whether
or not you can imagine an outlier real-world scenario that utilizes a code
path that tweaks the performance.

You have to bend over backwards to see that as anything other than dishonest.

~~~
Schlaefer
> You have to bend over backwards to see that as anything other than
> dishonest.

"Our new hot device has this awesome performance mode but those benchmarks
aren't aware for it and it won't be utilized in the device reviews. So let's
make sure it's used in these benchmark applications."

If they just flip the switches available to every developer/application it's
not necessarily dishonest just tech marketing.

~~~
joesb
1\. If this awesome performance mode is turned on automatically when the app
needs it. Then it must have turn on automatically for the benchmark without
actually checking the application name, right?

2\. Otherwise, if it require the application to be aware to utilize this
feature, then why hasn't Samsung announce this "mode" to developer to utilize
it yet?

------
sczkid
This is a minor point, but I'm really glad the article title uses "allegedly".
Even if it's quite probable Samsung is guilty here, I'm glad the article
doesn't adjudicate the way so many news articles do.

~~~
megablast
It comes off to me like these sites are scared of Samsung, they are a big
player now, with a lot of pull.

~~~
tadfisher
AnandTech is not one of those sites:

> Today, we are large enough to avoid these petty discussions of withholding
> review samples. Most manufacturers know that one way or another we'll get
> our hands on a product for review and don't try to play these sorts of
> games. Rarely we are faced with a manufacturer or advertiser who is looking
> to influence our content. We have a firm internal policy in place to deliver
> honest, balanced reviews to the best of our ability - regardless of external
> pressures. Fortunately, as I mentioned earlier, we have been around long
> enough and are large enough to avoid this being an issue in the vast
> majority of situations.

[http://www.anandtech.com/home/about](http://www.anandtech.com/home/about)

------
0x0
Note to self: Make sure to use "com.antutu.ABenchMark" as the internal app
name for my next android game.

~~~
JoshTriplett
You probably wouldn't want to, even if you needed the extra performance. You'd
get complaints from users about battery life, device heat, and similar things
that people running benchmarks often don't check for.

~~~
stephengillie
Maybe it could be an option in the settings?

[checkbox] Enable "High performance option"? Caution: will drain battery life
and may cause device to overheat!!

~~~
ygra
You can change tha app's name from within the app?

------
rsynnott
Samsung is merely being retro:
[http://blogs.msdn.com/b/oldnewthing/archive/2012/04/30/10298...](http://blogs.msdn.com/b/oldnewthing/archive/2012/04/30/10298919.aspx)

------
edderly
On mobile devices it would seem sensible to also run your benchmark until the
battery is "flat", it might be a crude heuristic but it probably the best you
can do with product devices where you can't easily break out power rails and
the like.

You should treat benchmarks with a heavy pinch of salt, simply because it's
caught up in the bullshit messaging world of marketing where the aim is just
to produce the bigger or smaller number where appropriate.

The necessity to satisfy marketing ignores a lot of things. Benchmarks often
translate poorly to real use.

Does GLbenchmark 2.7 provide any useful insight into the performance of the
top fifty games in the Android mobile market? Probably not with a few 3d heavy
exceptions.

Some benchmarks will never die, you'll still see Dhrystone trotted out to this
day.
[http://www.eejournal.com/archives/articles/20090602_coremark...](http://www.eejournal.com/archives/articles/20090602_coremark/)

And then you hit a grey area where software can behave badly on one piece of
hardware versus the other because of implementation. So if the benchmark hits
this area, is it a hardware problem or a benchmark software problem?

------
Dwolb
From an EE perspective, benchmark tests should include power consumption and
thermal profile measured by third party, calibrated equipment.

There are some errors induced through the act of measuring, but power
consumption and thermal profile checks would cut down on the amount of CPU/GPU
benchmark tampering.

------
Khaine
Stay classy Samsung

------
fidz
Reminds me about Nvidia with their fake benchmark several years ago

------
simula67
I wonder the extend to which we can trust performance benchmark results about
other closed source software like web browsers, operating systems, servers
etc.

------
induscreep
Anyone know if iPhone or any other phone does this kind of stuff? Downright
disgusting.

~~~
workbench
Since when has Apple relied on spec checklists to sell a product

~~~
corresation
Every iPhone release is a sea of specs: Number of GPU cores, pixel density,
thinnest, lightest, made of raw holistic unobtanium, number of apps in the App
Store, display metrics, etc.

Despite the truly ridiculous claims to the contrary, the iPhone has always
been about specs, where in many cases it is the leader (such as having the
perennial market leading GPUs)

