
HoloLens secret sauce: A 28nm customized 24-core DSP engine built by TSMC - runesoerensen
http://www.theregister.co.uk/2016/08/22/microsoft_hololens_hpu/
======
ericseppanen
For those who haven't had the pleasure: developing on Tensilica Xtensa cores
generally means living within 128-256KB of directly-accessible memory; a
windowed register file that makes writing your own exception handlers
"interesting"; a 6-year-old GCC bolted to a proprietary backend; per-seat
licensing fees to use the compiler; and a corporate owner that's only halfway
interested in the ecosystem they now control.

So yeah, kind of wishing it would just die and let ARM take over the embedded
space.

~~~
aylons
> a 6-year-old GCC bolted to a proprietary backend

Not only a violation of the GPL, but for a code owned by the FSF and even
Stallman himself.

That's a bold move. And very douchey.

~~~
the_mitsuhiko
Typically those are creative workarounds for the GPL not actual violations.

~~~
lmm
The question of whether what they've produced is a derivative work of GCC is
one for the courts, which tend to take a dim view of creative workarounds.

~~~
the_mitsuhiko
Absolutely not. The GPL boundary is very clearly defined and intentionally has
never been brought to court.

~~~
lmm
How so? The GPL grants permission to distribute derivative works in general
under the GPL, and additionally to distribute derivative works consisting of
object code that is the compiled form of system libraries (narrowly defined)
without including that source (though AIUI still under the GPL); that's all.
If someone were distributing a derivative work of a GPLed program not under
the GPL and therefore without permission (i.e. without being otherwise
licensed by the copyright holder), that would be copyright infringement. The
question of what is or is not a derivative work in the general case is
absolutely a question for the court. The FSF explicitly says as much:
[https://www.gnu.org/licenses/gpl-
faq.html#MereAggregation](https://www.gnu.org/licenses/gpl-
faq.html#MereAggregation)

~~~
the_mitsuhiko
All of this is irrelevant. The GPL stops at the process boundary and that's
exactly what these compilers do. They provide the modifications to GCC itself
under the GPL and then have separate executables that are proprietary that
feed things in and out of GCC.

~~~
nitrogen
It's still pretty clearly a violation of the intentions of the gcc developers,
and something that only exists to wrap one product seems very suspiciously
like a derivative work. I'd be curious to see this tested in court.

~~~
the_mitsuhiko
But the FSF would not be curious to test this in court because they know they
lose. I had this conversation many years ago with people that wrote extactly
this type of software and the general consensus was that the FSF would stay
away from probing the GPL there because there is angood chance that the ruling
would set a precedent they don't want. The FUD surrounding the license in that
regard is preferrable.

In general it's 2016 and no longer relevant. The world moved on to LLVM and
these GCC using toolchains are on their way out for good.

------
AndrewKemendo
_Holographic Processing Unit (HPU) chip used in its virtual reality HoloLens
specs_

It seems to be insurmountably hard for press to understand that virtual
reality and augmented reality are related but distinct concepts.

Beyond that, interesting teardown. The 10w number is great, but my guess is a
lot of the high power processing happens in the sensor suite because it's
using 4 repurposed IR sensor receiver combos to relay depth data in a highly
structured way. That means this processor is the glue between the IMU and RGBD
camera combo. I think in the end this can't scale down to consumer side with
this approach, not to mention the other hindrances to scaling down with the
protection system.

~~~
brador
Isn't VR just AR with the camera covered?

~~~
sitharus
Not really. VR is an entire immersive environment to itself, it doesn't have
to interact with reality.

AR has to blend in to the environment around, so it's a different class of
problem. It has to locate points in the scene to add things to and project in
a believable manner.

~~~
angersock
Turn off the lights. It's VR.

~~~
sitharus
Not really, turn out the lights and you'll stop seeing AR at all.

VR will, for example, create a room with a table and put a chess game on that
table. This all happens within the VR author's control and designs.

AR has to analyse the scene and find the table, find the angle the table is at
relative to the eyes, compute the position and perspective of the table and
the chess board to look correct on the table.

Both have the same end result - images projected to a user's eyes - but how
they get there is quite different.

------
markingram
HoloLens 3D miniMap of House with Synced Ladies +1
[https://www.youtube.com/watch?v=T-JvTZjbwNs](https://www.youtube.com/watch?v=T-JvTZjbwNs)
GG:)

~~~
rasz_pl
That cracked me up. Is that blue circle a gaze tracker? because it keeps
pointing at virtual boobs whenever he looks at the models :)) not to mention
sending avatar to the kitchen, priceless :D

~~~
shiftpgdn
It is! I feel like gaze tracking is one thing that's missing from VR headsets
that's in the hololens. Social VR won't work until your eye movement and
facial expressions can be translated into the application.

~~~
monk_e_boy
I'm sure gaze tracking would make rendering more efficient, just render the
stuff the user is looking at in high def, the rest in low polygon and low rez
textures.

~~~
Const-me
Unlike head, eyeballs rotate too fast.

Wikipedia says “the peak angular speed of the eye during a saccade reaches up
to 900°/s in humans”.

For reality-like experience, you need to have sub-millisecond rendering
latency. For a moderately-complex 3D scene, current GPUs can’t do anything
close to that.

~~~
rasz_pl
Modern mouse sensors can track ridiculous speeds and accelerations, 6000-10000
frames per second are the norm nowadays. Eye might rotate fast, but your brain
turns off the picture during saccades.

~~~
Const-me
You’re right, low latency sensor is a minor problems. Low latency rendering is
much harder to solve.

A good-looking 3D scene usually takes around 10-15 milliseconds to render. For
the last decades, GPUs were optimized for throughput, not for lower latencies.

One common method how existing VR products achieve lower latencies — they
render in a larger buffer, then shift the result to accommodate for head
rotation since when the rendering was started, then present to the VR headset.

With dynamic LOD based on eye targets this trick will not work. To show better
details at the center, you have to actually re-render your scene using better
LoDs/textures near the view center. And that, my friend, is going to take
10-15ms.

------
eganist
Lay speculation here:

If the first run is on a 28nm process, does this suggest the second generation
on e.g. Intel's 14nm process might yield a drastically more powerful HoloLens
model in the current form factor, or at least a more compact one capable of
all the same (assuming the optics were also compacted) for revision 2?

There's a whole ton of stuff I'm not taking into account such as the physical
size of the current HPU, but my guess is that this is the largest point of
improvement just going by the process alone.

~~~
sbierwagen

      e.g. Intel's 14nm process
    

Intel doesn't let anyone else use their fabs. TSMC has their own fabs, and
their own fab tech.

14nm is outdated, by the way. TSMC says they'll start shipping 10nm parts by
the end of the year:
[http://en.ctimes.com.tw/DispNews.asp?O=HJZ4GC65UYSSAA00NW](http://en.ctimes.com.tw/DispNews.asp?O=HJZ4GC65UYSSAA00NW)

~~~
mappu
_> Intel doesn't let anyone else use their fabs._

I don't think this is true any more.

~~~
eximius
Did they just buy a license for ARM or announce they'd let people use their
fabs for ARM?

------
ex3ndr
Well this explains a lot. In my university (Russia, Mathematic-Mechanic
faculty of SPBu) there are a lot of investments in computer vision stuff and
it it almost impossible to build fast and with low TDP software for CV without
SDP/FPGA/etc stuff.

------
markingram
This article gave a more in-depth look at optics (waveguides) and other parts
of Hololens: [http://www.tomshardware.com/news/microsoft-hololens-
componen...](http://www.tomshardware.com/news/microsoft-hololens-components-
hpu-28nm,32546.html)

------
oaf357
Only 10W for all that? While I'm a little sceptical that is an impressive
number.

~~~
bigiain
Yeah but - 10W all getting radiated inside a single 12x12mm BGA package? Isn't
that a lot? It sounds to me like that'd be running very hot...

I had a project with a Raspberry Pi2 which was only drawing ~2.5W all up
(including the wifi dongle), and it got flaky when the sun shone on the case
and got it a bit warm.

~~~
aidenn0
10W is probably on the high-end for passive cooling of such a package, but
zero problem with a fan.

~~~
imeron
For reference: an Intel Atom (e.g. Braswell) CPU has a TDP of 6W and a pretty
large passive heatsink.

~~~
rasz_pl
also for reference ~10W is Pentium ~100MHz territory. Afair 133MHz was the
first one that required fan on top of the heatsink.

~~~
lacksconfidence
Fwiw my 486dx2-66 had a fan, and when that fan died we noticed because of
corrupted memory crashing the system. Compared to modern CPU coolers it was
downright tiny, but required.

------
cloudjacker
I love to see when vendors actually modify a chip instead of using just off
the shelf parts.

But even then, low bar....

------
frik
Several independent review article about Hololens mention the limited view
(relatively small view angle), underwhelming and off putting. (You can
immediately spot a sponsored article if it doesn't even mention that issue at
all.) iPhone 6 and high end Android smartphones already have augment reality
apps with SLAM technology that is far more impressive than what MS PR is
suggestion Hololens might one day be able to offer - the E3 presentation in
2015 and 2016 were faked as we know in the meantime.

~~~
modeless
> iPhone 6 and high end Android smartphones already have augment reality apps
> with SLAM technology that is far more impressive

This is the most wrong comment I've seen on HN in a long time. The SLAM in
Hololens is incredibly impressive and _nothing_ on _any_ smartphone is _one
hundredth_ as good, with the possible exception of the still unreleased
Project Tango phone.

~~~
roninb
Could you possibly explain for those of us out of the loop as to why?
Otherwise I could simply say:

"This is the most wrong comment I've seen on HN in a long time. The SLAM in
smartphones are the _same_ as the one in Hololens. Project Tango should prove
_even more_ impressive."

And no one would be any wiser.

~~~
mcintyre1994
SLAM stands for "simultaneous location and mapping", it's about being in an
unknown environment which you need to both map and localise yourself in. I'm
familiar with it in the context of robotic navigation, but it seems Hololens
deals with it too - mapping the environment to place holograms, localising
within that environment because you need to be able to interact with them.

Most smartphones simply don't have to do this, they don't have to map their
environments without user input. They don't have to localise themselves within
a map of the environment that they've built themselves. Project Tango does do
AR and environment mapping on a smartphone, so it does deal with the SLAM
problem too.

------
ioquatix
TLDR; It's running Windows 10 :)

------
velox_io
This is a GPU rather the a DSP (the tricky part is rendering not filtering).
Core count is a bit of a pointless metric unless you know how big/ powerful a
core is.

I like the idea of a self contained VR headset with no wires, but I don't
think going to see on for ab a decade, there's just too much processing power
needed for a realistic experience. I hope I'm wrong!

Edit: It appears this is in fact DSP (digital signal processor). That's a huge
amount of power for dedicated signal processing, I'm intrigued to know what
can be done with it.

~~~
T-A
> self contained VR headset with no wires

[http://www.theverge.com/2016/8/16/12503948/intel-project-
all...](http://www.theverge.com/2016/8/16/12503948/intel-project-allow-vr-
headset-cordless-mixed-reality)

~~~
rasz_pl
castAR is also 'pivoting' from computer accessory to self contained unit

