
'Human brain' supercomputer with 1M processors switched on for first time - theaeolist
https://www.manchester.ac.uk/discover/news/human-brain-supercomputer-with-1million-processors-switched-on-for-first-time/
======
llcoolv
"‘SpiNNaker’ machine is capable of completing more than 200 million million
actions per second, with each of its chips having 100 million moving parts."

Could someone elaborate? I am probably missing something as I hadn't heard of
moving parts on a solid-state device?

~~~
rm999
I believe they are referring to the number of transistors per chip (which as
you correctly state are not "moving parts"). Also, fun fact: these chips are
7+ years old now!

[https://ieeexplore.ieee.org/document/6330636](https://ieeexplore.ieee.org/document/6330636)

~~~
wallace_f
The supply side of people who want to do journalism is full of so many people,
and the competition for jobs sends most to other career fields. Yet the
product of modern journalism is often so sad.

~~~
llcoolv
This is what caught me off guard - I also share this view on journalism and
would not have been surprised, but this piece is published by the Uni of
Manchester itself.

~~~
stochastic_monk
The press release office, as opposed to the researchers.

~~~
dkfellows
Our press office are definitely a law unto themselves, but the analogy was
drawn in the event. It's also inaccurate, as it doesn't count the extra
~billion gates per chip for the memory (as that's just a co-packaged standard
SDRAM module). It's just that we don't usually think about those as those
SDRAMs are very reliable and don't cause us much trouble at all.

------
PopePompous
From the article: "The newly formed million-processor-core ‘Spiking Neural
Network Architecture’ or ‘SpiNNaker’ machine is capable of completing more
than 200 million million actions per second, with each of its chips having 100
million moving parts."

I kinda doubt that.

~~~
mr_overalls
Science journalism is generally just awful.

I seriously think that a major reason for the lack of public scientific
literacy (I get an earful of anti-evolution, anti-climate change, anti-Big-
Bang crap from my conservative family) is the lack of a competent institution
for communicating these truths.

Outside of scientific journals, and the occasional pop-sci bestseller, the
average person has to rely on university press releases, bloggers, and
magazine writers - and they generally seem to be terrible at their jobs.

~~~
umvi
> anti-evolution, anti-climate change, anti-Big-Bang crap from my conservative
> family

At least anecdotally, my conservative family is like this because those topics
have been used in the past to attack/bludgeon their religious beliefs which
leads my family to dig in their heels, double down on their beliefs, and close
their minds to accepting them.

I've made a _lot_ of progress by instead showing how those things are not only
not anti-religious topics, but quite the opposite - they bring us closer to
the truth of how God accomplishes what he does. Once they feel that their core
beliefs are not being threatened, but merely augmented, it's much easier to
accept them.

~~~
darkmighty
Honestly, scientific literacy does jeopardize the "personal God" concept, and
in particular obviously the Christian mythology. There are still plenty of
christian scientists though, some more keen on the moral teaching and
community aspect of the religion, some interpret the texts as non-literal (a
tiny minority I presume cling to literal texts).

I mean, I'm all for more engagement and scientific literacy, but let's not
pretend there isn't some conflict here; specially for the more hardline
churches -- while catholicism in particular seems happy to transition its role
(into important lessons and social support) and turn dogma into allegory.

~~~
reaperducer
_let 's not pretend there isn't some conflict here_

Science and religion can get along just fine.

Half of the craters on the moon are named after priests. "Cells" are named for
the places where monks live. The Vatican has operated scientific observatories
for centuries. There's even one in Arizona that makes important discoveries to
this day.

The whole science vs. religion meme is something that the internet has
amplified out of proportion by latching onto fringe groups and smaller
denominations and holding them up as the only kind of religion that exists,
creating artificial levels conflict to justify their position.

~~~
mr_overalls
> Half of the craters on the moon are named after priests.

Sure, religious folks often don't have a problem with simple empirical
observations (I mean, the Catholic church _did_ get around to pardoning
Galileo 350 years after they murdered him).

But Lord help you if you apply the scientific standards of empirical rigor or
explanatory parsimony to any topics further afield. Most obviously, there
isn't a shred of evidence - none whatsoever - for the existence of a divine
being along the lines of what's posited by the Abrahamic religions.

~~~
dragonwriter
> Sure, religious folks often don't have a problem with simple empirical
> observations (I mean, the Catholic church _did_ get around to pardoning
> Galileo 350 years after they murdered him).

There's a lot of people you can reasonably argue were murdered by the Catholic
Church, but Galileo isn't one of them.

~~~
joelhoffman
GP was probably thinking of Giordano Bruno... Galileo died of old age under
house arrest after being convicted of heresy.

------
YeGoblynQueenne
Article author:

>> The world’s largest neuromorphic supercomputer designed and built to work
in _the same way a human brain does_

Project lead:

>> We’ve essentially created a machine that works _more like a brain_ than a
traditional computer

Press releases, ladies and gentlemen.

------
marmaduke
I went to a code jam with some of these guys (as part of the Human Brain
Project), the architecture is pretty interesting but it's lots and lots of
little ARM (v6?) processors on a grid interconnect, probably not too far from
Xeon Phi, even if it aims for neural like computation more so that a Phi.

~~~
gnufx
Definitely not like Phi, which is basically a SIMD computational system
(linear algebra box?) plus interesting memory configuration, at least in KNL.
The SpiNNaker chips have no hardware floating point.

~~~
dkfellows
The next generation will have single precision hardware floats, but that's
still at the prototype stage (with little bits of the processor running on a
monster FPGA in the lab).

The key however is that SpiNNaker is a MIMD system (the cores are really
independent of each other, except for a shared clock and chip-level shared co-
packaged SDRAM) with a very fancy fast multicast interconnect that's been
tuned for handling small source-routed packets without guaranteed delivery
(but with guaranteed _detection_ of failure to deliver). It's the almost
complete antithesis of MPI, and it is by using that well that we get great
performance in neural simulation. (I'm a software developer on the team.)

~~~
gnufx
I haven't had a chance to go back and read the literature or talk to people
more deeply, but what I've heard about SpiNNaker recently in conversation and
semi-technical talks has been confusing when it comes to comparisons. The
distinguishing features as presented are things I expect of large HPC systems.

I don't mean SpiNNaker isn't interesting, and I've been pointing it out as
such for years but it's been basically unknown even relatively locally.

~~~
dkfellows
It's basically very different in approach to many modern computers. The cores
are slow and low-powered, but the interconnect is very fast for routing small
packets to multiple destinations, which means that computational tasks that
would otherwise be utterly dominated by communication costs (e.g., neural
simulations) become a lot more tractable.

But since it's all done in soft realtime with very low level code (and no
hardware floats in the current hardware generation) and not much of an OS,
it's a very unusual platform for people to work with. Much more like
programming used to be like in the 1980s, if my memory serves me right. (One
of the key distinguishing things about SpiNNaker in the field of neuromorphic
systems is that actually has an OS at all. Most competitor systems are purely
bare metal, as they're put together by deep hardware hackers without
consulting software engineers.)

------
JoeAltmaier
Does that add up? Doesn't the human brain have several more zeroes on there?
Maybe 'worm brain' or 'lizard brain' would be a better description?

~~~
delecti
It's about 100B neurons in a human brain [0], but neurons are also _much_
slower. Individual neurons run at under 200Hz [1], which is probably a half
dozen orders of magnitude lower than these processors are running at. Maybe it
balances out?

[0] [https://www.verywellmind.com/how-many-neurons-are-in-the-
bra...](https://www.verywellmind.com/how-many-neurons-are-in-the-
brain-2794889) [1]
[https://en.wikipedia.org/wiki/Neural_oscillation](https://en.wikipedia.org/wiki/Neural_oscillation)

~~~
amelius
> Individual neurons run at under 200Hz

So anything your brain can think in 1 second needs at most a depth of 200
layers.

~~~
Retric
Each neuron operates on it's own clock cycle which messes with these
calculations.

------
peterlk
This article was pretty low on details, so I went out and collected some
links:

This looks more or less like another stab at the Cray connection machine[0],
but with modern hardware and a better framework about how neural nets can and
should work.

> The SpiNNaker engine is a massively-parallel multi-core computing system. It
> will contain up to 1,036,800 ARM9 cores and 7Tbytes of RAM distributed
> throughout the system in 57K nodes, each node being a System-in-Package
> (SiP) containing 18 cores plus a 128Mbyte off-die SDRAM (Synchronous Dynamic
> Random Access Memory). Each core has associated with it 64Kbytes of data
> tightly-coupled memory (DTCM) and 32Kbytes of instruction tightly-coupled
> memory (ITCM). The cores have a variety of ways of communicating with each
> other and with the memory, the dominant of which is by packets. These are 5-
> or 9-byte (40- or 72-bit) quanta of information that are transmitted around
> the system under the aegis of a bespoke concurrent hardware routing system.
> [1]

So, lots of relatively relatively tiny, interconnected nodes.

They built their own SoC to handle this. With a built-in router in the middle.
The router handles routing on the chip, and multicasts to its neighbors.

> The heart of the communications infrastructure is a bespoke multicast router
> that is able to replicate packets where necessary to implement the multicast
> function associated with sending the same packet to several different
> destinations. [2]

It also looks like they're developing dev boards [3]

So basically, this looks like a giant, really awesome, custom ARM cluster that
they want to do neural network stuff with.

If anyone from the team is here, I'd love to hear more about how this will be
used. Specifically, how will you prevent SpiNNaker from going down the same
path as the Connection Machine - (stops doing AI stuff because, say,
geneticists want to use it for protein sequencing)? Why do you see this as the
future over something like NVIDIA's new HGX-2 or clusters of TPUs?

[0]
[https://en.wikipedia.org/wiki/Connection_Machine](https://en.wikipedia.org/wiki/Connection_Machine)

[1]
[http://apt.cs.manchester.ac.uk/projects/SpiNNaker/architectu...](http://apt.cs.manchester.ac.uk/projects/SpiNNaker/architecture/)

[2]
[http://apt.cs.manchester.ac.uk/projects/SpiNNaker/SpiNNchip/](http://apt.cs.manchester.ac.uk/projects/SpiNNaker/SpiNNchip/)

[3]
[http://apt.cs.manchester.ac.uk/projects/SpiNNaker/hardware/i...](http://apt.cs.manchester.ac.uk/projects/SpiNNaker/hardware/index3.php)

~~~
dkfellows
> Specifically, how will you prevent SpiNNaker from going down the same path
> as the Connection Machine - (stops doing AI stuff because, say, geneticists
> want to use it for protein sequencing)?

I'm on the team. I can say that we're specifically funded to do and support
computational neuroscience. However, if someone comes along with money wanting
to do other kinds of projects on the hardware (and are able to handle the
special characteristics of the machine itself) then they're welcome. The
challenge is that it's non-conventional in a number of ways that make porting
code tricky: in particular, the messages are small, designed to be multicast
rather than unicast, the instruction memory per core is only 32kB, and there's
no hardware floating point at all in the current generation. (You can do
floating point in emulation. We do that in one of our projects.)

> Why do you see this as the future over something like NVIDIA's new HGX-2 or
> clusters of TPUs?

We see it as solving different problems. Those approaches you mention are
great for solving problems that resolve to big matrix operations; SpiNNaker is
better at tackling problems that are dominated _in terms of description_ by
communication. Neural simulations are really just vast hybrid ODE systems, but
where it is possible to break up the simulation into lots of communicating
domains (the synapses and neurons).

~~~
peterlk
I'm a bit late in checking up on this, but just wanted to say thanks for
taking the time to answer some questions

------
shshhdhs
> capable of completing more than 200 million million actions per second

> To reach this point it has taken £15million in funding, 20 years in
> conception and over 10 years in construction, with the initial build
> starting way back in 2006.

Wow, those numbers.. and 10 years to build... I’d be very excited to turn it
on!

~~~
Aeolun
15 million in funding for 1 million processors? So every processor cost only
15£? That sounds pretty good to me.

~~~
tyingq
_" The chip is a Globally Asynchronous Locally Synchronous (GALS) system with
18 ARM968 processor nodes"_[1]

So, 18 cores per chip. 55,556 chips. £270 per chip.

Still pretty good, considering the £15M is paying for more than a pile of
chips.

[1]
[http://apt.cs.manchester.ac.uk/projects/SpiNNaker/SpiNNchip/](http://apt.cs.manchester.ac.uk/projects/SpiNNaker/SpiNNchip/)

~~~
dkfellows
The unit cost of a chip isn't too much. The price of doing the design and
creating the masks for the silicon OTOH...

------
EamonnMR
> Biological neurons are basic brain cells present in the nervous system that
> communicate primarily by emitting ‘spikes’ of pure electro-chemical energy.

I don't think that those are terms of art.

------
nyrulez
This article, while obviously PR, is confusing to me. It seems to pin too much
emphasis on the nature of its hardware as if that's enough to "emulate" the
brain. If they don't have folks at the calibre of Deepmind to drive this
thing, can it really go very far?

What it could be useful for is neural structure modeling at a more primitive
layer, even if the end outcome isn't usable for practical consumption.

~~~
dkfellows
SpiNNaker is particularly for studying neural structure, especially on the
scale from small groups of neurons up to brain structures of a few million, on
timespans of a few seconds to a few hours at simulation timesteps on the size
order of a millisecond. That's a scale where doing the study _in vivo_ or _in
vitro_ is technically extremely challenging; signal analysers find that
awkward, either to get that many channels that fine or to get that density of
sampling. Or both; getting either is hard and getting both is crazy hard.

But being able to simulate neural networks that can do their learning on-line
and in real time, all while actively processing input (and in a controlled
fashion) is an interesting capability anyway, as it means SpiNNaker can
control physical robots in interesting ways (and those may be commercially
interesting). And it's low-power enough that doing this in the wild is
practical, rather than needing to upload everything into the Cloud for
analysis. That may also be commercially interesting.

------
bra-ket
Furber's lab built an interesting extension on SDM with N-of-M rank codes
[http://apt.cs.manchester.ac.uk/ftp/pub/apt/papers/sbf_TNN07_...](http://apt.cs.manchester.ac.uk/ftp/pub/apt/papers/sbf_TNN07_old.pdf)

------
yters
Some are saying this is no where near the amount of processing the human brain
has. But, it doesn't seem we need that much of the brain. There are plenty of
stories of highly functional and talented individuals with almost no brain,
and this machine would probably be in their ballpark.

~~~
neuromantik8086
I mean, a lot of the brain is devoted to sensation, so if you don't care about
simulating how the brain interprets certain aspects of sensation (motion,
depth, vision more generally) you could probably simulate other functions. For
memory, however, at least, there's a lot of evidence that you'll need to
simulate sensory systems to be able to accurately simulate recall [0].

[0]
[https://books.google.com/books?id=VjZyDwAAQBAJ&pg=PT597&lpg=...](https://books.google.com/books?id=VjZyDwAAQBAJ&pg=PT597&lpg=PT597&dq=cortical+reinstatement&source=bl&ots=raguZOS29j&sig=h9d66-nczgKMdii_U1ZWs_tuVIQ&hl=en&sa=X&ved=2ahUKEwichN6K7L3eAhXIVN8KHWTzD4w4FBDoATADegQIAxAB#v=onepage&q=cortical%20reinstatement&f=false)

------
melling
Steve Furber did some early design work in the ARM processor:

[https://en.m.wikipedia.org/wiki/Steve_Furber](https://en.m.wikipedia.org/wiki/Steve_Furber)

------
drannex
Happy to see Stanford R. Ovshinskys dreams being created, even thought it is
years past his death.

------
mrec
I'm getting NET::ERR_CERT_AUTHORITY_INVALID with Chrome - am I the only one?

~~~
tempay
No issues for me with firefox and everything looks in order

[https://www.ssllabs.com/ssltest/analyze.html?d=www.mancheste...](https://www.ssllabs.com/ssltest/analyze.html?d=www.manchester.ac.uk&latest)

~~~
mrec
Happens with IE too - suspect it's caused by a corp-installed Symantec
extension. I'll try again from home.

Thanks for checking!

------
qwerty456127
And what did it say?

~~~
gmuslera
Yes, now there is a God.

[http://thepequodblog.blogspot.com/2008/01/fredric-browns-
ans...](http://thepequodblog.blogspot.com/2008/01/fredric-browns-answer-short-
story-of.html)

------
sakers
Shortly after being switched on, researchers where found telling SpiNNaker
exactly what it could do with the lifetime supply of chocolate.

------
MBCook
Only time will tell if it’s a fair dinkum thinkum.

------
pixelbreaker
Won't be long before it's smoking Spice.

------
ctdonath
"...and even before its data banks had been connected, it deduced the
existence of rice pudding and income tax before anyone managed to turn it
off." \- H2G2

------
monk_e_boy
1M ... 1,000,000

Powers of 10 ... 10 fingers on the ape-man. Such a weird non computing number
to be thrilled with.

I'm always suspicious when numbers fit into powers of ten like that. Like,
somewhere in that build process the person who holds the purse strings doesn't
know binary.

~~~
SketchySeaBeast
Wouldn't any number you were going to stop at be arbitrary? If you can fit
1,000 to a rack, you're going to end up with a multiple of ten by the time
they are done. These processors don't need to adhere to 2^X, do they?

~~~
monk_e_boy
I put 1024 in my rack. Why did you choose 1,000?

~~~
vntok
Why did you choose 1024?

~~~
strainer
Its a nice fit for 10 bits

~~~
bencollier49
Brilliant

