
Natural image reconstruction from brain waves - soofy
https://www.biorxiv.org/content/10.1101/787101v3.full
======
anonytrary
I'm highly skeptical. I mean, a hash function that has four output states also
maps _anything_ to one of those four states. That doesn't mean it's some next-
level classifier.

The problem here is EEG. EEG bandwidth is not enough to capture that much
information. There is far too much noise introduced by the skull and muscles.
It's most likely physically impossible to do something like this with EEG.

What's likely happening here is that there's some large scale oscillations
that are sufficiently unique to discern the images from each other. This does
not mean they are reproducing the images. I am highly skeptical of the methods
used here -- they are almost certainly flawed.

I, too, once had dreams of conquering the planet with EEG when I was a grad
student. I quickly learned that physics makes this infeasible. Anyone who is
serious about BMIs are studying invasive BMIs and how to make them as safe as
possible. Going inside the brain is unavoidable, I'm afraid.

~~~
tudorw
There is at least one 'affordable' fNIRS device coming to market that looks
promising, [https://foc.us/fnirs-sensor/](https://foc.us/fnirs-sensor/)
There's a paper somewhere on using machine learning to help identify signal,
this one is specifically about pain,
[https://www.nature.com/articles/s41598-019-42098-w](https://www.nature.com/articles/s41598-019-42098-w)
Say for example you were making an insurance claim for neuropathic pain, this
kind of information could be very important.

~~~
jacquesm
Instead, it will be repurposed for lie detectors and 'terrorist mindset
detectors' in airports.

------
echelon
This model is incredibly overfit.

Video: [https://youtu.be/nf-P3b2AnZw](https://youtu.be/nf-P3b2AnZw)

Watch how it has preconceived notions of these scenes. It frequently fails to
reconstruct the correct scene from video, and it also turns completely blank
input into one of the scenes it was trained on.

------
unscrupulous_sw
Imagine the shitshow this will cause once law enforcement adopts this.

Currently eyewitness criminal sketches are still drawn by artist so they are
naturally low fidelity.

That will change once you can generate a photo of a face (like
[https://thispersondoesnotexist.com/](https://thispersondoesnotexist.com/))
based on your brain waves.

This will be disastrous on so many levels. The eyewitness might not have a
good sample of a minority race. The GAN dataset itself might also only be
trained on celebrity faces so it doesn't know how to generate anything else
(e.g., a teen).

But it will be deceptively high resolution so police will rely on it.

If you have a generic face your life is fucked.

~~~
Erlich_Bachman
OR they will just find out that this method can just as easily be producing a
picture that someone who is good at visualizing just made up in their mind and
is actually "looking at" in their mind's eye. This making this technique
useless as a form of truth seeking machine.

------
dr_dshiv
My research group is doing the same thing but with music. Music may be more
promising than images because of the Frequency Following Response -- a sort of
direct resonance effect in the brain in response to sound.

We have 24 subjects listening to 12 songs in random order, with 128 channel
EEG sampling at 1000hz. We can then label all these data points with the
musical features at the time the data is collected.

We don't have a public repo yet, but we are sharing data.

------
lbj
I dont think their model is working, and Im not sure it ever will. Simply
reading brainwaves, a bi-product as I understand it, of the actual neuron
activity, couldn't possibly give you an accurate result.

------
rangibaby
The end results are much, much better than I thought they would be. Luckily, I
think it would be easy to fool the training by thinking about a totally
different image to the baseline one. Idk if that would stand up to rubber hose
cryptanalysis, but there’s got to be a way that can.

~~~
dannyw
The end results show an overfit model. It's not predicting that specific input
out of an option space of everything; it's essentially predicting that mode
(out of the 4) and probably capturing things like "if brain's audio regions
are active, it's a waterfall, because waterfalls are loud and trigger that".

------
Roark66
I read in some article some time about use of SQUIDs
([https://en.wikipedia.org/wiki/SQUID](https://en.wikipedia.org/wiki/SQUID))
to map activity of a single neuron non-invasively. There was a lot of hype at
the time for brain-computer interfaces based on that, but then same as with
many technologies that were "just 5 years away" those 5 years came and went
with no deliverables expected.

------
snowwrestler
There’s a great movie called _Until The End Of The World_ that centers on this
kind of technology. Once the scientists get it to work, they realize that they
can record and play back their dreams, and they become addicted to watching
them.

------
rasz
Lena source image resulting in some other random woman "reconstruction" =
model over fitted AF. Put a dead fist and it will continue generating
"reconstructions".

------
olliej
It’s interesting to see that the reconstructed Lenna has the high quality
reconstruction, but of a generic woman.

~~~
viraptor
See the figure title:

> an original face image replaced by an image sample due to publication policy

~~~
olliej
Ah, I was having difficulty reading the text due to formatting.

------
d-d
What would a world look like where all thoughts are public?

~~~
quangio
* a LOT more weird porns

* we do not need passwords

* eventually, human will be more empathetic

* new educational system

~~~
csomar
> * we do not need passwords

Probably the opposite. All passwords are machine generated/stored. Everyone
uses an HSM.

