
Mental Overload Detection with Respiration Variability Spectrogram - koko20
http://aqibsaeed.github.io/2018-01-19-mental-overload-detection-with-respiration-variability-spectrogram/
======
sitkack
This would be great for having automated interrogation stations where one
attendant could oversee 20-50 individuals. Combine DL, TTS, STT, Eulerian
Video and this, we could have end to end personalized data collection.

The tortoise lays on its back, its belly baking in the hot sun, beating its
legs trying to turn itself over, but it can't. Not without your help. But
you're not helping.

~~~
FiveDegrees
TTS, STT?

~~~
barrkel
Text to speech, speech to text, is how I understand the two acronyms put
together.

------
randomdrake
Study: DeepBreath: Deep Learning of Breathing Patterns for Automatic Stress
Recognition using Low-Cost Thermal Imaging in Unconstrained Settings

Citation: Youngjun Cho, Nadia Bianchi-Berthouze, Simon J. Julier.
arXiv:1708.06026 [cs.HC]

Link: [https://arxiv.org/abs/1708.06026](https://arxiv.org/abs/1708.06026)

Abstract: We propose DeepBreath, a deep learning model which automatically
recognises people's psychological stress level (mental overload) from their
breathing patterns. Using a low cost thermal camera, we track a person's
breathing patterns as temperature changes around his/her nostril. The paper's
technical contribution is threefold. First of all, instead of creating hand-
crafted features to capture aspects of the breathing patterns, we transform
the uni-dimensional breathing signals into two dimensional respiration
variability spectrogram (RVS) sequences. The spectrograms easily capture the
complexity of the breathing dynamics. Second, a spatial pattern analysis based
on a deep Convolutional Neural Network (CNN) is directly applied to the
spectrogram sequences without the need of hand-crafting features. Finally, a
data augmentation technique, inspired from solutions for over-fitting problems
in deep learning, is applied to allow the CNN to learn with a small-scale
dataset from short-term measurements (e.g., up to a few hours). The model is
trained and tested with data collected from people exposed to two types of
cognitive tasks (Stroop Colour Word Test, Mental Computation test) with
sessions of different difficulty levels. Using normalised self-report as
ground truth, the CNN reaches 84.59% accuracy in discriminating between two
levels of stress and 56.52% in discriminating between three levels. In
addition, the CNN outperformed powerful shallow learning methods based on a
single layer neural network. Finally, the dataset of labelled thermal images
will be open to the community.

------
mirimir
Huh? Isn't "hypertension" high blood pressure? I get that "extreme tension",
as in "extreme stress", also makes sense. But is this recognized usage?

------
mindFilet
From the linked study, those spectrograms seem to represent a frequency of
nostril breathing versus... mouth breathing?

The automatic analysis of the source data’s thermal video capture seems to
look for hot nostril flares, from breathing through the nose only, as a
reliable measure of relaxed respiratory rate? The respiratory rate is either
assessed as a measurable number, ranked as a percentile, high or low, or
indeterminate/chaotic which is then presumed as correlated with elevated
mental activity?

Note that this spectrogram is not chemical analysis of respiratory exhaust
from exhaled gases. It’s not mass spectrometry, searching for CO2 and water
vapor content as an indicator of metabolic respiration, which is what the
title almost sounds like it could be about, without careful reading.

The study then goes on to state that other sensors (air flow meters, pressure
sensitive chest straps) could be used to extract respiratory rate over time,
and construct the same data interchangably. I’m not so sure that’s true, but
clearly the goal here is passive reads, probably for retrospective lie
detector types of analysis. Seems like a presuptive effort at precrime.

------
tankenmate
It would be interesting to see if this could have a positive application for
pilots and situations of loss of situational awareness or pilot load factor.

------
trevyn
Heart rate variability
([https://en.wikipedia.org/wiki/Heart_rate_variability](https://en.wikipedia.org/wiki/Heart_rate_variability))
may also be an interesting measure here.

~~~
nerdponx
When I did biofeedback therapy a couple of years ago, heart rate variability
was the main proxy they used for relaxation. The goal for session was to
obtain a steady heart rate.

~~~
trevyn
Counterintuitively, it's actually the other way around -- at rest, _more_
variability in your heart rate (over a roughly 20 second period) is correlated
with lower levels of stress. Deep, slow, regular breathing tends to increase
HRV, since heart rate is linked to respiration -- your heart beats a little
faster when you inhale, a little slower when you exhale. It's desirable to
have big, regular swings in heart rate, synchronized with your respiration.

Thankfully, biofeedback protocols will typically hide these details, and just
calculate a single metric for you to attempt to optimize.

~~~
nerdponx
No, I actually just remembered it wrong.

I remember what the chart looks like now, and yes the goal was in fact to have
your heart rate vary with breath.

When I started the session, my heart rate would jump around almost
arbitrarily. By the end, it would fall into a steady rhythm of rising and
falling as I breathed.

------
nerfhammer
Why do you need a neural net for this? isn't it just a simple increase in
variance?

------
phs318u
I've been looking for an excuse to get a FitBit or equivalent (hint: I'm not
particularly active other than a lot of walking). This would be awesome. I
think I'm one of those "hides it well" people when it comes to stress. Would
love something like this on my phone.

------
afpx
This reminds me of the professional gamers with collapsed lungs.

[https://news.ycombinator.com/item?id=15425256](https://news.ycombinator.com/item?id=15425256)

------
jdpigeon
Cool project.

BTW, is this line by line `x = Something()(x)` style of writing code common in
Tensorflow?

~~~
endianswap
For Keras, yes. It's known as the "functional model".

------
btown
Black Mirror episode idea: Luxury computer company starts including thermal
front-facing cameras, as well as heartbeat monitors in laptop palm rest areas,
in their hardware. Releases StressKit, an API which is promptly used by
companies to optimize their experiences. At first, it's great: Slack silences
notifications when someone's "in the zone," and Netflix has its greatest ever
season of hyper-optimized horror movies.

But then our protagonist, a programmer at a large software company, is told to
code a feature that subtly influences the behavior of users to drive them
towards microtransactions in their moments of greatest stress. Protagonist
finds this horrific and threatens to go public, at which point her colleagues,
fearful for their own careers, start using the prototype system to manipulate
her towards paranoia and madness. In true Black Mirror form, this escalates to
neverending existential horror by the end.

*mic drop

