
Signal processing is key to embedded machine learning - janjongboom
https://www.edgeimpulse.com/blog/dsp-key-embedded-ml
======
alephnil
Actually signal processing is already used for most machine learning of audio
signals, including speech recognition. The reason is that ML algorithms,
including deep learning has a hard time learning the information you can get
from a discrete Fourier transform.

Audio data in time domain are just too noisy for most machine learning, and
doing some signal processing as a preprocessor step often helps a lot.

Here it seems like he works with non-audio data, where this is less common.

~~~
taeric
This is just saying that signal processing is vital to the input sensors.
Which, doesn't seem new.

Yes, ml is dependent on getting data. Signal processing is vital to that.

I thought this was saying a new application of signal processing in ml.

------
kbumsik
It's not the key but the fundamentals that we have done for decades...

------
proverbialbunny
This is an awesome topic, but I'm somewhat annoyed they didn't dive into what
kind of DSP and instead turned the article into an advertisement.

Does anyone have any good further reading on the topic? (Books, articles,
classes, anything really.)

~~~
Optimal_Persona
The best DSP intro I've found, especially for the non-math whiz is Steven W.
Smith's DSP Guide:
[http://www.dspguide.com/pdfbook.htm](http://www.dspguide.com/pdfbook.htm)

~~~
proverbialbunny
I own a copy of this book. It is quite good as a reference once one has some
DSP experience. I wouldn't recommend it as an intro.

The book is quite good in that it is to the point and provides a good roadmap,
but to do so it often omits concrete examples and instead writes out the
algorithms in mathese. Knowing calculus should be enough.

------
ganzuul
The Kalman filter is basically an ML algo. The key here is to implement
already known linear optimization approximation versions of it in common
libraries.

~~~
srean
As an ML practitioner I am curious about how you decide what is ML vs what is
signal processing. I really cant tell. Its the same freaking problem, be it
information theory, machine learning, signal processing. All of us are stuck
at the same impasse -- do optimal rate-distortion of a continuous valued
signal _efficiently_ and _uniformly_.

~~~
ganzuul
Probably a matter of tradition, not substance. Same with control theory.

(Curious who downvoted me, when we are in agreement... Maybe the buttons are
too small and people just mis-click.)

~~~
srean
Dont worry about downvotes much. They can be quite erratic at times. I upvoyed
both of yours.

------
uoaei
It always comes down to representation. If you can use a deterministic,
efficient algorithm to represent the data in a more amenable manner, then the
ML system will have a much easier time "making sense" of the patterns inherent
in the data compared to a system that has to learn some abstract
transformation from raw data to useful representations.

------
taeric
My concern with a lot of signal processing techniques used in ml is that
sometimes they presuppose things that may not be true.

That is, signal processing had Nyquist's rates. And typically knows there is
an underlying signal. Does ml have either?

~~~
etrk
> _That is, signal processing had Nyquist 's rates. And typically knows there
> is an underlying signal. Does ml have either?_

What does this question mean? Every band-limited signal has a Nyquist rate.
Most signals of interest are well-contained within some finite bandwidth
(e.g., human voice). Sampling above this rate will get you very little.

If you're building an ML model to process a certain class of sampled signal
and you know, for example, 99% of the signal energy falls within a certain
frequency range, that should guide your choice of sample rate. If you're
sampling at too high a rate, your input layers may have far more parameters
than are needed or useful.

Whether or not a given ML input actually contains a signal of interest doesn't
seem relevant to how you sample and preprocess the signal.

~~~
taeric
Most machine learning is not on a band limited signal. I've literally seen
these tactics applied to demand forecasting. And I just can't square that they
should.

~~~
etrk
Ah, I see what you mean. Yes, if you're not dealing with approximately
bandlimited and sampled signals, then this wouldn't apply. The article is
about embedded devices processing sensor data (microphones, motion/light
sensors, accelerometers, etc.), and in those cases the signal of interest will
often be bandlimited.

~~~
taeric
Completely agreed. In those cases, these tactics are required.

