
MIT D4M: Mathematics of Big Data and Machine Learning [video] - espeed
https://www.youtube.com/watch?v=iCAZLl6nq4c&list=PLUl4u3cNGP62DPmPLrVyYfk3-Try_ftJJ&index=1
======
rdudekul
Beautiful explanations of key mathematical concepts such as linearity and
associated generalizations. Good to get a sense of how so many real life
complex phenomenon can be modeled by combining simple mathematical constructs.

------
melling
About 25 minutes in he explains that we had the fundamentals in the mid
1950’s. It took half a century for Moore’s Law to solve the problem.

------
sonabinu
Listened to this talk given on another campus! It's a great intro and makes
the concepts feel accessible

------
graycat
His first lecture is good in that he outlines how neural nets are trained to
recognize. E.g., can train a network to recognize cats, dogs, and parrots or
some such.

IMHO his material on _linearity_ is somewhat interesting but avoids the common
approach that what is _linear_ is a function. The usual definition goes,
function f is _linear_ provided

f(ax + by) = af(x) + bf(y)

for appropriate cases of a, b, x, y. In linear algebra, commonly f is a
matrix, a and b are numbers in a _field_ , usually the real or complex
numbers, and x and y are vectors.

Linearity continues quite broadly in math, especially analysis and geometry.
One text by G. F. Simmons stated that the two pillars of analysis in math were
linearity and continuity.

It appears that after the first lecture, the course is heavily from a paper he
and some others wrote on something of a software architecture to ease writing
software for the computing for training neural networks.

His architecture is based on his definition of _associative arrays_ which seem
to be one, if you will, data structure which can take data from, be regarded
as generalizations of, spreadsheets, matrices, maybe usually sparse, some
graphs (that is, nodes connected by arcs), etc. For the associative arrays, he
defines operations that generalize addition and multiplication of matrix
theory, etc. and shows that the operations obey a distributive law that he
regards as the key to linearity.

Then the general, infrastructure (interface) software _layer_ is supposed to
be small and easy to write.

Some of his examples of anomaly detection from "big data" (it really was big)
are impressive.

His framework might significantly help the productivity of building and using
neural networks.

First cut, it looks like the main interests in neural networks have been for
recognition of speech and images, but his examples likely provide good
evidence of other significant application areas.

As might expect from Lincoln Labs, he's fully serious and not just fooling
around or pushing hype.

His lecture notes and writing are relatively good.

He has a textbook available at Amazon.

For just the course materials, he has a ZIP file download of 35,802,178 bytes
that from just one use of a standard UNZIP command expands into about 640
files/directories (the number of files/directories of his course on my disk is
640 and nearly all of those are from the UNZIP command). Among the many files
are some well done PDF files.

He does admit that it is not known why neural networks work as well as they
do.

~~~
espeed
The first lecture is from this summer, the others are from an MIT mini-course
few years back when D4M was still incubating.

During that time D4M was developed into a standard, which is now GraphBLAS,
see my comment from yesterday...

Log(Graph): A Near-Optimal High-Performance Graph Representation (2018)

[https://news.ycombinator.com/item?id=18099520](https://news.ycombinator.com/item?id=18099520)

For an overview of GraphBLAS in the context of Heterogeneous High-Performance
Computing (HHPC) systems such as NVIDIA GPUs or Intel Xeon Phis, see the 2015
talk Scott McMillan ([https://insights.sei.cmu.edu/author/scott-
mcmillan/](https://insights.sei.cmu.edu/author/scott-mcmillan/)) gave at the
CMU Software Engineering Institute:

Graph Algorithms on Future Architectures [video]
[https://www.youtube.com/watch?v=-sIdS4cz7-4](https://www.youtube.com/watch?v=-sIdS4cz7-4)

