

Numenta open-sourced their Cortical Learning Algorithm - p1esk
http://numenta.org/

======
tlarkworthy
Hmmm, the white paper is not very encouraging:-

What is NOT in this document:

\- Although most aspects of the algorithms have been implemented and tested in
software, none of the test results are currently included.

-There is no description of how the algorithms can be applied to practical problems. Missing is a description of how you would convert data from a sensor or database into a distributed representation suitable for the algorithms.

So until someone has tried it on something, we don't know how it will perform
:s

~~~
tlarkworthy
BTW I was trying to install it on ubuntu 12.04 (an LTS) and could not get it
working. It wanted to change my numpy dist

------
Shooti
This Google tech talk provides the missing context:
[http://www.youtube.com/watch?v=4y43qwS8fl4](http://www.youtube.com/watch?v=4y43qwS8fl4)

------
eli_gottlieb
Ok, so what does this mean to those of us who don't do deep neuroscience-based
machine learning that often?

Basically, what problem (problems?) does this algorithm actually solve?

~~~
rdtsc
That was my main question. How does this compare (either experimentally or
theoretically) to already well established approaches.

No references in their paper.

The smells like Wolfram's "New Kind of Science" type work. And they are
welcome to prove me wrong, I like AI topics. But it is isolated from other
peer work, when AI field is not new. So kind of suspicious.

~~~
cr4zy
I've studied Numenta's CLA's, previously HTM's, as well as established ML
approaches fairly in-depth over the past couple of years. Hawkins definitely
takes a new approach to neural networks and focuses more on modeling the
brain's biology rather than building on known artificial neural network
concepts. HTM's did incorporate bayesian nets, but have been abandoned,
probably with the departure of Dileep George, who now runs
[http://vicarious.com/](http://vicarious.com/).

However, Hawkins work is similar to Deep Neural Nets in that both are
unsupervised neural nets (unlike perceptrons and back-prop) and both aim to
represent data as sparse representations. Also, here's a similar question I
answered on Quora: [http://www.quora.com/Machine-Learning/Does-Numenta-build-
on-...](http://www.quora.com/Machine-Learning/Does-Numenta-build-on-any-
existing-machine-learning-theories-e-g-NN-SVM-HMM-or-is-it-a-new-approach)

In addition, Numenta does use CLA's to solve real problems for customers:
[https://www.groksolutions.com/solutions.html](https://www.groksolutions.com/solutions.html)

Finally, Hawkins' book, On Intelligence, has at least inspired researchers
like Stanford's Andrew Ng at a philosophical level which he talks about IIRC
in his online ML course.
[https://class.coursera.org/ml/class](https://class.coursera.org/ml/class)

------
volokoumphetico
If I use this, could I make something that is better than machine learning
using neural networks?

~~~
derefr
Not better in the general case; most machine-learning algorithms do worse than
the naive solution ("throw a Bayes net at it") in most applications, but then
have a niche where they excel.

If I recall, Numenta's model is said to excel in the case where the data it's
being trained on is similar to the kind the human brain tends to process:
sensory data with temporally-local "features" (letters on a page, instruments
in a song), where the features form a hierarchy
(shapes->letters->words->sentences,
frequencies->notes->chords->riffs->melodies, etc.)

~~~
tensor
> most machine-learning algorithms do worse than the naive solution ("throw a
> Bayes net at it") in most applications

This is an extremely misleading statement. There is nothing naive about
probabilistic graphical models. It's a massive topic with a huge number of
different model variations, training methods, and techniques to avoid
overfitting or do feature selection.

More many probably you probably only want a linear classifier, and a
discriminative one at that. But if your data is not linearly separable, or has
structure, then you'll want something else again.

~~~
derefr
Hmm, I suppose I did phrase that in a way that would mislead someone who
didn't already know what I meant by it. :)

I didn't mean that Bayes nets are themselves naive--they're extremely good
models for many use-cases. In fact, they're _eventually_ ideal models for all
use-cases, given enough training, since all ML models are ways to approximate
Bayes' law, while Bayes nets just solve for Bayes' law directly. However, for
being so _generic_ , they have no embedded structure that allows them to
efficiently learn/represent/compress data in any _particular_ domain.[1]

When I have a Big Data problem, and I have a BNN library close to hand, I just
throw it at a random subsampling and see what happens. This is what I mean by
"naive": since it's a generic solution, you can try applying it to a problem
before even studying the structure of the problem domain, and you might just
detect a signal soon enough that you don't need to go any further.[2]

The _rest_ of the machine-learning toolbox, then, is basically a set of models
that you replace your BNN with when you have a model that hews closely to the
structure of your data (in est, a set of screwdrivers for different kinds of
screws.) The approximate model will get quicker training and clearer signals
for _that_ data, but not for just any data.

\---

[1] Another way to say it: a Bayes-net implementation is to gzip as other ML
models are to JPEG, MP3, etc. Bayes-nets can learn anything, but they can only
learn it as well as _the data can be used to model the data_.

[2] People do use linear regression as the same kind of general-purpose tool,
as you say--and linear classifier libraries are much easier to write, so
you'll find them as "batteries included" in more math packages--but if you
don't _know_ whether the data is linearly separable, linear models aren't
gonna give you even the slightest hint. A BNN is an extremely-inefficient, but
completely-general _probe_.

~~~
volokoumphetico
I have no clue about all that academic stuff, what's a good reading material
for such things?

