
The Tsetlin Machine outperforms neural networks - beagle3
https://cair.uia.no/milestones-and-discoveries/the-tsetlin-machine-outperforms-neural-networks/
======
kurthr
Just go straight to the paper or code... the write-up is almost content free.

Paper: [https://arxiv.org/abs/1804.01508](https://arxiv.org/abs/1804.01508)

Code:
[https://github.com/cair/TsetlinMachine](https://github.com/cair/TsetlinMachine)

------
xtacy
/r/MachineLearning comments on the same paper from April 2018:

[https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_t...](https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_the_tsetlin_machine_a_new_approach_to_ml/)

~~~
wetpaws
See also:
[https://www.reddit.com/r/MachineLearning/comments/ab8na3/pro...](https://www.reddit.com/r/MachineLearning/comments/ab8na3/project_new_tsetlin_machine_implementation_with/)

------
nl
The paper is actually pretty interesting - it's a pity the headline is
misleading.

The Reddit comment[1] is a good summary: _More accurately: "The Tsetlin
Machine - a new approach to ML - outperforms single layer Neural Networks,
SVMs, Random Forests, the Naive Bayes Classifier and Logistic Regression on
four carefully selected contrived datasets"_

Notably, on their (weird!) "Binary Iris" dataset, the NN appears to be
undertrained, and it is unclear what the headline "mean" accuracy figure is
actually a mean _of_.

However, once we get over that, it's quite a different approach, and I can
imagine places where it could be useful. Notably, as an anomaly detector it
would seem to have interesting properties like interpretable results similar
to a random forest.

[1]
[https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_t...](https://www.reddit.com/r/MachineLearning/comments/89yp8g/r_the_tsetlin_machine_a_new_approach_to_ml/dwujoxt/)

~~~
nestorD
They say that their results are interpretable due to being logical formulas
but, later, show a toy problem in which their formula has 10000 clauses. By
that definition deep-learning is probably interpretable.

However I do think that it is an interesting concept that would probably be
worth testing if your inputs and outputs have a natural mapping to booleans.

------
alexyak
The Tsetlin machine is based on Tsetlin’s automata, the so called finite
automata with linear tactics. The are basically counters of rewards (up) and
penalties (down) with various strategies of integration, proportionization and
differentiation. They have minimum necessary for digital emulation of the
perception functionality. They str not only due to Tsetlin but also yo
Krinsky, Krylov, Varshavsky. Mikhail Tsetlin is the most prominent creator.
It’s the work in the USSR in the 60s. So not new but very elegant!

------
simonw
The entire implementation appears to be 300 lines of Cython:
[https://github.com/cair/TsetlinMachineCython/blob/master/Tse...](https://github.com/cair/TsetlinMachineCython/blob/master/TsetlinMachine.pyx)

------
chvid
So what is a Tselin machine?

Maybe I am lazy but a don’t see an exact description.

I am assume it is a function that takes an input, a state and produces an
output. But what exactly is it?

A neural net is a fairly general construction. Is this not just a nn with a
particular activation function, a particular network layout?

~~~
shakna
For a detailed understanding, you need the paper. [0]

But, loosely speaking, it is a classifier that works best with less
information, utilising a series of formulas. It only really does bitwise ops
for recognition, and is easy to implement and fast.

If I were to describe it in more flowery terms, hashing and finite automata
had a baby.

[0] [https://arxiv.org/abs/1804.01508](https://arxiv.org/abs/1804.01508)

~~~
chvid
Why can't I just get a reasonable definition of what a Tsetlin machine is?

As far as I can read it a function that takes an input (a vector of real
numbers in 0..1 (?)), a state (in form of an integer) and produces an output
(in form of something?).

~~~
brownbat
What constitutes reasonable?

Here's an example of someone explaining something to five different levels of
audiences.
[https://www.youtube.com/watch?v=OWJCfOvochA](https://www.youtube.com/watch?v=OWJCfOvochA)

None of these approaches are prima facie "unreasonable." But if you get the
wrong level it won't feel useful.

I don't think it's crazy to have open teaching discussions on HN, I think you
lay out a fair hope that someone can break something down here. But I'm just
not sure which level you're looking for.

Precision and simplicity are often competing constraints.

~~~
chvid
Okay.

Let me try getting my point across with an insult instead.

Some dude in a marginal university in Norway has managed to convince a few
people, a couple of students and a writer for a glittery university magazine,
that he is a genius.

His approach is to do something trivial but bury it in a completely obscure
terminology. This technically sorta works on a toy example but not really on
more challenging tests. But it is very convincing for people who are not
trying to understand what is going on but rather are impressed by technical
words and academic rank.

That is why it is getting negative feedback here and on other technical
forums.

And that is why noone can answer a simple technical question.

~~~
majewsky
> Let me try getting my point across with an insult instead.

As a rule of thumb, when you write such a sentence, just stop and go outside
for a while.

~~~
chvid
Sorry about my choice of wording; I was just annoyed with the bold claims and
the apparent lack of definition of this specifc concept.

------
nestorD
The Tsetlin Machine being a specialized method, it makes sense that it beats
neural networks in its domain of predilection (decision from, potentially
noisy, binary inputs).

