
Tinn: A tiny neural network library written in C99 - flipnotic
https://github.com/glouw/tinn
======
stochastic_monk
kann [0] is a similarly small neural network library written in pure C. It's
quite fast for CPU-only, but only applicable for rather small problems.

[0]
[https://github.com/attractivechaos/kann](https://github.com/attractivechaos/kann)

~~~
attractivechaos
It is fair to say KANN "only applicable for rather small problems". However,
KANN is not so "similar" to Tinn. Tinn only implements MLP with a single
hidden layer. KANN supports a lot more features such as 1D/2D convolution,
RNN/LSTM, weight sharing and mini-batching. It is also more optimized.

~~~
stochastic_monk
You're absolutely right that kann is more optimized and full-featured. I
thought to mention your library because it is also a very portable neural
network library written in C.

kann is a stunning piece of engineering, and your code is magnificent, and
that is precisely why I mentioned it. I apologize, I did not clearly express
my intent.

~~~
attractivechaos
I just meant to add some clarification. I actually do appreciate that you
mentioned it. Thank you!

------
lavabender
Cranium is another alternative that uses BLAS for efficient matrix operations.

[https://github.com/100/Cranium](https://github.com/100/Cranium)

------
ptr
Just curious, is there a reason for passing around the Tinn struct by-value?
Will it get optimized?

~~~
kbumsik
I'm curious too. Even if it will be optimized isn't it still be better to
pass-by-reference?

~~~
tom_mellior
It might be easier to answer if you explained why you think it would be
better.

There may be cases where by-reference (yes, that's an absolutely acceptable
thing to say, even in C) is more efficient: If the struct is a kind of "this
pointer" (as in object-oriented programming) that is passed to a bunch of
functions but _relatively rarely used_ , then passing it by reference causes
much less register pressure. Passing it by value would mean occupying a lot of
registers with values that are probably not accessed.

On the other hand, if you expect the callee to do actual computation on all or
most of the fields of the struct you pass in, passing it by value should
probably be the way to go. This way, the values to be processed are already in
registers and don't have to be stored to the stack by the caller and loaded
back from the stack by the callee.

There is no general answer that is uniformly best, it depends on the
characteristics of the actual program and their interactions with the
compiler's optimizations. With inlining in particular, which should often
allow the compiler to generate the same code for both versions.

~~~
foo101
> There may be cases where by-reference (yes, that's an absolutely acceptable
> thing to say, even in C)

I don't want to be nit-picking. But I would like to know why "by-reference" is
an acceptable thing to say.

When I learnt C, C++, Java, etc. I was repeatedly told that pass by reference
exists only in C++. In C, a pointer is passed by value. In Java, a reference
is passed by value.

Doesn't calling "pass pointer by value" as "pass by reference" lead to
confusion and distortion of the true meaning of "by reference"?

~~~
pubby
What do you do when you want to find the value a pointer points to? You
dereference it of course!

C++ and Java have their own specific language-lawyer definitions of the word
reference, but outside of those languages, "reference" is a catch-all term
that includes pointers, handles, and indexes.

------
Fethbita
genann is also a similar small neural network library written in C.

[https://github.com/codeplea/genann](https://github.com/codeplea/genann)

------
p1esk
Single core? No AVX? What's the point of writing it in C?

~~~
Annatar
Minimal dependencies; people are starting to get fed up with having to
bootstrap and/or compile the planet just to be able to run some software. The
situation has gotten completely out of hand, especially if one is not on
GNU/Linux.

~~~
AstralStorm
This does nor preclude the use of threads or SIMD.

Essentially this library is a demo toy that will need a lot to work to make
useful in real conditions.

~~~
taneq
> Essentially this library is a demo toy

It's 200 lines of C, what were you expecting?

~~~
didymospl
Generally when I see links to github projects on HN I expect something useful
or extraordinary and this looks like my Neural Networks 101 assignment, only
far more polished. But people seem to like it(over 400 stars right now) so
maybe I am missing something.

~~~
torstenvl
So... if its usefulness is equal to something you've made, AND it's available
to the public, AND it's more polished... why exactly are you shitposting about
it?

Let people enjoy things.

~~~
didymospl
I just pointed out that this is a very basic code. It's definitely not
something I would use in any real life scenario(see other posts itt). And if
you want to learn about NN you'd be better off googling something like
'backpropagation example' as this project does is virtually not documented. I
personally don't find it useful and it takes up the space that can be filled
with more interesting content. How is that 'shitposting'?

~~~
hoffa
It's a refreshingly clean, tiny library without dependencies that can do some
cool stuff. Let people like things. Stop shitposting.

~~~
glouwbug
Thankyou for the kind words.

------
dvdplm
What exactly is the output of the example in `test.c`? I.e. what do the
numbers mean? The error in the prediction I guess, but the prediction of what?
What is the first entry in the semion data set? a "0"?

~~~
zimpenfish
The first row of output is the vector of numerals 0-9 where 1.0 indicates "it
was this digit."

The second row is the probability of each given the input.

e.g. This output means it's 98.6% certain that the input represents the digit
7.

    
    
        0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 
        0.000009 0.000005 0.000000 0.000000 0.003899 0.012933 0.000436 0.986013 0.000005 0.000000

------
chestervonwinch
To the author: in your github topics, "propogation" -> "propagation".

~~~
glouwbug
Thankyou, my friend ;)

------
sanxiyn
How about in 12 kilobytes of assembly? More featureful implementation here:
[https://github.com/dfouhey/caffe64](https://github.com/dfouhey/caffe64)

~~~
Annatar
“Installing a new neural network library is typically a tremendous pain
because of onerous dependencies, python version hell, complicated Makefiles,
and massive code bloat.”

Finally people have started speaking up and doing something about it! May we
see more of such mentality in the future, and computer industry will be a
better place to work!

~~~
eeZah7Ux
Developers still underestimate how much code bloat and dependency
proliferation harm security and administration. Static linking encourages even
more bloat by vendorizing dependencies.

Please make your applications modular. Optional dependencies are there for
this reason.

Most good packaging tools support them.

