
Hopfield Networks in Go - stehat
http://mlexplore.org/2017/03/12/hopfield-networks-in-go/
======
qznc
I encountered Hopfield networks in LLVM [0]. I still do not understand how you
would model Spill Code Placement as a Hopfield network.

[0] [https://github.com/alexjordan/LLVM-
TMS320C64X/blob/master/li...](https://github.com/alexjordan/LLVM-
TMS320C64X/blob/master/lib/CodeGen/SpillPlacement.cpp)

------
epberry
The number of neurons Hopfield networks can use seems like a serious
limitation. I think the max number in his graphs was 300? Does anyone know if
this is a real problem or not?

On another note, what is the state of the art in "'memory networks". I'm
thinking beyond LSTMs to a network that might be able to remember high
dimensional features across a large span of time.

~~~
curuinor
Usually if you properly use GPU the real limit is on the order of 20k. I think
the graph in his post is old as crap, actually

(if you sparsify connections and do some tricks with sparsification sky's the
limit, O(n * small multiplier over 1) with memory)

Hopfield nets don't do memory like LSTMs do. LSTM's already sparsify features
and memorize them a long time. There's a medium size literature on adding more
shit (NTM, Pointer net, attention) but nearly all of it is not that good (it
gets tested on bullshit)

------
est
Golang: a language where you divide statements with `if err != nil`

~~~
moreentropy
A language where it's natural to think about and properly handle every error
condition.

~~~
cr0sh
Someone who gets it.

In Go, this is one of the most difficult concepts of the language to wrap
one's head around. Moreso, it's a concept that makes you bang your head
against the wall at first; at least it did for me. After being so used to
"getting away with" not thinking about this in other languages, being forced
to think this way in Go can be maddening.

Then a light goes on.

...and then you look at your old code and go "eeesh"; in my case, I was coming
from a PHP background at the time I was learning Go.

Don't get me wrong - I still like PHP and other languages, and I don't use Go
much as it is, but once I wrapped my head around many of it's "strangeness"
(compared to other languages), I really appreciated it (because I recognised
that the whole point of the language was to help prevent common errors and in
many cases force better programming practices to that end), and began to enjoy
it.

~~~
vvanders
I hate to be the one who brings Rust into a Go thread but if you find forced
error handling to be a revelation then you'll find the borrow checker does the
same thing for your program structure and architecture.

There's also a ton of really nice error handlers[1] that let you ergonomically
deal with errors while still being forced to confront them.

[1] - [https://doc.rust-lang.org/book/error-handling.html](https://doc.rust-
lang.org/book/error-handling.html)

~~~
kmicklas
> I hate to be the one who brings Rust into a Go thread

You shouldn't feel bad for this. The Go programmers are legitimately unaware
that there's a better world out there.

~~~
greenhouse_gas
Honestly, not at this point of HN. I don't think there was one thread that
mentioned (or didn't mention) anything about Go which didn't have comments
criticizing it.

Everyone knows its problems, and everyone knows about Rust.

It's just that some people don't need such extreme Real Time that they can't
afford GC at the expense of the borrow-checker and lifetimes.

~~~
kmicklas
> Everyone knows its problems, and everyone knows about Rust.

I guess they've at least heard of it, yeah, but how many of them have actually
tried Rust (or any other language with a modern type system)?

> It's just that some people don't need such extreme Real Time that they can't
> afford GC at the expense of the borrow-checker and lifetimes.

They should probably be using Java, C#, Scala, ML, F#, Haskell, etc. then.

~~~
reality_czech
I have tried Rust, Go, Java, SML, Lisp, Javascript, and many other languages.
I prefer Go in many situations where simplicity, speed of compilation, and
portability are important. Rust is an interesting language with a lot of
potential but the jury is still out on how wide the adoption will be. There is
a steep learning curve, and the borrow checker has limitations.

One thing I don't appreciate is being lectured by pimply-faced nerds about how
Go programmers should embrace Java/C++ generics into their heart as their
personal saviors. I understand what Java and C++ are, I can choose to use them
or not, and it's none of your business.

