
Password Recovery Using Discrete Hopfield Neural Network in Python - itdxer
http://neupy.com/2015/09/21/password_recovery.html
======
hoare
Its refreshing to see other uses of neural networks other then the standard
deep learning stuff. keep up the good work!

EDIT: You might want to check the [Source] links in the docs. they throw a 404

~~~
itdxer
Thank you for your response. There are would be more interesting projects
related to neural networks.

And also thank you for detecting this problem with [source] link. I fixed it.

------
Houshalter
I wonder if this would work as a kind of spell check. Associate misspelled and
mistyped words with the correct word/spelling.

~~~
itdxer
It's an interesting idea. But I guess there are would be two main problems.

1\. Memory is limited. For this task you need to save a lot of words to make
possible use it in applications. To solve this problem you should save words
in a huge dimensional vectors.

2\. This network can't solve the problem of missing letters in words with a
high accuracy.

~~~
Houshalter
I was thinking of encoding words as letter pairs. E.g. the word "hello" would
have 1 'he', 1 'el', 1, 'll', and 1 'lo'. That seems more natural and robust
than the binary bits of it's ASCII encoding.

~~~
itdxer
There are two main advantage of ASCII approach. The first one is that vectors
weight is not very big and you can build relatively small matrix that easy to
store and faster to use. The second one is that usual encoding alternatives
give a vary sparse vectors and as a consequence there would a big number of
negative values (because we encode 0 as -1 value) inside the weight. For
instance, if you have something encoded as 1 (word, letter or anything like
that) with a big number of zeros (which would be encoded as -1 value in
Discrete Hopfield network) that would be very unlikely to see the same ones at
the same place multiple number of times and after some number of iterations
negative values will reduce value encoded as 1 inside of memory and these -1
values would be as a dominant value for specific pattern.

------
funkmaster_f
Yawn. Instead of making brand new neural network libraries, why not help
contribute to already mature libraries?

~~~
itdxer
I know that there are some great projects related to neural networks. But they
are focused on deep neural networks. Apart from this trend in neural networks
there are many other great ideas like Autoassociative memories, learning base
on associations, Adaptive resonance theory and so on. They are really cool and
you can make a lot of interesting projects with them. This was a major
inspiration for this library.

~~~
funkmaster_f
"But they are focused on deep neural networks" \- No. They're really not. If
anything, they're focused on making matrix operations faster and faster; where
the natural result is deeper and deeper networks. But, why not submit a pull
request to a highly mature library and add these features so that you can have
maximum impact instead of something fairly basic.

~~~
itdxer
Usually each library has it's own architecture. This architecture based on
some default classes that implements core functionality for models. I already
have a chance to check source code for the popular neural network libraries
and I found that their architecture requires some basic classes that are
related to the Backpropagation algorithm and it's variation. In neural
networks there are a lot of algorithms that really hard to compare and
generalize. To make your idea possible I see two ways for that. First one is
to change architecture for the main models classes in popular library. In
practice for the popular libraries it's a huge amount of work. Even if you
found possibility to generalize core architecture with other network that you
want implement, probably, you will broke a big part of library that works
before. The second one is build a new folder inside of existing library that
contains localized code that don't have relation to the core part. How it
differs from the decision of a single repository for a different library? In
practice its a good way to separate something without relation, because
further attempts to combine all in a big library will lead to breakage and
bugs that are difficult to debug and fix.

~~~
funkmaster_f
Different strokes for different folks, man. Help contribute to the
research/dev community by helping affect the research/dev community where it
matters.

