I wrote about this in my book ‘C++ Power Paradigms’ that I wrote about 25 years ago. I devoted a very long chapter to an implementation I called Vari-Gene where I started by using a small number of bits to represent weight parameters and slowly increased the number of bits per weight. At the time, I had lunch with John Koza and we discussed my idea. He said that it was a net idea but that it wouldn’t scale. He was correct, I never was able to train a large recurrent net. BTW, I didn’t see any source code and data mentioned in the first linked article. Any links?
