
An algebraic generalization for graph and tensor-based neural networks [pdf] - Katydid
http://www.cibcb2017.org/slides/CIBCB-2017-Ethan-Aug23.pdf
======
cs702
TL;DR: an early effort to establish a 'standard language' for representing and
symbolically manipulating (e.g., transforming) _every_ possible neural net
that could ever be constructed with graphs and tensors.

Very cool to see this sort of thing.

~~~
amelius
Two questions:

Does this work also allow cyclic structures such as recurrent neural networks
(RNNs)?

Does this work generalize to graphs in general?

~~~
cs702
In principle it should, because these guys are using matrices to represent
network structures, and every finite graph (cyclic or acyclic, directed or
undirected) can be represented by a matrix.

Examples:
[https://en.wikipedia.org/wiki/Adjacency_matrix#Examples](https://en.wikipedia.org/wiki/Adjacency_matrix#Examples)

------
minflynn
I used a very similar idea to create an indirect coding for neuroevolution
that was successful in solving some tasks. It's based on a generalization of
K^2 Trees and shares the same ideas of modularity and hierarchy.

~~~
cbennett
That sounds really interesting. What tasks did you attempt?

If you have open-sourced this in any sense, would enjoy to check it out and/or
contribute.

------
zitterbewegung
So what does this let you do other than having a standard notation for both
graph and tensor NN?

~~~
xaedes
See page 22:

" Modular Neuroevolution

• We are currently working on a neuroevolution framework based on Cartesian
genetic programming (CGP) and existing work on CGP-based ANNs.

• The algebraic framework introduced in this work will be the basis for the
genetic representation and operators.

• Evolved networks will be a mix of de novo evolved modules and existing
modules in the form of ANN layers, relational, and functional programs.

• The representation will be based on a mapping between algebraic expressions
and a recursive, modular adjacency structure. "

------
deepnotderp
Cool work, now hopefully we can stop using increasingly arbitrary notation in
papers... ;)

~~~
flor1s
It seems like "arbitrary" notation is also a problem in other fields of
mathematics such as Linear Algebra - perhaps depending on the background of
the author in math or CS. In Linear Algebra every text book has their own
favorite notation for matrices, row/column vectors (special letter such as c
for column vectors, roman letters, roman letters with an arrow on top) and
scalars (greek letters or the letter c).

~~~
gbrown
Now add the distinction between random and nonrandom.

------
eddieone
Search the CGP section, that blew my mind and it has become more rare these
days. Reminds me of Nexus, a science fiction book.

