Ask HN:Artificial Neural Networks for electronics engineers

[rick_2047]

Hi all,

I am a First year electronics engineering student.I am aiming at a scholarship which requires me to make a project, anything original and research based. I am still searching for ideas, and one that came into my mind was artificial neural networks in electronics. I thought it was perfect (mind you I have a very limited knowledge of ANN, so feel free to correct me).Neural networks are mathematical models of brain and if I could implement such a thing,even on a very very small scale, into simple circuits it would definitely result in a very impressive research project.

The immediate second thought was that this cannot be original. There has to be some work done on it as this is such an obvious thing. I found out about BEAM robotics and Nv Nets, but as this is still in the research phase there is not much material on it to read.

I couldn't find books,post,papers anything which will make me understand Neural Networks from the perspective of an electronics engineer. But I think this could be the mistake of my not-so-good searching skills. Can anyone please point me to such a resource?

[jacquesm]

There are chips that contain neural networks:

http://www.particle.kth.se/~lindsey/HardwareNNWCourse/

That's fairly dated, lots more to read:

http://www.google.com/search?q=neural+networks+in+hardware

The Japanese have done tons of research in this field and the producers of quite a few of the chips, there are lots of fields of applications for these, one of them is in process control.

[joshu]

It's not model of a brain, it's a model of a bunch of connected neurons.

I think it's probably easy to emulate a neural network in an embedded CPU. That seems somewhat boring.

It might be more fun to build a bunch of discrete neurons out of analog parts. Building a forward-propagation-only (perceptron) system should be pretty easy. Might be hard to actually train them.

[jjguy]

I spent some time studying Self-Organizing Maps, a neural network algorithm from Teuvo Kohonen, a Finnish professor.

The internet has plenty of resources:

http://www.google.com/search?q=self+organizing+maps

I implemented the two canonical 'hello world' projects in processing, but have yet to apply to a 'real' problem. The emergent behavior is fascinating, as is watching maps coalesce in real-time.

There are plenty of other algorithms available; a basic "back propagation" algorithm is widely used in OCR algorithms. Try reading some of these:

http://www.google.com/search?q=neural+networks+image+recogni...

Don't get too wrapped around the axle of trying to find immediate electronics engineering applications, but understand the algorithms and the applications will become self-evident.

[JohnIdol]

If you wanna implement logic operations using neural networks (you'll need 1 neuron for most of them - but XOR and NAND will require a few more and some tricks) and combine them to do some non-trivial computation - which is simple but good enough for a first year project - have a look at the first 3 chapters of this free book for an overview of this topic and have a look at the last chapter focusing on hardware for neural networks implementations (which will come in handy if we're not talking about software implementation) --> http://page.mi.fu-berlin.de/rojas/neural/

[paraschopra]

All neural networks do is to aggregate input values (which you can model with current) and multiply it by weights (which you can model by variable resistors) and finally output a squished sigmodial output (for which I am sure a circuit must exist). Make elementary units of such neurons and connect them together. Then train the network by manually changing resistors to emulate backpropagation algorithm.

Though why you want to do all this through circuits (instead of a computer) is beyond me.

[rick_2047 → paraschopra]

Because I want to show that these things can be done without a computational power. So my work can be further expanded to give IC's. Also it would be cool to have just circuits who have emergent behavior.

[paraschopra → rick_2047]

You are doing computation here. Moreover, it is not an emergent behavior, you are training your ANN here.

[rick_2047 → paraschopra]

Sorry for not making myself clear, by computation I meant just micro controllers and microprocessors. And I was wrong in stating that in such a way because computation is such a big umbrella.

Also the whole purpose of the project will be to make a customized circuit which could be faster than an embedded system.

[transmit101]

This is slightly more centred on evolutionary electronics, but nevertheless you might find some interesting papers here: http://www.cogs.susx.ac.uk/users/adrianth/ade.html

[rick_2047]

All this is quite good, and I was right in thinking it was just my not-so-good searching skills that did me in. But there are two big questions which I mean to ask

1)Are these projects too big and complicated for a first year student?I can put in as much hardwork as is required but sometimes somethings are beyond a persons comprehensions.

2)What should be the approach to learning the skills required to tackle such problems, considering a student with no previous knowledge of ANNs but some good maths and electronics skills?