Hacker News new | past | comments | ask | show | jobs | submit login
What's hidden in the hidden layers? (1989) [pdf] (cmu.edu)
124 points by sonabinu on Jan 2, 2018 | hide | past | web | favorite | 20 comments



It's from the August 1989 edition of Byte magazine, which has some more articles about neural networks: https://archive.org/stream/byte-magazine-1989-08/1989_08_BYT...


I like how "Doesn't Look Like IBM" was a feature on the clone PC ad on the last page.


It would be interesting to look deeper into IBM's move from "nobody gets fired by buying it" to a poisonous brand.


Back then, IBM was still deep in "nobody gets fired for buying them" territory.

It was just the ad style of the day. The same edition of Byte magazine had an Oracle ad that was full of DBase IV references (copies of negative articles about it, mostly) with only a small Oracle logo somewhere near the end.

Also stuff like https://78.media.tumblr.com/1fb5edc62a27f8f1c8ef07e611ec4281...


Nice. I didn't know they where working on self driving cars back in 1989! There's probably a lot to be learned. Today all the computing power of those racks can probably fit on a respery /nuc. Eg, you do not need a van, you could use an RC!


VaMoRs drove on the Autobahn in '87 [0].

[0] https://www.youtube.com/watch?v=2EH3R6c7Ufg


This looks so much fun and was probably unrivalled and academic spirit.


Take a look at following news and search for the thread started by guicho271828.

https://news.ycombinator.com/item?id=15593305


You need to look deeper into the ALVINN project (and the projects the preceded and came after it); it was one of the earlier attempts at a CNN-like architecture for self-driving vehicles.

A good start is here:

http://repository.cmu.edu/cgi/viewcontent.cgi?article=2874&c...

And yes, that van held a rack-mounted computing system that included a WARP systolic array processor:

https://en.wikipedia.org/wiki/WARP_(systolic_array)

...which is very similar to what we have today as a GPU; it was basically a "high-speed" (for its time) vector processing system (in other words, perfect for neural-network processing).

The processor on the vehicle ran the pre-trained model; training the model using similar machines took a really long time (weeks, iirc), even with the small number and sizes of layers they were working with.

Compare and contrast the ALVINN project (there's a lot of research papers and such on it - video, pictures, etc - dig into it if you are interested) to NVidia's End-to-End CNN self-driving vehicle:

https://devblogs.nvidia.com/parallelforall/deep-learning-sel...

You'll find a lot of parallels (and ALVINN - plus one or two of the other projects, iirc, is mentioned in their paper).

I'm familiar with a lot of this mainly thru my self-study of machine learning and deep learning via a combination of courses from Udacity and (now) Coursera. My first impressions about ALVINN actually came from articles and video about it when I was a kid in the 1980s; I didn't understand how it worked at the time, but I was really impressed and intrigued by it.

In 2011, I took part in (and ultimately completed) Andrew Ng's "ML Class" MOOC (this became one of Coursera's first courses) - and he made mention of ALVINN as a part of the course - and it inspired this guy, who was part of the class, to re-create it:

http://blog.davidsingleton.org/nnrccar/

So you aren't too off-base here; yes, that rack of machinery back in the 1980s has shrunk to pocket-size.

Earlier this year, as part of the course for Udacity's "Self-Driving Car Engineer" Nanodegree program, I implemented (using their simulator) a version of the NVidia End-to-End CNN architecture to drive the simulated car around the simulation track based only on what it learned from my own "driving" in the simulation.

Finally - if you are really interested in this - and want to compete on somewhat of a budget - there's this (among so many other similar competitions out there):

https://diyrobocars.com/


Can anybody explain figure 1?

As far as I can tell, you can't have hidden unit 1 inactive, while hidden unit 2 is active (as shown in Figure 1c)

Also, I can't see how we can make a matrix multiplication that maps

    (0,0) -> 0
    (0,1) -> 0
    (1,0) -> 0
    (1,1) -> 1
without adding a bias term. At least the weights shown in figure 1 are way off.


From the article: "According to optimistic predictions, by the year 2000, neural network technology will account for half the total revenue of the robotics and computer markets[]".


This is an interesting point! We are still only scratching the surface of the possibilities with what this technology can accomplish.


For as yet unknown values of "can".


So was this the first article on how todays AI works?


The research on artifical neural networks started already in the 50ths.

Take a look at this video.

https://www.youtube.com/watch?v=cNxadbrN_aI


Did you notice the link ends with an _aI suffix? :)


Nope ;-)


But would a neural network have noticed? :)


Only if trained to.


No




Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: