
Neural Network on a Commodore 64 (1987) - musha68k
http://www.fourmilab.ch/documents/commodore/BrainSim/
======
classichasclass
COMPUTE!'s Gazette published a three part series on an assembly language
neural network in 1990 for the C64, which is infinitely faster than this, but
much more opaque. The article was pretty thorough as I recall, though.

~~~
cbm-vic-20
Part 1:
[https://archive.org/details/1990-01-computegazette/page/n24](https://archive.org/details/1990-01-computegazette/page/n24)

Part 2:
[https://archive.org/details/1990-02-computegazette/page/n35](https://archive.org/details/1990-02-computegazette/page/n35)

Part 3:
[https://archive.org/details/1990-03-computegazette/page/n43](https://archive.org/details/1990-03-computegazette/page/n43)

------
kragen
John Walker said (in 1987):

> _If you walk up to a computer and ask it “what is the name of the famous
> bridge in the same state as Disneyland” you won 't have much success, but
> most people will immediately answer, “The Golden Gate Bridge”._

I prompted GPT-2 on talktotransformer.com with that question. The first two
times it gave me nothing coherent (a made-up URL about Minnesota, and
something about Georgetown), but the third time, it said:

 _What is the name of the famous bridge in the same state as Disneyland?

This is a double fact: it is a California Heritage Bridge, a nickname invented
by the California Turnpike Authority and Santa Clara County Transportation
Authority (whose name is, coincidentally, the original name of the Carson City
Parking Authority).

How did the first hippie, Kesey, travel across the Golden Gate Bridge?

In Kesey's famous classic "One Flew Over the Cuckoo's Nest" he ruminates the
great flight across a river, and then some._

This is almost entirely false — there is, for example, no such thing as a
"California Heritage Bridge", and I can almost guarantee you that the Carson
City Parking Authority, if it exists, was never called the "Santa Clara County
Transportation Authority". However, GPT-2 clearly did manage to make the
association with the Golden Gate Bridge, as well as several other things about
California.

My attempts to rephrase the question in a more GPT-2-friendly way, such as
"There is a famous bridge in the same state as Disneyland, named", were not
successful, although they did confabulate an entertainingly large variety of
bridge names.

~~~
Udik
I tried a similar experiment on GPT-2 a few days ago, giving prompts like "if
I drop a stone, it will" or "wood floats in water while a stone". Because that
thing definitely has some knowledge of the real world, otherwise it wouldn't
be able to produce chunks of semantically plausible text.

The results were disappointing though, it mostly rambles and only occasionally
completes the sentence correctly. I wonder if the engine could be tweaked to
produce more semantically consistent statements- maybe as a specific "style"
(as it's very well versed in reproducing writing styles and formats).

~~~
kragen
Yeah, it seems like the question format led it to imitate the "style" Q&A web
pages which are mostly infotainment, but mostly don't bother to mention facts
that are so well known as the existence of the Golden Gate Bridge. Imitating
IRC "style" produces people ignoring the question:

> _< bob> wut is the name of the famous bridge n the same state as disneyland

> <DocDrew> small

> <DeathTrip> lmao, you "drew" this up

> <DeathTrip> 'sup DocDrew

> <drewk> Wasn't James Cameron the guy to make the first ever 3D movie?

> <drewk> :p

> <drewk> Also the guy who claimed to invent digital video in the first
> place._

While adding timestamps and a space for @ leads it to imitate specifically
#bitcoin:

> _19:33 < bob> wut is the name of the famous bridge n the same state as
> disneyland if you wanna know 17:19:37 <nalixor> this is actually actually
> helpful. 17:19:47 <nalixor> what?! 17:20:12 <Pnutbot> We should note that
> I'm not an expert on the US government nor the wallet industry, but as an
> alts-trader I'd like to see this explored thoroughly. 17:20:21 <Pnutbot>
> there might be people with other knowledge like_

Alternatively:

> _19:33 < bob> wut is the name of the famous bridge n the same state as
> disneyland?

> 19:34 < warren> Old Alaska?

> 19:34 < warren> or Nome?

> 19:35 < warren> I'm not familiar with that town

> 19:36 < warren>
> [http://en.wikipedia.org/wiki/Alaska_state_homepage#Social_ex...](http://en.wikipedia.org/wiki/Alaska_state_homepage#Social_exercise)

> 19:36 < warren> sometimes is chosen by the change the spelling of homeport_

Imitating the "style" of a play results in a back-and-forth dialogue, but an
unhelpful, dramatic one:

> _John: What is the name of the famous bridge in the same state as
> Disneyland?

> Johnathan: Uh, no. No, that's not what this is.

> John: Okay.

> Jonathan: That is not a bridge. What is this? A super bridge?

> John: Uh, yeah, it's a bridge.

> Jonathan: And now we got them in our fire truck… the bridge.

> John: With their fire trucks.

> Jonathan: And we got them in our fire truck, and we got the roof lifted up_

Imitating the style of a screenplay results in another dramatic but unhelpful
conversation about bridges and murder:

> _JIM looks thoughtful.

> JIM

> (finally)

> What is the name of the famous bridge in the same state as Disneyland?

> TRACY

> No biggie. Like the bridge for Opus--

> INT. ANDY'S CAR - NIGHT

> JIM and TRACY arrive at a quiet house.

> TRACY (cont'd)

> ... it's this house in a country where you won't get pulled over.

> JIM

> Or shot.

> TRACY

> And where you are a reasonably happy_

Formatting it explicitly as a Q&A leads to especially unhelpful results:

> _Q: What is the name of the famous bridge in the same state as Disneyland?

> A: ???????

> Q: If I had an extra $1 billion in the bank, would I donate it to my state
> to fund a road?

> A: ???????

> Q: If I knew of a company that I was not allowed to talk about because of
> internal corporate policies, what company would I go to?

> A: ???????

> Q: What do most big brands have in common?

> A: ???????_

Imitating a numbered Q&A format is only slightly more effective:

> _5\. What is the name of the famous bridge in the same state as Disneyland?

> That bridge is called Yosemite Falls and if you want to know its name then
> click here to view Google Map for this spot.

> 6\. I have found a claim on Google that claims that this park is a National
> Park. Is this true?

> No, it is not true as this park is listed as a County Park. Although this
> park is considered by many to be a National Park it is completely within the
> boundaries of California. The park has plenty of hiking trails that could
> actually qualify it_

Using the beginning of the Wikipedia article about Disneyland, plus a slight
prompt, was effective:

> _Disneyland Park, originally Disneyland, is the first of two theme parks
> built at the Disneyland Resort in the same state as various famous bridges,
> such as_

After a few tries, GPT-2 completed this with:

> _the Golden Gate Bridge, and the San Francisco-Oakland Bay Bridge. The park
> was first opened to the public on July 15, 1955, and was designed by Robert
> E. Iger. In 2005, an expansion expansion was completed, adding thirteen new
> roller coasters and 13 new_

Even still, it often generates human-sounding nonsense:

> _the Hollywood Bowl and Golden Ledge. The Hollywood Walk of Fame is an
> atlas-style listing of the names of well-known celebrities in the Walt
> Disney Company who have accepted the honor of being among the park 's
> heroes. The current structure was built in 1939-1940 and_

And of course even when it gets it right it gets it wrong:

> _the Golden Gate Bridge and the San Francisco, USA bridges.

> Worlds Fair of America and World's Fair of Texas [ edit ]

> In 1969, Disney established the World's Fair of America, a giant mechanical
> thing and show meant to feature large scale attractions like a Big_

------
dang
A thread from 2017:
[https://news.ycombinator.com/item?id=15882683](https://news.ycombinator.com/item?id=15882683)

------
cromwellian
Wow, a blast from the past. I ported this program to 6502 assembly and added
more features, and used it for a school project.

Then I read all of the articles on neural networks claiming they were a dead
end and went off to other things :)

------
binarymax
I love historic pieces like this and it's wonderful to see these early
experiments.

It's really easy to get this to run! Download VICE Disc Image of this Program,
unzip it (to get the .d64 file), then go here:
[https://virtualconsoles.com/online-
emulators/c64/](https://virtualconsoles.com/online-emulators/c64/) and drag in
the .d64 file to see it run.

Another note, for those not familiar with the brilliant author: "This site is
developed and maintained by John Walker, founder of Autodesk, Inc. and co-
author of AutoCAD. "

------
melling
Has anyone rewritten this in something modern? Probably less than 100 lines of
Python, for example. Can skip load/save, .... I think he uses the C64 ROMs to
get the bitmaps. All those pokes around 1024 are writes to the screen RAM.

    
    
        # READ Bitmap for selected character from C64 ROM
        310 k=asc(a$)
        320 if a$>="0" and a$<="9" then k=k+64:goto340 # process numbers 0..9
        330 if a$ < "a" or a$ > "z" then 500 # Maybe a function key
        340 gosub 970:print "fetch ";a$
        350 l%=0
        360 k=(k-64)*8+53248
        370 poke56333,127:poke 1,peek(1)and251
        380 fori=0to6:poke49408+i,peek(k+i):next # Copy character's image from character ROM (53248) to RAM
        390 poke 1,peek(1) or 4:poke 56333,129
        #
        # Look up bits that make character 6x7 = 42 values
        # Lighted dots are stored as 1 and background dots as −1
        400 for i = 0 to 6
        410    j% = peek(49408+i)/2 # byte value - divide by 2 is shift right.  Removes first bit
        420    for k=1 to 6 # 6 wide
        430      l%=l%+1 # counter 1..42
        440      f1%(l%) = -1 + (2 * (j% and 1)) # 2 * (1 or 0) => field[counter] = 1 or -1
        450      j%=j%/2 # Shifting right again to get the next bit in first position for the AND operator. xxxx xxx1
        460    next k
        470 next i
    
        # Function keys
        510 if j%=1 then gosub 1000:goto 280 # train()
        520 if j%=5 then gosub 1080:goto 90  # print_matrix()
        530 if j%=2 then gosub 1210:goto 280 # randomize_ten_percent()
        540 if j%=6 then gosub 1680:goto 280 # clear_memory()
        550 if j%=3 then gosub 1290:goto 280 # recall_pattern()
        560 if j%=7 then print chr$(147);:close15:end
        570 if j%=4 then gosub 1800:goto 90  # save_to_disk()
        580 if j%=8 then gosub 1990:goto 90  # restore_from_disk()
    
     1000 rem train on pattern in f1% def train()
     1010 gosub 970:print "training"
     1020 for i = 1 to 42
     1030   for j = 1 to 42
     1040     m%(i,j)=m%(i,j)+f1%(i)*f1%(j) # matrix[i][j] = matrix[i][j] + field1[i] * field1[j]
     1050   next j
     1060 next i
     1070 return
    
     1210 rem randomise 10 percent of f1% def randomize_ten_percent()
     1220 gosub 970:print "random"
     1230 for i=1 to 42
     1240   if rnd(0) > 0.1 then 1260 # Skip > 10%
     1250   f1%(i)=-f1%(i) # negate
     1260 next i
     1270 gosub 750 # update field f2 on screen
     1280 return

~~~
tenebrisalietum
The "poke56333,127" is disabling the background keyboard reading IRQ by
disabling the "IRQ on timer out" function of the CIA 1 used to trigger it.

This is needed because the "poke 1" that follows maps in the 4K character ROM
over the I/O devices at D000-DFFF; if you don't do this the system would
freeze.

------
fortran77
I got my Master's Degree in CS in 1988. For one project, I did a Neural
Network on an IBM-PC, in Turbo Pascal, which is barely a 16-bit computer, back
in this year. We were training it for various logic functions, and very simple
"simulated retina" problems with patterns on 16x16 grids.

~~~
kragen
_Barely_ a 16-bit computer? The IBM PC's CPU could address a mebibyte of RAM —
that's 20 bits! In a single process! You could actually put 640 KiB of RAM in
it, I'm pretty sure. A computer that was _barely_ 16 bits would at most have a
16-bit address space, maybe less, though I don't know of any that took that
option.

~~~
fortran77
The 8088 had an 8-bit data bus, 16-bit registers, with a segmented scheme that
would allow 20-bit memory addressing.

The 8086 had a 16-bit data bus. See
[https://en.wikipedia.org/wiki/Intel_8088](https://en.wikipedia.org/wiki/Intel_8088)

Usually the "bitness" of the computer is the width of the general registers.
Most 8-bit computers had a 16-bit address counter (for 64K of memory), with
8-bit general regisgters.

~~~
kragen
Yes, all of that is correct, although I think you have conflated "address bus"
with "program counter".

