Beware, there's a bug on ARM11-based Raspberry Pis (BCM2835 on Pi 1/Zero/Zero W and CM1) that crashes the VM when executing drawing functions. I'm currently investigating the problem.
FYI, your commits are signed with "Your Name <you@example.com>". You'll want to fix your git config. For these existing commits, you can use git-filter-repo [0] to rewrite the author name and email.
It seems to be a problem with the USB controller driver in circle. Startup hangs with the message "XHCI: HW reset failed".
I was under the impression that the hardware of the 8GB Raspberry Pi 4 was identical to the versions with less memory except for (obviously) the memory chip and some fixes for USB-C power delivery. Maybe it's a subtle timing problem - another item on my to-do list...
I'll have to give this a try on my Raspberry Pi 4 4GB. I've setup Ubuntu, Arch, Kali, Raspbian, and Raspberry Pi OS on my Pis, but this'll certainly be the most exotic, given that it's not Linux. Do you know of any bare metal Forth- or Lisp-based systems for RPi?
There's the Interim OS project [1] (already discussed in another comment here) as a bare-metal Lisp and it might be interesting to port Movitz [2], a bare-metal Lisp project for x86 that seems to be abandoned for quite some time now.
There also seems to be a bare metal Forth implementation, pijFORTHos [3], but I haven't played around with that one so far.
Not in the current version, here the drivers are implemented in C/C++ in the circle bare-metal library [1].
In general, I think Smalltalk is not that well suited as a systems programming language since garbage collection might mess with your timing and you would also need to find a way to handle asynchronous events (such as interrupts).
Nevertheless, Smalltalk has been used for low-level control of robots, e.g. in NXTalk [2] for the Mindstorms NXT ARM7 platform. However, the VM also relied on C code (the LegOS project) for driver code.
There is an almost all-Smalltalk bare metal system for X86, CogNOS. It runs on top of NopSys, about 3500 lines of C and 600 lines of assembler for booting, accessing device registers, and responding to interrupts. But the file system etc. are in Smalltalk.
Do you know anything about the status of this project? I started following it on Github a couple of years ago when it appeared but it does not seem like anyone is working on it
It’s interesting that the 8GB Pi4 won’t work, but the 4GB version will. Any clues as to why? I would assume that they are the same, just with more memory.
Thanks for mentioning this - I would certainly appreciate input on this. Testing software on half a dozen different Raspberry Pi models currently involves quite a bit of cable un- and replugging...
We know that smalltalk message passing was inspired by biology, and the ultimate design vision was for objects to be ephemeral machines themselves.
Would it be possible to create a farm of Pis, where some objects were on some subset of them, and some objects on another? Even a simplistic example with two Pis, with the object space split between them? Can we use the network to perform language-level message passing?
Btw., the biological inspiration is now in a way reversed and a huge networked cluster of ARM nodes is currently used to simulate the brain in the SpiNNaker Project [1]. This project is headed by Steve Furber at Manchester University, one of the inventors of the ARM processor (with Sophie Wilson).
With more than a million ARM9 cores and about 7 TB of distributed memory this is a bit of a different scale than a cluster of Raspberry Pis, though ;). SpiNNaker also reminds me of Dan Hillis' Connection Machine [2], but taking advantage of 40 years of progress with Moore's Law...
> Would it be possible to create a farm of Pis, where some objects were on some subset of them, and some objects on another? Even a simplistic example with two Pis, with the object space split between them? Can we use the network to perform language-level message passing?
That's a distributed erlang setup, and I'm rather certain erlang runs on raspi. Pretty sure I even saw Erlang on Xen worked on raspi at one point, so your raspi could be a bare-metal erlang node. Not sure that's maintained / still working though, EoX seems dead.
That idea crossed my mind, too - in fact, I have a SOPINE cluster here [1] (built by Pine64) that would be a good platform for experiments with distributed objects. The Turing Pi is a very similar product for Raspberry Pi Compute Modules [2], just a bit more expensive. Of course, this approach would work as well with a bunch of Raspberry Pis connected via an Ethernet switch.
There was some research in the late '80s on building a distributed Smalltalk [3,4] which is a nice inspiration for such a project. So it's definitely possible to build a distributed Smalltalk and it would be interesting to investigate how well such a system would work given today's hardware performance and networking throughput.
Distributed Smalltalk is just one of the ideas for building distributed non-shared-memory systems from the 1980s. This was an interesting time in general, people experimented with different approaches for distributed systems hardware and software. Some examples include the inmos Transputer, which had bit-serial communication links [5] and support for CSP-based (paper [6] and book [7]) communication in the instruction set (later these ideas were picked up again by David May and his colleagues for the XMOS embedded processors [8]); on the operating system side there were systems such as Plan 9 and Inferno [9], invented by the Unix team at Bell labs, Tanenbaum's (the Minix guy) Amoeba [10] and Ousterhout's (otherwise known for tcl/tk) Sprite [11].
Some of these ideas seem to be gaining popularity again (e.g., CSP-based communication for goroutines [12]) and I hope my students here are interested in investigation these approaches to building distributed systems, e.g. as an alternative platform for IoT applications.
[3] Marcel Schelvis, Eddy Bledoeg, The Implementation of a Distributed Smalltalk, ECOOP 1988 - www.lirmm.fr/~ducour/Doc-objets/ECOOP/papers/0322/03220212.pdf
[10] Andrew S. Tanenbaum, M. Frans Kaashoek, Robbert van Renesse and Henri E. Bal (1991). The Amoeba distributed operating system — a status report. Computer Communications 14 - https://www.cs.vu.nl/pub/amoeba/Intro.pdf
[12] James Whitney, Chandler Gifford and Maria Pantoja, Distributed execution of communicating sequential process-style concurrency: Golang case study, The Journal of Supercomputing volume 75, pages 1396–1409(2019)
Thank you for this synopsis! It's been several years since I've had the time to really dig into the (extensive) prior work in this area. You've given me some great links!
I have the smalltalk blue book on my shelf and have always, always wondered about the possibility of bringing back some of the radical ideas from the 60s/70s/80s in a language like Ruby (being popular). Live objects and snapshots are the two that I was most enamored with way back in the day, but I'm sure there are lots of others.
This isn't terribly old (2014) but would love thoughts on it (if you have any!). It's a prototype of a smalltalk-like "System Browser" implemented in Ruby and Qt on OS X:
The idea is that in the same way that Clojure is a sort of "improved" lisp on the JVM for performance reasons, syntax enhancements, and access to the standard library of Java, I see no reason why Ruby couldn't be the same thing for Smalltalk. Access to the ruby standard library, bringing an interactive live environment to a vibrant programming community and prioritizing interoperability with the modern world (as opposed to much of the smalltalk world which still sort of sits a "world" apart)
Thanks for the link - that looks really interesting! I will have to give it a try to make some informed statement, but I think it's definitely worth looking at. Some parts of Ruby also seem to be inspired by Smalltalk, so that might be a good fit. There's also a Smalltalk-80 VM by Rochus Keller based on the Lua JIT compiler [8] (and a very nice Qt-based Smalltalk browser), that might also go in the same direction.
In general, I think there are two major interesting aspects in systems like the Ruby system browser and Smalltalk. On the one hand one can discover the system - you can dig into the implementation without using a disassembler or reverse engineering - and on the other hand it is possible to change the system behavior at runtime (though this is, of course, dangerous).
I would love to see at least the discoverability aspect to be more pervasive in standard operating systems. There are already partial approaches to this, such as DTrace/systemtap and the proc filesystem, but having discoverability as a first class concept - which might fit well to Plan 9's (and to some extent Unix's) "everything is a file" approach - would be nice.
The ability to change the system at runtime is also fascinating. I already mentioned the research of Stephen Kell in another comment [1] and there's a great video of one of his talks [2] in which he shows his approach to dynamically alter a program's behavior on Unix using shared library mechanisms (you can find the code on github [3]). I worked on related ideas in my PhD thesis 15 years ago, maybe it's time to dig these out again ;-).
However, these dynamic modifications should also be safe and accessible. I'm thinking of something related to AppleScript here, which allows to automate/scripts an amazingly large number of functions on OS X. Combining this with a browser that enables a user to discover the accessible OS functionality seems to be a good idea.
The persistent image ideas might also be useful again in the context of persistent main memory systems, which are becoming available now, e.g. with Intel Crosspoint [6]. There's so much prior work in persistent object systems also on the OS side that is almost forgotten nowadays, such as the Eumel/L3 system by Jochen Liedtke [4] (the inventor of the L4 microkernel) and the BirliX OS [5]. Sounds like a good time to re-evaluate these 1980's-era approaches.
I see some more recent work, such as Ksplice [7], as interesting related technology. What is really missing is an integration of many of these ideas into a consistent system.
[1] https://www.cs.kent.ac.uk/people/staff/srk21/
[2] https://www.youtube.com/watch?v=LwicN2u6Dro
[3] https://github.com/stephenrkell/liballocs
[4] Jochen Liedtke, A Persistent System in Real Use - Experiences of the First 13 Years, Third International Workshop on Object Orientation in Operating Systems 1993 - http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.53.7112
[5] W. E. Kühnhauser, H. Härtig, O. C. Kowalski, W. Lux, Mechanisms for Persistence and Security in BirliX, International Workshop on Computer Architectures to Support Security and Persistence of Information 1990, https://link.springer.com/chapter/10.1007/978-1-4471-3178-6_22
[6] https://en.wikipedia.org/wiki/3D_XPoint
[7] Arnold, Jeff and Kaashoek, M. Frans, Ksplice: Automatic Rebootless Kernel Updates, EuroSys 2009 - https://ksplice.oracle.com/doc/ksplice.pdf
> However, these dynamic modifications should also be safe and accessible. I'm thinking of something related to AppleScript here, which allows to automate/scripts an amazingly large number of functions on OS X. Combining this with a browser that enables a user to discover the accessible OS functionality seems to be a good idea.
I have been thinking a lot lately about HyperCard and Applescript. They represent a line of thinking that has more or less been abandoned by the commercial side of personal computing, which is really too bad. Applescript still exists, sure, but they've really let it die on the vine. Back in the late 90s I hoped they would make the whole front end of the OS in a HyperCard like system, so that everything could be manipulated by the user. Don't like how this part of the Finder works? No problem, just peel back a layer and see what's going on.
That's definitely a shame - and, as far as I know none of the open source Unix(-like) systems has exposed a similar functionality. I think D-bus should enable something like it, but I haven't had a closer look at that...
The possibilities of bringing back this way of thinking are tantalizing to me. The intersection of end user programming a discoverable system with the pervasiveness of the cloud could be really awesome. Something like a mashup of Automator, Hypercard, Heroku, and Smalltalk.
The only reason this doesn't exist is simply that nobody has decided to build it yet.
> The only reason this doesn't exist is simply that nobody has decided to build it yet.
It also takes a lot of time and effort, and so needs to be funded along with minimal interference from the bureaucracy of a business. Smalltalk and Hypercard came out of precisely those kind of environments. Today the goals of businesses are much different -- much more short term and quarterly focused. I don't have much faith that FOSS can come up with it either, since so much depends on part time work of loosely connected people.
I'm really interested in these types of topics too, so I'll add some more info to this thread.
I agree that things like Smalltalk could be a great alternative platform for IoT and edge computing. [1][2]
With GPIO access [3], very cool debugging abilities [4], and some capabilities for on-device AI/ML with TensorFlow [5], Smalltalk already has some real potential in this area. I've even tried Smalltalk on Amazon AWS (ARM) and it worked really well there too [6] with the ability to make a Docker Swarm if desired [7].
Full disclosure, I work with Instantiations (the company that develops VA Smalltalk), and they have developed a really cool VM that runs on ARM. Their interpreter VM is working on both ARM 32 / ARM 64 platforms at this point, and we've been doing a lot of testing of the LLVM-based VM with JIT compiler, PICs, and everything. And yes, it works for ARM v6, v7 and v8 so far. Small runtime images (around 1MB) or even the full blown VM with full IDE/language can be run on Linux ARM platforms (not bare metal yet...but maybe at some point.)
Feel free to reach out to me for more details or discussion.
Seems like a neat idea. I've always wanted a tiny LispOS or SmalltalkOS that I could carry around and use on a Pi. I just wish it would also have the modern necessities like web browser...etc.
One of the inspirations for my "crosstalk" Smalltalk system was Lukas Hartmann's Lisp-based "Interim OS" bare-metal system for the Raspberry (http://interim-os.com/), so you might want to give this a try.
I think Interim only runs on the Raspberry Pi 2 at the moment; porting Interim OS to use the circle library (which I use for crosstalk) should also be feasible and would help to support other Raspberry versions.
Lukas is also the creator of the MNT Reform open source notebook (using a Freescale i.MX8 quadcore ARM Cortex-A53) - see https://www.crowdsupply.com/mnt/reform for the currently running crowdfunding campaign.
Modern web browsing is, of course, more complex than the whole rest of the Smalltalk system...
I really like language-based operating systems. It is cool you did this.
My ideal though would be – it is all written in the language (Smalltalk/Lisp/whatever), with no use of other languages such as C/C++ (except possibly assembler, in which case my ideal is it comes with an assembler written in the language); and it is fully self-hosting in terms of the build process.
Of course, I realise my ideal is a lot more work than what you've done, and none of us has infinite time in which to tinker with things...
(I myself have some toy programming language interpreters I wrote, some in C, some in Java – I keep on thinking "oh, I should get rid of the C/Java, make it generate native code and then rewrite its core in itself" – but that's a lot of work and I've never had time for that.)
Great that you like it - thanks! Some of the work of Stephen R. Kell at Kent University also covers language-based OSes and how some of their ideas might be adapted in Unix (see [1] and [2]).
The Smalltalk-80 VM is specified in Smalltalk itself (in the "Blue Book" [3]) and the Squeak VM (a Smalltalk derivative) was written in a subset of Squeak itself [4].
The great thing is that it's a lot of fun to build this (at least for me) because it's actually possible to understand the whole system. I hope my students will have fun working on topics like these, too.
[4] Dan Ingalls, Ted Kaehler, John Maloney, Scott Wallace, and Alan Kay. “Back to The Future: The Story of Squeak, A Practical Smalltalk Written in Itself.” In Proceedings of the 12th ACM SIGPLAN Conference on Object-Oriented Programming (OOPSLA) 1997, 318–26, 1997 - http://files.squeak.org/docs/OOPSLA.Squeak.html
I'm a backer of that project and was looking forward to doing bare-metal experiments once the RISC-V alternate SOC comes out (I have implemented some of the RISC-V ISA in Squeak). Are you also a backer? It would be dope to get this working on the ARM cortex
I have already spent some money on the Pinebook Pro, so I hope that my next notebook after this will also be RISC V-based...
But the MNT reform (and the Purism Librem 5, which uses a similar SoC) is a great project and an interesting platform for bare-metal projects, since the CPU documentation is open and pretty good (I think the old Motorola heritage of Freescale shows here...) and there is also open source GPU support in the Etnaviv project [1].
I've been eyeing the new PinePhone as a potential candidate for a reimagining of Dynabook using Smalltalk-78 (which was built for the Notetaker) in a portable form factor.
The PinePhone is a very interesting platform. Unfortunately, the circle library I'm using only supports Raspberry Pis so far, so porting crosstalk to the Allwinner A64 in the PinePhone (even though the processor cores are the same as in a Raspberry Pi 3) is quite an effort.
Considering the large number of Allwinner-based systems out there (by Pine64, Olimex and many others), it might still be worth to spend some time on a port of circle. I can imagine this would make a nice project for Google's Summer of Code.
Smalltalk-78 shouldn't be significantly different to Smalltalk-80, so a bare metal version of it sounds feasible.
There's already a project to run Smalltalk-78 in a JavaScript-based VM [1].
It is interesting that you prefer Smalltalk-78 to -80. I also see that forks of Little Smalltalk 3 (which doesn't have metaclasses) are far more common that of LST 4 (which has them).
Very neat! How much work would it be to write ROS bindings so you could control robots easily with it? Googling around, I found one team that has done it with Pharo: http://car.imt-lille-douai.fr/category/software/pharos/
Robot control would be a fascinating applications (but my students had to program our robots in C [1] ;-)).
I am not an expert in ROS, but there have been other projects using Smalltalk for robotics, such as NXTalk [2] and a Squeak-based system [3]. A more general paper discusses the challenges in using Smalltalk for robotics applications [4].
So it should be possible to do this on the Smalltalk-80 bare-metal port. One problem that will definitely show up, though, is the limited memory space and number of objects of Smalltalk-80, since it uses a 16 bit VM. Modern robotics applications involving complex SLAM approaches, object recognition or LiDAR might be a bit challenging for this, so I think a more modern 32 bit based VM such as Squeak or Pharo is more appropriate here.
I love the idea of bare-metal ports to the Pi. I saw snakeware posted here not too long ago. Are there other examples? ColorForth would be interesting as well
In addition to the Interim Lisp system I mentioned in another comment, there are a number of bare-metal applications for the Raspberry on github (mostly ports of games).
Could someone share what one gains with such 'baremetal' ports? I see only the loss of common ground, well tested libraries and functionality.
It doesn't take a lot of Linux know-how to boot you favorite interpreter from /etc/rc.local or replacing /sbin/init (losing the libraries again, but at least keeping the kernel with the filesystem, network support and all the hardware drivers).
In general, lower code complexity results in fewer bugs and security problems, less code implies lower resource usage, so you could run the system with a smaller main memory, a slower CPU or lower power/energy consumption. In addition, if you plan to target real-time applications, setting up Linux to guarantee your real-time constraints is not trivial.
All of this is not that relevant for running Smalltalk-80 on the Pi, since Smalltalk was running happily on 16-bit machines with one or a few MBs of RAM.
So I guess the ultimate reason for doing this is "it can be done" - for the fun of hacking :-).
Accelerating the Smalltalk VM BitBlt primitives using the VPU would certainly be interesting. Currently, drawing is rather slow since lots of the functionality is implemented in interpreted bytecodes and rendering to the framebuffer is completely unoptimized.
Slight tangent, but: I first encountered a function named BitBlt when using the Win32 C API a long time ago. I had no idea that it originated in Smalltalk-72 in 1975 and the function name was just carried forward into the Windows API.
BitBlt was also used as the drawing primitive for an early graphics terminal developed at Bell Labs, the appropriately named "Blit" (https://en.wikipedia.org/wiki/Blit_(computer_terminal). Hardware implementations of BitBlt were available in the Amiga and Atari ST blitter chips (that's how I learned about BitBlt in the late '80s).
I had a quick look at the Win32 BitBlt API and it seems to be quite similar. Can you remember how much work was actually involved in mapping the Smalltalk primitives to Win32?
I probably shouldn't have described it quite like that.
This was in the days of Windows 2 (1988 or so) and I was hacking PPS Smalltalk to support larger, non-VGA displays. I ran the VM in 386 protected mode, because it needed large memory, where it blt-ed to an upper memory frame buffer. This was sort of a "bounce buffer" which in turn was blt-ed onto a Windows canvas by my Windows app which also reflected input events up to the VM.
Bill Gates was impressed when he saw a demo (or so I was told). Of course, he didn't realize the level of hackery involved to make it happen.
I believe Circle now has some OpenGL / GPU acceleration support. So textures could be used for bitblt.
I was thinking more along the lines of... I wonder if there's anything in the VPU (or other support hardware) that would be especially useful for accelerating a bytecode VM. But I can't really think of anything.
The screenshots are boring - it looks like any other Smalltalk-80 system :).
A picture of the setup to test one of my earliest versions can be found in my blog article (this needs to be updated) [1] and another more current one in my linkedin article [2] (I hope the direct link to the picture works for you). Both pictures at home using a crappy TV in Corona times, unfortunately...
Beware, there's a bug on ARM11-based Raspberry Pis (BCM2835 on Pi 1/Zero/Zero W and CM1) that crashes the VM when executing drawing functions. I'm currently investigating the problem.