> I've been working privately on such a "systems programming REPL" in my free ti...

matheusmoreira · on Feb 10, 2023

Thanks, that's a nice project I didn't know about! Always happy to see more projects along these lines!! I'm not sure to what extent it permits systems programming though. I searched the repository for common system calls like mmap and didn't find anything. I assume it links to either libc or JVM.

I suppose I'm challenging myself. What I had in mind is much lower level: a Lisp thing where I can use the Linux system calls directly. It's gonna look like this:

  ; mmap some memory
  (set memory (mmap 0 4096 '(read write) '(private anonymous) -1 0))

  ; query the kernel for some data
  ; terminal size for example
  ; have the kernel put the data at the start of that memory
  (ioctl 1 'TIOCGWINSZ memory)

  ; memory now points to a struct winsize
  ; decode the 4 unsigned shorts
  ; first two unsigned shorts are the terminal's rows and columns

The language runtime is completely freestanding: it doesn't link to any library at all, not even libc. I made it so eval supports a special system-call function which executes a Linux system call from C, and I want to build literally everything else on top of that. I want to be able to run strace on any coreutils binaries, see what system calls they make and then implement the same thing on top of the system-call primitive. It should be possible to make a coreutils module that contains an mv function, for example.

  ; boils down to:
  ; (renameat2 'fd-cwd "file" 'fd-cwd "renamed" 'no-replace)
  (mv "file" "renamed")

I had to use static allocation to pre-allocate a stack of Lisp cells when the process is loaded just to get it to evaluate at all. Now I'm trying to get the garbage collector to work so I can get it to bootstrap to a point where it can allocate memory, read files and load more code. I wish I had something real to show for all this effort but right now it's not real yet.

kazinator · on Feb 11, 2023

TXR Lisp:

  (defvarl TIOCGWINSZ #x5413)

  (typedef winsize (struct winsize
                     (ws-row ushort)
                     (ws-col ushort)
                     (ws-xpixel ushort)
                     (ws-ypixel ushort)))

  (with-dyn-lib nil
    (deffi ioctl-winsz "ioctl" int (int ulong : (ptr-out winsize))))

Then:

  $ txr -i winsz.tl 
  TXR doesn't really whip the llama's ass so much as the lambda's.
  1> (let ((ws (new winsize)))
       (ioctl-winsz 0 TIOCGWINSZ ws)
       ws)
  #S(winsize ws-row 37 ws-col 80 ws-xpixel 0 ws-ypixel 0)

matheusmoreira · on Feb 11, 2023

That's amazing! I'll definifely look at it for inspiration.

Am I correct in assuming that it depends on the C library for its system call support? The code seems to be a Lisp equivalent of:

  self  = dlopen(NULL, flags);
  ioctl = dlsym(self, "ioctl"):

I'd like to have system call support as a feature built right into eval. Maybe a JIT compiler that emits code with Linux system call calling conventions whenever eval encounters a (system-call ...) form.

Do you know more low level Lisp systems?

lispm · on Feb 11, 2023

Basically all/most Common Lisp implementations have a foreign function interface. Those who run on some UNIX or Linux need support for low level access.

See for example the SBCL sources for mmap and ioctl.