
A Python Interpreter Written in Python (2016) - jstrieb
http://aosabook.org/en/500L/a-python-interpreter-written-in-python.html
======
chubot
I found this code very useful. I've been spelunking in CPython and mostly I
understand it, and can find my way around.

But ceval.c is another beast entirely, being full of macros and gotos, not to
mention being 5000 lines long.

The interpreter loop starts here:
[https://github.com/python/cpython/blob/master/Python/ceval.c...](https://github.com/python/cpython/blob/master/Python/ceval.c#L540)

So I appreciate seeing the algorithm laid out in Python. In particular it
clarifies that there are three separate stacks:

1\. call stack

2\. block stack for try/except, loops, etc.

3\. value stack for evaluating expressions

It also clarifies how generators work, which IMO is very difficult to follow
from the C source (i.e. without a design doc to go along).

I wrote about some of my recent work here:
[https://www.reddit.com/r/oilshell/comments/8b0n6z/opyreadmem...](https://www.reddit.com/r/oilshell/comments/8b0n6z/opyreadmemd_at_oilshelloil/)

BTW I have shell scripts running under _triple interpretation_ : CPython,
byterun, and OSH itself :) This is just an experiment toward writing my own
VM, not for the final product. The release binary doesn't use byterun at all.

EDIT: There is also a companion bytecode compiler that I mention here:
[http://www.oilshell.org/blog/2018/03/27.html](http://www.oilshell.org/blog/2018/03/27.html)
(but I'm not using it, I'm using the one that used to be in the Python 2
stdlib, which is entirely separate from the one used by the interpreter
itself.)

~~~
00ajcr
Philip Guo has an excellent set of video lectures on CPython internals, which
includes an overview of parts of ceval.c: [http://pgbovine.net/cpython-
internals.htm](http://pgbovine.net/cpython-internals.htm)

~~~
chubot
I've watched several of those, and they are good. Though for me, playing with
code is a different type of learning than watching videos. I should go back
and watch the one on ceval though.

------
asperous
Just in case anyone doesn't know, there's actually a real, live project that
does this:

[https://pypy.org/](https://pypy.org/)

(Technically the base interpreter is a subset/restricted version of Python
however). The reason this is done (my understanding), is that a the python
source code for the base interpreter can be fed into a JIT generator, which
produces a Python interpreter that can perform JIT optimizations.

~~~
masklinn
> Technically the base interpreter is a subset/restricted version of Python
> however

The base interpreter is _written_ in a subset/restricted version of Python,
but it's still Python. You can run Pypy as an interpreter on top of CPython or
pypy.

And IIRC the restrictions mostly have to do with static type inference so
you're mostly limited in how dynamic/weird the program gets the other big
limits being magic methods being unavailable (for user-defined types) and most
of the stdlib being off-limits.

So it's Python in a straightjacket, but for most interpreter implementations
it's probably fine, as long as you're not using an overly interesting parsing
strategy (hello pratt parsers) an interpreter tends not to use the more
dynamic/odd corners of the language I think.

~~~
xapata
There's also interesting pieces like sections not in try/except have reduced
checking for out-of-bounds and type mismatch. I don't grok it yet, mostly
because I've had no need to use that flavor of Python. But in some ways it's
providing flexibility instead of constraints. Flexibility for the interpreter,
not necessarily the programmer.

------
0x7f800000
But can the interpreter interpret itself?

~~~
niroze
It isn't a LISP :P

~~~
chrisseaton
You don't need a language to be a LISP in order for them to be able to
interpret themselves. There are several Java interpreters which can interpret
themselves, for example.

~~~
kazinator
You just need it to be a Lisp in order to do it in such a way that you get
confused whether you're in the interpreted language or the host one.

~~~
yorwba
Byterun confuses host and interpreter in plenty of places.

It uses the host Python for attribute resolution (all code in getters and
setters runs on the host), exception handling (which combines with the
previous point to make some NameErrors impossible to catch) and even function
calls (but all functions are wrapped so that their _call__ switches back to
byterun as the interpreter).

Because of that confusion, byterun isn't very useful if you want to be really
independent from the host interpreter; it's just too easy to escape from the
VM. As a learning exercise however, it is helpful for understanding Python's
innards at a level higher than C.

~~~
kazinator
I mean actually getting confused.

------
sethgecko
I made something pretty similar the other day. An interpreter for the bitcoin
scripting language with a stack and the most used opcodes. Code here :
[https://github.com/mcdallas/cryptotools/blob/master/btctools...](https://github.com/mcdallas/cryptotools/blob/master/btctools/opcodes.py#L226)

------
kgoutham93
Slightly off topic... If processor can only understand assembly grammar,
what's the difference between an interpreter and a compiler?

I understand that interpreter can directly execute the intermediate byte code
(giving us portability benefits), but at some point it must convert the byte
code into platform independent machine code. Right?

What exactly differentiates an interpreter and a compiler?

------
jwilk
(2016), according to the HTTP headers:

    
    
      Last-Modified: Sat, 09 Jul 2016 12:15:59 GMT

~~~
codetrotter
I’m going to start having my server send

    
    
        Last-Modified: Mon, 20 Apr 3018 13:37:00 GMT
    

Then we will see what people say when my stuff is posted :^)

Then again my stuff probably wouldn’t get much attention to begin with.

But if it did...

People would have to admit that they are all living in the present whereas I
was living in the future.

~~~
make3
pay your registrar to return a WHOIS on Mars.
[https://news.nationalgeographic.com/2016/10/planets-maps-
exp...](https://news.nationalgeographic.com/2016/10/planets-maps-exploring-
mars-space-science/)

------
CodeArtisan
an evaluator that is written in the same language that it evaluates is said to
be metacircular[1]. Note that this is not the case here because it's not a
python interpreter but a cpython bytecode interpreter. It would have been
metacircular if it were written in that bytecode.

[1]
[https://sarabander.github.io/sicp/html/4_002e1.xhtml#g_t4_00...](https://sarabander.github.io/sicp/html/4_002e1.xhtml#g_t4_002e1)

------
carlmr
Ouroboros would be a good icon for this project.

~~~
fermigier
It's already the PyPy logo: [https://pypy.org/image/pypy-
logo.png](https://pypy.org/image/pypy-logo.png)

