Hey, awesome! And it produces a true goto as well, working on the bytecode level. I wonder if there's a way to remove the extra nops?
This is really, really useful for doing language-to-language translation. Without the ability to arbitrarily branch from basic block to basic block, it's possible for the basic block graph produced by a program in one language to be simply inexpressible in your target language. You end up having to fake it with a while loop and switch statement, which has horrible performance implications.
(Emscripten has some exotic algorithms to try and express the source program's basic block graph using the target language's structured programming constructs. This is vitally important to make goto-less Javascript perform well, but it can't work in all cases, and some C functions will force Emscripten to fall back to loop-and-switch. And, of course, these functions are typically the ones which have been carefully hand-optimised for speed...)
> I wonder if there's a way to remove the extra nops?
Yes, technically it would be possible to remove the code instead injecting NOPs. But then I'd have to adjust the jump targets. However, I don't think these NOPs are too bad. Note that goto jumps directly after the NOP ramp of the label, and the NOPs of the goto itself are never reached. The only scenario where NOPs are actually seen by the interpreter is when the natural code path visits a label.
EDIT: Instead re-assembling the bytecode and adjusting jumps, I went to use JUMP_FORWARD(3) instead 7 NOPs now. Note that there are still 4 NOPs left to fill the gap, these however are never executed as they are skipped by the preceding jump instruction.
https://github.com/snoack/python-goto/commit/2b0f5e5069cbb88...
It'd be interesting to know whether peculiar basic block graphs upset PyPy's JIT. Apparently Lua's JIT is much less good at optimising programs that don't look like normal Lua.
This is cute but of course horribly unsafe since it doesn't respect control statements.
Some examples you'd expect to work that will crash the interpreter:
@with_goto
def fun1():
while True:
for x in [1]:
goto .next
label .next
@with_goto
def fun2():
goto .next
for x in [1]:
label .next
@with_goto
def fun3():
while True:
try:
goto .next
finally:
print('skipped')
label .next
Note how Lisp's TAGBODY/GO doesn't have this problem.
(loop
(tagbody
(loop for x in '(1 2 3)
do (go next))
next
(print 'out)))
(OUT is printed repeatedly.)
How it works is that tagbody contains a bunch of labels mixed with forms. The tagbody establishes an exit point for GO expressions used in these forms. When a (go label) occurs, it works by abandoning the current sub-form being evaluated, and initiating a control transfer to the exit point, which then transfers control to the appropriate label.
The upshot is that whatever form is interrupted will cleanly unwind, as necessary:
Note that you can only have labels and GO in certain forms like TAGBODY and PROG, not just willy nilly in any construct. The labels of a given TAGBODY are all on the same level of nesting: immediate children of the TAGBODY form. SO this isn't possible:
Here, impossible is not considered a label associated with the tagbody, so the (go impossible) is branching to a nonexistent label. If it were allowed, of course it would create the problem of what becomes of the initialization of x.
Thus, compilers only have to reason about GO within specific constructs, and rely on those GO's to only be performing abandonment followed by a simple lateral move.
Jumps out of loops and other blocks work now. However, there are some limitations:
1. As I only have 4 bytes left to inject POP_BLOCK instructions, you can't exit more than 4 blocks with a goto. But this is handled and results into a SyntaxError rather than crashing Python.
2. Jumps into loops still doesn't work. However, it's handled now, and causes a SyntaxError as well.
3. When jumping out of a try- or with-block the finalizer is skipped.
it = iter(foo)
while True:
try:
i = next(it)
catch StopIteration:
break
print(42)
and jumping to print(42) would skip the init and first iteration expression.
Of course that's non-obvious behaviour, so in practice you would make it a compile time error. A careful language designer might even decide to not support goto at all. :)
If you're interested in this sort of thing, you might also check out https://github.com/llllllllll/codetransformer, which is a general-purpose library for doing these sorts of shenanigans in an almost-sane way. (Disclaimer: I'm one of the library authors).
As an example, I've got a PR open right now that lets you do things like:
@mutable_locals
def f():
out = []
x = 1
out.append(x)
locals().update({'x': 2, 'y': 3})
out.append(x)
out.append(y)
return out
assert f() == [1, 2, 3]
which works using a combination of ctypes hackery and replacing all LOAD_FAST instructions with appropriately-resolved LOAD_NAME instructions.
The idea that it needs optimizing, no doubt because someone's using it in a hot code, is hilarious. That's either the best satire or the worst truth I've heard all day.
In e.g. fortran there are labelled loops. So one can break out of an inner loop back to the main program flow. I know that this wouldn't be pythonic but sometimes I just want to finish the one off script and get back to something more fun and I miss this ability. I still don't think arbitrary gotos are great.
You can resolve this in basically every language by reworking your code in smaller pieces, and simply returning from a loop. Jumping left and right should only be signalling that you are Doing It Wrong (tm)
Can you do that without causing performance degradation e.g. through cache thrashing though? Fortran performance can vary significantly when you move stuff around.
I've seen exceptions masquerading as goto in the wild in multiple python codebases (and not because I added them). `raise HttpResponse(status=401, body="unauthorized")` inside some utility method is little more than goto exit in a lot of C code.
>I think goto's are fine, and they are often more readable than large amounts of indentation. That's _especially_ true if the code flow isn't actually naturally indented (in this case it is, so I don't think using goto is in any way _clearer_ than not, but in general goto's can be quite good for readability).
>Of course, in stupid languages like Pascal, where labels cannot be descriptive, goto's can be bad. But that's not the fault of the goto, that's the braindamage of the language designer.
Goto is absolutely the right tool for: "the instruction pointer finds itself here, and at this critical juncture in the algorithm, one would rather prefer that it were over there, for want of correctness, if not beauty."
One job is when auto-generating source code. It's much, much easier to generate a graph as a collection of blocks connected by various goto's and if statements.
This does not adversely affect modern optimizers - they'll figure out the loops from the goto graph. At least the D compiler does.
This is really, really useful for doing language-to-language translation. Without the ability to arbitrarily branch from basic block to basic block, it's possible for the basic block graph produced by a program in one language to be simply inexpressible in your target language. You end up having to fake it with a while loop and switch statement, which has horrible performance implications.
(Emscripten has some exotic algorithms to try and express the source program's basic block graph using the target language's structured programming constructs. This is vitally important to make goto-less Javascript perform well, but it can't work in all cases, and some C functions will force Emscripten to fall back to loop-and-switch. And, of course, these functions are typically the ones which have been carefully hand-optimised for speed...)