
Porting Python to GRUB: “Networking without an operating system” - d33
https://lwn.net/Articles/692638/
======
JoshTriplett
I'm one of the authors of BITS, and the presenter of the PyCon presentation
this article discusses. Happy to answer any questions here.

~~~
yitchelle
I am always interested to hear when a language is ported to a new plaform, in
this case Python (or a subset of) on GRUB. The question would be why? Is this
because you have lots of test cases already in Python and can easily deployed
those test cases on using this?

~~~
JoshTriplett
Originally, it was because we had a lot of one-off test cases _not_ written in
Python. Before BITS, most BIOS test programs consisted of one-off DOS
programs, or later one-off EFI programs. If you wanted a new test, or you
wanted a customer to quickly gather some information for you, you sent them a
new test program, or a bootable image. Now, many of those one-off tests have
become Python one-liners, and you can use Python to explore the system with an
interactive REPL.

Python also lets us provide additional APIs to make scripting easy. For
instance, we include the complete ACPICA interpreter, so you can evaluate an
arbitrary ACPI method with arguments and process the result. And we have an
FFI with EFI support, so you can locate and call an arbitrary EFI protocol.

~~~
djsumdog
Wow. I thought this was one of those "just because you can," projects, but it
looks like there are some real practical uses for firmware developers/testers.

~~~
JoshTriplett
Yeah, we wrote BITS almost entirely for practical reasons. Several groups at
Intel use it to test both the BIOS and the CPU itself. (You have to have
enough functionality to boot an OS, but once you have that, BITS can easily
test things like CPU power management or MSR configuration.)

The module that displays presentation slides using
EFI_GRAPHICS_OUTPUT_PROTOCOL, on the other hand, was definitely "because we
could".
[https://www.youtube.com/watch?v=bYQ_lq5dcvM#t=22m56s](https://www.youtube.com/watch?v=bYQ_lq5dcvM#t=22m56s)

------
vanderZwan
Was the talk recorded and can we expect it on the Pycon 2016 youtube
channel[0] at some point? It's not there right now as far as I can tell.

[0]
[https://www.youtube.com/c/pycon2016/videos](https://www.youtube.com/c/pycon2016/videos)

~~~
JoshTriplett
Yes:
[https://www.youtube.com/watch?v=AlkKvetGFSk](https://www.youtube.com/watch?v=AlkKvetGFSk)

The first few minutes of the talk got cut off; the conference organizers have
the original footage and plan to re-cut and re-upload it at some point.

The talk from PyCon 2015 on the Python port itself is available as well:
[https://www.youtube.com/watch?v=bYQ_lq5dcvM](https://www.youtube.com/watch?v=bYQ_lq5dcvM)

~~~
vanderZwan
Huh, I loaded all the way back to Pycon 2015 videos CTRL+F "Josh" or
"Triplett" didn't find anything on my laptop. Strange

Anyway, thanks for the links!

------
brian_herman
Could you make a python rumpkernel with this?

------
denfromufa
How is this different from micropython?

~~~
detaro
Micropython is a independent implementation of Python with some
microcontroller-specific extensions and able to run on smallish
microcontrollers, this is a slightly modified "standard" CPython für x86.

------
sandGorgon
why would they not use lua ? which is pretty popular (and pretty much
designed) as embedded scripting languages. Not to mention much more
lightweight than python.. and has a much smaller footprint.

Nginx is a pretty good example of this.

~~~
JoshTriplett
> why would they not use lua ?

The target audience for this is BIOS developers and firmware engineers.
Several existing development tools in that area already support Python,
including Simics and the Intel ITP, so the target audience is used to it.

Lua also has several other impedance mismatches with the target audience, such
as 1-based array indexing (remember, the target audience writes assembly and
C).

While people often describe Lua as "easy to embed", personally, I don't find
Lua's stack-based C API easy at all; in particular, the references to stack-
based indexes that change when you push or pop the stack feels like working
with stack-based parameters in assembly without a base pointer, where the
offset of a given location can change at any time. I much prefer the Python C
API.

Finally, Lua would require additional libraries to provide a useful
environment, precisely because it can assume so little about its environment.
For instance, we'd need to bundle one of the Lua socket libraries in addition
to Lua itself, as well as an HTTP library. Python's "batteries included"
standard library works nicely here.

~~~
sandGorgon
thank you for that. That explains a lot ! But Lua is already in Grub right ?
wouldnt it have been more effective to just extend that anyway ?

~~~
JoshTriplett
Lua isn't part of upstream GRUB; it lives in a separate module, "grub-extras".
As far as I can tell, it also wouldn't be easy to extend that with additional
functionality or modules without forking it, due to the way GRUB builds out-
of-tree modules.

Note that this port of Python is almost as old as grub's Lua; we've been
working on BITS since ~2009, and it has had Python support since May 2011.

For reference, the Python port consists of less than 1000 lines of code, most
of which provides compatibility implementations of standard C and POSIX
functions that Python expects. (Plus another couple thousand lines
implementing additional C Python modules like "_smp", "_acpi", and "_efi", but
those are getting smaller as we move more code into Python using ctypes.)

------
kbradero
is BITS building for ARM ?

~~~
JoshTriplett
No. We don't have any need for that ourselves, but if someone wanted to work
on that, and produced a clean patch that wasn't filled with #ifdefs in C and
ifs in Python, we'd take it.

The biggest challenge would be adding equivalents of the assembly that
implements SMP support (waking up other CPUs and putting them into a loop
ready to run code on request).

