
Micro Python – a lean and efficient implementation of Python 3 - maxerickson
https://mail.python.org/pipermail/python-list/2014-June/672994.html
======
ngoldbaum
It was just pointed out to me that micropython starts an order of magnitude
faster than both python2 and python3:

    
    
        $ time ./micropython -c 'print(1)' 
        1
        ./micropython -c 'print(1)'  0.00s user 0.00s system 0% cpu 0.002 total
        
        $ time ./python2 -c 'print(1)'
        1
        python2 -c 'print(1)'  0.01s user 0.00s system 52% cpu 0.019 total
    
        $ time ./python3 -c 'print(1)'
        1
        python3 -c 'print(1)'  0.03s user 0.00s system 85% cpu 0.035 total

~~~
chubot
Yeah this is one of my longstanding annoyances with Python... the interpreter
is quite slow to start up because every import generates a ton of stat()
calls. Moreover, there is a tendency to use a very long PYTHONPATH, which
increases the number of stats.

It's basically doing random access I/O (the slowest thing your computer can
do) proportional to (large constant factor) * (num imports in program) *
(length of PYTHONPATH).

When you can use it, a shebang of

#!/usr/bin/python -S

can speed things up substantially.

Perl starts up an order of magnitude faster too (more like 3ms than 30ms).
Ruby seems to have the same problem as Python.

~~~
pekk
What are you doing that you are noticing this routinely and finding it to be a
large annoyance?

~~~
price
Any command-line script in Python that uses a significant codebase or a
significant number of libraries. For me the main example is Mercurial -- it
takes about 200ms (on my Mac) to run even a trivial command, and that's plenty
long enough to get in the way.

It's about 60ms on my other laptop (Linux), and that's just at the threshold
of noticeable and annoying.

------
michaelhoffman
> Supports almost full Python 3 syntax, including yield (compiles 99.99% of
> the Python 3 standard library).

What parts of Python 3 syntax are missing? Which parts of the library don't
compile?

~~~
maxerickson
It's discussed some here:

[https://github.com/micropython/micropython/wiki/Differences](https://github.com/micropython/micropython/wiki/Differences)

But I guess they are still in the process of writing that/filling it out.

The biggest difference listed is probably no Unicode. Another significant
point is that lots of library functions are only partially implemented.

~~~
AceJohnny2
> The biggest difference listed is probably no Unicode.

Funny, considering that was the core reason for Python3 in the first place
(though I can't immediately come up with a reference).

~~~
maxerickson
This is sort of the design document for Python 3:

[http://legacy.python.org/dev/peps/pep-3100/](http://legacy.python.org/dev/peps/pep-3100/)

Along with this one:

[http://legacy.python.org/dev/peps/pep-3000/](http://legacy.python.org/dev/peps/pep-3000/)

(Of course there was lots and lots of thought and discussion that is not
reflected in those documents)

I would characterize the Unicode/bytes change to simply be the most visible
and disruptive change, it didn't really stand apart as far as being a reason,
it was one of many.

~~~
AceJohnny2
From what I remember from various posts, most other changes could be done
gradually on the 2.x branch. Unicode couldn't.

I may be mischaracterising it :)

That said, I suppose it's wise to base MicroPython on the "live" branch of
Python (3.x), not the legacy one (2.x), even if Unicode support is too costly
for them.

------
stinos
Slowly gaining traction it seems. Has been posted before without even getting
comments
([https://news.ycombinator.com/item?id=6763123](https://news.ycombinator.com/item?id=6763123))
and then some comments
([https://news.ycombinator.com/item?id=6996692](https://news.ycombinator.com/item?id=6996692))
but already has more points now :]

~~~
mech4bg
The kickstarter for the controller boards got a few votes too, but didn't get
much traction:
[https://news.ycombinator.com/item?id=6727326](https://news.ycombinator.com/item?id=6727326)

Glad to see people getting on board now, it's pretty awesome. Fitting Python
onto such a resource limited microcontroller is a great achievement.

------
ethanpil
WOW!! I see this as a serious competitor to Lua. I think lua because so
popular because of its small footprint and easy embedability... Perhaps this
is the first step to a Python revolution...

I expect to see a lot of interesting devices and software embedding Micro
Python.

~~~
plorkyeran
It's still well over an order of magnitude bigger than Lua. MicroPython is
~200k non-blank-non-comment lines of code (as reported by cloc), while Lua 5.1
is 12k.

~~~
carbon12
Actually, most of that SLOC is in the stmhal/ port, of which ~100k is the ST
library.

The true count comes from the py/ directory, for which cloc gives only 25k.
And there are a lot of extra bits there, eg the inline assembler, the Thumb
and X64 assembler helpers, etc.

EDIT: without the additional assemblers, cloc gives 22k. Remember that Python
has ints, floats, complex and bignum, and these are all included in Micro
Python. So that 22k SLOC includes complex arithmetic and a 1k bignum
implementation.

------
thomasahle
It always strikes me, that the things that parts of python that create
problems for people trying to optimize it, are all things of relatively small
importance. Like __new__, __del__ semantics and changing the stack by
inspection. I wish Python3 had just let go of the slow scripting-only parts.

~~~
dbpatterson
The problem is that often while regular users are not directly using those
features, they are using libraries that do use them. As a concrete example,
people writing webapps with Django probably (often) aren't using some of the
crazier class metaprogramming, but they are using Django's models, which use
lots of it.

------
knappador
Attn: Kivy can really use this for mobile deployements, but we use Cython and
almost everyone needs cpython C modules. We need to investigate making drop-in
replacements for Python.h and other cpython headers to stub out reference
counting etc, which micro-python doesn't use. The compiler will just skip the
calls entirely in some cases.

If these drop-in replacements are technically feasible, not only does Cython
magically work, but so does a lot of the Python ecosystem. There's probably
more work to get linking and other aspects working, but this might also be a
model for moving to alternative Python implementations in general. As long as
straight Python "just works" and the headers are available for compiling C
modules, we're very close to having a sensible alternative to cpython that can
grow without being wedded to it.

Please comment on technical requirements. Issue opened here:
[https://github.com/micropython/micropython/issues/658](https://github.com/micropython/micropython/issues/658)

------
meepmorp
The project's website, from the linked post:

[http://micropython.org](http://micropython.org)

Apparently, there's a kickstarter for a dev board that runs this version of
python. Looks interesting.

~~~
analog31
Got one!

Haven't fired it up yet.

------
justincormack
There is a talk about it here from a few weeks ago
[https://www.youtube.com/watch?v=hmGISrwPtyA](https://www.youtube.com/watch?v=hmGISrwPtyA)

------
ant_sz
This project provide a so-called pyboard. It is interesting! The board's
design can be found here
[https://github.com/micropython/pyboard](https://github.com/micropython/pyboard)

------
ionforce
Does it strike anyone else as odd that garbage collection is used in such a
resource constrained environment?

Also tangent question, what is it about languages like Python and Ruby that
make it more amenable to reimplementation than Perl?

~~~
Demiurge
To take a stab at the second question, I think BNF that fits on one page
[https://docs.python.org/3.4/reference/grammar.html](https://docs.python.org/3.4/reference/grammar.html)
vs [http://perldoc.perl.org/perlfaq7.html#Can-I-get-a-
BNF%2fyacc...](http://perldoc.perl.org/perlfaq7.html#Can-I-get-a-
BNF%2fyacc%2fRE-for-the-Perl-language%3f)

This basically means Perl is very complex and its grammar can be self
contradicting, such that behavior is undefined. C++ has a similar problem to a
lesser extent.

~~~
riffraff
To expand on the non-syntax, perl has an incredible amount of language-level
features, which may appear very weird to people who have only seen it from
afar.

For example, perl formats[0] are language-level support for generating
formatted text reports and charts, which is basically a whole sublanguage
(much like perl regexen).

[0]
[http://perldoc.perl.org/perlform.html](http://perldoc.perl.org/perlform.html)

~~~
Demiurge
That's pretty crazy. I used Perl a lot, but haven't seen that feature :)

~~~
emmelaich
Printing nicely formatted reports was one of Larry Wall's original motivations
for Perl.

------
andybak
Is this a route to a good Python stack for mobile?

Any interest from the Kivy team or related projects?

~~~
pekk
People are constantly running Java on mobile and that's very heavy. With phone
storage into the gigs. There's no reason Python couldn't have been used before
Micropython, except that vendors didn't want to allow it in.

~~~
andybak
It's about:

a) executable size (packaging a large runtime with every app b) RAM usage

Python suffers on both counts on mobile.

------
dekhn
This is pretty neat, I'm gaining an appreciate for microcontrollers and using
high level languages is attractive.

That said, the alternative I'm exploring is to upload a standard Firmata
firmware to the microcontroller, then drive it remotely, say from python on a
full computer (like raspberry pi).

i think the interestng area comes when you can actually put a fairly "smart"
microcontroller firmware on the device (GRBL) and then program it remotely,
say with a scripting language. At that point, the boundaries between a
firmware that is a dvice controller, and a firmware that is an open-ended
remotely drivable VM starts to break down. Interesting area.

------
daGrevis
If someone is wondering what's going on with unicode support:

> No unicode support is actually implemented. Python3 calls for strict
> difference between str and bytes data types (unlike Python2, which has
> neutral unified data type for strings and binary data, and separates out
> unicode data type). MicroPython faithfully implements str/bytes separation,
> but currently, underlying str implementation is the same as bytes. This
> means strings in MicroPython are not unicode, but 8-bit characters (fully
> binary-clean).

~~~
maxerickson
This was linked in a later post to the mailing list thread:

[https://github.com/micropython/micropython/issues/657](https://github.com/micropython/micropython/issues/657)

They plan on improving the Unicode support.

------
thearn4
Just got my board (kickstarter backer). Can't wait to take a few hours to play
with it!

~~~
Fuzzwah
Could you share what projects you can imagine this being great for?

I'm excited by the idea of it all but I can't seem to come up with a project
that I'd want to mess with enough to purchase one.

Maybe I just need more coffee....

------
ForHackernews
This is pretty cool. Could it run on something like an Arduino?

~~~
eric_bullington
If you're talking about the traditional 8-bit AVR Arduinos, the answer is
almost certainly no. The AVR boards like the ATmega328 (on the Uno) only have
a few kbyte of RAM available. But MicroPython apparently can run on some
pretty tiny and inexpensive boards: namely the Arm Cortex microcontrollers.

They are a bit more expensive than the chip in the Arduino, but well under $20
per chip. You can buy very nice evaluation Cortex boards for ~$10 from ST and
Freescale[1,2].

Incidentally, Arduino now makes a board that uses the Arm Cortex chip: the
Arduino Due[3]. I believe it's around $30.

1\.
[http://www.element14.com/community/docs/DOC-46626/l/element1...](http://www.element14.com/community/docs/DOC-46626/l/element14-presents-
freescale-kinetis-l-based-freedom-development-board)

2\.
[http://www.st.com/web/catalog/tools/FM116/SC959/SS1532/PF252...](http://www.st.com/web/catalog/tools/FM116/SC959/SS1532/PF252419)

3\.
[http://arduino.cc/en/Main/arduinoBoardDue](http://arduino.cc/en/Main/arduinoBoardDue)

~~~
chillingeffect
It appears from [0] that the chip of choice is the STM32F045RGT (datasheet
[1]). This is from the Cortex M4f series, which includes such wonderful things
as a hardware floating-point unit. That is wonderful news, although, this
board appears to have no external memory, so it would be limited to 128kB.

[0]
[https://raw.githubusercontent.com/micropython/pyboard/master...](https://raw.githubusercontent.com/micropython/pyboard/master/schem.png)
[1] [http://www.alldatasheet.com/datasheet-
pdf/pdf/510587/STMICRO...](http://www.alldatasheet.com/datasheet-
pdf/pdf/510587/STMICROELECTRONICS/STM32F405RG.html)

~~~
justincormack
You can get versions with external memory.

------
rectangletangle
Anyone know the "scale" of this re-implementation? Is it simply a fork of
CPython with some performance tweaks, or is it a primarily new code-base?
Regardless it's pretty cool.

~~~
keeperofdakeys
It'd definitely be a new code-base. In terms of comparison, I'd guess this
uses "cheaper" algorithms, that use less memory, and don't scale as well.
There are many things that CPython does behind the background that most people
don't fully appreciate.

Edit: Here is a good document on the differences
[https://github.com/micropython/micropython/wiki/Differences](https://github.com/micropython/micropython/wiki/Differences)

------
kelvin0
I am truly very happy that he was able to fund this cool project. The funding
went very well it seems, more more than 6x the initial proposed funding!

I might just quit my job and do something similar!

------
ashish01
Is there a way to try it on windows ?

~~~
corysama
[https://github.com/micropython/micropython/tree/master/windo...](https://github.com/micropython/micropython/tree/master/windows)

> This is experimental, community-supported Windows port of MicroPython. It is
> based on Unix port, and expected to remain so. The port requires additional
> testing, debugging, and patches. Please consider to contribute.

~~~
stinos
It works ok though: all tests pass both for msvc and mingw

------
lerouxb
So upset that I missed this kickstarter. I've been obsessively refreshing the
homepage to see if I can buy a micropython board yet.

------
neil_s
Nice! How much of this optimization can be ported back to CPython, keeping
compatibility but improving memory usage and speed?

~~~
carbon12
A lot of the internals are significantly different, and would be difficult to
back port. Eg, a Micro Python object is a machine word which might be a
pointer to a real object, a small integer, or an interned string. By stuffing
small integers (and strings) into the unused bit-space of a machine word,
Micro Python can do a lot of integer operations without allocating heap RAM.
This is in contrast to CPython where all integers are on the heap (although
the ones below 257 are cached).

Another example: calling a function in CPython requires a heap allocation of
the call frame. In Micro Python this is generally not needed (unless the
function call is significantly complicated).

~~~
phkahler
So never mind back porting. Can the implementation be extended to 100 percent
compatibility without losing the benefits? In other words, could it replace
cpython?

~~~
ecnahc515
I would rather have pypy replace cpython than this. Seems like a better bet
too due to the design goals.

------
lacion
i got one of the dev boards like a month ago, i was part of the kickstarter.

this thing is amazing, it took me a week to stop using arduino.

------
sandGorgon
we use a lot of python on the windows platform - especially the win32 and
multiprocessing modules. Would love a variant of micropython on windows that
can support them.

There a large number of embedded Windows CE devices that run in retail.

------
zobzu
any benchmark vs cpython avail? (not startup speed). just curious.

------
brassattax
Anyone else read that as "Monty Python"?

