1. For early iterations / similar projects, could all-metal screw terminals[1] accept power internally?
2. Would supporting UXN/Varvara[2] be an option?
More on these questions below after initial comments.
## Initial Comments
> if you aren't worried about going batteryless, or about mass production, you could literally buy an alphasmart neo and wire its keyboard and display up to an esp32 or something
I could, but it's a limited in availability. The keyboard is pretty important in my opinion.
i'm glad to hear you find it appealing! i'll explain some of my thinking on these issues,
some of which is specific to my own eccentric objectives (nobody designs computers to last decades) and some of which is more generally applicable
screw terminals or post terminals are a good idea if you're going to have charging
and want longevity. but charging implies batteries.
and batteries themselves induce lots of failures; they typically
have a design lifetime of two years, more than an order of magnitude too low
for the zorzpad. by itself that's not a fatal flaw, because you could replace
them, but it becomes fatal in combination with either of the two supplementary
flaws:
1. battery shelf life is limited to about 10 years, except for lithium thionyl chloride and so-called 'thermal batteries' that use a molten-salt electrolyte (kept frozen during storage). consequently, if your computer needs batteries,
after 10 years you're dependent on access to not just a market but a full-fledged industrial economy to keep your computer running. a friend of mine in rural romania
has been trying to build a usable
lead-acid battery from scratch to provide energy storage for his solar panels, and it's motherfucking difficult without that supply chain
2. batteries have a lot of failure modes that destroy the devices they power. overheating, swelling, leaking corrosive electrolytes, and too much voltage are four of them. if these failure modes were very rare (like once in 100 battery-years) that might be okay, but they aren't
for me these are fatal flaws, but for others they may not be. if you can afford batteries, you can run for one hell of a long time on a single 18650 lipo on a milliwatt. a coin cell can deliver a milliwatt for a month
as for uxn, uxn is the first attempt at frugal "write once, run anywhere" that's good enough to criticize. everyone should read devine's talk about its aspirations (https://100r.co/site/weathering_software_winter.html)
however, uxn is not frugal with cpu, or fast, nor is it good for
programmer productivity, and it’s not all that simple. It’s designed
for interpretation, which inherently means you’re throwing away 90% or
95% of even a single-threaded cpu, and it’s not designed as a
compilation target, which means that you’re stuck programming at the
assembly-language level. This is pretty fucking fun, but that’s
because it makes easy things hard, not hard things easy; the offgrid
notes lionize oulipo (https://100r.co/site/working_offgrid_efficiently.html)
as a result of the efficiency problems, the platforms with complete uxn
implementations (see https://github.com/hundredrabbits/awesome-uxn#emulators)
are the luxurious ones: microsoft windows, linux, amigaos, the essence
operating system, anything that can run sdl or lua and love2d, gameboy
advance, nintendo 64, nintendo ds, playdate, and the nook ereader: all
32-bit or 64-bit platforms with ram of 4 mebibytes or more, except
that the gba only has 384 kibibytes, and there were amigas with as
little as 512k (though i don’t know if they can run uxn). more
limited machines like the gameboy, ms-dos, and the zx spectrum do not
have complete uxn implementations
(384k is the same amount of ram as the apollo3, so it's probably feasible, as you say)
the uxn machine code is basically forth. forth is pretty aesthetically appealing to me, and i've gotten to the point where forth code is almost as easy for me to write as c code (though i have to look things up in the manual more often and have more bugs). however, it's much, much harder for me to read forth than it is to read c. it's harder for me to read forth code than to read assembly for any of 8086, arm, risc-v, i386, or amd64. possibly i just haven't written enough forth to have the enlightenment experience yet, but currently my hypothesis is that forth is just hard to read and that's it, and that the really great thing about forth is not actually the language but the interactive development experience
the varvara display model is not a good fit for the memory-in-pixel displays, which only have one bit per pixel, while varvara requires two.
also, evaluating things 60 times a second is not a good fit for low-power systems of any kind.
and i'm not sure how well the existing varvara applications will work at 400×240 or tolerate a requirement
to use fonts many pixels tall in order to be readable
as for tools for software development, program launching, editing text, editing fonts, editing other graphics, and sequencing music, i'm confident i can write those myself if i have to. i've written a compiler in a subset of scheme that compiles itself, a compiler in a forth-like language that compiles itself, various toy text editors, various virtual machines in under 150 lines of code, and various pieces of music synthesis software, and i've designed a few fonts of my own (though not using software i wrote)
maybe i should set up a patreon for this or something, maybe people would be interested in supporting the project
TL;DR: Thank you for confirming some mismatch in display hardware and project goals
> also, evaluating things 60 times a second is not a good fit for low-power systems of any kind.
Agreed. As to a terminal-mode UXN, you pointed out:
> as for uxn, uxn is the first attempt at frugal "write once, run anywhere" that's good enough to criticize. everyone should read devine's talk about its aspirations
I think it's the primary use case like Java Applets and Flash preceded JS as iterations of sorta-portable and low efficiency tools which make up for it with sheer volume of high-familiarity material.
> nor is it good for programmer productivity
I'm curious about this. Could you please elaborate?
When trying earlier uxntal versions, the worst part seemed to be the string syntax. If it wasn't because it seemed easier at the time, my guess is the pain point might be intentional nudges away from making things the designer doesn't like.
> and it’s not all that simple.
This is where I get confused:
* Can you explain more about your views on this?
* Do you favor a specific structure of hardware, OS code, and user scripts, or whatever ends up offering the best use of power and long-term device durability?
> i'm not sure how well the existing varvara applications will work at 400×240 or tolerate a requirement to use fonts many pixels tall in order to be readable
Counting mW and decades implies text > GUI. Despite 100r & fans favoring GUI, there's an UXN terminal mode and a few math and terminal libraries written for it.
As to GUI, many of the ecosystem's GUI programs are FOSS with room to shrink the graphics. There's a decent 4x6 px ASCII font which is CC0 and could help, but as fair warning, it's awful at accents and umlauts.
> possibly i just haven't written enough forth to have the enlightenment experience yet, but currently my hypothesis is that forth is just hard to read and that's it, and that the really great thing about forth is not actually the language but the interactive development experience
This seems to be a universal opinion since REPL isn't rare anymore. Being portable and easy to bootstrap are probably the main draw now. In addition to 100r's work, I think remember hearing ${NAME}boot or another project used it to simplify porting.
> the varvara display model is not a good fit for the memory-in-pixel displays, which only have one bit per pixel, while varvara requires two.
Ah, this is what I suspected. I wasn't sure I understood the display you mentioned, so thank you for confirming the hardware mismatch.
Although your blog showed you'd considered two identical side-by-side displays[1], I have a radical idea which might not be a good one:
1. A "Full" display of W x H characters (the Sharp display or whatever you replace it with)
2. A "Fallback" 1 or 2 line display (2nd line reserved for editor info / UI?)
A single 1-bit editable line is so hard to work with that ed[2] isn't even installed by default on many Linux distros. However, people did once got real work done with it and similar tools.
As a funny sidenote, this is where non-colorForth[3] forths may align with UXN's dev. He's a Plan9 fan and uses its Acme editor[4]. So Varvara is theoretically capable of syntax highlighting, but he refuses to implement it as far as I know.
> as for tools for software development, program launching, editing text, editing fonts, editing other graphics, and sequencing music, i'm confident i can write those myself if i have to. i've written a compiler in a subset of scheme that compiles itself, a compiler in a forth-like language that compiles itself, various toy text editors, various virtual machines in under 150 lines of code, and various pieces of music synthesis software
You may be pleasantly surprised by how many people will show up if you start building and releasing tools which:
* let them solve problems or have fun
* allow extending the tool
* can be used to build their own tools
Many will will even contribute back. To paraphrase the Gren[5] maintainer's recent livestream:
> we've had more engagement in our time on Discord than all our years on Zulip.[6]
> maybe i should set up a patreon for this or something, maybe people would be interested in supporting the project
Based on your blog, I think you can safely skip to the footnotes:
* GitHub sponsors might be worthwhile if you're willing to use non-fully FOSS platforms
* Don't make people feel like they're paying for their own work
* Do let the system do real-ish creative work and self-host, even if only as emulators
If you'd like, I can provide more specific suggestions.
i'll reply in more detail later but for the moment i just want to clarify that i'm not za3k, although i've been collaborating with him; his priorities for the zorchpad are a bit different from my priorities for the zorzpad
you said:
> I think it's the primary use case like Java Applets and Flash preceded JS as iterations of sorta-portable and low efficiency tools which make up for it with sheer volume of high-familiarity material.
i wasn't able to parse this sentence. could you unpack it a bit?
you said:
> If you'd like, I can provide more specific suggestions.
yes, please! even if our priorities aren't exactly aligned i can surely learn a lot from you
np! I'll comment now in case it gets weird about a double-comment on the same parent by the same user.
> > I think it's the primary use case like Java Applets and Flash preceded JS as iterations of sorta-portable and low efficiency tools which make up for it with sheer volume of high-familiarity material.
> i wasn't able to parse this sentence. could you unpack it a bit?
Software that doesn't care about being well-made or efficient. It doesn't matter because it's either fun or useful.
> > If you'd like, I can provide more specific suggestions.
> yes, please! even if our priorities aren't exactly aligned i can surely learn a lot from you
This is a long topic, but some of it comes down to Decker[1] vs Octo[2]'s differences:
* Decker can be used to make and share things that process data
* The data itself can be exported and shared, including as gifs
* Octo can't really, but it does predate the AI commnity's "character cards" by shoving game data into a "cartridge" gif
* Octo has LISP Curse[3] issues
I'm serious about the LISP curse thing. In addition to awful function pointers due to ISA limitations, everyone who likes Octo tends to implement their own emulator, tooling, etc and then eventually start down the path of compilers.
I haven't gone that far yet, but I did implement a prototyping-oriented terminal library[4] for it. Since Gulrak's chiplet preprocessor[5] is so good, I didn't bother with writing my own.
> i just want to clarify that i'm not za3k
Ty for the reminder. On that note, the larger font sizes you brought up are seeming more important in this moment. I don't think I can deal with 4 x 6 fonts on tiny screens like I once could. HN's defaults are already small enough.
with respect to winestock's account of the 'lisp curse', i think he's wrong about lisp's difficulties. he's almost right, but he's wrong. lisp's problems are somewhat social but mostly technical
basically in software there's an inherent tradeoff between flexibility (which winestock calls 'expressiveness'), comprehensibility, and performance. a glib and slightly wrong way of relating the first two is that flexibility is when you can make software do things its author didn't know it could do, and comprehensibility is when you can't. bugs are deficiencies of comprehensibility: when the thing you didn't know your code could do was the wrong thing. flexibility also usually costs performance because when your program doesn't know how, say, addition is going to be evaluated, or what function is being called, it has to check, and that takes time. (it also frustrates optimizers.) today c++ has gotten pretty far in the direction of flexibility without performance costs, but only at such vast costs to comprehensibility that large codebases routinely minimize their use of those facilities
comprehensibility is critical to collaboration, and i think that's the real technical reason lisp programs are so often islands
dynamic typing is one example of these tradeoffs. some wag commented that dynamic typing is what you should use when the type-correctness of your program is so difficult to prove that you can't convince a compiler, but also so trivial that you don't need a compiler's help. when you're writing the code, you need to reason about why it doesn't contain type errors. in c, you have to write down your reasoning as part of the program, and in lisp, you don't, but you still have to do it, or your program won't work. when someone comes along to modify your program, they need that reasoning in order to modify the program correctly, and often, in lisp, they have to reconstruct it from scratch
this is also a difficulty in python, but python is much less flexible in other ways than lisp, and this makes it much more comprehensible. despite its rebellion against curly braces, its pop infix syntax enables programmers inculcated into the ways of javascript, java, c#, c, or c++ to grasp large parts of it intuitively. in lisp, the meaning of (f g) depends on context: it can be five characters in a string, a call to the function f with the value g, an assignment of g to a new lexical variable f, a conditional that evaluates to g when f is true, or a list of the two symbols f and g. in python, all of these but the first are written with different syntaxes, so less mental processing is required to distinguish them
in traditional lisp, people tend to use lists a lot more than they should, because you can read and print them. so you end up with things like a list of a cons of two integers, a symbol, and a string, which is why we have functions like cdar and caddr. this kind of thing is not as common nowadays, because in common lisp we have defstruct and clos, and r7rs scheme finally adopted srfi-9 records (and r6rs had its own rather appalling record system), although redefining a record type in the repl is pretty dodgy. but it's still common, and it has the same problem as dynamic typing, only worse, because applying cadr to the wrong kind of list usually isn't even a runtime error, much like accessing a memory location as the wrong type in forth isn't
this kind of thing makes it significantly easier to get productive in an unfamiliar codebase in python or especially c than in lisp
40 years ago lisp was vastly more capable than the alternatives, along many axes. smalltalk was an exception, but smalltalk wasn't really available to most people. both as a programming language and as a development environment, lisp was like technology from the future, but you could use it in 01984. but the alternatives started getting less bad, borrowing lisp's best features one by one, and often adding improvements incompatible with other features of lisp
as 'lightweight languages' like python have proliferated and matured, and as alternative systems languages like c++ and golang have become more expressive, the user base of lisp has been progressively eroded to the hardest of hardcore flexibility enthusiasts, perhaps with the exception of those who gravitate to forth instead. and hardcore flexibility enthusiasts sometimes aren't the easiest people to collaborate with on a codebase, because sometimes that requires them to do things your way rather than having the flexibility to do it their own way. so that's how social factors get into it, from my point of view. i don't think the problem is that people are scratching their own itches, or that they have less incentive to collaborate because they don't need other people's help to get those itches thoroughly scratched; i think the social problem is who the remaining lisp programmers are
there are more technical problems (i find that when i rewrite python code in scheme or common lisp it's not just less readable but also significantly longer) but i don't think they're relevant to octo
TL;DR: You may be right about LISP issues, but my intent was show how IO and social factors seem key to platform success
I'll skip discussing Python for the moment because I think there are a lot of things wrong with it. If I try, we'll be here forever, but they mostly come down to not enough consistency and it being pointless to discuss for reasons which will become clear below.
Whatever the nature of the cause/effect of the LISP ecosystem, I see Octo's issues as similar in result:
1. There is no way to load and persist useful data in any popular CHIP-8 variant
2. Once people get a taste, the lack of IO means they do one of two things:
* leave
* become emulator or assembly developers
3. They few who stay in Octo or CHIP-8 are interested in writing their own tools, libraries, and ISA variants
Keep in mind, I still see this as success. This is based on my understanding of the Octo author's post saying farewell to running OctoJams[1] and my time as a participant. It got me interested in assembly and has helped many other people learn and begin to use low-level tools.
Compare this to Uxn:
* Although flawed, it has useful IO capabilities
* People have built applications which they use to perform real work every day
* An already-familiar ecosystem of tools exists to cater to the interests of similarly-minded people
> 40 years ago lisp was vastly more capable than the alternatives, along many axes. smalltalk was an exception, but smalltalk wasn't really available to most people. both as a programming language and as a development environment, lisp was like technology from the future, but you could use it in 01984.
Yes, I think that's getting at what I mean. One of the motivating factors for 100r was that Xcode is bad fit for people who live on a boat with solar-powered computers. So are Visual Studio or PyCharm.
Although the Uxn ecoysystem is far from optimal, it's still better than those heavy IDEs. Better yet, it was already mostly here a few years ago. Even to this day, it feels like it's from some point(s) in a better timeline where the world wasn't full of leftpad/polyfill/AI or whatever the next ridiculous crisis will be.
In other words, Uxn is a pidgin[2] of ideas "solarpunk" or even just small, understandable computer types like. At the same time, it does this without too much purism holding it back:
* Simple enough and familiar enough mish-mash of forth and low-color, plane-based display
* Low enough power that its cheaper than Xcode to run without more effort
* Enough IO capabilities to get things done
Since it already had all of this as of a few years ago, it's had time for social effects to take off. For example, you can share a picture, a music track, or text
made using Uxn-based software. Those files will still work even if the Uxn spec evolves in backward-incompatible ways again. Its design choices are also inherently good for (or limited to) producing art aligned with the aesthetics of the creators. That seems to be part of a self-reinforcing popularity loop at this point.
Although it's theoretically possible to make similar things using CHIP-8 based tools, nobody bothers. Even if you record the output of music sequencers which have been written for it, you can't really save your work or load in data from elsewhere. In theory, there's the 16 bytes of the persistent state registers in XO-CHIP which could be repurposed from a calculator-like thing into IO, but the fact I'm mentioning it should underline the main issue in that community: purity concerns. Those limit it more than having "real" 16 buttons do. Yes, you could do some interesting multiplexing or even just bit-shift every other button press to make a pretend terminal-like keyboard, but the purity focus inadvertently kills interest more than silly function pointer syntax.
Decker is the complete opposite of Octo in this. It also goes even farther than Uxn in giving up purity concerns. Sure, it's complicated and inefficient, but it is inherently built around sharing widgets with others in a mishmash of HyperCard and Web 1.0 revival spirit:
1. You can make gifs with it, and this is probably its most popular use in the form of Wigglypaint[3]
2. When asked "um, can I borrow source from it?", the answer is "Absolutely!"[4]
This is why as much as certain Python projects frustrate me, I'm not trying to fix their inherent Python-ness right now. I just accept they're what they are. As you pointed out, it's better than the alternatives in many cases. Before you ask, I won't go as far as calling JavaScript is good, but I'll admit it's shareable. :P
yeah, the chip-8 design is less ambitious than the uxn design and can only really handle very limited programs. on the other hand, it does run on much smaller hardware
i wouldn't go so far as to say uxn is flawed; that suggests it's fundamentally unsound. as far as i know, it isn't; it's a perfectly workable design, and it's better than anything else so far for frugal write-once-run-anywhere. i think a different design would be better in important ways, but vaporware is always better than actually implemented systems
> One of the motivating factors for 100r was that Xcode is bad fit for people who live on a boat with solar-powered computers. So are Visual Studio or PyCharm.
that's true, but you're damning uxn with faint praise here. i suspected that if you benchmarked left¹ (which, incidentally, does do syntax highlighting) against a default configuration of vim or gnu emacs, you'd find that left is the one that consumes the most power
but then i tried it, and also compared an extremely minimal editor called `ae`, and left seems to use a third as much power as emacs, but five times as much as vim and 300 times as much as ae
ae and left could plausibly run on the zorzpad. vim and emacs cannot; they need far too much ram. ae and left also have in common that they both lack undo, but left is dramatically more capable and easier to use
— emacs power usage —
i've been running my current emacs process for a week and (not counting cpu time used by the x server) it's used 34 minutes and 15 seconds of cpu, which is about 0.3% of one core of this amd ryzen 3500u. if we estimate that the cpu draws 29 more watts from the battery when running full-speed than when completely idle, and can run about 3 instructions per clock per core on each of 4 cores and 3.1 gigahertz working out to 36 billion instructions per second, emacs is consuming about 90 milliwatts and running about 100 million instructions per second on average
that's a lot higher than i expected, and possibly actually higher than left (i haven't tested) but it's certainly not in the same league as vscode. (this is emacs 28.2, and somewhat to my surprise, system-configuration-options tells me it's built --with-native-compilation, so perhaps it's not using the inefficient old bytecode interpreter.)
as a more precise test, to test emacs's built-in ide functionality rather than gtk and my proliferation of elisp packages, i ran `valgrind --tool=cachegrind emacs -q -nw --no-site-file` and wrote this c program in it:
#include <stdio.h>
int main(int argc, char **argv)
{
char buf[256];
printf("What's your name? ");
fflush(stdout);
fgets(buf, sizeof buf, stdin);
for (char *p = buf; *p; p++) if (*p == '\n') *p = '\0';
printf("Oh, hi, %s! Lovely to meet you!\n", buf);
return 0;
}
syntax highlighting was enabled, but -nw runs it in the terminal. i compiled it, fixed bugs in it (at first i didn't have any but i wanted to ensure i was doing a fair test), jumped to error message locations, jumped to system header files, looked up manual pages in it, ran a unix shell in it, and ran the program in the unix shell and interacted with it
this took about four minutes and 8.8 billion instructions. (just starting up emacs and shutting it down that way takes 595 million, but i wanted to get an estimate of the steady state.) this is about 30–40 million instructions per second, not counting the instructions of the compiler, shell, terminal emulator, and x-windows server; so 100 million per second with a more elaborate configuration and larger files seems like a plausible estimate
— ae power usage —
i wrote the same program again in anthony howe's 'ant's editor' from 01991⁰, which is about 300 lines of c in its non-obfuscated form, and is roughly the simplest programming editor you can actually use; the user interface is a stripped-down version of vi
where you exit insert mode with control-l, write the file with
capital w, and exit with capital q. this took about 7 minutes (i kept hitting the wrong keys) and 39 million instructions, of which about 0.8 million were startup and shutdown. that's about 90 thousand instructions per second, or 100 microwatts: a thousand times faster than emacs, and within the capabilities of a commodore 64 or apple ][. of course that again doesn't account for the window system, compiler, and terminal emulator, but it does account for ncurses computing the minimal updates necessary to the screen, so it's most of the work needed to avoid redrawing unchanged areas
38 million instructions divided by 278 bytes of output is about 137000 instructions per byte, but of course moving around in a larger file takes longer
— uxn left power usage —
running uxnemu left.rom in the same way from https://rabbits.srht.site/uxn/uxn-essentials-lin64.tar.gz takes 481 million instructions to start up and shut down. writing the same c in left.rom took about 5 minutes and 3.4 billion instructions; 3 billion instructions over 5 minutes is about 10 million instructions per second. this suggests that it uses about a third as much power as emacs and 110 times as much power as ae
10 million interpreted bytecode instructions per second is really pushing the limits of what the zorzpad can do. also its window is by default 684×374, about 33% bigger than the zorzpad's two screens put together, and it doesn't seem to be resizable (which i assume means it's not designed to be able to handle being resized)
— vim power usage —
finally, i did it again with vim in a terminal emulator, with syntax highlighting and manual page lookup, and vim took 680 million instructions, exactly one fifth of what left took. it took me less time, but i don't think vim does any background computation. as with ae, vim's time includes the time to compute minimal screen updates (confirmed with `asciinema rec vimtest`)
aha, i finally figured out that yes, indeed, my elisp code is being compiled to native code and saved as shared library files in /usr/lib/emacs/28.2/native-lisp/28.2-e4556eb6. the functions have names like F76632d63616c6c2d6261636b656e64_vc_call_backend_0@@Base (the hex string decodes to 'vc-call-backend'), and all the calls are indirected through registers, but `objdump -d` can decode it into recognizable amd64 assembly language, with function prologues and epilogues, nop-padding to 8-byte boundaries, tests followed by conditional jumps, that kind of thing. so emacs doesn't really have an excuse for being so slow
i thought that maybe the earlier version of left in c would be more efficient, so i git cloned https://git.sr.ht/~rabbits/left, checked out 4f127602e4e9c27171ef8f6c11f2bc7698c6157c, and built the last c version of the editor. a simple startup and shutdown cost 407 million instructions, and writing the same c program in it again in more or less the same way took 2½ minutes and 1.8 billion instructions. 1.4 billion instructions in 150 seconds are about 9 million instructions per second, which is surprisingly about the same as the version running in uxn. but just leaving it open for 2¼ minutes also used only 424 million instructions, so maybe it makes more sense to compare the 1.4 billion against the 8.8 billion of emacs, the 0.039 billion of ae, the 3.4 billion of uxn left, and the 0.68 billion of vim, since it seems to grow primarily with activity rather than just time open
Someone just told me to have a look at the thread and I'm very happy I did! It's been really good for me to read back you two's exchange. I'm not here to defend uxn or anything like that, I was only wondering, could you do the same test with uxn11(instead of uxnemu)? I don't personally use uxnemu, I find it's too demanding for my laptop, I would love to have the data for uxn11 in comparison to uxnemu from your system if you can run X11 programs.
oh hi! delighted to hear from you! i hope it's clear i'm not here to attack uxn either
let's see about uxn11... initial signs are good, 175711 instructions to start up and shutdown rather than hundreds of millions. but after using it to go through the whole left editing session, it's 175645 instructions; that suggests the real work is being done in a child process
yeah, strace confirms there's a child process being spawned off. i'll have to see if i can disable that to get more accurate measurements with valgrind, later today
i'm interested to hear your thoughts about how left and other uxn apps might be made usable under the constraints of the zorzpad display hardware: two 400×240 screens, each 35×58mm, with only black and white (no greyscale). left itself seems like it might be relatively easy to run usably on one of them?
Of course, I've really enjoyed your exploration of uxn, and all your comments are accurate.
Uxn11 has a special device to spawn linux processes(like playing a mp3 on disk with aplay), it can be disabled in devices/console, I wonder why it would be acting up, left doesn't do any request to the console's special ports I think, I'll have to double-check.
Left is pretty heavy, it does A LOT, it's definitely not a simple text editor, not only does it do a lot of work locating symbols and navigation strings, the front-end uses a proportional font and does a lot of positioning for it.
I don't think left would be a good candidate for zorzpad, I can imagine a simpler IDE that uses fixed width font, and doesn't try to make any sense of the tal code, syntax highlight, and doesn't support utf-8 glyphs - But Left is not it.
I've ported the classic macintosh notepad application, which I now use daily for taking notes, it has proportional font support, but is monochrome and has a small window, doesn't do anything fancy. Expanding this into a proper text editor with scrolling might be more realistic than trying to fit left to a monochrome screen.
But really, I think it'd be better to write a whole new text editor specifically for zorzpad, left has a very specific goal in mind, and I can imagine one like it designed specifically for the zorzpad. Writing text editors, or block editors, is an art that shouldn't be forgotten, anyone who's got an opportunity to write one, should do it.
I've read as much of your works as I could find, you've been a big influence on the work I do nowadays, it's an honor to read a message from you. Thank you.
aw, thanks, i'm really flattered—i admire your work a lot, and i'm glad to hear i've contributed somewhat to it. you're very welcome
as for proportional fonts, a few years back i wrote a microbenchmark for proportional font layout with word wrap to see how cheap i could get it. it's http://canonical.org/~kragen/sw/dev3/propfont.c (maybe make sure you redirect stdout to a file if you run it), and it uses the n×6 font i designed for http://canonical.org/~kragen/bible-columns, derived from janne kujala's 4×6 font. the notes in the comments say that on my old laptop it ran at 70 megabytes per second, which is probably about 60 instructions per byte. if you redrew 32 lines of text with 32 characters each (about what you could fit on the zorzpad's two screens) it would be about 60000 instructions, about a microsecond at the apollo3's theoretical 60 dmips; that might be a reasonable thing to do after each keystroke. and there are some notes in there about how you could optimize it more
running the same benchmark on my current laptop i get numbers from 64 to 68 megabytes per second, but for comparability with the editors, i should say that cachegrind measures 1,482,006,535 instructions for running 100000 iterations of rendering 126 bytes, which works out to about 118 instructions per byte and 120 000 instructions for the screen-redrawing example
(of course that computation takes about 2 microseconds, while actually updating the screen will take 50000 microseconds according to the datasheet, 16700 microseconds according to some other people's experimental results, so the computation may not be the tall pole in the tent here)
propfont.c might get a lot faster if its pixels were bits instead of bytes, especially on something like the thumb-2 apollo3, which has bitfield update instructions. then again, it would get slower with a larger pixel font size
I'm currently away from reliable network and I can't manage to load the bible-columns image, I will try once we return to Canada.
I've added proportional fonts to Left because, after writing loads of THINK Pascal, I felt like I needed it, and I still enjoy this a lot more than fixed-width, but it annoys a lot of people, even even I like to write space-padded files from time to time so Left has to support both.
If I was golfing an editor(one that I would actually have to use each day), I think I would make a block editor, fixed-width, and by its block-paginated design would do away with gap buffers altogether by needed to move very little data around.
An early version of Left tried something different where it would only display one @symbol at a time. In Uxn, "scope" or objects are defined with @label, and methods by &method, making it so it's well usable if you have a list of objects, and methods a-la Smalltalk's System Navigator. During editing, you never insert into the text file, but in a gap buffer made of only the object or method being edited. I always found that design pretty neat, the main downsize is when you want to add non-source files, then you have to contend with editing to a single large buffer.
The font encoding for Left is https://wiki.xxiivv.com/site/ufx_format.html, which allows me to calculate the sprite address glyph width sort-of quickly, the issue is that varvara has no blitter, everything is drawn as tiles so it creates a bit of overdraw for each glyph. If you have a blitter, then I think there would be pretty smart encodings for drawing proportional fonts that would make them more efficient.
yup! I'll never change it, most the projects I've made use it in some way. There's little variance in how I implement the drawing, but the specs are good enough for anything I might want to do with fonts :)
usually the issue with that image is not its byte size (it's 4 megs) but its ram usage (about 100 megapixels)
my best thought on how to handle proportional fonts is a slight variant on nick gravgaard's 'elastic tabstops'; i think we should use tabs to separate columns and \v and \f as delimiters to nest tables, so that we can do pre-css-style html nested table layout in plain ascii text files
with respect to moving very little data around, the bulk of the data that needs to be moved is pixels to display; a character is 1–3 bytes in the editor buffer, but in a 16-pixel-tall font, it averages about 128 pixels. that's 512 bytes in 32-bit rgba or 16 bytes in 1 bit per pixel. on the zorzpad, without interpretation overhead, i think i can roughly guesstimate about 100 instructions, and at a nominal 25 picojoules per instruction, that's about 2.5 nanojoules
i still haven't measured, but if i recall correctly, the ls027b7dh01 memory lcd datasheet says that maintaining the display statically costs about 50 microwatts, while inverting all the pixels once a second raises that to 175 microwatts, which is to say 125 microjoules to invert all 96000 pixels, which works out to 1.3 nanojoules per pixel. so updating those 128 or so pixels will cost about 170 nanojoules, which is a lot more than 2.5 nanojoules
i'm unclear on how much of this 125 microjoules is due to simply shifting in new lines of 400 pixels, regardless of their contents, and how much is due to the actual inversion of color of the pixels; after all, they invert polarity several times a second just to maintain the display without damaging it. if the expensive part is shifting in the pixels, then to display a single new 16-pixel-high letterform, we're updating 16 × 400 = 6400 pixels rather than 128, costing 8300 nanojoules. typing at 60 words per minute (5 letters plus a space) would then dissipate another 50 microwatts if you fully updated the display after every letter. when you're typing into previously empty space,
you could maybe update only a third of the pixel rows after each letter to cut down on this, completing the delayed updates after a few hundred milliseconds if you stop typing
it might turn out that the most important thing to do to minimize power usage is to minimize the amount of the screen that gets regularly updated, and last night i was thinking about how in bsd talk and on mit its, text would wrap around from the bottom of the window back to the top rather than scrolling. emacs does a less aggressive version of this where it does scroll, but it scrolls by half a screenful at once when the cursor goes off the screen, so that updates to the whole screen only happen every ten or twenty lines of movement or typing, rather than after every line. this was important on terminals like the adm3a that didn't have escape sequences to insert and delete lines
narrowing the view to a single symbol might avoid the need to spend energy drawing lines of text from other symbols, perhaps repeatedly as line insertion or deletion moves them around on the screen
the adm3a also didn't have escape sequences to insert or delete characters, and vi (strongly influenced by the adm3a) does a peculiar thing with its 'c' change command; if you say, for example, ct. to change all the text before the following period, it doesn't erase that text immediately, but puts a $ at the end of it so that you know what you're changing. this avoids the need to shift the rest of the line right after each character. (vim does not do this.) if the big energy suck is actually changing pixels, as i think it is on e-ink, rather than shifting them through a shift register, that might be a useful strategy to adopt: open up space to type into, with some kind of indicator that it doesn't contain spaces or any other characters. like a visible manifestation of a buffer gap, but measured in pixels rather than bytes
i don't think there's a reasonable way to hack a hardware blitter into the zorzpad, but doing it in software is very reasonable
with respect to software that doesn't need to be efficient, sure, there are lots of things that are usable without being efficient. but in rek and devine's notes on working offgrid, which are part of the background motivation for uxn/varvara, they say:
> Computers are generally power-sucking vampires. Choosing different
software, operating systems, or working from machines with a lower
draw (ARM) or even throttling the CPU, are some of the many things
we do to lower our power requirements. The way that software is
built has a substantial impact on the power consumption of a system,
it is shocking how cpu-intensive modern programs can be.
so uxn/varvara is not intended for software that doesn't need to be efficient, very much the contrary.
so, from my point of view, the fact that logic-intensive programs running in uxn consume 20× more energy than they need to is basically a mistake; rek and devine made choices in its design that keep it from meeting their own goals. which isn't to say it's valueless, just that it's possible to do 20× better
what i mean by 'logic-intensive' is programs that spend most of their time running uxn code doing some kind of user-defined computation, like compilation, npc pathfinding, or numerical integration. if your program spends most of its cpu blitting sprites onto the screen, well, that's taken care of by the varvara code, and there's nothing in the varvara definition that requires that to be inefficient. but uxn itself is very hard to implement efficiently, which creates pressure to push more complexity into varvara, and varvara has ended up fairly complex and therefore more difficult than necessary to implement at all. and i think that's another failure of uxn/varvara to fulfill its own ideals. or anyway it does much worse according to its own ideals than a better design would
how much complexity does varvara impose on you? in https://news.ycombinator.com/item?id=32219386 i said the uxn/varvara implementation for the nintendo ds is 5200 lines of c, so roughly speaking it's about 20× more complexity than uxn itself
and that's what i mean by 'and it's not that simple'. in the comment i linked above, i pointed out that chifir (the first archival virtual machine good enough to criticize, which is a somewhat different purpose) took me 75 lines of c to implement, and adding graphical output to it with yeso required another 30 lines of code. you might reasonably wonder how much complexity i'm sweeping under the carpet of yeso, since surely some of those 5200 lines of code in the uxn/varvara implementation for the nintendo ds are imposed by the ds platform and not by varvara. the parts of yeso used by my chifir implementation compiled for x-windows are yeso.h, yeso-xlib.c, and yeso-pic.c, which total 518 lines of code according to either sloccount or cloc.
still, uxn, even with varvara, is much better than anything else out there; like i said, it's the first effort in this direction that's good enough to criticize
i don't understand what you mean about octo vs. decker but possibly that's because i haven't tried to use either one
you definitely can't deal with a 4×6 font on the ls027b7dh01 display i'm using. it's 35mm tall, including the pixel-less borders, and 240 pixels tall. so 6 pixels is 0.88 mm, or a 2.5 point font. even young people and nearsighted people typically need a magnifier to read a 2.5 point font.
> I have a radical idea which might not be a good one:
> 1. A "Full" display of W x H characters (the Sharp display or whatever you replace it with)
> 2. A "Fallback" 1 or 2 line display (2nd line reserved for editor info / UI?)
i've thought about this, especially with respect to e-ink screens. some versions of the barnes & noble nook work this way: there's a small color lcd touchscreen for interactive stuff, and a large black-and-white e-paper display for bulk information display. i thought that a small reflective lcd like cheap scientific calculators use would be enough for a line or two of text, permitting rapid interaction without putting excessive demands on the e-paper's slow and power-hungry refresh
but i think that sharp's memory lcds are so much better than e-paper that there's no real benefit to that approach now
> A single 1-bit editable line is so hard to work with that ed[2] isn't even installed by default on many Linux distros. However, people did once got real work done with it and similar tools.
it's not that command lines are hard to work with; it's that ed's user interface is designed for 110-baud teletypes, where erasing is impossible, and typing out a 10-character error message means forcing the user to wait for an entire second, so it's preferable to just say '?\r\n'. user interfaces designed for more capable hardware, such as an adm-3a connected over a 1200-baud modem, can be immensely easier to use, without abandoning the command-line paradigm. you'll note that ex(1) is installed by default on every linux distribution except alpine (which does, oddly enough, have an ed(1)), and i know a programmer who still uses it as his preferred editor, preferring it even to vi
i've certainly done plenty of real work with tcsh and bash, where the primary user interface paradigm is editing a single line of text to your satisfaction, submitting it for execution, meditating on the results, and then repeating. not to mention the python repl, gdb's command line, octave, units(1), r, pari/gp, sqlite, psql, swi-prolog, guile, ngspice, etc. i like direct-manipulation interfaces a lot, but they're harder to design, and especially hard to design for complex problem-solving activities. it's just that correcting a typo a few words ago isn't really a complex problem-solving activity, so, by my lights, even a relatively dumb
direct-manipulation text editor can beat a command-line editor like ex most of the time.
especially if their bandwidth to your eyes
is measured in megabits per second rather than kilobits per second
> you'll note that ex(1) is installed by default on every linux distribution except alpine
On my system, it seems to be set up as an alias of vim ("Entering Ex mode"). Huh. A bit more reading after that has been an interesting historical detour.
> i know a programmer who still uses it as his preferred editor, preferring it even to vi
I've never tried ed before this comment. Others might see it as strange and clearly teletype-oriented, but I think it's refreshingly explicit about every action taken. Even more than ex, it's a vim ancestor where commands and state are persistently visible instead of being hidden.
Thank you for mentioning this, but now I'm a little worried I might start wanting to use ed too.
> i've certainly done plenty of real work with tcsh and bash, where the primary user interface paradigm is editing a single line of text to your satisfaction, submitting it for execution, meditating on the results, and then repeating.
I think you've hit on something important here. Web-oriented workflows with live reload (and Flutter[1]) inherit this. I think it means the form of the REPL itself isn't what we should be focusing on, but easy iteration and maybe cached results to save power and time.
On the topic of accessibility, do you have any opinions on Hedy[2] and whether a long-lived, low-power system might have room for something like it? The site is annoyingly JS-filled, but the idea is interesting: a language where the names in the symbol table are more overtly a user-localizable value.
(trying not to respond at too great length to avoid being overwhelming)
vi has been a mode of ex since the inception of vi; early versions of vim (up to 3.0.0 i think) didn't implement ex mode, but later it was added. the : commands of vi are exactly the ex commands (: is the ex prompt; one of the improvements over ed enabled by higher-bandwidth terminals was explicit feedback to distinguish command mode from insert mode)
i don't think of ed as very strange. teco, now, teco is strange. ed is just a more taciturn vi. but i would never describe ed as making state persistently visible!
if you're finding yourself tempted by ed you should probably try sam instead
i don't understand the nature of hedy but it sounds like just another scripting language; what would make it especially difficult or costly to implement? i don't understand what you mean by 'user-localizable' or 'more overtly'
I might quote you on this if I have to explain it to others.
> if you're finding yourself tempted by ed you should probably try sam instead
Ty, I may try it then.
> i don't understand the nature of hedy but it sounds like just another scripting language; what would make it especially difficult or costly to implement? i don't understand what you mean by 'user-localizable' or 'more overtly'
The language drop-down on the editor / try-it page translates not just page text, but the scripting language keywords itself. I haven't had time to dig into it yet, but it's inherently an internationally oriented scripting language by design since the goal is to allow adding your own language to the supported set of keyword display languages.
I do not think that it should be necessary to make the keywords in the programming language to be languages other than American English, and I also do not think that it should be necessary for musical notation to be in languages other than Italian.
(I do not speak Italian, but that does not mean that I cannot play piano or that I cannot write music. I speak Canadian rather than American, but that does not mean that I cannot program a computer.)
But if people want to use languages other than English for computers and Italian for music, they can do that if they want to (and some people do); this is not an intention to prevent people to do such a thing if they want to do, just to say that I think it is unnecessary, and why I think it is probably better not to do.
However, documentation, comments, etc, can be in any languages you want to do, and documentation should be available in many languages in order to make it understandable by many people.
Using different languages in the program itself (or in music, etc) can make it more difficult to understand and to find information; but making the documentation in many languages makes it easier; you can read documentation in the language you undertsand and can then understand a program even if it is English, that you might not have written by yourself. So, I think this way is more useful.
i see. visual basic did that too; while i'm sympathetic to complaints of cultural imperialism, i'm not sure that having your keywords in a foreign language such as english is actually worse than being cut off from the international community of other programmers in your language so that you can't find answers with a full-text search engine. but there's an enormous space of possible programming language designs that haven't really been tried
TL;DR: My concerns are more directly practical ones about approaches (formal and humor-based) to overcoming language barriers between devs across language families
> visual basic did that too
I haven't seen this on recent versions. An old MS support thread[1] suggests VB's editor has been English-only for the past decade-ish. Do you remember whether it's a built-in feature or an add-on? For example, did it get handled by replacing text, or storing ASTs of built-ins and remapping the display name? I'm not seeing much in search results.
I also haven't had time for an in-depth look at how Hedy handles their translation, but they do use Weblate to manage everything[2]. Their keywords page even has percentages for partial languages[3]. For Microsoft or whoever added a simliar feature to VB, I'd be surprised if they didn't have a way to do your own customization for an organization.
> while i'm sympathetic to complaints of cultural imperialism
My concerns are much more practical. Collaboration across language barriers:
* is fairly easy when the devs are all familiar with the same writing systems
* can become mutually frustrating across writing system families, especially between Latin-derived and logographic[4] CJK[5] systems
My best example involves attempting to discuss the same differences across the same writing system gap. I had to do some careful reading to make any sense of things. However, it led to some immediately useful conclusions about glyph cache and texture atlas / sprite sheet issues for everyone involved.
Fun fact: limited space in rewritable text editing tools may go back at least as far as the Romans.
However, de novo writing writing systems seem to have very few enduring examples. As far as I know, Korean Hangul[6] seems to be the oldest and it still experienced significant hurdles for hundreds of years.
Part of it seems to be that symbol systems tend to be domain-specific if they even have grammar, let alone executable properties. For example:
* Extensions to flowchart concepts such as Drakon[7]
* Toys such as Scratch[8], LEGO's old MindStorms environment, etc
* Non-executable domain-specific markup (7 Wonders[9] and other card / board games)
This goes all the way back to the Sumerians. Although we no longer use symbols primarily as primarily a means of agricultural contract record keeping, we did some of their timekeeping conventions. Our mathematical notation is better as I understand it, but still awful and inconsistent from an outsider's perspective.
That last point got me thinking. Diagramming and working demos both help with graphics, games, and geometry questions. The first two also seem to be specific cases of the third, which is a human interest predating digital or mechanical computers.
However, low-power or emulator developer communication seems to tend toward the following instead of clay cylinders or tablets:
* Non-freeform monospace text and text boxes (although the Left editor has both nice font support and syntax editing now, thank you for updating me on that)
* Audio calls or screen sharing text boxes or GUIs (more text and symbols)
Aside from inventing conlangs[10], we don't really invent new human languages, nor do we apply them at project scale. Instead, we seem to tend to come up with neologisms and project-specific slang.
Is this a feature or a bug? I see two extremes with regard to bridging grammatical structure and writing system gaps:
1. Reject: attempt to formalize how Esperanto did (or the way you said VB did)
2. Embrace: encourage and combine it with the following:
* The way Forths tend toward being a family of dialects, each specific to an ISA or machine
* Naturally emerging labels and slang
The second direction seems easier.
To me, Python seems to have "regional" dialects specific to industries or codebase purposes. It's not a standard, but an emergent thing the same way I've heard Newgrounds[11] projects had a reputation for offensive variable names.
Python had humor once too. Unlike Newgrounds Python's humor was Monty Python references in example code instead of choosing variable names based on shock value. Also, Python seems to have lost the humor as it "grew up"
It seems like this sense of humor is important to early-phase software and hardware ecosystems. For example, the Uxn / Varvara ecosystem we've been discussing has plenty of it. Potato[12] isn't quite a joke with a punchline, but it is an irreverent reference to low-power computers with "bad" graphics.
Community coherence is something Octo had issues with, but both Uxn and Decker seem to be doing well. My guess is humor as a key factor.
Do you have any thoughts on this and other parts of community building or emergence?
* In your specific area(s) of interest? (low-power, long-term hardware)?
* For retro languages+hardware?
* In general?
I also realize I might be bringing way too much process and theory into this. Do you still use the email listen on your about page?
It'd be nice to have a fallback in case this thread locks or something. It's been rare to have such long-lasting discussions, so I don't remember how thread locking works on this site.
so, the translated visual basic thing is a thing i remember hearing about vba in the 90s, without any personal experience. but if memory serves (and i'm not just confabulating the whole thing) it was built-in, not an add-on
i think it's reasonable to describe graphical notations such as drakon as having a grammar. you have some finite set of atomic features (often things like shapes or arrows, but sometimes other characteristics such as size, orientation, and color) which are combined in some finite set of ways to produce an infinite set of possible 'sentences'. the only non-grammatical aspect of this is that some of these features are used in an analog way, for example with a length proportional to a time lapse or an orientation that indicates a cardinal direction, while verbal features are strictly digital and arbitrary—but we often use similar paralinguistic features in speech and especially in sign language
def edits1(word):
"All edits that are one edit away from `word`."
letters = 'abcdefghijklmnopqrstuvwxyz'
splits = [(word[:i], word[i:]) for i in range(len(word) + 1)]
deletes = [L + R[1:] for L, R in splits if R]
transposes = [L + R[1] + R[0] + R[2:] for L, R in splits if len(R)>1]
replaces = [L + c + R[1:] for L, R in splits if R for c in letters]
inserts = [L + c + R for L, R in splits for c in letters]
return set(deletes + transposes + replaces + inserts)
in seven lines of code, it computes all the potentially incorrect words that can be made from a given correct word with a single edit. so, for example, for 'antidisestableshmentarianism', it returns a set of 1482 words such as 'antidisestauleshmentarianism', 'antidisestableshmentarianlism', 'antidiseitableshmentarianism', 'antidisestablesjhmentarianism', and 'antidiseptableshmentarianism', totaling 42194 bytes. how would you do this in uxntal?
here's another part of norvig's program. this part tabulates the case-smashed frequency of every word in its 6-megabyte training set (which presumably consists only of correctly spelled words):
import re
from collections import Counter
def words(text): return re.findall(r'\w+', text.lower())
WORDS = Counter(words(open('big.txt').read()))
this takes about 340 milliseconds one core of on my laptop here, which runs at about 6000 MIPS, so it would take about 34 seconds on a machine running at 60 MIPS, maybe a little longer on the apollo3. there are 32198 distinct words in the training set, totaling 244015 characters; the most common word ('the') occurs 79809 times, and the longest word ('disproportionately') is 18 characters. so plausibly you could represent this hash table without any compression in about 500k, though cpython requires about 70 megabytes. ram compression could plausibly get those 500k down to the 384k the apollo3 has without trying to swap to offchip flash
finding the best correction for a word requiring two corrections like 'slowlyyy' takes 70ms, so plausibly it would take 10 seconds or so on the apollo3. (you could maybe do this in the background in a text editor.) (if it were compiled to efficient code, it would probably be closer to 300 milliseconds on the apollo3, because cpython's interpretive overhead is a factor of about 40.) 'disproportionatelyyy' takes 370ms. here's the rest of the correction code:
def P(word, N=sum(WORDS.values())):
"Probability of `word`."
return WORDS[word] / N
def correction(word):
"Most probable spelling correction for word."
return max(candidates(word), key=P)
def candidates(word):
"Generate possible spelling corrections for word."
return (known([word]) or known(edits1(word)) or known(edits2(word)) or [word])
def known(words):
"The subset of `words` that appear in the dictionary of WORDS."
return set(w for w in words if w in WORDS)
def edits2(word):
"All edits that are two edits away from `word`."
return (e2 for e1 in edits1(word) for e2 in edits1(e1))
note that this requires you to have two such `edits1` sets in memory at once, though you could plausibly avoid that problem by tolerating more duplicates (double letters provoke duplicates in deletes, transposes, and replaces)
norvig doesn't tell us exactly how long it took him to write the code, but he did it in a single airplane flight, except for some minor bugs which took years to find. more importantly, though, it's very easy code to read, so you can easily understand how it works in order to modify it. and that's the most important factor for programming productivity
here are some things in this code that are more difficult to write and much more difficult to read in uxntal:
- managing more than 64k of data (uxn's memory addresses are 16 bits)
- dynamically allocating lists of things such as the (left, right) tuples in splits
- dynamic memory allocation in general
- string concatenation
- eliminating duplicates from a set of strings
- iterating over the words in a text file
- generating a sequence of outputs from a sequence of inputs with a filtering predicate and a transformation function [f(x, y) for x, y in xys if p(x, y)]
- generating a lazy flat sequence of outputs from a nested loop (return (z for y in f(x) for z in f(y)))
- hash tables
- incrementally eliminating duplicates from a sequence of candidates that turn out to be valid words (set(w for w in words if w in WORDS))
- counting the number of occurrences of each string in a lazy sequence of strings
- floating-point arithmetic (which would be fairly easy to eliminate in this case, but not in many other cases; this deficiency in uxn is especially galling since the apollo3 has fast hardware floating point)
- finding the highest-rated item of a lazy sequence of candidates according to some scoring function
and all of that is on top of the general readability tax imposed by postfix syntax, where even figuring out which arguments are being passed to which subroutine is a mental challenge and a frequent source of bugs
note that these are mostly not deficiencies you can really patch with a library. i didn't mention that the program uses regular expressions, for example, because you can certainly implement regular expressions in uxntal. they're things you probably need to address at the level of language semantics, or virtual machine semantics in the case of the address space problem. and they're not tightly tied to cpython being implemented grossly inefficiently; pypy implements all the necessary semantics, and common lisp and c++ have similar facilities in most cases, though their handling of lazy sequences is a weakness that is particularly important on hardware with limited memory like the apollo3
so that's what i mean when i say that uxn is designed to make easy things hard, rather than making hard things easy
you say:
> the pain point might be intentional nudges away from making things the designer doesn't like
the thing is, i don't really care whether rek and devine think that autocorrecting misspellings is a bad thing to do; i want the computer to be a means of expression for my ideas, indeed for everyone's ideas, not for the ideas of a singular designer. that's the apple walled-garden mindset, and it's anathema to me. and, though i could be wrong about this, i think rek and devine would probably agree
this doesn't look bad at all! considerably better than common lisp, in particular. but i think the flatter structure of the python improves readability, and the independence of the different clauses facilitates interactive incremental testing:
>>> word = 'the'
>>> letters = 'abcdefghijklmnopqrstuvwxyz'
>>> splits = [(word[:i], word[i:]) for i in range(len(word) + 1)]
>>> splits
[('', 'the'), ('t', 'he'), ('th', 'e'), ('the', '')]
>>> deletes = [L + R[1:] for L, R in splits if R]
>>> deletes
['he', 'te', 'th']
but lisps are generally pretty good at that kind of thing, so i imagine you could formulate it slightly differently in txr lisp to support that kind of thing (i just don't know txr lisp)
as a semantic question, is this materializing the whole list (as the python does) or are the `add` calls inserting into the hash table as the loops run, thus eliminating duplicates?
I had a bug somewhere, so I selectively off some of the add expressions. They can be commented out with #; or by flipping add to list or identity to throw away the value.
The add is something which pairs with build. Lisp doesn't have "bag-like" lists. For those times when they are helpful, we can have procedural list building syntax. The build macro creates an environment in which a number of operators like add that build up or otherwise operate on an implicit list. When the build form terminates, it returns the list. (Its sister buildn returns the last form like progn).
In this function, I could just have used (push expr stack) because we don't care about the order; there would be no nreverse. That would be a better idea, actually.
We could also add the strings to a table directly, like (set [h (join ...)] t).
The hash table is built by the hash-list call. It associates the elements of the list with themselves, so if "a" occurs in the list, the key "a" is associated with value "a".
thanks! it sounds like a pretty effective system, although the form of incremental development you're describing is editing and rebuilding the program, more like c than the kind of repl flow i was talking about
the use of [] for indexing rather than clojure's list building (or as conventional superparentheses) is appealing
In TXR, we can easily recall the entire function definition at the REPL, and resubmit it, without requiring any external IDE.
Furthermore, if you have the kind of workflow where you have individual REPL commands produce results that are used by subsequent commands, the TXR Lisp listener has good support for that. When you recall an input line from history, you can use Ctrl-X Enter to execute it rather than just Enter. When you use Ctrl-X Enter, it will keep the history position and move to the next line in history rather than return to the current context. So using Ctrl-X Enter multiple times, you can resubmit a sequence of historic lines in order.
you might want to switch to the standard gnu readline keybinding for 'submit the current input line for execution and recall the next line in history', which is control-o. aside from the synergistic effect of being able to use the same keybinding in txr, bash, python, etc., it's a command which you frequently want to use several times in a row, and binding such a command to a sequence of two keystrokes makes it disproportionately more clumsy. you may have noticed recent versions of emacs permit you to run a keyboard macro repeatedly with c-x e e e, and it's a huge usability improvement
I see in the readline sources that it was added in 2021. (The git history of Chet Ramey's repositories is horrible; he just adds big patch bombs with no individual commits.)
I independently conceived the feature in 2015. I had not seen it anywhere before that, nor have I seen it since!
commit 51322f6c66d153d2f008a227c5e517f6fb34dbab
Author: Kaz Kylheku <kaz@kylheku.com>
Date: Thu Dec 31 05:30:46 2015 -0800
linenoise: submit and stay in history.
I see the readline source code gives credit to copying this feature from the Korn shell. In misc.c there is a comment:
/* The equivalent of the Korn shell C-o operate-and-get-next-history-line
editing command. */
(Where would Bash imagination be without Korn.)
I see in the ksh93 sources that this appeared in or before 2012. (Again, patch bomb wall.)
What buildn will do with the list is simply lose it. The last form can extract it and do do something with it.
When you might use it is when the goal isn't to build a list which escapes. The construct supports queuing semantics (insert at one end, take from the other), so you can use buildn to express a breadth-first traversal that doesn't return anything, or returns something other than the list:
(defun bf-map (tree visit-fn)
(buildn
(add tree)
(whilet ((item (del))) ;; (del) from front
(if (atom item)
[visit-fn item]
(each ((el item))
(add el)))))) ;; (add) to back
so, i don't want to be unkind, but i think this is a misconception, or a pair of misconceptions, and i think i know where one of them comes from. i'll try to steelman it before explaining why it's wrong
the first assertion is that counting milliwatts (or actually microwatts in this case) implies that text beats guis. the second assertion is that counting decades implies that text beats guis. i have no idea where the second assertion comes from, so i'll focus on the first
a first plausible justification for the first assertion is that power usage (over leakage) is proportional to the number of toggling bits (or perhaps cubic or quadratic in the number of toggling bits if voltage scaling is an option) and a character generator toggles many fewer bits than all the computation needed to draw a gui, so applications written for a character-cell text terminal can use much less power than gui applications. a second plausible justification is that we have many examples of applications written for character-cell text terminals which use much less computation (and therefore power) than any existing gui alternative
the first justification has two problems: first, it's comparing solutions to the wrong problem, and second, character-cell text terminals aren't even the best solution to that problem; they're the best solution to a significantly different problem which we don't have
the problem that the first justification is trying to solve is the problem of generating a video signal that is useful according to the goals of the application. in that context, the character generator is running all the time as the beam refreshes the phosphors on the crt over and over again. it is true that the character generator can compute the pixels to send to the crt much faster and with less transistors than a cpu can. but in the zorzpad, the display itself is bistable, like the plato gas plasma terminals from the 01960s. there's a flip-flop printed on the glass in amorphous silicon next to each pixel. so we don't have to consume any computational power refreshing a static display (though we do have to reverse the polarity to it a few times a second to prevent gradual damage to the lcd). we only have to send it updated lines of 400 bits. we don't have to generate a continuous video signal
moreover, the fact that a character generator would toggle less bits than the cpu to generate a line of pixels is somewhat irrelevant, because ambiq doesn't sell a character generator chip, and a character generator implemented in conventional cmos from another vendor (or on an fpga) would use orders of magnitude more power than the subthreshold-logic ambiq microcontroller, so we'd have to do the character generation process on the cpu anyway. at that point, and especially considering that the memory-in-pixel lcd display hardware doesn't impose any timing constraints on us the way an ntsc or vga signal does, there's no extra cost to generating those lines in a more flexible way. even without a framebuffer, generating lines of pixels on demand, we could support glyphs positioned with pixel precision rather than character-cell precision, proportional fonts, multiple sizes, polygons, rectangles, stipple patterns, overlaid sprites, and so on
also, though, generating a line of pixels or a vga video signal from a framebuffer requires less bit toggling than generating it from a character generator. with the framebuffer you literally just read sequential words from ram and send them to the display with no processing. what character generators save isn't power; it's memory. with a character generator you can drive a 1280×1024 display with an 8×16 font (256 characters, say, so 4096 bytes of font memory) displaying 64 lines of 160 columns, requiring 10240 bytes of glyph indices, for a total of 14336 bytes of memory. for the traditional 80×25 you only need 2k of modifiable memory and a font rom. and you can do it with strictly sequential memory access (many early character-cell text terminals used delay-line memory or shift-register memory) and meeting the strict hard-real-time guarantees required by the crt hardware being driven
in this case, 400×240 1-bit pixels would require 12000 bytes of framebuffer per screen, and 24k is about 6% of the apollo3's total 384k of sram. in theory the apollo3 needs about 2.1 cycles per 32-bit word to copy data into the framebuffer without rotating it, a number i haven't tested, so 12600 cycles to redraw both screens from scratch; at 48 megahertz that's about 260 microseconds, so it doesn't use much of the chip's available horsepower even at submilliwatt power levels. the display datasheet only guarantees being able to draw two megapixels per second, so redrawing the screen should take 48000 microseconds, although others have reported that it works reliably at three times that speed. hopefully i can use the cpu's built-in spi peripheral to spew out buffered spi data to the screen while the cpu goes back into deep-sleep mode. if i use a character generator implemented in software to avoid having a framebuffer, i have to wake up the cpu periodically to generate new lines of data, using more power, not less. (which might be worth it for the memory savings, we'll see.)
even without enough memory for a framebuffer, though, character generators weren't the only way to generate hard-real-time video signals. the nintendo (nes) only had 2k+256 bytes of vram, for example, and you could do pretty elaborate graphics with that. it used tiles (which are sort of similar to font glyphs and can in fact be used for font glyphs) and sprites (which aren't), plus scrolling registers. some other hardware supported display lists of straight lines instead of or in addition to glyph indices
the second justification is basically a cohort confounder. yes, vs code uses a lot more cpu than vim. but vim was originally written for the amiga, and vs code is written in electron. applications that run in character-cell text terminals do empirically tend to be a lot faster than gui applications, but that's largely because they're older, and so they had to be written with more attention to efficiency. remember that geos provided a standard wimp interface on the commodore 64, with mouse pointers, sliders, pulldown menus, proportional fonts, overlapping windows, and so on, and all that in color without a color framebuffer, adding extra work. it was slow, especially when it had to load data from the floppy drive, but it was usable. the commodore 64 is about 0.02 dhrystone mips (https://netlib.org/performance/html/dhrystone.data.col0.html) and the apollo3 is about 60, so running something like geos on it at 10× the commodore speed ought to use about .3% of its full-speed cpu. or 3% if it's running in a simple bytecode interpreter
so that's why i think submilliwatt computing doesn't require abjuring the gui
does that make sense? have i just totally misunderstood you?
> Do you favor a specific structure of hardware, OS code, and user scripts, or whatever ends up offering the best use of power and long-term device durability?
well, i think it's critical for the zorzpad to be able to rebuild its own operating system code, because otherwise it's dependent on external computers to modify its own operating system, and the external computers might have become incompatible or untrustworthy at some point. this poses tricky problems of recovery from bad operating-system updates; probably having multiple redundant microcontrollers in the device is the best option, so that on bricking one of them, i can restore its old configuration using the other
ideally, though, the core code capable of bricking it would be a small tcb, not everything that runs on the system the way it is in ms-dos or templeos. that kind of isolation is usually implemented with an mmu, but the apollo3 doesn't have an mmu, just an mpu. so currently my plan is to implement it with a low-level virtual machine instead; initially an interpreter (my prototype seems to have about 10× interpretive overhead, but probably providing a small number of extra primitives will be adequate to eliminate the worst hotspots from things like updating the display and digital signal processing) but maybe later a simple jit compiler. like most microcontrollers, the apollo3's memory/throughput quotient comes to about a millisecond, rather than the second or so that has been typical for human-interactive computation for the last 60 years, so i suspect i'll be leaning pretty hard to the side of sacrificing speed to reduce memory usage
in addition to providing the protection needed to write experimental code without risk of bricking the machine, the virtual machine can also provide services like virtual memory (including compressed ram), transactional concurrency, and transparent persistence
the nand flash chips i'm using (s34ms01g2) are supposed to consume 27 milliwatts when running, but their power budget at the nominal 1-milliwatt full power is only 300 microwatts. at full speed they're capable of 133 megabytes per second, so at 300 microwatts they might be capable of 1.5 megabytes per second. i can't write to them nearly that fast without wearing them out early, but i can load data from them that fast. in particular that means that the 24 kilobytes of a full screen can be loaded from flash every 16 milliseconds, so you can do 60fps video from flash. reloading all 384k of ram, for example to switch to a completely different workspace, would take a quarter of a second. i feel like this is enough of a quantitative difference from the speed of floppy disks on a commodore 64 or a macintosh 128 that it becomes a qualitative difference, and for applications like the spell corrector example i gave in the other comment, it might be possible to use the flash almost as if it were ram, using the ram as a cache of what's in the flash
but maybe when i gain more experience with the hardware, all that stuff will turn out to have been the wrong approach, and i'll have to go with something else
1. For early iterations / similar projects, could all-metal screw terminals[1] accept power internally?
2. Would supporting UXN/Varvara[2] be an option?
More on these questions below after initial comments.
## Initial Comments
> if you aren't worried about going batteryless, or about mass production, you could literally buy an alphasmart neo and wire its keyboard and display up to an esp32 or something
I could, but it's a limited in availability. The keyboard is pretty important in my opinion.
> memory-in-pixel lcd displays
I vaguely remember hearing about this, but don't have the EE knowledge to judge the benefits of these. Data sheet for anyone interested: https://www.sharpsde.com/fileadmin/products/Displays/Specs/L...
## 1. The screw terminal:
Disclaimer: I'm not an EE specialist.
My understanding is that if the power loss isn't too great, an all-metal internal screw terminal[1] might improve device durability:
* The power source could be replaceable. AA, AAA, LiPo, solar, etc
* If you don't solder to the metal part, even sandpaper or a conveniently shaped rock could remove oxidation
* For a case, internal screw terminals could turn a charging port into an easily replaceable component
## 2. UXN/Varvara:
From your blog, I see an "artemis apollo 3 board"[3] is being used. From a Sparkfun page[4], it seems to have enough ram to host graphical Varvara.
I was initially doubtful of UXN despite loving the idea, but:
1. The UXN community seems to have built a self-hosting ecosystem[5] of:
2. The UXN VM is light: 64k system ram, 4-bit 2-plane color, and misc state and debug3. The core UXN VM is simple: a minimal implementation fits in under 150 lines[6] of C89
[1]: https://en.wikipedia.org/wiki/Screw_terminal
[2]: https://wiki.xxiivv.com/site/varvara.html
[3]: https://blog.za3k.com/zorchpad-update-cardboard-mockup-mk1/
[4]: https://www.sparkfun.com/artemis
[5]: https://github.com/hundredrabbits/awesome-uxn
[6]: https://wiki.xxiivv.com/etc/uxnmin.c.txt