Well, how much say do we have over the architecture? Paging/banking hardware is pretty common on 6502-based systems. Such hardware decouples the virtual address space (what the CPU sees) from the physical address space. Typically they are used to extend memory space (e.g. the Apple //e's 80-column card) but they can also be used as a mass indirection mechanism (e.g. the NES's various "mappers", which granted are used more by its graphics coprocessor). The act of paging is "instantaneous" since you are just setting a hardware register.
So if you are designing the hardware, give it an extra 4 KiB or so (16 pages), mapped through a paging circuit to the zero page. On entry to a function, increment the current page; on exit, decrement. Now you have a way to push all "local variables" that is faster than even pushing all the registers. You could even get fancy and XOR bits 7-4 of the page register with bits 4-7 of the paged address lines, so the compiler can choose anywhere from 16 256-byte frames up to 256 16-byte frames.
There's also the Apple ///'s approach to almost-transparent banking on a 6502. The hardware could redirect the zero page to (almost) anywhere in the 64K address space (foreshadowing the 65C816's movable zero page), and it maintained a sort of second zero page called the "X" page at a fixed offset from the zero page. Then on any indirect access to memory through the zero page, it would check the corresponding byte in the "X" page and if it was valid it would use it as additional address bits. As implemented in the ///, programs could thus directly access up to 512KB without doing any traditional banking or paging.
Of course, it was also possible to do traditional banking on the /// so that program code could reside anywhere in the 512K, but I thought the X page thing was pretty neat when I finally figured it out.
Great tidbits! I'm reminded also of the 6809's movable "zero" page (the "direct page"), though it has a 16-bit stack register so the trick isn't really necessary.
Coincidentally, I was just reading this the other day. I've been programming for the Nintendo NES. The most common high level language that people use for NES Homebrew development is C with cc65[1]. The output of the compiler is passable, but C is just not well suited to the 6502 processor. Eventually I'd like to write a compiler for a high level language that used these ideas.
I've wanted to implement a better C compiler myself, and I think it would work better if it were built from the ground up instead of from Small-C, as cc65 was.
I would split the parameter and computation stacks, instead of using one as cc65 does. The parameter stack would work the same as cc65's, but the computation stack would be a FORTH-like stack indexed with the zp,X addressing mode.
Second, I would have it build an AST instead of having the parser generate the code. This would open doors to big optimizations, such as automatically detecting when to use 8-bit ops (complicated by C's promotion rules), etc.
Finally, I would implement far pointers for memory-banked systems--most prominently the NES, which cc65 doesn't handle banking on as far as I know.
Another think that could be done is using the zero page as a bank of 16/32/etc. registers and treat the 6502 like a RISC.
cc65 is an amazing project that's enabled many developers. It's Small-C pedigree is limiting though, and inefficient on a machine that it wasn't designed for.
We use cc65 in the Oric-1/Atmos scene (look it up, great little computer..) and it generates fairly decent code. Most of the Oric hackers nevertheless use C as scaffolding for plain old assembly routines.
Action! rocked! So fast to compile and run. Too bad it came along too late in the Atari lifecycle to matter. It's big trick was to disallow recursion, which meant local variables and function args/results could be allocated statically, which worked around the 6502's limitations.
It was an orange cartridge and a small yellow ringed binder. It was one of e few if not the only non Atari sourced cartridges for the Atari cartridge I ever saw or owned.
"Use fixed zero page locations for
locals and parameter passing, simulating a
conventional CPU with a large register file. This
eliminates most stack operations, except for
saving/restoring the zero page"
The 65c816 allows the "zero page" (called the direct page there) to be relocated anywhere in the lower 64k. So if one targets that platform the saving/restoring operation could simply be one of pushing/popping the direct page location to/from the stack when entering or exiting function stack frames. Then just keep all program code above the 64k boundary.
I implemented a forth based language on C64 in 1983, it was tiny and fast. The 6502 was well suited to this. The REPL nature of forth languages is also perfect for machines with limited offline storage.
The danger of Forth, is that much like with LISP, as soon as a programmer gets competent in it, they think "You know what the world needs? Another version of my favorite programming language".
Isn't this a general attitude? I think it applies to many programming languages, in particular at developers who know only their one favorite language.
I learned Forth on a 6502 besides Basic and Assembler, and it was the language with the best performance/memory ratio. It was about 10 times slower than Assembler, and very efficient in memory consumption. Memory was a really big issue at that time. And Forth as a programming language is not bad. You just need a lot of discipline (like at Lisp) because it is so extremely powerful. I wonder why Forth isn't used for firmware since buggy code could be exchanged at runtime.
True, although Forth and Lisp have the advantage of making (at least toy implementations) a project of a few spare evenings due to their simplicity.
I was also into Forth in the 1980s on the Apple ][, and have always meant to get back into it (or one of the new Forth-y languages like Factor) one day.
I did a lot of stuff on Paul Lutus' GraFORTH and TransFORTH. One of my projects on my 5th year in college ended up being a stack-based CPU inspired by Forth.
BTW, Mr. Lutus appears around here from time to time.
Not 6502 but the Zardoz 65816 compiler from back in the day (early nineties) had different memory models and one of the best allowed you to use the direct page register (essentially a moveable zero page) to access local variables in your functions. If you made some concessions to how you coded your C, you could end up with reasonably efficient code all things considered. It would be pretty cool to see a C like language that compiled efficiently for a stock 6502.
The author has more notes on the same topic elsewhere on his website, too (including reviews of Forths for 6502, PLASMA, and others): http://www.dwheeler.com/6502/
I think 6502 machines can do better with a version of BASIC that can compile into machine language binary files. One that has advanced commands for moving sprites and checking for collisions, etc. A BASIC compiler if you will?
But then you lose one of main reasons why so many programs were written in a mix of BASIC+assembly. BASIC's tokens were effectively a "virtual machine" with more functionality and a more expressive/"memory efficient" assembly language. Many of the basic programs on the apple ][ simply couldn't have been written in straight C/assembly without running out of RAM. Of course, its possible to build another virtual machine in assembly, or simply call directly into the BASIC routines in ROM, but that doesn't really buy much over just writing the application in BASIC looking for the places where BASIC is to slow, or doesn't support an operation you need and calling assembly.
I remember the day I broke into Ultima and discovered that it was mostly BASIC. I was like WOW, game programmers write in basic too. It validated the way I had hacked together my own ultima clone. (many years after the initial ultima, I was to young in the early 80's to be writing code).
White Lightning [1] [2]. The original version is Forth, but it also offers BASIC extensions and I seem to remember there's a "compiler" though a at least some of the BASIC compilers for 6502 based platforms did little more than turn the BASIC into a bunch calls into the BASIC interpreter to sidestep the interpreter loop so that might not say much.
EDIT: Here's the manual for Basic Lightning [3] which mentions in the preface that a compiler was due to be released in 1985.
EDIT2: Ah, it changed name to Laser BASIC when the compiler was released [4]
I won't ever forget typing in seemingly endless columns of hex numbers from a magazine to obtain both the interpreter and the compiler... (here are some original scans & documentation: http://www.tmeyer.de/atari/index.html).
Back in 8-bit days I didn't have access to the Turbo Basic XL, but was reading about it. I was really excited about the new features (speed, graphics commands, control structures) and at the same time miserable that I can't use it. The local computer shop offered a lot of Polish Atari games (they were great [1]), but not so much tools.
Anyway, once I bought a new game "Kolony", stored on a cartridge. It was a multiplayer game, asking about the number of players before start. Accidentally, I pressed CTRL+3, which caused some error and I got the BASIC prompt. It turned out that the whole game was written in Turbo Basic XL and the cartridge included the interpreter as well. I just have to type NEW to erase the game code itself and start programming in the new BASIC flavour.
I always wondered why they didn't make disks available for sell with the programs on them? The two assembly programs I remember the best having to type out myself for the Apple //e was one that added double high resolution graphics commands to Applesift Basic and one typing out shape tables of letters so you could easily mix text with graphics.
So if you are designing the hardware, give it an extra 4 KiB or so (16 pages), mapped through a paging circuit to the zero page. On entry to a function, increment the current page; on exit, decrement. Now you have a way to push all "local variables" that is faster than even pushing all the registers. You could even get fancy and XOR bits 7-4 of the page register with bits 4-7 of the paged address lines, so the compiler can choose anywhere from 16 256-byte frames up to 256 16-byte frames.