This is fantastic news. Speedups across the board and lots of small goodies everywhere!
Previously I had been using macros to bolt-on support for definitions in expression context, but that was buggy at best. If you have time you should look at the algorithm behind this letrectification. The paper is called "Fixing letrec (Reloaded)" and can be found here: https://legacy.cs.indiana.edu/~dyb/pubs/letrec-reloaded.pdf
This means letrec is now just as fast as let, which means (define ...) at the top of functions now is as fast as (let ...). For small functions called often, that could add quite a bit of overhead.
I usually use Common Lisp and when I use a Scheme-like language I usually use Racket. However, projects likes Guile (great for scripting applications, the web framework is cool), Chez, and Gambit are much appreciated (I use them all, at least a few times a year).
I know that Lisp languages are not the best tool for every application, but using Lisp languages makes me happy so I try to use them whenever possible.
Excited by the new release. I learned a little bit of Guile for scripting in Make, and I dig it.
It's a little hard to include in my own designs though, as much as I'd like to do it. If I had a wish-list, it would be:
- Make some of the dependencies (especially libgc) internal.
- Reduce the installed size.
- Consider adding a license exception that allows using Guile without inheriting GPLv3's Installation Instructions requirement, which prevents inclusion on a lot of commercial products (even as a standalone program for running scripts).
The Guile team is doing great work, and I'll enjoy trying out 3.0 once it hits my distro packages. I just wish that it slotted better into more systems.
There's some humor in the idea that Guile started out with the stated goal of being a Tcl killer. Years later, a Guile install is almost 8 times the size of a Tcl install and it introduces significant external dependencies. The licensing is also worse for inclusion in larger systems.
Just a note from someone who was bitten by GNU make and guile integration last year. Huge numbers of systems have old make versions with no support for guile extensions¹, and many systems that do have new enough releases require a separate package² to use it.
And yeah, I'd written all my fancy integrations before figuring out they'd be worthless :/ It did allow me to noodle over the build system for long enough to come with some real improvements though, so it wasn't all wasted.
I had the same experience. I think the slow adoption of Make's Guile extension is precisely because of its larger dependency load. If Guile had the same set of dependencies as GNU Make, I bet we'd see more distros turn it on by default.
FWIW, I switched that same project to meson¹ about six months later with no pushback whatsoever. I expected there to be some, but it went through review with uniform agreement.
I'm not sure what that says. `apt install make-guile` vs `apt install meson` don't seem all that different, if it is just about dependencies. I think that means your point on licensing is probably the bigger issue, or that perhaps I just picked the wrong day to open a PR with a guile extension…
My phrasing was a little ambiguous maybe. What I meant was that the regular `make` package from most distros isn't compiled with support for Guile. So _distros_ are the ones that aren't adopting it.
Make's Guile extensions will probably never see widespread use until distro packagers compile them in by default. And distro packagers don't necessarily want to do that, because it would mean a bunch of extra dependencies (including ~100MB of Guile) just to compile a core build-tool.
If `make-guile` remains an optional (and incompatible) package distinct from vanilla `make`, users like us won't be able to rely on its availability. Which means it won't see big adoption by users. And many distros aren't interested in increasing `make`'s dependency footprint to support a userbase that doesn't exist because of chicken-and-egg reasons.
I feel like the single most important reason Guile is not more popular is the focus on Linux. It is not even as easy to install it on Mac, afaict.
I've seen unofficial binary builds for windows but they are hard to find... and probably they use cygwin?
I wish Guile was readily available in other platforms (I wish their downloads page looked like this [1]...), but I'm guessing the main reason it is not is more philosophical than practical.
Can't understand why a Lisp would be hard to build on a Mac or elsewhere. What are the dependencies besides some common C libraries (e.g. for math, unicode handling, etc)?
> Can't understand why a Lisp would be hard to build on a Mac or elsewhere.
Huge number of complex and extremely platform-specific dependencies and modules, like garbage collectors, foreign function interfaces, native code generators, etc, etc. All inherently very platform-specific, fragile, and dependent on specific tooling. Very much not plain C code.
I build Guile on the Mac regularly. I need to pass the paths to these 4 libraries to configure: libgmp, libunistring, libiconv, and libreadline. The libraries themselves I get from MacPorts. Then make check && make install. Doesn't look all that awful to me to be honest.
Doing it using libraries installed by homebrew is borderline impossible unless you write your own brew formula. I have up on trying to use deps from homebrew and went with macports, and voilà: worked flawlessly.
It is not unique to guile either. Compiling things on Mac os brings back memories from Redhat 8. I have had so much more luck with WSL or even cygwin than Mac os for at least a dozen open source apps.
I've tried to build it on Mac in the past (years ago). I spent hours fiddling with it, and finally gave up and ran brew install guile. It might be better now?
I have built some other GNU packages without issue. In particular, I remember having easy success with CLISP. But at the same time, I have not needed to try to build a lot of packages, as many are available as pre-built binaries already.
Is there still any work being done on Guile Emacs? As I recall it basically worked a few years ago, but it doesn't seem to be maintained. I'm curious if these 3.0 speedups would translate there too.
Guile runs elisp just fine, although there are quite some low hanging optimization fruit to be picked.
From the Emacs side I strongly doubt it will happen. The discussions that have been going on for faaaar less intrusive changes is staggering. Reading the discussions about the portable dumper (new in the upcoming emacs 27, proposed in 2014) is a nice 4 hours. That was in just about every measurable way a step forward for Emacs, yet it took almost 4 years for it to be mainlined,despite glibc more or less deprecating the old unexec from under emacs' feet.
I might be misrepresenting just about everything here, but one thing I am pretty certain of: guile Emacs will never happen, despite the promises of running elisp on guile.
While this may be the case, my understanding was that once the codebases were combined and guile-emacs builds were brought online, they encountered a severe performance drop from vanilla Emacs due to back-and-forth string conversion between Guile's Unicode strings and the Emacs internal enhanced string type. I thought that the project was abandoned after finding no obvious resolution to the problem short of a complete rewrite of all Emacs string handling.
I thought people at my workplace loved to schedule meetings and talk endlessly about things that could have been easily implemented in less time than the meetings took. Then I remember emacs-devel... and it's infuriating. There's bikeshedding, and then there's how to paint the bikeshed, discussions about bikeshed compliance, if bikeshed should be called GNU/Bikeshed, if the bikeshed even supports non open-source bikes and how to make the bikeshed not compatible with them, and that's before the feature itself is even discussed. Even if it is already implemented and there's an actual patch.
> guile Emacs will never happen
Universe heat death is still far away, there's always hope.
changing anything in GNU (with a few exceptions) is like pulling teeth. it is endlessly frustrating. every proposed change, no matter how small, will be met with essay length responses from the old guard about why it shouldn't be changed. just one example of many: gnu.org is still managed via a cvs repo because rms doesn't like git and it doesn't matter to him that newcomers find it to be a roadblock. it's amazing that anyone has the patience to swim against that current and improve things.
I think it could potentially happen by way of a fork, similar to the history of GCC and EGCS, but it would take a dedicated group of hackers to make it happen. Not one guy doing all the work and a bunch of Guile fans on the sidelines cheering for him.
No, I don't think anyone's working on this. It is also very unclear if it is really worth the effort (for a good overview see https://lwn.net/Articles/615235/ from Stefan Monnier, Emacs' ex-maintainer). For speeding up Emacs Lisp, I think the current work done by Andrea Corallo using libgccjit is more promising. Another big boost would probably be to replace the current regex engine with something better.
thanks for the news, and yeah considering the amount of time emacs extension spend re-searching faster regexes would probably translate to big speed increases
I would like to ask a genuine question (not trying to provoke anyone!). Where is a good use case for Scheme these days (in any form?). I really love the idea of the language but I can't figure out where it really does a _better_ job that these things:
-- JavaScript/Typescript for in-browser or even some server-side stuff (e.g., NodeJS)
-- Python seems to be dominant for non-browser code that doesn't need to be fast (or which needs to call Python libraries)
-- C/C++/Rust/C# in the performance space
-- And then the workhorse Bash/ZSH etc for command-line script-fu
I am sincerely asking, what's the nice good fit for a Lisp these days? I know Emacs uses it as its internal language -- fair enough. But other than that.
Thanks and I did not mean to hurt anyone's feelings, I just really am curious.
I'd say scheme fits into the python niche rather well. It's faster, easy to learn, and usually has good C interopt (or great interopt in the case of ones like chicken).
<stands on soapbox>
Scheme: fix your damn documentation
I have to know that SRFI (scheme request for implementation) is a thing. Then I have to look over hundreds of them to find the one I want (do I want SRFI 69, 90, 125, or 126 for hash tables?). Then I have to find that obscure doc somewhere that says which SRFI are supported. Then I have to read the specification doc as the only source of documentation on https://srfi.schemers.org/.
I once needed to get some complex data out of a very restricted interface (syntax trees from a compiler plugin, where the only communication channel was stderr). The AST nodes provided a generic-programming interface, so I made a dead simple tree traversal which spat out the node type followed by each child, wrapped in a pair of parentheses to disambiguate structure.
Since these dumps turn out to be s-expressions, it seemed like a good idea to do further processing using a Lisp. I picked Racket (a Scheme) and it was great: parsing was a no-op, and the language made recursing through the structure and pattern-matching on the contents really simple.
The only downside was that I wrote a lot of "contracts" (dynamically-checked types), since I'm more comfortable in statically-typed languages (Haskell, StandardML, etc.). Turns out that Racket contracts are REALLY slow, so I ended up using a macro to discard them unless we were running the test suite :(
Scheme has been the 50th most popular language for 50 years running. It has never been the best language for any particular use case, but it has been good enough to inspire a community to keep it running and relevant all this time.
If you made the same kind of list of 4 languages that dominate all use cases back in 1990 (say), Scheme wouldn't have made the list then either; BUT the 4 languages would be different than they are now. If you want the most frictionless and popular language for the tech stack of the moment, use the trendy language du jour, but you have to live with neverending churn. If you want a language that is perpetually "good enough," use something like Scheme.
There are use cases where longevity and inoffensiveness are important. Embedded scripting, the original motivation for Guile, is one of them. Scheme is my go-to for little personal utility programs. After a long day of dealing with headaches from trendy environment du jour, I want to come home to something that's bulletproof, just works, and I already know. The minimalism and clarity make it a good first learning language. It seems like it'd be good for code "for the ages" like reference implementations and government software, but AFAIK there hasn't been much of that.
If you've ever felt a sense of flexibility and freedom writing JS... Scheme takes it one step further. Guile is a great, well documented Scheme implementation.
I prefer typing and less flexibility on larger projects, but Guile is my go-to for for small/medium programs.
Uh, this is all nice and such I suppose, but how do I install it? The README file in the git repository refers to an INSTALL file, which doesn't seem to exist as well as to a ./configure script which doesn't either.
./configure is a generated file. It's not committed to the repository. If you download a release (like "guile-3.0.0.tar.gz"), it has the files you're looking for.
Or you can learn autotools and do it yourself, but I don't recommend opening that can of worms.
Hmmh, it's been a decade or two since I used the autotools last, so I'll need a hint. I find it slightly disturbing that the README file is of little help though.
Further, if I try naively
$ sh -x ./autogen.sh
then that fails on a dependency on libtoolize, of which I never have heard of and which doesn't seem to be known to bananian.
libtoolize is a program provided by the libtool package. It's pretty common in the GNU ecosystem, and is used for managing some of the more subtle aspects of build-time and runtime linking.
Agreed that the README needs much better documentation of the package-time dependencies (autoconf, automake, m4, perl, etc), and the build-time dependencies (libtool, libgc, plus a bunch of other libraries).
Yeah, that was just my ignorance which a quick search on the Imterwebs remedied. With the bananian (Debian) package 'libtool' installed (and flex and libunistring-dev)
$ sh -x ./autogen.sh
completes successfully and I got a ./configure and a INSTALL file (don't recall having seen an INSTALL file be autogenerated, but my memory might very well be failing me there). The './configure' also completes successfully (after a while -- seems like autoconfigure still believes that there are other Unices besides Linux left ;-} but 'make' eventually fails with
--8<--
[..]
CC libguile_3.0_la-intrinsics.lo
CC libguile_3.0_la-ioext.lo
CC libguile_3.0_la-jit.lo
jit.c:232:1: error: initializer element is not constant
static const jit_gpr_t THREAD = JIT_V0;
^
jit.c:237:1: error: initializer element is not constant
static const jit_gpr_t SP = JIT_R0;
^
[..]
-->8--
This is correct, however much of the work we've done to make Racket work using the Chez VM could be re-used to build a Guile-backed version of Racket. It's not clear whether that would be of significant value, but it's doable.
The only reason Guile still exists is the stubbornes of GNU project to accept better alternative solutions over the course of time and push their own as the only true.
...which is a great thing if you want to write code that will still run in 30 years. If you're writing code geared toward a company's quarterly report, the long-term doesn't matter, but that's not everyone's goal.
I've been a Guile user since 2001 or 2002 and all these years it's been a workhorse for me. Even before there was a compiler or before the compiler started getting better, Guile was a solid choice for embedding. Each of the alternatives had issues.
Now that Scheme runs faster I get to write more Scheme and less C and I couldn't be happier.
Previously I had been using macros to bolt-on support for definitions in expression context, but that was buggy at best. If you have time you should look at the algorithm behind this letrectification. The paper is called "Fixing letrec (Reloaded)" and can be found here: https://legacy.cs.indiana.edu/~dyb/pubs/letrec-reloaded.pdf
This means letrec is now just as fast as let, which means (define ...) at the top of functions now is as fast as (let ...). For small functions called often, that could add quite a bit of overhead.