Although I reimplemented the commonly used list processing functions (those from R5RS and part of those from SRFI 1) manually, I think the 'right thing' to do is to write an analyzer that is able to take the strict functions (the code from SRFI 1) and produce lazy variants; basically, a lazy sublanguage. I haven't gotten around doing that yet (it needs access to lexical code analyzis, and will want to optimize so that only essential parts are delayed). (I know Racket implements a lazy sublanguage, so they must have done that.)
Programming language implementations often retain references to memory a bit longer than necessary for running the programs written in that language (i.e. memory is not freed even though the program can't access it anymore), sometimes on purpose because it can help with introspection (debugging). Usually the assumption is that subexpressions or subroutine calls will return quickly and/or that the memory retained will not grow, both of which can be voided when using lazy sequences (the subexpression can take a long time, and on top of that, grow the stream whose head is being retained by the language implementation). This leads to leaking and potentially running out of memory, which I think is what SRFI 40 suffered from. I agree that SRFI 41 is very weird; I think it went to great contortions because of such language implementation issues--I'm still not totally sure, in any case the impression I got was that the SRFI 40/41 implementors didn't really completely understand what was going on. The post here says
> Both SRFI 41 and lazy-seq don't have this problem.
Now my guess is that lazy-seq doesn't suffer just because Chicken does not retain memory too long (for the cases that the author tested). The author doesn't tell whether he has tested SRFI 40 for leaking on Chicken, it might well not have leaked, either!
On another note, I like how Scheme, like lisps in general, treats cons cells simply as pairs of any data type, not a sequence building block. Clojure does away with that generality, but at least buys interfaces (genericism for processing various kinds of sequences) and perhaps (I don't know in detail) better interoperation with Java in return. Scheme already has the promise data type (their naming of those predates the different use of the same term in JavaScript) and |delay| and |force| to create and evaluate them, which the library here reimplements as |lazy-seq| and |realized-lazy-seq|. Perhaps this introduces a promise with a different type tag, which would be one ingredient to help build a generic sequences type, but since it uses a make-lazy-seq that appears to be provided from outside the library, I'm not sure (the article doesn't go into specifics, either.) Also, on a smaller note, why lazy-each and not lazy-for-each (which would be consistent with traditional naming in Scheme)?
https://github.com/pflanze/chj-schemelib/blob/master/stream....
which is using
https://github.com/pflanze/chj-schemelib/blob/master/lazy.sc... https://github.com/pflanze/chj-schemelib/blob/master/define-...
and I've also implemented about the same in Perl here (work in progress):
https://github.com/pflanze/functional-perl
(which is used in http://ml2json.christianjaeger.ch/ .)
Although I reimplemented the commonly used list processing functions (those from R5RS and part of those from SRFI 1) manually, I think the 'right thing' to do is to write an analyzer that is able to take the strict functions (the code from SRFI 1) and produce lazy variants; basically, a lazy sublanguage. I haven't gotten around doing that yet (it needs access to lexical code analyzis, and will want to optimize so that only essential parts are delayed). (I know Racket implements a lazy sublanguage, so they must have done that.)
Programming language implementations often retain references to memory a bit longer than necessary for running the programs written in that language (i.e. memory is not freed even though the program can't access it anymore), sometimes on purpose because it can help with introspection (debugging). Usually the assumption is that subexpressions or subroutine calls will return quickly and/or that the memory retained will not grow, both of which can be voided when using lazy sequences (the subexpression can take a long time, and on top of that, grow the stream whose head is being retained by the language implementation). This leads to leaking and potentially running out of memory, which I think is what SRFI 40 suffered from. I agree that SRFI 41 is very weird; I think it went to great contortions because of such language implementation issues--I'm still not totally sure, in any case the impression I got was that the SRFI 40/41 implementors didn't really completely understand what was going on. The post here says
> Both SRFI 41 and lazy-seq don't have this problem.
Now my guess is that lazy-seq doesn't suffer just because Chicken does not retain memory too long (for the cases that the author tested). The author doesn't tell whether he has tested SRFI 40 for leaking on Chicken, it might well not have leaked, either!
On another note, I like how Scheme, like lisps in general, treats cons cells simply as pairs of any data type, not a sequence building block. Clojure does away with that generality, but at least buys interfaces (genericism for processing various kinds of sequences) and perhaps (I don't know in detail) better interoperation with Java in return. Scheme already has the promise data type (their naming of those predates the different use of the same term in JavaScript) and |delay| and |force| to create and evaluate them, which the library here reimplements as |lazy-seq| and |realized-lazy-seq|. Perhaps this introduces a promise with a different type tag, which would be one ingredient to help build a generic sequences type, but since it uses a make-lazy-seq that appears to be provided from outside the library, I'm not sure (the article doesn't go into specifics, either.) Also, on a smaller note, why lazy-each and not lazy-for-each (which would be consistent with traditional naming in Scheme)?