LambdaExpression is a part of the dynamic language runtime (DLR) and is absolutely useful if you are into run-time code generation. The analogous contract for this in the lisp world would be quoting, but wouldn’t be as fine grained as the DLR since it limits what abstraction can be applied (eg if you want to compute what parameters or what operations are needed at run-time).
Given the performance it allows for run-time generated code, nothing in production really beats the DLR. It is a shame Microsoft nerfed it in UWP because they ripped out the jit in favor of ahead of time compilation.
I'm not familiar with how .NET runtime works, but why does AOT compilation nerf runtime code generation?
Take SBCL - an open source Common Lisp implementation that compiles everything by default. That includes any run-time generated code. The code is simply AOT-compiled at runtime, at the point of generation.
They don’t include a JIT in UWP, so run time generated code is interpreted rather than dynamically compiled.
Run-time generated code in the DLR is very programmatic using the full C# language (eg expressions are values that you can store in a field to be retrieved later and completed), they aren’t just simply quoted templates. So an AOT compilation of run-time generates code is very impossible (consider if you used the DLR to execute code for your own live programming language layered on top...).
It is if you want ahead of time compilation to work on your generated code. People often confuse macros with full programmatic code generation, but they aren’t equivalent at all, of course.
How is that possible, given that macros are arbitrary Lisp procedures, which are transforming arbitrary input expressions (and world state) to output expressions (and new world state)?
Also given that code generation is not limited to macros in Lisp. Any function can generate code and hand it over to the embedded AOT compiler.
I’m talking about the quoting inside to implement macros, they don’t allow for generalized run-time code generation.
An embedded AOT compiler is compiling code at run-time, which isn’t allowed without writable code pages (which similarly makes the JIT disallowed as well).
Sorry, I still don't understand. What do you mean by quoting vs. generalized code generation? In Lisp the macro can be a procedure, which takes expressions and then uses an arbitrary computation to generate new expressions. Using a quoting mechanism and code templates is fully optional.
It can be, but generally isn’t as most macros don’t require generalized code generation, so quoting works just fine. Going back to the first comment I was replying to:
> (In one case, I knew an experienced C# programmer to build and compile an expression at run-time using LambdaExpression [1]. It takes about 10 times as many lines to achieve the same thing, and you have to write in a style that looks nothing at all like a normal function, so in practice nobody ever does this. In contrast, the way to accomplish this in Lisp takes one character, and the code looks identical to a normal function, so it's not unheard of.)
I assume they mean that they didn’t need to manipulate expressions in a general way because what they wanted was a simple macro that lisp supports with nice template/quote syntax, whereas C# doesn’t support macros at all but you can do something hacky with general run-time code generation.
For others reading this, this was for security reasons (UWP sandbox restrictions). Turns out having memory that is both writable and executable isn’t great for sandboxes.
It's a sad state of things. Given that for any runtime-generated code, you can make an equivalent by building an interpreter and interpreting data - only significantly slower - this doesn't seem to me to be buying any security at all, and the cost is a deal-breaking level of inconvenience for the programmer.
If you’ve found an exploitable memory safety bug, the goal is usually to execute some attack payload and do something useful to an attacker. This generally involves injecting some sort of code. If you can write to executable memory, this is easy. If the attack target contains an interpreter, and you can convince the interpreter to interpret your payload, you also win. If the target doesn’t contain an interpreter, your job is considerably harder.
In locked-down experiments, e.g. iOS but also plenty of SELinux-ish things, there may be no interpreters available and no ability to execute newly delivered code. This does add some degree of security, and it requires a lack of JIT support.
We live in an age where we can circumvent processor hardware security using prefetching. I’m not sure what we can really say about security anymore.
Also, the DLR wasn’t a very popular feature used by .net developers, even if it was really well done (wrt to performance and static type compatibility). None of the DLR languages ever took off (ironpython and ironruby), and very very few use the API to generate code, it’s nerfing isn’t inconveniencing many programmers.
Given the performance it allows for run-time generated code, nothing in production really beats the DLR. It is a shame Microsoft nerfed it in UWP because they ripped out the jit in favor of ahead of time compilation.