A while ago, I was using Lua as a scripting language for my games but wanted something between Lisp and Python, a language which would run on a VM to be able to export only the bytecode, and to be able to code in a lisp like syntax.
And ArkScript was born. I wanted to keep it tiny, under a megabyte, with a few keywords to do everything.
I think I can say I achieve what I wanted to do, now we are working with a small team to improve the code, the documentation, and the standard library. The language is indeed small and fast (even if we can do better, and I'm still seeking for more speed), with immutability (with 'let'), and closures (note that we can read the data captured by a closure easily, as a small object, through for notation: object.field).
Interesting project. By chance have you heard of fennel[0]? It is a compile to Lua language with s-expression syntax. Also I believe you can export LuaJIT's bytecode with string.dump and it would be portable across platforms. Not trying to diminish your work, however. This could be useful for having a "real" Lisp embedded in-game. I don't think I've seen a scripting language with immutability either.
Would it be possible to write a REPL using ArkScript or use the compiler from within the language?
Also, I'm really curious how you got so many people to work on the language. How did you accomplish this?
I've never heard of fennel before, looks interesting!
Yes it's possible to launch a REPL by using the `-r` or `--repl` option when launching ArkScript executable.
A few people contributing on the language are people I know from some programming discord servers, and the others came to work on the project after discovering it thanks to the hacktoberfest tags I put on the issue.
Along the same lines, Embeddable Common Lisp can optionally be configured to use a bytecode interpreter. I have no idea if said bytecode can be exported and moved between platforms though.
Great project. Looked around the repo to understand how the FFI works without annotations. Maybe there is a section I missed. I'm curious how you translate between ArkScript types and C++ types? And what is the error-handling like when the programmer gets it wrong?
Also, if you create objects by making all variables within a function scope accessible outside the function, do you have a means for private local variables that don't get exported?
Thanks!
Some people are going to be a bit angry about what I'm going to say, but the language is 100% dynamically typed, so we retrieve `Ark::Value` from the language, and in C++ we check the data type by using `value.valueType()`, we can then retrive the content of the value by using the right getter (number, string, etc).
When an ArkScript developer get a program wrong, the virtual machine is designed to throw the code away at runtime, without being able to recover. It could sound strange, but my idea behind that is to throw away the exception design (as used in C++) to have a error code design as in C, or even better, a result design as in Rust.
Everything is exported actually, if you care about users being able to modify values, don't worry, we can only read the values captured / created in a closure, only the closure itself can modify them. The privacy is a convention, by prefixing the variable name with an underscore.
How do you know what types and how many arguments you need to conform to the foreign function? Every language I'm familiar with requires the programmer to declare the types and number of arguments.
That's where it's tricky, the compiler doesn't know, it's checked at runtime by the C++ function, which receives a list of Values, count them (actually quite convenient if we need to make variadic argument functions) and decide if it's fine or not, then does type checking on the needed argument. I know it sounds crazy because it can cause performance loss, but in a dynamic language like this one that's a cost I must pay.
That's not what you'd typically understand as an FFI; that's just an interpreter / VM API. A foreign function interface would generally allow something running in the VM to call arbitrary external (hence foreign) functions that were not written against the VM's API, i.e. don't have a binding.
For example, this is what FFI could look like in some language:
You're right, I'm using the wrong name for this. At the beginning the goal was to have what you described as a FFI and what I thought a primitive "FFI" would look like (eg. what it is today, a system using the VM API), but I never managed to do both and the name stayed, and now I'm here a bit confused, messing up and mixing words..
Wait but when you `(import "librandom.so")` and call `(random)` in your examples, that's an FFI. But I guess I'm getting the picture anyway. When there's a foreign function call, you count the number of arguments and convert them to their C++ types and call a helper function for making a call against some function pointer with so many arguments? I'm still confused though how you'd pass strings or function pointers for example.
Since an example is worth a thousand words, here is a function for the http module, counting argument, checking the types and then using the values to create an http client (to make http requests to web servers): https://github.com/ArkScript-lang/modules/blob/refactoring/h...
That's not exactly an FFI, that's an extension since you're coding against ArkScript's C++ API. An FFI is when you use only ArkScript to make calls against other languages that don't know about ArkScript's API.
The wiki (https://github.com/ArkScript-lang/Ark/wiki/Embedding#creatin...) calls them “plugins” and they’re also called “modules”, which I think is a lot clearer than “FFI”. Essentially it’s a bunch of native functions that are exported with C linkage and compiled into a set of shared libraries, which are then loaded and the functions in them looked up at runtime. On the other (C++) side it’s similar to e.g. a JNI binding or any other language where you can pull values out of the runtime.
Presumably the lisp-like syntax and immutable variables by default. Lua has some functional features, but it isn't a primarily functional language like this.
It could have been compiled to Lua bytecode, but I felt like having an external VM and ArkScript would have been too much. I always hate when I download a lib and 100 smaller libs come because the main lib relies on it
This also seems odd to me because Lua can be embedded in a binary easily or made into a shared library since it is just C with no dependencies of its own.
Interesting, makes me think of Naughy Dog's GOOL/GOAL, also a Lisp-like scripting language that was used internally for many games, including Crash Bandicoot on PSX
And ArkScript was born. I wanted to keep it tiny, under a megabyte, with a few keywords to do everything.
I think I can say I achieve what I wanted to do, now we are working with a small team to improve the code, the documentation, and the standard library. The language is indeed small and fast (even if we can do better, and I'm still seeking for more speed), with immutability (with 'let'), and closures (note that we can read the data captured by a closure easily, as a small object, through for notation: object.field).