Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: ArkScript, a small and fast language for scripting video games (github.com)
100 points by jackrabbit_ 3 days ago | hide | past | web | favorite | 42 comments

A while ago, I was using Lua as a scripting language for my games but wanted something between Lisp and Python, a language which would run on a VM to be able to export only the bytecode, and to be able to code in a lisp like syntax.

And ArkScript was born. I wanted to keep it tiny, under a megabyte, with a few keywords to do everything.

I think I can say I achieve what I wanted to do, now we are working with a small team to improve the code, the documentation, and the standard library. The language is indeed small and fast (even if we can do better, and I'm still seeking for more speed), with immutability (with 'let'), and closures (note that we can read the data captured by a closure easily, as a small object, through for notation: object.field).

Interesting project. By chance have you heard of fennel[0]? It is a compile to Lua language with s-expression syntax. Also I believe you can export LuaJIT's bytecode with string.dump and it would be portable across platforms. Not trying to diminish your work, however. This could be useful for having a "real" Lisp embedded in-game. I don't think I've seen a scripting language with immutability either.

Would it be possible to write a REPL using ArkScript or use the compiler from within the language?

Also, I'm really curious how you got so many people to work on the language. How did you accomplish this?

[0] https://github.com/bakpakin/Fennel

I've never heard of fennel before, looks interesting!

Yes it's possible to launch a REPL by using the `-r` or `--repl` option when launching ArkScript executable.

A few people contributing on the language are people I know from some programming discord servers, and the others came to work on the project after discovering it thanks to the hacktoberfest tags I put on the issue.

What is a scripting language for you?

Because I think (could be wrong) elm, purescript, clojurescript and few others are immutable.

Along the same lines, Embeddable Common Lisp can optionally be configured to use a bytecode interpreter. I have no idea if said bytecode can be exported and moved between platforms though.

Great project. Looked around the repo to understand how the FFI works without annotations. Maybe there is a section I missed. I'm curious how you translate between ArkScript types and C++ types? And what is the error-handling like when the programmer gets it wrong?

Also, if you create objects by making all variables within a function scope accessible outside the function, do you have a means for private local variables that don't get exported?

Thanks! Some people are going to be a bit angry about what I'm going to say, but the language is 100% dynamically typed, so we retrieve `Ark::Value` from the language, and in C++ we check the data type by using `value.valueType()`, we can then retrive the content of the value by using the right getter (number, string, etc). When an ArkScript developer get a program wrong, the virtual machine is designed to throw the code away at runtime, without being able to recover. It could sound strange, but my idea behind that is to throw away the exception design (as used in C++) to have a error code design as in C, or even better, a result design as in Rust. Everything is exported actually, if you care about users being able to modify values, don't worry, we can only read the values captured / created in a closure, only the closure itself can modify them. The privacy is a convention, by prefixing the variable name with an underscore.

How do you know what types and how many arguments you need to conform to the foreign function? Every language I'm familiar with requires the programmer to declare the types and number of arguments.

That's where it's tricky, the compiler doesn't know, it's checked at runtime by the C++ function, which receives a list of Values, count them (actually quite convenient if we need to make variadic argument functions) and decide if it's fine or not, then does type checking on the needed argument. I know it sounds crazy because it can cause performance loss, but in a dynamic language like this one that's a cost I must pay.

That's not what you'd typically understand as an FFI; that's just an interpreter / VM API. A foreign function interface would generally allow something running in the VM to call arbitrary external (hence foreign) functions that were not written against the VM's API, i.e. don't have a binding.

For example, this is what FFI could look like in some language:

   lib = ffi.load("kernel32)
   lib.TerminateProcess.args = (ffi.voidptr, ffi.unsigned)
   lib.TerminateProcess.returns = ffi.int
   lib.TerminateProcess.callconv = "stdcall"
   lib.TerminateProcess(1234, 56)
Arguably you neither want nor need FFI for a game scripting language.

You're right, I'm using the wrong name for this. At the beginning the goal was to have what you described as a FFI and what I thought a primitive "FFI" would look like (eg. what it is today, a system using the VM API), but I never managed to do both and the name stayed, and now I'm here a bit confused, messing up and mixing words..

Right, I'm confused /curious about FFI calls not regular function calls.

I'm sorry, I used the wrong word (see the details in the above comment: https://news.ycombinator.com/item?id=22698450) for this..

It's not a FFI but an interface using the VM API.

Wait but when you `(import "librandom.so")` and call `(random)` in your examples, that's an FFI. But I guess I'm getting the picture anyway. When there's a foreign function call, you count the number of arguments and convert them to their C++ types and call a helper function for making a call against some function pointer with so many arguments? I'm still confused though how you'd pass strings or function pointers for example.

That's the idea yes.

Since an example is worth a thousand words, here is a function for the http module, counting argument, checking the types and then using the values to create an http client (to make http requests to web servers): https://github.com/ArkScript-lang/modules/blob/refactoring/h...

That's not exactly an FFI, that's an extension since you're coding against ArkScript's C++ API. An FFI is when you use only ArkScript to make calls against other languages that don't know about ArkScript's API.

The wiki (https://github.com/ArkScript-lang/Ark/wiki/Embedding#creatin...) calls them “plugins” and they’re also called “modules”, which I think is a lot clearer than “FFI”. Essentially it’s a bunch of native functions that are exported with C linkage and compiled into a set of shared libraries, which are then loaded and the functions in them looked up at runtime. On the other (C++) side it’s similar to e.g. a JNI binding or any other language where you can pull values out of the runtime.

> An FFI is when you use only ArkScript to make calls against other languages that don't know about ArkScript's API

I should definitly rename this part as stated in another comment, since the "FFI" isn't a real one, I messed up with the vocabulary

Super fun. This is a great project.


That all sounds like lua, where does lua not work?

Presumably the lisp-like syntax and immutable variables by default. Lua has some functional features, but it isn't a primarily functional language like this.

What is it missing though? It seems like this could be a different syntax compiled to Lua bytecode.

It could have been compiled to Lua bytecode, but I felt like having an external VM and ArkScript would have been too much. I always hate when I download a lib and 100 smaller libs come because the main lib relies on it

Small libraries are an accountability nightmare. Reducing the number of dependencies on a core library is very important.

This also seems odd to me because Lua can be embedded in a binary easily or made into a shared library since it is just C with no dependencies of its own.

Interesting, makes me think of Naughy Dog's GOOL/GOAL, also a Lisp-like scripting language that was used internally for many games, including Crash Bandicoot on PSX

Exactly how can a language that's 4x slower than CPython, on the only benchmark posted, be considered "fast"?

This is a very heavy test of non optimizable recursion, and (benchmark not published yet) ArkScript is only 2 times slower than JavaScript. It sounds a lot, but please note that I'm actively working on improving those numbers, and I'll add other benchmarks (fast allocations of lists and more) to have a better idea of what the language is capable of in term of speed.

> This is a very heavy test of non optimizable recursion, and (benchmark not published yet) ArkScript is only 2 times slower than JavaScript

That statement does not help. Which implementation of Javascript and what kind of benchmark is only 2x faster?

For starters, here is a good overview of Javascript interpreter performance:


Woops sorry

I used the Ackermann Péter fucntion, still the same parameters (3, 6), on spidermoney (js engine for Firefox 74.0, windows 10, i7 8k)

Thanks for the link, I'll definitely dig down into that!

I'll be very surprised if game scripting languages don't all devolve to whatever-compiles-to-webassembly. Not only will choices be more plentiful but the ecosystem will be stronger and safer than what any individual author could built before long.

I'd predict the opposite. Game developers tend to be hostile towards web technology. Not having that ecosystem in your pipeline is a bonus. Safety isn't something game developers value highly.

WASM would just make everything more complicated for no good reason. You can't do JIT-compilation on the consoles or iOS, so it's not going be performing any better than a simple interpreter.

If you go the interpreter scripting route, what you really want is trivial and comfortable binding to native (engine) code. You likely want your engine functions to be intrinsics in the VM.

Otherwise, you'll want to go the AOT compiled scripting route, which offers better performance. Then you just do the "scripting" in C++ (UE4) or C# (Unity).

Slightly tangentially, as someone that worked on games entirely in C# more than a decade or so again, it still sometimes makes me sad that most C# work in games is just Unity scripting today.

C# is a very performant and efficient language that I thought a lot more games would be written in top-to-bottom today, at least in indie circles. (Sure, performance tuning C# is a different art to C++ performance tuning, but it is a very similar art [know your data structures, keep an eye on your allocations, pool memory where you can].)

That "Safety isn't something game developers value highly" says a lot about why that C# bubble popped and C# right now is "merely" that weird scripting language choice for Unity.

OT but why do you need a separate language for scripting? Why not just use the same language as you use for the rest of your game?

Since language engines are usually in C++, it takes tons of time to compile (and link!) when you made the simplest change. It could be really annoying really fast if you need to recompile everything every time you change a mob's HP pool or anything.

The second reason is modding: World of Warcraft is scripted using Lua (like many games) and players can overwrite a lot of things via addons themselves being written in Lua.

Game logic can then be implemented in a safe, "clean" language where game designers can't break the underlying game engine.

It's also partly due to C++ being the Lingua Franca of game development - if it were a slightly less complicated/difficult to deploy language you could use both (I have built a toy GUI app that live compiles D code into an object file, and calls an entry point with a nice OOP interface into a made up data structure, it worked pretty well but I don't know much about games so I don't have much use for it). C++ is also ridiculously slow to compile (at it's beautiful worst) so having AI logic (say) in the main codebase is a bit of an issue for designers.

I also saw a Rust game engine using a scriping language that leans heavily on Rust concepts.

I think the scripting language was called Dyon.

Another reason besides those mentioned is it lets you deliver new content w/o having to deliver a new binary. This is especially important if you're targeting iOS, since there's no good way to force users to update (the best you can do is put up a dialog saying, "please update to continue").

Good question actually! I knew and I know that making a scripting fast enough to make video games is really tough, so I wasn't expecting to be able to make a language able to make games on its own (understand: not as a scripting language, but for the whole project as you suggested) ; also even if I want to throw away a few things from C++, I can't leave static typing and effeciency of C++: I think mixing the pros of C++ and the pros of my language was a good catch, the core in C++ (strong, fast, using optimizations I can't make in ArkScript), the scripting of actors, entities and all in ArkScript (dynamic language, easier to script entites behavior with).

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact