Hacker News new | past | comments | ask | show | jobs | submit login
JAI Primer: A Programming Language for Games (sites.google.com)
92 points by tasoeur on Oct 7, 2015 | hide | past | favorite | 65 comments



As of this writing there are no public compilers for Jai, so all information in this text is collated from Jon Blow's YouTube videos.

Until the language is actually released, at least in an alpha version, writing a tutorial is just speculation. Anything can change in the actual language implementation.

That being said, the author initiative is laudable if you don't want to watch a few hours of Youtube videos.


I have a friend who writes a lot of C++ code for signal processing, software sound synthesis mainly.

Watching Jon Blow's videos I could tell this was something my friend would be interested in, because the problem space is very similar: you need realtime performance, can't afford even the slightest garbage collection pause, etc. However, I could not tell my friend "go watch videos for six hours".

Thanks to Jorge Rodríguez, I can email my friend a document that will take him less than half an hour to read.


This is what Jon Blow said about C++ on Twitter: [https://twitter.com/jonathan_blow/status/561371202350878720]

"If you want to never get any work done, use a programming language like this: https://isocpp.org/blog/2012/11/universal-references-in-c11-... "

I used to enjoy programming in C++, even though I always thought it was too complex. Too many things to learn and too many approaches. For example, I never could grasp template-meta-programming thing. May be I didn't dedicated enough time, but that's the point. If it takes so much time to learn the language and its complexities, you have much less time to program "things" with it.


Rule of thumb for doing a programming language: Write the debugger first. Or at least, very, very early. This will inform you of poor choices you've made, and you're going to need a debugger anyway.


What poor choices are revealed by writing a debugger in particular?

At this point, he compiles Jai to C, and the compiler isn't too gigantic yet, so there's not a huge surface area to look for compiler bugs.


> What poor choices are revealed by writing a debugger in particular?

Well, for one, you can't just transpile to ANSI C, compile, and end up with sensible debug symbols in your binary. You'll end up with debug symbols for the C code, not the original language. So you end up writing in one language and debugging in another. This is one reason most new languages target LLVM.


Compiling to C with #line directive actually gives a good enough debugging experience, similar to compiling to JavaScript with source map. This is much easier than generating debug info, even with LLVM.


I don't think it's good enough. You can't inspect variables, types will be wrong, any namespacing you're doing will leak into the debugger, single step won't give you the right semantics unless your language is extremely close to C, gdb pretty printing won't know how to print your language's values, etc.

Backtraces will sort of work (although the name mangling will leak through), but little else will. You really need to generate DWARF directly in your frontend in order to have an acceptable debugging experience.


> Well, for one, you can't just transpile to ANSI C, compile, and end up with sensible debug symbols in your binary

Doesn't that depend entirely on how similar your language is to C? ;)


Doesn't it currently have a compile to c, and a to byte code? (which allows for compile time run of things)


I love the idea of seamlessly switching between AoS and SoA for data types.


For those who don't follow, the acronyms mean "array of structures" vs "structure of arrays". These are two orthohonal ways to represent data, both of which have advantages which make them attractive. Being able to switch the same data around between the two is cool and innovative.


I like that idea too, but what I like even more is the bold originality in it. I think hackers who make these new programming languages should try to be original, and experiment with weird things that actually help them solve whatever problems they feel they have during development, because language is pretty much a programmer's user interface.

And also this particular feature is close to my heart, because what I would like to see as a feature(s) in programming languages that the programmer would only supply the data schema, and compiler selects the actual data structure based on run time performance characteristics of the program.


The #run directive addresses some major pain points for me. Is there anything analogous in other languages?


Yeah, similar constructs exist in D and Nim.

I don't know well enough how they compare to give any thorough analysis, though.


I don't know what Nim do but AFAIK in D you're (voluntarily) limited in what you can do at compile time because in effect with Jai compiling a program is running it, so this can have security implications..


In theory a good optimizing compiler should see that some code doesn't depend on anything external and do it, but I'm not sure to what extent they do. GCC with higher optimization settings does do some pretty crazy stuff, so I would expect it to figure out something like in the example code pretty easily.


All Lisps have macros, which are the same only more powerful (as they're allowed to transform code).


Yes, constexpr in C++.


I'm excited for Jai, and if it ever makes it out into the world I look forward to building some games with it.

However, I just see C++ continuing to get better and better for games programming. The recent appearance of the Core Guidelines [1] (and the eventual codified form for compilers) will be a huge leap. I wish we had full #run-type control over the compile phase, but full constexpr support is a step of the way there.

1. https://github.com/isocpp/CppCoreGuidelines/blob/master/CppC...


Aside from the aos/soa I feel like Rust hits a lot of the same points.

Nice to see #run similar to elixir macros.


Yep, a lot of people have noticed that. He's mentioned Rust in a few of his videos (and the compiler devs are always taking notes about his grievances) but he doesn't seem interested in working with the Rust community in any way. He considers Rust too ideologically opinionated to be of use to him.


When you do new things, one of the challenges is other people's mental prototyping process. Crowds line up to say, "it's like x," when it's not. They just don't grok you yet. It takes time for people to get used to unfamiliar ideas.

Rust is a multi-purpose language, with an emphasis on clever type systems. That's not Blow's focus.

He wants there to be a clear path between simple code he's writing and the fast compiled code that results. He likes C, specifically this quality: you can reason about what the hardware is doing, and write C, and count on it doing exactly what you want. His emphasis in the work he's doing is to make a memory-foundry language that stresses this kind of power, with less of C's warts (and non of C++'s).

Mundane example: he has created a not-annoying function pointer syntax. Less obvious: he has created a high-level syntax that appears to be array-of-structs, but which you can /reason/ will be compiled to struct-of-arrays for caching purposes.


From the perspective of someone like Blow, Rust's concept of safety is like childproofing a house by putting latches on all the cabinets while ignoring the loaded gun on the coffee table.


I used to have similar thoughts. That said having used Rust for ~1 month now I'm much father along the line towards drinking the kool-aid.

I've yet to see any deal breakers(custom allocators are coming) and I've not yet crashed once(!) in my entire time using it. Try that with C/C++/ObjC/etc. With functional approaches to polymorphism combined with an ownership model that I really like and I think we could see Rust make some serious in-roads in the next few years.


I don't follow the analogy. What's Rust's loaded gun?


It's not Rust's loaded gun, which is sort of the problem. Basically, I'm talking about the sorts of relatively low-level APIs whose use is somewhat obligatory for Getting Stuff Done in a performant way, and for which higher-level interfaces typically deny sufficient control and/or hurt performance by heavily wrapping (and often quietly copying/rearranging) the critical data structures and operations. Things like ioctl() calls, OpenGL/DirectX, and so on. Misusing these (or just encountering bugs in them) can pretty easily trash your application -- and sometimes even your kernel -- in really messy ways.


> Basically, I'm talking about the sorts of relatively low-level APIs whose use is somewhat obligatory for Getting Stuff Done in a performant way, and for which higher-level interfaces typically deny sufficient control and/or hurt performance by heavily wrapping (and often quietly copying/rearranging) the critical data structures and operations. Things like ioctl() calls, OpenGL/DirectX, and so on. Misusing these (or just encountering bugs in them) can pretty easily trash your application -- and sometimes even your kernel -- in really messy ways.

OK, but that's why glium is so popular (quite possibly the preferred OpenGL binding in the community nowadays?), which has very low overhead while maintaining OpenGL safety.

ioctls have similar wrappers for the important ones.


glium seems pretty cool (arguably still immature, but certainly not the joke of some wrappers I've seen where the examples/ directory just has adaptations of NeHe 1-5 or so, and anything else requires fiddling with the GL entry points yourself), but I'm not convinced that it's possible to both have low overhead and effectively enforce safety for something like OpenGL. Maybe tomaka will prove me wrong, but I suspect that the worst problems are in the DX/GL/driver stack.

Anyway, the general principle I'm trying to get at here is that when you're in a field that requires specialized expertise beyond general-purpose programming, the domain problems tend to get much, much bigger than the problems posed by generic "unsafe" operations. It's sort of an Amdahl's Law thing. Even a language that gets (lack of) pointers and array bounds checking and the like 100% correct can only possibly save 5% of your hair from being pulled out.


> Even a language that gets (lack of) pointers and array bounds checking and the like 100% correct can only possibly save 5% of your hair from being pulled out.

That's not my experience, and I work in a very specialized domain (at the moment, OpenGL, in fact). It doesn't eliminate all the bugs, but memory safety saves me a lot of time.


Sure; I don't mean to deny that such workloads exist. I was a bit sloppy with my writing; I meant for that sentence to illustrate by describing the situation around the product I work on (which isn't OpenGL-oriented, but does involve a lot of things happening in third-party kernel code and other address spaces).


Jai also seems rather "ideologically opinionated" to me (which isn't a bad thing in my opinion).


> He considers Rust too ideologically opinionated to be of use to him.

He's not wrong. The rust community approaches everything from the standpoint that we should be forcing devs to code in a particular way because safety. What I like about Blow is that, so far, he's big into giving the dev options along with some (completely optional) syntactic sugar. The focus is on functionality, not ideology.


The idea that a programming language should focus more on helping people do things (functionality) rather than preventing people do things (safety) is an ideological stance, just as much as the reverse. Describing one side as functionality and the other side as ideology is not a good description.

I understand that many people's ideology align better with Jai compared to Rust. On the other hand, many other people's ideology align better with Rust compared to Jai.


> The rust community approaches everything from the standpoint that we should be forcing devs to code in a particular way because safety

No, I'd actually be perfectly fine allowing the safety features to be turned off, and have suggested it several times. Nobody has seemed particularly interested, though, because what people tend to find is that safety ends up being good for productivity as well.


I don't think that's entirely true. We had many people interested in turning off bound checks with compile option, but it was rejected.


Those who don't learn from history are doomed to repeat it.

My favorite Hoare quote given at his ACM award speech in 1980, referring to his work in Algol during the late 60's:

"Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to - they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980, language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law."


In Jai it is easy to, for example, have bounds checks on during the entire development process to aid in finding bugs, then turn off bounds checks in a dozen critical loops for testing and release, then turn bounds checking on in one loop that still proved crash-y.

Blow has said many times: he doesn't think the language should be prevent you from shooting yourself in the foot, because sometimes that means preventing you from doing what you want to do.

He usually asks: what is something simple we can do that will give us 90% of the benefit, without adding a layer of overhead. Rather than Rust's "Safe All The Things!" approach, bounds checking is just another tool you can use or not use as is appropriate for the context.


> In Jai it is easy to, for example, have bounds checks on during the entire development process to aid in finding bugs, then turn off bounds checks in a dozen critical loops for testing and release, then turn bounds checking on in one loop that still proved crash-y.

That's pretty much exactly Rust's philosophy as well.

> Rather than Rust's "Safe All The Things!" approach, bounds checking is just another tool you can use or not use as is appropriate for the context.

Rust's philosophy is not "Safe All The Things". It has "unsafe" for a reason.

Anyway, I really dislike the overfocus on bounds checks, because they're pretty much zero overhead if properly implemented and optimized.


Most Common Lisp implementations, for what it's worth, take this approach as well. For tight loops where you've removed the invariants and proved its bounds removing the run-time checks can give a non-trivial performance boost to production systems.


If you mean the (speed 3 safety 0), it's more in line with pcwalton's idea to have a switch for generally turning off safety checks, not just bounds checks - because the other checks typically affect performance more.


Most of the checks affect compile time performance in a minor way, but also have no runtime overhead. The borrow checker is an entirely compile-time thing, for example.


> We had many people interested in turning off bound checks with compile option, but it was rejected.

Can you cite where this happened? This sounds like it was an executive decision, but as I recall the community was opposed to a switch to turn off bounds checks. (Rightly so: the overhead of bounds checks is near zero.)

As I recall, there were some people asking for a bounds check switch, but they were mostly people coming from the D community and new to Rust who were asking for a copy of the -noboundscheck switch (without benchmarks to show that it would help). The more interesting option would be something to turn off the lifetime, borrow, and mutability checks. But when I suggested this, almost nobody seemed particularly interested.


Yes, I was referring to request to provide copy of -noboundscheck. You stated that nobody seemed interested in turning off safety features, so I provided a counterexample.

As a matter of fact, I am interested in -noboundscheck switch. The motivation is so that people can easily confirm bound checks overhead is zero by themselves, because it seems many people find this hard to believe. I didn't submit a pull request implementing this (it is easy to implement) because I thought it was fait accompli. Was I mistaken and you will support such pull request?


I actually intend to benchmark this myself in the future and write a comprehensive blog post on my findings. Personally, I would oppose any RFC that attempted to add such a compiler flag, especially if its only purpose was to prove a point.


It'd need to go through an RFC process, but I would be neutral and not opposed to such an RFC.


because safety

I'd rather play a game that performs a little poorly than a game that crashes. I gave up on Dragon Age Origins due to how buggy it was, for example. I've also refrained from buying games that looked great due to bug reports. I can't tolerate a broken product, but I can tolerate one that doesn't perform quite as well as I'd like.

Therefore, I think giving up on safety in favour of performance is a bad idea. Sure, its possible to write good code in any language, but the fact is people make mistakes, a lot and software is, in general, extremely buggy.

Of course, I understand that not everyone thinks like me and the "average gamer" is possibly much less forgiving for performance problems than I am, so Blow of course is doing what he feels best serves his goals for the market he's in.


The CVE database is full of proofs of what happens when a language that disregards safety like C gets adopted.

Profilers do exist.


Any Rust people around willing to speculate about adding data layout options? Would it be easy or hard, and would it be something current Rust hackers might consider doing or would it take a volunteer with a mission?


We already provide a repr attribute which controls representation to some degree, though it doesn't do anything like this.

At the moment, I think it would be a volunteer with a mission: I'm not aware of anyone who's even directly indicated that they'd like this feature anywhere official yet, let alone made a proposal. That doesn't mean it's bad, I just mean it's not directly on our radar at this time.


Lots and lots of people have indicated interest in more precise data layout options. Someone even tried to make AoS vs SoA in a macro.


Watching him occasionally stream his sessions shows me that what he is doing is just funzies.

His tooling is so far behind anything all other languages have that I would never bother to learn this language, because it was "for games."

If I wanted to make a game I would learn any engine using any number of language bindings.


He's been working on this project mostly alone for only a few months, how can you possibly expect the tooling to be on par with anything else.

There's a reason it's not released.

And it's great that you can just pick up an engine, but someone's gotta write that engine, and there's a reason triple a studios aren't using unity for major releases; you'll never get quite as good performance out of a general purpose engine as you would from a custom one.


> there's a reason triple a studios aren't using unity for major releases

They are, however, using Unreal Engine for tons of major releases.


But most of those major releases require some amount of custom work on the engine. They still need C++ programmers who can understand what's going on under the hood.


Hearthstone (by Blizzard) is a Unity game, not that performance is a huge concern for card games. It's a pretty thin client mainly for displaying the animations/artwork while most of the logic runs serverside.

Still, it's a very profitable game by a triple A company.


Hearthstone isn't what anyone would normally call a AAA game though. Until the end of the beta they only had 15 employees working on it.

They basically had to use an established engine because they didn't have the budget to build their own.


>Hearthstone (by Blizzard) is a Unity game,

And I asked myself why Hearthstone stresses my system so much. Now it's clear.


Endless Legend, Endless Space by Amplitude Games are all made in Unity. While not AAA FPS, they are certainly not trivial games.


JAI is for games in the sense that reflects the experience of a well known game developer that writes game engines and games.


And also in the sense that it allows you to easily remove the safety checks when you need higher performance while Rust doesn't necessarily allow it (array access for example). So I'd say that Rust seems more suited to write an OS, a webbrowser as in both cases safety is critical and Jai for games as performance is critical. For other applications which needs low latency (so no GC) but are neither safety critical nor performance critical? Jai's 'look and feel' seems to me much more seducing than Rust (assuming/hoping Jai becomes real).


Rust does let you do unsafe array indexing and other unsafe stuff. But it doesn't support switching the semantics of safe code to unsafe with a build flag, you need to explicitly use "unsafe" in code.


> But it doesn't support switching the semantics of safe code to unsafe with a build flag

Jai allow this for array indexing AND it also allow the reverse: reinserting array checks even when there is an 'unsafe access' operation: this way even if you made a mistake in adding an 'unsafe access' operation it's easy to find where the error is (assuming the error is reproducible of course).


I haven't seen any cases in which adding a global build flag to turn off bounds checks would actually improve performance. People overfocus on bounds checks because they're an easy thing to understand, but they're just one of many performance issues when creating a new language, and not a particularly interesting one at that since APIs, compiler optimizations, and branch predictors all conspire to drive their overhead down to near-zero (in my experience).


Sure, but I think that having one global flag to remove bound checks is useful as it allows you to measure this bounds checks performance impact easily. And then when you know how long it takes to find an index out-of-bound error and you see that the removal performance improve is small.. Well, you remove the flag.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: