
Using Lua coroutines to create an RPG dialogue system - stevekemp
http://lua.space/gamedev/using-lua-coroutines-to-create-rpg
======
shinymark
I was the producer and an engineer on Grim Fandango Remastered, a remaster of
one of the first games to use Lua (2.5, later 3.0 alpha). And the lead engine
programmer on the original game in 1998, Bret Mogilefsky, added coroutines to
the Lua language in the game engine before they were officially added as an
official language feature. They are very important for scripting adventure
gameplay in an intuitive way for non-engineers and helped the original team
ship Grim Fandango in 1998. It was similar to what LucasArts had done with
SCUMM, their adventure game domain specific language, previously.

To reference SomeCallMeTim's comment - the way the save game system worked in
Grim Fandango was to serialize the entire Lua environment! Stack, all code,
etc! And too jamesu - when you patch the game you patch the code that was
serialized into the save game! The story behind all of this in Grim is pretty
interesting.

If anyone is interested in more details please come to the Game Developers
Conference this year as I'm giving a talk on the technical details of how the
original game and the remaster works. Oliver Franzke and I will cover both the
Grim Fandango and Day of the Tentacle remasters.

~~~
veli_joza
I'd love to hear all the gritty details about game internals, but I cannot
attend GDC (wrong continent). Please submit the talk to HN once it's
available.

------
SomeCallMeTim
I've done this kind of thing in Lua before, but what troubles me about it is
the fact that a lot of the state of your game ends up encoded in the Lua
stack.

What happens when the user wants to save? There's Pluto [1], which is somewhat
slow for complex state. And it doesn't work on LuaJIT code -- and I tend to
like to run games on LuaJIT.

And it's opaque: If you're debugging some kind of interaction in the game, you
can't just look at the save state to see what's where.

I used to be a big fan of Lua, and this kind of AI and behavior coding was a
component of that. But ... I think that, on balance, actually creating a data
format that represents the state and "coding" using that plus state machines
is actually a more robust and useful way to accomplish the same thing.

[1] [http://lua-users.org/wiki/PlutoLibrary](http://lua-
users.org/wiki/PlutoLibrary)

~~~
pcwalton
> I think that, on balance, actually creating a data format that represents
> the state and "coding" using that plus state machines is actually a more
> robust and useful way to accomplish the same thing.

Manually coding up a "state machine" is kind of reinventing the Lua
interpreter, no?

~~~
nine_k
Coding a state machine manually is indeed suboptimal.

Computers are good at generating code. If you want a very fast FSM, they will
build it for you from a compact representation (cf. a regex).

But for game dialogs speed is probably a secondary concern. I suspect that a
table-driven FSM interpreter with a fully transparent internal state that can
be saved / loaded is not very hard to create. Your dialogs' input language
becomes the control table, which is possibly no less designer-friendly than
Lua coroutines.

~~~
optionalparens
Speed is not always a secondary concern for dialog. Certainly in many games
where the dialog is very fixed, sure. However more and more, dialog systems
actually need to be pretty fast. The simplest reason is you generally have
only a few or even 1 frame to process dialog and you certainly don't want it
tying up your frame's update cycle. It's easy enough most of the time as a
place where you can shave off time with some good optimizations anyway for
simpler cases.

An example I see a lot where it needs to happen fast is spatial queries or AI-
related dialog. For instance if I walk by someone or something, that has to
trigger a query on the dialog engine for someone to say something to the
player. It's not just the dialog, but the sound assets, playback, etc. that
need to happen and they need to happen in a way that isn't annoying - i.e.
player is far away already before dialog starts or has to "wait" in place.

Sometimes as I mentioned, AI also gets thrown into the mix in different ways.
The simplest is that you want some piece of dialog to be triggered by some
other game states - ex: if player killed dragon, say this, otherwise say that.
It seems simple, but when you start throwing in lots of world states like in
open world games and want some decent variation and natural dialog that isn't
awful and repetitive, it gets harder. Now add on spatial and other
requirements for a dialog query and things start to suck. To this end, it's
why a lot of dialog systems suck. It's not just laziness, rather sometimes
people actually can't get their dialog systems to work fast enough during
update cycles and end up dropping features because of the cost vs. benefit for
most games.

Dialog seems simple, but is surprisingly complicated in the greater ecosystem
of other game systems. But again, this depends on the game and if you just
have a side-scroller where you repeat the same 5 catch phrases, then yeah,
dialog is simple and speed is not a problem.

------
dividuum
For simple dialogs that don't have to be saved (see the other discussions),
this seems like an easy way to quickly put any format of dialog system
together without reinventing your own DSL to encode those dialogs as state
machine(s). You can easily compose dialogs just by calling functions and
thanks to Luas tail call optimization you could even jump between dialogs
without the fear of running out of stack space.

I'm not saying it's the best solution but it's pretty simple and probably easy
to understand. I've used a similar approach way back in a networked
multiplayer game where players could telnet into the game server and use text
based menus to control the game. You can see the relevant Lua code here:
[https://github.com/dividuum/infon/blob/master/server.lua](https://github.com/dividuum/infon/blob/master/server.lua).
I don't want to imagine how this code would look like without the use of
coroutines.

------
hypothete
Here's a similar implementation on the PICO-8 platform, which uses a subset of
Lua:
[http://www.lexaloffle.com/bbs/?tid=3833](http://www.lexaloffle.com/bbs/?tid=3833)
This author demonstrates a multiple choice ask() function as well.

------
managun
That's interesting. One of my pet projects is a compiler for Bethesda games
quests - having source in text/plain file(s), that's the motivation - and as I
am designing the underlying language, maybe I should consider coroutines as
well...

------
optionalparens
I can see how it might be fun to do something like this for a blog post or
simple project, but in a real game this just doesn't work for many reasons,
most listed here already.

There are tons of ways of doing dialog system. Mostly it comes down to your
specific game and I highly recommend against generic dialog systems. Rather,
the most you can do at a generic level is try to come up with a good way to
store dialog and retrieve it in the ways you need. This isn't the best thing
I've seen or worked with, but here's an easy to understand and documented
method and presentation about it form Valve:

[http://www.gdcvault.com/play/1015317/AI-driven-Dynamic-
Dialo...](http://www.gdcvault.com/play/1015317/AI-driven-Dynamic-Dialog-
through)

Anyway, I've been programming games and game engines for several decades, and
coroutines are always in the list of things that you can throw in the junk
pile along with other cool things like continuations, various theoretical data
structures, fancy dispatch mechanisms, events/signals, application-wide
functional purity, etc. Of course all of these things are useful, but they
tend to have problems that make them of limited use in professional game
development.

Here's a brief checklist of show-stoppers off the top of my head I tend to go
through before introducing anything "creative" into a game, and especially a
game engine. Most of the aforementioned items fail one or more of these.

\- Performs awful in common cases or when applied to actual game-like
conditions vs. theoretical

\- Murders cache lines

\- Hard or impossible to debug

\- Hard to save, load, serialize, deserialize, or snapshot

\- Doesn't work over networks

\- Doesn't play nice with libraries or data coming from other libraries or
languages (ex: 3rd party physics lib)

\- Unpredictable execution start or end times

\- Non-determinant memory usage or allocation patterns

\- Risk of stack-overflows

\- Non-portable or problems on specific target architectures

\- Requires "religion" meaning it infects all your code or forces you to write
all of your code a certain way, everywhere

\- Fragments memory

\- Long blocking or uninterruptable processing

Conversely, besides the obvious opposites of most of the above, I look for the
following when introducing some major data structure or conceptual item like
coroutines to a game or game engine:

\- Leads to predictable, consistent code and resource usage

\- Data-driven

\- Editor friendly (as a corollary to the above)

\- Clear debugging story

\- Both simple and easy if possible. No, they are not the same.

\- Plays well with the world around it

\- Portable

\- Fast

\- Loved by the CPU, GPU, and/or compiler, and produces reasonable assembly on
target hardware

\- Easy for someone else to jump in and understand the flow and how it works,
assuming they are not a moron.

Pretty much these two lists mean that games tend to be made of meat and
potatoes things. Simple data structures like arrays, custom allocators,
predictable branching and execution patterns, up-front memory allocation, and
CPU and GPU loving goodness. Pretty much anything stashing things away and
building up massive states or jumping around all over the place, pointer
chasing, and so-on should always be a no-go.

Anyway, if you're building a truly simple game, as in a 2 day sort of affair,
do whatever you want. If you want to build something even remotely complex
that you will work on for awhile, it's best to do things the right way which
means detach yourself from everything you think you know about most of
computer science and think in terms of pipelines, CPU instructions, caches,
and predictability among other things. This often means programming against
the grain of your language a bit, throwing out sugar, std libraries, and all
kinds of things. It really sucks, but that's the essence of good game
programming. I hope one day it will be different, but as long as there's still
need to push gameplay and graphical boundaries, professional developers almost
always need to squeeze out what they can.

~~~
davexunit
>Anyway, I've been programming games and game engines for several decades, and
coroutines are always in the list of things that you can throw in the junk
pile along with other cool things like continuations, various theoretical data
structures, fancy dispatch mechanisms, events/signals, application-wide
functional purity, etc.

This is one of the worst outlooks on programming I've ever seen. Did you know
that any control flow operator such as try/catch is really an application of
continuations?

~~~
optionalparens
Not sure what your point is here, could you elaborate? It seems to me maybe
you are missing the point.

As I said, this is about programming games, primarily in C/C++ or a related
language (though applies mostly to others), not programming in general. If you
enter into other high-performance domains, you will encounter the same issues.
Conversely, other domains such as general application programming do not
adhere to this.

Regarding try/catch, I am well aware of low-level implementations in various
languages. Did you know that in most game engines, you don't use that much
try/catch for this exact reason? If you're littering your game code with
try/catch everywhere, you are doing something wrong. Not to say you don't have
any, but there are other error handling and return methods or you just let
things bubble-up and handle them there. What would you actually do in most
try/catch situations deep inside a game loop outside of things where resources
can leak horribly like IO?

When you're working on a game, there are of course areas you can compromise.
It also as I mentioned depends on the nature of your game. I'm speaking
strictly of properly designed, professional games that have to be competitive
in terms of features and run at a smooth frame rate while doing many things
(i.e. your average AAA console game).

In case you still doubt what I am saying, go look at GDC talks, game dev
papers, etc. and you'll see the same thing. As far as someone people around
here might be more familiar with as a source, I seem to recall there were
multiple times Jonathan Blow mentioning exactly what I did (maybe a talk he
gave at Berkley?). Anyway, the point is that as programmers we learn all these
cool and fascinating things, but in game dev, you tend to have to let go of
"cool" things and spin your thinking a bit towards things like predictability,
performance, stability, transparency, and composeability.

------
Pica_soO
Im wondering, could one auto-convert JavaScript- librarys to lua-code?

~~~
etiene
There are some JS to Lua transpilers around, but I'm not sure how robust they
are. I want to try some of them later to try to convert some handy JS
libraries and feed them into LuaRocks.

~~~
Pica_soO
The Git-hub link, please?

------
EekSnakePond
Robust RPG dialogue and trigger system for JavaScript:
[https://github.com/codeotter/thusspokenpc](https://github.com/codeotter/thusspokenpc)

