
My RustConf 2018 Closing Keynote - AlexeyBrin
https://kyren.github.io/2018/09/14/rustconf-talk.html
======
spenrose
Wonderful inductive reasoning from why object oriented programming of a game
is challenging in Rust to some larger lessons about managing large type
systems. I felt the discussion of crossing the barrier to Lua hinted towards
some larger truths about types in distributed systems.

------
ufo
Could someone help me understand how "generational indexes" differ from usual
indexes? Does this have to do with how you allocate the indexes or with how
you use them? Are there run-time tests to check if the index is stale? (For
example, check if the generation of the index I am holding matches the
generation of the object stored in the entity array).

~~~
steveklabnik
I released a crate that implements a doubly linked list this way.

Here's an index. As you can see, it's got an index, but also a generation
[https://github.com/steveklabnik/indexlist/blob/master/src/li...](https://github.com/steveklabnik/indexlist/blob/master/src/lib.rs#L174-L178)

The list itself keeps track of the current generation:
[https://github.com/steveklabnik/indexlist/blob/master/src/li...](https://github.com/steveklabnik/indexlist/blob/master/src/lib.rs#L104)

When an entry is removed, this is increased:
[https://github.com/steveklabnik/indexlist/blob/master/src/li...](https://github.com/steveklabnik/indexlist/blob/master/src/lib.rs#L680-L682)

Each entry also keeps track of the generation it was inserted with:
[https://github.com/steveklabnik/indexlist/blob/master/src/li...](https://github.com/steveklabnik/indexlist/blob/master/src/lib.rs#L119)

When you call get, which takes an Index, the generation is checked
[https://github.com/steveklabnik/indexlist/blob/master/src/li...](https://github.com/steveklabnik/indexlist/blob/master/src/lib.rs#L577-L582)

Same as if you try to delete something at a given index:
[https://github.com/steveklabnik/indexlist/blob/master/src/li...](https://github.com/steveklabnik/indexlist/blob/master/src/lib.rs#L661)

Does that make sense?

~~~
ufo
Thanks. It wasn't as clear to me in the article that there was a run-time
check but it makes much more sense now.

~~~
steveklabnik
You're welcome.

One interesting property of the current implementation is that you can use an
Index across multiple IndexLists. That may or may not be what you want.
There's a way to add a compile-time check to prevent this, but I haven't
implemented it yet
([https://github.com/steveklabnik/indexlist/issues/7](https://github.com/steveklabnik/indexlist/issues/7))

------
setr
I can understand the fast lookup, but if I understand correctly, every object
in the game is an entity, and every entity is represented in _every_ vector?
So if you have a hundred game properties, an object that only utilizes three
properties still has to be allocated a hundred times over?

Also I'm not sure how a system is meant to recognize which entities to do its
work on, without iterating over the _entire_ associated vector, every frame?

It seems like there's a lot of waste in this model

I'm also not sure why struct of arrays is better than an array of structs,
where each struct implements each component as a trait (but still relies on
the system to do the work), at least logically. It would definitely be more
memory efficient if there's a high number of components that are rarely used,
if I understand things correctly

~~~
djur
The talk presents a fairly simplified version of what game engines actually
do.

Components that are only used by a minority of entities can be kept in a map.
The talk refers to `specs`, which is an ECS system implemented in Rust. It
allows you to use different backends (a vector, a couple types of map, etc.)
and it uses bitsets to optimize determining which entities have which
components: [https://slide-rs.github.io/specs/05_storages.html](https://slide-
rs.github.io/specs/05_storages.html)

As to why "struct of arrays" might be preferred to "array of structs", it's
all about locality of data. Game Programming Patterns explains it way better
than I possibly could: [http://gameprogrammingpatterns.com/data-
locality.html](http://gameprogrammingpatterns.com/data-locality.html)

------
jimbo1qaz
Are the arrays untyped? If all enemies are the the same struct with various
data components (state, speed, etc.), how would you implement different
behaviors based on enemy _type_? Function pointers/references? Does the game
loop have a 20-enemy-long switch statement which calls 20 different functions
based on which enemy _type_ is present? Does the game store 20 optional
behavior components per enemy, and run 20 different if-component-present tests
_per frame per enemy_?

~~~
annywhey
In short: sum types.

If behavior is controlled from the top level(as is suggested by the cases
where the main loop grows to 10's of thousands of lines) your behavior
branching also appears there. The data is still driving everything, but if the
data model is inherently complex it bubbles up to the surface. And you can use
sum types(whether provided by the language or in less structured "match a
magic constant" type deals) to check your coverage. During prototyping, you
can lean on dynamic types instead and get a similar outcome with more fluidity
in model changes and lower debuggability.

That feels instinctually wrong if you're dosed up on polymorphism and expect a
kind of "ultimate abstraction" where any of the enemy types could be totally
different and you just write to an interface, but - the thing you want, and
that is the practical situation in most gameplay logic, is not "wholly bespoke
behavior per type of enemy, hide everything behind a facade and either
duplicate code or use a complex indirection to reuse it where it's the same"
but "98% of the behavior is on the same code path and reads top to bottom,
except for the one or two important pieces of business logic that isn't". The
latter is both more compact in total SLOC and by being surgically precise,
gives more visibility into what the code does at the moment you try to debug
or extend it - which is what we're really aiming for.

And if you have really complex behaviors, you do end up with Turing-complete
script logic embedded in the data, just naturally growing out of data model
extension to cover FSM logic. That indicates that you are never really "giving
up power" at any point in this process - you're just gradually moving things
up the abstraction chain in a controlled fashion, until your main loop is
really an interpreter that deals with "opcodes" and "commands" more than it
does any particular algorithm or data layout.

And then when you get there, you can go, "well, now that I have a virtual
machine, let's optimize around that..." and then you start writing some tech
to lower the overhead of the interpreter...and it just goes from there. You
never have to jump off that train, but you can usually ship something well
before you get it even to the Turing-complete level of programmability, too.

------
k__
I have to say, Starbound was a nightmare perormance-wise...

------
dang
[https://news.ycombinator.com/item?id=17977906](https://news.ycombinator.com/item?id=17977906)

~~~
meheleventyone
I don’t think this is a dupe as this is a text transcript with a lot of
additional detail and commentary.

~~~
dang
Ok, we've unduped it.

~~~
meheleventyone
Awesome, thanks a lot.

------
Ericson2314
This is really good for comparing different eras if game design. But damn,
when will the gaming community realize they need something like GC?

~~~
Ericson2314
The SQL bit is also good. Basically somebody needs to bring capability
ideology (highly related to magic pointer types in Rust, btw) to relational
data organization to limit access patterns (no arbitrary joins) which will
both allow for more modularity and perhaps distributed DBs that actually
scale.

~~~
adamch
Your suggestion reminds me of Eve [1]. I remember when I first read the
design/explainers, the closest analogy I could make was "all variables are
backed by a database, and all manipulations are SQL". Not a very accurate
statement, but it's how I wrapped my head around it.

[1] [http://witheve.com/](http://witheve.com/)

