
Closed Source Engines Are a Big Risk - ingve
http://stephaniehurlburt.com/blog/2016/9/16/closed-source-engines-are-a-big-risk
======
flipgimble
I fully agree with the author about the risk of Unity as a closed source
engine.

Unity isn't just closed source, but your main access to the engine is through
an ancient Mono .NET environment that hasn't been updated in years due to
licensing issues. The latest re-licensing of Xamarin should help here, but as
with many other features Unity will be afraid to make upgrades for fear of
braking compatibility.

This means that most of the Unity game code that will set your game apart has
to be written in C# 3.0, and your memory use and performance is heavily taxed,
unless you resort to platform specific native plugins. As a game developer
this scales down your ambition down to mostly casual games. The type of AAA
games that Unity advertises are mostly distinguished by large amount of
content, but nothing innovative in gameplay or technology besides what the
engine provides. I can't imagine anyone starting a No Man's Sky or even a next
Minecraft in Unity.

This is sad, because about 5 years ago I was cheering for Unity and having a
lot of fun with some prototypes. They know they are a mill for casual clone
games too, since they are shifting their business focus to online developer
services like advertising and multiplayer hosting with per-player fees.

~~~
douche
Microsoft really ought to buy Unity. Selfishly, it might prod them along to
expand their .Net Core (or whatever they are calling it this week) offerings
further.

XNA used to sit in this space, and it was a pretty nifty little game
development platform for C#. Monogame has carried on, and expanded the target
platforms nicely, but hasn't really made many other improvements, from what
I've seen, getting back into it after a few years away.

~~~
Fr0styMatt88
It's unfortunate but Unity is a prime example of what technical debt can do to
a product it seems.

[https://feedback.unity3d.com/suggestions/change-font-size-
in...](https://feedback.unity3d.com/suggestions/change-font-size-in-unity-
editor)

[http://forum.unity3d.com/threads/when-will-unity-support-
hig...](http://forum.unity3d.com/threads/when-will-unity-support-high-dpi-on-
windows.424958/)

~~~
izym
I'm not quite sure how those two threads indicate technical debt. Could you
elaborate?

~~~
loup-vaillant
First thread is asking for a simple fix ("change font size" feature), that
would really improve the usability of the editor. It has not been done for 7
years, which suggests that it's not an easy fix at all. It should be.

Second threads ask for support for high DPI. It's a simple scale issue, but
somehow hard to do it on Windows.

Stuff that should be easy but isn't is a strong indicator of technical debt.

~~~
user5994461
> Stuff that should be easy but isn't is a strong indicator of technical debt.

It can also be an indicator that there is noone working on it. It's common to
have fix which are relatively easy but noone to implement them.

~~~
loup-vaillant
Doesn't apply to obvious wins such as variable font sizes or high DPI. Those
items are necessarily high priority, because of their low hanging fruit
nature.

Or, they just get their priorities wrong, meaning, they're being stupid. Not a
safe assumption in my opinion.

~~~
user5994461
font sizes and high DPI are far from trivial changes.

> Or, they just get their priorities wrong

Most open source projects have very few or no people working at all. It's
common to find bug reports opened for years that were never fixed. They don't
have to be complex, just non-trivial.

------
TazeTSchnitzel
They are, and it's why I've vowed never to use them again.

I've contributed to Gang Garrison 2 (GG2), an open-source 2D multiplayer
shooter (a "demake" of Team Fortress 2) for several years.

GG2 is built with Game Maker 8.0, an obsolete commercial game engine/RAD IDE
product. Why? Because Game Maker 8.1 is buggy and broke things we needed, and
its successor, GameMaker: Studio, broke even more (and is also much more
expensive). We did try to move to GM8.1, but we had to give up on it.

This is unfortunate, because Game Maker 8.0, being now completely unsupported,
does not even have working DRM servers. We literally have to pirate the
product in order to continue development.

Game Maker 8.0 also produces executables that crash on recent versions of
Windows when too many sounds are played simultaneously. This is probably fixed
in newer versions, but we can't use them. So we have to work around it.

And “black box” bugs? This is a consistent theme of development with Game
Maker. It's filled with them. There are lots of workarounds in our source code
for such bugs. And there are some bugs we have never been able to work around,
so sometimes the game crashes with inexplicable (i.e. incorrect) error
messages.

Because of these experiences, any game I make in future will use an open-
source engine. Proprietary engines are just too much of a risk.

~~~
user5994461
> [Proprietary] engines are just too much of a risk.

ALL software are at risk of being poorly maintained and abandoned.

open-source VS free VS closed-source VS paid is not directly relevant to the
direction and the future of a software.

What you need to do is a proper risk analysis before you pick an engine (or
any important dependency for the matter).

~~~
Cpoll
But OS means you can fix the problem yourself instead of being at the mercy of
the maintainer. We can use the parent post as an example:

> Game Maker 8.0 also produces executables that crash on recent versions of
> Windows when too many sounds are played simultaneously.

If GameMaker were open-source, they could fix this. They could probably even
backport a fix from 8.1 or Studio.

------
tedajax
This has been my experience with Unity as well. You can spin something up
super fast but actually tightening the screws and getting every little last
bug out and getting all the little interactions between gameobjects working
just right takes a really long time simply due to quirks and bugs that require
odd workarounds.

Unity is a wonderful prototyping tool but using it for production can be
painful. That said I build engines for work so in my free time I like to just
make games and using Unity for that purpose is refreshing and fun because the
games I make in my spare time are simple enough that they rarely hit any
limitations of the engine.

------
buzzybee
The focus of a lot of engine dev is shifting away from AAA. For the AAA model,
everything was about heavily optimizing the runtime performance of mostly-
static scenes towards large teams delivering complex assets. The engineering
team has skills and depth - enough to support every release with custom
technology work. But new projects are often much smaller in scope, and that
has a consequence for all the technology.

With smaller projects, pressure moves back towards development cost, ease of
deployment and maintenance, which is a lot more like the model for small-site
webdev. The necessary optimization for the runtime is more often at the level
of just using correct algorithms, good-enough memory management, and feeding
the GPU correctly. In turn, this is going to have a commodifying effect on
engine runtimes - if your scene is already easily hitting its quality target
at 144hz or whatever the high end is today, your concerns are just going to
shift towards how fast you can iterate and debug, add and tune new assets,
create one-off behaviors, and run off final builds for external use. You can
hire for design rather than coding talent.

This favors a "game creation system" model like Ren'Py or Puzzlescript, where
the underlying formats and protocols still allow a substantial degree of
customization, including the possibility of moving the data to a different
runtime altogether, but the developer's interface works on abstractions all
tuned around that particular type of game. This path follows what's happened
to audio with the development of synthesizers; after the early modular
synthesizers proved the concept, a commercial market arose for delivering
"complete instruments" that are less flexible in abstract, but are more
portable, and offer an easier mix of performance and programmability, through
presets and limited customization parameters. And today, for every popular
genre, you can get started by "buying the sound" in the form of sample loops
and synth presets, and then build out your own work by hanging off
customizations. New genres appear simultaneously with the successful
commercialization of new technologies - e.g. drum machines, or Autotune.
There's no reason why games won't go in the same direction, given enough time.

------
GuiA
More generally, not having access to the source of the components most
critical to your business as a tech company is always a giant risk and should
be avoided as much as possible, if not at all costs.

------
vvanders
If you're using an engine you should be negotiating source access. I've never
worked on any title where we didn't have complete source for both the editor
and runtime.

Also when you evaluate engines what is there _today_ is what you'll ship with,
don't plan on features down the road. A lot of studios learned that one the
hard way with UE3 and PS3.

Lastly two devs seems like a really small number for even more than the most
simple of indie games. For a lot of the reasons that seem implied in the
article. Every game I've worked on needed to retrofit a different engine in
significant ways unless it was a direct clone of the platform/genre it came
from.

------
immigrantsheep
I'm a bit confused about this article. One one hand Unity is criticized
because it's a black box. The project was complicated, early days of VR and so
on. "It was not fun. It was well worth the high rates we charged. It didn't
help that we're still in the early days of VR, I'm sure."

On the other hand they did a very small project that was simple and for that
they built their own engine and that was a lot of fun. "Building our own
actually turned out to be a joyful experience. It was a simple app, didn't
require a lot of features. It was an unexpected, fun breath of fresh air."

While I see how building something small and simple can really be a joyful
experience, building something huge and complex for a client and supporting VR
in this day and age is probably far from that.

Am I reading it wrong or is this apples and oranges?

And while I agree that having the source for the game engine is always
preferable, often reading the comments (usually on facebook groups) makes it
sound like EVERYONE has 20 years of experience in C++ and could actually fix
everything as long as the source was provided.

~~~
heisenbit
Engines tend to orchestrate the flow of control. This breaks your app into
small pieces that are easy to reason about. When things don't work however you
are helpless as there is a black box in the center of your application with
black internal state.

Having the sources is often useful as it helps you to * partition the black
box * observe the internal state * observe critical control flows * maybe not
100% diagnose and fix but mitigate an issue

Generally engines and frameworks are bigger than needed. They may be well
tested but the sheer volume means there are a lot of corner case defects of
holes left. Frameworks are particularly risky as they are supposed to be
adapted to new situations and as a rule writing software for future
requirements is fraught with problems.

Special purpose frameworks often beat general frameworks hands down in size,
performance and cost. Going with a large established framework from a
"commercial" vendor is often seen as a safe choice by management and as the
article shows it is not. Then on the other hand there have been plenty of
teams going down naively the road of custom engines and finding that they did
not have to re-invent the wheel but build an advanced gasoline engine and were
failing at that. As always understanding your requirements, options and
capabilities priceless.

~~~
immigrantsheep
My point was actually something else. On one hand it seems they've been
struggling with Unity on a big project (where source might have helped) and on
the other she said how they made this little app with their own little engine.

If you're building something simple and small It's easy-ish to build
everything yourself. While if you're doing something big, the cost/benefit of
doing it from scratch might be something else entirely.

My point was that for the second little project Unity might have worked
brilliantly anyway. Or godot, or maybe any little 3D engine. While doing the
first thing for a client, with VR support and God knows what might have been
really really difficult.

Not sure if I made my point any clearer at this point really :D

~~~
loup-vaillant
So, you need a more relevant example, right?

I think _The Witness_ might do it: it's a relatively big game, build with a
custom engine. And it's quite clear it couldn't have been build with a classic
engine such as Unity though telling why would be a bit spoiley. Likewise,
Braid couldn't have been made with a generic 2D engine: its time mechanics are
just too alien.

Another reason why Jonathan Blow chose not to use any 3rd party engine is, he
wants his games to last. He wants the ability to port them to new platforms,
and that ultimately means owning the code —if only to separate clearly the
platform dependent parts from the rest. (I hope he'll release the code at some
point, so others can continue that work.)

Being able to fix issues was also a reason. When you not only have the source
code, but also have _written_ the damn engine, you can fix anything.

~~~
immigrantsheep
No, you missed my point. I'm perfectly clear why Jonathan Blow made it's own
engine and why anyone who's capable of that would want the same thing. I was
talking about the article and about some members of the game community and for
that I have two criticisms

First: in the article she says that they struggled on one very complex project
while using Unity. And then she says (or that's my understanding) that she
made a completely different project on her own engine. That's why I said
apples and oranges. I can't build unity, u4 or frostibe from scratch but if
you want Pong or a simple platformer, sure, I can do that from nothing. That
would be me saying: "well I had serious problems recreating the Witness in
Unity but look at my space invaders which i made in Elm for browser without an
engine".

The second problem I have is with some members of the gamedev community. Feel
like a lot of people are ready to jump on the "I wish the source was
available" train while very few of them (or at least that's my perception)
know what to do with the engine source. Actually I could extend this point to
a lot of advocate of the open source community. There is this false perception
that anyone can fix anything that's broken because the source is available and
yet a lot of those apps are sadly still very much broken and sometimes the
authors of those apps have difficulties in finding people who might help them.
But if you read any post about it online it's usually "closed source sucks and
open source is brilliant cause we can always fix that anytime".

Sorry, this is a lot of generalization and obviously not everyone is like that
but I tried to make point clearer this time 'this all.

~~~
loup-vaillant
Okay.

About the second problem, I think it is one of control. Source code access at
least gives you a fighting chance.

That said, I have used Qt a couple times, and never dared to delve in its
source code, it's just too big. If I have a problem, I just work around it. If
you want control over your engine/framework, you pretty much have to have
written it in the first place.

~~~
immigrantsheep
Here I agree very much with you. With stuff so complex I can't imagine going
through all that code. That's why I don't get people saying "if only I had the
this or that source code".

~~~
loup-vaillant
Well, there _is_ a middle ground: when you haven't written that big
framework/engine, but _could_ have, delving into the source code in an as-
needed basis may sound like a decent option.

Of course, you're hoping your framework is fit enough for your purpose, so
that you rarely have to inspect it —if at all.

------
qwertyuiop924
Well, that's why Unreal is starting to get back into it. It's also why id
should be licensing their engines, and open sourcing them.

Bethesda/id/Zenimax, are you just afraid of money, success, graditude, and
forming long-lived communities around your games that can mod and hack them,
to keep the games feeling fresh, and thus people buying them, decades into the
future?

~~~
greyskull
This comment comes up often in the gaming community. My guess: they've done
the math and the business gains do not outweigh the inputs. Between the
codebase sanitation and the licensing issues with third-party products, that
would be a pretty massive headache. There's the intellectual property argument
to make as well.

For modding specifically, that also dictates how your game is built and
requires extra work to do it right. Not every game needs it. The reality also
is that it's easier to sell people games on a shorter cycle when the previous
one doesn't have legs. I imagine the vast majority of the gaming population
don't think twice about modability - they're happy playing a focused game and
eventually leaving it behind.

~~~
Fr0styMatt88
To be honest I'm not sure that the game engines market isn't already
saturated, raising the bar even further for a big AAA newcomer.

Like others have said it isn't just about 'opening your engine' and leaving it
at that. All of a sudden you have to polish up your editor for external
consumption (ooops.... the editor is crashing on this PC config that we've
never had in the office?), write documentation and tutorials, putting out
really good examples, engaging with this new type of community, etc.

Is there honestly room in the market for an engine without a clear niche these
days? I mean you seem to have Unity and UE4 at the top tiers, then engines
carving out a niche (Godot comes to mind), your lower-level stuff (how's OGRE
doing these days?). There's a few random ones on Steam even (Leadwerks,
GameMaker Studio, RPG Maker, etc) for other use-cases.

I think of CryEngine 3 -- that's the most recent move by a big player into the
engines-for-everyone space. How is it doing these days? My impression is that
it's struggling to get mindshare, but it has been awhile since I've looked.
I'd be wary of bringing a new engine into the space after watching how that's
going for them.

~~~
Pfhreak
> I think of CryEngine 3 -- that's the most recent move by a big player into
> the engines-for-everyone space.

There's also Lumberyard, Amazon's new entry into the game engine market:
[https://aws.amazon.com/lumberyard/](https://aws.amazon.com/lumberyard/)

It's not Open Source, but you do get the engine and all the source for free.
It's built on technology from CryEngine, Double Helix, AWS, Twitch, with a
significant number of bugfixes, improvements, and features.

I'm genuinely excited to see where it goes, but, in full disclosure, I work
for Amazon Games Studios (though not on Lumberyard).

~~~
Drdrdrq
"Q. Is Lumberyard “open source”? No. We make the source code available to
enable you to fully customize your game, but your rights are limited by the
Lumberyard Service Terms. For example, you may not publicly release the
Lumberyard engine source code, or use it to release your own game engine."

[https://aws.amazon.com/lumberyard/faq/](https://aws.amazon.com/lumberyard/faq/)

------
floatboth
[https://godotengine.org](https://godotengine.org) anyone?

~~~
cyborgx7
Been using this recently. It doesn't seem as incredibly simple as it sounds
using Unity is, from how people talk (never used it myself), but it's pretty
fun and has a lot of potential. I'm kinda hoping it will eventually manage to
replace unity as the go to indie game engine.

------
userbinator
I thought it was about ECUs, automotive systems, and that other sort of
engine. The title is equally suitable.

With regards to debugging software, I think being able to do so effectively
even without source is an important and rather rare skill. And sometimes, even
with the source, the compiler can introduce its own issues. Being able to read
Asm and follow along with what the CPU is actually doing is exceptionally
useful, especially when working with lower-level languages like C/C++. One of
my friends has a saying along the lines of "if you can read Asm, software has
no barriers to understanding."

~~~
pandaman
There is "reading asm" and there is "debugging millions LOC compiled with an
optimizing compiler". The later is possible but the time requirements would be
just impractical for game development. Even having the source without the
ability to modify is almost useless.

~~~
Hydraulix989
If he confused the definition of "engine," he's likely coming from the auto
world, where things are (_mostly_, not everyone drives $100k Elon Musk
mobiles) still microcontrollers with kilobytes of flash and RtOSes. There,
it's still entirely possible to debug execution interleavings and the like by
just staring at the ASM.

~~~
wolrah
I think your information is a bit behind.

My old car, a 2002 BMW 325i, fits that description with a 25 MHz 16 bit
Infineon C167 processor and 512kb of flash in its Siemens MS43 ECU. That's
properly a microcontroller in my mind.

My new car, a 2015 Ford Fiesta ST, has a 133MHz 32 bit Infineon TriCore
processor and 4MB of flash in its Bosch MED17 ECU. That's more than many PCs
had until the late 90s, so while it's definitely running a RTOS I have trouble
calling it a microcontroller.

That's not even a new ECU either, a friend's '09 VW GTI had basically the same
unit. Anything with direct injection almost definitely has a 32 bit ECU,
especially if a turbo is involved. I'd imagine a higher end car these days has
something more potent (or just doubles them up, like BMW used to do on V12s).

------
frik
What is the current situation with Unreal and CryEngine? (both are at least
state-of-the-art)

------
bartread
Yeees.... but, in Unity's defence, for an indie dev I can see the appeal
because it allows you to target all of the platforms (not actually an
exaggeration).

DISCLAIMER: I _don 't_ use Unity because, at the time I started getting back
into game dev, I think you still needed a plugin to run in browser whereas I
wanted to target purely HTML5, JavaScript and CSS for maximum device
compatibility (hah!). That may no longer be true.

The thing is, I could make the same point as this post but about developing
cross-browser games with standardised web technologies. You can waste days at
a time fixing behavioural quirks that appear in only one particular browser -
I've done exactly that fixing bugs in iOS Safari, whereas I don't even bother
supporting Edge in Windows 10 Phone (yet) because very few people use it.

I've had a full screen bug in iOS appear multiple times in both of the games
I'm currently working on - a "nearly done" version of Star Castle
([https://arcade.ly/games/starcastle/](https://arcade.ly/games/starcastle/)),
and a very alpha-ish take on Asteroids
([https://arcade.ly/games/asteroids/](https://arcade.ly/games/asteroids/) \-
takes a while to load due to graphics pre-generation, which I need to sort
out) - due to very minor changes in markup and CSS. They largely share both an
engine, along with many front-end components, so issues like that tend to
affect both games.

Actually, my "engine" really isn't an engine at all: more an assorted set of
libraries of useful functionality that can be bolted together to help build
games of a certain type.

Back to the point, some browsers are open source but it really doesn't help
because I can't (easily) change what's deployed on millions of peoples'
computers around the world. Having access to the source _might_ make it
quicker/easier for me to come up with a workaround. I doubt it though because,
for example, Firefox has a _lot_ of source code, which it would take weeks to
understand before you could figure out why it was behaving oddly in a
particular case. It would just be easier to create a workaround. (Of course,
it's fragile - the next release of FF might break your workaround, and then
you have to have a workaround for the workaround for that release and later.)

I could make the same points about building on top of any platform, open
source or otherwise. I've spent a lot of years professionally doing .NET and
Java development. With .NET I spent years building desktop tools for
developers and DBAs, and we couldn't redistribute a custom runtime so, again,
you have to put in workarounds sometimes.

I'd still say the leg up these platforms gave us was worth the hassle of
having to deal with some bizarre pieces of behaviour (and we did occasionally
encounter some really nasty issues, like race conditions involving database
connections, and suchlike).

With web development on an OSS platform and framework I can see it might be
different. You have complete control over your deployment environment, so you
can put the fix directly in the Fx and submit a patch, if that's the best way
to deal with it.

Likewise, if you're distributing the runtime yourself you have control and can
apply the fix in its code rather than a workaround in yours. But whilst it's
easy to say that the reality probably wouldn't be, and it might cause
headaches in the future.

The costs of building your own engine are pretty clear:

\- You start from nothing, so it's harder to make progress initially,

\- If you want to hire people well, there's a pool of Unity developers, but
nobody with experience of your engine,

\- Say you start developing for PC but want to release a PS4 version of your
game: adding support for more platforms is going to be a _lot_ of work (I
don't just mean dev: Unity is tested across all platforms they support, but
now _you_ have to do that for your engine),

\- When your game engine gets large and complex it's going to start exhibiting
hard to debug behavioural quirks, just like everybody else's; you might be
able to figure out why more easily, because you have the source, but that
doesn't mean the fixes will always be easy.

So, yeah, closed source game engines are a risk, but then so are open source
engines, as well as engines you create yourself: they're just a different set
of risks. Which you choose should be defined by the objectives of your
project(s).

~~~
moron4hire
Unity's WebGL support has, in my experience, largely been in name only. It's
just such a horrifically bad experience that I just can't imagine anyone
taking it seriously.

~~~
daredevildave
Especially if you want to support mobile:
[http://blog.playcanvas.com/playcanvas-versus-unity-
webgl/](http://blog.playcanvas.com/playcanvas-versus-unity-webgl/)

~~~
bartread
That result doesn't surprise me too much - it's another reason I chose not to
go with unity. The total JS payload for Star Citadel is still <100K, which
makes a huge difference when you're talking about hitting the site from a 3G
connection. The total page size runs into the megabytes but this can be
attributed to two factors:

\- Music and sound effects, which are queued up and loaded in the background
(it's even possible to play the game with missing music and SFX, and you'll
start to hear them once they've loaded).

\- Google Adsense content. This ticks me off because some of the ads are
_very_ poorly behaved. Some load relatively large amounts of content, and can
often block page loading. I've therefore pushed the Adsense script itself
right to the bottom of the page so it doesn't result in other requests from
the game being blocked. I'd vastly prefer it if Adsense restricted ads to
plain text or simple images, banning active content and Flash (which, at least
up until recently, was still appearing). The reason for the ads is I need to
cover the costs of the site, never mind actually earn a living, but I want to
do so in a way that doesn't detract from the player experience.

~~~
moron4hire
Stripe integration for a few dollars "support the developer if you can" is
really easy to setup and I've gotten a lot more money out of it (tens of
dollars) then I've ever gotten out of advertisements.

~~~
bartread
Interesting - thanks, I'll have a look.

------
known
Closed source is organized syndicate;

------
spullara
The Pro subscription from Unity gives you source code access.

~~~
binarycrusader
No it does not, at least not to the full engine and editor. Source access is
separate and the price is unlisted, although rumours are that it's six
figures:

[https://store.unity.com/products/unity-
pro](https://store.unity.com/products/unity-pro)

~~~
doyouevensunbro
Source is $40k up to 50 seats, and then it jumps to $100k. It's pretty
obscene. The worst is that Unity will fight you if you say you need the
source, like it's a crazy request.

~~~
user5994461
Think how much are the salaries to fill 50 seats => $40k is a drop in the sea.

I can understand that this pricing model doesn't appeal to amateurs and micro
teams working in their free time. I can also understand that Unity doesn't
care about this [non] customer base.

------
newobj
Shipping a game with Unity is hard/risky? News flash: shipping any game is
hard/risky.

Source: shipped games, all had last minute hard to find problems. In-house
engines every time.

