
Porting the Unity Editor to Linux: Stuff I Wish We’d Done Then - mariuz
http://natoshabard.com/post/122670082502/porting-the-unity-editor-to-linux-stuff-i-wish
======
wtbob
Wow, I wouldn't have thought that folks would be sloppy about case, but I
guess it makes sense if all one's used to is a case-insensitive system.

> Same policy as with our runtime; in order to keep our own sanity, we will
> officially support Ubuntu Linux.

…

> Installer will (most likely – it’s one of the things we didn’t do yet) just
> be a .deb package.

I really wish folks would support _Debian_ first, and let Ubuntu support flow
from that.

~~~
jebblue
I agree in both cases. There would be no Ubuntu today without Debian (although
I've never installed it because I tried years ago and found it to be too
difficult which is why I went with Ubuntu after becoming less than enchanted
with RedHat) and really I thought case issues were a thing of the 90's, early
2000's at the latest. I've read that Debian has gotten easier to install
though but I'm not sure from their web site which one. They need to get
someone with a marketing background to help them with their web site.

~~~
natosha_bard
I can totally sympathize, and we'd like to support every distribution. :-( But
the simple statistics show that we get the most bang for our buck by
supporting Ubuntu because of its desktop market share.

The great thing about the Linux community is that it's full if crazy smart
people who will go out of their way to make sure stuff is working on their
distribution. And of course if something isn't working on your distro, it'll
be just like the player -- send us a bug report or poke us on the forums, and
we'll do what we can. But in order to not spread our resources too thin
(especially considering that this is starting off as an experiment), we have
to set some boundaries, and we need to do what will make sense for the largest
number of users.

And case sensitivity problems will exist as long as we have a mixture of some
platforms (Windows/OS X) using case-insensitive filesystems by default, and
others (Linux) using case-sensitive filesystems by default.

~~~
fnordfnordfnord
You haven't been asked to support every distribution, just Debian, and
possibly Ubuntu. Please pay attention to this advice. Supporting Debian
practically gets you Ubuntu support for free, as well as a number of other
popular distributions.

~~~
tormeh
Steam and most Linux games can be run on any distribution with a bit of work
on the user's part. The problem with officially supporting something is that
suddenly the vendor is responsible for the game not working. That shit costs
time and money. You say Debian, but sooner or later someone will demand
support for Damn Small Linux. It's better to just stop at Ubuntu and let users
of other distros install the right dependencies themselves.

~~~
fnordfnordfnord
Perhaps you're new to Linux. Let me assure you that Debian is a very popular
distribution upon which many other distributions are based, and is in no way
equivalent to suggesting support for one or a number of niche or less popular
distros. I don't even use Debian myself, but I suggest it because I know that
if Debian is supported then getting support for my preferred distribution will
be effortless; the same is true for Ubuntu.

------
evmar
I worked on the Linux version of a large program and after some years of it I
mostly concluded that it's hard to know in advance which things work
everywhere and which won't, unless you ensure your programmers are familiar
with the quirks of all the platforms you intend to run on (which isn't really
practical).

(I was working on the Windows -> Linux path, but for some cross-OS examples
that aren't that I was surprised to learn that while toupper follows your
locale on Linux, OS X uses the "C" locale by default. Or that shm_open has a
tiny size limit on OS X.)

I think the best you can do is (1) write a lot of tests; (2) whenever you add
an OS-specific snippet, use the same ifdef string so it's easy to grep for.

------
hyperion2010
> it’s the place where case sensitivity problems really show up

Who made the decision to have case insensitive file systems? If I had a time
machine they would be high on the list of people to visit and give a stern
talking to.

~~~
reitzensteinm
I don't think the problem is having case insensitive file systems, the problem
is having both on different platforms. A similar problem as with different
endians. If everything were case insensitive, it wouldn't cause issues.

While as a programmer, I prefer the level of precision case sensitivity
provides, I'm hard pressed to make the case that it is better for an average
computer user.

~~~
ploxiln
... which exact type of case insensitivity would you do? Only for latin
letters in ascii? or such letters plus just one accent type? or also less
familiar symbols used in other writing systems? Did you know that different
languages can contradict one another? [1] Are you also going to normalize
unicode representation, like Mac OS X does? (it uses a _variant_ of NFD
normalization, which is arguably the wrong one to do but that's life) (I only
know this because I've been tripped up by it)

Basically, you can't "just do case insensitive everywhere", that doesn't fix
the confusion and inconsistency.

[1]
[http://www.i18nguy.com/unicode/turkish-i18n.html](http://www.i18nguy.com/unicode/turkish-i18n.html)
(see table titled "English vs. Turkish Case Mappings") (this is just the first
example I could find with a quick search)

~~~
reitzensteinm
Point taken; I made the mistake of writing a prescription for the world based
on my narrow English speaking world view.

I stand by what I said for ASCII, but in a Unicode world, I think you're
right. When the mapping stops being simple, the principle of least surprise
reverses in the favor of case sensitivity.

------
jebblue
I gave Steam (should be Valve) credit for their work to bring games to Linux,
I should have mentioned Unity. Some of the best games I've played in the past
2 years were based on Unity. The Unreal platform brings new promises and will
be nice to see games unfold from those on Linux. My money is good for Linux
games for my lifetime which hopefully is still decades.

------
Superpelican
Having switched to OS X about a year ago after years of Linux usage, I can say
that personally I really like the case insensitivity OS X has by default. It
doesn't whine every time I accidentally (don't) press caps lock. Case
sensitive file systems must be the dumbest invention ever. Who is seriously
going to distinguish his/her files by capital and non-capital letters?
"Test.txt" and "test.txt" should be the same!

~~~
4ydx
This only applies to ASCII (at best). Case insensitivity is very annoying.

------
mkozlows
So this is framed as a "things we should have done back then", but I think
that the part about case-sensitive filenames show why that wouldn't work:

"Of course we tried to be smart in the early days, but if you don’t set up a
way to actually verify that what you’re doing works on a case-sensitive file
system, then it will never fail that some well-intentioned programmer throws a
toLower() in somewhere and ruins the party."

Because yeah, if nobody's running or testing a code path, it will absolutely
decay. You can't write anticipatory code to work for hypothetical future
requirements that you don't necessarily understand and are never testing. ﻿

~~~
natosha_bard
Right, it's in the category of "things I wish we would have done back then"
because it would have been easy to solve: we could have simply had everyone
install OS X (which is what Unity was originally written on) on a case-
sensitive HFS+ filesystem, but we didn't do that. Ooops. Oh well. :-)

------
GFK_of_xmaspast
"The C++11 standard is, ahem, vague in some places and different compilers
choose to interpret the standard, well, differently."

I'm a little curious to know what they were doing.

EDIT: Dammit, I should have kept reading, the very next sentence has: " Lots
of compilation errors that involve this-c++-template-thingy-with-lots-of-
angle-brackets does not match that-c++-template-thingy-with-lots-of-angle-
brackets-that-has-a-const-somewhere-in-the-middle."

and personally I would have thought that of course you need the consts to
match and it would have been this way for literal decades now.

------
ianlevesque
Gtk does sound like a better choice than raw X11, particularly as Wayland,
Mir, and friends continue to grow.

~~~
JupiterMoon
Just curious. Why not go for QT for the whole GUI on all platforms?

~~~
jebblue
It seems Gtk has won, why don't the QT developers and supporters move to the
Gtk camp and work to make it better rather then keep QT alive which just
wastes efforts.

~~~
sirtaj
Because for professional development, there's really no comparison; Qt is on a
different level from GTK entirely. I just did a keyword search on Monster
India - Qt: 69 jobs, GTK: 3 jobs.

~~~
htns
Yeah, just about the only reasons to use GTK+ in new projects are the plain C
API and slightly better bindings. Qt bindings support deteriorated a little
from Qt 4 to Qt 5 (python is there but still no java bindings as far as I can
tell), whereas GTK+ isn't expanding in features so it doesn't get to have such
problems.

------
mynegation
About 12 years ago I worked on porting pretty large C/C++ codebase from Unix
to be compiled also on Windows. What helped is that C/C++ part of the software
had no GUI components (that was handled by Java) and by that time it has
already supported Linux, Solaris, and HP-UX, which later were dropped, but we
already had some experience of multi-platform support.

Yes, file system case-sensitivity and dispersed strictly hardcoded pre-
processor directives were the problem, but they were the least of the problems
for me (every codebase is different, so I am not doubting posted article). The
differences between POSIX and WinAPI, GNU-isms in the code (Windows version
was compiled by Visual C++ 2002 at the time), and support differences between
C++ STL version were bigger problems.

My approach was to choose a C library specifically designed to support
multiple platforms - Apache Portable Runtime (I also considered QT, NSPR, and
GTK at the time). This helped tremendously and I would recommend doing the
same to anyone doing similar effort or - better yet - start using something
like that at the design stage.

------
natosha_bard
So based on all of the feedback we've gotten, I'm going to look at making a
platform-agnostic installer as well as a .deb package, for people who want to
run Unity on non-Ubuntu installations.

We're probably going to still only offer "official" support for Ubuntu, since
that is what we build/test on, but we'll try to fix what we can when problems
are supported on other distros.

------
anon4
I still don't understand why so many are jumping on the Linux gaming
bandwagon. I mean, it's nice for me and all, I just don't really see the
business gains for them.

~~~
tormeh
Costs are slightly lower than sales, I would expect. It's also a political
issue when it comes to Humble Bundles and crowdfunding.

Costs have dropped with C++11, Unity and Steam for Linux. Sales have risen
since Linux' market share is rising (very _very_ slowly). Also of note is that
Linux users seem to have beefier setups than their Windows peers, according to
Steam Survey (12GB RAM vs 8, 2GB VRAM vs 1, bigger screens and more cores).
More money spent on hardware correlates with more money being spent on games.

------
kayoone
Looking forward to this but given the fact that Unity is not the most stable
piece of software on it's main platforms, i am still a bit skeptical.

------
white-flame
Their biggest failure is not self-hosting the editor as a Unity application
itself. Any target they support could have become an editor platform.

~~~
natosha_bard
The Editor needs to do all kinds of complex window management, and other
stuff, and also uses all sorts of 3rd party middleware (like Enlighten, for
global illumination) that wouldn't work as being part of the runtime itself.

The editor is, however, largely written in Unity (uses the same APIs, etc, in
addition to a bunch of private ones).

------
akavel
I'm scratching my head after this article, not really sure if the author
really believes in what he's complaining on fully? For me, those are kinda
nice "wishes", in that if they happened, the porting team would have less work
to do now, sure. But, actually, especially those specific things listed - they
sound to me like things, which would be unnecessarily hard and _really_ costly
things to do "in the past", assuming they were initially aiming just for
Windows (and maybe OSX, don't know it well). So, like, I feel like doing them
that way originally was actually a _very good decision_. I'm even starting to
think: if the author has _just_ those things to complain about, then maybe
actually the codebase should be described as written fairly portably and
future-proof...

To explain my argument, I'll try to elaborate a bit more on the points raised
by the author:

 _1\. Case sensitivity in paths._

So, as others mentioned too: on a case insensitive filesystem like Windows,
you'd have no chance to verify (i.e. test) "case-sensitive-correctness"
anyway. So, you'd fail on this here and there anyway. Ok, maybe using the OSX
FS with case-sensitive option would be a guard. But, OTOH, I believe OSX has
it case-insensitive by default anyway, no? So, by testing in case-sensitive,
you're actually testing the "least used path", which sounds not so smart,
actually... especially given that switching from case-sensitive to case-
insenstive (i.e. the other way) has its own surprises, then...

 _2\. No OS #ifdefs_

Um;... so... like... what else to use instead?...

 _3. "Assumptions":_

 _3.1. Compilers._

This point seems actually to be more "no C++11". OK; yeah; sure; but remember:
then you'll get maimed for being backwards, "corporate-ish", you'll get no
"smart young hackers", people will get more frustrated...

Not that you'd really have a good alternative to C++ (Java/Python need huge
and messy runtime env; and if you need performance of C++, you need
performance of C++; if you started now, you could _maybe_ at least try using
Go as a kinda compromise, but depends on your actual needs).

You know, whichever way you choose, you have pros and cons. And, supposedly,
some time ago when the decision was made, the pros were actually outweighing
the cons; so, it wouldn't sound so smart to me to choose otherwise...

 _3.2. & 3.3. "Assumptions about [GUI]"_

Sorry to say, but: GUIs are just _not portable_. _Every_ GUI environment has
different "idioms", "interface guidelines", etc. Windows has different. OSX
has different. Linux has different. Heck, different _versions_ of the OSes
have them different! On Linux, even in the same "OS version" (um, distro?
kernel?) you have multiple "desktop environments" with different widgets,
layouts, etc (KDE, Gnome, ...). Conversely, on Windows, Microsoft likes to
sometimes pull you in multiple directions at the same time too, e.g. when new
MS Office introduces new widgets, or new Explorer does the same, or you have
Metro+classic, or whatever. Dunno about OSX; I dare suppose it's not all roses
there either.

I seem to believe now, that if you want to have a cross-platform app with a
GUI, you should really write the GUI for each platform from scratch (only
trying to reuse some parts). Or, consciously break with staying consistent
with any OS interface guidelines.

As to "copy & paste": on Linux it has actually _totally different_ underlying
mechanism(s) than on Windows. So, you probably have to rewrite even low level
parts of it very differently. Not just menu entries.

So - I don't really see what's there to complain on. The app was written for
some 2 OSes; now guys port it to a third one; awesome! just, sorry, you know,
there's gonna be some work needed in order to complete that. <shrug>

~~~
natosha_bard
Hi! I'm glad to see that the post is promoting constructive dialog about what
does or doesn't make portable code, which is what the point actually was (in
addition to shedding a bit of light about the state of porting Unity to
Linux).

I'll clarify a few things:

1\. Unity was originally written for Mac OS X, then later ported to Windows
(not the other way around). Given that it's more than possible to have a case-
sensitive HFS+ filesystems (since, I think OS X Panther released in 2003?),
IMO it would have been more future proof for us to be sure to develop on a
case-sensitive filesystem. Unity users over the years have tried to run Unity
on a case-sensitive HFS+ filesystem and failed, so it would have been nice for
all of those guys. :-)

2\. Note that the point in the article was not to avoid using OS-specific
defines (that's impossible), it was to make sure to do something sane (like
#error) in the #else case that would make it immediately clear something need
to be implemented there.

3.1. Yes, the higher-level point is indeed, "Be smart/cautious about what
language features to use for the sake of portability".

3.2 & 3.3. Of course it's just a matter of opinion, but I wish that we'd
considered a more platform-agnostic approach to GUI handling when the number
of platforms that Unity was ported to went from 1 to 2. I (and others) are
still a fan of the menu-system-written-entirely-in-Unity-GUI idea, but others
(within Unity), for very good reasons, disagree. Fair enough.

Anyway, thanks for sharing your thoughts; when I write about anything I'm
always looking forward to hearing a different perspective, and one of the main
goals I had in writing the post was to promote constructive dialog about what
it means to write "portable" code. :-)

~~~
akavel
Thanks for the reply! Most of them make much more sense now. As to 1. and 2.,
I'd say you could want to consider merging those clarifications into the
original article. In 1., arguably some background knowledge of Unity could be
expected from the reader, and maybe I'm just not the intended target. But
especially in 2., I'm afraid I can't agree that those clarifications are
there, just hidden between words. There are very little words in 2. in the
article. As to 3.2 & 3.3, I don't really get the clarifications unfortunately;
I think it shows even more that I don't really have knowledge of Unity, so I'm
starting to think maybe that wasn't really an article for me to start with.
Thanks for the interesting and insightful discussion!

------
justthistime_
With .NET Core, is there any reason to not just drop IL2CPP completely?

(The whole thing only existed in the first place because they are stuck on a
completely outdated Mono version, due to Xamarin changing the license of newer
Mono versions trying to extract money from Unity.)

~~~
carussell
Can we kill off the Mono libel already?

Mono changed the license _so that it could be used in Unity to begin with_.
Mono was available under a copyleft license. To everyone. Including Unity.
Unity couldn't/wouldn't accept it under those terms, so folks at now-Xamarin
said, "We'll offer it to you under a commercial license if you won't take the
copyleft offering." That's standard fare. Unity said, "Okay, deal." Mono
continued to improve over the years. Unity was interested in those
improvements. Mono said, "Okay. You can get those under a commercial license,
too." Unity balked.

So what we're really talking about is that Unity wants the improvements that
have been made to Mono, but doesn't want to pay for them.

Get this: They're still free at any point to accept Mono under the copyleft
license that's available to _literally everyone_ , but they won't accept it
under any terms other than their own. That's it.

~~~
azakai
That seems like a very unfair description of the situation.

Do you know the specific licensing terms that Xamarin offered to Unity?
Probably none of us here do. Given that, why would you assume Unity was the
unreasonable party, and not Xamarin?

Regarding the copyleft license, while it is an option for everyone in general,
it isn't an option for Unity, which needs to ship on platforms that do not
allow such licenses (iOS, game consoles). Products like Unity will always need
a commercial license as long as such platforms exist.

~~~
carussell
> Given that, why would you assume Unity was the unreasonable party, and not
> Xamarin?

I don't know what this means. What we know of the situation is that Unity
doesn't want to pay a commercial license for Mono. This is even the way that
pro-Unity/anti-Mono folks frame the situation. What do you have in mind here?

> Regarding the copyleft license, while it is an option for everyone in
> general, it isn't an option for Unity

That's all well and good, but it doesn't have much to with any obligations
Mono has. Those are completely synthetic restrictions that the platform
vendors are placing on the people targeting them. How does Mono get the blame
for Apple and Nintendo's choices here? Why do Apple and Nintendo get treated
as bedrock? Even so, that's bedrock that Unity knows they're working with.
It's their choice to target those platforms.

EDIT:

> That seems like a very unfair description of the situation.

Unfair. Every reference to the Unity/Mono relationhip I've ever come across
has been from someone who heard Unity's creative retelling of the conflict and
parroted it exactly, down to overtly identifying Mono as the bad guy.

~~~
azakai
It means: Xamarin offered a commercial license to Unity for $X under Y terms.
We don't know what X and Y are, since this was not a public negotiation. X and
Y could be reasonable, or they could be totally unacceptable. They could be
offered in good faith, or they could not.

We just don't know the answer, and you're assuming the party offering X and Y
is reasonable. But it might not be. We don't know.

edit to your edit: if you heard people on the internet being wrong in some
way, that doesn't justify you being wrong in the opposite way.

~~~
carussell
There's no "opposite" emanating here. I'm making no assumptions beyond what we
do know, and what I describe is what we do know, sans Unity's spin. There's
only me saying, "I'm a little sick of seeing Unity's going swallowed without
critique." You characterize that as unfair somehow.

Companies A and B couldn't come to agreement to license something under terms
Y that include $X in payment. Okay. There's neutrality in that statement. My
comment says that (and no more).

What let's not do is say, "That makes company B a bunch of jerks." That _is_
what the comment I'm responding to does, and it exists in a sea of others with
the same tack.

Overton window and that.

~~~
azakai
You said

> So what we're really talking about is that Unity wants the improvements that
> have been made to Mono, but doesn't want to pay for them.

which paints Unity as the bad guy, when for all we know, it might not be.
Perhaps Unity is totally OK with paying for a license, but the Xamarin terms
would have bankrupted it.

But thank you for clarifying what you meant. I agree there is no point in
saying either party is the bad guy, since there is not enough public
information to know.

~~~
carussell
> which paints Unity as the bad guy

 _In what way at all?_ Entity A doesn't want to pay entity B $X. That's the
only thing the comment of mine that you quote says.

There are shops around that sell quadcopters for hundreds of dollars. I want a
quadcopter. I don't want to pay hundreds of dollars. Have I just painted
myself as a bad guy?

I repeat: the _only_ reason my comment sounds pro-Mono is because, at neutral,
it resists the spin of the one that paints Unity as a victim and Mono as a
fiend.

~~~
azakai
"X wants Z, but doesn't want to pay Y for Z" implies several things. Perhaps
you didn't intend them. But almost every reader will make those assumptions.

In particular, it sounds like Y is offering Z at a standard rate, as in "Sam
wants a Chipotle burrito, but doesn't want to pay Chipotle for the burrito."
Sam doesn't look good here - does Sam expect to get the burrito for free? Why
does Sam think he is entitled to that?

Readers may not realize that there isn't a standard, objective rate for
commercial licenses for Mono. Especially for the single biggest consumer of
Mono, Unity, which uses it in products that its customers ship - a
particularly complex situation.

Again, maybe you didn't intend to, but it sounded like you were saying
something very negative about Unity. A less biased way to say it might be

"Unity and Mono could not arrive at mutually acceptable terms for Unity to
license newer versions of Mono."

~~~
carussell
The only thing I'm saying are very negative things towards people who read (or
heard the story of) Unity's PR-level attempt to make it sound like the people
behind Mono are jerks, and then bought in to it full force and now repeat the
line at every opportune moment (and then some inopportune ones, too, like the
comment I replied to).

What we also don't know are the terms for the first negotiation. It's as
possible that Mono made one offer and Unity countered with an insultingly
lower one, then Mono accepted due to being a poor position to negotiate
something "reasonable", but are no longer in a position to have to accept
whatever is offered, now that Xamarin is off and running.

We don't know. Hypotheses non fingo.

