
Xplain – Explaining X11 for the rest of us (2017) - pcr910303
https://magcius.github.io/xplain/article/
======
Jasper_
Author here. I haven't updated this project in a few years. I burned out on it
almost immediately. I wrote a little retrospective for it on here before, the
last time it was posted:
[https://news.ycombinator.com/item?id=21041340](https://news.ycombinator.com/item?id=21041340)

In the days since, I've joined the games industry, fell in love with it there,
and am working a bit more on more video game-focused content:

[https://noclip.website](https://noclip.website)

[https://www.youtube.com/watch?v=mnxs6CR6Zrk](https://www.youtube.com/watch?v=mnxs6CR6Zrk)

[https://www.youtube.com/watch?v=8rCRsOLiO7k](https://www.youtube.com/watch?v=8rCRsOLiO7k)

If anybody has any questions about game graphics, Linux graphics, or X11, feel
free to ask.

~~~
exikyut
I just checked Xplain out and wow, it's a bit expanded from how I remember it
from a few years ago, I'm definitely going to be fine-tooth-combing through it
at some point, particularly the section on regions which looks like it'll help
me crack the overlapping-window problem. The rasterization section also looks
like it'll be fun to "??!?" through, thanks for making that!

(Compositing was also interesting to look through; I always thought it was
extremely curious how no X11 windowmanagers ever did hybrid compositing, where
they only composited minimized windows into thumbnails but left actual open
windows alone without inducing ≈1 frame of extra latency. I'm very curious if
Wayland can avoid that, but that's just idle curiosity at this point.)

NB. noclip.website is throwing "program not valid" from
`this.gl.useProgram(e.gl_program)` (line 100,924 :D) with a HD Graphics 3000
on oldish i5-3360M (HP 8470p). Chances are it needs a beefier GPU.

~~~
Crespyl
Regarding "hybrid" compositors, I believe there used to be a few that had an
"un-redirect fullscreen windows" feature that essentially did what you're
suggesting. I think there was some sort of difficulty with doing this for
regular/non-fullscreen windows and still having a consistent/smooth experience
with the rest of the features you'd want out of a compositor.

These days, at least in KWin land, there's just an "allow applications to
block compositing" feature that automatically disables compositing (globally)
when it makes sense. I'm not sure exactly what rules trigger it (simply
putting, say, FireFox, into fullscreen doesn't do it, but Dota 2 does), but it
seems to work pretty well.

~~~
badsectoracula
I implemented fullscreen OpenGL in my 3D engine a few years ago and this was
kind of an interesting part. It was one of the hardest to get right,
especially if you want to avoid dictating sort of WM your users will have to
use (which IMO is one of X11 strengths over pretty much any other desktop
GUI).

Some window managers treat any window that covers the entire screen as being
fine to bypass composition. Other window managers need hints - which will be
ignored by some other window managers. And a few compositors simply seem to
ignore any hint or window state, but they will disable composition for
fullscreen windows that also bypass window management (which in general is
kind of a "nuclear option" for fullscreen modes since it also bypasses other
window manager functionality like alt-tab, etc). Also as a cherry on top, i
had a window manager which refused to treat the window as a fullscreen window
(added borders, etc) unless i set up things in a very specific order (which i
found out via trial and error because normally it shouldn't be an issue - and
wasn't in every other window manager i tried).

(that last part was kinda interesting because i downloaded every window
manager i could find on Debian and after i tested my code and debugged it with
everything i could get my hands on, i sent it to a friend of mine to try it
out and he immediately told me that in his computer he got borders :-P)

Should be noted that neither SDL1 nor SDL2 (which are pretty much recommended
by people who haven't bothered to create their own engines as "obvious"
solutions to not have to bother with OS-specific stuff) handled all the above
(SDL1 always bypasses the window manager, which in general works but disables
alt-tabbing and other WM functionality whereas SDL2 assumes all window
managers support the fullscreen hint, which is certainly not the case -
especially with more minimalistic window managers - while compositor support
is hit and miss).

------
jhardy54
This is great messaging. I'm happy to enable JavaScript when authors
communicate why they use it.

> This article relies a lot of interactive demos that use JavaScript. I
> understand you're disabling JavaScript because you've been burned by heavy
> scripts and tracking on other sites, but the article really doesn't work
> without the ability to play with it. I write every line of code in these
> demos from scratch, minus the snippet of Google Analytics that lets me know
> when people read these so I can join in the conversation. Feel free to block
> Google Analytics.

This is a billion times better than a white screen with the words "JS required
to use this app" when you're trying to read a simple document.

------
bjarneh
Very cool, hard to imagine the constraints they had when that was created.
Reminds me of an interview I heard with Bill Joy a while back (creator of vi +
worked on the Java language in its beginnings).

"We solved problems that do not exist anymore" — Bill Joy.

~~~
kjs3
Constraints? It was generally developed on '3-M' machines: more or less around
1MIPS of processing power (68k, VAX), 1M of memory, and 1M pixels (e.g.
1024x1024).

While X11 is pretty grokky these days and definitely showing it's age, I'm not
sure that the problems X11 solved don't exist any more. In particular, the
idea of running an app on one machine and displaying it more or less
transparently. It's just that with very performant desktops, this became the
exception rather than the norm.

~~~
Jasper_
OK, if I really want one meme about X11 to die, it's this "network
transparency" one. Nobody has ever explained to me what it means, what its
goals are, what constraints it has on the system or the protocol. I can't find
any definition of it in any official X11 documentation, protocol or otherwise.

The typical answer I see is "you can send the same bits locally that you do
over the wire", but nobody ever explains why that's even desirable for a
network protocol. Realities of distributed systems vs. local ones have a wide
variety of tradeoffs. Transparent, distributed RPC was tried in the 90s and
largely judged a failure. We know how to design much more resilient systems
than that.

Not to mention that ever since the introduction of the MIT-SHM protocol, which
is used by most everything to transfer image data, it hasn't even been true in
practice for the last 20 years. Most modern enhancements to X11 have been
about taking the X server out of the loop for as much as possible.

To make X11 usable over the network, even advocates of the principle usually
use something like NX which is a network-designed fork of the protocol, which
introduces a large number of new features aimed at network that wouldn't be
required over a network, things like lossy compression, multiple independent
streams so we don't have reach head-of-line blocking with roundtrips, and
client-side rendering.

Problems like remote desktop and streaming can be solved by other solutions
than "shove the same bits over the wire and hope they're helpful on the other
side".

~~~
tgot
Network transparency meant that when I was doing Finite Element Analysis on
the beefiest Sun workstation they sold, running a proprietary application, I
could log in from home, run the application on the Sun, and see the results of
the prior run.

I could edit the model, regenerate the mesh if needed, and kick off another
big batch run, and then drop the X11 frontend GUI.

Over _dial-up_ , from a Windows box to a Sun workstation, in the mid 90s. And
it was could be secure, tunneled through SSH.

~~~
cycloptic
I'm assuming that was sending OpenGL 1.0 commands over the network, or doing
some kind of equivalent. Using X11 drawing commands for that would have
resulted in miserable frame rates over dial-up if you ever tried to rotate or
zoom the mesh. In any case that's not really network transparency nor is it
really X11, the program likely had to be built in a very specific way using
display lists.

~~~
ddingus
CAD and Simulation Apps were written in those specific ways.

You are right.

Where that was done, performance was excellent.

~~~
cycloptic
It just seems wrong in that case to to say that X is network transparent. The
real concern is that OpenGL 1.0 was capable of running over the network, and
in order to use it effectively application developers had to take network
operation into consideration, and the server had to support the non-standard
extension required to use it correctly. In some circumstances using display
lists locally could actually reduce performance, so the application may not
have wanted to take that path in all cases:
[https://www.opengl.org/archives/resources/faq/technical/disp...](https://www.opengl.org/archives/resources/faq/technical/displaylist.htm)

Generally if your application has any code that does this:

    
    
        if (client is remote)
            ...
        else if (client is local)
            ...
    

Then I wouldn't say the protocol you're using is network transparent.

~~~
ddingus
Open GL could run over the network because X could.

Sgi used X fully. They wrote the book on all that, including GLX, and it was
flat out awesome.

The Apps I used all ran display and pick lists. They worked very well over the
wire and that use case for used a lot.

The quibbles are really getting past the basic idea of running an app remotely
on ones local display.

That happened and worked well and had some advantages. I personally built some
pretty big systems for modeling and simulation that were cake to admin and
very effective on many other fronts.

Notice I did not say network transparent in my comments above.

Multi-use graphical computing is much more accurate when it comes to how X
worked and what it delivered people.

------
drewg123
I remember my undergrad graphics class in 1991 or so. We did it in C and X11
on Sun and DEC workstations, and we had to learn X as an implementation
detail. Our final project was animated 3-d wireframe models of objects,
written in Xlib primitives. No toolkits or modern libraries. One of the
assignments was a "game", and I wrote a tetris-like typing game. The projects
in that class were insanely hard compared to any other class I'd taken.

The amazing thing is that, at least as of 10 years ago, my projects for that
class _STILL_ compiled on 32-bit FreeBSD and Linux.

Oh, and to learn X11, we used
[https://www.amazon.com/gp/product/0134999975/ref=x_gr_w_bb_s...](https://www.amazon.com/gp/product/0134999975/ref=x_gr_w_bb_sout?ie=UTF8&tag=x_gr_w_bb_sout-20&linkCode=as2&camp=1789&creative=9325&creativeASIN=0134999975&SubscriptionId=1MGPYB6YW3HWK55XCGG2)
I still remember sitting on a bench outside the college bookstore, cracking
that book open and feeling both excited and hopelessly lost.

~~~
formerly_proven
> The amazing thing is that, at least as of 10 years ago, my projects for that
> class STILL compiled on 32-bit FreeBSD and Linux.

i686 binaries from the 90s compiled for then-Linux and linked against then-
xlib _still_ run on AMD64 Linux hosts (provided you have the necessary
multilib libraries around).

------
david_draco
I wish hacking on X and getting patches in had the same coolness factor as
hacking on the Linux kernel. The hacking complexity is certainly comparable. X
is used by a very large number of people, and contributions can make a big
difference.

~~~
cycloptic
There's still a good number of graphics hackers, but a lot of code has moved
out of Xorg and as a result most of the hackers have moved on to working on
Mesa, Wayland, and in-kernel GPU drivers. So there is some overlap there with
the kernel hackers.

------
dang
If curious see also

2019
[https://news.ycombinator.com/item?id=21039848](https://news.ycombinator.com/item?id=21039848)

also 2019
[https://news.ycombinator.com/item?id=21118633](https://news.ycombinator.com/item?id=21118633)

2016
[https://news.ycombinator.com/item?id=12042548](https://news.ycombinator.com/item?id=12042548)

2014
[https://news.ycombinator.com/item?id=8019346](https://news.ycombinator.com/item?id=8019346)

2013
[https://news.ycombinator.com/item?id=6978274](https://news.ycombinator.com/item?id=6978274)

------
SXX
Really recommend that speech as well since it's give good understanding what's
wrong with X11 and why Wayland was created.

The real story behind Wayland and X:

[https://www.youtube.com/watch?v=GWQh_DmDLKQ](https://www.youtube.com/watch?v=GWQh_DmDLKQ)

