Hacker News new | past | comments | ask | show | jobs | submit login
Xplain – Explaining X11 for the rest of us (2017) (magcius.github.io)
326 points by goranmoomin on Aug 18, 2020 | hide | past | favorite | 89 comments



Author here. I haven't updated this project in a few years. I burned out on it almost immediately. I wrote a little retrospective for it on here before, the last time it was posted: https://news.ycombinator.com/item?id=21041340

In the days since, I've joined the games industry, fell in love with it there, and am working a bit more on more video game-focused content:

https://noclip.website

https://www.youtube.com/watch?v=mnxs6CR6Zrk

https://www.youtube.com/watch?v=8rCRsOLiO7k

If anybody has any questions about game graphics, Linux graphics, or X11, feel free to ask.


I just checked Xplain out and wow, it's a bit expanded from how I remember it from a few years ago, I'm definitely going to be fine-tooth-combing through it at some point, particularly the section on regions which looks like it'll help me crack the overlapping-window problem. The rasterization section also looks like it'll be fun to "??!?" through, thanks for making that!

(Compositing was also interesting to look through; I always thought it was extremely curious how no X11 windowmanagers ever did hybrid compositing, where they only composited minimized windows into thumbnails but left actual open windows alone without inducing ≈1 frame of extra latency. I'm very curious if Wayland can avoid that, but that's just idle curiosity at this point.)

NB. noclip.website is throwing "program not valid" from `this.gl.useProgram(e.gl_program)` (line 100,924 :D) with a HD Graphics 3000 on oldish i5-3360M (HP 8470p). Chances are it needs a beefier GPU.


Regarding "hybrid" compositors, I believe there used to be a few that had an "un-redirect fullscreen windows" feature that essentially did what you're suggesting. I think there was some sort of difficulty with doing this for regular/non-fullscreen windows and still having a consistent/smooth experience with the rest of the features you'd want out of a compositor.

These days, at least in KWin land, there's just an "allow applications to block compositing" feature that automatically disables compositing (globally) when it makes sense. I'm not sure exactly what rules trigger it (simply putting, say, FireFox, into fullscreen doesn't do it, but Dota 2 does), but it seems to work pretty well.


I implemented fullscreen OpenGL in my 3D engine a few years ago and this was kind of an interesting part. It was one of the hardest to get right, especially if you want to avoid dictating sort of WM your users will have to use (which IMO is one of X11 strengths over pretty much any other desktop GUI).

Some window managers treat any window that covers the entire screen as being fine to bypass composition. Other window managers need hints - which will be ignored by some other window managers. And a few compositors simply seem to ignore any hint or window state, but they will disable composition for fullscreen windows that also bypass window management (which in general is kind of a "nuclear option" for fullscreen modes since it also bypasses other window manager functionality like alt-tab, etc). Also as a cherry on top, i had a window manager which refused to treat the window as a fullscreen window (added borders, etc) unless i set up things in a very specific order (which i found out via trial and error because normally it shouldn't be an issue - and wasn't in every other window manager i tried).

(that last part was kinda interesting because i downloaded every window manager i could find on Debian and after i tested my code and debugged it with everything i could get my hands on, i sent it to a friend of mine to try it out and he immediately told me that in his computer he got borders :-P)

Should be noted that neither SDL1 nor SDL2 (which are pretty much recommended by people who haven't bothered to create their own engines as "obvious" solutions to not have to bother with OS-specific stuff) handled all the above (SDL1 always bypasses the window manager, which in general works but disables alt-tabbing and other WM functionality whereas SDL2 assumes all window managers support the fullscreen hint, which is certainly not the case - especially with more minimalistic window managers - while compositor support is hit and miss).


SDL2 sets this on windows by default, which is likely what's doing it:

https://specifications.freedesktop.org/wm-spec/wm-spec-lates...

Compositing is required for any sort of antialiasing, especially including window borders, so it's difficult to turn off without having a lot of artifacts. And it also solves vsync which a lot of raw X11 applications can't possibly render correctly with.


Nothing I can do about old GPUs... I need a working WebGL implementation. You can try a simpler game like Mario Kart DS to see if that works.


(Thanks for replying! https://get.webgl.org/webgl2/ is green and shows a spinning cube, but yeah, it's quite old.)


Amazing work! I can understand how you can burn out on it though. The explanations are very friendly and easy to follow, so don't be deterred if you want to create more EXPLANATION content!


I have no desire to continue the Explanations series. The videos are where I'm spending my focus now.


OT: Noclip is amazing, thank you. It's sorely needed in an industry that's (mostly unintentionally) very good at erasing its history.


Thank you for putting this together. This answered many of the questions I've had about X for multiple decades.


I had never heard of noclip.website... our office stopped for a couple minutes.


What year do you suggest we put in the title above? https://news.ycombinator.com/item?id=21039848 said 2017 but clearly the project goes back earlier.


The first article (X Window System Basics) was written in 2013. The last article (Basic 2D Rasterization) was written in 2017. Take your pick.


I pick 2017.


This is great messaging. I'm happy to enable JavaScript when authors communicate why they use it.

> This article relies a lot of interactive demos that use JavaScript. I understand you're disabling JavaScript because you've been burned by heavy scripts and tracking on other sites, but the article really doesn't work without the ability to play with it. I write every line of code in these demos from scratch, minus the snippet of Google Analytics that lets me know when people read these so I can join in the conversation. Feel free to block Google Analytics.

This is a billion times better than a white screen with the words "JS required to use this app" when you're trying to read a simple document.


Very cool, hard to imagine the constraints they had when that was created. Reminds me of an interview I heard with Bill Joy a while back (creator of vi + worked on the Java language in its beginnings).

"We solved problems that do not exist anymore" — Bill Joy.


“X is rasterop on wheels” — Bill Joy in Unix Today 6/25 page 5.

https://donhopkins.com/home/archive/NeWS/news-info.txt

"..., Richie reminded the audience that Steve Jobs stood at the same podium a few years back and announced that X was brain-dead and would soon die.

"He was half-right," Ritchie said. "Sometimes when you fill a vacuum, it still sucks."

http://www.anvari.org/fortune/Miscellaneous_Collections/1332...

https://news.ycombinator.com/item?id=22502125


Thankfully, not only do we have a modern replacement for X in Wayland, we also have a modern replacement for NeWS. It's called "the browser"/Electron.


Constraints? It was generally developed on '3-M' machines: more or less around 1MIPS of processing power (68k, VAX), 1M of memory, and 1M pixels (e.g. 1024x1024).

While X11 is pretty grokky these days and definitely showing it's age, I'm not sure that the problems X11 solved don't exist any more. In particular, the idea of running an app on one machine and displaying it more or less transparently. It's just that with very performant desktops, this became the exception rather than the norm.


OK, if I really want one meme about X11 to die, it's this "network transparency" one. Nobody has ever explained to me what it means, what its goals are, what constraints it has on the system or the protocol. I can't find any definition of it in any official X11 documentation, protocol or otherwise.

The typical answer I see is "you can send the same bits locally that you do over the wire", but nobody ever explains why that's even desirable for a network protocol. Realities of distributed systems vs. local ones have a wide variety of tradeoffs. Transparent, distributed RPC was tried in the 90s and largely judged a failure. We know how to design much more resilient systems than that.

Not to mention that ever since the introduction of the MIT-SHM protocol, which is used by most everything to transfer image data, it hasn't even been true in practice for the last 20 years. Most modern enhancements to X11 have been about taking the X server out of the loop for as much as possible.

To make X11 usable over the network, even advocates of the principle usually use something like NX which is a network-designed fork of the protocol, which introduces a large number of new features aimed at network that wouldn't be required over a network, things like lossy compression, multiple independent streams so we don't have reach head-of-line blocking with roundtrips, and client-side rendering.

Problems like remote desktop and streaming can be solved by other solutions than "shove the same bits over the wire and hope they're helpful on the other side".


I'm not clear on what you're missing. What meme? The protocol is well documented. The basic abstraction is that there is a separation between the 'processing' part of an application and the 'display' part of the application and those two parts communicate using the X11 protocol. That conversation can happen over a local or a remote socket, or shared memory. The practical use cases are that I can have an application running on my local machine, or one on the other side of the world, and the 'display' portion is consistent. Not much different than, say how Citrix uses it's own protocol to draw remote displays.

It allowed a 'mainframe' model where you could have a very fast, expensive application servers that could display to less expensive local resources, and mix the display with local applications. So my heavyweight applications could run on the monster Sun or Cray in the datacenter, but my clock, calc and notepad app ran locally, but all in the same display.

Yes, over the years things like MIT-SHM and NX came along to paper over problematic issues in X11. And yes, there are other, possibly better ways so solve it now. But in the late 80s and through the 90s, it was pretty cool stuff.


It's possible I'm a bit jaded because I spent a long time working on things like Wayland and next-generation graphics technology, where people would suddenly show up and name-drop the terminology "network transparency" as if it meant something, or solved useful problems.

I just find it funny that CORBA-style RPC is now considered a dated, bad approach in the world of distributed systems, but the same thing in the X11 world is considered way ahead of its time and a design ideal.


> I spent a long time working on things like Wayland and next-generation graphics technology

This is really sad. You are insulting and dismissing technology which did something well over 20 years ago that you think is not possible today. And instead of thinking "hmm maybe I should look at the older technology and see how it did it" you just dismiss it as a "meme" and tell people that the actual technology they used is impossible. So much effort is wasted in our industry because of NIH syndrome...

I seriously hope that the Wayland people didn't just completely ignore the positives of Xwindows!


I wrote the linked article explaining X11's internals and design goals in a neutral manner. I've worked on X11 and Xorg ecosystem, worked on the server, I worked on the window managers, the toolkits, applications, compositors, you name it.

I think I'm allowed to have opinions with 40-year hindsight saying that some design decisions have not aged well. I'm not saying X11 is a meme; I'm saying the words "network transparency" ( which never appears once in the documentation; seriously, try to find it https://www.x.org/releases/current/doc/xproto/x11protocol.ht... ) are unhelpful, have unclear motivations and goals, and have a massive restraining influence on what a modern protocol can look like and do.


Aaahh! I get it now.

Network transparency: you can run a process remotely and see the same interface that you would have seen if you had (or had been able to) run it locally. Nothing more, nothing less. Mostly marketing.


We're so desperate for network transparency that we're abusing hypertext web browsers as application platforms, because how else can a machine in a datacenter put a GUI on an end user's screen today?


That's not network transparency. A significant part of the application code is running on the local machine and even if the webserver/API server is turned off there are many applications that work offline and only need a remote connection to download the application itself and sync data periodically.

Your definition of network transparency is so weak even an email client would count as a network transparent application even though 100% of the email client is running on the local machine.

A definition of network transparency that is based around the current behavior of X11 would require you to fully own the remote machine. The remote machine is the real computer that is executing application code and all the display server does is output pixels to a screen and send inputs back to the real machine. When you're using a browser or an email client or a smartphone app the real machine is the device you are holding in your hand, not some server sitting in a data center.

Let me rephrase my point. sshing into a remote machine and running vim there would be considered network transparency. Running vim on your own machine on a mounted NFS share would not be an example of network transparency.

Now transfer this to your web browser argument: How many people are actively receiving control over a google server when they log into gmail? Zero.


> how else can a machine in a datacenter put a GUI on an end user's screen today?

A lot of companies are doing it with Citrix. That's a framework which provides Windows users with what 99% of the Unix users asking for X network transparency are really asking for: the ability to run an app or optionally an entire desktop on the server and have it render locally, where it can be used and controlled. There are some optional file sharing things as well, and audio, etc.

I'm sure the network protocols underneath are appalling, but the product does what people want.


> which did something well over 20 years ago

Not sure that was ever true. 20 years ago, MS remote desktop (called "terminal server" at that time) was vastly superior to X11 for remoting.


"RDS was first released as "Terminal Server" in "Windows NT Server 4.0 Terminal Server Edition" a stand-alone operating system including SP3 and fixes built in. Starting with Windows 2000, it was an optional role and became a mainstay of the Windows NT family of operating systems and was improved with each version of Windows. The rename to "Remote Desktop Services" occurred with Windows Server 2008 R2 in 2009."

X predates Windows NT by more than 10 years and is a completely different beast than remote desktop systems. (Disclaimer: I used to log into a local box, run an unpopular window manager on a remote server, and then spread my workload over a number of remote machines. Terminals, email, gui apps, and yes, the cheesy web browser of the time. Seamlessly. (No desktop-in-a-window. Same files everywhere (Thanks, NFS!), same apps (more work to set up than in Plan 9, but didn't have Plan 9's disadvantages, either) everywhere. And that was half a decade before Windows NT.)


> by more than 10 years

Good point. Still, from 2020 both “available for 21 years” and “available for 33 years” more or less equal to “available since forever”.

> is a completely different beast than remote desktop systems

Pretty much the same. I think you’re confusing remote desktop with VNC, TeamViewer and similar. RDP doesn’t just send bitmaps, it sends GDI draw commands, later versions supplemented them with DirectX9 and D3D11 draw calls. The server implements that by replacing GPU driver with a special version which sends draw commands to connected client, as opposed to rendering stuff on the display.


> I spent a long time working on things like Wayland and next-generation graphics technology

I don't know you, so this might come off as overly aggressive. But very little that happens in the FOSS/Linux space and has to do with graphics I would describe with "next gen". Wayland for example is meant as the "ring to rule them all" as a way to drive displays, yet it doesn't even support color management, so it's increasingly useless in a world of non-sRGB displays. X11 does color management (in a cludgy way), for example.


Linux graphics is mostly driven by volunteer work. There is ongoing work to try and get color management done properly (though color management is an extraordinary difficult topic and you could make the argument no display system gets it correct)

https://lists.freedesktop.org/archives/wayland-devel/2019-No...


https://www.omg.org/gettingstarted/history_of_corba.htm

CORBA 1.1 (February 1992) This was the first widely published version of the CORBA specification. It closed many ambiguities in the original specification; added interfaces for the Basic Object Adapter and memory management; clarified the Interface Repository, and clarified ambiguities in the object model.

https://en.m.wikipedia.org/wiki/X_Window_System

X originated at the Project Athena at Massachusetts Institute of Technology (MIT) in 1984.[1] The X protocol has been at version 11 (hence "X11") since September 1987.


> Nobody has ever explained to me what it means, what its goals are, what constraints it has on the system or the protocol.

In today's world, what I think people want to do (and certainly what I want to do) is ssh to a machine, run a graphical program on that machine and see and interact with it in my graphical environment. X11 with ssh tunneling lets me do that. It's not a great experience, for all sorts of reasons, but it's an experience, and people who have been around for a while remember that it used to work better. It's usable enough on many applications to work in a pinch, or for ocassional work (I always run mythtv-setup via remote X, because it's inconvenient otherwise; sometimes I run a graphical browser over remote X to access localnetwork only servers from outside my home, but only if I can't get proxying to work, because browsers on remote X are awful in 2020 in ways they weren't in 2005)

I haven't looked in a while, and maybe waypipe fills this gap now, but I think at least some people are frustrated because Wayland is supposed to replace X11, but it doesn't hit all their use cases, and they may not care that much about the things it does better because they were used to the old ways. (Of course, if those things include hardware support, that will be a forcing function).

I certainly understand your frustration though. You built something new and better, and people won't use it because they're stuck on something that barely works.


It's not a great experience, and it doesn't work at all with everything that uses video acceleration (like a modern browser, and as a corollary all Electron applications).

In contrast Windows RDP protocol works much better, even if it's sending the whole desktop as a video stream, and works even on modest connections (like a 20mbit/s ADSL).

In fact nowadays I would argue that the method used by Xorg is no longer ideal: the commands that needs to be sent between the application and the GPU are too big to be sent on a network connection. Instead we got much better video compression algorithms, so it's better instead of sending on the network the commands to draw the application to just have the server GPU render the application and then send a compressed video stream to the client.


> it doesn't work at all with everything that uses video acceleration (like a modern browser, and as a corollary all Electron applications).

Works for me. 15 years ago it was a bit more problematic, but worked still.

> works even on modest connections (like a 20mbit/s ADSL)

This is not modest by any means. Vnc works good on 56 kbit, and marginally at 24kbit.


> It's not a great experience, and it doesn't work at all with everything that uses video acceleration (like a modern browser, and as a corollary all Electron applications).

I'm posting this with Firefox 79 running on FreeBSD, displayed on Windows (VcXsrv) via remote X over ssh (gigE lan because I'm at home). Not sure how much modern I need to get. chromium-84 also installed and worked enough. Scrolling is glitchy, and I don't dare watch a video[1], but either would work enough if I need to use a graphical browser to administer some network device remotely (I used to be able to use w3m for a lot of those)

I'm sure RDP would be better, but it doesn't fit my use case. I don't want a whole desktop, just one window, and I do need it to tunnel through ssh, and it's nice when it's a simple setting, so I can just run a program and it shows up. If there's an easy way to tunnel RDP that I'm not aware of, I'm happy to learn, especially if the experience is better, but if it's super fiddly, I'd rather use what I know and what works, even if it's laggy and awful.

If RDP worked like X, I'd love to setup a beefy windows server in my basement and play games on it via RDP from my PCs around the house. If anyone has tips on that, would love to read them. (Requirements: single server, three simultaneous users, single GPU ideal, but flexible; want to play Civ V etc with the family. Cost (including licensing) needs to be comparable with three low tier desktops (i3-8300 ))

> In fact nowadays I would argue that the method used by Xorg is no longer ideal

Again --- I know it's not ideal, it's just less fiddly and I already know how to use it, and it how to make it work hasn't changed significantly in the last 15 years (mostly the windows X servers are better and crash less), and desktop applications tend to be more synchronous and harder to use over high latency connections. I'm not looking to run all my applications from the other side of the country, I just want to occasionally run a couple things from home when I'm not there, or to setup my HTPC with a mouse and keyboard instead of a remote control.

I think everyone will hate me for suggesting it, but I could imagine an X11 extension where if the client is not local to the server, it would switch to rendering everything locally and sending compressed video; I don't expect anybody to build that, but there's no reason you couldn't do it. It wouldn't solve important issues like head of line blocking, of course.

[1] Ok, for giggles, I tried to watch something on youtube. It was surprisingly decent. Lots of tearing, and I don't know where the audio went (clearly, not over X which doesn't do that, not sure if it was playing on the X client host), and it took a long time to stop the video because I'm guessing a whole ton of frames were buffered. But, I got the idea of the video, and it didn't kill the session. If I were doing this over my WAN, it would be a lot worse, clearly.


I understand what you want but as someone who spends a non insignificant time watching youtube videos over what amounts to a glorified VNC (it's not VNC though) I don't see why a video compression based solution wouldn't work for you. The only downside is that real time video encoding is very CPU intensive if you're running it on a budget VPS. However, CPU based HEVC decoding is also really bad on a VPS. On a dedicated server or machine it's going to be insignificant though.

>If RDP worked like X, I'd love to setup a beefy windows server in my basement and play games on it via RDP from my PCs around the house. If anyone has tips on that, would love to read them.

Let's be honest. Using X for this would have absolutely atrocious results. Steam already has a solution for that and it is based on video encoding. You can stream games from a local machine to any computer with the Steam client installed on it. There is also dedicated hardware to stream to a TV and use a controller instead so you might not even need more than one computer. [0]

[0] https://store.steampowered.com/steamlink/about/


> If RDP worked like X, I'd love to setup a beefy windows server in my basement and play games on it via RDP from my PCs around the house. If anyone has tips on that, would love to read them. (Requirements: single server, three simultaneous users, single GPU ideal, but flexible; want to play Civ V etc with the family. Cost (including licensing) needs to be comparable with three low tier desktops (i3-8300 ))

Steam's in-home streaming is close to what you want. However, I don't think it supports multiple simultaneous users. Perhaps there is a way to hack that to work somehow, assuming you had three Steam accounts with a copy of the game each.


Search for XPRA.


XPRA sounds like screen for X? Plus maybe a betterish protocol, but also maybe a bit more fiddly?


Network transparency meant that when I was doing Finite Element Analysis on the beefiest Sun workstation they sold, running a proprietary application, I could log in from home, run the application on the Sun, and see the results of the prior run.

I could edit the model, regenerate the mesh if needed, and kick off another big batch run, and then drop the X11 frontend GUI.

Over dial-up, from a Windows box to a Sun workstation, in the mid 90s. And it was could be secure, tunneled through SSH.


I'm assuming that was sending OpenGL 1.0 commands over the network, or doing some kind of equivalent. Using X11 drawing commands for that would have resulted in miserable frame rates over dial-up if you ever tried to rotate or zoom the mesh. In any case that's not really network transparency nor is it really X11, the program likely had to be built in a very specific way using display lists.


CAD and Simulation Apps were written in those specific ways.

You are right.

Where that was done, performance was excellent.


It just seems wrong in that case to to say that X is network transparent. The real concern is that OpenGL 1.0 was capable of running over the network, and in order to use it effectively application developers had to take network operation into consideration, and the server had to support the non-standard extension required to use it correctly. In some circumstances using display lists locally could actually reduce performance, so the application may not have wanted to take that path in all cases: https://www.opengl.org/archives/resources/faq/technical/disp...

Generally if your application has any code that does this:

    if (client is remote)
        ...
    else if (client is local)
        ...
Then I wouldn't say the protocol you're using is network transparent.


Open GL could run over the network because X could.

Sgi used X fully. They wrote the book on all that, including GLX, and it was flat out awesome.

The Apps I used all ran display and pick lists. They worked very well over the wire and that use case for used a lot.

The quibbles are really getting past the basic idea of running an app remotely on ones local display.

That happened and worked well and had some advantages. I personally built some pretty big systems for modeling and simulation that were cake to admin and very effective on many other fronts.

Notice I did not say network transparent in my comments above.

Multi-use graphical computing is much more accurate when it comes to how X worked and what it delivered people.


BTW, display performance on those was great local. A small hit locally is not that big of a deal. Never has been.

Users will employ detail limits, model scope limits, whatever to get the UX they need.

Developers can dual path it, or provide options. And they will provide options because not all users have latest and greatest. They never do.

In the end, it is mostly a wash for most things.

The big gains were had in other areas.

In the CAD space, sequential CPU is far more of a bottleneck. Mid to lower grade GFX subsystems perform more than good enough for a ton of cases today. Can't get a fast enough sequential compute CPU. And while there is serious work to improve multi threaded geometry, fact is most important data is running on crazy complex software that needs the highest sequential compute it can get.

Big data actually sees a gain with the X way of doing things.

Huge model data running over shared resources is a hard problem. And it continues to be one. Mix in multi user and it takes serious tools to manage it all and perform.

In the 90's, many of us were doing those things, multi user, revision control, concurrent access, you name it on big models and fast, local file systems. There was software with all that well integrated. We did not have cloud yet. Not really.

The app server model rocked!

X made all that pretty easy. One setup, and just connect users running whatever they want to, so long as a respectable X server is available, they are good to go.

One OS, the whole box dedicated to one app, fast storage, big caches, multiple CPUS all tuned to get that job done well and perform.

Once that work is done, doing things the X way means it stays done, and users just run the app on the server. Bonus is they can't get at that data directly. Lots of silly problems just go away.

And, should that system need to be preserved over a long period of time? Just do that, or package it all up and emulate it.

In all cases, a user just connects to run on whatever they feel like running on.

Those of us still talking about how X does things see many advantages. Like anything, it is not the be all end all. It is a very nice capability and a "unixey" way of doing things being lost.


It is quite possible it just sent draw line requests as a list of vertexes after doing all the math on the client, which core x supports just fine.


That takes huge amounts of bandwidth. There's a reason display lists were necessary to get it to work on dial-up connection.


It means that _today_, I can run an IDE on my 32-core home desktop and interact with it seamlessly on my 13-inch laptop at the office. I'm not sure whether this is possible with Wayland.


https://wayland.freedesktop.org/faq.html#heading_toc_j_8:

Is Wayland network transparent / does it support remote rendering?

No, that is outside the scope of Wayland. To support remote rendering you need to define a rendering API, which is something I've been very careful to avoid doing. The reason Wayland is so simple and feasible at all is that I'm sidestepping this big task and pushing it to the clients. It's an interesting challenge, a very big task and it's hard to get right, but essentially orthogonal to what Wayland tries to achieve.

This doesn't mean that remote rendering won't be possible with Wayland, it just means that you will have to put a remote rendering server on top of Wayland. One such server could be the X.org server, but other options include an RDP server, a VNC server or somebody could even invent their own new remote rendering model. Which is a feature when you think about it; layering X.org on top of Wayland has very little overhead, but the other types of remote rendering servers no longer requires X.org, and experimenting with new protocols is easier.

It is also possible to put a remoting protocol into a wayland compositor, either a standalone remoting compositor or as a part of a full desktop compositor. This will let us forward native Wayland applications. The standalone compositor could let you log into a server and run an application back on your desktop. Building the forwarding into the desktop compositor could let you export or share a window on the fly with a remote wayland compositor, for example, a friend's desktop.


See waypipe for an example of how this can be done "natively": https://gitlab.freedesktop.org/mstoeckl/waypipe/


All of that said, "ssh -Y" works pretty well for me. It's pretty usable.

And from a user perspective I personally like the "application connects to render" model much better than VNC or RDP style entire-desktop streaming.


IIRC it used to work a lot better (like, a whole lot better) but modern X apps mostly end up sending bitmaps over the wire, rather than native X rendering instructions.

Agree that it works better than other remote desktop options on Linux. Overall though I get better performance with RDP to Win10 running Linux in Virtualbox (!) than any other option I've tried though, including Linux on bare metal with a variety of remote desktop options, and Virtualbox's remote desktop server.


NX partially solves the problem that applications don't use the X primitives anymore and are way to chatty with the X server (which leads to many unnecessary roundtrips which is the real killer for remote X11). Old applications work great even without NX or any hacks.. just anything modern is quite sluggish.

And it's less about the actual protocol, but the desired behavior. I want to launch applications on a remote machine and have the windows appear on my local machine as if I was running them locally. And so far there is nothing except X11 that can really do that..


>And so far there is nothing except X11 that can really do that.

In terms of Wayland there is Waypipe. And I believe Citrix has been doing this forever on Windows, although I've never used their products.

In my opinion Xorg by itself can't really do it correctly, I find running remote clients to be basically unusable without Xpra. And in order to really use that effectively, you have to be running all your clients within Xpra from the start.


Back in the 90s, I had a serial cable under the carpet from my computer to my couch. My computer would access the internet over PPP and my laptop would PPP over to my computer. Then I would launch netscape and browse the web from the comfort of my couch.

I have had a couple other use cases over the years like virt-manager at work, and a graphical management tool that I made for my son's minecraft server.

Sure there were other solutions but X11 over the wire worked well.


Not that anyone feels it is important today, but X11 is multi user graphical computing.

A display server can display anything any user connected to it wants to display.

Actual 90's era use cases:

Application server. One copy of 3D CAD, one database, 30 users all running on a beefy machine well up to the task.

Each ran via X talking to the app server. Admin was cake, data big model access was fast, no user could actually touch the data except through use of the program.

User having trouble, push a window onto my display. I fix it, close window.

Display wall, conferencing. Big wall display, multiple users display data on it, one management user handling windows, etc...

Display wall, many small displays, each running via workstation. App sends display data to all of them.

Distributed computing resources:

Once, just for fun, I had an application data shared via a machine running nfs, fonts served by another machine, yet another machine running the application, yet another handling window management, all displayed on yet another machine serving the display and taking input.

(BTW, it was possible to literally kill the window manager and launch a different one and the display would shift a bit and the user continued along still running their app, but now using a different window manager!)

User could click on an icon, all those machines contributed to the app experience. Was hard to tell the difference between that and it running all locally.

Why?

Linux user wanted to run CAD unavailable on Linux, was developer who wanted a Linux box. So I used X to push the whole experience to their machine and distributed it for fun once it was working well just to learn and really push things, frankly.

While running, it was hard to tell the Linux machine was not an expensive commercial Unix machine.

All that was over 10-T and 100-T ethernet BTW.

Well designed X applications, particularly those using the SGI GLX extensions ran very well on local LANs. Poorly designed ones were nowhere near the quality of experience.

That skill was expensive. I think this is a big part of why so many struggled with X. A browser, for example, would not be comparable, pretty terrible.

Displaying web pages sucked, yet modeling a freaking car was amazing!

At the time, the X implementation of multi user graphical computing allowed for these and other combinations of users, machines, displays and applications.

It was well thought out multi user graphical computing!

The users could be local, as in a multi head machine offering two sets of keyboard, mouse, display, or they could arrive over the network.

Same as the Unix it was built on top of.

I think that is what people really are trying to get at when they say "network transparency"

Not all these use, or the set of possible use cases X made possible, make sense today.

But, when it was done?

Nobody knew, so they made a system as capable and that employed the best ideas in multi user computing, so that it was all possible.

That's why people who got to understand and really use X appreciate it as we see them do today.


I'm afraid I'm dreadfully confused by your comment.

"The typical answer I see is "you can send the same bits locally that you do over the wire", but nobody ever explains why that's even desirable for a network protocol."

Yes, you can use a network protocol to chat between local processes as well as between remote processes. Desirable? It's impossible to avoid...

"Transparent, distributed RPC was tried in the 90s and largely judged a failure."

True (ahem[1]), but irrelevant. X isn't a remote procedure call system.

"Not to mention that ever since the introduction of the MIT-SHM protocol, which is used by most everything to transfer image data, it hasn't even been true in practice for the last 20 years. Most modern enhancements to X11 have been about taking the X server out of the loop for as much as possible."

This is true; network speed hasn't kept up with graphics power and requirements for the most part, so processing has been pushed to the client. On the other hand, HTTP/HTML/JS/CSS can be seen as a poor-man's reimplementation of X (or more closely, of Plan 9 client-ui/server-processor protocols). And for the 20 years previous, clients were underpowered in comparison with a big honking machine you could log into remotely (or at least a dedicated machine that didn't have to bear the weight of a graphical interface).

"To make X11 usable over the network..."

X11 has been usable over the network since 1987 (and X < 11 before that) (https://en.wikipedia.org/wiki/X_Window_System#Origin_and_ear...).

"...even advocates of the principle usually use something like NX which is a network-designed fork of the protocol..."

I'm not sure what you mean be NX. The...

"NX technology, commonly known as NX, is a proprietary software application for remote access, desktop sharing, virtual desktop on Linux and file transfer between computers. It is developed by the Luxembourg-based company NoMachine which also lends its name to the software." (wikipedia)

which has nothing to do with X other than being descended from one of the low-bandwidth X extensions. (Not the low-bandwith X, and low-bandwith means dial-up. Plain ol' X was pretty chirpy on a SPARCstation with 10MB ethernet.)

"...multiple independent streams so we don't have reach head-of-line blocking with roundtrips..."

Welcome to the wonderful world of TCP.

"...and client-side rendering."

Once upon a time, there was a thing called a Sun Ray (https://en.wikipedia.org/wiki/Sun_Ray), basically a (non-3d) graphics card glued to an ethernet card, that was used as a thin client for a Sun server. I didn't see one until after I'd used an IBM X-terminal (a smallish pizzabox that ran the X server and pointed everything else at a bigger IBM box) for several years and the latter was much better than the former.

[1] Protobuffs, et al., notwithstanding.


I'm saying that "a local protocol that can function as a networked protocol" isn't always an ideal design decision. We should consider designs that allow different local and remote protocols, and consider the tradeoffs in each space separately.

> Yes, you can use a network protocol to chat between local processes as well as between remote processes. Desirable? It's impossible to avoid...

Using a network protocol to chat between local processes comes with a lot of drawbacks. UNIX local sockets support features like datagrams and FD passing, and of course you can communicate resources in a shared namespace (e.g. a filesystem paths). So limiting yourself to the lowest common denominator of a TCP socket restrains the kinds of tools your protocol can make use of.

> X isn't a remote procedure call system.

It has requests, and it has replies. Xlib will block on a reply by default. That's the building blocks of a classic RPC system. It has all the messiness of distributed systems, but none of the innovations from the past 40 years. Error handling is particularly gnarly in Xlib; since all resources are global, another client can stomp on or delete your window behind your back. You can't be sure of anything before you try, and get an error back. In theory, you can make a lot of requests without blocking, but that's a super annoying process and isn't easy to do in practice; lots of information requires getting a reply back from the server.

> which has nothing to do with X other than being descended from one of the low-bandwidth X extensions

It has a lot to do with X11! It's championed as an example of the success of X11's "network transparency", but instead is very much a custom design suited for the network.


> Xlib will block on a reply by default

> Error handling is particularly gnarly in Xlib;

I don't think anyone strongly objects to the consensus that Xlib is a bad binding of the X protocol for these reasons. But libxcb has been usable for quite some time by now.


I keep reading what you write, and also was there (I used X10 at one point, and built the Athena platform from 9-track tape). I dunno...the whole screed strikes me as bizarre.

You say things like "Xlib will block" and "error handling is particularly gnarly in Xlib", when Xlib is only one possible implementation of the X protocol. There are other possibilities like XCB. I have no idea if it has the same issues as Xlib, but the inability to separate protocol from implementation speaks to a specific bias.

Further, you obsess over the use of the term "network transparency" like a previous generation used the term "commie", even cite up-thread "documentation" where the term isn't used. But no less that Jim Gettys used it to describe X11: https://static.usenix.org/event/usenix02/tech/freenix/full_p.... You might find it inappropriate, but it doesn't seem to be confounding vernacular to most other commenters.

I dunno...I think we've gone beyond butthurt because people didn't agree with you on the direction a project should go, to some obsession that has nothing to do with the technology.


> Nobody has ever explained to me what it means, what its goals are, what constraints it has on the system or the protocol

You answer it later on. They want something like NX. We can fill in the details as one pleases.

> We know how to design much more resilient systems than that.

I think you are overstimating the progress of computer/network engineering. Anyway, CORBA's "transparency" might be a thing of the past, but what's the point here? Unless you plan to make a graphics API explicitly asynchronous, what alternatives you have? Sending PostScript over the wire? :/

The users want graphic primitives sent through the wire. There is not much alternative other than CORBA style RPC.

> Not to mention that ever since the introduction of the MIT-SHM protocol, which is used by most everything to transfer image data, it hasn't even been true in practice for the last 20 years

That is not true. See how many commercial Qt engagements were stuck with Qt 4 because Qt 5 initially forced to use Qt's local rendering and only pushing the final pixmap to the server. Multi-million CAD machines slowing down to a halt because the clients were suddenly sending megabits through the pipes * . Eventually they had to backpedal and Qt5 now offers coreX/Xrender rendering too. _Most_ toolkits _to this day_ still render with X by default, and people do notice when this changes (see Mate). Sure there are exceptions and likely amongside the most used software (e.g. browsers), but I'd hardly claim "anything but MIT-SHM is all that matters for the past 20 years".

> is a network-designed fork of the protocol,

Yet compatible (to a high degree, at least). Up to the point the "transparency" illusion is maintained.

> Problems like remote desktop and streaming can be solved by other solutions

The biggest problem is that once you drop the codeflow on most opensource programs that renders using X, it's code that's rotten forever, and this entire method of "network transparency" will never be workable again in the free desktop world. Everyone and their mother will draw in a different way.

We'll have to resort to full video streaming or the like that have their own share of disadvatanges. There is absolutely no technical reason behind this, it's just a "code is done by volunteers" problem.

* And please don't say VNC/video here, since this is CAD. It's mostly a several megapixel black bitmap with a couple million white lines rendered on it. Relatively easy for a "drawing primitives over the wire" solution, insane for anything that tries to encode it as realtime video.


>See how many commercial Qt engagements were stuck with Qt 4 because Qt 5 initially forced to use Qt's local rendering and only pushing the final pixmap to the server. Multi-million CAD machines slowing down to a halt because the clients were suddenly sending megabits through the pipes

Is there a bug report for this in the Qt bug tracker that you can share?

>It's mostly a several megapixel black bitmap with a couple million white lines rendered on it. Relatively easy for a "drawing primitives over the wire" solution, insane for anything that tries to encode it as realtime video.

I think your math is off. In the X11 protocol, each line segment is transmitted as 8 bytes. Transmitting two million white lines uncompressed is going to take up 16 megabytes. Do that 60 times a second and now you have a bitrate of 960MB/s. Compare this to youtube's recommendations which suggest a maximum bitrate of around 10MB/s for 4K HDR 60FPS video: https://support.google.com/youtube/answer/1722171

To me, the compressed video solution is the clear winner.


> Is there a bug report for this in the Qt bug tracker that you can share?

"A" bug report? There is like tens of them. A 5 minute search reveals many:

Main bug:

* https://bugreports.qt.io/browse/QTBUG-56851

Similar bugs:

* https://bugreports.qt.io/browse/QTBUG-67170

* https://bugreports.qt.io/browse/QTBUG-47069

Bugs already filed back in qt4 release when the default was changed (but the x11 code could still be enabled):

* https://bugreports.qt.io/browse/QTBUG-19636

* https://bugreports.qt.io/browse/QTBUG-25761

People complaining whenever the new X11 render path gets broken on qt5:

* https://bugreports.qt.io/browse/QTBUG-70387

In stackoverflow, plenty:

* https://stackoverflow.com/questions/22017200/is-there-a-way-...

* https://stackoverflow.com/questions/28782162/serious-perform...

Bringing back the option to draw using X:

* https://twitter.com/peha23/status/1037294277859131392

* https://sourceforge.net/p/qwt/mailman/message/36418978/

> compressed video solution is the clear winner.

This is enlightening, but it is still comparing apples to oranges, or in this case, compressed to uncompressed. The cost of encoding video cannot be disregarded. And for starters, even if you use hardware encoding, they hardly do any type of variable bitrate; you are likely to get almost the same bitrate even if the screen only changes slightly (good luck sending partial damages to a hardware encoder). On the other hand, NX compresses all traffic by default, and so does SSH (but not by default).

Not to mention that compressed video at this rate would show artifacts. Even at bluray rates artifacts can be seen in motion film; let's not even get started what you would see for this type of content.

Do note that I am highly biased here since we (my company) constantly evaluates potential solutions almost every other week to do remoting (e.g. RDP/extremeFX, PCOIP, Blast, you name it) , and _every_ _single_ _time_ customers end up falling back to an ANCIENT NX version (cause the newer ones also use video "sometimes") which is so full of bugs I consider it the a modern equivalent of torture. (By the way, xpra, freenx, and every other open source NX fork are also the same -- ancient NX 3.xx versions). I really would like to kill that with fire but there's absolutely nothing comparable in terms of (in)sensitivity to high-latency links, low bandwidth usage, image quality, and even economy (no GPU video encode farm thanks).


Thank you for those bugs. They seem to further the point that X11 is not network transparent, seeing as how the solution was for Qt to support two separate xcb backends, one for local rendering and one for remote rendering. And actually a third for local hardware rendering using GL.

I have no comment on NX, I've never used it.


Qt _needs_ a non-X11 backend, seeing that it runs on platforms where X11 is not prevalent. Network transparency is another story and not related whatsoever to this since even if you push bitmaps X11 is still technically network 'transparent' (the quotes are due to the increased bandwidth usage).


Those 3-M machine don't have enough system memory to render every window to an off-screen pixmap and then compose all the windows onto the framebuffer. Each window (even those entirely occluded by other windows) might need up to 1MB of RAM for the pixmap (for a full-screen window on an 8-bit display).

So the solution (which isn't needed anymore) is to render everything directly to the framebuffer. That means you only need num_pixels * bit_depth worth of bits on the X server, but it also means that the X server has to discard the contents of any part of part of a window that you can't see (i.e., the parts that are occluded by other windows "on top").

So when the user moves or closes a window (and reveals another window below), you don't have data for the newly-visible pixels. That's why X has a mechanism to keep track of which areas become un-occluded, and notify the client that it need those pixels again.

Wayland and Mir are born in a world that generally doesn't have those constraints. We have dozens of GB of system ram, and GB of graphics memory today. Even a $25 Raspberry Pi has gigabytes of memory. So it's no problem to give 50 MB of system memory to every window (or, heck, even GPU memory) for use as an HDR+alpha 4k pixmap, and then compose those on the GPU (without using any system memory bandwidth). The application can just write to the pixmap whenever it feels like it, and let the compositor handle the rest.

Even back in the mid-1990's (so, 10 years after X), low-end SGIs and Suns with 24 bit graphics often shipped with a little as 64MB of system memory. Composing off-screen pixmaps becomes a less-obvious architectural choice when each large window costs you almost 1/20th of your system memory. By the time you open your mail, browser, terminal emulator, a couple "real" applications, and the desktop background, you're spending 1/3rd of your memory on pixmaps.


Yes...this is one of the key innovations that X had that probably isn't so useful now. Efficient use of memory was critical. In the mid-1990s, low end Sun's could actually ship as a supported config with 24-bit video and as little as 20MB of memory (16M system ram + 4M vsimm) like the the SX/CG14 in the SS20. There were 24-bit cards for earlier, mc68k Suns that could run in even less, but were more for rendering and of questionable utility for interactive use.

Another thing this efficiency allowed was the X Terminal. It was something akin the a serial terminal but running X, modest processor and memory, diskless (some had floppies for file transfer), usually running a couple of built-in local clients (xclock, etc), and costing far less than a full workstation. Most of them I recall being either monochrome or 8-bit color, but around this time I had an NCD X Terminal with 24-bit graphics, a MIPS processor and 16M ram + 8M vram, and network (streaming) audio for about a fraction the cost of an SGI Indy. Nice machine.


Some of our university computer labs in the mid-90s were set up this way: a room full of X Terminals that you could use to connect to the big Alpha machines in the datacenter, or the Sun machines if you needed the software that ran on them, etc. When Linux with X came around, it was awesome to be able to dial into the modem bank and get connected to these machines from home.


I deployed a couple of thousand NCD xterms at a Very Large airline backed by a cluster of very large Sun machines running a customer service application. Would have been late 90s. Very successful once the network performance issues were wrung out. Good times.


I remember working in a lab full of X terminals. They were NCDs, now that I recall. They all ran off of DEC Alpha or DECstation servers. That environment was pretty snappy.


Solution: X11-over-websockets with display in Javascript.

(Actually now wondering whether this nonsense already exists?)


Xpra has a way to do this: https://xpra.org/trac/wiki/Clients/HTML5

There's a similar thing for Wayland too: https://github.com/udevbe/greenfield

And GTK has yet another way to do this built-in, that uses its own protocol: https://developer.gnome.org/gtk3/stable/gtk-broadway.html


X also solved problems for a future that never happened. The whole client/server design was originally intended for a world where compute and storage were much more expensive than network bandwidth. I remember in the early 90s it was just kind of received wisdom that networks were going to get a lot faster. They didn't, processors did.

Which isn't to say it isn't great that X makes it blindingly easy to put a window on someone else's screen.


And yet, there is this newfangled streaming of videogames over the net...


I guess the first question is how much people spend.

Millennials spend a fair bit it seems.

$112 a month

https://venturebeat.com/2019/06/06/superdata-millennials-spe...

This makes a console between 2 and 4 months of gaming budget.

A network focused console still requires sone hardware or runs on a pc that costs more than most consoles.

The big benefit would appear to be killing the secondary markets for used games dead which is a hard benefit to sell to your users. It also only saves you money up front as they pretty much must charge more than a console over time to cover their own costs.

Then you have the problem of latency which can only be rendered slightly less crappy with substantial hardware at the isp, unpredictable demand hello covid, and annoying Netflix like account sharing.

The real problem is that the more money your target audience has the better off they would be buying a desktop pc. Nothing up front sounds great to someone who can afford to pay 20 a month but then you can't eat. If someone can pay 99 why not buy a Playstation or pc?

I predict it's a bad idea whose time will never come.


For which X is entrely inappropriate and needs a completely different approach.


So I went back and thought about whether you could extend the X approach to streaming of 3D games. I came up with the following walls:

* Licensing issues * 3D games can max out the PCI bus, network traffic wouldn't stand a chance * How the heck do you keep up with the frames on less powerful hardware? * You'd need a properly portable 3D graphics API

So the White Queen would probably think it's fairly doable before breakfast.


I remember my undergrad graphics class in 1991 or so. We did it in C and X11 on Sun and DEC workstations, and we had to learn X as an implementation detail. Our final project was animated 3-d wireframe models of objects, written in Xlib primitives. No toolkits or modern libraries. One of the assignments was a "game", and I wrote a tetris-like typing game. The projects in that class were insanely hard compared to any other class I'd taken.

The amazing thing is that, at least as of 10 years ago, my projects for that class STILL compiled on 32-bit FreeBSD and Linux.

Oh, and to learn X11, we used https://www.amazon.com/gp/product/0134999975/ref=x_gr_w_bb_s... I still remember sitting on a bench outside the college bookstore, cracking that book open and feeling both excited and hopelessly lost.


> The amazing thing is that, at least as of 10 years ago, my projects for that class STILL compiled on 32-bit FreeBSD and Linux.

i686 binaries from the 90s compiled for then-Linux and linked against then-xlib still run on AMD64 Linux hosts (provided you have the necessary multilib libraries around).


I wish hacking on X and getting patches in had the same coolness factor as hacking on the Linux kernel. The hacking complexity is certainly comparable. X is used by a very large number of people, and contributions can make a big difference.


There's still a good number of graphics hackers, but a lot of code has moved out of Xorg and as a result most of the hackers have moved on to working on Mesa, Wayland, and in-kernel GPU drivers. So there is some overlap there with the kernel hackers.


FWIW i played around with Xorg xserver code a while some time ago and i found it quite easy to grasp things in it... or at least not any more difficult than any other relatively large codebase i've worked with (though the X server doesn't really have a large codebase and the entire Xorg project is broken into multiple small libraries that make things easier to grasp).

My guess why you do not see as many people working on Xorg is that... it generally works for the majority of people. Personally i'm 100% certain i could fix any issue i have with it, but i never really felt anything missing from Xorg, it works perfectly fine for what i'm doing with it.


The difference is that (sadly) no one seems to care much about desktop Linux. Only Android this Android that Android those...



Really recommend that speech as well since it's give good understanding what's wrong with X11 and why Wayland was created.

The real story behind Wayland and X:

https://www.youtube.com/watch?v=GWQh_DmDLKQ




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: