A Neighbourly Solution to the 'X is Deprecated?!' Conundrum
29 Oct 2020
Recent posts about the state of Xorg and its future has been stirring the internets as of late and, as almost always, it is best to stay clear of the comments sections. The more insightful post is On Abandoning the X Server.
There are a few details in it that should be emphasised before we move on.
Though the code happens to implement an unfortunate
specification, the code itself is quite well structured, easy to hack on, and
not far off from being easily embeddable.
From my own experiences with Xorg internals, I agree completely. A whole lot of the code there is noticably better than corresponding paths in certain Wayland compositors. There is more thought; domain expertise; engineering and pure elbow grease behind it than you might have been led to believe -- if you have only listened in to the collective moans in various discussion groups.
So is Xorg abandoned? To the extent that means using it to actually
control the display, and not just keep X apps running, I'd say yes. But xserver
is more than xfree86. Xwayland, Xwin, Xephyr, Xvnc, Xvfb: these are projects
with real value that we should not give up. A better way to say it is that we
can finally abandon xfree86.
There are some nuances here that many will miss as it requires you to know something about the architecture of the X server. The main thing is to not conflate ‘xfree86’ with rest of what you think of as X, hence why the blog post separates out XFree86-the-project from 'xfree86 the hardware driver'. It is unfortunate that the predecessor to Xorg was also called XFree86. Naming things is hard and all that.
If you look at the xorg code, recommended reading for many who can understand the language, you will see that the device-dependent code used to drive displays (hw/ in the source tree) has a few backends, mentioned above. Venture into the hw/xfree86 part and you will hopefully see why Adam Jackson and others deserve the software engineering equivalent of a Purple Heart for their service to your desktop, possibly alongside anyone that ever had to work on ASN.1.
Rightfully throw Xorg-xfree86 into the fires of Mt. Doom. If you still need Xorg-xfree86 to be your graphics card driver, you have bigger things to worry about, such as plain old bitrot.
Is there something else? Yes. I’m not going to say Arcan – it is very likely that there is only a small percentage of the stakeholders we would ever get along with. That is fine. The goals and agenda reach much further than many will ever care to travel -- unless you really want to push the boundaries of computing, or you are actively targeted by nefarious individuals; our funding is not based on popularity or mass adoption, but rather sticking to the shadows.
The Compromise
So what can we do instead? The current Wayland maintainer has libliftoff for smoothing over the rather ‘unergonomic’ APIs that are used to get the open, modern, graphics stack up and running. Write a DDX backend that uses it. hw/liftoff!
This will get liftoff the lift-off and mileage it needs to become good enough for Wayland compositors to use as their default, and as a slightly more polite migration path in order for NVIDIA to be less contemptible in the unlikely event that they are one day struck by some capitalist version of religious insight. Thus, it improves parts of the Wayland compositor situation that is, politely put, chaotic.
Those that don’t care about what Wayland tries to achieve can stay with X and not worry about their graphics breaking after a kernel update, or well, more than usual -- Linux gotta Linux. NVIDIA blob users can continue down that path and stop bothering Wayland developers, yet still run their Steam games without submitting to open source ideals.
Next steps. This is for the window managers. Take libarcan-shmif-server and embed as a module in Xorg, fork it off even. If you are unaware, this is the IPC system part of Arcan. Both the server and the client sides of it are fairly trivial to use and are written with developers, not toolkits, in mind.
It is beyond feature parity with the rest of X, but has a comparable view on capabilities and division of responsibility. You will only get a fraction of the benefits of Arcan as a display server, but without other dependencies or friction - many of the problems that X clients are plagued with can be worked around while leaving the X11 protocol and its family of extensions to rest. Hacks relying on facilities like uinput can be put down.
The security around these clients will be much tighter and they can be gradually tuned to the specification of the developer. It lets XFCE, AwesomeWM, WindowMaker and the tens to hundreds of others WM projects to continue to improve their respective ecosystems, specialized widgets and other tools, rather than to burn resources they don't have to rewrite themselves with bugs that won't be discovered or fixed in time to matter.
It also gives the conservative side of BSDs a portable way of improving their use of the graphics stack without reworking fundamentals they care little about. The OpenBSD fork 'Xenocara' can be dropped.
This way, the heritage is preserved and kept alive, with a band aid to live out another decade or three. It will still work for the marginalized that have little interest- or option- in going elsewhere. It will be less painful to maintain and work with.
The Reality
Wayland only is not an option that will ever work across the board. It is not a replacement, it is a fundamental change of principles. Stop trying to market it as some kind of inevitable transition. There are strong differences that will not get smoothed over regardless of how many 'protocols' you define; Wayland is Policy over Mechanism, X11 is Mechanism over Policy. This is the real barrier. Not unsubstantiated claims of 'security'. It is a tectonic shift and no matter your feelings about that, we won't all be in the same country or continent anymore.
Heralding a 'Screenshot Protocol' and similar exercises as the fix that will bridge this gap will only serve to distract from the fact that the power of all these little tools and hacks from the crowd of X11 users comes from the 'screenshot' being just one case of 'give me the contents of a specific window and all of its children' or 'composite these windows to an offscreen buffer and send me the results' as the building blocks available to be creative with.
The mechanism approach provides a huge set of possibilities and opens up for experimentation and 'works for me' kind of hacks. The downside is that it surrenders control on the server side and are much harder to make robust.
The policy aproach will only do what both sides agree to, and that agreement has the final say. If you violate this contract, you are to be blamed. However, that contract has to be bikeshedded to near state-space exhaustion and carefully screened for conflicts as more contracts are added to the pile.
The current trajectory is Gnomelanders and Swaylanders and Kwinners and so on continuing to lock down singular uses cases and with political gunplay try to push that as the one solution that will convince a few more bread crumbs. The hand that controls the toolkit will eventually just decide. This will erode the strength of the policy contract.
This has already led to a substantial amount of dissonance -- it turns out that the 'opting-server-side decoration protocol' in the mix had implications for how to interpret the 'subsurface protocol' and the 'redshift gamma protocol' will clash with whatever 'color management protocol' that materialize and so on. To get a feel for how involved such policies become, just read the specification and you will see why no implementation arrives at the same calculation.
Take the perspective of a client developer chasing after the tumbleweed of 'protocols' drifting around and try to answer 'what am I supposed to implement and use'? To me it looked like like a Picasso painting of ill-fitting- and internally conflicted ideas. Let this continue a few cycles more and X11 will look clean and balanced by comparison. Someone should propose a desktop icon protocol for the sake of it, then again, someone probably already has.
In Conclusion
This approach will force Wayland to demonstrate its worth through the virtues of its properties alone or by inventing actually compelling features rather than by throwing shade on the elderly; or reimplementing the past through rebranded and traced outlines from shadows cast by grander window managers of yore.
The proposal leaves us with three well-defined paths.
- Wayland lets your own solipsist thiefdoms expand towards the horizon, while trying to save embedded from the Android expanse.
- People that have invested their hearts and souls into the X ecosystem can continue to use their computers without fear of a kernel upgrade leaving them with broken graphics, or a switch of display server paradigm leaving them with an incompatible mental model of how system graphics work, and perhaps, one day, stop whining about Wayland ruining their day and leave the devs alone.
- Arcan lets you play with crazy ideas at a low initial cost. The ones that might revolutionise your world or fail miserably. It will continue to pushing things towards whatever is beyond the Twilight Zone in order to answer profound questions such as 'what happens to productivity if your heartbeat aligns with the blink rate of the cursor' (well that's actually be done since forever and the RESULTS WOULD SHOCK YOU! - but it still illustrates the point).
It might even turn out so well that one of these paths will have a fighting chance against the open desktop being further marginalised as a thin client in the Azure clouded future; nothing more than a silhouette behind unwashed Windows, a virtualized ghost of its former self.