An ugly hack has been reported within the Linux kernel that has been in the mainstream for over three years. Due to a buggy X.Org Server/xf86-video-modesetting DDX, the Linux kernel has been enforcing different behavior on whether a process starts with “X” and in turn disabling atomic mode configuration support .
Linux security researcher Jason Donenfeld (who is also known for creating WireGuard) stumbled upon this ugly code hack inside the kernel.
Donenfeld commented this weekend on the core mailing list:
This reverts 26b1d3b527e7 (“drm/atomic: remove atomic toys from X”), a rootkit-like hack that doesn’t have to be inside a general-purpose kernel. It’s the kind of debugging trick I’ll use momentarily but never commit, or some kind of malware trick to hide the first process from babies.
The backstory is that some userspace code, xorg-server, has a config DDX so it’s not actually coded correctly. Since no one wanted to maintain X11 anymore, instead of fixing the buggy code, the kernel was tweaked to avoid having to touch X11. A bummer, but fair enough: if the kernel no longer wants to support some userspace API, the right thing to do is organize an elegant fallback where userspace thinks it’s not available in a manageable way.
However, the *way* to do that is to simply check `current->comm[0] == ‘X’`, and disable it just for that case. That means it’s *not* simply a matter of the kernel no longer wanting to support a particular userspace API, but rather it’s the kernel not wanting to support the xorg server, in theory, but in reality, it turns out they are all processes start with ‘X’.
Playing games with current->comm like this is obviously wrong, and it’s quite surprising that this has ever been done.
the engage to this kernel with the first “X” character check was done in September 2019. The argument in that Linux kernel committed at the time was:
The -modesetting ddx has a totally broken idea of how atomic works:
– does not disable old connectors, assuming they are automatically disabled as with legacy setcrtc
– assumes ASYNC_FLIP is hardwired to the atomic ioctl
– not a single call to TEST_ONLY[In other words] the implementation is a 1:1 translation of legacy ioctls to atomic, which is a) broken b) meaningless.
We already have bugs on both i915 and amdgpu-DC that prevent us from enabling neat features.
If anyone ever cares about the atomic in X, we can easily add a new atomic level (req->value == 2) for X to retrieve the shiny toys.
Since these broken versions of -modesetting have been shipped, there’s really no other way out of this predicament.
The “good” news is that since then, on the userspace side in 2019, the xf86 video mode configuration code went ahead and disabled atomic support by default. So technically, if you’re running an X.Org stack updated within the last three years, this kernel hack is no longer necessary since user space is bypassing the atomic API.
We’ll see if Linus Torvalds agrees to this hack being removed since, after all, it goes against his “don’t break userspace” principle by rolling back the system if you stick to an X server stack. org outdated and wants to run with a future kernel version. We’ll see, but it’s surprising that it took three years to criticize this dirty code.
