For those wanting to build a Wayland-only Linux desktop experience without carrying any aging X11 baggage, GNOME 47 will be able to optionally offer Wayland-only support without carrying X11/X.Org support. This Mutter merge request landed today that allows compiling Mutter with X11 support disabled. That landed today along with this GNOME Shell merge request for being able to disable X11 support too.
I think that could be solved with a XDG portal
@possiblylinux127 That would definitely be part of it, I assume. Does Wayland already track text rendering and its contents?
Because somehow text from any UI would need to be detected.
The actual implementation would be per desktop. The desktop draws to the screen and then the apps connect to the desktop. We already have a window capture XDG portal that is used by things like OBS. We could huild a simular portal for just text on the screen. We would just need some way of either recognizing text or even better some sort of image to text engine like what is in Firefox.
Are you saying to use an image to text engine just for what are text fields in applications? That sounds horribly inefficient…
It would work for images as well. It doesn’t need to be exclusively used but sometimes text is rendered at a pixel level.