Multiple Mouse Cursors in Wayland

QuaternionsBhop

I've always thought this was a cool idea. It's obviously been on the minds of Wayland protocol developers since the beginning, otherwise it wouldn't be possible!

ben-schaaf

To add to the long list of caveats: This also breaks assumptions GUI toolkit and applications make about only one of their windows being focused at a time: `gtk_application_get_active_window`, `QApplication::activeWindow`, etc.

McGlockenshire

The original title, "State of multi-player Wayland" is a much better summary of the article. This ain't just about the pointers! In fact, the author is very explicit about that: > I think there could be a term that’s describing this concept even better – in the title of this post, I picked “multi-player”, to make clear that it’s mostly about multiple people who are collaborating. I’ve also seen “multi-cursor” or “multi-pointer” (which are missing the keyboard component).

evertheylen

It's not as weird as you might think: laptops may have touchscreens (which can be multi-touch) on top of the "normal" cursor. IIRC, X.org can't really handle such cases cleanly.

nomel

Another possible use case might be mixed reality/VR displays, for simultaneous mouse cursor + eye tracking/control.

kelnos

I'm building a compositor now, and I've been thinking about multi-seat from the beginning, but I know there are lots of places that assume a single seat, and so this sort of thing won't work for me. It's actually a lot more work than you'd think to support this...

jchw

On the note of the two different kinds of seats, a tangent: better multi physical seat support is something I am passionate about, specifically, I want Windows-style "Fast User Switching" on Linux with RDP. Today's Linux RDP support usually acts like screen mirroring; you get a mirror of a physical seat. With GNOME I believe there is also now support to do remote login: you can log in remotely, and even disconnect and reconnect to a remote login session. But what you can't do is continue a physical session remotely or a remote session physically, and the reason why is sorta to do with seats. Sort of. See, when you start a graphical session on modern systemd-based Linux, a new logind session is created, and it is immutably tied to a seat. So you can't just remove the seat from a running logind session and then make it remote, or add the seat to a running logind session and make it physical. I actually tried patching systemd-logind to do this, but when I proposed it upstream, Lennart Poettering himself actually took the time to explain that this wasn't the intended way to do it: instead, you would actually just create a new logind session and drop the old one entirely, or literally whatever you want. Which to me sounded perposterous, but then I realized that it really wasn't that ridiculous. As it was, when you boot a desktop using systemd-logind, the desktop itself doesn't typically run much of anything directly in the session context anyway - the compositor and almost all services actually run as user services, as in systemd-land having more than one active desktop session per user isn't really supported. Okay so we can do this! And so I set out to prototype this with KDE, but there are still a few issues... - First and foremost, $XDG_SESSION_ID. Believe it gets set by the systemd PAM module somehow? I don't fully recall. Either way, it gets used by various programs to determine the current logind session, but if our graphical session can "roam" between them we need a mechanism that can handle changing over time. I propose we stuff the current logind session ID into a file in $XDG_RUNTIME_DIR instead. - Secondly, compositors need to be redesigned to handle this. Compositors today can hotplug displays and input devices - that's a good start. Unfortunately, at least kwin can't switch backends at runtime. You can really go one of two ways, and I tried a bit of both and either can work but both are a bit ugly: one is to make a single backend that can handle either case and the other is to allow switching the backend at runtime. Either way, when running remotely the card* DRM nodes are unavailable so you need to switch to renderD* nodes instead. You can also use both, and only use card* for scanout when available, but it seems a bit trickier. - Finally, I genuinely have no idea how this could or should be handled in non-systemd environments. I have admittedly forgotten how that all works. It would be rather lame to simply require logind for this to work, but sadly it is the path of least resistance at least at first. And maybe some more, but otherwise the concept does indeed broadly work. I mainly have only tested it in virtual machines with DRM native context, but I can see everything working switching in either direction at least once before things start to get screwed up. Even OpenGL clients and audio playback can move seamlessly! It's just like what Windows accomplished over 20 years ago! Aside from convenience and being cool, it would also solve a couple problems. No need to worry about the limit of only one session per user: just move your session with you. No need to worry about your physical session being unlocked when remotely connected: when you log in this way your physical seat gets disconnected from the session entirely and the display manager takes it back over. I actually would really like to at least make some progress on ground work that would be needed to make this happen, although I'm not really that sure that anybody cares. But either way this post's brief discussion about physical vs logical seats activated a few neurons so I couldn't help but dump this out.

jauntywundrkind

On the web, PointerEvents and touch events both support multiple different cursors. https://developer.mozilla.org/en-US/docs/Web/API/Touch_event... https://developer.mozilla.org/en-US/docs/Web/API/PointerEven...

Dwedit

Focus of controls is the hard part. You end up needing a Per-seat version of focus in an application.

zImPatrick

cant read the entire article on ios because the left part is just somehow cut off :(

ofalkaed

I have never liked split keyboards but I might if each keyboard was also a mouse with its own cursor, it seems fun to develop the skill to work that way. Would also be useful at times if the the Thinkpad trackpoint and trackpad could each have their own cursor and I could leave one somewhere. Or maybe a cursor stack like pushd/popd, registers would probably work better; instead of virtual desktops just give me one massive desktop with cursor locations I can jump between, switch to cursor 3 it saves the location of the current cursor and jumps.

jumperabg

Imagine working with two separate mouses at the same time - no keyboard.

chmod775

The more "typical" use for a feature like this is drawing tablets and touch screens. You don't necessarily want a touch/drawing tablet input to move your actual "main" mouse cursor.

AuthAuth

I have been dreaming about a split keyboard with dual trackballs and two cursors for the past few months. This may be the kick i need to actually get on it.

bsder

Maybe RedHat should contract this guy to finally get pointer warp implemented for Wayland everywhere it needs to be.

kahf56

Now I know why Web devs, frontend developers hate us

RobotToaster

After using a space mouse for CAD it always seems weird how dual mice isn't a more common thing.

eighty8days

This feels like a step in the right direction from this talk on desktop UX. Finally some cool innovation here! https://youtube.com/watch?v=1fZTOjd_bOQ