An exciting new development has emerged in the world of desktop environments, with the proposal of an experimental plugin for the KWin composite manager that transforms KDE into a virtual reality system. This innovative plugin allows users to create interfaces not on physical monitors, but in the form of virtual screens in 3D space, which can be accessed through augmented reality glasses or 3D helmets.
This groundbreaking plugin offers support for various features such as working with floating windows, combining physical and virtual screens, arbitrary positioning of screens in 3D space, control using a keyboard without the need for a mouse or VR controllers, and a mode for tracking the user’s position. To handle the rendering aspect, the module Qt Quick 3D Xr, linked to the runtime OpenXR, is utilized. The platform was tested in conjunction with various devices including the Monado platform with Rokid Max and HP G2 augmented reality glasses, as well as the WiVRn server with the Quest 3 3D helmet.
To facilitate user input, the implementation of a KwinVrInputDevice device has been introduced, allowing for the movement of the pointer akin to a mouse but not restricted to the boundaries of the output area. To position windows in 3D space, standard KWin capabilities have been modified slightly for pop-up windows and expanded to enable the movement of windows outside the output area. In order to operate effectively, patches need to be applied to Qt and XWayland. The majority of these patches have already been integrated into the Qt codebase and will be included in releases 6.10.2 and 6.11, leaving only 4 patches yet to be ported.