Developers from Collabora project , which, with the support of Valve, is developing a library with elements for interacting with traditional desktops inside three-dimensional environments formed using 3D glasses and virtual reality helmets. The library code is written in C language and under the MIT license. Ready builds for ΠΈ .
Currently, for Linux, there are already tools for direct output to virtual reality helmets (Vulkan extensions VK_EXT_acquire_xlib_display for X11 and VK_EXT_acquire_wl_display for Wayland), but there is no support at the level of correct rendering of windows in 3D space and synchronization of the screen refresh rate. The goal of the xrdesktop project is to develop methods that will allow classic XNUMXD screen-oriented, keyboard-and-mouse-oriented interfaces to be used in virtual environments.
The xrdesktop components extend existing window and compositing managers with the ability to use virtual reality runtime systems to draw windows and the desktop in 3D virtual environments. xrdesktop promotes the idea of ββintegrating into existing desktop environments without the need to launch a separate dedicated compositing manager and allowing existing custom configurations used with a conventional monitor to be used with XNUMXD helmets.
The project architecture implies the ability to integrate with any desktops, but at the current stage of development, components for supporting virtual reality helmets are implemented for KDE and GNOME. For KDE, support for 3D helmets is implemented through a Compiz-like plugin, and for GNOME through a set of patches for GNOME Shell. These components mirror existing windows to the virtual environment of the 3D helmets in the form of a separate scene or in an overlay mode, in which desktop windows can be superimposed on other running virtual reality applications.
In addition to the rendering engines, xrdesktop provides components to enable navigation and input using specialized spatial controllers such as the Valve Index and VIVE Wand. Xrdesktop generates normal input events based on information from VR controllers, simulating the use of a keyboard and mouse.
xrdesktop includes several libraries that render window textures for the VR runtime using OpenVR, as well as an API-based system for rendering a full desktop in a 3D environment. Since xrdesktop does not provide its own window manager, integration work with existing window managers is required (xrdesktop can be ported to any X11 or Wayland window manager). On the graphics driver side, it requires a driver that supports the Vulkan API and the VK_KHR_external_memory extension.
The main components of xrdesktop:
- - glib wrapper for Vulkan, providing classes for handling devices, shaders and texture initialization from memory or DMA buffers;
- - API for abstracting programming interfaces for developing virtual reality applications. Currently only OpenVR is supported, but support for the OpenXR standard will be added soon;
- - a library for synthesizing input events, such as mouse movement, clicks and key presses, implemented in the form of backends for xdo, xi2 and Clutter;
- - a library for managing windows in a 3D environment, a set of related widgets and backends for rendering a scene;
- ΠΈ - plugin for KWin for integration with KDE and Plasma applet for putting KWin into output mode on a 3D helmet;
- ΠΈ - a set of patches for GNOME Shell to integrate support for xrdesktop and an add-on for switching output to a 3D helmet in GNOME Shell.
The project supports several methods of organizing interaction with the desktop and windows in a virtual environment, which can be used to capture windows, scale, move, rotate, overlay a sphere, pin and hide windows, use the control menu, and control two hands simultaneously using multiple controllers.
Source: opennet.ru