DIgital DYnaMic and respOnsible twinS for XR

ended

The digital transformation and the availability of more diverse and cost-effective methods for 3D capture has led to the creation of digitised representations of parts of our public spaces, machinery and processes, so-called Digital Twins. These Digital Twins are an important basis for building advanced XR applications for cityscapes and the industrial environments. These XR applications need high-fidelity models for accurate localization, which are kept in-sync with the real world. However, currently (i) creating high-fidelity Digital Twins is costly, (ii) integrating data from heterogeneous sensors is still challenging and (iii) the update of Digital Twins requires explicit manual intervention. The vision of DIDYMOS-XR is to enable advanced, more realistic and more dynamic extended reality (XR) applications, powered through artificial intelligence. The project thus focuses on advancing technologies for creating large-scale digital twins, synchronised with the real world. DIDYMOS-XR will research and develop methods for reconstruction and mapping from heterogeneous inputs, including static and mobile sensors, AI-based data fusion, scene understanding and rendering. Digtal twin based XR applications also require means of accurately positioning in the environment, aware of the dynamics of the scene, also addresed by the project. The capture of scenes at scale, as well as using cameras and other sensor data for synchronising the digital representation, bears the risk of capturing personal and sensitive data. Hence, the technologies resulting from DIDYMOS-XR must be and will be ethical and privacy-aware by design. In order to demonstrate and validate the enabling technologies researched and developed in DIDYMOS-XR, the project will address use cases of XR applications in two domains that differ in their scale and characteristics. One is urban planning and smart mobility, the other is the collaboration with autonomous mobile robots at an industrial production facility.