Perceptual Realities: Optimizing XR through Perceptually-Informed Experiences

ongoing

Extended Reality (XR) technologies—including Virtual Reality (VR), Mixed Reality (MR), and Augmented Reality (AR)—are rapidly evolving, offering immersive experiences that blend the physical and virtual worlds. While progress has been made in understanding basic perceptual and behavioral processes to create high-quality, engaging content, current perceptual models often oversimplify how users experience these environments. To create truly immersive and effective XR experiences, we need a deeper understanding of how users perceive complex, highly realistic immersive scenarios. Whether to optimize visual fidelity, manage attention, or improve system performance, human perception plays a central role in the XR experience. PROXIE aims to address this by studying perception in XR environments that mirror the complexity of real-world XR applications, pursuing ecological validity. Rather than focusing on simplified, constrained scenarios as in existing research, we will explore how perception is shaped by both the environment and the tasks users perform. By understanding how users process multisensory inputs while actively engaging in realistic, interactive tasks such as search or free exploration, we can inform content generation, visualization algorithms, and even hardware design to improve XR experiences. Our goal is to develop computational models of perception that predict how users perceive and respond to immersive environments—whether fully virtual or augmented—based on sensory inputs and tasks. These models will anticipate how users process their surroundings, allowing XR systems to adapt in real time. For example, the models may help optimize visual rendering, sensory feedback, and the overall user experience to match user needs. This will enhance applications such as telepresence, training, and simulations, ensuring XR experiences feel natural and responsive, bridging the gap between laboratory experiments and real-world use.