Embodied interaction based auditory digital twin and application in UAV

DOI:	10.11809/bqzbgcxb2023.12.034
Keywords:	digital twin; military virtual simulation; extended reality; embodied interaction; spatial audio
Abstract:	Auditory digital twin technology applications are essential in extended reality, military virtual simulation, and game interaction to achieve an immersive audio visual experience. Spatialized rendering of audio must be consistent with visual feedback and minimal latency response time to user actions for embodied interaction. The volatility of sound poses a serious challenge to realistic embodied interaction and immersive rendering, as well as significant computational costs. From closed to open, static to dynamic, virtual scenes in mixed indoor and outdoor areas to an extended reality experience that blends reality with reality, there is a dramatic increase in computational complexity. This paper takes an audio perspective to realize a “human centered” mapping relationship between a simulated listener and a digital twin, describes the interaction between the listener and the virtual environment through an auditory digital twin, establishes an auditory digital twin system for UAV combat scenarios based on embodied interaction, realizes spatial audio twin and multisensory embodied interaction under extended reality experience, and discusses the application in UAV combat scenarios.
Issue:	Vol. 44 No. 12 (2023)
Published:	2023-12-28
PDF HTML