Objective: Quantify, predict, and enhance squad-level shared situational awareness (SA) and understanding (SU) across volatile, uncertain, complex and ambiguous operating environments leading to demonstrated increases in mission effectiveness.
Tactical Awareness via Collective Knowledge (TACK) uses an opportunistic sensing approach that exploits gaze and physiological data to provide real-time estimates of human SA across heterogeneous teams. Dynamic estimates of Squad SA, combined with other Soldier-borne sensors, can be used to reason over the environment and inform adaptive systems. Currently, TACK uses gaze position and passive neural (EEG) data from multiple individuals to extract Regions of Interest (ROIs) from the environment to share across heterogeneous teams. Computer vision (CV) is used to integrate ROIs with physiology to provide real-time estimate of Soldier state and Squad SA.
Advanced Computational Approaches
Under ACA we developed a deep-learning algorithm to detect the discrete neural responses that occur when humans visually perceive an object of interest, or stimulus relevant to the current objective. Importantly, we linked neural and eye-movement activity to enable application in naturalistic viewing environments. By simultaneously combining the outputs from multiple individuals, we built a common representation of the environment that can be shared across teams or with autonomous agents
Brain Computer Interaction
Under BCI we developed the expertise and software tools for real-time exploitation of physiological signals within virtual environments.
Gaze Behavior During Navigation and Visual Search of an Open-World Virtual Environment
Leah R Enders, Robert J Smith, Stephen M Gordon, Anthony J Ries, Jonathan Touryan (2021) Frontiers in Psychology