Researchers develop innovative Soldier decision-making system

By U.S. Army CCDC Army Research Laboratory Public AffairsOctober 27, 2020

Army research allows warfighters to retrieve relevant information quickly in support of mission-critical decisions
Geographically-distributed users, which researchers depict as a gray avatar, collaborate in a shared virtual environment populated with mission graphics and interactable objects.
Geographically-distributed users, which researchers depict as a gray avatar, collaborate in a shared virtual environment populated with mission graphics and interactable objects. (Photo Credit: U.S. Army) VIEW ORIGINAL

ADELPHI, Md. -- Army researchers developed a system that populates immersive environments with complex, heterogeneous data, allowing warfighters to retrieve relevant information quickly in support of mission-critical decisions.

Inspired by the exciting potential of Cross-Reality, or XR, many scientists and engineers throughout the Department of Defense, commercial and academic spaces have developed remarkable systems to explore what is possible in the XR domain, said research psychologist Dr. Mark Dennison, with the U.S. Army Combat Capabilities Development Command’s Army Research Laboratory.

This work by the laboratory is unique in that it is one of the first to focus on creating an XR common operating environment that enables interoperability between commercial, off the shelf XR systems and program of record systems, heterogeneous sensors and big data analytics over a secure, encrypted network.

Augmented-, mixed- and virtual-reality--collectively known as cross-reality--present an opportunity to redefine how we interact with computers and information, Dennison said. XR enables researchers to completely rethink how virtual elements can coexist with the physical world to create an information space that crosses the cyber and physical domains.

“This work aims to create a framework for enabling interoperability between XR technologies and big data analytics to accelerate decision-making and enable shared understanding between co-located and distributed users,” Dennison said.
An interactable display expands to show the richness of the information available in the AURORA system.
An interactable display expands to show the richness of the information available in the AURORA system. (Photo Credit: U.S. Army) VIEW ORIGINAL

Until now, there has been a limited amount of foundational and applied research into how human cognitive processes are impacted when engaging complex information in immersive environments.

By linking the Accelerated User Reasoning: Operations, Research, and Analysis for Cross Reality, or auroraXR, system with the Elastic Stack, a big-data analytics architecture used throughout the DOD, industry and academia, scientists and engineers are now able to populate immersive environments with complex, heterogeneous data.

“These rich datasets enable CCDC ARL scientists to investigate how complex decision-making processes are enhanced or suffer detriments in XR environments, especially in comparison to existing, fielded information systems,” Dennison said.

The system was developed by Stormfish Scientific Corporation, which is under contract with the laboratory.

A non-immersive interface connects to the AURORA virtual environment allows users to collaborate without wearing a head-mounted display.
A non-immersive interface connects to the AURORA virtual environment allows users to collaborate without wearing a head-mounted display. (Photo Credit: U.S. Army) VIEW ORIGINAL
“Military decision-making requires the convergence of large volumes of data from a wide spectrum of sources that cross domains,” Dennison said. “Much of this data may be irrelevant or redundant when attempting to filter down to critical elements necessary to answer a specific question. Therefore, the Army requires novel information meditation tools to assist human decision makers with identifying and retrieving relevant data quickly enough to support mission-critical decisions for which the windows of opportunity may be less than a few minutes."

For example, Dennison said, an Army brigade commander may need to leverage intelligence, surveillance and reconnaissance, or ISR, assets from the Air Force while simultaneously tracking Army ground units co-located with Marines.

“This information space is only exacerbated by gray asset data, such as social media feeds, which may dynamically change mission planning within complex urban terrain,” Dennison said. “Technologies that identify, transmit and display only that information which is relevant to a particular unit’s mission are critical for Combined Joint All-Domain Command and Control, or CJADC2, decision-making.”

This research addresses the lack of a secure XR networking solution that meets DOD networking and security requirements, and provides a near turn-key solution for linking foundational research to prototyping systems.

This effort also supports the Adaptive Cross Reality Information Mediation, or AXRIM, task under the Robust, Resilient, and Intelligent C3I, or RRIC3I, project in the Army’s Network Cross-Functional Team.

For its contributions towards this ARL-funded research, Stormfish Scientific Corporation celebrated its selection as the 2020 Public Sector Elastic Search Award Honoree within the Cluster Award (Technology Innovation) category.

This work shapes the Army’s understanding of how an immersive common operating picture will augment decision-making in the mobile, mounted and command post computing environments across echelons, Dennison said. Researchers are eager to continue working in this area to operationalize science for transformational overmatch and provide Soldiers with unmatched capabilities on the battlefield, he said.

Visit the laboratory's Media Center to discover more Army science and technology stories
(Photo Credit: U.S. Army) VIEW ORIGINAL

CCDC Army Research Laboratory is an element of the U.S. Army Combat Capabilities Development Command. As the Army’s corporate research laboratory, ARL is operationalizing science to achieve transformational overmatch. Through collaboration across the command’s core technical competencies, CCDC leads in the discovery, development and delivery of the technology-based capabilities required to make Soldiers more successful at winning the nation’s wars and come home safely. CCDC is a major subordinate command of the Army Futures Command.