Intelligent & Autonomous Systems

Research concentrates on understanding and exploiting interactions between information and intelligent systems, such as software agents or robots. Information can be thought of as data in context. In order to fully exploit that data, the context must be taken into account. The data can then be used in providing automated intelligence: perception, reasoning, planning, collaborating, and decision-making. These broad issues in automated intelligence can be applied to a wide range of systems and environments, like cyber virtual environments or decision support. Aspects of Intelligent Systems complement research conducted in the Sciences for Maneuver Campaign, which focuses on Intelligent Systems concepts applied to vehicles or robotic platforms.

Human Information Interaction (ALC)

Solutions to improve systems for uniform battlefield information comprehension that support relevant and timely decision making using machine intelligence and (large and sparse) data-driven decision models that increase understanding of risk, improve situational awareness, and minimize cognitive burden.  Methods and algorithms that automate reasoning, learn and predict Soldier information requirements and preference, minimize information search, and accelerate human and machine decision-making.

Principal Investigator:

Adrienne Raglin,, (301) 394-0210

Natural Language Interaction (ALC)

Natural Language Interaction (NLI) focuses on theories, models, and technologies for natural communication between humans & systems. The two areas of focus are:


  • Natural Language Understanding: Leveraging linguistic features and capturing the semantics and intent embedded in language to support analysis, manipulation, and interpretation of communication between people, information, and software agents.

  • Natural Language Generation: Using NLP algorithms and models to create reasonable, relevant, and intuitive communication from computational systems for Soldiers

  • Information Extraction: Research in linguistics and machine learning towards an adaptable Analyst Pipeline of NLP tools for situational understanding.

  • Dialog Management Systems: Fundamental questions in Dialogue Management Systems that enable robots to act semi-autonomously and interact with Soldiers using natural language.


This work is involved in creating adaptable HLT for Army operations. It consists of applied research in multitask machine learning and Neural Machine Translation (NMT) for training automated speech recognition models for language sub-populations and military domain-specific tasks.

Principal Coordinator:

Reginald L. Hobbs, (301) 394-1981

Social Intelligent Agents (ALC)

Social Intelligent Agents (SIA) is about training software agents to utilize available data analytics, task details, and social context for inferring intent during human-agent collaboration for more effective teaming. The two focus areas are:


  • Using data science to model context and intent

  • Defining informatics techniques for determining value of information

  • Developing synergistic human-agent information exploitation capabilities


  • Using social sensing, social computing, social dynamics, and social network analysis to extract context in a multi-domain battlespace

  • Leveraging social models to capture knowledge of relevant behaviors, events, & tasks

  • Exploiting crowdsourcing and reinforcement learning methods to train software agents on inferring intent

Principal Coordinator:

Reginald L. Hobbs, (301) 394-1981

Joint Text and Video Analytics (ALC)

The planned program will develop methods for integrating (NL) text & video analytics to enhance speed and accuracy of situational awareness (SA). Currently, automatic text analysis and video analysis have been pursued independently even though the two forms co-occur frequently with respect to a given SA, resulting in pervasive inefficiencies and in loss of potentially exploitable data. Our methods will target problems of retrieving images or video by NL query, generating NL descriptions of images or video (referring-expression or summary generation), and jointly exploiting video and text when elements of both are present in the data, such as making use of one medium to help parse the other and making use of both media to construct integrated SA. Toward developing these methods, we will build new annotated, aligned, text and video datasets.

Principal Investigator:

Heesung Kwon,, (301) 394-2501
Clare Voss,, (301) 394-5615

Computational Intelligence (ALC)

Research seeks computationally feasible techniques and theories to ad­dress the synergistic integration of components of intelligent behavior, including perception, reasoning, planning/execution, and decision making. These behaviors can be individual or collaborative. This includes: (1) semantic perception and scene understanding investigating innovative concepts for recognizing and under­standing the world in which the system’s intelligent agents operate; (2) robot planning and behaviors in complex environments; (3) reasoning under uncertainty;

Principal Investigators:

Stuart Young,, (301) 394-5618

Reasoning and Decision Making Under Uncertainty (ALC)

Research in developing a method for analogical and deductive reasoning and decision making by automated systems such as intelligence analysis systems or battlefield robots. In order to deal with the inherent uncertainty that comes with military operations, this approach needs to be able to handle approximate and partial information. Our method includes both a rich common-sense knowledge base for deductive reasoning and a distributional semantic vector space trained on large volumes of text corpora for analogical reasoning at query-time. This combination gives the system the ability to answer questions approximately when full information is not available. Our system can, for example, reason that because a tank is similar to a truck, it probably contains a steering mechanism and a transmission.

Principal Investigator:

Doug Summers-Stay,, (301) 394 -1990