Evaluation of Visualization Tools for Computer Network Defense Analysts: Display Design, Methods, and Results for a User Study

Report No. ARL-TR-7869
Authors: Christopher J Garneau; Robert F Erbacher
Date/Pages: November 2016; 98 pages
Abstract: Computer network defense (CND) analysts serve an increasingly vital role in the defense of our nation's computing infrastructure. An important component of their work is the monitoring of suspicious activity identified by an intrusion detection system (IDS). While analysts are trained to quickly recognize abnormal patterns in textual log files, humans are generally not well suited for such processing in any large quantity. Many authors have proposed the use of visualization techniques to aid the cyber security analysts' search activities; however, such techniques are not widely used by analysts. This report describes an evaluation of 2 graphical displays (a "parallel coordinates" display and a "node-link" display) compared against a traditional tabular arrangement with the goal of better understanding analyst performance and obtaining subjective feedback on the graphical alternatives. Both expert analysts and novices (students) participated in the study. Results show that analysts generally preferred familiar tools but were able to use some graphical alternatives (node-link) to achieve similar performance in less time. Students were not found to be effective surrogates for experienced analysts for research/validation of techniques. This report describes the development and design of the displays and the experiment, and provides insight into analyst needs and evidence on effective methods for validating cyber defense visualization tools based on results obtained.
Distribution: Approved for public release
  Download Report ( 2.999 MBytes )
If you are visually impaired or need a physical copy of this report, please visit and contact DTIC.
 

Last Update / Reviewed: November 1, 2016