Benchmarking the Benchmarks

Report No. ARL-TR-2805
Authors: Daniel M. Pressel and Jelani Clay
Date/Pages: September 2002; 33 pages
Abstract: Benchmarks can be useful in estimating the performance of a computer system when it is not possible or practical to test out the new system with an actual workload. In the field of high performance computing, some common benchmarks are the various versions of Linpack, the various versions of the Numerical Aerospace Simulation Systems Division of NASA Ames Research Center (NAS) benchmarks, and the STREAMS benchmark, as well as older and less frequently referenced benchmarks such as the Livermore Loops. There are also those who recommend estimating the performance based solely on the peak speed of the computer systems. Unfortunately, the per processor levels of performance measured using these benchmarks can vary by 1 to 2 orders of magnitude for the same system. Therefore, one has to ask, which benchmark(s) should we be looking at? This report attempts to answer that question by comparing the measured performance for a variety of real world codes to the measured performance of the standard benchmarks when run of systems of interest to the Department of Defense (DOD) High Performance Computing Modernization Program.
Distribution: Approved for public release
  Download Report ( 0.381 MBytes )
If you are visually impaired or need a physical copy of this report, please visit and contact DTIC.

Last Update / Reviewed: September 1, 2002