NIST Research Computing Environment Benchmarking Study Report

By Kevin Gunn , Christine Harvey , Kana Shah , Megan Rinard

This report presents an evaluation of the Research Computing Environment at the National Institute of Standards and Technology (NIST) and compares it with similar institutions across five key components.

Download Resources

Our study focuses on five key components of research computing: High Performance Computing (HPC), Scientific Software Portfolio (SSP), Research Data Management Services (RDMS), Research Computing Support Staffing (RCSS), and Networking Services.

Overall, NIST compares well to the other study participants. It outperforms the institutions in this study in RDMS and RCSS. There are some opportunities for improvement, however, especially with respect to its HPC infrastructure and scientific software provisioning. 

Key findings include:

  1. High Performance Computing (HPC): NIST operates multiple HPC systems, many of which are specific to certain research groups. NIST's computational capability is reported as lower than all participants, even when scaled by the size and impact of the research program. NIST's HPC spending is lower than that reported by other study participants, with more spent on labor but significantly less on hardware, software, and capital investments.
  2. Scientific Software Portfolio (SSP): NIST provides a larger variety of centrally managed scientific software to researchers as part of its RCE than the other participants. The practice common to all participants is that the majority of scientific software is provisioned either on a fee-for-service or a bring-your-own-software model.
  3. Research Data Management Services (RDMS): Although data in this area is limited to what the participants could provide, NIST is employing research data management best practices as they apply to federally funded research. More data from additional research institutions would help NIST better gauge its RDM spending and storage capacity.
  4. Research Computing Support Staffing (RCSS): NIST invests much more than other participants in RCSS, both in terms of dollars and people. However, NIST’s relatively high number of support staff is inclusive of staff in NIST’s research laboratories – a perspective apparently not measured by other participants.
  5. Networking Services: Central funding of the network infrastructure is a norm among the study participants. However, the scarcity of data from participants makes it difficult to draw any firm conclusions related to networking.

The report concludes that NIST could benefit from a deeper dive into what comparable institutions characterize as RCE as well as how to gauge whether RCE support levels are appropriate to an institution’s research program.