Homeland Security: Refining Best Practices in Systems EngineeringNovember 2013
In 2010, a year after earning her graduate degree in cognitive psychology, Bethany Casto joined MITRE as a data management systems engineer in the Homeland Security Systems Engineering and Development Institute (HSSEDI™), the federally funded research and development center we manage for the Department of Homeland Security. The opportunity to help improve the capabilities of DHS through systems engineering intrigued her. It also enabled her to put her expertise in research methods to work.
"One of my first tasks at MITRE was to research the development of best practices in systems engineering," Casto says. Working closely with MITRE subject matter experts, she helped produce a Web portal to disseminate information on best practices in concept-of-operations requirements and other systems engineering tasks.
An Inside View of DHS Mission Priorities
The project gave her an inside perspective on several of DHS' key technology focus areas, including biometrics (the technology of measuring biological data to determine a person's identity). Her next task was to help develop a test strategy, framework, and test methods for assessing the performance of commercially available iris-recognition systems, which identify people by the unique characteristics of their eyes.
Casto and the HSSEDI team developed a standard protocol for DHS to use in evaluating these systems. The team defined test methods and metrics for system physical characteristics, data quality and format, and performance characteristics. This included metrics such as how easy the system is to use and whether people can understand the feedback. "You have to consider the user's needs and capabilities in systems engineering," she says. "You can't expect the person who uses the technology to pay attention to a tremendous number of details at once."
The team delved into issues relating to technical performance, standards compliance, security, human factors, and sustainability, which means minimizing the number of parts that comprise a system. This helps reduce the ecological impact of their replacement. "There are additional environmental factors to consider. If the system is deployed on a Coast Guard cutter, you need to think about its resistance to wind and saltwater," she explains. The team collaborated with device vendors as well as DHS and other government officials to develop the protocol.
With her growing store of biometrics knowledge, Casto transitioned to providing systems engineering and technical guidance to the DHS Science & Technology (S&T) Directorate. S&T manages science and technology research, from development through transition, for Department components and first responders. Casto's work includes system testing and software development to enhance operational capabilities that support S&T's mission of providing knowledge products and innovative technology solutions for the homeland security enterprise.
Finding the Answers
"To be successful in a project like this, you need good writing skills and research capabilities," she says. "You must have technical knowledge in many areas. You need to be persistent in pulling the thread to find the answers.
"When you engage with the people who will use the system, you must be diligent to find their deeper technical needs. They know what the outcome should be, but not necessarily how to get there with their current system."
As an example, Casto cites a project where the sponsor intended to purchase a certain facial-recognition system. MITRE research showed the system—and others like it—wouldn't meet performance requirements in all operational settings. "They really wanted to catch bad guys, but we showed that the systems wouldn't work in the ways they were expecting them to. They could have wasted a lot of money on systems that weren't able to do what they wanted."
Casto now works most of the time at a sponsor site. "I find it helpful to sit near my project team, which includes government staff and commercial contractors. It's great to engage with them on a daily basis. My day-to-day work is very collaborative."
At the same time, she is taking graduate classes in systems engineering at Johns Hopkins University. Though her schedule is demanding, she enjoys devoting some of her personal time to singing alto in her church choir and taking Shotokan karate lessons.
—by Maria S. Lee
Are you a good fit for MITRE, too? Explore our current Job Openings.