 |
 |
 |
 |
| |
|||||
|
|
|
Home > News & Events > MITRE Publications > The MITRE Digest > | |||||||||||||||||||
Zapping System Bugs Through Performance Engineering May 2006
What would happen if engineers at the world's biggest automakers didn't agree on a master blueprint before they designed a car? If teams responsible for components such as the transmission, engine, and suspension worked independently, problems would undoubtedly crop up during the car's assembly. For example, the engine might be too large and heavy for the suspension to support it, or the transmission might not be able to handle the engine's energy levels. Clearly, it makes sense for people to work together to ensure individual systems meet overall performance goals. It's only logical that this process, known as performance engineering (PE), be incorporated right from the start when building a complex system. Yet it's common for systems to be designed without incorporating PE. "Most people don't believe their systems will experience performance problems," says Tim Kehoe, business lead of the Performance Engineering group in MITRE's Software Engineering section. Unfortunately, many systems do—and discovering glitches during system testing or after deployment is often costly and time-consuming. That's why hardware and software performance engineering is so important to the success of large-scale information technology programs. A group of engineers at MITRE is collaborating and emphasizing how the PE practice assures systems meet their operational performance goals. "PE includes the processes, techniques, and tools used to measure, model, and analyze the ability of an information system to meet its performance demands throughout the entire life cycle," explains Kehoe. PE is a formal engineering discipline, and the MITRE team's mission is to encourage PE to be addressed throughout every phase of a sponsor's system life cycle. "We advocate taking an active approach that significantly reduces performance risks," he says. Getting It Done By helping sponsors overcome PE problems in current systems, MITRE hopes to avoid issues with future systems. "One element we're focusing on is expectations," says Kehoe. "It's commonly believed that after system development, tuning will alleviate performance issues found late in a program's life cycle." The term "tuning" refers to making midstream changes to a system that will help it perform better, whether it's revising a few lines of code, adding additional equipment such as disk drives, or tweaking system configuration parameters, like increasing the size of buffers used to move information. However, it's been proven that tuning offers only marginal improvement once an ineffective performance design has been put in place. "We need to get away from a reactive posture and engage performance early—throughout every phase of a system's development," Kehoe emphasizes. To that end, MITRE engineers have developed a number of PE tools, such as the Modeling Environment for SOA Analysis (MESA). MESA provides end-to-end modeling and simulation environments for studying service-oriented architectures (SOA). "The idea behind an SOA is to develop small modular services, deploy them on a common infrastructure, and then compose them as needed to complete a task," explains Patrick Van Metre, a senior software systems engineer in MITRE's Center for Innovative Computing and Informatics. For example, online retailers can use services provided by Amazon.com to supplement their own sites by including content from Amazon's extensive storehouse of reviews and product data. "The MESA toolkit is flexible and its capabilities can be expanded, and it allows models of SOA infrastructures to be integrated with other models, like business-user process models," he says. "MESA addresses PE challenges, such as how to plan and construct an SOA. It also offers capacity planning for SOA deployment." MESA has been applied to provide insights into performance considerations with SOA strategies for military net-centric operations. It has been implemented on several specific net-centric engineering projects to highlight the importance of evolving and refining performance metrics specifications. MESA may also be used when there are performance variables to consider. For example, a government agency might have a centralized authentication server or a distributed network of server installations; in either scenario, the servers need to incorporate authentication systems that compare credentials with data that users must supply, such as name/password combinations or digital certificates. Constant Testing, Consistent Improvement The key, Van Metre says, is to use modeling and simulation technologies in concert with experimentation and prototyping as a means to evolve, refine, and manage performance expectations. "Prototyping and experimentation provide the data we can use in creating performance models to assess 'what-if' conditions," he explains. Similarly, another set of tools MITRE uses to assist its customers with PE is based on the standard Unified Modeling Language (UML), and a recent extension to it, called the UML Profile for Schedulability, Performance, and Time. While UML focuses on describing a system's structure, functionality, and data, the Profile centers on enabling quantitative predictions, supporting the interoperability of performance analysis tools with UML-based design tools. UML is widely used for software-intensive systems such as the U.S. Army's Future Combat Systems and the U.S. Air Force's E-10A program. (The E-10A is a long range, air-to-air and air-to-ground surveillance weapons system that can locate, track, and classify ground targets such as insurgent convoys.) "We analyzed the performance of the E-10A airplane in a time sensitive targeting scenario," explains Tom Wheeler, a senior principal software systems engineer. He says the UML model was well received by the government and contractor. From this experience, engineers demonstrated that integrated design and analysis is a promising systems engineering approach, because it allows developers and analysts to work off the same set of data. Additionally, this approach allows for analysis of end-to-end system behavior and timing. Practicing What's Promoted MITRE itself, in fact, makes its own investment in PE for the corporation's internal products. For example, the MITRE Information Infrastructure (MII) is a continually evolving corporate intranet site that includes human resources information, technical data, project tracking features, a company phonebook, and more. "The improvements in the MII's reliability since it was created ten years ago are due to the concentrated efforts of PE," says Joel Jacobs, MITRE's acting Chief Information Officer. "We have successfully deployed a system that tells us when there is a problem in the network, and it has tremendously improved the way we manage systems. The MII has proven so successful that it often serves as a model for sponsors." Clearly, there is consensus throughout the company that PE is indispensable. "Performance tools are a critical part of enterprise systems engineering, because MITRE supports customers as they move towards net-centric operations," says John Wilson, a director of information technologies in MITRE's defense work area. "If performance isn't evaluated from the start, problems will crop up program after program, system after system." —by Cheryl Balian Related Information Articles and News
Technical Papers
Websites |
||||||||||||||||||||
| Page last updated: May 25, 2006 | Top of page |
Solutions That Make a Difference.® |
|
|
