Planning for Successful User Adoption


Definition: Capability and technology transition strategies are intended to increase the likelihood that users will adopt a new application. The methods underpinning these strategies derive from the disciplines of usability engineering and organizational change management.

  • Usability measures how intuitive, efficient, and task-enabling an application is from the user perspective.
  • Usability engineering refers to the structured methods applied to achieve usability in user interface design during all phases of the application development life cycle.
  • Organizational change management encompasses the art and science of moving an organization from its current state to a desired future state.

Keywords: organizational change, transition strategies, usability engineering, user adoption

MITRE SE Roles and Expectations: MITRE systems engineers (SEs) are expected to develop and recommend strategies to eliminate, reduce, and manage end-user reluctance to use newly deployed capabilities or technology or outright reject them. They are expected to define requirements for capability and technology transitions in requests for proposals (RFPs) and related acquisition documents and assess the quality of bidder responses as a key element of the source selection process. MITRE SEs are also expected to monitor and evaluate contractor capability and technology adoption efforts and the acquisition program's overall processes and recommend changes when warranted. Throughout the application development life cycle, MITRE SEs serving in project management, development, and oversight roles should emphasize the importance of transition plans in achieving successful end-user adoption.

Background

Everyone agrees that users and developers need to work together to build systems. The question is "how [1]?" In developing mission-critical applications that involve significant investment of time and resources, how can we ensure that the application will actually be used once deployed? Two primary factors impede user adoption of mission applications: (1) the lack of acquisition program and user mission management support, and (2) the failure to develop a tool as "seen through the user's eyes [2]." Although there is ample evidence that some applications developed and deployed by the government are never used or require massive redesign after roll-out, there is a dearth of baseline metrics for understanding the extent of the problem or developing "best practice" solutions. Thus we still rely on intuition and anecdotal evidence to guide our understanding of the problems and solutions to improve outcomes.

Today's standard practice for achieving user adoption of a new application is to follow the engineering development life cycle, define requirements, and establish user groups as sounding boards. But even with enthusiastic early adopters providing inputs, the number of times that users reject sponsor applications at the point of deployment does not inspire confidence in this approach. Many of these applications require years of subsequent releases to get them right, whereas others simply die a slow death as the funding runs out, having been deployed but never widely adopted.

Scope

This article describes strategies for stimulating user adoption of mission applications, including usability engineering to align the application with user tasks and organizational change management strategies to ensure the organizational readiness of the stakeholders to promote and accept change.

Strategies for Stimulating User Adoption

Usability Engineering

Usability measures how intuitive, efficient, and task-enabling an application is. Usability engineering refers to the structured methods applied to achieve usability in user interface design and process enablement. MITRE experience suggests that relying exclusively on user groups to derive requirements and engage users in an application development effort is not a promising path. User groups are necessary but insufficient for engaging users. They attract early adopters and technical enthusiasts, and though they provide valuable input, their limited representation quickly compromises the engagement process.

Usability engineering can fill the representation gap. It provides formal methods for identifying the different types of users, modeling the tasks they perform, and deriving usage conventions for the new application from those used in the existing software environment [3]. It employs structured methods such as field observations, task assessments, heuristic evaluations, and cognitive walkthroughs to obtain actionable feedback from representative users. Usability engineering provides a method for determining which users to engage, the kind of information needed to create usable designs, the optimum time to gather different types of user inputs in the application development life cycle, the most promising approaches to soliciting user inputs, and how to resolve disagreements among users. Further, it provides an objective means of measuring the progress toward establishing the usability of an application by developing and periodically assessing quantifiable usability requirements. Ideally, therefore, usability engineering begins early in the acquisition process and operates throughout the life cycle of application development.

Organizational Change Management

Organizational change management is a discipline that moves an organization from its current state to a desired future state. It derives its methodologies from the social and behavioral sciences, particularly organizational psychology, communication theory, and leadership development. The deployment of enterprise mission applications typically involves some type of change, such as the imposition of new workflows, business processes, quality standards, and/or metrics for measuring effectiveness. Addressing the organizational dimensions of deploying new technology is critical to establishing manager buy-in and engendering user adoption. For more on organizational change management, see the other articles within the Transformation Planning and Organizational Change topic.

Some acquisition stakeholders interpret application transition narrowly as hardware and software transition. The focus of transition should be extended to include the broader organizational issues relevant to user adoption. This includes: (1) leadership engagement to involve business managers in promoting new technology and ensuring organizational readiness, (2) release strategies designed to optimize end user adoption, (3) communication strategies that prepare stakeholders for the coming change, and (4) training that equips end users to perform tasks with the new technology. The fifth element is usability engineering, as described above, which provides a quantitative means of ensuring the usability of an application as it evolves [4]. In combination, these strategies address the factors that affect organization and site readiness to embrace change in both processes and technology.

Best Practices and Lessons Learned

Build the right multi-disciplinary systems engineering team. Recognize that MITRE's contribution to acquisition, oversight, or development activities may call for usability engineers and organizational change specialists to be embedded in the acquisition and/or systems engineering team. This is a form of the wisdom that says, "build teams with the skill sets needed to achieve a successful outcome."

Include a transition strategy in the RFP. In addition to detailing requirements, the RFP should include a transition strategy that delineates the role of government, contractors, and if applicable, Federally Funded Research and Development Centers (FFRDCs), in stimulating user adoption of a new application. This will ensure that the strategy has been discussed and agreed to by users and other stakeholders before issuing the RFP. Bidders should be asked to detail the methods and personnel they will employ in executing this strategy. The contractor transition strategy should be a major source selection criterion.

Convene a transition team on day one. Consider including the following team members: (1) contractor transition lead, (2) government transition lead, (3) mission-side transition lead, (4) usability engineering lead on development side, (5) independent assessor usability lead, (6) organizational change management lead, and (7) training lead. Consideration should be given to establishing a role for an independent usability assessor, in an organizational change capacity, or as the lead or co-lead of the transition team (in concert with or in lieu of a government team member). The team should monitor risk, measure progress, and steer the program toward developing an application that is quantifiably verified as usable. The transition team and program manager should develop and agree on a process for verifying transition-readiness.

Identify and prioritize user types. Define the different user types and rank their needs. If contention surfaces among requirements, schedule, and budget, this will ensure that necessary tradeoffs are informed by mission priorities.

Continue usability assessment after an application is deployed. Continue to survey the rate of user adoption, assess ease of use, determine the effectiveness of training, etc. Usability engineering starts at program inception and continues after deployment to identify operational or field-specific problems [5].

Recognize "red flags" and address them early. Develop early indicators to alert you that the organization's change approach is likely to founder. These indicators may appear at the very beginning of an effort and continue through deployment. Recognizing the red flags and addressing them early can get you back on track. Here are some examples:

  • IT expects to lead transformational change for the mission: IT can partner, IT can support, but IT cannot lead an organizational change initiative for the mission side. Without mission leadership, the effort to introduce transformational technology will fail.
  • Transition team is buried or marginalized: We have all seen it. Program management pays lip service to user adoption when necessary to get through control gates or milestones. If management and the mission are not aligned and committed to supporting transition strategies, the ultimate outlook for application adoption is grim.
  • Expectation that transition will be handled by the developers: Developers play a key role in building user-centric applications, but expertise in usability engineering and organizational change is fundamental to a successful process.
  • Increasingly greater reliance on training as the "cure-all": A growing need for training is often an early indicator of a decline in usability and design quality. Avoid the temptation to rely on "training" as a workaround in lieu of building usable, intuitive systems.

Consider convening an enterprise-wide developer group. Involve developers throughout the application life cycle. In one program for a single sponsor, a monthly meeting was introduced where developers come together to harmonize interface designs and usage conventions across major enterprise applications under development. This is an initial effort in the early stages of implementation, but the expectation is that it will lead to more usable systems. Though the early results appear promising, the ultimate impact of this strategy is still being assessed.

Expect resistance. Do not assume that project managers will be rewarded for advocating usability. Many organizations still reward project managers on the basis of their adherence to schedule and budget, regardless of adoption outcomes. In fact, the norm in many organizations is initial user rejection of applications, followed by years of subsequent releases to "get it usable." These dysfunctional practices are difficult to turn around, particularly when they are entrenched in organizational culture and reward systems.

Expect formidable challenges when asked to introduce transition planning at the end of the development life cycle. You may be called in "after the fact" when initial application transition has failed. With enough time, money, and expertise, it may be possible to determine what went wrong and how to correct it, but turning around a failed deployment is a daunting task.

Bottom Line

To stimulate user adoption of transformational technology, the systems engineering effort requires an interdisciplinary team that includes embedded experts in organizational change and usability engineering who can leverage relevant social and behavioral methods.

Adoption of new technologies requires a transition team that can develop an overarching plan, manage the activities as a coherent whole, and measure progress toward the development of a usable system. The team should begin these activities at program inception and continue throughout the acquisition phase and after deployment.

References and Resources

  1. For a fundamental introduction to technology adoption, see Rogers, E. M., 1995, Diffusion of Innovations, 4th Ed., New York, The Free Press.
  2. Adelman, L., P. Lehner, and A. Michelson, September 27, 2009, Why Professionals Are Reluctant to Use Judgment and Decision Aids: Review of the Literature and Implications for the Development Process.
  3. MITRE has significant experience pairing developers with analysts in "small cells" to elicit requirements and build tools in real-time for a discrete set of users. Organizations developing technology for a small group of users should consider adopting the "small cell" approach. See Games, R., and L. Costa, April 2002, Better Intelligence Analysis Through Information Technology—Lessons Learned from the Analysis Cell Initiative.
  4. For more on usability engineering, see Bradley, R., T. Sienknecht, M. Kerchner, and A. Michelson, October 2005, Incorporating Usability Engineering into Application Development, draft briefing; Sienknecht, T., May 2007, Incorporating Usability Engineering into Application Development, draft briefing; and Bradley, R., E. Darling, J. Doughty, and T. Sienknecht, September 4, 2007, Findings from the Agile Development and Usability Engineering TEM, MITRE briefing.
  5. For an excellent example of post-deployment usability engineering, see Kerchner, M., June 2006, A "Dynamic Methodology for Improving the Search Experience," Information Technology and Libraries, Vol. 25, No. 2, pp. 78–87.

Publications

Download the SEG

MITRE's Systems Engineering Guide

Download for EPUB
Download for Amazon Kindle
Download a PDF

Questions?
Contact the SEG Team