The project team, working with a user research practitioner, conducted user research to identify root causes and solutions to the variety of UX problems that end users were facing. Subsequent activities included iterative rounds of usability tests and configuration changes to build an optimized business process that better matched end users' natural workflow. With some very simple configuration changes, average task completion times dropped from 45 min. to 23.5 min. In addition, several modifications were implemented to address critical usability issues, and those that were not addressed through modification were channeled back to the vendor to improve future releases. On future deployments, the training team shifted their focus from "quantity" (e.g., percentage of users trained) to "quality" (effectiveness) to verify that end users were adequately prepared for upcoming changes. Lastly and most importantly, multiple parties were involved in improving the overall UX, including stakeholders, end users, technical support personnel, training personnel, and user readiness representatives. Although the initial deployment provided some hard lessons, the result across Intel IT has been positive as it has increased focus on the importance of UX considerations when purchasing, configuring, and deploying OTS capabilities.

Qualitative Research Methods and Outputs

To avoid similar scenarios, Intel IT is increasingly utilizing user research techniques to proactively understand UX considerations. Using these techniques optimizes the effectiveness of technology deployment projects and allows organizations to incorporate user concerns in longer-term technology roadmap planning.

User research practitioners utilize both quantitative and qualitative methods drawn from various fields. Traditional quantitative methods tell you what people are doing and can provide some detail around user's attitudinal and behavioral patterns. However, qualitative research techniques, such as ethnography or contextual inquiry, provide the insight and rich context needed to understand the why behind users' actions. "Although traditional market research techniques such as surveys and focus groups can identify trends and tell us what computer users are doing, they cannot necessarily describe why. And yet the answer to "why" is a key to defining products that will be readily adopted" [8].

Primarily, qualitative research techniques are exploratory and are often most useful to examine areas about which little is known. These techniques can also provide a detailed understanding of existing contextual work practices and workflows. They are useful in exploring the user environment and can help identify factors that might increase resistance to adopting new systems (organizational structure, cultural, values, attitudes, perceptions, beliefs, etc.). There are various methods used within the field of qualitative research, but we primarily use the ethnographic methods of direct observation and unstructured or semi-unstructured interviews to gain the highest quality contextual data we can about our users (Intel employees).

Ethnography is the study of people in their natural environment. The ethnographic method was initially developed within the discipline of anthropology but it has been used and extended throughout various social science disciplines, including sociology, psychology, information science, and human-computer interaction, among others. Interviews and direct observation (observing people while they work) are the primary methods used. Other methods include group interviews, video documentaries, or photo, audio and written journals or diaries. The ethnographic method essentially involves a researcher immersing him or herself within a specific context and establishing relationships and rapport with people within that context. The rapport the researchers develop allows them to interact with people through actual participation in that environment and through observation and dialogue to uncover people's attitudes, beliefs, cultural constructs, and values that shape their behavior.

Contextual inquiry is a method from the overall Contextual Design methodology [9] utilized by UCD and Human Computer Interaction (HCI) practitioners. It borrows much of its technique from ethnography but differs because the researcher actively seeks understanding while actions are happening. Regardless of the specific qualitative research method used, its main advantage is that it solves one of the major challenges of design—understanding users' real needs and the way users work. Work processes often become so automatic that end users find it difficult to articulate specifically what they are doing and why. In these cases, outside observations of work practices often lead to key questions and insights that result in improvement opportunities.

The benefits of qualitative research include the following:

• Gain holistic understanding of employees and how and why they work the way they do.
• Identify unarticulated needs or behaviors.
• Allow natural patterns to emerge to guide understanding as opposed to starting with a narrow research question.
• Learn intricate details about difficult-to-study phenomena.
• Understand differences in cultural responses to technologies or workflow processes.
• Identify problems with systems or technologies that users attribute to their own failings, and thus do not report.
• Add richness to and support for quantitative data.
• Gain a rich understanding of the context within which employees work (social, cultural, physical).

User researchers generally summarize the major themes and patterns in the data for decision makers and then use these to develop various kinds of visualizations of the data that help describe the users and their needs and goals in vivid ways. These visualizations include user profiles that describe key characteristics of user groups or "Day in the Life" narratives. They may also include scenarios that represent one particular activity observed during the field research. Various types of flow diagrams that show workflow, information flow, and relationships between different job roles or processes are also effective ways to demonstrate the user's current experiences during their workday. Typologies or classifications based on certain factors are another common type of output. Researchers often collect various kinds of artifacts from users that may include work products/outputs, documents, photos, or images. All of the outputs or deliverables generated by user research describe the user and his/her current experiences.

Applying User Research to an OTS Deployoment

In the case of OTS deployments, user research can provide key insights about users and usage that will help guide purchase decisions, application configuration, and necessary business process engineering efforts. Ideally, conducting user research occurs prior to the initial OTS selection with the resulting data continually influencing the configuration and deployment processes. However, user research is still beneficial to programs where the purchase decision has already been made but configuration and deployment have yet to occur. One example of how user research has been used to enhance the UX of OTS products is a study conducted earlier this year on Intel's Platform Product Life Cycle (PPLC). Study results identified organizational structure and cultural issues that the selected tool would not have addressed directly. For example, the PPLC data showed that about 50% of the issues discovered were in fact related to organizational culture and structure, areas that are critical to understand when deploying any major new capability, but that are often overshadowed by business or technical considerations.

Many of the organizational issues discovered were directly related to efforts undertaken at a corporate level to transition Intel to a platform company. Developing the new OTS product has primarily focused on business processes and new tool capabilities for management of the platform product life cycle. The study was able to highlight the importance of understanding the context within which an OTS product is deployed as well as validate some of the design choices that had already been made.

By proactively understanding issues prior to deployment, user research practitioners were able to work directly with the transition change management team to address gaps and mitigate risks to increase adoption rates. It has also allowed the team and other decision makers to understand more about any possible organizational roadblocks that may make adoption of the new tool difficult. Findings like these highlight the value that user research brings to an IT organization. Through understanding and addressing all aspects of user experience before making initial OTS purchase decisions, decision makers and deployment teams are better able to effectively configure and deploy valuable capabilities, increasing adoption rates of end users.

See Appendix A at the end of this article for an overview of how to utilize user research to optimize OTS deployments.