Chapter 4
Finding the Right People: The Foundation of Valid Research
Involving the right participants is fundamental to good research-if you don't understand who your users are, you can't provide them with the right solution. Few organizations truly have "everyone" as their audience. Even large entities like governments or companies like Amazon need to segment their audience into specific user groups based on behaviors and needs.
Using inappropriate participants (like recruiting English literature undergraduates to test mathematics software) will produce invalid results. Having a small number of appropriate users is always preferable to having many inappropriate ones. Participants can be recruited through specialized research agencies with participant databases, or by using your organization's customer databases (with proper permissions).
Effective recruitment requires two key tools: a recruitment brief detailing who you want to involve and how many; and a recruitment screener questionnaire to identify suitable participants. When creating these materials, focus on both demographics and behaviors-demographics alone are insufficient, as demonstrated by the Prince Charles/Ozzy Osbourne example where demographically identical profiles represent vastly different people.
A good recruitment screener includes: an introduction explaining the screening process, logistics information, demographic questions, behavior questions, self-assessment of confidence levels, technology usage questions, confidentiality disclosures, and accessibility requirements. The screener should include options for behaviors you both do and don't want to capture, and typically excludes participants who work in the same industry or in website/app design.
Paying participants is the most effective way to ensure participation and attendance. Incentive amounts vary based on participant type, research format, session length, location, and participant profession. When paying with vouchers for your own organization, be aware this may positively bias participants toward your products. For ethical research, incentives should never be related to what's being researched.
Making your products accessible requires considering the full spectrum of human capability. While accessibility deserves its own book, several approaches can help: use automated testing tools which identify 20-30% of issues; conduct manual testing with assistive technologies like screen readers and speech recognition software; hire specialists to audit your work early in development; and perform usability testing with users who rely on assistive technologies, including both novice and experienced users.
Chapter 5
Asking Questions That Reveal Rather Than Lead
The quality of your research depends heavily on how you ask questions. Effective research questions follow three key rules that help ensure authentic, unbiased responses from participants. First, use inclusive language by avoiding jargon and technical terms, using plain English to ensure all participants understand questions fully. When technical terms are necessary, clearly explain them in simple terms and provide examples. For instance, instead of asking about a user's "navigation paradigm preferences," ask how they prefer to move through websites or find information. This prevents participants from either feeling uncomfortable asking for clarification or worse, guessing at answers they don't understand.
Second, use neutral open-ended questions in exploratory research that don't assume certain answers or elicit one-word responses. Simple questions like "Do you like this?" can influence participants toward positive answers because people naturally avoid hurting others' feelings and tend to agree with authority figures. More balanced formulations include "What, if anything, do you like about this?" followed by "What, if anything, do you dislike about this?" Adding "if anything" gives participants explicit permission to express negative opinions. Other effective neutral questions include "How do you typically handle this situation?" or "Walk me through your thought process when you encounter this type of problem."
Third, avoid using key words from interface labels in research tasks and questions to prevent artificially directing participants. For example, when testing how users find events on a charity website, saying "Please imagine you want to get involved with raising money by taking part in a sports event" uses the exact navigation label "sports event" from the interface. A better approach would be "Please imagine you want to do a sports activity to raise money" which tests whether users can naturally discover the correct section. Similarly, if testing a "Help Center," ask participants "Where would you go if you had a problem with your account?" rather than directly mentioning "help" or "support."
Once you're comfortable with research principles, you'll recognize situations requiring flexibility. Document when and why you break rules to account for this during analysis. Three scenarios justify breaking standard practices: with non-responsive participants who give minimal answers, you may need direct, leading questions to extract valuable insights. For example, after several open-ended questions yield little information, you might ask "Did you find the checkout process confusing?" to prompt more detailed feedback. During usability testing, you might need to assist struggling participants to prevent excessive frustration, especially when testing early prototypes or complex systems. And in surveys, closed questions are appropriate when properly crafted with response options like agreement scales, multiple choice selections, or rating systems that capture specific data points needed for quantitative analysis.
Remember to maintain detailed notes about any deviations from standard questioning techniques, including the context and rationale for these decisions. This documentation helps maintain research integrity and provides important context during data analysis and reporting phases.
Chapter 6
Navigating the Ethics of Human Research
Data protection is essential for user research and shouldn't be skipped. It involves the fair and proper use of people's information as part of their fundamental right to privacy. Since the first edition of Marsh's book, new regulations like GDPR and CCPA have been implemented globally. Good data protection practices build public trust and aren't just strict rules but principles for assessing risks.
Personal data collected during research must be kept accurate, secure, used transparently, and limited to what's necessary. This includes both information you collect about participants and data you create by recording research sessions. You need to consider how you handle data throughout the entire research process, including collecting, recording, storing, using, analyzing, combining, disclosing and deleting it.
Informed consent is essential for collecting and using participant data. For consent to be truly informed, participants must understand who's conducting the research, its purpose, what data you're collecting, what will happen during the session, how results will be shared, that participation is voluntary with the right to withdraw, and how long data will be kept. You must also disclose if sessions are being observed or recorded.
People aged 16 and older can give their own consent. For participants under 13 in the UK (under 16 in many other countries), parent/guardian consent is required. Send consent forms digitally before sessions to save research time. Digital signature services like DocuSign work well, though alternative formats may be needed for accessibility. For international research, translate consent materials into appropriate languages.
Consent forms aren't needed when you don't collect personal data, typically in unmoderated research where participants voluntarily take part without incentives, such as short surveys, first-click tests, card sorting, tree testing, or brief usability tests with screen recording only. However, if offering incentives, you'll need at minimum a data privacy statement explaining how contact information will be used.
When sharing research findings, particularly through quotes, images, video and audio clips, you must protect participants' privacy by anonymizing data wherever possible. This means editing content to accurately reflect what happened while ensuring participants cannot be identified. For difficult-to-anonymize content like video showing faces, consider blurring, bleeping, or removing sensitive information.
Chapter 7
Choosing the Right Research Method for Your Questions
User research is an iterative and continuous process with different methodologies appropriate at different times throughout development. There isn't one correct way to combine methods, but there are common approaches that can be effective. Always start with your research question rather than a predetermined method.
Quantitative data reveals what is happening-how products are used, user attitudes, and the scale of issues-but doesn't explain why. Qualitative research complements this by exploring user motivations and the reasoning behind behaviors and attitudes. Combining both approaches allows researchers to gain comprehensive understanding, using quantitative data to identify patterns and qualitative insights to explain them.
Different scenarios call for different combinations of research methods. For developing new products or improving existing ones, you might begin with a stakeholder workshop to understand current situations and requirements, followed by user interviews or contextual inquiry to validate assumptions, then prototype design, usability testing, and iteration based on findings.
When reorganizing content and structure, you could start with usability testing to understand existing problems, use card sorting to test new structures, create draft information architecture, validate with tree testing, and refine with content testing. Alternatively, you might begin with content testing, then use open card sorting, test draft content, refine with closed card sorting, validate with tree testing, and finish with usability testing.
For exploring new concepts, research typically moves from broad to specific questions. Start with stakeholder workshops to generate ideas, use ethnography or contextual inquiry for broad understanding, check direction with stakeholders, create prototypes, test with users, and iterate with increasingly refined prototypes while maintaining stakeholder alignment.
Usability testing involves researchers observing users completing tasks with a product or service to identify problems. Moderated testing offers several advantages: greater control allowing researchers to answer questions and probe deeper; flexibility to adapt research based on unexpected findings; cost-effectiveness requiring minimal equipment; robust results that convince stakeholders through direct user feedback; and accommodation of accessibility testing with assistive technology users.
Content testing evaluates how understandable and suitable your content is for its intended audience. Good digital content is clear, actionable, and navigable. Users recognize it instantly and become frustrated when it's lacking. Creating quality content requires significant effort but builds trust and confidence while reducing errors by both organizations and users.
Card sorting helps understand how people think about and associate things by having them group related items. This method is particularly useful when designing website structure or information architecture, making content easier to find through proper organization and labeling. It reveals how people mentally organize information, making it valuable for designing website navigation and grouping related content.
Chapter 8
From Data to Insight: The Art of Analysis
After collecting data, the real work begins-transforming raw information into actionable insights. Content analysis offers techniques to help analyze large amounts of qualitative data by codifying and categorizing it. This interpretative process involves tagging data with codes-brief descriptions of what's occurring without interpretation. Patterns can be characterized by similarity, difference, frequency, sequence, correspondence, or causation.
When starting analysis, you can use predetermined codes, no codes (completely open exploration), or a combination of preset and emergent codes. The combination approach is often least biased while remaining methodical, allowing unexpected findings to emerge. Codes should be comprehensive, mutually exclusive, and understandable to others. Multiple coding passes help ensure consistency, and having multiple coders compare results helps avoid bias.
Affinity diagramming (or mapping) helps identify patterns from qualitative data by writing ideas on sticky notes and grouping them around themes. This method works well for usability testing, interviews, surveys, and other qualitative research. It's particularly suitable for agile environments, providing a quick first analysis to show stakeholders early research themes.
The process follows a structured approach: First, combine individual work (highlighting interesting data and writing discrete points on sticky notes) with group work. Then, conduct initial sorting by sharing raw data among the group, having participants highlight relevant points and write them on sticky notes. Finally, analyze and produce themes by having participants take turns placing and grouping sticky notes, explaining their reasoning, discussing duplications, naming groups, and identifying relationships between themes.
Agile analysis provides methods for processing qualitative research data within fast-paced environments. Two key approaches include face-to-face synchronous analysis and the rainbow spreadsheet method. In the synchronous approach, observers note observations on sticky notes during sessions and organize them into predetermined themes between sessions. The rainbow spreadsheet method uses color-coding to track whether participants validated team assumptions, providing a visual summary that's particularly useful for usability testing in agile contexts.
When analyzing qualitative usability data, you can organize findings by task, product features, design categories, or journey flow. The process involves becoming familiar with your data, identifying issues without categorizing them initially, grouping similar issues together, and identifying patterns between groups. After grouping issues, count instances in each group to show relative importance. Consider context-even low-frequency issues may be critical if they affect key user journeys.
Chapter 9
Communicating Research for Maximum Impact
After analyzing research data, communicating findings effectively is crucial. The format depends on research objectives and available time. Rather than spending excessive time on visual presentation, focus on substance and the story your data tells. Prioritize making your work digestible over creating glossy graphics.
When making research recommendations, consult with stakeholders who will implement changes before formally sharing suggestions. Your recommendations might include more targeted research (clearly explaining why), changes to internal processes, digital product modifications, brand realignment, product proposition adjustments, or content improvements. It's helpful to categorize recommendations as short-term (cheap, quick fixes), medium-term (medium budget, medium fix), or long-term (expensive, hard to fix).
The length and detail of your research report should match your objectives, project stage, available time, and audience. For executives, create concise summaries, while developers and analysts need detailed reports explaining necessary work. Reports should be practical and visual rather than text-heavy, often presented in PowerPoint instead of Word documents.
A comprehensive research report should include a title page, contents list, executive summary, research objectives and methodology, test materials, participant summary, guidance on reading the report, logically grouped results, research limitations, and conclusions with next steps. The executive summary should work as a standalone document covering research purpose, methods, key findings, recommendations, limitations, and a positive summary sentence.
Video recordings provide powerful evidence of user experiences, especially for stakeholders who couldn't observe the research firsthand. When creating video highlights, be strategic to save time: identify specific issues to showcase, use timestamped notes to locate key moments, keep clips concise while providing enough context, include multiple examples of important issues, and label participants for clarity.
Experience maps visualize end-to-end user experiences for particular user groups accomplishing specific goals, while customer journey maps show how people use specific products or services. These visualizations represent touchpoints, emotions, expectations, motivations and questions across channels and time periods, helping identify problem areas and innovation opportunities.
Personas are fictional profiles based on real data that represent user groups with shared interests, goals or characteristics. They bring user groups to life, helping organizations engage with them on a human level to build better products and services. Effective personas capture characteristics and behaviors most relevant to design decisions while feeling like real individuals.
Chapter 10
Creating a Culture of Continuous Learning
User research isn't a one-time activity but an ongoing process of learning and refinement. The most successful organizations integrate research into their regular workflows, creating a culture where decisions are routinely informed by user insights rather than assumptions or opinions.
When user research isn't common in your organization, getting time, resources and budget can be difficult. People may not want to admit there's a problem with products or services. The best way to demonstrate benefits is showing evidence of what research can do, though this creates a catch-22 situation. Guerrilla research provides a quick, cheap way to gather initial evidence that demonstrates user issues, how users think and feel (which may differ from organizational assumptions), real-world behaviors, and insights for improvement.
For established research practices, consider implementing an atomic research approach that offers a systematic way to transform research data into actionable insights. This knowledge management method combines facts from multiple experiments to create contextual insights that inform decisions. In this model, experiments generate data, facts are extracted observations, facts combine to create insights, and insights lead to findings and decisions. This approach enables powerful triangulation by connecting these atomic units, creating persuasive evidence for decision-making while allowing research knowledge to be reused rather than unnecessarily repeated.
Accurate timescales for research completion depend on numerous factors including research type, participant characteristics, and internal approval processes. When incorporating user research into workflows, particularly in agile environments, different approaches are needed for new versus existing products. Discovery phases typically require significant upfront research investment (4-12 weeks), while alpha phases (6-12+ weeks) involve iterative research and design validation.
For qualitative research, analysis typically takes longer than the research itself-20 hours of interviews might require 40-60 hours of analysis. Dual-track agile approaches can help integrate research with design and development, allowing the research track to proceed alongside but sometimes ahead of design work to accommodate recruitment and analysis needs.
The ultimate goal isn't just to conduct good research but to create products and services that truly meet user needs. By understanding people's behavior, motivations, and needs in specific contexts, organizations can design experiences that feel intuitive and valuable. As Marsh notes, user research is inherently iterative, requiring repeated engagement. The learning in user research never stops-and that's precisely what makes it both challenging and rewarding.
