Emerging technologies such as artificial intelligence, virtual reality, chatbots, blockchain, and connected devices present a new universe of opportunities for product managers. Modern product teams have an abundance of ideas on how to improve customer experience and reshape industries using these technologies.
The challenge, however, is that most products fail to achieve meaningful customer adoption. The lack of customer adoption is often a result of building a product that customers don’t actually need. You can reduce the risk of this by conducting product research and running experiments throughout the product lifecycle.
If you conduct product research the right way, you will identify market opportunities and gain meaningful customer insights that will help you make decisions with confidence. Product research can help you early in the product lifecycle, to test value propositions and identify ideal customer segments, and later in the product lifecycle, to hone your usability and branding.
However, many product managers fail to properly incorporate product research into their development workflows. If you simply ask customers what they want you to build, or rely on outdated research strategies, you won’t gain the insights and data you need to make the right decisions. This article covers how product research compares to market research and the best strategies for conducting product research.
Is product research the right tool for the job?
The difference between market research and product research largely boils to sample size and learning objectives.
Market research provides insights on markets and large demographics of customers. It typically involves large-scale focus groups and surveys.
Market research is an excellent approach for:
- Sizing a market
- Ethnographic research
- Measuring sentiment of an established brand
Market research can provide meaningful insights for branding and new product development but does not sufficiently meet the needs of modern product managers because it’s large-scale rather than iterative.
Product research, on the other hand, involves running a larger number of tests with smaller sample sizes of potential users and iterating on the idea based on the insights from each test.
Product research is an excellent approach for:
- Validating new product and feature ideas
- Prioritizing features
- Identifying target customer segments
- Improving user experience
A sample size of 100 users may not seem significant enough to provide meaningful insight, but it’s actually a major benefit of product research. Using a small sample size enables product teams to run more tests, thereby mitigating bias that might be present in any single test, and build on the solution based on the insights gained throughout the process. Small tests also let product teams identify glaring product risks sooner and avoid wasting additional resources.
What do you need to research?
In “High-Impact Experimentation is Product Management,” Holly Hester-Reilly, a product consultant and former Product Owner at Shutterstock, shared her framework for determining what product assumptions to research. Holly plots her assumptions on a four-quadrant diagram. The diagram maps “the negative impact if the concept isn’t viable” on the Y-axis, and “the likelihood that the concept isn’t viable” on the X-axis.
Holly then tests the assumptions in the upper right-hand quadrant: assumptions that have a high likelihood of not being viable and would have a major negative impact if they aren’t viable. This framework helps her get answers to her most critical product questions. For example, if the success of your product is dependent on users having a need to find the best tortillas in their city, and there’s a significant risk that they don’t have a need to find best tortillas, it would be important to research that assumption.
What are the best product research strategies?
Once you’ve identified what you need to research, it’s time to pick the right approach for the job. Fortunately, there’s a strategy for answering just about any product question you’d want to answer.
At Alpha, we break our tests into six categories: user discovery, competitive analysis, concept testing, usability testing, UX refinement, and feature prioritization. Depending on the test, we incorporate a mix of surveys, rapid prototyping, and moderated usability sessions. Let’s take a closer look at each of these six categories of product research and learn best practices from the guests of This is Product Management.
1. User discovery
User discovery tests are great for validating that users have a given problem and getting initial feedback on solution ideas. This type of research is typically conducted through surveys and interviews. That doesn’t mean that you simply ask users what they want. The value of the insights you gain from this research is dependent on the questions you ask.
In “Asking the Right Questions is Product Management,” Noopur Bakshi, Senior Product Manager at Adobe, shared her best advice for gaining meaningful insights through interviews and surveys. She advises listeners to be open-minded rather than seeking to validate your hypotheses. Noopur asks open-ended questions about users’ needs and current behaviors. User discovery interviews are also an opportunity to introduce your ideas and make a qualitative assessment of users’ interest.
2. Competitive analysis
Competitive analysis can help you learn what users are currently doing to solve the problem your product solves. It shows you how users are acting, rather than just what they’re saying. If you have an idea for a chatbot to help people find the right doctor, and you learn that users are already using products to find doctors, you’ve validated that there is some demand to solve that problem.
Getting feedback on competing products can also help you identify gaps in the market. For example, you might learn that your target users aren’t satisfied with the product recommendations they receive when shopping on Amazon and that they’re actively seeking a different shopping experience with better recommendations. This insight might encourage an e-commerce product team to test a new personalized product recommendation feature on their store.
To conduct competitive analysis, survey users about their experience with your competitors or ask them to use your competitors’ products and provide feedback.
3. Product concept testing
Surveys, interviews, and competitive analysis help you learn more about your users’ pain points and what they’re currently doing to solve the problem that your product solves. Once you’ve discovered that there could be a market opportunity, it’s time to start testing your product idea.
Rapid prototyping enables you to provide users with a visual representation of your product idea without having to spend all the engineering hours that would be required to actually build it. This enables you to gain insights that are more specific to the solution than the insights you gain from surveys or interviews.
Prototyping is ideal for testing value propositions, onboarding flows, and feature sets. A prototype can be as simple as a static landing page that displays the value propositions of the product or as complex as a multi-page mobile onboarding experience.
4. Feature prioritization
Does your restaurant discover app need to include a feature that enables users to share their location with their friends when they’re meeting at the restaurant they’ve selected? Does it need to include a feature that enables users to make restaurant reservations from within the app? Will any of these features make users more inclined to sign up and use the product?
Split testing landing pages is a great approach to prioritizing your feature set. Send 100 users to a landing page that includes the location sharing feature and another 100 users to a landing page that doesn’t include the location sharing feature. See which one more users sign up for and get their feedback on the feature. While the results of a single test of this nature are not sufficient to provide a definitive answer to whether you should include the location sharing feature, it will provide you with a direction for further testing and identify any glaring risks.
5. UX refinement
In “User Experience is Product Management,” Sarah Doody defined user experience (UX) as the user’s entire journey of using your product. This includes every touch point they have with your brand both online and offline. Sarah refines the UX of her products by conducting surveys, conducting moderated testing sessions, and visiting users in their homes and offices.
If you’re managing a B2B product, visiting customers at their office is a great product research strategy. Customer visits allow you to see how your clients experience the problem and how they incorporate your product into their workflow.
In “Research and Development is Product Management,” Marc Rubner, VP of Product Management and Marketing at Blackboard, shared a story about managing a product for banks while at American Express. Customers loved the product in almost every market — except for Tokyo. So Marc got on a plane to Tokyo to visit customers and understand why they weren’t using the product. He found that bank reps were so well-trained on their legacy process that they didn’t actually need to incorporate Marc’s product into their workflow.
6. Usability testing
Usability testing helps you put the finishing touches on your product. This includes honing your branding, design, and copywriting. If you’re wondering what color your sign up button should be, this is the test for you. Ask users to complete an action on your app, such as searching for a restaurant, and get feedback from them in real-time. Consider a split test if you’re deciding between two different color schemes.
Sarah Doody emphasized the importance of getting feedback from users while they’re using the product, as opposed to afterward, to get gain more meaningful insights Ask users to say what they’re thinking out loud as they use your product. Users may not otherwise remember to share some specific feedback.
Interpret your research and iterate accordingly
When conducting product research, it’s critical to find a balance between intuition and data. Remember, product research utilizes a larger volume of tests with a smaller sample size of users as compared to traditional market research. Therefore, no single test is sufficient to provide a definitive “green light” or “red light” answer. Each test will provide you with directional insights to inform future testing.
If you learn that 60% of users prefer onboarding experience A over onboarding experience B, it doesn’t mean that you should definitely launch onboarding experience A. It does, however, mean that you may want to take what you learned from the test to improve one or both of the onboarding experiences and conduct another test.
Running multiple tests enables you to continuously iterate and turn a good idea into a great product that users love. Look for patterns in the feedback you gain from users and increase the fidelity of your prototypes and the sample size of your tests as you become more confident that you’re building the right products and features. The value of product research comes from gaining insights throughout the product lifecycle.