So you have an idea for a new product concept. You have a vision, a theoretical business model, and the ability to execute. If all goes to plan, you will improve the lives of millions of consumers and fundamentally reshape your industry.
But what are the first steps to turning a good idea into a successful product – to go from “Zero to One,” as Peter Thiel describes?
It’s important to first acknowledge that this is not an easy task. Most product concepts fail to become successful businesses. The most common reason for this is that, as Steve Blank said, “no plan survives first contact with customers.” In other words, product managers need to continually adapt to changing customer needs and behaviors.
Fortunately, the product leaders and entrepreneurs who I’ve interviewed on This is Product Management have shared the strategies and tactics that have contributed to their success in doing just that.
The most accomplished guests have mastered the art and science of collecting user insights, using that data to make good decisions, and iterating quickly. During this process, they find ways to create a better user experience and increase the chances of building a successful product.
This article covers seven strategies I’ve learned for testing a product concept and making decisions that lead to successful products.
1. Make small bets
Turning a product concept into a successful business requires making a nearly infinite number of decisions – ranging from what color your logo should be to what market to serve.
However, in product management, it’s difficult to predict the outcome of a decision. Unlike in a game such as chess, you never have all the information you need to know what the best decision is. If you’re creating a new experience, you don’t have historical usage data to rely on, and customer needs and market dynamics are always changing.
In “Thinking in Bets is Product Management,” World Series of Poker Champion turned business consultant, author, and speaker, Annie Duke, provides strategies for making good decisions when you don’t have all of the information that you need (for example, every product decision, ever).
She recommends considering all of the potential outcomes of your decisions and then estimating the probability of each potential outcome. For example, “what is the probability that this product concept can become a billion dollar business?”, “What is the probability that ‘feature set A’ will lead to a greater conversion rate than ‘feature set B’?”, and so on. Annie then recommends gathering information to get a more accurate estimate of the probability each outcome.
For product managers, this means running experiments that provide data on what users want and using that data to build products that have a greater probability of achieving success.
2. Run tests with real users
Nearly every leader I’ve interviewed incorporates user feedback into their decision making process. The first step they take after coming up with a new product or feature concept is to test it with real users. Running tests with a small group of real users gives them the data they need to know if the idea has real potential to succeed, or it needs to be refined.
There are several effective ways to conduct product research and test new concepts, including:
- Conducting user discovery surveys and interviews
- Analyzing user behavior with competitive products
- Creating visual prototypes or landing pages to test with users
These strategies, even if deployed on a small population of users, can provide you with critical data that will help you make a decision to either invest additional resources into the idea, iterate on it, or scrap it all together.
If the data tells you that the small population of users like the idea, it doesn’t necessarily guarantee that building the product is the right decision. It does, however, provide you with data to indicate that building the product is more likely to be the right decision. Incorporate the insights you glean from testing to upgrade your product concept and then run larger experiments once you have the data to give you confidence in your decisions.
3. Select the right audience
In “Vision and Strategy is Product Management,” Marty Cagan, Author of Inspired: How To Create Products Customers Love, describes product strategy as “a series of product-market fits.” In other words, he identifies specific markets the product will serve over time. For example, Airbnb initially focused on people traveling to cities for major conferences in events. After conquering that segment and many more, the company is now working to serve business travelers.
When running experiments, it’s critical to source an audience that matches your target market. When defining your audience, think less about customer personas and more about your customers’ “Jobs to Be Done.” You’ll gain more relevant and actionable insights if you source an audience based on past experiences, behaviors, and perceptions, instead of by demographic factors.
For example, if you’re working on a healthcare product, consider an audience such as “Aetna customers who need medical coverage for diabetes” instead of an audience such as “Florida-based real-estate brokers in their thirties with two children and under $300,000 in household income.”
For early product concept testing, the goal is to gain insights quickly and use those insights to inform the next series test. You don’t need a massive sample size of thousands of users. It’s better to iterate quickly with a small sample size of users than to not be able to iterate at all.
4. Synthesize your insights
After running a few tests, you’ll have some great data to help you decide what to build. Here’s what those results look like on Alpha:
Synthesize the data and feedback you gain from each experiment. What did you learn? How likely is it that your concept has legs? What can you change to make it more likely to succeed?
Build on what users liked and scrap what they didn’t. Run more tests to explore interesting new insights that came up.
Holly Hester-Reilly, Product Consultant and former Product Owner at Shutterstock, shared a case study on how she did this in “High-Impact Experimentation is Product Management.” While she was working on an online learning platform, she found that users who were looking for new jobs were most engaged with the platform. This was a promising growth opportunity for her team to explore.
While launching a new product to help social media managers to create graphics, Holly tested a prototype with her target users. She learn that users needed greater customization capabilities. The team would not have known that this was a need until running these tests, but after running the tests, they were more confident that building additional customization features would pay off.
5. Make larger bets
The product concept testing strategies that I’ve covered so far are designed to provide you with actionable user insights, quickly. These insights are directional in nature. They equip you with confidence that you are building the right product and provide opportunities for further improving it. As you gain additional data, and additional confidence that your product concept will be successful, run more robust experiments.
In “Revenue is Product Management,” Jeff Morris Jr., Director of Product Management, Revenue at Tinder, shared a behind the scenes look at the launch of Tinder Gold, the company’s flagship monetization product. Instead of making a large investment in a big launch, Jeff and his team tested Tinder Gold in a few cities to validate that they should roll it out to the rest of the world. Tinder has since become the #1 revenue generating app in the Apple app store.
6. Continue iterating
Experimentation and iteration doesn’t stop after launch. There are always improvements to make, particularly as consumer needs and behaviors continue to change.
Sam Nordstrom, Product Manager at Intuit, provided an in-depth look at how he uses data to iterate on live products in “Becoming Data-Driven is Product Management.” He analyzes the five key moments along his users’ journey – discovery, engagement, conversion, retention, and referral – to get and indication of how much value users are getting from the product. Sam then works to determine where users are dropping off in this journey and to improve the user experience accordingly. This approach lead Sam to discover that engagement on Intuit’s invoicing product was low because users were sending invoices as attachments via email. He then iterated on the product so that users could send invoices from within their inbox – the competing behavior that they were already comfortable with.
In “Mastering Quantitative Research is Product Management,” Suhail Doshi, CEO of Mixpanel, provides several strategies for making better decisions using in-product analytics. I particularly appreciated how he uses paired metrics to evaluate the success of experiments. Measuring success based on two metrics, instead of just one, hinders you from hitting one metric, such as conversion, at the expense of another, such as retention.
7. Develop your product concept with speed
You now have a framework and several steps for developing your product concept. In today’s world, where consumer needs and behaviors are rapidly changing, speed is on of the most critical ingredients for success.
The most effective product leaders don’t wait for the perfect idea or the perfect plan to get started. They start running tests, use what they learn to inform the next set of tests, and eventually develop concepts into groundbreaking products that customers love.
Amazon CEO, Jeff Bezos, shares why speed is so important in his 2016 letter to shareholders:
“If you’re good at course correcting, being wrong may be less costly than you think, whereas being slow is going to be expensive for sure.”
You need to be able to experiment and iterate rapidly because if you don’t, you will fall behind whichever competitor does.