Developing an Experimentation Organization is Product Management

As we move into the digital age and as the number of touchpoints with customers explodes, there will be many decisions businesses have to make. How will they make them? Hopefully not by intuition and gut. The only way to adjudicate all these decisions is through experiments.”

Stefan Thomke, professor of business administration at Harvard Business School and author of Experimentation Works: The Surprising Power of Business Experiments for Innovation, discusses how experimentation can influence an organization’s ability to innovate. 

Stefan Thomke has been an authority on experimentation for 25 years, long before the concept went mainstream in product management.

Before joining Harvard Business School as a professor in 1995, Stefan worked as an electrical engineer. That’s where he first discovered the business benefits of running disciplined experiments. He has been studying the principles of experimentation since then, and, drawing from decades of research, has authored several articles and books on the topic.

In his latest book, Experimentation Works: The Surprising Power of Business Experiments, Stefan makes the case that any organization looking to innovate and evaluate new ideas must run experiments.

“Mark Okerstrom, who was the CEO of Expedia, told me that in an increasingly digital world, if you don’t do large scale experimentation in the long term, and in many industries the short term, you’re dead. That kind of caught my attention. That was the starting point of this new book,” Stefan says.

These days, nearly every organization wants and needs to innovate their offerings. But innovation carries a lot of uncertainty and risk around product development, scaling, customer experience, and market opportunity.

Stefan believes experimentation is the missing link. When the answer isn’t obvious or the problem is too broad, experimentation can help manage the uncertainty and mitigate the risk.

“What’s the problem with innovation? Well, we usually don’t know what works. Innovation activities can be highly variable,” he says. “Experimentation is a surefire way of dealing with some of these challenges that we face in innovation. And I think investing in experimentation and building experimentation capabilities makes a ton of sense. And it pays off in a very big way.”

In a fast-moving digital world, experimentation needs to be the engine of innovation. A company’s ability to create new products will depend on its ability to create a culture of experimentation.

According to Stefan, companies must revamp their processes and understand the impact of new technologies. From there, leaders must set clear goals for experimentation, encourage curiosity in their teams, and accept challenges to their opinions.

In this episode, you’ll learn a lot about innovation, creating a culture of experimentation, designing good business experiments, and leadership strategies.

Here are the highlights: 

Stefan Thomke

Professor at Harvard Business School and author of Experimentation Works: The Surprising Power of Business Experiments

Put our guest's advice into action.

Alpha is the fastest way to learn about your future customers.

Learn More

Subscribe to receive the latest episodes and exclusive resources

Related Episodes


Scaling Experimentation is Product Management

Ben Singer, Innovation Design Strategist at Humana, shares how he’s helped teams across the company embrace experimentation and overcome the challenges that innovators face at large organizations.

Listen to Episode

Embedding Experimentation is Product Management

Anuraag Verma, Global Head of Strategic Transformation at Alpha, shares how large organizations can use experimentation principles to de-risk decisions, test critical assumptions, and get innovation to stick -- and scale.

Listen to Episode

High-Impact Experimentation is Product Management

Holly Hester-Reilly, Product Consultant and former Product Owner at Shutterstock, shares how she runs experiments to test critical assumptions and make better decisions throughout the product lifecycle.

Listen to Episode

Read our 2020 Product Management Insights Report

Get Report