Listening to Users at Scale is Product Management

James Colgan, Senior Director of Product Management  for Outlook Mobile Enterprise at Microsoft, shares his approach to scaling a product and generating user feedback for an app with over 100 million monthly active users.


Microsoft Outlook has 100 million active monthly users. James Colgan is in charge of making sure that they’re happy. As the Senior Director of Product Management for Microsoft Outlook Mobile Enterprise, he works across multiple teams to drive adoption within the largest companies in the world.

Outlook Mobile is the largest-scale project that James has worked on, but he’s stuck to the tried and true principles that he’s developed throughout his career.

First, Outlook Mobile needs to be trusted by IT and loved by end users, James says. Second, he’s committed to moving fast. His team ships new features to the iOS and Android app stores on a weekly basis. This gives his team the agility and ability to be responsive to challenges and opportunities in the market. When operating at scale, James warns that it can be tempting to slow things down to be more contemplative and analytical, but says that this is a false tradeoff. His team is both data-driven and fast.

There wasn’t much data for James to rely on at first, so he and his team turned to app reviews aggregated across the iOS and Android app stores and focused on the 1-star reviews. They then prioritized feature roll outs that addressed common complaints. Though the app started at around 2.2 stars, today it is up to 4.7 out of 5.

The feedback his team was processing was, however, very North American- and European-centric. They had blindspots in Asia and South America. The team had some data but didn’t understand the “why” behind it. James sent teams to various countries in those regions to get a better sense of the context of how consumers used Mobile Outlook in their work and home lives by going to their homes and asking how they communicate and manage their time. They found that the work-personal life balance differed greatly among countries, which proved crucial in understanding why and how people use Outlook.

Mobile Outlook is unique in its highly developed in-app support, wherein users can have an instant message chat session with an Outlook Mobile expert. This helps the user get more value out of the app and acts as key signal for user pain points, which in turn feeds into how the team prioritizes new features.

The product manager is responsible for digesting all of the user insights from these various forms of research. James says that it’s crucial to contextualize long-term goals so that they can develop a tactical execution roadmap, which is actionable for engineering teams.

Product managers also have to be willing to recalibrate when presented with new insights. James had originally assumed that the more engaged users are, the more likely they would be to become monthly active users. But when looking at the data, he saw that the average session length was only 22 seconds long. Therefore, rather than focusing on email composition, he and his team shifted to focus on the consumption of information and ease of use for app navigation.

James says that there’s a common misconception that in order to increase the value of your product, you need to add more features. Often a product is born through the creation of a couple important features, but once the product devolves into a collection of features, then the user has a difficult time using it and loses sight of where your product fits into their life. He says that sometimes the need to pare back unnecessary bells and whistles.

You’ll learn a lot from this episode about customer-centricity, user experience, and user growth.

Here are the highlights:

James Colgan

Senior Director of Product Management for Outlook Mobile at Microsoft

Put our guest's advice into action.

Alpha is the world's most powerful product management platform for running experiments and generating on-demand insights.

Learn More

Subscribe to receive the latest episodes and exclusive resources

Related Episodes

187

Scaling Experimentation is Product Management

Ben Singer, Innovation Design Strategist at Humana, shares how he’s helped teams across the company embrace experimentation and overcome the challenges that innovators face at large organizations.

Listen to Episode
59

Adopting Mobile is Product Management

Warren Wan, Director of Product at Under Armour Connected Fitness, discusses what opportunities mobile presents for large companies, how to transition from a startup into a large company, and his user engagement lessons learned while at MyFitnessPal.

Listen to Episode
52

Building a Customer-Driven Product Team is Product Management

David Cancel, CEO of Drift and formerly Chief Product Officer at Hubspot, shares how to build great product teams and how autonomy increases employee retention and customer-centricity.

Listen to Episode

Unlock key insights from the first 200 episodes of This is Product Management!

Learn More