Product teams are facing an unprecedented paradox: we've never had more qualitative customer data, yet we've never felt further from truly understanding our customers. In this post we discuss:

• The Feedback Fragmentation Tax: How more qualitative data + more tools + more owners has led to less insight and worse product outcomes

• The four different types of tax stemming from Feedback Fragmentation and their impact

• Why solving this is more important than ever in ord shift to AI-Native Product Teams

• How to fix it with Triple AAA Insights: Aggregate, Analyze, Act

• Reforge has acquired Monterey.ai to launch Reforge Insight Analytics

How The #1 Job Of Product Is Becoming Impossible

The fundamental job of product teams is to understand the customer. This probably sounds like an obvious statement. But that’s becoming nearly impossible. Not because we don't want to listen, but because we're drowning in a sea of customer feedback that we can't effectively process. There are three trends I’ve been paying close attention to for the past few years that are converging.

AI Is Enabling Companies To Collect More Qualitative Data

The amount of qualitative data companies are collecting from customers is accelerating fast. Sales call recordings, success call recordings, customer support emails, customer research calls, user research surveys, slack community groups, app store reviews, product review sites, social media comments, in-product feedback forms, and a lot more.

This trend has been fueled by two things:

1. AI has made things like audio recording transcripts easy and cheap enabling us to capture data where we weren’t able to before.

2. Customers are more vocal than ever publishing their thoughts and opinions on products publicly on social media, review sites, community groups, and more.

But That Data Is Fragmented Across Many Tools

But more data isn’t better if you can’t put it to use. If you look at the average midsized company they are capturing qualitative data across 8+ places:

• Internal Data - Tools like Gong, Zendesk, Intercom, Grain, Slack Channels, Sprig, SurveyMonkey, UserTesting, Google Sheets and CSVs.

• External Data - Customer comments across social media (LinkedIn, Twitter, etc), communities (Slack Groups, Facebook Groups), and review sites (App Store, G2, Capterra).

Each one of these tools creates it’s own mini-data silo.

Those Tools Owned By A Disparate Set Of Functions

These tools are each owned by a disparate set of functions within a company. Here is what a typical B2B and B2C company looks like:

• Sales own the CRM and sales recording tools.

• Customer support owns help desk.

• Success owns the success platform and call recordings

• User Research owns user research tools and repositories

• Marketing owns social media and review sites.

• Data owns data warehouse

• Product owns Google Sheets, in product feedback tool, etc

The Feedback Fragmentation Tax

More Data, Less Insights, Worse Product Outcomes

The fundamental challenge isn't just connecting with customers - it's transforming the massive amount of customer feedback into actionable product intelligence. There is a vast hidden cost that companies pay when that feedback is fragmented across tools, teams, and systems. - The Feedback Fragmentation Tax. Despite having more customer data than ever before, this tax is growing on product teams making it harder to build winning products.

The Four Types Of Feedback Fragmentation Tax

There are different types of fragmentation tax, but one common theme. They all add up to lost revenue or bloated costs.

⌚️ The Time Tax

The cost of teams endlessly hunting for and recreating customer insights that already exist.

• Product teams spend hours hunting across tools for relevant feedback

• Teams repeatedly "discover" insights that already exist somewhere

• Knowledge gets recreated rather than reused

• Decisions that should take days stretch into weeks or months

• More people and process to aggregate, analyze, and distribute the feedback create libraries of decks and docs that become insight graveyards.

As one VP of Product that we surveyed explains:

"Our team spent a ungodly amount of hours last week just trying to find relevant customer feedback for one feature decision. The data exists—it's in our sales calls, support tickets, and research reports. But it's like trying to piece together a puzzle where the pieces are scattered across twenty different rooms."

☎️ The Translation Tax

The erosion of customer truth as feedback gets filtered through layers of hand-offs.

• Instead of the builders having direct access to the feedback, teams compensate with more meetings and hand offs to translate the feedback

• Customer stories get stripped of crucial details as they move through organizations

• Instead of building intuition from direct customer quotes and conversations, product teams rely on filtered summaries that lose critical context

• Product decisions get made on sanitized summaries instead of rich customer narratives

From a Director of Product at a leading SaaS company:

"We've created this elaborate game of telephone. Our CSM team talks to customers, summarize it for researchers and product ops, who summarize it for product managers, who summarize it for engineers, who try to build what they think customers want. By the time we ship, the real voice of the customer is so diluted it's unrecognizable."

🤔 The Trust Tax

The friction of product decisions when every department claims their slice of customer feedback is the most important.

• Different teams don't fully trust data because they can't see the complete picture

• Every department believes their slice of customer insight should drive product direction

• Product teams spend more time defending decisions than making them

• Roadmap discussions become political negotiations rather than strategic planning

• Instead of clear ownership over product direction, product teams face constant pressure from other departments trying to dictate the roadmap

As one CPO recently shared:

"We're no longer driving product strategy - we're negotiating it. Sales wants X based on their customer calls, Success wants Y based on their feedback, and Marketing wants Z based on market research."

🏗️ The Technical Debt Tax

The mounting cost of maintaining features without clear evidence of their value.

• Dead-weight features persist without clear evidence of value or impact

• Valuable engineering resources get drained maintaining features with unknown or minimal usage

• Product complexity grows creating customer confusion and frustration

A frustrated CTO recently shared:

"We just did an audit of our product. We're actively maintaining over 100 features, but we only have confidence in < 50% of them. The rest? We're too scared to remove them. The quantitative data doesn’t tell the whole story and because the customer feedback is so fragmented and not tied to other key data like customer type, we can't tell if they're actually creating impact or not. It's probably costing us millions annually in engineering resources to maintain features that might be completely useless."

Qualitative Data Is More Important Now Than Ever

In the race to build better products, we've become addicted to quantitative data. Every click and conversion is meticulously tracked and analyzed. But as AI reshapes product development, and we shift to AI-native product teams, the teams that win won't be the ones with the most sophisticated dashboards to show what people are doing—they'll be the ones who can rapidly capture and leverage qualitative insights from their customers to understand why those behaviors matter.

An Over-Reliance on Quantitative Data

For the past twenty years, quantitative data has dominated product development. We've built sophisticated dashboards, A/B testing frameworks, and metric-tracking systems. While quant data is important, there are three problems with this:

1. Metrics tell you the what, but have always struggled with the why.

2. Too many metrics dehumanize the customer.

3. An over-focus on quant data blinds teams to emerging opportunities that aren’t yet measurable.

Joff Redfern, Partner at Menlo Ventures, angel investor in Reforge and previous CPO of Atlassian said:

“Since the lean product movement we went all in on quant data because qualitative data was too hard. Metrics led to more metrics and this dehumanizes the customer in product development.”

Qualitative data has always been the other half of the picture. But it’s historically been high friction, slow and expensive. Despite that, the “why” in qualitative data is where the magic often emerges:

• The right words to tap into your customer’s problems.

• The insight that informs a new breakout feature idea.

• The narrative that breaks through the market noise.

Behzod Sirjani is a former user research at Facebook and Slack and an advisor to Figma, Dropbox, Replit, and OpenAI. He is the creator of Reforge’s Mastering Customer Feedback course he says:

“Great product teams have always used both qualitative and quantitative data. They start with qual to understand what people are doing and why, then identify the quantitative metrics that best track those key behaviors. Sadly, too many companies skip the first step and instead focus on what they think matters. This stops them from building the best experiences for their customers.”

AI Accelerates The Need For Low Friction, Fast, Inexpensive Qual Data At The Fingertips Of Builders

As we shift to AI-native product teams, qualitative data from your customers becomes 10X more important.

1. AI Tooling Requires Customer Understanding AI is enabling product teams to draft PRDs, product strategies, GTM narratives in addition to writing code and designing faster. But to get the most out of those tools, they need an understanding of your customers at scale.

2. Feature Execution Becomes Commoditized As AI accelerates product development and competitors can replicate features faster than ever, differentiation will from deeper customer understanding, not just feature execution.

3. Rapid Shifts In Customer Expectations AI is transforming how users interact with products. These rapid shifts in user expectations mean we can’t rely solely on historical usage data to guide decisions.

4. AI Experiences Are Not Easily Quantified AI experiences are non-deterministic. Measuring their success with traditional product metrics paints an inaccurate picture. Understanding users’ qualitative perception becomes more important.

Putting qualitative data on an equal or greater level with quantitative data is a strategic imperative for building an AI-native product team. As Zach Cohen from Andresseen Horowitz put it:

“With the emergence of LLMs, web-based agents, and multimodal models, we can now collect, comprehend, and integrate unstructured data with quantitative information to achieve a more holistic understanding…The future of analysis isn’t just numerical; it’s contextual and dynamic…This convergence of qualitative and quantitative data will be a strategic wedge for building the large, AI-native companies of the future.”

Acquiring Monterey.ai To Create Reforge Insight Analytics

We believe in this problem so much, that Reforge has acquired Monterey.AI to create Reforge Insight Analytics help product teams solve The Feedback Fragmentation Tax. I had the opportunity to talk to 10+ teams working on this problem. By far, the Monterey team of Chun Jiang, Ben Kramer, Cole Hoffer, and Jacob Hubbard was the most thoughtful, fast-acting, and talented teams. We are beyond excited to have them joining Reforge. Let me show you how we are solving this problem.

Triple AAA Insights: Aggregate, Analyze, Act

No, not the AAA the auto industry. In gaming, there are games called AAA (Triple A). These are the franchises that create the blockbuster value. Similar with qualitative data. How do you go from raw feedback to the insights that help you create blockbuster products?

• Aggregate - Eliminate the disparate data silos.

• Analyze - Find unique insights you never would have before.

• Act - Seamlessly weave them into the every day of your entire product team.

Aggregate

The first step is to aggregate all the disparate data silos into one place. This is the foundation because if take garbage in, it’s going to be garbage out. Monterey by Reforge solves this:

• Smart Cross-Platform Integration: Unifies feedback data from Zendesk, Intercom, Front, Discord, Slack, Gong, and many other sources into a single repository

• Quality Data Ingestion: Each data source isn’t create equally. It ingests each one differently to ensure quality data in.

• Feedback Capture: In-App Widget, Chrome Extension, External Customer Portal to easily capture more feedback in a more personalized way.

Analyze

Analysis of qualitative data use to take an immense amount of hours often losing key insights in the process. With Monterey by Reforge you can:

• Insight Reporting Tied To Impact - Automatically identify top complaints and requests driving key business KPIs (ARR, Retention, NPS, App Rating, etc.).

• Customer Segmentation - Analyzes customer sentiment across data sources with filtering by ARR, demographics, and more

• Realtime Anomaly Detection and Notification: Highlights new insights and alerts teams to anomalies in real time

• Intelligent Search & Chat: Uses natural language to search and compare product features, helping teams build segment-specific customer understanding

• Deep Dive Interactive Charts: Create interactive charts across data sources with advanced filtering.

• Compare and Find Root Causes: Compare feedback across metadata and dive into the root causes in the most iterative way.

Act

It’s not good enough to just aggregate and analyze the data. For it to create product outcomes it needs to be applied on an every day basis. Monterey by Reforge works in the tools your team already uses.

• Incorporate Feedback In JIRA, Notion, Google, more - Get instant customer feedback incorporated in all the tools your team uses in product development.

• Draft Based On Customer Insights - Instant drafting of Weekly/Monthly Updates, PRD’s, Product Strategies, Launch Posts, and more based on your customer insights.

• Route To The Right Team In The Right Place - Automatically direct feedback to the right teams in Slack and other tools they are using.

• Find The Perfect Users To Survey - Auto generate a list of the right users to target based on the questions you need answers to.

• Close The Loop With Users - Personalized emails when feedback is addressed.

Historical Solutions Make Feedback Fragmentation Worse

This problem has been tried to be solved before. Most teams have gone through a few waves.

Stage 1: Scale User Research Teams

The first attempt at solving this was by trying to scale user research and/or product ops teams. Scaling these teams creates a few problems:

• Creates gatekeepers and bottlenecks around customer research

• Puts up walls between the builders and the customers.

• You end up with endless presentation decks, Google Docs, confluence pages, and more that rarely get read.

• Slow cycles between question and answer to be slow and expensive.

It was the wrong tool for the problem. User research teams should be used for very large, forward-looking, ambiguous decisions that need an intense methodology.

Stage 2: ChatGPT, Claude, Etc

The past couple years some individuals have started to use ChatGPT, Claude and other horizontal AI tools to assist them. While it helped eliminate some incremental problems, teams quickly run into a number of new ones.

Data sources aren’t aggregated in one place. They can handle a limited amount of qual data. They don’t marry the qual data with the quant data. Each individual uses their own prompts and ways to analyze the data. Lots of manual work to export, import, export over and over.

The end result is that ChatGPT and Claude don’t filter through the noise and bring the best practices and insights in highly sophisticated product development context; the insights can only be delivered when ai really learns about your business / internal data sources.

Stage 3: Reforge Insight Analytics

We are now entering the next phase with robust qualitative data platforms enabled by AI. Prior to acquiring Monterey we had been using the product for months internally at Reforge. These are just some of the results we’ve experienced:

• From lost/repetitive insight to finding new, unique insights we could have never done with humans or manual analysis.

• Spending hundreds of hours manually processing feedback, to customer understanding happening instantly as new feedback comes in.

• Slow cycles between feedback and product decisions, to accelerated product decisions.

• A few people being bottlenecks of feedback, to the entire product team (engineers, designers, and PMs) enabled with it at their finger tips.

Reforge is moving beyond expert education. Reforge Insight Analytics is our first in a suite of unified tools to help transition to becoming an AI-native product team. In the next couple of months we’ll be launching more tools. Subscribe here, to get notified when they launch.