An innovation lead at a consumer company dreams up a new product attribute. A chief merchandising officer at a retail company wants to put a trendy product on the shelf. In the past, these executives would conduct market research to get real-world input from consumers before moving forward. Today, there’s another option: running a synthetic panel.
The So What
Synthetic panels are GenAI-based tools that can respond to surveys, based on personas with defined demographic and psychographic attributes. In that way, they can generate research outputs that are as insightful as if they came from real people.
In a recent study, we tested synthetic panels through a conjoint analysis and found that they were able to predict the choices of real-world consumers for a new beverage with 92% accuracy (with fine-tuning of research outputs over time). We’ve seen similar accuracy rates through client work with leading food and beverage, household goods, and apparel companies, and we’re working with our clients on synthetic panel applications in other industries as well.
In our experience, synthetic panels can enhance the product innovation cycle in three ways:
- Early Screening of Initial Concepts. Synthetic panels can help to quickly refine innovative ideas in the pipeline. For example, fast reactions to initial sketches can eliminate the need for multiple rounds of creating samples, which can be costly.
- Attribute Selection. Panels can also quickly assess how consumers will react to certain product attributes (for example, an organic version of a grocery product) and model their potential impact on sales.
- Pricing and Promotional Strategies. Digital consumers can predict whether real consumers will buy an item at a particular price or promotional discount.
Collectively, those benefits lead to more efficient market research and higher success rates in innovation. As a result, companies can increase sales, even as they reduce operating costs in areas like sampling, quality assurance, or markdowns.
Stay ahead with BCG insights on artificial intelligence
What Else
As with any AI tool, companies need to understand where the technology works best, where it may fall short, and how they can mitigate any risks from it.
A Complement to Traditional Market Research. Synthetic panels can’t replace traditional market research, which will always be the primary solution for applications like assessing a radically new product idea. But for certain use cases—like predicting the impact of pricing changes, gauging how well a product fits into an assortment, or testing marketing claims—synthetic panels can be a valuable tool.
With the right level of training, fine-tuning, and oversight, these tools enable companies to iteratively test a wider range of variables and attributes than if they relied exclusively on human respondents. Leaders can generate reliable results at lower costs and be more confident about the potential impact of changes before actually executing them.
The Limits of Synthetic Models. As with all forms of AI, synthetic models face inherent model risks from the underlying training data, which may be outdated or inaccurate. They can also produce seemingly reasonable results that overlook minority views, and they only consider attributes that human-based surveys have already assessed.
Another consideration is measurement risk: the confirmation bias that AI models can fall prey to. A recent study found that synthetic respondents can sometimes infer the researchers’ hypothesis and produce data that artificially confirms it.
Now What
Given these constraints, companies need a systematic approach to implementing synthetic panels and enhancing the product innovation cycle.
Start by determining which decisions you will allow synthetic panels to influence. Group decisions into tiers:
- Low-risk, high-iteration decisions like ideation, naming options, and basic product claims, where synthetic panels can serve as a primary research solution.
- Medium-risk decisions like packaging selection and product attributes, where these panels can support traditional research outputs.
- High-risk decisions like regulated claims or forecasting, where synthetic panels should be subordinate to human testing.
Notably, companies should avoid synthetic panels in applications where limited training data exists, along with those involving culturally sensitive topics.
Run paired studies to calibrate results. Pick two or three categories and run a synthetic-panel study alongside a high-quality human panel study, with a behavioral check where possible (for example, an in-market A/B test, conjoint validation, or sales pilot).
Choose vendors based on overall quality, not demo quality. Consider factors like whether the model is trained or conditioned on real panel or behavioral data and how often it’s refreshed. It is also important to assess if the vendor publishes comparisons against human samples and whether a synthetic respondent can remain consistent across a long interview and over time. In addition, make sure the vendor has clear data processing agreements and enterprise-grade security.
Build a playbook for consistent use across your organization. That playbook should include standards for prompts to reduce variance, required disclosure language when synthetic data is used (both internally and externally), and quality checks like hallucination filters, distribution sanity checks, and drift monitoring.
Focus on developing capabilities, not just adopting tools. The organizations that win with synthetic research build governance with legal and privacy teams early and train their researchers and marketers on what to do when synthetic panels fail. They also redesign their innovation cycles around rapid iterations, such as weekly learning loops instead of quarterly reviews, to build up organizational capabilities with the technology over time.
The bottom line? With the right oversight, synthetic panels can supplement traditional research and dramatically accelerate innovation cycles. In that way, they can help companies in all industries gain a competitive edge.
