Managing Director & Senior Partner
Traditional retailers generate and capture a deluge of data—most notably, customer transaction histories that can reveal detailed product affinities and promotional and marketing response rates. Now the emergence of big data and advanced analytical tools and techniques can connect data with a larger context. Big data can explain the who, what, when, where, why, and how of retailing.
Although some leading companies have gained a reputation for deft data handling, most retailers have not yet built the analytical capabilities and internal processes necessary to take advantage of the deep well of information they can access. Merchants and marketers often rely on tactics that worked last year, with only slight modifications. Sometimes their promotions end up discounting the wrong items and hurting rather than helping sales. Too frequently, they simply rely on consumer goods companies or suppliers, with their different incentives and motivations, to tell them what to do.
In the end, many retailers have not figured out where and why they are winning and where and why they are losing. They struggle to discover which prices, promotions, and store locations are working best. They have a hard time taking advantage of all the contextual information around transactions that could make a difference in sales. In effect, they know the outcomes of millions of real-time experiments, but they are not able to look at and learn from them.
All this leads to missed opportunities. Ultimately, it opens doors to online and direct sellers, which often have better data and more sophisticated analytics.
Big data can help turn this situation around. But in such a fluid technology landscape, it can be hard for retailers to tell where to focus their big-data efforts: which projects are a flash in the pan, and which ones will create growth opportunities and competitive advantage?
In our work with retailers across a range of market segments, we see three opportunities that offer high potential in the near term. Exploiting them can generate a significant increase in revenues and profits for retailers.
Boosting the Effectiveness of Promotions. Promotions are a common source of success and frustration for retailers. Overall, promotions tend to have a dramatic effect on retail sales. But once retailers start discounting, they are often hesitant to reduce promotional activity, because of the steady pressure to increase top-line growth. Making matters worse, most retailers do not have a handle on how their promotions actually perform.
Finally, many retailers rely heavily on suppliers for the selection of promotions. However, vendors often confuse retailers by providing conflicting consumer research in support of their brands. And they tend not to care if a promotion boosts sales for their product line while having the unintended consequence of lowering sales in the overall product category.
As a result, 30 to 50 percent of promotions have no positive impact on sales and margins. Even worse, many of them reduce profits without leading to additional sales.
Understanding the performance of promotions is difficult, because the necessary analysis is complex and because processes are interdependent. As a first step, retailers need to establish an accurate baseline of sales without promotions to understand the direct lift from a promotion and the cost of the discount. Retailers also need to tease out the secondary effects of promotions. Cannibalization can result when the promoted item reduces sales of other, substitutable items. A “halo effect” can result from the sale of additional products when shoppers come in for the discounted product. And the “pull forward effect” can depress item and category sales following a promotion, since consumers have already stocked up on the products. (See Exhibit 1.)
In addition, retailers must determine how much support vendors are really providing for promotions. That includes finding out how much of the funding that manufacturers provide is in fact associated with promotional activity. Retailers also need to capture the operational implications of promotions, including incremental marketing, supply-chain, and store-labor costs. Doing this kind of analysis on a single promotion requires good data and techniques but is not all that complicated for large retailers. However, executing this analysis across the thousands of item level promotions running concurrently and isolating external effects such as season and weather can be extremely complex, even for sophisticated retailers. It can be hard enough to determine what happened as a result of a promotion, let alone what might have happened if the company did not run the promotion. Even more challenging is to translate this analysis into tangible actions that buyers can take to improve performance.
In short, the analysis requires a huge amount of varied and detailed big data and advanced analytical techniques to sort out the interdependencies. But the payoff is worth the effort. We consistently see potential improvements in profit margins of around 1 percent of sales, depending on a retailer’s starting position and the share of income it derives from items on promotion. That can represent 20 percent of a retailer’s entire net income, which can be invested in lower prices, better store layouts, and improved shareholder returns.
One grocery retailer achieved these kinds of results after seeing poor performance from promotions. As a result of the kinds of advanced analytical techniques described above, the grocer was able to tailor promotions to products, with some brands receiving additional promotions, others a price cut, and some no promotion.
Targeting Pricing Precisely. Retailers often struggle to set the right prices while trying to provide a consistent pricing experience to customers across stores. We find that pricing zones, or areas with similar pricing, are typically determined on the basis of how a retailer organizes its activities geographically. Setting prices in this way often causes retailers to place a great deal of emphasis on city, state, and country borders. When people cross these sometimes arbitrary boundaries to shop, however, they see different prices from the same retailer.
As a result, customers might cherry-pick the best prices from nearby locations. Also, less price-sensitive customers might pay less than they may have been willing to, and more price-sensitive customers might not buy at all. Frequently, retailers create too few pricing zones, which limits their ability to respond in a more nuanced way to competitive pressures.
A better approach is to use individual customer behavior as the foundation for establishing pricing zones. Rich transactional data from loyalty programs and credit cards across thousands of stores can be used to understand where individual customers tend to shop. That data can be layered onto an analysis of current store locations and pricing zones. All this information can be combined, analyzed, and mapped to show geographic clusters of price awareness based on observed shopping behavior rather than artificially set boundaries.
This is the kind of complex, messy information from which big-data projects excel at generating insight. As a result of an advanced analytical approach, retailers can determine the likelihood of an individual customer shopping at different stores. They can then justify charging different prices across a region, be confident that pricing performance will not erode, and ensure that customers will not be likely to encounter noticeably different prices within the area where they usually shop.
Consider the case of a national retailer that used state boundaries to establish one set of prices across most of the Greater Atlanta area for a group of products essential to drawing customers to stores. An analysis of loyalty card data revealed considerable complexity in customers’ shopping patterns across Georgia and adjacent states. Even within Atlanta, the retailer discovered a wide diversity of customer demographics and shopping behaviors. For instance, because of Atlanta’s sprawl and difficult commutes, stores serving the same cluster of customers tended to be located along commuting corridors.
By examining where customers shopped, the retailer identified about a dozen unique store clusters in Georgia alone. (See Exhibit 2.) The analysis gave the retailer a strong rationale to establish as much as a 30 percent price difference for the set of products across store clusters, within a region that previously had had only one set of prices. The retailer raised prices in some store clusters while still offering the lowest available price in the area. At the same time, it lowered prices in other store clusters just enough to beat the local competition. Customers throughout the region saw competitive prices in their shopping area, while the retailer generated $5 million in additional profits within a single category from the combination of higher prices and additional store visits. Through a big-data analysis of billions of transactions, the retailer was able to scale the approach to the entire U.S., resulting in a much more refined approach to pricing, without causing customers to notice the multiple price points.
Understanding the Value of a Network. Retailers often view the value of their network of stores—the “last mile” of retail—in terms of the value of the land and the buildings. A network of brick-and-mortar stores is seen as a fixed cost tethered to a physical location. Because of the long-term costs of site location, real estate, and construction, changing a network of hundreds of stores can be time consuming and expensive. These days, executives often look at a network as a burden, rather than an opportunity. That burden is made even heavier as sales migrate online and leave many brick-and-mortar stores with too much space or an insufficient network to meet demand.
Big data reveals another way to think about store networks. One specialty retailer was able to unlock value from its retail footprint with an advanced analysis of its network. Consumer research suggested that a well-known technology brand had not achieved its full potential for sales among a key customer segment, in part because its existing direct-sales network was spotty in many areas. The retailer believed it could help the brand provide the necessary coverage for those businesses, but it needed proof.
The retailer decided to analyze how well 12 million businesses and 117 million households in the United States were covered by its network, by its competitors’
networks, and by additional retailers that could be partners with the technology brand. It used geoanalytical techniques to examine 3.4 billion point-to-point customer trips in order to identify gaps in the technology brand’s retail and partner network.
The results were clear: The specialty retailer was 37 percent closer on average to U.S. households and businesses than its competitors were. In fact, the retailer had the best coverage of key customer segments when compared with competitors and other potential partners in almost all U.S. regions. The company determined the true value of its last mile by looking at it through the lens of a complementary player.
As a result, the retailer was able to partner with the technology brand to generate $40 million in incremental revenues from its existing network at little additional cost. It also gained a new understanding of the value of its footprint. We believe that other retailers can achieve results from their network on the same relative order of magnitude by using a wide variety of tactics, aided greatly by big data and geoanalytics.
As retailers explore these opportunities aided by big data, they should take the following initial steps:
Retailers cannot afford to pass over these kinds of tangible big-data opportunities. Companies that excel at tapping into them can achieve measurable growth. The value they create will be long lasting.
ABOUT BOSTON CONSULTING GROUP
Boston Consulting Group partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities. BCG was the pioneer in business strategy when it was founded in 1963. Today, we work closely with clients to embrace a transformational approach aimed at benefiting all stakeholders—empowering organizations to grow, build sustainable competitive advantage, and drive positive societal impact.
Our diverse, global teams bring deep industry and functional expertise and a range of perspectives that question the status quo and spark change. BCG delivers solutions through leading-edge management consulting, technology and design, and corporate and digital ventures. We work in a uniquely collaborative model across the firm and throughout all levels of the client organization, fueled by the goal of helping our clients thrive and enabling them to make the world a better place.
© Boston Consulting Group 2023. All rights reserved.
For information or permission to reprint, please contact BCG at firstname.lastname@example.org. To find the latest BCG content and register to receive e-alerts on this topic or others, please visit bcg.com. Follow Boston Consulting Group on Facebook and X (formerly Twitter).