Managing in Times of Big Data and Analytics
This is the third in a series of articles on setting and executing digital strategies with speed, foresight, and savvy. The first article explores how companies can frame and prioritize digital opportunities, while the second article looks at the actions companies can take to make their digital strategies a reality.
Digital technologies are profoundly transforming not just products, services, and business processes but also management itself and the inner workings of companies. A new generation of executives is relying on data analytics and artificial intelligence to support and make decisions.
Our aim is to demystify these once-arcane fields and show how they have moved out of the laboratory and into the executive suite. Executives need to address several fundamental quandaries: How should you, your team, and your organization evolve? What does it mean for leadership and management behavior, as well as for organizational structure and the composition of teams? What skills will employees need, and what levels of performance can you expect from them? How will the role of IT systems shift as their ease of use, flexibility, and intelligence increase?
We are at the cusp of a new era of data-driven management. The rules are still being written, but the broad outlines of what needs to change are sharpening. As an executive, you have a chance to tap into new sources of power and knowledge, but only if you upgrade your own analytical arsenal and model the change in behavior that you expect of your people.
Data-driven management touches the entire organization. It is not limited to a particular function or activity. Many executives, however, see data-driven management only from within their own silo. Marketing may see it as an exercise in consumer insight and churn reduction. Manufacturing may view it as a tool to optimize processes and effectiveness. Bankers may focus on the potential to use data analytics to reduce fraud.
Viewed more holistically, data-driven management refracts into five major sets of activities: innovative offerings and business models; operational analytics; customer and commercial analytics; enterprise and risk analytics; and the underlying systems and capabilities that support all of the above. (See Exhibit 1.)
Even at the functional program level, data-driven management is a challenging activity that can create substantial value—or destroy it through ill-fated projects. Consequently, executives should refrain from delegating initiatives to lower-level teams, which may be constrained by traditional processes and structures. In particular, they should learn how to avoid the stumbling blocks that others have encountered along the way. (See “Five Levers to Create Value from Data Analytics.”)
Many organizations struggle to embed analytics into their day-to-day business. Attention to five areas will help you leverage data for business and avoid common pitfalls.
Framing Initiatives: Bridging the Divide Between the Business and the Quants. Standalone analytics teams commonly face challenges similar to those of IT departments. When business teams set the agenda, analytics people often do not know how to execute what they view as ill-posed requests. The converse is also true: business leaders frequently struggle to create value from initiatives suggested by self-directed data analytics teams. Solutions to this dilemma vary, but increasingly companies are providing a tighter integration between the quants and the execs, often by relying on the fast and frequent feedback mandated by agile methodologies.
Data Collection and Analysis: Managing an Embarrassment of Riches. Acquiring, preparing, and integrating data are more labor-intensive than analyzing it. Internally, firms struggle with moving their data across organizational boundaries and properly cleaning, standardizing, and checking it. External data, which is frequently unstructured and “noisy,” is even harder to harvest. Not surprisingly, several high-profile data analytics providers are primarily sophisticated data integrators. Still, their technologies and methodologies do boost companies’ ability to make sense of a jumble of data sets and eliminate the need for drawn-out, painful database integration projects.
Review and Decision Making: Opening the Black Box. As pricing, marketing, trading, and other activities become increasingly automated, companies must make sure that they maintain full transparency. A major airline introduced a highly successful pricing engine whose performance gradually deteriorated. When senior executives eventually broke open the system, which had effectively become a black box, they discovered that its algorithm did not address critical and widely available customer experience and competitive information. The airline now relies on a new system that is intuitive, flexible, and open—and that can be overridden. This lesson speaks to the need for business executives to get their hands dirty with data and insist on transparency and clarity.
Implementation: Anticipating Substantial Change. Companies that generate insight from data must initiate change management programs to realize value from this new approach. Several utilities, for example, were unprepared to handle fraudulent uses that their new smart-meter-based detection algorithms flagged, and they did not have a plan to turn free riders into paying customers. Likewise, a retailer introduced an analytics system to help its forecasters, who rejected the system’s recommendations because they felt threatened. Both companies failed to put in place change management programs that anticipated and addressed potential outcomes. Data, in other words, is a powerful beast, so prepare your people and processes to leverage its strength.
Improvements and Foresight: Thinking Big, Acting Small. Companies need to have both a big-picture target state for a data analytics initiative and a set of manageable goals along the way. The CEO of a large electrical company told us that the most important thing he learned from failures was to “slice the white elephants” and launch minimally viable products that could be improved over time. In other words, he wants to play the double game.
Faced with a flurry of promising initiatives and the challenge of managing the portfolio of “use cases” for analytics (its myriad applications across the business), it is easy for executives to fail to shift their focus from today’s weather to the broader climate, from tactics to strategy. They frequently do not think deeply enough about how to create an effective data-driven company as a whole, leaving their business exposed today and unprepared for things to come.
Executives who want to sharpen their analytic edge and manage a data-driven company need to start with themselves, then move on to their team and finally to their organization. Such a transformation requires actions along four critical dimensions, as well as a double-game perspective. (See Exhibit 2.)
Leadership: An Analytics-Driven Business Mindset. The best data-driven companies have a data-driven culture. Executives at these companies—ranging from digital attackers, such as Amazon, to century-old companies like Procter & Gamble—make decisions on the basis of rich, near-real-time data. Data analytics have evolved into a core general-management skill, similar to corporate finance and cost accounting. While executives would never base their decisions entirely on such information, they also would not proceed without it. They are comfortable leading discussions and teams centered on data analytics.
Data-driven companies embrace the democratization of data. Managers and employees can access and interpret a wide range of company data using intuitive self-service tools. They explore topics and generate answers more quickly and with less friction than if they had to go to specialized data analytics or business intelligence teams.
At Amazon, many managers spend 5% to 10% of their time working directly with databases, constantly striving to define and measure the right metrics. In meetings, they are expected to condense their findings into concise, data-rich documents that will frame subsequent discussions and decisions. This culture extends all the way to recruiting, with candidates for management positions assessed on their skills and eagerness to get their hands dirty with data.
At P&G, managers have access to standardized datasets, powerful data visualization tools, intuitive dashboards, and immersive conference rooms with large wall-screen data displays. More than 50,000 employees also have access to a “decision cockpit” that displays critical data in near real time. These methodologies collectively form a sort of common language.
Organization: Cooperative and Agile Ways of Working. The organizational challenges of creating a data-driven company fall into two main areas:
A company’s choices will depend on its specific capabilities and its need to boost entrepreneurial activity, reduce friction, and foster cooperative performance. A global beverage company wanted to retain local entrepreneurship, its core strength, while leveraging analytics to improve the customer experience and its own go-to-market activities. Local teams, however, lacked data analytics skills and scale. Headquarters could provide the data analytics but was too far removed from regional operations.
Ultimately the company decided to create a global SWAT team consisting of ambitious, business-savvy executives paired with analytics and IT executives. This team worked with local teams to launch projects for specific use cases in individual markets and was responsible for platform development and knowledge transfer. The approach balanced speed, scale, and cross-regional fertilization with strong local buy-in and skills building.
A conglomerate whose many different businesses each required specific know-how faced a related centralization-versus-decentralization challenge. Being more technology savvy, this company embedded dedicated analytics teams into every business unit and also created a unit in the center. The headquarters unit provided support by staying attuned to the latest in algorithmic research and technology, building common technology platforms, and supporting the businesses through decentralized skills development.
In parallel, companies should improve the cooperation between business owners and quants by embracing agile principles that originated in software development two decades ago. Agile is becoming the method of choice for organizations aiming to transform areas where time to market is critical.
At a leading retailer, for example, the analytics team developed new applications with business owners by forming small, multidisciplinary project teams. These teams had authority to act independently. Working in two-week “sprints,” they created working versions of an application and then sought direct user feedback that they used to build the next version. This iterative approach drastically improved mutual understanding, shortened development time, and reduced delivery risks.
Skills: Simultaneously Hiring and Transforming. Organizations need to evolve and refresh their skills to address the fast pace and changing requirements of data analytics and other technologies. In doing so, they need to build up their dedicated analytics teams—predominantly by hiring—as well as train their core workforce.
AIG, a leading insurance company, created a “science team,” hiring 90% of its 130 members externally. Recognizing the need to blend data analytics into the fabric of the business, the team recruited behavioral economists, psychologists, and change management experts in addition to analytics specialists. This cross-functional team created not only sophisticated novel solutions but also creative ways to implement them. (See “How AIG Moved Toward Evidence-Based Decision Making,” Harvard Business Review, October 1, 2014.)
Other companies have hired “data driven” officers, ranging from middle managers to senior leaders, as a way to create a new mindset. An executive from a major online retailer put it bluntly: “Experienced external hires for management positions are often not used to drilling down to the raw data. They come from the ‘aggregated’ world. I rather focus on hiring young and clever people.”
In addition to building dedicated units of analytics specialists, leading consulting companies have become models for how to train the broader workforce. For them, the scarce resource is generalist consultants at all levels, who can bridge the gap between data analytics and business opportunities, not data scientists and IT specialists. These companies are making heavy use of modern intuitive analytical and visualization tools and are rapidly expanding and tailoring their development programs so that consultants can conduct rigorous data analysis and tightly frame the tougher challenges for the specialists. When data analytics is widely applied, innovation and entrepreneurship start to flourish.
Systems: Investing in Data and Steering the New Technologies. Historically, discussions about data and IT systems have been tedious and technical, delivery has been slow and expensive, and productivity results disappointing. So why bother?
The answer is rather simple. With data analytics developing into a source of competitive advantage, and with speed, ease of use, and machine intelligence changing the role of IT, executives have no choice but to embrace the topic.
Data has become a form of currency that companies use to generate business value. P&G, for instance, is constantly investing in new sources of data and improving data quality. Its approach varies by market. In mature markets, P&G receives high-quality data from retailers via data warehouses. In some emerging markets, mom-and-pop shops are still a major distribution channel, and they cannot afford to make large technology investments. So P&G leverages mobile phones to provide support in ordering, store design, and product placement, while concurrently collecting business data.
Building the system infrastructure to support data analytics can be tricky on many fronts. The technology is new and rapidly evolving. And companies must make critical choices about the optimal technology stack and the best vendors. To master these challenges, they must take several critical steps:
When embarking on transformational programs, companies and executives easily fall into one of two traps. They become either too tactical or too visionary. The unrelenting rush of real-time data can trigger a flood of short-term, detail-oriented discussions and a reflex to make instant decisions, neglecting to define longer-term aspirations. Conversely, a singular focus on the long game misses immediate opportunities. An effective way to balance action and direction is to simultaneously extrapolate and retropolate.
Extrapolation focuses on creating and following a roadmap of specific and relevant short-term data-driven opportunities. Retropolation starts with long-term (and possibly extreme) scenarios and desired target states and derives requirements to reach them. An overlay of the two perspectives exposes gaps in current initiatives, forcing executives to adjust the speed and direction of their programs.
The most notable gap often involves the use of artificial intelligence and the automation of cognitive processes. The good news is that executives—when they understand the difference between what they are doing now and what the future will bring—often speed up and broaden the use of data analytics.
Finding the right balance between the short and long term also involves a review of governance. A leading machinery vendor is playing the double game by establishing a data analytics leadership council. The goal of the cross-functional group, which consists of senior business leaders, data analytics experts, and the CIO, is to explicitly balance the different time horizons, approaches, and investments.
Most important, avoid procrastination. To quote a famous Russian proverb, as adapted during the collapse of the Eastern bloc, “Who comes too late is punished by life.”
The following questions should help executives create a data-driven organization:
How can we strengthen our analytics-driven mindset?
How can we improve the organizational context so that individuals can do their best work?
How can we transform the organization’s skills through training and hiring?
Where should we expand investments in data and systems renewal?
How can we further strengthen our double game?