There is a gap between where most organizations are today and where they will need to be to succeed in the coming decade. The companies that win in the 2020s will be designed to constantly learn and adapt to changing realities, combine artificial and human intelligence in new ways, and harness the benefits of broader business ecosystems. Reaching this necessary future state will require a fundamental transformation.
This change effort will be challenging. Many businesses have deeply entrenched operating systems that are predicated on hierarchy and human decision making. They will need to redesign their internal processes and build new capabilities and business models. Furthermore, this will not be a one-time change effort: the dynamic nature of business will require organizations to build capabilities for ongoing large-scale change to keep up with evolving technology and competition.
Traditional approaches to enacting organizational change are generally not very effective. Change management is generally thought of as one-size-fits-all and based on plausible rules of thumb. But our research shows that only about one in four transformations succeeds in the short and long run, and the success rate has been trending downward. Meanwhile, the stakes are extremely high: the cumulative difference between success and failure for the largest transformations over a decade can add up to the company’s entire market value.
Leaders need to take a new approach to change—one that deploys evidence, analytics, and emerging technology. In other words, leaders must apply the emerging science of organizational change, which is based on five key components. (See Exhibit 1.)
When it comes to understanding how to enact change, business leaders have often relied on intuition and experience. In a typical transformation effort, the program design, choice of tactics and value levers, and ongoing management are often based on little more than subjective data such as customer surveys and progress reports. But the increasing availability of data, together with novel analytical approaches, has made it possible to empirically decode what really works and what doesn’t. Leaders thus need an evidence-based approach to transformation.
Our empirical analysis of hundreds of large companies that experienced major change reveals a number of lessons. In the short term, the most successful companies articulated a compelling story to reset investors’ expectations in addition to improving efficiency. (See Exhibit 2.) In the longer term, they took actions to increase revenue growth, such as spending more on R&D. They launched formal transformation programs, rather than a series of ad hoc improvements, and invested in them sufficiently. And they initiated their transformations preemptively, when things were still going well, instead of in reaction to declining performance.
To succeed in the next decade, leaders can apply such an evidence-based approach to all types of change situations—turning transformation from a reactive necessity into a competitive opportunity. For example, empirical analysis can help companies successfully acquire and transform underperforming businesses: our research shows that although such “turnaround M&A” deals are very risky, there are demonstrable ways to beat the odds. These include launching turnaround initiatives quickly and setting ambitious synergy goals, as well as giving attention to key soft factors—for example, companies with a well-defined purpose had significantly better outcomes in turnaround deals, demonstrating the importance of motivating employees on the change journey.
A similar approach can also help companies respond to changing external conditions, such as an economic slowdown: while most companies see performance decline during a downturn, a minority thrive—and historical analysis can identify what sets them apart.
What steps can leaders take to adopt an evidence-based transformation approach?
Organizational change is often seen as a single type of challenge that calls for a single type of change management in all situations. Accordingly, most change efforts follow a recipe with common ingredients: for example, centralized program offices, periodic pulse checks, measurement against predefined milestones, and a one-shot process with a clear end date.
In reality, there are many types of organizational change that present very different challenges and have very different requirements. Leaders need to de-average organizational transformation into various components and understand the right approach for each.
Change can be considered as movement across a “landscape” of possibility, where each point corresponds to a different possible state of the organization. Organizations try to seek “higher ground,” which corresponds to higher performance. Different change situations can be considered along two dimensions: Is the target destination clear (the ends)? And is there a clear path to get from here to there (the means)? (See Exhibit 3.)
De-averaging the challenge this way reveals five types of change strategies, each of which requires a fundamentally different approach to change management:
Major transformation programs, such as the ones many companies will have to undergo to reinvent themselves for the next decade, require a composite of these strategies—in which various change management approaches are applied in sequence, or in different parts of the business simultaneously. Companies therefore need to develop capabilities to tackle each type of change effectively.
A few key imperatives can help leaders leverage the required variety of change strategies:
Businesses have traditionally been managed with a “mechanical” mindset. This mentality assumes that everything that needs to be known can be known, everything that needs to happen can be planned, and all necessary change can be enacted through direct intervention.
However, companies are composed of people who interact with one another and with a complex dynamic environment. So businesses, like other biological systems, behave like nested complex adaptive systems. Lower-level systems (such as individuals) are embedded in higher-level systems (such as teams, business units, companies, industries, national economies, and societies)—and changes in any system can cause unintended and unpredictable effects in others.
Interactions between individuals or systems are becoming even more complex today, because employees, companies, and economies are more connected as a result of digitization, and because production is starting to be organized in dynamic multicompany ecosystems rather than traditional static supply chains. Therefore, mechanical approaches to change management are increasingly inadequate. Instead, leaders need to employ a “biological” approach, which is more realistic about what can be known and directly controlled.
Biological management involves several principles:
These principles point to new strategies for enacting change in business. To address a complex task (for instance, shifting a company’s culture), direct interventions (such as mandating individual behaviors) are unlikely to bring about the required change. Indirect interventions—those that change the mindset, assumptions, and context that underpin employees’ actions—often prove to be more effective because they touch the deeper, more persistent drivers of behavior.
Biological approaches also enable the orchestration of external change, such as shaping the behavior of other players in a business ecosystem. For example, early in the evolution of its Taobao e-commerce platform, Alibaba wanted to expand the range of offerings by making it easier for small or inexperienced sellers to join the market. Rather than addressing this challenge through direct actions, the company set up Taobao University—a platform on which established sellers could produce certified training materials for new sellers. By finding this indirect leverage point, Alibaba was able to capture the best wisdom from its marketplace, transmit it to potential sellers with greater scale and specificity, and improve the quality of services on Taobao, which ultimately became the world’s largest e-commerce website.
What steps can leaders take to embed biological thinking in change management?
Large-scale organizational change often results in a need for new capabilities, which may be found by reallocating workers within the enterprise or by identifying new talent externally. To execute change effectively, it is therefore necessary to develop strategies for identifying individuals’ unique skills and matching them to the right roles.
The challenge of aligning skills with positions, like many other aspects of large-scale change, has generally been based on subjective judgments of individuals’ track record of performance in different roles. But advances in science and technology are unlocking new possibilities. The study of neuroscience, and advances in testing technology, allow for the rapid, scalable identification of cognitive and emotional traits—which are more objective than self-reported survey measures or judgements based on interviews. And AI can now find and refine complex relationships between these traits and job performance across very large sets of traits and roles.
For example, our research (in collaboration with pymetrics, a startup using neuroscience and AI to help companies improve hiring processes, and Professor TejPavan Gandhok of the Indian School of Business) used digital games to assess individuals’ cognitive and emotional traits as well as their capabilities in various simulated problem-solving environments. We found that different neuro-traits reliably predicted success in different situations, suggesting that science can indeed play a helpful role in identifying talent to fill new roles. And we found that very few individuals were successful in all environments—demonstrating the need to effectively align skills with roles.
Furthermore, there were clear differences in how well certain capabilities could be learned over time. Some capabilities could be learned effectively, indicating that businesses are able to develop them internally over time—while others could only be learned very slowly, indicating a need to acquire them externally if they cannot be found within the organization. (See Exhibit 4.)
How can leaders ensure they have the proper capabilities to successfully execute change programs?
As science and technology advance, more tools for managing change in complex dynamic environments will emerge. The leaders who are willing to let go of established models and embrace this frontier will have an advantage in transforming their organizations in the coming decade.
Some emerging lessons from science and technology include:
Businesses may be able to identify early-warning signals of impending shifts—for instance, the threat that their current growth engines will run out of steam—and understand how to change accordingly. For example, Thorton Tomasetti, a leading engineering firm, has adopted new metrics to measure its vitality in order to identify signs of deterioration before they show up in measures of financial performance.
However, in the next decade, forward- looking companies will likely leverage increasingly powerful AI capabilities as an essential part of their change programs. For example, machine learning has already shown remarkable ability in predicting the dynamics of some chaotic systems, such as weather.7 Notes: 7 Natalie Wolchover, “Machine Learning’s ‘Amazing’ Ability to Predict Chaos,” Quanta Magazine, April 2018. Similar technologies could identify disruptions in the business environment or diagnose organizational health in real time.
For example, according to analytics startup KeenCorp, deep semantic patterns in Enron’s internal emails identified latent tensions in the organization, which could have served as a sign of trouble to observers even if they did not know the full extent of its fraud. 8 Notes: 8 Frank Partnoy, “What Your Boss Could Learn by Reading the Whole Company’s Emails,” The Atlantic, September 2018. This type of analysis could be applied not only to legal risks, but to strategic risks as well.
Major, ongoing change will be necessary to succeed in the next decade. But successfully enacting organizational change is highly challenging. By recognizing the complexity of change, and using lessons from science and analytics to address it, leaders can ensure their companies are best positioned to win the ’20s.
The BCG Henderson Institute is Boston Consulting Group’s strategy think tank, dedicated to exploring and developing valuable new insights from business, technology, and science by embracing the powerful technology of ideas. The Institute engages leaders in provocative discussion and experimentation to expand the boundaries of business theory and practice and to translate innovative ideas from within and beyond business. For more ideas and inspiration from the Institute, please visit our Latest Thinking page and follow us on LinkedIn and Twitter.