Managing Director & Senior Partner, Chairman of the BCG Henderson Institute
San Francisco - Bay Area
“Creating a great culture, finding the right people, managing them to do great things, and solving problems creatively and systematically are challenges faced by all organizations. What differentiates [organizations] is how they approach these challenges.”
Ray Dalio, Founder, President, and CIO, Bridgewater Associates
Business, at its heart, is about solving problems. Problem solving is performed both explicitly by analysts and computers and implicitly by your organization as a whole. And the way your organization is designed—the structure, processes, communication policies, incentives, training, and talent management you have in place—shapes the way your problems are approached and solved. Many organizations, however, lack explicit strategies for problem solving. This has come at little cost to these organizations historically, given that many of the problems they faced could be solved using straightforward, well-known methods. But today’s business environment, characterized by sharply rising complexity and hence increasingly complicated problems, is putting a rising premium on more sophisticated approaches to problem solving.
The rise in complexity—defined as the number of calculation steps required to reach a solution—is being driven by the rapid growth of three variables: data, interconnectedness, and the speed of change.
First, the volume and variety of data available are expanding exponentially. From 2002 through 2012, the amount of digital data generated annually increased from 5 to 2,800 exabytes, or roughly 400 times the number of grains of sand in the world.
Second, companies, individuals, and machines are increasingly interconnected. Also from 2002 through 2012, the number of Internet users rose from 500 million to 2.4 billion. But the degree of interconnectedness among those users is increasing even more sharply. While the number of Facebook users increased from 13 million to 700 million from 2007 through 2011, for example, the number of connections between those users increased from 600 million to 70 billion.
And third, the rate of change in the business environment is accelerating. Whereas it took 64 years for the telephone to reach 40 percent penetration in the U.S., it took the mobile phone only 18 years to reach that level and smart phones only 10 years to do so. For Facebook, it took only 4 years.
Many problems, by their sheer nature, are rising in complexity even more quickly than the growth rate of these variables would suggest. Take the challenge of parsing Facebook’s user population, for example. While the number of connections among Facebook users increased by a factor of 11 from 2007 through 2011, the problem of identifying the largest number of users who are not “friends” now takes 1.4 x 10^51,599,743 times the number of computations it took a decade ago. This sharp rise in complexity means that, for the most complex problems—those involving many variables or a high degree of interconnectedness—finding an exact solution becomes infeasible.
Fortunately, there are a growing number of ways to reach approximate solutions to such problems, including metaheuristics, new approximation algorithms, and social problem-solving methods such as crowd sourcing. Some companies are already taking advantage of these new tools and techniques, and, in the process, they are raising the competitive bar for problem solving in their industries.
Understanding how to frame and approach the key problems a company faces—and having the ability to solve these problems more accurately, quickly, or economically than the competition solves its problems—will be an increasingly critical competitive differentiator among businesses going forward. This capability demands change across the organization; it is far more than simply applying “big data” to existing approaches and behaviors. We draw a distinction between “classical enterprises” and “ingenious enterprises.” Classical enterprises approach the management and operation of their core business without thinking explicitly about how to solve problems. Ingenious enterprises, in contrast, see problem solving as a critical capability and have explicit strategies for managing it.
The effectiveness of a company’s problem solving, as measured along the dimensions of cost, speed, and accuracy, is influenced by five elements: strategy (that is, the core of the company’s problem-solving approach, which drives decisions about the other elements), framing, data selection, choice and implementation of a solution method, and selection of problem solvers. (See Exhibit 1.) Classical enterprises typically lack a strategy for problem solving. They try to frame problems as simply as possible, to employ limited amounts of data and a limited repertoire of approaches, to aim for exact solutions, and to keep the effort entirely in-house. Ingenious enterprises approach problems very differently. They develop explicit problem-solving strategies that take into account the types of problems encountered. They frame each problem in a way that allows it to be solved optimally. They employ large volumes of data, if necessary; vary the problem-solving methods based on the complexity of the problems at hand, using artful approximations as needed; and turn to social networks and other open-problem-solving approaches when doing so is beneficial.
Google in the search engine arena circa 1997 is an example of how an ingenious enterprise tackles a highly complex problem—and the results that such an approach can yield. Before the advent of Google, users faced the choice between human-curated directories, such as LookSmart, that were slow to update their results and expensive to maintain (emphasizing accuracy over speed and low cost) and automated search engines, such as Lycos, that were susceptible to manipulation and often returned low-quality results (emphasizing speed and low cost over accuracy). Google managed to break this compromise by taking a new approach to the problem. (See Exhibit 2.) Rather than relying on a massive database of Web pages or an army of editors, Google changed both the method and the data. It created PageRank—an algorithm that approximates the significance of a page on the basis of the number of links it has received from other pages—as a heuristic to identify and rank relevant results. By changing the problem-solving elements, Google was able to deliver the holy grail of acceptable speed, low cost, and accuracy. And, in the process, Google transformed itself into a company with a market capitalization of more than $250 billion as of April 2013.
The high-technology sector is an obvious hotbed for ingenious enterprises. But such problem-solving ingenuity can also be found in other industries. Take property insurers’ response to Hurricane Andrew. Prior to Andrew, the industry’s practice for estimating potential losses from large catastrophes was based largely on extrapolation from historical data. The problem with this, in hindsight, was that there were limited relevant data from which to draw. Hence, the industry was caught flat-footed when, in 1992, Andrew struck, with much greater force than anticipated: rather than losses ranging from a few hundred million dollars to a maximum of $8 billion, as the industry had expected, Andrew’s damages exceeded $15 billion (in 1992 dollars). The industry had solved the problem—but with very low accuracy. As a result, 11 insurers in Florida went bankrupt.
In response, the industry changed both the way it framed the problem as well as the method and data it employed. Rather than focusing only on previous catastrophes to estimate the potential costs of damage from future storms, insurers increased the complexity of their calculations by factoring in potential changes in other variables (for example, building codes and the amount of housing stock). With the help of specialized risk-management firms, the industry also took a more sophisticated approach to predicting the occurrence of future catastrophic storms by incorporating advances in the modeling of climate change. The value of these changes became evident when Hurricane Katrina hit in 2005: despite claims that were almost double those related to Hurricane Andrew, only one insurer went bankrupt.
Bridgewater Associates, one of the world’s largest hedge funds, is illustrative of an ingenious enterprise that has embedded problem solving into its organization. Ray Dalio (Bridgewater’s founder, president, and CIO), who has compiled his insights into a book, Principles, espouses such management principles as the following: Consider the organization a machine built for particular goals. Evaluate the organization’s performance against those goals and make adjustments as needed. Diagnose your problems to understand what the root causes are. Think not only of the first-order consequences but also of the second- and third-order ones of any solution. Create a culture in which criticism is encouraged and making mistakes is acceptable as long as the mistakes are analyzed and learned from. Build your organization in a way that enables effective problem-solving and train your entire workforce in the problem-solving methods that you want them to use.
Bridgewater has done more than simply create a lofty corporate vision—it has incorporated specific problem-solving practices into its organizational behavior. The effort has clearly paid off for the company, which currently has more than $150 billion under management and whose flagship fund has generated average annual returns of 21 percent (gross of fees) over a span of more than two decades.
Ingenious enterprises stand to gain an increasingly powerful competitive advantage over their competitors as complexity in the business landscape continues to rise. Managers who aspire for their company to join the ranks of these organizations should do the following:
Identify your problems. The first and most important step is to identify and define the most critical challenges your company faces. Focus on high-value problems whose complexity rises quickly with an increase in input variables. Look beyond your industry to understand where similar problems occur. Do not oversimplify your problems or obscure them in industry-specific jargon but take a broad view and identify the problems’ underlying structure.
Benchmark your algorithms. Gauge where your company stands versus its key competitors with regard to the cost, speed, and accuracy of its problem solving. What approaches and methods are competitors using? Identify gaps in your performance. Find out how companies in other industries have solved problems similar to yours. Are some organizations changing the game the same way Google changed Web search?
Develop a repertoire. Create a toolbox of techniques for problem solving. Embrace both explicit algorithms and implicit organizational techniques. Develop a framework for when each technique should be used and for how the techniques should be deployed. Practice what you preach. Make sure your organization embraces and embodies the problem-solving techniques that you are advocating.
Make problem solving a key capability. Reengineer your systems to reward creativity in problem solving. Train your organization and managers in problem framing and solving. Create the incentives and structure that enable them to break from established models of thinking and try new approaches. Make problem solving a key criterion in talent management.
Evaluate and experiment. Measure the results of your problem solving constantly. Try out new methods, uses of data, and framing techniques. Compare the results with those yielded by your old techniques to ensure progress. Vary the composition and structure of your problem-solving teams. Identify opportunities where external partners and open-problem-solving techniques can bring new ideas and solutions to your problems.
Today, most companies define themselves on the basis of their competitive standing within their industry. Increasingly, however, as complexity in the business environment continues to rise, the most successful companies will be those that define their prospective competitiveness by a different metric: how, and how well, they approach and solve problems.