Prediction in Uncertain Times

We live in interesting times. Britain voted to leave the European Union. We then saw a reality TV star and real estate developer win the American presidency. These events were clearly momentous in their own right. But, interestingly, they both happened against the predictions of hundreds of pollsters and quantitative political analysts, at the very time when we were trumpeting the progress of and prospects for big data, analytics, and artificial intelligence.

It’s certainly been a chastening time for data-driven forecasters. After all, elections are some of the most studied events in the world, with stable rules, a simple binary outcome, and access to vast amounts of current and historical data. One journalist even declared after the election, “It was the day the data died.”

The juxtaposition of these events with recent progress in analytics and artificial intelligence and increasingly bold claims about how they will change our lives is striking. It seems that data and smart algorithms could generate tremendous value for businesses, whether it’s improving promotional effectiveness and operational efficiency or enabling self-tuning recommendation engines. Yet Brexit and Trump serve to remind us that data and analytics can just as easily lead us astray.

What the political upsets of the past year underscored is that advanced analytics cannot be a substitute for human insight and ingenuity. Those events reinforced some of the ideas that we began to explore in a recent HBR piece about the challenges that businesses must overcome in order to take advantage of smarter algorithms and the need for humans and machines to integrate their unique strengths. We see four main lessons for business leaders.

• In unfamiliar territory, complement technology with human insight. The combination of data and sophisticated algorithms is extremely powerful given a familiar situation with enough prior data. Nate Silver at FiveThirtyEight was able to correctly predict 50 out of 50 states in the 2012 US presidential election because it was “business as usual” with known characters. People, however, still have a decisive advantage in navigating unfamiliar territories, and the campaign we experienced this time was anything but business as usual. It was therefore pundits like Michael Moore and Scott Adams who saw the outcome more clearly—people who paid attention not to business as usual but to the unquantifiable, human aspects of this unique election.
• Go upstream to understand the base assumptions of the technology. Even the most powerful algorithms require a defined problem and a definite scope. They don’t have the ability to think outside that particular scope. For now, only people can go “meta” and reframe a question. “Is this even the right question?” and “Are we using the right model?” are questions that only people can answer. As it turns out, the pollsters were not using the right model. Their prediction algorithms themselves were adequate, but the upstream assumptions embedded in the model, such as voter turnout and response rates, turned out to be highly fallible. We should have known from the implosion of Long-Term Capital Management (LTCM) in the late 1990s. Sometimes, the critical insight lies outside the model, and only people can find it.
• Question technological black boxes. Human beings have important roles to play in extracting value from data. To play this role, they need to understand, examine, and validate algorithmic output and provide feedback to the model driving it. Every political analyst said that this election was unprecedented; nevertheless, pollsters continued to be blasé about the limitations of their approaches, and journalists did not or could not look “under the hood” of technological black boxes. The same risk applies in business: reductive visuals and sophisticated algorithms can become black boxes that hide unstated assumptions and the messiness of the underlying reality.
• Do not rely on prediction without structural analysis. Analysts distinguish causal and predictive models. The latter aim to predict events accurately without any regard for the causal structure between input and output variables, while the former are used to better understand isolated relationships between inputs and outputs. In other words, some models predict the future without understanding it, and others understand parts of the future without being able to comprehensively predict it. In reality, you need both. Causal analysis may—in comparison to predictive models—look clumsier, but it allows us to uncover previously hidden causal relationships that drive key elements of shallower prediction models.

Put succinctly: it would be a basic mistake to think that technology takes humans out of the equation; rather, it makes human ingenuity more necessary than ever. Man and machines are complements. This means that the business value of algorithms and data can be “unlocked” only by better human-machine integration.

Hannah Arendt thought that the fundamental error of political philosophy since Plato was that it aspired to formulate an “ideal system” with theoretical principles so rationally compelling that they eliminate the messy, unpredictable nature of actual human interactions. We fear that our current approach to technology has fallen into the same trap. Reality—especially as it pertains to human action—is complex and messy. Something will always fall out of the analytical model; something important will always linger outside its scope. Therefore, instead of dogmatic use of data, we need the plurality of perspectives that combining human and machine capabilities makes possible.

In other words, we need to remind ourselves that critical thought still lies upstream of data science. Business leaders are right to integrate data and analytics into their business processes. But human ingenuity is still the key to unlocking the power of technology, even in the coming “Age of Machines.” We believe that this idea should be uplifting for business leaders in reaffirming their value even as technology progresses, providing they sharpen their ability to use it critically.

The BCG Henderson Institute is Boston Consulting Group’s strategy think tank, dedicated to exploring and developing valuable new insights from business, technology, and science by embracing the powerful technology of ideas. The Institute engages leaders in provocative discussion and experimentation to expand the boundaries of business theory and practice and to translate innovative ideas from within and beyond business. For more ideas and inspiration from the Institute, please visit Featured Insights.