(Not) Lost in Foresight

Certainty vs Contingency: The Problem of Gray Swans

Futures researchers generally embrace the contingent character of the many conceiv­able tomorrows. But they part ways in their different approaches to coping with futures complexity. Forecasters make probability judgments about short-term events that can be checked against the unfolding reality in due time. This allows for transparent evalu­ations of the accuracy of these predictions. Foresighters, in contrast, create plausible scenarios of conceivable mid- to long-term developments that primarily serve the pur­pose of raising awareness of what might lie ahead.

Foresighters and forecasters agree, how­ever, that many imaginable developments can be categorized as so-called gray swans: events that are to be expected because of their frequent occurrence, such as pandemics, natural disasters, or political crises. Still, it is extremely difficult to accurately predict when and where the bird will land next and what the outcome of its landing will be. There was, for instance, no shortage of warnings over the past decade about a possible global pandemic. COVID-19 is not a black swan – that is, a completely un­expected event. Alas, the warnings proved to be too unspecific for policymakers.

Political authorities crave certainty, there­fore they frequently complain about major surprise disruptions because they expect precise and actionable forewarning. In a way, this is quite reassuring as their anger reflects that hardly any policymaker will make decisions lightly in times of crisis. Most governments prefer to avoid such situa­tions because under conditions of un­cer­tain­ty about future developments poor choices are bound to be made. COVID-19 is currently the most prominent example of this.

Following reverse logic, fewer surprises would mean more certainty in decision making. Accordingly, political authorities frequently resort to reasoning based on past experience. This convenient way of framing the future is reflected in everyday policymaking practices. Administrations routine­ly engage in incremental planning in terms of policy development, programming, and budget requirements for the next year, or, as is the case for the EU, for multiple years. Of course, many decision makers are mind­ful of the problems that are inherent in inter­preting the past as prologue to the future. In military affairs, the proverbial phrase “generals are always prepared to fight the last war” – attributed to Winston Churchill – exemplifies both this practice and its critique.

What If There Are Too Many What Ifs?

Futures complexity thus collides with every­day policymaking. A lot of the myriad policy proposals, recommendations, and warnings that compete for the attention of decision makers originate with futures researchers. Numerous international organizations, research institutes (including SWP), con­sul­tancies and think tanks, not to mention governmental foresight units, expert coun­cils and intelligence agencies, are trying to make policymakers aware of future risks and threats. It is therefore not surprising that decision makers, when confronted with an ever-growing number of risk assess­ments, scenarios, early warnings, “What ifs…” and “For your eyes only”-reports, start saying, “I don’t have the time or resources to deal with all these issues simultaneously, so I have to set priorities. Can you help me with that? But wait, you have been wrong before, haven’t you?”

They have a point. Research has shown that experts are not very good at accurately predicting political events in the future. Of course, foresighters and forecasters could insist that they had issued warnings about the major disruptions of the past two dec­ades, including COVID-19, and that policy­makers failed to take appropriate action; to which policymakers would prob­ably reply that these warnings were too vague and un­specific in that they did not precisely pre­dict what would happen where and when – a reference to the problem of gray swans. But such back and forth will neither im­prove preparations for future contingencies nor facilitate the policy uptake of analysts’ recommendations. Both the futures and the policymaking communities should aim to move beyond the blame game of “Why didn’t you warn us?” vs “We told you so!”

In order to avoid disappointment, and to make the expected boost in demand sustainable, it is necessary to clearly com­municate to policymakers the different goals for which various scientific methods of futures analysis aim. Foresight helps political authorities to structure thinking about the more distant future by raising awareness of emerging political, economic or social developments and the range of their possible impacts on international affairs. Sketching out possible mid- to long-term futures requires methodological approaches, such as horizon scanning or trend analysis, that differ from those aim­ing to predict the outcome of a concrete political event, for instance the result of an election that is scheduled for a specified date. Forecasting an election result can help policymakers to make more informed deci­sions even under conditions of uncertainty. Here, quality criteria such as assessment accuracy and process transparency should be emphasized.

Policymakers in turn should acknowl­edge that setting priorities for preparations against hypothetical events and develop­ments is not always a rational, evidence-informed process in which subject matter expertise holds sway. Deciding which of the many possible risks in the future will take priority is an eminently political process. Precautionary policies require governments to make an investment, the costs of which depend on the nature of the threat that is prioritized. Investments in military capa­bilities, for instance, are guided by com­peting assessments of the most likely and pressing security challenges; e.g. should resources be spent on protection against territorial or cyber aggression? In this con­text, it is then hard to avoid the resurfacing of one of the most basic questions of politi­cal science: who gets what, when, how?

How to Increase Policy Relevance

The inherently political nature of determin­ing priorities does not make futures analysis irrelevant. The question is, rather, how to make it politically significant while at the same time minimizing attempts at its malign politicization. Futures researchers will need to engage more frequently with politics to better understand daily routines and work requirements, including recog­nition of the enormous amount of infor­mation decision makers must process. Simply throwing more reports and studies about hypothetical events at policymakers does not automatically produce expected results, namely precautionary political intervention. To the contrary, knowledge overload can paralyze decision making processes and also provide a smokescreen that allows policymakers to deflect incon­venient or unpopular measures that would mitigate future risks and threats.

Therefore, the futures community must get better at processing information about futures’ contingencies. Structuring and curating futures should include pointing out which of the many conceivable events and developments might deserve special attention in political deliberations. Policy recommendations could be framed accord­ing to the needs of different audiences in governments and parliaments: short-term forecasts of geopolitical events for the ex­ecu­tive branch, and distant future implications of various scenarios for legislators. Of course, the process of inferring these policy recommendations must be based on trans­parent criteria.

Prediction accuracy should also be an im­portant driver for policy relevance. Studies show that the accuracy of predictions can be systematically increased through prac­tice and training. Analyzing a multi-year geopolitical forecasting tournament with several thousand participants, researchers found that some participants get it right significantly more often than others. About two percent put forth consistently accurate forecasts. When assigned to teams, the com­bined accuracy of their forecasts improved even more. Different cognitive styles, diver­sity and multi-perspectivity were typical features of the best teams. They achieved about 30 percent higher prediction accu­racy than competing teams that had access to classified information.

Geopolitical Forecasts for the EU

The EU could harvest this knowledge. Policy­makers should insist that transparently gen­erated predictions inform decisions as to which of the many conceivable events in the future should have priority. In the EU, an interinstitutional platform for a geo­political forecast tournament could be orga­nized for this purpose [I am indebted to Leopold Schmertzing for this idea]. Such a platform would help to identify the best forecasters among EU staff across all par­ticipating insti­tutions and without regard to professional status or seniority. Given its international composition and supra­national identity, the EU could easily assemble teams characterized by various cognitive styles, diversity and multi-per­spectivity, and then train them to improve their performance. As Vice-President of the European Commission for Interinstitutional Relations and Foresight, Maroš Šefčovič would be perfectly positioned to lead such an initiative.

The executive branch of government would benefit from such a platform, as the following thought experiment demon­strates. Predictions generated by the best forecasting teams on the probability of open military hostilities between Ukraine and Russia in the next six months, for example, would add additional assessments to the insights of intelligence agencies, diplomatic services and experts. Inserting information from a source that has a veri­fiable track record of accuracy into policy debates on how to react to Russia’s aggres­sive posture could help to make these debates more objective and evidence-informed.

In April 2020, the UK government be­came one of the first to launch a geopolitical forecast tournament. Here, participating civil servants answer a broad range of ques­tions based on publicly available informa­tion, for example, about the probability of Chinese aggression against Taiwan and the rate of decline of COVID-19 infections worldwide. Because prediction accuracy can be judged against real-world outcomes, the best forecasters can easily be identified and assembled in teams tasked with specific missions.

Better Policymaking in the Future

Of course, the benchmark for this approach is not perfect anticipation of the future but rather better policymaking in the future. Predictions work best for hypothetical politi­cal events within a timeframe of 12 months ahead. But the potential 30 percent higher forecast accuracy referred to above implies a considerable reduction of the num­ber of short-term crises for which prepa­rations should be taken immediately, thereby creating a decision making advan­tage. Costly and awkward precautionary measures – such as military deployments, purchasing intensive care units, or wearing masks – could be more convincingly jus­ti­fied to crucial audiences, including political competitors, courts of auditors, the media and the general public.

Given the limitations of short-term fore­casting, foresight will remain instrumental as a method to structure thinking and pro­duce plausible scenarios about developments the EU may face in the more distant future. Past reports produced by the Euro­pean Strategy and Policy Analysis System (ESPAS) – a collaboration between some of the most important EU institutions – on, for example, “Challenges and Choices for Europe” until 2030, illustrate this approach.

Scenarios tend to be more normatively charged than predictions of concrete politi­cal events due to the longer time horizon that they cover. In a democratic regime, debates over different interpretations of the desirability of emerging futures typically fall within the domain of the legislative branch of government. Hence, the obvious place for normative deliberations about these issues at the level of EU institutions is the European Parliament – the staunchest supporter of the idea of the Conference on the Future of Europe.

It is therefore commendable that the Euro­pean Parliamentary Research Service (EPRS) already has a strong focus on stra­tegic and scientific foresight. Two features of the EPRS’s work stand out in this regard: its quality products – as acknowledged by the futures community – and its close interaction with experts, policymakers and legislators when designing inquiries. How­ever, it would be highly desirable to further enlarge the range of futures expertise that is available to parliamentarians. Given a more uncertain world characterized by in­creasing global connectivity, cross-sectoral interdependence, and declining political cohesion within the international community, more rather than less informed debates and discussions on the geopolitical and nor­mative aspects of the future are needed.

Selected References

Brozus, Lars, ed., While We Were Planning. Unexpected Developments in International Politics. Foresight Contributions 2018, SWP Research Paper 5/2018 (Berlin: Stiftung Wissenschaft und Politik, September 2018)

Horowitz Michael C., and Tetlock, Philip E., “Trending Upward. How the Intelligence Community Can better See into the Future”, Foreign Policy (online), 7 September 2012

Horowitz, Michael C., Ciocca, Julia, Kahn, Lauren, and Ruhl, Christian, Keeping Score: A New Approach to Geopolitical Forecasting, Perry World House White Paper, February 2021

“How Spooks Are Turning to Superforecast­ing in the Cosmic Bazaar”, The Economist (online), 17 April 2021

Mazarr, Michael J., “The Swans to Worry About Are Gray”, in idem, Rethinking Risk in National Security (New York: Palgrave Macmillan, 2016)

Polchar, Joshua, Unboxing the Future. Finding the Futures Hidden in Plain Sight, Brief 19 (Paris: European Union Institute for Security Studies [EUISS], August 2020)

Scoblic, J. Peter, and Tetlock, Philip E., “A Better Crystal Ball. The Right Way to Think About the Future”, Foreign Affairs (online), November/December 2020

Tetlock, Philip E., and Mellers, Barbara A., “Intelligent Management of Intelligence Agencies: Beyond Accountability Ping-pong”, The American Psychologist 66, no. 6 (2011): 542–54.

Tetlock, Philip E., and Gardner, Dan, Super­forecasting. The Art and Science of Prediction (New York, NY: Crown Publishing Group, September 2016)

Tetlock, Philip E., Mellers, Barbara A., and Scoblic, J. Peter, “Bringing Probability Judgments into Policy Debates via Fore­casting Tournaments”, Science 355, no. 6324 (3 February 2017): 481–83