I’ve been struck by how weather forecasters face some of the same types of difficulties  that beset macroeconomics. Both have to rely on “natural experiments” (ie events) as a way to test whether they have an understanding of how the system works. In each case, the systems they work with are extremely complex and chaotic. Weather forecasters make a tremendous effort to continuously assess how well their forecasts perform. They recognize that such forecast verification is crucial for progress to be made. The verification process itself is not trivial. Often the most important predictions are knowing when there is a heightened risk of an extreme event such as a severe storm moving from the ocean over the land. It can be much more valuable to have better (though still probabilistic) awareness of possible extreme events than to have precision for normal weather. Weather forecasters fully recognize that; they use a variety of different types of modeling systems and wide variety of verification approaches. An analogy between weather forecasting and macroeconomic forecasting is not a novel idea and the BIS has made the analogy with severe storm prediction methods to call for more probabilistic forecasting to help spot risks of financial crisis.

There is great controversy over whether or not the 2008 global economic crisis was predicted by heterodox economists employing stock flow consistent models. The mainstream macroeconomists claim that such predictions of imminent crisis were continuously being issued by various mavericks and so (just as a stopped clock is right twice a day) harebrained doomsayers eventually got lucky. There is also controversy over whether macroeconomics is entangled with politics. Some economists profess that they are politically impartial scientific observers and technocrats whilst others say that they (perhaps unwittingly) are party to an inextricably political process. Margret Thatcher famously acknowledged that for her,

Economics are the method; the object is to change the heart and soul.

To my mind both these controversies need to be dealt with head on by a comprehensive and transparent program of continuous verification of macroeconomic approaches. It is not enough to have occasional ad hoc predictions leading to an unresolvable mess of anecdotes as to what approach does or doesn’t provide insight. It is also vital for the public to see how the economic policies being implemented are predicted to influence future outcomes. If an economic policy is being implemented and the widespread prediction is that it is going to greatly increase the risk of economic crisis or depression, then the public deserve to know. Comprehensive public forecasts would throw up an endless stream of material for journalists to discuss and for politicians and their economic advisers to be held to account over.

A verification system would work best if every macroeconomics program hoping for respectability continuously provided a standardized and wide-ranging set of predictions (including probabilistic assessments of tail risks) stretching for various time spans into the future (from weeks to many years) for a wide set of countries. It is crucial that every participant provided forecasts for the wide set of countries or otherwise the data would be too thin to spot whether any branch of macroeconomics had managed to develop predictive power. The predictions would be most valuable if they included metrics that had direct real importance. As well as financial measures such as  household income (and discretionary income level) level and wealth at each decile, inflation, GDP,government debt level and current account deficit; I think it would be useful to include unemployment, labor participation, energy use and child mortality rates.

I think much of the problem with macroeconomics stems from a misplaced reverence for microfounded mathematical models. It is great whenever a phenomenon can be successfully described mathematically but we need to face facts and accept when that is still beyond us. Many fields of science make progress without being at a stage where mathematical modelling can provide much insight. Some people make valiant attempts to mathematically model the development of embryos based on diffusion of proteins and such like. As yet however that approach hasn’t been what has driven the bulk of the advances that have been made in that field. The useful models have largely been crude flow diagrams that provide little more than an expected direction for an outcome. Nevertheless it is much much better to actually know whether something is likely to increase or decrease than to have a fancy model that gives you the wrong answer. What I’m calling for is for macroeconomists to simply do their best to predict what the economy will do. The methodology could be as crude as simply having a check list of “good” and “bad” economic policies. Let’s just see what predictive method works best. Bringing this back around to the comparison with weather forecasting, some of the current controversies in macroeconomics are more akin to arguments as to how gross geography influences climate. Some economists predicted that the fiscal regime in Latvia would cause a depression when others said it was a recipe for prosperity. That dichotomy of opinion is as extreme as if we still hadn’t established whether the weather was cooler near the poles or near the equator.

Economic policies conducted in various parts of the world over the years do seem (with hindsight) to have made profound transformations to how economies have developed. Consider how Singapore has transformed from the poverty it found itself in at independence to having widespread affluence. The financialization and crisis in Iceland was dramatic but the people there didn’t subsequently suffer the real poverty that for instance occurred in Russia in the 1990s. Perhaps the most shocking economic phenomenon has been the persistent extreme poverty of the world’s poorest two billion people. If the globalized capitalist economy is seen as an integrated system that we are all responsible for, then that is a shameful failure causing millions of avoidable deaths. If, in the developed world, we enact some facilitation for capital flight or agricultural tariff adjustment that led to forecasts for increased child mortality rates across Africa, then the public might recoil with a “not in my name” type protest. Comprehensive forecasts could help to expose such issues.

Macroeconomists often make pronouncements about the worthiness or otherwise of various policies such as monetary policies, tax structures, labor regulations or deficit levels. I simply want them to pin themselves down to making formal predictions as to the outcomes from the myriad of natural experiments that our world economy continuously provides us with as events unfold. Some macroeconomists might counter with the view that their subject is about deep theoretical insight rather than petty forecasting. My view is that such insight is as worthwhile as it is useful for informing a predictive understanding.

Related stuff on the web:

WWRP/WGNE Joint Working Group on Forecast Verification Research

Continually improving our forecasts -Met Office

Moving towards probability forecasting -Bank of International Settlements

Economics Education and Unlearning -Post-Crash Economics Society

Who Are These Economists, Anyway? -James K Galbraith 

What does it mean to have “predicted the crisis”? -Noah Smith

Simon Wren-Lewis Defends the Status Quo -Nick Edmonds

 Are macroeconomics methods politically biased?- Noah Smith

Can economists forecast recessions? Some evidence from the Great Recession- Hites Ahir and Prakash Loungani (link added 30May2014)

Macroeconomic model comparisons and forecast competitions -Wieland and Wolters (link added 20Oct2014)

On Macroeconomic Forecasting- Simon Wren-Lewis (link added 01Jan2015)

Forecast errors -Bank of England (link added 01Jan2015 ht Simon Wren-Lewis)