Sigrún Davíðsdóttir's Icelog

Reinhart & Rogoff – a digest of the debate (updated)

with 4 comments

As so many others I’m reading with interest the debate on the 2010 article by Carmen Reinhart and Kenneth Rogoff, Growth in the Time of Debt. Apart from the issues at stake, I find it in general interesting to observe how certain theories catch on and then there are invariably people who attach their careers to these theories, thus turning into crusaders for certain ideas instead of crusading for the quest for good solutions. The debate on Reinhart and Rogoff chrystallises all these tendencies.

The article by Thomas Herndon, Michael Ash and Robert Pollin, PERI Institute University of Massachusetts Amherst, appeared April 15 (with a correction posted on April 17). Their abstract reads (emphasis mine):

Herndon, Ash and Pollin replicate Reinhart and Rogoff_ and find that coding errors, selective exclusion of available data, and unconventional weighting of summary statistics lead to serious errors that inaccurately represent the relationship between public debt and GDP growth among 20 advanced economies in the post-war period. They find that when properly calculated, the average real GDP growth rate for countries carrying a public-debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0:1 percent as published in Reinhart and Rogo_ff. That is, contrary to RR, average GDP growth at public debt/GDP ratios over 90 percent is not dramatically different than when debt/GDP ratios are lower.

The authors also show how the relationship between public debt and GDP growth varies significantly by time period and country. Overall, the evidence we review contradicts Reinhart and Rogoff_’s claim to have identified an important stylized fact, that public debt loads greater than 90 percent of GDP consistently reduce GDP growth.

Thus, their conclusions are both striking and clear, which is why this has come as a bombshell into the debate on Eurozone policies that so far and to a high degree reflect the R&R doctrine. To be fair, there has earlier been criticism from near and far on this debt-to-GDP correlation – but this seems to be the first study that picks apart the basis for the R&R doctrine.

Here is a digest of some relevant articles so far:

Michael Konczal, writing on the Roosevelt Institute’s blog Next New Deal, summarises the Hendon Ash Pollin debate. As to the status of the R&R paper he writes:

Their “main result is that…median growth rates for countries with public debt over 90 percent of GDP are roughly one percent lower than otherwise; average (mean) growth rates are several percent lower.” Countries with debt-to-GDP ratios above 90 percent have a slightly negative average growth rate, in fact.

This has been one of the most cited stats in the public debate during the Great Recession. Paul Ryan’s Path to Prosperity budget states their study “found conclusive empirical evidence that [debt] exceeding 90 percent of the economy has a significant negative effect on economic growth.” The Washington Posteditorial board takes it as an economic consensus view, stating that “debt-to-GDP could keep rising — and stick dangerously near the 90 percent mark that economists regard as a threat to sustainable economic growth.” 

So far, R&R’s data has not been scrutinised. Hendon, Ash and Pollin got the underlying data from R&R and detect selective exclusions, unconventional weighting and coding error. As to the coding error, Konczal checked it himself, with data from one of the three authors:

This error is needed to get the results they published, and it would go a long way to explaining why it has been impossible for others to replicate these results. If this error turns out to be an actual mistake Reinhart-Rogoff made, well, all I can hope is that future historians note that one of the core empirical points providing the intellectual foundation for the global move to austerity in the early 2010s was based on someone accidentally not updating a row formula in Excel.

Konczal concludes in an update on his blog:

UPDATE: People are responding to the Excel error, and that is important to document. But from a data point of view, the exclusion of the Post-World War II data is particularly troublesome, as that is driving the negative results. This needs to be explained, as does the weighting, which compresses the long periods of average growth and high debt.

Next new Deal has a further digest to the debate but got the brilliant idea of getting an econometrician to read the article by Massachusetts trio. A colleague of the trio Arindrajit Dube took the task on with a zest, pondering on Growth in a Time Before Debt. I.a. he throws up a graph of Past and Future Growth Rates and Current Debt-to-GDP Ratio, concluding:

As is evident, current period debt-to-GDP is a pretty poor predictor of future GDP growth at debt-to-GDP ratios of 30 or greater—the range where one might expect to find a tipping point dynamic.  But it does a great job predicting past growth.

This pattern is a telltale sign of reverse causality.  Why would this happen? Why would a fall in growth increase the debt-to-GDP ratio? One reason is just algebraic. The ratio has a numerator (debt) and denominator (GDP): any fall in GDP will mechanically boost the ratio.  Even if GDP growth doesn’t become negative, continuous growth in debt coupled with a GDP growth slowdown will also lead to a rise in the debt-to-GDP ratio. 

Besides, there is also a less mechanical story. A recession leads to increased spending through automatic stabilizers such as unemployment insurance. And governments usually finance these using greater borrowing, as undergraduate macro-economics textbooks tell us governments should do. This is what happened in the U.S. during the past recession. For all of these reasons, we should expect reverse causality to be a problem here, and these bivariate plots are consistent with such a story.

His conclusion is:

All in all, these simple exercises suggest that the raw correlation between debt-to-GDP ratio and GDP growth probably reflects a fair amount of reverse casualty. We can’t simply use correlations like those used by RR (or ones presented here) to identify causal estimates.

R&R have now responded to the critique, on FT Data Blog ­– with FT Chris Cook’s own digest, i.a. comparing date from R&R and the Massachusetts trio. R&R admit to the coding error – but deny it has the dramatic results as the Massachusetts trio claim:

On the first point, we reiterate that Herndon, Ash and Pollin accurately point out the coding error that omits several countries from the averages in figure 2. Full stop. HAP are on point. The authors show our accidental omission has a fairly marginal effect on the 0-to-90-per-cent buckets in figure 2. However, it leads to a notable change in the average growth rate for the over-90-per-cent debt group. The median growth rate we report is the right order of magnitude.

Our interpretation of the errant data point in figure 2 was fortunately tempered somewhat by the parallel weight given to the median GDP growth rate for the various levels of debt in our discussion, an issue HAP selectively ignore.

Further, R&R deny selective exclusions and defend the weighting – and they stand by their earlier conclusions:

So do where does this leave matters on debt and growth? Do Herndon et al. get dramatically different results on the relatively short post war sample they focus on? Not really. They, too, find lower growth associated with periods when debt is over 90 per cent. Put differently, growth at high debt levels is a little more than half of the growth rate at the lowest levels of debt.

On The Daily Beast Megan McArdle digests the material so far, concluding that the R&R doctrine still has merits:

This doesn’t overthrow the broader scope of their work, such as the finding that you tend to get slow growth after financial crises.  They certainly seem to have called that one right.  

There is now a deluge of comments and of course Paul Krugman has a thing to say here, pointing out that actually the pundits should indeed be blamed. Not only R&R – after all, they have not been the only economists peddling austerity. Krugman cites a recent article in the Washington Post:

If [debt projections are] even slightly off, debt-to-GDP could keep rising — and stick dangerously near the 90 percent mark that economists regard as a threat to sustainable economic growth.

Krugman concludes:

Not “some economists”, let alone “some economists who have been sharply criticized by other economists with equally good credentials”, but “economists”.

This is deciding what you want to believe, finding someone who tells you what you want to hear, and pretending that there are no other voices. It’s deeply irresponsible — and you can’t blame Reinhart-Rogoff for that mistake.

*Professor Simon Wren-Lewis, Oxford, has just posted an aside (read the whole piece, it’s short, with a telling graph) to the R&R debate, An understandable mistake, further underlining the importance of government spending in the time of recession. But first to the R&R debate:

No, I’m not talking about coding in excel – as someone who has in the past done plenty of empirical work, my overriding reaction is empathy with the researchers concerned. There – not so much by the grace of god but because no one bothered to check what we did – go us. What I’m talking about is the weak global recovery and a primary reason for it. But there is a link, which I will come to at the end.

Wren-Lewis wants to draw the attention to a new blog on Vox EU, Why is this global recovery different?, from the latest IMF WEO by Ayhan Kose, Prkash Loungani and Marco E Terrones. According to Wren Lewis “…it tells a story in pictures (particularly comprehensive and clear pictures) that I and others – most notably Paul Krugman – have been telling for some time.”

…government spending in the advanced economies has grown at a much slower rate in this recovery than in previous recoveries, and of course this recovery has been significantly slower as a result. In contrast, government spending in the emerging economies has been as rapid during the recovery as before the recession, and they have recovered rapidly from recession. Go into detail within the advanced economies group, and the pattern is clear: the greater the contraction in government spending relative to previous recoveries, the slower the recovery has been.
So we have one clear reason why the recovery from the Great Recession has been weak, and it also explains why it has been weaker in some countries than others. I’m sure its not the only reason, but is anyone seriously arguing anymore that it is not an important factor explaining our weak recovery? So the real mistake that Reinhart and Rogoff made was to push the high debt issue at the wrong time. It was I believe an honest mistake: high government debt is a concern, if only (but not only) because it makes politicians do the wrong thing in a serious recession. And of course the mistake would have happened anyway (in part because of Greece, and partly because of those who see reducing the size of the state as the overriding priority) – academics probably overestimate the importance of the research that politicians use as cover.  But when history tells the story of why the Great Recession was so prolonged, charts like the one here will be what is shown.

In the spirit of their study – and much in the spirit of Wren-Lewis, Krugman and others, Pollin and Ash have now written an article for the FT, most topically on Austerity after Reinhart and Rogoff, where they conclude (emphasis mine):

The case for austerity has never relied entirely on Prof Reinhart and Prof Rogoff. But the other major claims made recently by austerity hawks have also not held up well. Focusing on the US case, austerity supporters circa 2009-10 consistently argued (frequently in this newspaper) that the large US deficits would lead to dangerously high inflation and interest rates. Neither of these predictions came true. In fact, both inflation and the interest rates on US Treasuries were at historic lows in the four years, 2009-12, during which government deficits were at their peak.

 It is also not true that the large deficits have created an unsustainable burden on US government finances. In fact, since 2009, the US government’s interest payments on debt have been at historically low levels, not historic highs, despite the government’s rising level of indebtedness. This is precisely because the US Treasury has been able to borrow at low rates throughout these high deficit years.

We are not suggesting that governments should be free to borrow and spend profligately. But government deficit spending, pursued judiciously, remains the single most effective tool we have to fight against mass unemployment caused by severe recessions. Recent research by Prof Reinhart and Prof Rogoff, along with all related arguments by austerity proponents, does nothing to contradict this fundamental point.

Only a few years ago social unrest, rising from horrendous unemployment in debt-ridden European countries, was a state  of affairs few people expected to see. Keeping in mind that the brutal focus on austerity and little else is only aggravating the situation, the study by Massachusetts trio is such a timely and weighty addition to the debate. It cannot be said any clearer:

… government deficit spending, pursued judiciously, remains the single most effective tool we have to fight against mass unemployment caused by severe recessions.

*The update is this post from Simon Wren-Lewis.

Follow me on Twitter for running updates.

Written by Sigrún Davídsdóttir

April 18th, 2013 at 11:55 am

Posted in Iceland

4 Responses to 'Reinhart & Rogoff – a digest of the debate (updated)'

Subscribe to comments with RSS or TrackBack to 'Reinhart & Rogoff – a digest of the debate (updated)'.

  1. Yep !
    In the fog of economic war one wonders about the minutia of such calculus.
    What is the logic in using such coarse indicator like GDP ? When the GDP increases after a car accident. The mental inertia which prevent us to think in terms of GINI indicator should be questioned, maybe moneyed interest can’t think otherwise.
    Bottom line is being able to provide a decent sustainable living for 7 billion people.
    To try to relieve the anguishing uncertainty of our economic future with such fuzzy mathematics is normal human endeavor, not very efficient though.


    18 Apr 13 at 1:19 pm

  2. The story finally gets more traction.

    I wait with trepidation for a correlation between corruption and GDP growth.
    For instance when one consider the amount spent by Greece in buying weapons, tanks, submarines, war ships, put in relation with the corruption bribes and the effects of GDP growth (or lack of it) in Greece, Germany, France.
    A nice equation.


    20 Apr 13 at 7:27 am

  3. Sigrún,

    For some reason the debate you summarize in your blogpost brings Bram Stoker’s 1887 book “Dracula” to my mind. If you read it you may recall the sweet, gentle and good-natured economic system Lucy Westerna, who was turned into a raging seductress by Dracula’s feeding upon her, despite the efforts of Professor van Helsing and Doctor Seward and Johnathan Harkness to restore her to health, and, failing that, to simply keep her, first, alive, then, failing that, un-zombied and finally dead. They gave her transfusion after transfusion, pouring dollars and euros into her veins, first from their own veins, then from others, brom the ECB Bloodbank, and, if I recal correctly, they even waylaid the oficers of three Icelandic ships that happened to be in harbour, draining them and their entire cargos, pouring all into poor dying Lucy, while the American Fed was also transfusing dollars directly to Dracula.

    Alas, as the story ended, they were not able to get Dracula fat enough he could no longer fit his bulk through the window casement into Lucy’s bedroom, and so poor Lucy became a zombie economy and began preying on children, sucking educations, possibilities, opportunities and so on out of one after another of the next generation, and the next, and the next…

    The problem was that no one figured out that they needed to stop the blood they were transfusing into Lucy being drained away into Dracula, or, for the Americans, that feeding an economic 1% vampire directly does not sate it, or reduce its appetite, or leave anything extra to stimulate the economy the parasite is reducing (or restore the lost strengths to all those, honest enterprises, pension funds and others with obligations of their own, drained to transfuse Lucy, and so only feed the vampire. Nor would draining them more do any more, of course. The Klugman and Keynes solution requires a staunching of the outflow for the inflow to be effective.

    And meanwhile, the discussion amongst the academics, whether the benefit Lucy receives from transfusions over 90% becomes less as the pints tick up, is, well, I suppose, interesting, but it certainly is academic…

    As would be any discussion of effects of blood-types in transfusions in an 1887 novel.


    22 Apr 13 at 12:51 am

  4. Recovery is slow after a financial crisis? Brilliant, McArdle! Whom will you compliment next, someone who predicts the sun will come up in the east tomorrow? If the only example you can give of something R&R got right is an item that falls in the “Captain Obvious” category, it’s time to call their work what it is, i.e. a mistake, and policies relying on them what they are, i.e. tragedies. Wren-Lewis makes it clear that the austerity “solution” has been part of the problem. If anyone is still arguing against MMT, he’s either an idiot or a hired gun.

    Knute Rife

    23 Apr 13 at 8:18 pm

Leave a Reply