De Rerum Natura: Stupid Statistics from the Federal Reserve

« Previous | Main | Next »

Stupid Statistics from the Federal Reserve

Posted by Reed A. Cartwright on July 28, 2004 1:45 PM

In the news today, economists at the federal reserve say “belief in hell boasts growth”.

Economists searching for reasons why some nations are richer than others have found that those with a wide belief in hell are less corrupt and more prosperous, according to a report by the Federal Reserve (news - web sites) Bank of St. Louis.

Turning to the actual report, should make anyone with any training in statistical analysis pound their heads against the wall. First rule, correlation is not causation. But what about their correlation?

Yeap that is the graph that they use to show the negative correlation between fear of hell and corruption. Powerful isn’t it. Notice what is missing? There is no significance placed on the correlation. Since they provide their data in Excel format, I decided to use PopTools to do a Monte Carlo analysis of their data.

Simply put, I randomized the relationship between fear-of-hell and corruption rankings and calculated a correlation coeficient for the random data. I did this ten thousand times and used this to determine if the observed data was significantly negative. It was not. A random sample of 10000 replicates produced a 95% confidence interval of -.34 to .33. This means that the observed correlation, -.34, is just barely not significant.

There are many methdological problems with the paper, but their own data don’t support the conclusion that the authors want to make.

Update

The actual claim is that belief in hell means that you are more prosperous. However, the paper does not even report the correlation for GDP and belief in hell. This is because it is 0.15, which is not even close to being borderline significant.

It gets even better. If you calculate a correlation between their raw data and not the rankings, you get a negative correlation (-0.21) between per capita and belief in hell. This exactly contradictory with claims made in the paper. In fact, based on additional Monte Carlo analysis, this correlation is approximately what you would expect if per capita and belief in hell were independent of one another: average -0.22, lower cl -0.59, upper cl 0.19.

I feel that these authors are knowingly being dishonest with their data.

TrackBack URL for this entry: http://scit.us/cgi-bin/mt/mt-tb.fcgi/207.

Comment: #166

Posted by Ed Hessler on July 28, 2004 2:24 PM

Thanks, Reed, for posting the report and as importantly (or more) for your work with the data. Have you sent it to them? It would seem to me that they might have thought of doing this but their econo blinkers might have prevented it. How quickly church-types seized on this and will. And thanks too for your thoughtful comments on the new, emerging GA standards for science. Cheers.

Comment: #167

Posted by John on July 29, 2004 11:04 PM

Also interesting, a quadratic regression equation shows a medium start, then a rise, then a drop to a point lower than that. Also, cubic and quartic regressions are even worse. It actually goes from a low point, then it rises, then another low point (but not as low as before), and then starts rising again. So in other words, too much fear of hell is also bad, and no fear of hell is best, at least according to cubic and quadratic. Just like the saying, “Torture numbers and they’ll confess to anything”