Stupid Statistics, Second Pass

Yesterday I sent the following letter to The Regional Economist about the stupid article, “Fear of Hell Might Fire Up the Economy.”

The analysis of the data in the article is flawed. No significance values are attached to the reported correlations. It is obvious from the chart of belief in hell versus corruption that the values are not well correlated. A Monte Carlo analysis confirms this. The 95% confidence interval of these rankings is approximately -0.34 to -.33. This means that the observed value, -0.34, is just barely not significant. An analysis not reported in the paper, correlation between belief in hell and per capita, results in a value of 0.15 which is clearly not significant.

Going further, a comparison of the raw data reported in the article finds that per capita and percent belief in hell are actually negatively correlated at -0.21. Doing another Monte Carlo analysis finds that this is close to what we expect if the two were independent of one another. The average is -0.22, and the 95% confidence interval is -0.59 to 0.19.

Clearly the reported correlations between belief in hell and corruption and belief in hell and per capita are not significant, which means that any attempt to conclude association between them is flawed. I hope it was not the intention of the authors to mislead their readers.

Well the article has now been changed (again), and all the original work, looking at corruption levels, has been removed. From the editorial note that now appears:

Thanks to the keen eyes of a number of readers, however, we have discovered that the charts used in both of these versions of the article contained errors. Consequently, the version below does not include discussions of the correlations between religiosity, corruption and per capita income.

It was not the charts that contained errors; it was their entire methodology. Garbage in, garbage out. Continuing,

It is important to note that this has no bearing on the results in the literature that are discussed in the article. It is not uncommon, for example, for simple correlations between two variables to provide different answers from regressions that control for a longer list of variables.

I’ve briefly looked at the Barro and McCleary paper, and I have to say that I am not impressed with their results. They do a complicated analysis of multiple factors, which reveals that “fear of hell” is slightly positively related with economics, after other effects have been removed. In other words, “fear of hell” might influence economies, but it is not a major factor. The regression that they find in their most detailed model is minor, 0.0174, with a standard error of 0.0083. This means that the “95% confidence interval” gets very close to zero. I think that this paper needs a Monte Carlo analysis where the initial data is randomly resorted based on a reasonable model and then reanalysized. I feel that this is an excellent way to see if the regressions are significantly different from zero.

TrackBack URL for this entry:

Use KwickXML Formatting to markup your comments, acceptable tags: <b> <blockquote> <br> <code> <em> <email> <h1> <h2> <h3> <h4> <h5> <h6> <i> <li> <list> <ol> <p> <qref> <quote> <s> <strong> <sub> <sup> <u> <ul> <url>. You may need to refresh before you will see your comment.

Remember personal info?