I have been troubled by Paul Collier’s research and policy advocacy for some time. In this essay he goes even further in directions I argued were dangerous in his previous work. Collier wants to de facto recolonize the “bottom billion,” and he justifies his position with research that is based on one logical fallacy, one mistaken assumption, and a multitude of fatally flawed statistical exercises.
The logical fallacy leads to the conclusion that the poorest countries systematically fall behind everybody else in economic growth. Of course they do! Collier selected countries that were on the bottom at the end of a specific period, so naturally they would be more likely to have had among the worst growth rates in the world over the preceding period. This ex post selection bias makes the test of poor-country divergence invalid. The correct test would be to see who is poor at the beginning of the period and then see if they have worse growth than richer countries in the following years. When the test is run this way, there is no evidence that poor countries grow more slowly than richer countries.
Collier’s bottom billion is in fact a constantly changing mishmash of initially poor countries with average growth combined with other countries that were initially richer countries that had sharply negative growth. (Côte d’Ivoire and Zimbabwe are classic examples of the latter.)
Collier’s mistaken assumption is that below-average growth usually persists. A large body of research over two decades has shown that growth is extremely volatile and that there is strong reversion to the mean—countries with below-average growth in one period tend to move back to the average in the following period. The same countries do not show up period after period as growth disasters.
So the combination of the logical fallacy and the mistaken assumption lead to Collier’s incorrect assertion that the membership and income of the bottom billion will never change.
Lastly, Collier’s work is built on deeply problematic statistical analysis. Valid statistical results must meet stringent conditions. The usual standard for labeling a result “significant” is that it could have occurred by chance only one out of twenty times, assuming a statistical exercise is run only once. An unfortunately all-too-common practice called “data mining” involves running twenty statistical exercises and then reporting only the one that produces a “significant” result (which will have happened by chance). Collier comes close to admitting that he does exactly that. In a previous article, he wrote: “The reference model is reached after a series of iterations in which insignificant variables are deleted and variants of economic, social, geographic and historical explanatory variables are then tested in turn.” One common symptom of data mining is that the (spurious) results disappear when new data emerges—exactly what Collier reports in Wars, Guns, and Votes: “Our previous results got overturned by the new data.” But this is not the scientist in the lab coat retesting a well-defined hypothesis. Collier’s convoluted stories are made up after the fact to fit whatever random collection of data points he is working with at the moment. So the specious rationalizations keep changing—too bad for those who took the precise recommendations in The Bottom Billion as gospel.
Yet even if Collier got valid correlations, which he has not, correlation does not equal causation, which he also fails to address. As Daron Acemoglu of MIT commented on Collier’s earlier research on civil wars: “The correlations that are interpreted as causal effects are really no more than correlations. . . . It is too early to jump to policy conclusions.”
Given that Collier’s evidence base collapses when subject to scrutiny, it is all the more disturbing that his policy recommendations are remarkably interventionist. Collier tells bottom-billion societies that are recovering from civil war that they must accept international “peacekeepers” (a nice euphemism to help us forget that these are soldiers who kill people), whose deployment is decided by the Western powers.
Even more outlandish, Western powers will use military force to pass judgment on elections. If the West decides that a country’s election is free and fair, then the West will invade that country if its military subsequently overthrows the government. If the West decides that an incumbent stole an election, then the West will withdraw the anti-coup guarantee, lending Western encouragement to a coup. As Collier nonchalantly notes, “an incumbent who stole an election would face a heightened risk of a coup.” Collier also recommends in Wars, Guns, and Votes that the West forbid an election altogether in some circumstances that the West judges (based on Collier’s invalid results) to be dangerous.
Foreign armies invade and control the whole political process—yes, the word “neocolonial” is overused, but with Collier’s recommendations I think one could drop the “neo.” Such aggressive interventions will almost certainly have unintended negative consequences. Will Western violence beget more local violence? Will there be a violent backlash against foreign intervention? Foreign armies will likely kill some innocents in the process of peacekeeping. The motto “first, do no harm” puts the burden of proof on the academic interveners. Collier falls short by intercontinental ballistic missile range.
Remarkably enough, Collier puts the burden of proof on non-intervention. “At some point,” he writes, “doubt becomes an excuse for inaction, while the problems of insecurity remain real enough.” Elsewhere Collier alludes to his doubters as “professional skeptics.” Actually, doubt is a superb reason for inaction. If being a professional skeptic entails scrutinizing the logic, the assumptions, and the evidence base and finding them all invalid when they do not meet normal academic standards, I plead guilty. If being a professional skeptic means putting a large burden of proof on those who want to invade poor countries, run poor people’s societies for them, and deny poor people their own democratic rights, I plead twice guilty. I wish we had many more professional skeptics, including above all from the poor countries themselves, to scrutinize the extreme recommendations of academic armchair warriors.