This is based on a comment I made here.
When trying to tease out what sorts of policies work and what do not, people often make cross-country comparisons of, for example, expenditure on education (such as, as a % of GDP). Trouble is, expenditure on education is a remarkably useless measure.
An obvious indicator of that is that expenditure on education and education outcomes are poorly correlated. But, particularly in a situation where most expenditure on education is government expenditure, there is nothing surprising about this. Government expenditure is merely the cost of inputs, and the cost of inputs does not tell you the value of something. Businesses seek to have the cost of their inputs be considerably less than the price they sell a good or service for and to avoid the cost of inputs being larger than what they can sell it for. While price is not a perfect measure of value, it is a great deal better than merely the cost of inputs. This is a petite version of the economic/socialist calculation problem and it afflicts all government expenditure figures which are not market-priced, merely input-costed.
A point that extends across the range of government expenditure. Do people, think, for example, that expenditure per ISIS fighter and expenditure per Iraqi soldier tells us anything about their relative military effectiveness?
Regarding education, comparing (say) Latin American expenditure on education and East Asian expenditure on education is almost completely useless. Family effort and expectations about education matter greatly.
Latin American societies are based on various levels of “social mercantilism”, where very uneven provision of public goods (such as property rights recognition) and bureaucratic approval systems are used to protect “insiders” against “outsiders”. While East Asian societies are noted for certain sorts of government interventions, they are more of the “market augmenting” type than of the “market controlling” type. And their markets are generally very open to firm entry (at least for domestic firms).
So building enough schools and legal environment so everyone can go to school is going to operate differently in a society with strong education attachment and expectations (East Asia) compared to a society where which family you belong to with what connections counts so much more (Latin America). To put it another way, expenditure on education in a low-Gini ratio society with markets open to firm entry are going to operate very differently than expenditures on education in a high-Gini ratio society riddled with bureaucratic barriers to entry.
Hence, comparing Latin American and East Asian (input) expenditures on education is going to be almost completely useless, given the vastly different contexts.
Much the same point counts with the term “government intervention”. Market-augmenting actions operate very differently than market-controlling actions, yet both count as “interventions”. The real measure is how much government actions raise or lower transaction costs, commercial risks or otherwise get in the way of/promote gains from trade, but you cannot tell that from merely calling government actions “government interventions”.
This is not to say that there is no point to trying to compare large numbers of cases statistically. On the contrary, it can be a great way to sift through possible explanations. The trick is to be aware of the weaknesses in data (generally avoid mere input measures), and how very much context matters.