That measure, it does not mean what you think it means

By Lorenzo

This is based on a comment I made here

When trying to tease out what sorts of policies work and what do not, people often make cross-country comparisons of, for example, expenditure on education (such as, as a % of GDP). Trouble is, expenditure on education is a remarkably useless measure.

An obvious indicator of that is that expenditure on education and education outcomes are poorly correlated. But, particularly in a situation where most expenditure on education is government expenditure, there is nothing surprising about this. Government expenditure is merely the cost of inputs, and the cost of inputs does not tell you the value of something. Businesses seek to have the cost of their inputs be considerably less than the price they sell a good or service for and to avoid the cost of inputs being larger than what they can sell it for. While price is not a perfect measure of value, it is a great deal better than merely the cost of inputs. This is a petite version of the economic/socialist calculation problem and it afflicts all government expenditure figures which are not market-priced, merely input-costed.

A point that extends across the range of government expenditure. Do people, think, for example, that expenditure per ISIS fighter and expenditure per Iraqi soldier tells us anything about their relative military effectiveness?

Regarding education, comparing (say) Latin American expenditure on education and East Asian expenditure on education is almost completely useless. Family effort and expectations about education matter greatly.

Must get good score to get a good job and make my family proud.

Latin American societies are based on various levels of “social mercantilism”, where very uneven provision of public goods (such as property rights recognition) and bureaucratic approval systems are used to protect “insiders” against “outsiders”. While East Asian societies are noted for certain sorts of government interventions, they are more of the “market augmenting” type than of the “market controlling” type. And their markets are generally very open to firm entry (at least for domestic firms).

So building enough schools and legal environment so everyone can go to school is going to operate differently in a society with strong education attachment and expectations (East Asia) compared to a society where which family you belong to with what connections counts so much more (Latin America). To put it another way, expenditure on education in a low-Gini ratio society with markets open to firm entry are going to operate very differently than expenditures on education in a high-Gini ratio society riddled with bureaucratic barriers to entry.

You trying to tell us that it is not about who our family is?

Hence, comparing Latin American and East Asian (input) expenditures on education is going to be almost completely useless, given the vastly different contexts.

Much the same point counts with the term “government intervention”. Market-augmenting actions operate very differently than market-controlling actions, yet both count as “interventions”. The real measure is how much government actions raise or lower transaction costs, commercial risks or otherwise get in the way of/promote gains from trade, but you cannot tell that from merely calling government actions “government interventions”.

This is not to say that there is no point to trying to compare large numbers of cases statistically. On the contrary, it can be a great way to sift through possible explanations. The trick is to be aware of the weaknesses in data (generally avoid mere input measures), and how very much context matters.


  1. conrad
    Posted August 23, 2014 at 12:49 pm | Permalink

    I think its all contextually sensitive. In the case of education, for example, comparing the total input into universities, for example, is not necessarily worthless because it really does predict success in some areas (e.g., the health of your biotech industries which, in the US, get massive more or less free inputs). Similarly, Australia is not the US, but it’s hard to argue that the massive difference in health-care expenditure is because things are working well in the US.

    More importantly, if we move away from money, then the story can still be important. For example, there are massive cross-country difference on maths performance. This is important, because all of the factors that people like to look at and arguable about endlessly, generally because they are easy to look at, never predict much variance (e.g., proportion of private-to-public schools, autonomy of schools, individual teacher performance, voucher usage, etc.).

    This basically leaves you with factors that are harder to examine. One of these is that which you mention (culture), but there are other things where the nitty-gritty really does matter (e.g., mathematics performance, literacy). If you want good mathematics performance, for example, the biggest easily controllable factor is what you teach from about 3 years old and on. But no-one cares about this because it’s too hard to think about. Most people don’t know the slightest thing about what really should be taught and why and nor how mathematics even develops in children (probably and unfortunately including most edumacation bureaucrats in Australia). Similarly, literacy teaching really makes a big difference to outcomes, and some places do it better than others. But all we here about is how children in Year 11 (or whatever) are getting biased questions in their curriculum. By that age, all of things that matter that you need to teach are long gone.

    So used intelligent, we can learn something useful, but the solution is at the other end of the spectrum in terms of what we need to compare.

  2. conrad
    Posted August 23, 2014 at 12:51 pm | Permalink

    Excuse the typos 🙂

  3. Posted August 23, 2014 at 2:52 pm | Permalink

    [email protected] is all terribly sensible and informative, thank you.

    [email protected] You’re excused 🙂

Post a Comment

Your email is never published nor shared. Required fields are marked *