Financial incentives for schoolkids - new evidence

Massive hat-tip to this US blogger who has been following the Capital Gains programme (one of those where school kids are paid for attendance, reading books, grades etc).

His posts led to this report (PDF) by Roland Fryer, who is the economics professor in the US apparently most associated with the 'Green for Grades' type programmes. As I have said before, I think the results of these initiatives will tell us a lot, one way or another, about how financial incentives affect motivation in a very important area.

It's an interesting report, if not the most clearly written in terms of expressing the results, and I'm resisting the temptation to cherry pick the bits that support my hunch (that financial incentives aren't going to be very effective in terms of academic performance). So here are some of the conclusions:

The results from our incentive experiments are interesting and in some cases quite surprising. Remarkably, incentives for output did not increase achievement. Paying students for performance on standardized tests yielded treatment effects for seventh graders between -.018 (.035) and -.030 (.063) standard deviations in mathematics and .018 (.018) and .033 (.032) standard deviations in reading. The programs in which fourth graders were paid for their test scores exhibited similar results. Rewarding ninth graders for their grades yielded increases in their grade point average of .093 (.057) and .131 (.078), but had no effect on achievement test scores in math or reading.

Conversely, incentives can be a cost-effective strategy to raise achievement among even the poorest minority students in the lowest performing schools if the incentives are given for certain inputs to the educational production function. Paying students to read books yields a large and statistically significant increase in reading comprehension between .180 (.075) and .249 (.103) standard deviations, increases vocabulary between .051 (.068) and .071 (.093) standard deviations, and increases language between .136 (.080) and .186 (.107) standard deviations. The estimated impacts on vocabulary scores are not significant; increases in language are marginally significant. Similarly, paying students for attendance, good behavior, wearing their uniforms, and turning in their homework increases reading achievement between .152 (.092) and .179 (.106) standard deviations, and increases mathematics achievement between .114 (.106) and .134 (.122) standard deviations. The point estimates are moderate in size, but we do not have enough statistical power to make confident conclusions. The effects of incentives in Washington, DC, on reading achievement are marginally significant in reading and statistically insignificant in math.

I think if we cut through the 'inputs' and 'outputs' terminology the story seems to be - if you pay kids to turn up, more will turn up, and if you pay them to read books, more will read books (which is A Good Thing). But if you reward kids for doing well in tests it makes little or no difference. Obviously this story has much longer to run, and I take my hat off to Fryer for trying something different to address a real problem. But it looks to me like a pretty significant setback for those who think that financial incentives are the answer to a lack of educational attainment. This, I humbly submit, is a rather big deal.

UPDATE: Actually I think I've been a little unfair. The point about the impact on reading books is important, as the report indicates that leads to improved vocabulary etc. And improved attendence likewise (you might accidentally learn something even if you only turned up for the money). Anyway, read the thing yourself and let me know what you think...