How Randomistas Can Help Fight Inequality
By Andrew Leigh
Member, Australian Parliament
You might think some theories are so obvious they don’t need to be proven.
- To discourage early pregnancy, ask teens to care for a baby doll programmed to demand attention at all hours.
- Juvenile delinquents can be ‘scared straight’ by spending a day in jail to see how tough prison really is.
- What young unemployed men most need is job training.
Each statement sounds completely reasonable, doesn’t it? Unfortunately, randomized trials showed that girls paired with an infant simulator for a week were twice as likely to become teenage mothers. Scared Straight programs increased crime. And job training programs for unemployed youths have often produced disappointing results.
The methodology behind randomized trials is remarkably simple: assign participants to an experimental or a control group based on the toss of a coin or a computerized random number generator. The difference between the two groups tells you the impact of the policy. When it comes to tackling disadvantage, randomized trials don’t just spotlight failure, they can also shine a light down new paths for addressing poverty.
In the 1960s, researchers commenced a randomized evaluation of the Perry Preschool program designed for disadvantaged children in Michigan. The study found that by the time the participants were in their twenties, those who had been to preschool were more likely to own a car, own a home and have a steady job. They were also less likely to use drugs and less likely to be on welfare. By age forty, a quarter of those in the preschool group had been to jail, compared with half of the control group. Every dollar spent on the program returned between $7 and $12 to society.
In the classroom, the U.K. Education Endowment Foundation has so far commissioned over a hundred evaluations, many of them randomized, to test what works in the classroom. Among those randomized studies that produced positive results are a Singaporean-designed mathematics teaching program and a philosophy-based intervention encouraging students to become more engaged in classroom discussion.
Educational “randomistas” – as proponents of the trials are sometimes known – are also evaluating how to get more low-income children to university. In Ohio and North Carolina, researchers worked with tax preparation company H&R Block to identify low-income families with a child about to finish high school. Half of these families were randomly offered assistance in completing a university financial aid application, a process that took about eight minutes. The intervention boosted college attendance by a quarter.
If government isn’t working well, the most affluent can turn to other alternatives. They can rely on private healthcare, private education, and private security. They are less likely to be unemployed, and have family resources to draw upon in hard times. For the top 1 percent, dysfunctional government may be annoying, but not life-threatening.
But for the most vulnerable, government can mean the difference between getting a good education or struggling through life unable to read and write. Those who depend on government depend on government programs actually working.
We live in a world in which failure is surprisingly common. In medicine, out of the 10 drugs that look promising in lab tests, only one ends up getting approval. In education, just one tenth of the randomized trials commissioned by the What Works Clearinghouse, a project of the U.S. Department of Education, produced positive effects. In business, only a fifth of Google’s randomized experiments showed actual product improvement.
It’s no different in social policy. In the words of the late sociologist Peter Rossi: “The better designed the impact assessment of a social program, the more likely is the resulting estimate of net impact to be zero.”
Conducting more randomized evaluations isn’t an excuse to give up on the fight against inequality. We don’t abandon the search for a cure for cancer just because most cancer drugs to emerge from the laboratory don’t make it through clinical trials. Similarly, the goals of cutting crime, raising test scores, and providing jobs for everyone who wants them should be pursued even if a specific program comes up short.
The more we ask the question “What’s your evidence?,” the more likely we are to find out what works – and what does not. By evaluating social policies, discarding those that don’t work, and boosting those that do, government can have a far greater impact on reducing disadvantage. An experimenting society is likely to end up a more equal society.
Skepticism isn’t the enemy of optimism: it’s the channel through which our desire to solve big problems translates into real results. Given the chance, randomistas can deliver a more equal world, one coin toss at a time.
***
Reposted from Inequality.org
