In Data, Making Things Equal

There’s more than one reason why the sales cycle in school districts takes 18 months or so. One of the reasons is controlling for outcomes. Nobody wants to pay money for something that is unproven — like online learning — even though it’s clearly changed the lives of millions of college-age online students. There’s no outcome evidence. There also isn’t an objective third party to control value and truth in terms of data. Shafeen Charania blogs about business thinking that offers control, but also liberty to create and accelerate.

Say someone came up to you and said “I want you do this job, I’m not going to tell you how, I’m not even going to tell you what success looks like, you have to figure that out for yourself, but I will pay you a ton of money. Oh and by the way, I’m almost never going to check up on you.” Would you be interested? Well – duh!! You get paid and you get to say what you have to do get paid, and they may not even check that you actually did the work – how sweet is that!
Does this happen often? Yep, and especially in governments and other ultra-large organizations.

And the federal government has pursued this line of thinking with education before.

The second example is the No Child Left Behind Act – while conceptually a good thing, it is hampered by the weird jurisdictional dance that the Feds have to play with the States and local governments on education. The concept was good – let’s figure out a way to push schools to graduate every student, and also improve what is taught.
Using the only power they had (money), the Feds paid for outcomes, but left it to the States to define those outcomes, define their baseline, and define how to measure progress. The results speak for themselves – students got dumber, money got wasted, everyone was more frustrated.

How do you guard against failure with the current Race to the Top initiative, which is an attempt at correcting year’s of bad guidance? Chanaria talks generally about business thinking, but I like what he says here, and I think it can be applied to decision-making about blended or online learning services and products.
It’s the basis of why there needs to be a foundation of good data analysis, let alone the objective retrieval and delivery of such data to the people who need to see if things are working.

Separate performance from measurement. This should be obvious but it isn’t. If the people you’re paying are also measuring the results and reporting them back, there is NO realistic expectation that you will ever get the truth. Consider the simple fact that they get paid if their results look good, and they get paid more if their results look better…