Additional Math Pages & Resources

Tuesday, January 10, 2012

Evidence-Based Education, Part II


Yesterday in the Excel Math blog I complained about the complexity of evidence-based decision-making. I said the process is too complicated. Its very complexity and complication drives out simple but valid options. I normally talk about using elementary math in everyday life, but any curriculum selection decision-making process transcends simple math. [click image to enlarge]

 Paraphrasing  Benjamin Graham very slightly:

Normally, mathematics is expected to produce precise and dependable results; but in the stock market, the more elaborate and abstruse the math, the more uncertain and speculative are the conclusions.

In 44 years of Wall Street experience, I have never seen calculations made about stock values or investment strategies that needed to go beyond simple arithmetic or elementary algebra. 

Whenever calculus is brought in, or higher algebra, take it as a warning that the speaker is about to substitute theory for experience, and make speculation seem to be investment!

Keep this quote in mind, and substitute procurement for investment as appropriate!

Acquisition and Procurement
With a few clicks of the mouse you can investigate dozens of purchasing projects that end up being scrapped due to complexity. [ Notorious examples include US Air Force tanker aircraft contracts and the UK's NHS computer system, etc. ] I have participated in several smaller but (in)famous development projects myself.

In the Education business, we as a society (superintendent, school board, principal) appear to be going down the same path. We decide that those closest to the action (teachers, parents) can't make decisions on their own.

The process, by its very complexity, demands that decisions be made by professional evidence-sifters - pushed upwards to a higher authority, who makes a judgement affecting more than one school or district - perhaps even a whole state.

One advantage from a School District point of view is the implied credibility of this decision-making process:
  • Who would criticize decisions made on the basis of facts? (Can you go wrong buying IBM?)
  • Are you setting yourself up against our authority? 
  • After all this, why would you debate our decision? 
However, it is simplistic to think that decision-making is funded, pursued and concluded in a completely objective way. Even if we do set our biases aside, refrain from emotional influences and spurn illicit offers of personal enrichment, the "evidence-based" process implies
  • We eliminate smaller companies, resource-poor researchers, non-expert witnesses, or advocates from either extreme end of the spectrum
  • Few decision-makers will re-study something that has been panned before
  • Only the most popular products are worthy of study
  • Only the most readable papers get published
  • Only the most attractive ideas get funding
And we layer this evidence-based evaluation onto an already laborious adoption process (6 years in California)! If it takes longer to choose the curriculum than a student stays in a public elementary school, are we over-thinking it?
I'll close with an illustration from an earlier blog on objective measures for the subjective topic of pain. Please select the level of pain you associate with involvement in a procurement project: