There’s been much talk lately about problems with the No Child Left Behind law. So many states are falling behind on meeting some of its provisions that the Obama administration has announced it will issue waivers to free them from its shortcomings. But maybe the problem isn’t with the specifics of certain provisions of the law. Maybe the problem is with its whole approach: evaluating the quality of a student’s education by giving them a standardized test.

Cathy N. Davidson, who teaches at Duke University, believes that approach is out of place in today’s world. In a Washington Post opinion piece she notes that “When Frederick J. Kelly invented the multiple-choice test in 1914, he was addressing a national crisis.” That crisis was caused by an explosion in the number of secondary school students:

The ranks of students attending secondary school had swollen from 200,000 in 1890 to more than 1.5 millionas immigrants streamed onto American shores, and as new laws made two years of high school compulsory for everyone and not simply a desirable option for the college bound. World War I added to the problem, creating a teacher shortage with men fighting abroad and women working in factories at home.

According to Davidson, Kelly drew on the mechanical mindset of the time to propose a solution:

The country needed to process students quickly and efficiently. If Henry Ford could turn out Model Ts “for the great multitude,” surely there was an equivalent way, Kelly wrote in his dissertation at Kansas State Teachers College, to streamline schooling. What he came up with was the Kansas Silent Reading Test, sometimes called the “item-response” or “bubble” test.

Such tests have been with us ever since, from high school aptitude tests (like SATs) to end-of-course and/or exit exams used by many states to meet the requirements of No Child Left Behind. Unfortunately, as Davidson points out, these tests are ineffective:

In a decade of researching digital education, I have never heard an educator, parent or student say that the tests work well as teaching tools. Beyond the flaws of these rigid exams — which do not measure complex, connected, interactive skills — there is little room in the current curriculum or even in the current division of disciplines (reading, writing, math, natural sciences and social studies) for lessons about key questions that affect students’ daily lives.

Interestingly (to me anyway), there’s been a similar conundrum in the audit field. Auditing has traditionally focused on hard controls (things like procedures, segregation of duties, supervisory oversight, etc.). However many of the biggest business failures – like Enron, Tyco and the recent debacle on Wall St. – were rooted in soft controls (things like employee morale, ethics, philosophy, values, integrity, etc.).

What we’ve learned is that having great hard controls won’t matter if a company’s leaders and/or staff are not inclined to follow them. Author James Roth described a situation he encountered during the Savings & Loan crisis:

During the S&L crisis, I was working for a banking organization in Minneapolis, where a $3.5 billion S&L failure occurred. After the S&L was dissolved, our bank acquired six of the S&L’S branches. When we began the first audit of those branches, we expected to find their internal control systems riddled with holes.

We were surprised to find instead that those branches were beautifully controlled according to our audit tests. They had every policy, procedure, and checklist imaginable. A teller could not swipe $10 from her drawer without getting caught. In spite of the control activities, the president and founder was able to play games; and he, his daughter, and several members of the upper management team went to jail.

After that, I asked myself if I had been an internal auditor working at that organization and using our control activity-based audit program, would I have had the foggiest notion of what was happening? Not from my audit work. The only way you would ever find out something like that is if someone tipped you off. That sort of realization makes you feel really uncomfortable about your status as a professional evaluator of control.

As Roth noted seven years ago in talking about the Sarbanes-Oxley Act that was passed after the failures of major companies like Enron and Tyco:

Companies are spending enormous time, energy, and consulting fees documenting and testing detailed financial control activities to comply with Section 404 of the U.S. Sarbanes-Oxley Act of 2002. But if all this resource-intensive work had been performed at Enron, WorldCom, or Parmalat, would it have prevented what happened? For that matter, would it have prevented any of the recent financial reporting disasters? Such frauds are not caused by noncompliance with low-level accounting procedures. The root cause is always a breakdown in the control environment, usually the ethical climate and the behavior of executives. Testing of accounting procedures to the extent it is done today is not only expensive, but also–for the most part–misses the point.

Why have auditors traditionally focused on hard controls rather than soft when examining businesses? In large part because they’re much easier to measure and evaluate. Why do politicians and administrators favor standardized tests when evaluating schools? Most likely for the same reason.

It reminds me of an old joke:

A cop saw a blonde down on her knees under a streetlight. “Can I help you?” he asked.
Replied the blonde, “I dropped my diamond ring and I’m looking for it.”
Asked the cop “did you drop it right here?”
“No,” she responded, “I dropped it about a block away, but the light’s better here.”

Just being able to test for something doesn’t mean the results will be relevant.

If we really want to ensure that America’s children are being properly educated, it’s not enough to simply make them take standardized tests. As with auditing, we must make sure those tests aren’t missing the point.