Quick version of how Bush fooled everyone by claiming that the two RAND reports conflicted

• Both RAND studies used the same data (NAEP)
• Bush took office in 1994
• Texas implemented high-stakes testing in 1994. This caused a spike in the 1996 4th grade math score since the NAEP and TASS are similar for 4th grade math (drilling for the TASS bumped up the NAEP just like if you study how to do square roots before you took an exam, you'll do a lot better than if you didn't study)
• The original RAND study (Grissmer) looked at the period from 1990 to 1996. Thus, the 1996 spiked math score put Texas off the charts. The author should have factored in "coached" vs. "non-coached" states, or ran the analysis from 1994 backwards, and from 1994 forwards. By failing to account for the spike, you get wierd results, in this case, one datapoint that causes Texas to move off the charts improvement-wise when scores were added together.
• Had Grissmer confined the analysis to the data from 1994 onward, i.e., after Bush was elected Governor, he'd find Texas was flat. He admitted this on the phone to me.
• Klein avoided the suspect exam (the 4th grade math exam can be influenced by routine drilling because it is just very basic memorization rather than real thinking like the 8th grade test) and focused on following a 4th grader to 8th grade 4 years later. He did this in reading (1994 to 1998) and math (1992 to 1996). In both cases, Texas improved no more than the rest of the country!! This agrees with what Grissmer said about the post 1994 data. If there was truly a educational gain in Texas, there would be no possible way for it not to show up on Klein's tables. It's impossible. Nobody has disputed that. That is a key point that nobody seems to want to point out. No detractor can explain why this supposed mass improvement didn't show up in the Klein analysis. Naturally, the press ignores this. They don't go any deeper than saying the two reports disagreed. They never asked "why" they disagreed.
• Klein also found the achievement gap widened.
• Klein's study is much more difficult to attack than Grissmer because Klein used raw NAEP scores, and he computed each ethnic group separately. So there were no fudge factors or assumptions like in Grissmer's report.
• It's possible that there is more than one explanation that explains the 1996 spike. But it doesn't matter since Klein showed that there was no progress over the period studied and Klein's stuff is hard to challenge. Nobody has ever challenged Klein's data or approach.

Bottom line: 

For both reports, using only data from 1994 onward, we find RAND both reports agree: nothing special is happening in Texas and the achievement gap is getting worse, not better.

Try it yourself step by step... How to prove to yourself that the national test data shows no improvement in Texas since 1994 (in 2 minutes)

Steve Kirsch Political Home Page