Quick version of how Bush fooled everyone by claiming that the two RAND
reports conflicted
- Both RAND studies used the same data (NAEP)
- Bush took office in 1994
- Texas implemented high-stakes testing in 1994. This caused a spike in the
1996 4th grade math score since the NAEP and TASS are similar for 4th grade
math (drilling for the TASS bumped up the NAEP just like if you study how to
do square roots before you took an exam, you'll do a lot better than if you
didn't study)
- The original RAND study (Grissmer) looked at the period from 1990 to 1996. Thus,
the 1996 spiked math score put Texas off the charts. The author should have
factored in "coached" vs. "non-coached" states, or ran
the analysis from 1994 backwards, and from 1994 forwards. By failing to
account for the spike, you get wierd results, in this case, one datapoint
that causes Texas to move off the charts improvement-wise when scores were
added together.
- Had Grissmer confined the analysis to the data from 1994 onward, i.e.,
after Bush was elected Governor, he'd find
Texas was flat. He admitted this on the phone to me.
- Klein avoided the suspect exam (the 4th grade math exam can be influenced
by routine drilling because it is just very basic memorization rather than
real thinking like the 8th grade test) and focused on following a 4th grader to
8th grade 4 years later. He did this in reading (1994 to 1998) and math
(1992 to 1996). In both cases, Texas improved no more than the rest of the
country!! This agrees with what Grissmer said about the post 1994 data. If
there was truly a educational gain in Texas, there would be no possible way
for it not to show up on Klein's tables. It's impossible. Nobody has
disputed that. That is a key point that nobody seems to want to point
out. No detractor can explain why this supposed mass improvement didn't show
up in the Klein analysis. Naturally, the press ignores this. They don't
go any deeper than saying the two reports disagreed. They never asked "why"
they disagreed.
- Klein also found the achievement gap widened.
- Klein's study is much more difficult to attack than Grissmer
because Klein used raw NAEP scores, and he computed each ethnic group
separately. So there were no fudge factors or assumptions like in Grissmer's
report.
- It's possible that there is more than one explanation that explains the
1996 spike. But it doesn't matter since Klein showed that there was no
progress over the period studied and Klein's stuff is hard to challenge.
Nobody has ever challenged Klein's data or approach.
Bottom line:
For both reports, using only data from 1994 onward, we find RAND both
reports agree: nothing special is happening in Texas and the achievement gap
is getting worse, not better.
Try it yourself step by step... How to prove to yourself that the national test
data shows no improvement in Texas since 1994 (in 2 minutes)
Steve Kirsch
Political Home Page
|