How to verify the NAEP data yourself

Suppose you want to figure out how whites in Texas did relative to the rest of the country. You would do the following using white data only from public schools only from the tables below (or directly from NAEP site):

• Texas gain=(1998 8th grade reading score) - (1994 4th grade reading score) [from the Texas tables]
• US gain=(1998 8th grade reading score) - (1994 4th grade reading score) [from the US table]
• Texas improvement=(Texas gain)-(US gain)

So now let's do it using real data for whites which we get from the tables below:

• Texas gain=273-227=46
• US gain=270-223=47
• Texas improvement=46-47= -1

So in other words, Texas did not improve as much as the US as a whole (that's the part that Bush says Al Gore was responsible for).

The calculation for other groups is just as easy.

You can load the data yourself from the NAEP site, or  use the data below which is copied from those pages.

If you want to load the data yourself, go to:
http://nces.ed.gov/nationsreportcard/TABLES/index.shtml

The tables to load to verify Table 2 of Klein's paper are:

• 1998, Reading, 8 [continue], National-Public, Student Data [Begin Search], Select "Total Student Population" link
• 1998, Reading, 4 [continue], National-Public, Student Data [Begin Search], Select "Total Student Population" link
• 1998, Reading, 8 [continue], Texas, Student Data [Begin Search], Select "Total Student Population" link
• 1994, Reading, 4 [continue], Texas, Student Data [Begin Search], Select "Total Student Population" link

Note: No "fuzzy math" is required! A Texas 8th grader could do the calculation! You don't have to do any normalization if you look at each racial group separately. You just extract the 12 numbers from the publicly available 4 tables (as noted below), and subtract the scores (see Table 2 in Klein's paper). If Grissmer had interpreted data only from 1994 onwards, he would have made exactly the same conclusion. The data is unambiguous. No fuzzy math. No assumptions. It shows clearly that there is nothing special going on in Texas. All external independent external evidence shows that to be the case! 

Notes:

•  The argument about using the earlier study to discredit the new study is also somewhat of a red-herring. The “Texas Miracle” was based on state TAAS scores. The earlier study didn’t use TAAS scores.
•  The argument that Texas NAEP scores are above the US average especially for Blacks and Latinos is also bogus. It's been that way for a long time (for at least a decade).

Bottom line:

• The only independent data available (the same data source used by Grissmer, who later added the 1998 data but didn't publish it) shows nothing special going on in Texas relative to the rest of the country
• Klein's paper shows that the state TAAS scores are not trustworthy, the achievement gap widened under Bush, and that there was no improvement in academic performance

THE NEW NEWS IS THIS:

• There is now ZERO independent trusted data to back Bush's claim.
• Everything we see which is trusted independently (SAT scores, etc.) shows ZERO to decline.

The only NAEP data you can use for the comparison if you want to look at 1994 onwards is 1994 4th grade Reading vs. 1998 8th grade reading. Other datasets are not available because you can't get the right pairs, e.g., you can't get 8th grade 1994 reading and 12th grade 1998 reading, you can't get 4th grade math in 1994 vs. 4th grade math in 1998, etc.

Questions? Contact me: stk@propel.com

Here is the data, collected for your convenience (I've highlighted the 12 numbers you want are in Blue/Bold). They exactly match the data in Table 2.

 

 

NAEP ID: COHORT		TOTAL STUDENT POPULATION
		DEMOGRAPHIC AND DERIVED VARIABLES

---

	N	TOTAL			%  MISSING
PUBLIC SCHOOLS
-- TOTAL --	9091	100 (****)			0
		261 (0.8)			****
RACE/ETHNICITY
WHITE	5013	100 (****)			0
		270 (0.9)			****
BLACK	1565	100 (****)			0
		241 (1.6)			****
HISPANIC	1934	100 (****)			0
		243 (2.1)			****
ASIAN/PACIFIC ISLANDER	456	100 (****)			0
		269 (4.0)			****
AMERICAN INDIAN	106	100 (****)			0
		248 (4.8)			****
MISSING  responses were not included in the calculation of percentages for the other categories.
(****)  Standard error estimates cannot be accurately determined.
****  Sample size is insufficient to permit a reliable estimate.
The nature of the sample does not allow accurate determination of the variability of the statistic.

 

 

NAEP ID: COHORT		TOTAL STUDENT POPULATION
		DEMOGRAPHIC AND DERIVED VARIABLES

---

	YEAR	N	TOTAL			%  MISSING
PUBLIC SCHOOLS
-- TOTAL --	1998	6300	100 (****)			0
			215 (0.8)			****
	1994	6030	100 (****)			0
			212 (1.1)			****
	1992	5045	100 (****)			0
			215 (1.0)			****
RACE/ETHNICITY
WHITE	1998	3314	100 (****)			0
			225 (0.9)			****
	1994	3419	100 (****)			0
			223 (1.3)			****
	1992	2897	100 (****)			0
			223 (1.3)			****
BLACK	1998	1162	100 (****)			0
			193 (1.8)			****
	1994	1175	100 (****)			0
			186 (1.7)			****
	1992	957	100 (****)			0
			192 (1.6)			****
HISPANIC	1998	1455	100 (****)			0
			195 (1.9)			****
	1994	995	100 (****)			0
			188 (2.7)			****
	1992	897	100 (****)			0
			199 (2.2)			****
ASIAN/PACIFIC ISLANDER	1998	206	100 (****)			0
			222 (2.8)			****
	1994	244	100 (****)			0
			227 (4.9)			****
	1992	174	100 (****)			0
			214 (3.6)			****
AMERICAN INDIAN	1998	151	100 (****)			0
			200 (3.2)			****
	1994	146	100 (****)			0
			200 (3.6)			****
	1992	105	100 (****)			0
			205 (4.9)			****
MISSING  responses were not included in the calculation of percentages for the other categories.
(****)  Standard error estimates cannot be accurately determined.
****  Sample size is insufficient to permit a reliable estimate.
Significantly different from 1994

 

 

NAEP ID: COHORT		TOTAL STUDENT POPULATION
		DEMOGRAPHIC AND DERIVED VARIABLES

---

	YEAR	N	TOTAL			%  MISSING
PUBLIC SCHOOLS
-- TOTAL --	1998	2241	100 (****)			0
			217 (2.1)			****
	1994	2454	100 (****)			0
			212 (1.9)			****
	1992	2571	100 (****)			0
			213 (1.6)			****
RACE/ETHNICITY
WHITE	1998	1080	100 (****)			0
			232 (1.9)			****
	1994	1176	100 (****)			0
			227 (1.7)			****
	1992	1188	100 (****)			0
			224 (2.1)			****
BLACK	1998	356	100 (****)			0
			197 (3.5)			****
	1994	296	100 (****)			0
			191 (4.4)			****
	1992	364	100 (****)			0
			200 (2.5)			****
HISPANIC	1998	718	100 (****)			0
			204 (2.7)			****
	1994	886	100 (****)			0
			198 (1.9)			****
	1992	928	100 (****)			0
			201 (1.8)			****
ASIAN/PACIFIC ISLANDER	1998	55	**** (****)			****
			**** (****)			****
	1994	67	100 (****)			0
			221 (6.6)			****
	1992	47	**** (****)			****
			**** (****)			****
AMERICAN INDIAN	1998	30	**** (****)			****
			**** (****)			****
	1994	27	**** (****)			****
			**** (****)			****
	1992	42	**** (****)			****
			**** (****)			****
MISSING  responses were not included in the calculation of percentages for the other categories.
(****)  Standard error estimates cannot be accurately determined.
****  Sample size is insufficient to permit a reliable estimate.
Significantly different from 1992

Steve Kirsch Political Home Page