It's so easy, a Texas 4th grader could figure this one out!

How to verify the NAEP data yourself

Suppose you want to figure out how whites in Texas did relative to the rest of the country. You would do the following using white data only from public schools only from the tables below (or directly from NAEP site):

  • Texas gain=(1998 8th grade reading score) - (1994 4th grade reading score) [from the Texas tables]
  • US gain=(1998 8th grade reading score) - (1994 4th grade reading score) [from the US table]
  • Texas improvement=(Texas gain)-(US gain)

So now let's do it using real data for whites which we get from the tables below:

  • Texas gain=273-227=46
  • US gain=270-223=47
  • Texas improvement=46-47= -1

So in other words, Texas did not improve as much as the US as a whole (that's the part that Bush says Al Gore was responsible for).

The calculation for other groups is just as easy.

You can load the data yourself from the NAEP site, or  use the data below which is copied from those pages.

If you want to load the data yourself, go to:
http://nces.ed.gov/nationsreportcard/TABLES/index.shtml

The tables to load to verify Table 2 of Klein's paper are:

  • 1998, Reading, 8 [continue], National-Public, Student Data [Begin Search], Select "Total Student Population" link
  • 1998, Reading, 4 [continue], National-Public, Student Data [Begin Search], Select "Total Student Population" link
  • 1998, Reading, 8 [continue], Texas, Student Data [Begin Search], Select "Total Student Population" link
  • 1994, Reading, 4 [continue], Texas, Student Data [Begin Search], Select "Total Student Population" link

Note: No "fuzzy math" is required! A Texas 8th grader could do the calculation! You don't have to do any normalization if you look at each racial group separately. You just extract the 12 numbers from the publicly available 4 tables (as noted below), and subtract the scores (see Table 2 in Klein's paper). If Grissmer had interpreted data only from 1994 onwards, he would have made exactly the same conclusion. The data is unambiguous. No fuzzy math. No assumptions. It shows clearly that there is nothing special going on in Texas. All external independent external evidence shows that to be the case! 

Notes:

  •  The argument about using the earlier study to discredit the new study is also somewhat of a red-herring. The “Texas Miracle” was based on state TAAS scores. The earlier study didn’t use TAAS scores.
  •  The argument that Texas NAEP scores are above the US average especially for Blacks and Latinos is also bogus. It's been that way for a long time (for at least a decade).

Bottom line:

  • The only independent data available (the same data source used by Grissmer, who later added the 1998 data but didn't publish it) shows nothing special going on in Texas relative to the rest of the country
  • Klein's paper shows that the state TAAS scores are not trustworthy, the achievement gap widened under Bush, and that there was no improvement in academic performance

THE NEW NEWS IS THIS:

  • There is now ZERO independent trusted data to back Bush's claim.
  • Everything we see which is trusted independently (SAT scores, etc.) shows ZERO to decline.

The only NAEP data you can use for the comparison if you want to look at 1994 onwards is 1994 4th grade Reading vs. 1998 8th grade reading. Other datasets are not available because you can't get the right pairs, e.g., you can't get 8th grade 1994 reading and 12th grade 1998 reading, you can't get 4th grade math in 1994 vs. 4th grade math in 1998, etc.

Questions? Contact me: stk@propel.com

Here is the data, collected for your convenience (I've highlighted the 12 numbers you want are in Blue/Bold). They exactly match the data in Table 2.

NAEP 1998 National Public-Only Reading Summary Data for Grade 8 Students
Weighted Percentages and Average Composite Scale Scores (Standard Errors in Parentheses)

 

 
NAEP ID: COHORT   TOTAL STUDENT POPULATION
  DEMOGRAPHIC AND DERIVED VARIABLES

N TOTAL     %  MISSING
 
PUBLIC SCHOOLS
-- TOTAL --   9091   100 (****)    0
      261 (0.8)    ****
 
RACE/ETHNICITY
WHITE   5013   100 (****)    0
      270 (0.9)    ****
BLACK   1565   100 (****)    0
      241 (1.6)    ****
HISPANIC   1934   100 (****)    0
      243 (2.1)    ****
ASIAN/PACIFIC ISLANDER   456   100 (****)    0
      269 (4.0)    ****
AMERICAN INDIAN   106   100 (****)    0
      248 (4.8)  Variability of this statistic cannot be accurately determined  ****
MISSING  responses were not included in the calculation of percentages for the other categories.
(****)  Standard error estimates cannot be accurately determined.
****  Sample size is insufficient to permit a reliable estimate.
Variability of this statistic cannot be accurately determined  The nature of the sample does not allow accurate determination of the variability of the statistic.
NAEP 1998, 1994, and 1992 National Public-Only Reading Summary Data for Grade 4 Students
Weighted Percentages and Average Composite Scale Scores (Standard Errors in Parentheses)

 

 
NAEP ID: COHORT   TOTAL STUDENT POPULATION
  DEMOGRAPHIC AND DERIVED VARIABLES

YEAR N TOTAL     %  MISSING
 
PUBLIC SCHOOLS
-- TOTAL --   1998  6300   100 (****)    0
        215 (0.8)  Significantly different from 1994  ****
   1994  6030   100 (****)    0
        212 (1.1)    ****
   1992  5045   100 (****)    0
        215 (1.0)    ****
 
RACE/ETHNICITY
WHITE   1998  3314   100 (****)    0
        225 (0.9)    ****
   1994  3419   100 (****)    0
        223 (1.3)    ****
   1992  2897   100 (****)    0
        223 (1.3)    ****
BLACK   1998  1162   100 (****)    0
        193 (1.8)  Significantly different from 1994  ****
   1994  1175   100 (****)    0
        186 (1.7)    ****
   1992  957   100 (****)    0
        192 (1.6)    ****
HISPANIC   1998  1455   100 (****)    0
        195 (1.9)    ****
   1994  995   100 (****)    0
        188 (2.7)    ****
   1992  897   100 (****)    0
        199 (2.2)    ****
ASIAN/PACIFIC ISLANDER   1998  206   100 (****)    0
        222 (2.8)    ****
   1994  244   100 (****)    0
        227 (4.9)    ****
   1992  174   100 (****)    0
        214 (3.6)    ****
AMERICAN INDIAN   1998  151   100 (****)    0
        200 (3.2)    ****
   1994  146   100 (****)    0
        200 (3.6)    ****
   1992  105   100 (****)    0
        205 (4.9)    ****
MISSING  responses were not included in the calculation of percentages for the other categories.
(****)  Standard error estimates cannot be accurately determined.
****  Sample size is insufficient to permit a reliable estimate.
Significantly different from 1994  Significantly different from 1994
NAEP 1998 Texas Reading Summary Data Tables for Grade 8 Student Data
Weighted Percentages and Average Composite Scale Scores (Standard Errors in Parentheses)

 

 
NAEP ID: COHORT   TOTAL STUDENT POPULATION
  DEMOGRAPHIC AND DERIVED VARIABLES

N TOTAL     %  MISSING
 
PUBLIC SCHOOLS
-- TOTAL --   2318   100 (****)    0
      262 (1.5)    ****
 
RACE/ETHNICITY
WHITE   1106   100 (****)    0
      273 (1.6)    ****
BLACK   293   100 (****)    0
      245 (3.1)    ****
HISPANIC   805   100 (****)    0
      252 (2.1)    ****
ASIAN/PACIFIC ISLANDER   89   100 (****)    0
      277 (4.2)    ****
AMERICAN INDIAN   24   **** (****)    ****
      **** (****)    ****
MISSING  responses were not included in the calculation of percentages for the other categories.
(****)  Standard error estimates cannot be accurately determined.
****  Sample size is insufficient to permit a reliable estimate.
NAEP 1998, 1994, and 1992 Texas Reading Summary Data Tables for Grade 4 Student Data
Weighted Percentages and Average Composite Scale Scores (Standard Errors in Parentheses)

 

 
NAEP ID: COHORT   TOTAL STUDENT POPULATION
  DEMOGRAPHIC AND DERIVED VARIABLES

YEAR N TOTAL     %  MISSING
 
PUBLIC SCHOOLS
-- TOTAL --   1998  2241   100 (****)    0
        217 (2.1)    ****
   1994  2454   100 (****)    0
        212 (1.9)    ****
   1992  2571   100 (****)    0
        213 (1.6)    ****
 
RACE/ETHNICITY
WHITE   1998  1080   100 (****)    0
        232 (1.9)  Significantly different from 1992  ****
   1994  1176   100 (****)    0
        227 (1.7)    ****
   1992  1188   100 (****)    0
        224 (2.1)    ****
BLACK   1998  356   100 (****)    0
        197 (3.5)    ****
   1994  296   100 (****)    0
        191 (4.4)    ****
   1992  364   100 (****)    0
        200 (2.5)    ****
HISPANIC   1998  718   100 (****)    0
        204 (2.7)    ****
   1994  886   100 (****)    0
        198 (1.9)    ****
   1992  928   100 (****)    0
        201 (1.8)    ****
ASIAN/PACIFIC ISLANDER   1998  55   **** (****)    ****
        **** (****)    ****
   1994  67   100 (****)    0
        221 (6.6)    ****
   1992  47   **** (****)    ****
        **** (****)    ****
AMERICAN INDIAN   1998  30   **** (****)    ****
        **** (****)    ****
   1994  27   **** (****)    ****
        **** (****)    ****
   1992  42   **** (****)    ****
        **** (****)    ****
MISSING  responses were not included in the calculation of percentages for the other categories.
(****)  Standard error estimates cannot be accurately determined.
****  Sample size is insufficient to permit a reliable estimate.
Significantly different from 1992  Significantly different from 1992

Steve Kirsch Political Home Page