Epilogue—State of the NAEP: Good lord! Over the weekend, a rational discussion almost broke out concerning the nation’s low-income schools!
First, Gene Lyons wrote this piece at Salon. Among other things, he noted that today’s black kids score better on the NAEP math test than their white counterparts did in the not-too-distant past. (Black fourth graders scored higher this year than white fourth graders in 1990.)
This Sunday, in response to Lyons, Kevin Drum offered this post at Mother Jones. He then offered this second, follow-up post. A discussion had almost broken out! Such things simply aren’t done!
Might we offer a point about liberal interest in low-income schools? On Sunday, Drum’s two posts about the NAEP received 17 and nine comments. That same day, his post about left-eared versus right-eared hearing attracted 53 comments; a post about democracy in Europe garnered 36. As has long been clear, the liberal world pays little attention to low-income schools or the kids who attend them. We quit on black kids a long time ago—although we love to bleat and bray about all those conservative racists.
To his vast credit, Drum does pay attention to such topics. We thought we’d offer a few reactions to his twin Sunday posts.
How happy should American be about the state of NAEP scores? We’re not sure, but we do think Kevin tends to tilt toward gloom a tad when he reviews these data. Before explaining that impression, let’s establish a few basic points:
The NAEP runs two separate studies: Just so you’ll know, the National Assessment of Education Progress (the NAEP) conducts two parallel studies—the so-called “Main NAEP” and the “Long-Term Trend” study. At the NAEP’s central site, you can choose which study you want to review.
The “Main NAEP” tests students in reading and math (and some other subjects) in fourth, eighth and twelfth grades. The “Long-Term Trend” study tests 9-year-olds, 13-year-olds and 17-year olds in the same basic subjects.
Which study is more valuable? We have no idea. If education reporters ever chose to report on this “gold standard” testing program, they might even ask NAEP officials and other experts about the rationale for running twin studies.
In the past, we have presented the results from both studies. We know of no particular reason to favor one over the other.
The problem with 17-year-olds: In its own “Main NAEP” reports, the NAEP stresses the scores of fourth- and eighth-graders while downplaying or ignoring the scores of twelfth-graders. We’ve never seen anyone explain this practice. But then, “education reporters” rarely explain anything in our failing culture.
For ourselves, we stay away from the scores of the oldest students in these twin studies; we do so because of the drop-out factor. Consider the Long-Term Trend study. At ages 9 and 13, almost all kids are still in school. But by the age of 17, a lot of kids have dropped out of school—and the NAEP only tests the student population.
But uh-oh! Over the years, the drop-out rate has changed, in ways which are hard to measure. This means that an “apples to oranges” factor may be in play when we compare average scores over time at this age level.
If education reporters ever did real reporting, they might ask NAEP officials and other experts to speak to this general issue. But as far as we know, more kids remain in school at age 17 than was the case in the past. If true, this complicates any attempt to compare average scores over time.
A one-time change in procedure: During the 1990s, the NAEP instituted a basic, one-time change in its testing procedure. It began permitting “accommodations”—allowing “students with disabilities and English language learners” to take the tests under special conditions. Plainly, this creates a bit of an apples-to-oranges problem if we want to compare average scores from before and after the institution of these procedures.
In our own past work on NAEP scores, we have tried to adjust for this one-time change. On the surface, this can make a difference in the size of score gains.
(For the most part, student populations recorded lower average scores when the “students with disabilities and English language learners” were first added into the mix. The NAEP measured this change in the years when this change in procedure was instituted.)
Having made these observations, we would offer these reactions to Drum’s recent posts on this topic. We’ve gained a lot from reading Kevin’s post on education topics over the years. But in our view, he tends to tilt a tad toward gloom when he assesses the NAEP. A few more observations:
The score gains in math count as news: Math and reading are the two basic skills measured by the NAEP. On both NAEP studies, gains in math scores have been larger than gains in reading scores.
But please note: The gains in math scores have tended to range from very large to enormous. If there were no score gains in reading at all, the apparent gains in math skills would be a massive news story. At a time when citizens are constantly told that nothing has worked in the public schools, it’s simply astounding that these score gains have gone unremarked, unreported.
How big are the gains in math scores? On Drum’s second post from Sunday, his graph shows black eighth-graders scoring 2.5 years higher in math in 2011 than their counterparts from 1992. If those apparent gains are real, they represent an astounding success story, even if no gains have been accomplished in reading at all. That said, your nation’s “education reporters” have persistently failed and/or refused to report these score gains, even as they robotically refer to the NAEP as the “gold standard” of testing.
But then, you live inside a hall or mirrors. Your “press corps” is run by chimps.
Here’s where Drum’s tendency toward gloom surfaced in a third recent post, although he isn’t one of the chimps. He did his first report on the latest NAEP scores a few weeks ago (click here). But he only cited the reading scores, completely skipping math.
Over the past ten to twenty years, NAEP math scores seem to indicate very large progress. This is a very large news story. In even a modestly rational world, the public would have been told.
How large are the score gains in reading: How large are the score gains in reading? It depends on which study you use and when you start your comparison. As Drum noted in his second Sunday post, the reading gains look a bit better on the “Main NAEP” than in the “Long-Term Trend” study. In neither case are the score gains in reading as large as the score gains in math.
That said, let’s do the following: Let’s look at the gains in reading on the Main NAEP starting in 1998, the first year “accommodations” were permitted. This permits a clean, apples-to-apples assessment of the two student populations. These are the gains in reading scores recorded by black and Hispanic kids over that 13-year period:
Score gains, NAEP reading tests, 1998-2011You can assess the size of those score gains yourself. But here are the corresponding gains in math scores. In this case, we’ll measure from 1996, the first year accommodations were permitted in math:
Black fourth-graders: 12 points (1.2 years)
Hispanic fourth-graders: 13 points (1.3 years)
Black eighth-graders: 5 points (0.5 years)
Hispanic eighth-graders: 9 points (0.9 years)
Score gains, NAEP math tests, 1996-2011Those score gains are extremely large. It’s astounding that your nation’s “education reporters” haven’t told the public about these apparent gains in math skills—especially at a time when public school teachers are under relentless attack, from the nation's irate billionaires, for their inept performance.
Black fourth-graders: 26 points (2.6 years)
Hispanic fourth-graders: 22 points (2.2 years)
Black eighth-graders: 22 points (2.2 years)
Hispanic eighth-graders: 19 points (1.9 years)
But then, we don’t live in a functioning democracy. We don’t have a functioning intellectual or journalistic culture. We live in a balls-out idiocracy, just as Mike Judge showed us. Significant facts almost never get reported in this idiocratic world.
What do those NAEP scores really mean? Have math skills really improved that much? We don’t know, and there is no chance that the actors hired to pose as “education reporters” will ever try to find out. But one other group must be named at this time:
That would be us liberals.
We liberals must be the most horrible people found on the face of the earth. When it comes to insults to black kids, we’d put Lawrence and Rachel among the worst—but we do seem to function quite well as a group. We love to parade about the land, bragging about our vast racial greatness. But we quit on black kids a long time ago. With the rare exception of people like Drum, you couldn’t make us discuss their interests if you had a big can of orange spray aimed at our grandmothers’ faces.
What have black kids ever done to make us liberals hate them so? We can’t answer that question. But were the shoe on the other tribal foot, R- and B-bombs would fall on the land, causing vast destruction.
Dear lord, how good it would make us feel! How we love to announce our own greatness!