NAEP VERSUS NEWT: Winerip in the Bloomberg years!

WEDNESDAY, DECEMBER 21, 2011

EPILOGUE—WHERE WAS THE SO-CALLED PRESS CORPS: As with Paul Krugman, so too with Michael Winerip:

On balance, we don’t like it when these valuable players get snarky. And uh-oh! This Monday, there was a bit of snark in Winerip’s “On Education” column, and in the headline which topped his report in our hard-copy Times:

Headline: “10 Years of State Exams Whose Results Never Need Be Questioned.” A bit of snark could be spotted there, and in Winerip’s report.

In his brutal report, Winerip reviews the horrible history of New York’s statewide public school testing program during the glorious Bloomberg years. We will suggest that you read every word, but here are two of his bookends:
JUNE 2003 Scores on the state algebra test are so poorly calibrated that 70 percent of seniors fail. After a statewide outcry, officials agree to throw out the results. The Princeton Review says that ranking New York first was a mistake. “We’re going to have to come up with a fiasco index for a state like New York that messes up a lot of people’s lives,” a spokesman says.

OCTOBER 2003 A special panel appointed to investigate the state math fiasco concludes that the test “can’t accurately predict performance,” was created “on the cheap” and was full of exam questions that were “poorly worded” and “confusing.”

[...]

JULY 2010 Finally someone—Dr. Tisch, the chancellor of the Board of Regents—has the sense to stand up at a news conference and say that the state test scores are so ridiculously inflated that only a fool would take them seriously, thereby unmasking the mayor, the chancellor and the former state commissioner. State scores are to be scaled down immediately, so that the 68.8 percent English proficiency rate at the start of the news conference becomes a 42.4 proficiency rate by the end of the news conference.
Oof! There’s more, a whole lot more. We’ll suggest you read every word.

Remember, Winerip is reviewing the history of the statewide testing program in the state of New York alone. The horrible bungles he describes do not invalidate all such testing; in particular, the horrible bungles of New York State do not invalidate the work done by the National Assessment of Educational Progress, a federal testing program which is commonly called the gold standard of educational testing. That said, it should be noted that the types of bungles Winerip details have occurred in other statewide programs. In the middle part of the last decade, North Carolina “recalibrated” its statewide testing program too; as in New York, passing rates fell precipitously. Meanwhile, Virginia ran a statewide scam which was uncovered right here at THE HOWLER. When we uncovered this scam, the Washington Post and other Virginia papers politely refused to report this unpleasant story.

Darlings! It just isn’t done!

Almost surely, the NAEP isn’t run in the bungle-laden way described in Winerip’s report. In our view, citizens should be very skeptical about current statewide testing programs; we know of no reason to have similar concerns about the NAEP, although we’d like to see the press corps report about the workings of that vital program. That said, as you read Winerip’s horrible history of the New York State testing program, we will suggest you ask one question:

Where the heck was the New York Times? While these endless bungles occurred, where was our greatest newspaper?

In most of the instances Winerip lists, we can’t answer that question. We can, however, talk about two of the episodes he describes. Here’s the way Winerip recalls events from 2005:
SPRING 2005 New York City fourth graders make record gains on the state English test, with 59 percent scoring as proficient, compared with 49 percent the year before. “Amazing results” that “should put a smile on the face of everybody in the city,” says Mayor Michael R. Bloomberg, who happily recites the numbers on his way to re-election.

FALL 2005 The federal tests (the National Assessment of Educational Progress), which are considered more rigorous than the state tests, show a drop in New York City reading scores. On the eighth-grade test, 19 percent are proficient in 2005, compared with 22 percent in 2003. Asked if city and state officials had hyped the state test results, Merryl H. Tisch, a Regent, says, “They have never, ever, ever exaggerated.”
Where was the New York Times? Editorially, the Times was kissing the keister of its city’s billionaire mayor, and rolling its eyes at city teachers who said the statewide tests were simply getting easier. Gail Collins was in charge at the time, in her pre-Romney’s dog incarnation. To see her fawn to power and sneer at the proles while failing to grasp the shape of the story, see THE DAILY HOWLER, 10/6/05.

Second instance: Where was the New York Times in 2010, when Tisch finally “had the sense to stand up at a news conference and say that the state test scores are so ridiculously inflated that only a fool would take them seriously?” (By the way, good for Tisch!) In our view, the Times was trying very hard to pretend that nothing much had happened. In our view, the New York Post did a much better job reporting this major disaster. See THE DAILY HOWLER, 3/17/11.

Winerip snarked a bit this week; because his work is so valuable, we wish he wouldn’t. For our money, he put this thumb on the scale at times, making it seem that things have gone haywire in the schools of the Empire State. At one point, he used SAT scores as a measure of progress; that’s almost always a bad idea. In our view, he sometimes cherry-picked NAEP scores a tad. Were New York State’s eighth-grade reading scores really “dismal” in 2007? In fact, the state outscored the nation by three points in eighth-grade reading that year; the state's black kids, white kids and low-income kids all outscored their national peers. The state’s NAEP scores have slid a bit in recent years. But that fact deserves a careful look from this uber-valuable journalist, one of the few we have.

Quick review: Our series began a few weeks back, when a pair of events coincided. First, Newt Gingrich made a somewhat bombastic remark about “really poor children in really poor neighborhoods.” At the same time, the NAEP released its new TUDA scores, scores which reflect the academic performance of many of those same deserving children.

The liberal world ranted and stormed about Newt, sometimes failing to grasp what he had actually said. As we stormed, we completely ignored those new NAEP scores, the scores which help reflect the life chances of those deserving kids. (Except for Kevin Drum.)

The liberal world, and the mainstream press, have clowned about those children for years. There’s little sign that anyone cares about these kids; they exist so we can kiss major billionaire keister and/or denounce our rivals as racist. But does anyone give a flying fig about these kids, except to the extent that their lives can be put to to these uses?

The track record is quite poor, for the mainstream press and the liberal world. If you doubt that assessment, read Winerip’s report and ask yourself this:

Where the heck was the New York Times? At one of the mayor’s parties?

9 comments:

  1. Thank you, Bob, for a sad and fascinating discussion of problematic state test design and the flawed reporting of those problems.

    Having been involved some in testing, I would love to see a detailed explanation of how the state tests came to be so flawed. After all, creating and administering tests is a routine chore done by millions of teachers at all levels. So, precisely how did the NY state tests get so screwed up?

    I can imagine various possibilities. Maybe some individual intentionally made the tests easier in order to justify social promotion. Maybe a committee accidentally created an unbalanced test. Maybe some high-level administrator ordered a test-design committee to make the test easier. But, these are just wild guesses as to what really happened.

    There must be lots of people who were involved in cooking the books on various state exams. I wish one of them would give a blow by blow account of the process. I wish some enterprising reporter would seek out such a person and write that blow by blow account.

    ReplyDelete
  2. Is the Merryl Tisch of 2005 the same Dr. Tisch quoted in July, 2010?

    ReplyDelete
  3. Is the Merryl Tisch of 2005 the same Dr. Tisch quoted in July, 2010?

    ReplyDelete
  4. "Where the heck was the New York Times?" Some wags are calling her "The Gray Lazy."

    ReplyDelete
  5. My only interest here is, are some kids in NY learning algebra?

    If so, then their program(s) are not a failure. And traditionally, most Americans can't learn it anyway.

    This may not be a good thing, but it's nothing new, or anything immediately culpable or blameworthy.

    I'd love to see some of these outraged pundits recite the Quadratic Formula-- or use it !

    ReplyDelete
  6. "The United States spends far more on education than any other nation, with paltry results," writes David Brooks, taking the US's "paltry results" for granted. See http://www.nytimes.com/2011/12/27/opinion/brooks-midlife-crisis-economics.html?_r=1

    ReplyDelete