Ideology, Rather Than Science, Continues to Control NAEP

In the important Substack commentary linked below, Paul Thomas offers an update on the 40 year-old battle to return NAEP (National Assessment of Educational Progress) testing to the status of a useful and scientifically-grounded assessment tool, rather than the high-stakes political bludgeon created by Chester Finn and his band of renegade right-wingers (including Diane Ravitch) to pummel public schools in the U.S. for failure to reach out-of-reach proficiency levels that were instituted to regularly demonstrate the failure of public education. 

In a 2008 article in The School Administrator, Gerald Bracey provided this snapshot history of the unresolved controversy: 

The [NAEP] governing board hired a team of three well-known evaluators and psychometricians to evaluate the process — Daniel Stufflebeam of Western Michigan University, Richard Jaeger of the University of North Carolina at Greensboro and Michael Scriven of NOVA Southeastern University. The team delivered its final report on Aug. 23, 1991. This process does not work, the team averred, saying: "[T]he technical difficulties are extremely serious … these standards and the results obtained from them should under no circumstances be used as a baseline or benchmark … the procedures used in the exercise should under no circumstances be used as a model.” 

NAGB, led by Chester E. Finn Jr., summarily fired the team, or at least tried to. Because the researchers already had delivered the final report, the contract required payment. 

The inappropriate use of these levels continues today. The achievement levels have been rejected by the Government Accountability Office, the National Academy of Sciences, the National Academy of Education, the Center for Research in Evaluation, Student Standards and Testing and the Brookings Institution, as well as by individual psychometricians. 

 I have repeatedly observed that the NAEP results do not mesh with those from international comparisons. In the 1995 Trends in International Mathematics and Science Study, or TIMSS, assessment, American 4th graders finished third among 26 participating nations in science, but the NAEP science results from the same year stated that only 31 percent of them were proficient or better. 

 As you will see from the piece below, little has changed in the ongoing war except that minority rule by fascist bastards has become more intense.

NAEP 2026: The Politics of Testing by Paul Thomas

"Despite the best of intentions, however, in practice, reading assessments sometimes negatively impact students and their learning opportunities." Forzani, et al. (2022)

Read on Substack

Nationalize, Stigmatize, Penalize: Corporate Education's Recurring Fever Dream Recast

by Peter Green:

The Failed Case for Super-NAEPery by Peter Greene

Read on Substack

Gov. Bill Lee to Sacrifice Third Graders to Help Him in 2024?

Revised January 24, 2020

Prior to this week, the last time a Tennessee governor brought the General Assembly into Special Session to pass education legislation was January 2010, and it was all about getting more federal money to pay for the schools that state politicians are unwilling to support.  

Responding to call by Gov. Phil Bredesen (D), the TN legislature quickly met in January 2010 and passed into state law the necessary policies that would make the state a shoo-in for a half-billion dollar federal grant to be provided by Obama's Race to to the Top (RttT).  A core component of Tennessee's half-baked program was to use the money for a test score based teacher evaluation scheme, which added to Tennessee's reputation as an educational laughingstock.

Just this week, Governor Bill Lee (R), has ridden herd over another special session on education, this one ostensibly to address 1) what educational snake oil salesmen call the Covid-related "learning loss," and 2) the reading proficiency emergency at Tennessee's elementary schools.

Since "learning loss" scam is covered in this recent post, let me focus for a moment on Bill Lee's other manufactured crisis, reading literacy. 

Less than two years ago, Lee's Commissioner of Education, Penny Schwinn was providing this spin on recent NAEP results for Chalkbeat:

Tennessee jumped from 34th to 25th in fourth-grade math and moved up three spots to 31st in fourth-grade reading, according to calculations provided by the state education department. For eighth-graders, Tennessee now ranks 30th in both reading and math, up from 38th and 35th, respectively.

Legislation introduced this week by Lee's team adopts an entirely different spin on reading proficiency by TN students:

(3) In 2019, Tennessee's third grade English language arts proficiency rate was thirty-six and nine-tenths percent (36.9%);

(4) In 2019, Tennessee's eighth grade English language arts proficiency rate was twenty-seven and one-tenth percent (27.1%); and

(5) In 2019, Tennessee ranked thirty-first in the nation in fourth grade reading proficiency and thirtieth in eighth grade reading proficiency.

A couple of things are notable about these claims.  First, there are no third grade proficiency rates provided by NAEP.  Grade 4 rates, check.  Grade 8 rates, check.  No grade 3.  

Second, the mendacious and misleading NAEP proficiency rates cited by Lee's folks are simply wrong: 4th grade proficiency in 2019, according to NAEP, was 35%, and 8th grade proficiency was 32%.  

The reason that Lee's henchmen want to mislead the public with manufactured scores for 3rd graders is explained here by Amy Frogge:

. . . here’s the biggest concern about the “learning loss” bill: It will require districts to hold back third graders who are not deemed “proficient” in standardized testing. (Proficiency rates can be manipulated by the state through cut scores.) If you google the term “Mississippi miracle,” you will find that Mississippi used this very same trick to create the appearance of a sudden increase on NAEP test scores. Holding back low-performing third graders creates the illusion of huge one-time testing gains, and implementation of the bill would take place just in time for the 2023 NAEP tests. This is not about best serving the children of Tennessee; it’s about gaming the system. Furthermore, the costs for holding back large numbers of third graders, as mandated by this bill, would be astronomical.

It's not as if we don't know the harmful effects of grade retention on children.  We do.  Even so, Lee up for, um, a presidential bid? in 2024, and he has consciously decided to sacrifice the welfare of Tennessee's poorest children, who inevitably will comprise the vast majority of 3rd grade failures, for the PR boost that he hopes voters will attribute to his miraculous education interventions, just passed.

Wishing Away Goliath, Part 2 w/updates

Jim Horn
Part 1 can be found here. 

Updated May 3, 2020

Just over ten years ago when Diane Ravitch signaled that she was about to leave the corporate ed reform ship and walk ashore as an apostle of the Resistance, I and many others were hoping to read The Death and Life of the Great American School System as a roadmap to the inner workings of the corrupt education reform establishment.  Finally, I thought, if the world was ever to know the full story of the seminal events that had misshaped, corrupted, and, otherwise, damaged public education over the three prior decades, an insider-backslider like Diane Ravitch might be the one who could spill the beans.  After all, as an historian, wouldn't she want to set the record straight as part of her redemption plan?

What was the inside scoop, for instance, on the manipulations of NAEP cut scores to make American students appear underprepared and their schools grossly negligent in preparing them?  Diane was on the scene when those cut scores were set in stone by Checker Finn, who served as President of NAEP's Governing Board in 1989:

In an article titled "A Test Everyone Will Fail," the late Gerald Bracey (2007/2011) wrote this about the manipulated NAEP cut scores and why nothing has ever been done since 1989 to make them realistic metrics of American student achievement:

. . . these numbers are useful as scare techniques and bludgeons. If you can batter people into believing that the schools are in awful shape, you can make them anxious about their future and you can control them. In the 1980s the schools-suck-bloc used such numbers to make us fearful that Japan, now emerging from a 15-year-long recession-stagnation was going to take away all of our markets; today India and China play the role of economic ogres.

Diane served on the National Assessment Governing Board, which oversees the National Assessment of Educational Progress (NAEP), from 1997 to 2004.  During her tenure, she did nothing to change the cut scores, which the National Academy of Science described in 1993 as "fundamentally flawed."  

In fact, at a Hoover Institution press briefing in 2001, Koret Task Force member, Diane Ravitch, bragged:

The National Assessment of Educational Progress has gotten, I think, much stronger. When Checker [Finn] was chairman of the NAGBE Board, the governing board, they created real standards so that now we are able to report periodically to the nation about where American kids are in terms of learning different important subject areas in their school studies.

And to this day, Diane has said nothing in any of her three books written since 2010 about this ongoing scandal. The same scandalously high proficiency levels remain scandalously high and "fundamentally flawed," and they are the same unreachable proficiency targets that "reformers" use to hammer schools whenever new NAEP results are published.

And so it quickly became clear even in my initial reading of The Death and Life . . . that Ravitch was much less interested in blowing the whistle than preaching a message of redemption to her former allies.  Rather than sharing the facts about the NAEP scandal or the two other prominent scandals that will be the focus of Parts 3 and 4, Diane has offered the education reform establishment homilies, tsk-tsks, and finger wagging. 

Meanwhile, the real history American education policy's greatest manipulations remains untold by someone with all the facts to tell it.  To this day, Diane regularly presents the education reform establishment as simply misguided and misinformed individuals and organizations with good intentions, rather than as an ideologically and cash-driven movement to monetize and privatize public institutions.  

Again, NAEP Shows Failure of Testing Reforms

The new NAEP scores are just out, and, per always, the hand-wringing has already begun.  Compared to 2017, scores disappoint, especially for reading.  

Below are charts showing the trajectory of scores over time for 4th and 8th grade students in Math and Reading.  Below the charts are a few observations.

When we look at these NAEP scores over time, one thing that stands out is the general tendency for scores to improve, however gradually.  When you look at the NAEP scores beginning in the early 70s to early 19990s through today, that trend is consistent--a gradual improvement over time, with bursts of punctuated improvement or decline.

A second aspect that stands out is that the rate that NAEP scores increased was greater before testing accountability became mandated by federal law than it was after.  From 1990 to 2002, before NCLB's testing accountability regime was passed or took hold, the rate of NAEP score growth for almost all grade levels and percentile rankings was higher than it was after 2002, when the NCLB testing madness began in earnest.

Also note that the longer that NCLB remained in effect, NAEP score growth slowed or even declined for some categories.  

NAEP score growth during the first 7 years of NCLB was greater than the last seven years of NCLB.

And when we look at the struggling students (10th and 25th percentiles) who were specifically targeted by NCLB testing accountability reforms, we see even more graphic examples of NCLB ineffectiveness in raising academic performance as measured by test scores.  

Without exception, 10th and 25th percentile students in 4th and 8th grade Math and Reading show declining NAEP scores from 2009 to 2019.  In fact, scores for 8th grade Reading students in the 10th and 25th percentiles actually declined over the past 20 years, from 1999 to 2019.  

We cannot forget that these sharp declines in Reading accompanied a growing focus on direct instruction and phonics as the officially sanctioned approach to reading.  We should not forget, either, the decline of school libraries in public schools during the past 20s and the near total absence of libraries and, certainly, of librarians in the 7,000+ charter chain gang schools across the U. S.

Get Ready for Betsy's Propaganda on New NAEP Results

It's been thirty years or so since conservative thought leaders like Checker Finn and Diane Ravitch came up with the brilliant ploy to misuse NAEP test results from the "Nation's report card" as an ongoing cudgel against public schools.  The simple and elegant plan was to make NAEP's "Proficiency" cut scores unattainable by the majority of American students.  

By raising proficiency targets to the unattainable level, the scores would show most student not meeting "proficiency"standards.  This news, repeated every few years when new NAEP scores were published, could be used to demonstrate the failure of American schools/teachers and, thus, demonstrate the need for more and more testing accountability standards and assessments, along with the need for corporate and mayoral steering of schools.

Gerald Bracey and every other testing experts understood the scam, and Bracey had the courage to write about it and talk about it and try to make the public aware.  To no avail.  

The story of American public school failure became the ascendant meme, despite scientific evidence to the contrary (read Bracey's story of the suppressed Sandia Report, which was quashed by the U. S. Department of Education).

The fact that NAEP's fanciful cut scores remain unaltered today almost three decades later is a strong testament to the power of the U.S. Chamber of Commerce and the corporate education industry to shape policy.

And that is why this press release is now necessary (now more than ever) for all to read and to share with school boards, parents, teachers, and media folks.

NAEP TERM “PROFICIENT” IS MISLEADING

STATEMENT OF JAMES HARVEY

EXECUTIVE DIRECTOR

NATIONAL SUPERINTENDENTS ROUNDTABLE

FOR IMMEDIATE RELEASE

SEATTLE, April 9, 2018 – As the U.S Department of Education prepares to release the latest findings from the National Assessment of Educational Progress (NAEP), the American people should understand that the misleading term “proficient” sets a performance benchmark beyond the reach of most students in the world.

A detailed analysis released in January concluded that the vast majority of students in most countries could not demonstrate proficiency as NAEP defines the term.

The authors of the analysis, the National Superintendents Roundtable and the Horace Mann League, linked NAEP’s proficiency benchmark to the performance of students around the world on international assessments such as TIMSS (Trends in International Mathematics and Science Study).

The report on this work (How High the Bar?) concluded that:

• In no nation do even 40 percent of students meet the NAEP Proficient benchmark in Grade 4 reading.
• Only one nation has 50 percent or more of its students meeting the Proficient benchmark in Grade 8 science (Singapore).
• Just three nations have 50 percent or more of their students meeting the Proficient benchmark in Grade 8 math (Singapore, Republic of Korea, and Japan). 

Citing U.S. Department of Education documents, the report criticized the Department for misusing the term "Proficient." The term, as the Department acknowledges, does not mean performing at grade level. Surprisingly, according to the Department’s statements, it does not even mean proficient, as most people understand the term.

Roundtable and Horace Mann League officials have insisted that the problem can be addressed without lowering standards by changing the term “proficient” to “high.” Without such a change, they maintain, the misuse of the term will continue to confuse both the public and educators, as in the past it has confused U.S. Secretaries of Education.


CONTACT: JAMES HARVEY: Office (206) 526-5336
                                                  Cell    (206) 579-9272

******************************************

National Superintendents Roundtable
9425 35th Avenue, NE, Suite E
Seattle, WA 98115

After 25 Years of Education Reform, NAEP Results Down

Since Ronald Reagan began the war on public education, school has become an increasingly unpleasant and miseducative place to learn and to work.  With the coming of corporatists' NCLB and RTTT, children have learned less and suffered more, humane teachers have been run out of the profession, and we now have corporate know-nothings in charge of schools where learning needs are the most severe.

Is there anything to recommend another 30 years of testing, segregation, and corporate control?  Please let me know if there is.

Here is a useful overview of the latest NAEP results for 12th graders.  Links are provided in the video:

NYTimes Economics Columnist w/Physics Degree Has Ed Solution

by Jim Horn

As a veteran educator of educators, I am accustomed to hearing how much people, regardless of education, training, or native wit, believe they know about education and school policy, especially when they find out how I make a living. 

"History and philosophy of education, huh?  Well, let me tell you why schools are so bad. . ."

And listening to them is easier than explaining how history or philosophy may be relevant to policy talk, implementation, and practice, if the powerful elites who make policy were not committed to remaking the same self-serving system that the protectors of the social status quo have been imposing for 150 years at least.  

Today's businessmen who run the paternalistic and segregated schools for the disadvantaged are no less enthralled by rote learning and soul-crushing discipline systems than were the Latin grammar school teachers of the 19th Century, who cried out in protest when Horace Mann suggested "moral suasion" as an alternative to beating schoolchildren into submission.

Nor are today's edupreneurs any less excited about the use of racist "achievement" testing than were the social efficiency zealots like Stanford U.'s Elwood Cubberley a hundred years ago, who declared in the foreword of Louis Terman's 1914 classic, The Measure of Intelligence,

Questions relating to the choice of studies, vocational guidance, schoolroom procedure, the grading of pupils, promotional schemes, the study of the retardation of children in the schools, juvenile delinquency, and the proper handling of subnormals on the one hand and gifted children on the other— all alike acquire new meaning and significance when viewed in the light of the measurement of intelligence as outlined in this volume. . . .More than all other forms of data combined, such tests give the necessary information from which a pupil’s possibilities of future mental growth can be foretold, and upon which his further education can be most profitably directed. 

Take, for instance, the ignorant remarks by modern day efficiency "expert," economics reporter, and former NYTimes editorial board member, Eduardo Porter.  Porter has taken Cubberley's fascination with data one step further, to claim that raising test scores, in fact, is the best indicator that inequality has been, or is soon to be, eradicated:

Narrowing proficiency gaps that emerge way before college would probably do more to increase the nation’s college graduation rate than offering universal community college, easier terms on student loans or more financial aid.

How does Eduardo know this?  Can't say, since he does not offer any evidence to support his outrageous claim.  

Nor does Eduardo, whose degree is in quantum physics, say anything about new evidence that points to more and more underemployment for those who do graduate from college:

The dire experience of these workers with college degrees displacing workers with less formal education stands in strong contrast to the widely-held view in economic and policymaking circles that the main problem facing the U.S. economy is a shortage of highly-educated workers. If college-educated workers were in short supply, then we would expect their wages to rise as employers attempted to lure them away from their competitors. Yet the inflation-adjusted value of the wages of college-educated workers has barely increased in the 21st century.

What’s more, between 2000 and 2014 (the last year for which complete data are available), the employment of college-educated workers has increased much more rapidly in low-earning industries than in high-earnings ones. If there weren’t enough college graduates to go around, then the opposite should be happening because high-earnings industries would presumably be outcompeting low-earnings industries to hire college-educated workers. Our new analysis of the data from the U.S. Census Bureau’s Quarterly Workforce Indicators strongly suggests that college-educated workers are more likely to “filter down” the job ladder than to climb it.

Eduardo does offer some data, however, that suggests that our past three decades of fixation on testing proficiency while forgetting about rising resegregation contributes nothing positive for either test scores or diversity in schools.  

Extrapolating from Sean Reardon's paper on NAEP trends, Eduardo claims the black-white achievement gap has dropped by "as much as 50 percent compared with what it was 30 to 40 years before." 

What we see in NAEP trends, however, is that the test score gaps were lowest when school desegregation was at its highest (click charts below), a correlation that is entirely lost on the middle class neolibs at the New York Times who require education fixes that don't work in order to maintain and grow the class divisions that provide the unscalable wall between Eduardo and those he presumes to save by raising their test scores.

Those Who Cannot Remember the Past

 The past is never dead; it's not even past.--Wm. Faulkner

Yesterday Diane Ravitch gave space for Peter Greene and Catherine Gewertz to have their say on Common Core proficiency targets, which have been arbitrarily adjusted to match the impossibly high proficiency targets of NAEP.  

I am wondering why Ravitch, instead of remaining mum on her intricate knowledge of the history of cut scores for big testing, does not explain it for us, as she began to do in a review of Waiting for Superman in 2010:

I served as a member of the governing board for the national tests for seven years, and I know how misleading Guggenheim’s figures are. NAEP doesn’t measure performance in terms of grade-level achievement. The highest level of performance, “advanced,” is equivalent to an A+, representing the highest possible academic performance. The next level, “proficient,” is equivalent to an A or a very strong B. The next level is “basic,” which probably translates into a C grade. The film assumes that any student below proficient is “below grade level.” But it would be far more fitting to worry about students who are “below basic,” who are 25 percent of the national sample, not 70 percent.. . .

She might explain, too, I guess, why she sat on her, um, Board, for seven years and did nothing to help realign the cut scores toward reality.   Oh, I almost forgot, she was on the Checker Finn team then, trying to make sure that public school students look as stupid as possible in order to bash teachers and to enthuse the public for privatization solutions in the pre-charter days.

In 2013, Ravitch said this in defense of her old pal: "The [NAEP] achievement levels were created when Checker Finn was chair of NAGB. I think they are defensible if people understand that the achievement levels do not represent grade levels." 

Oh, why would anyone think "proficient" and "below basic" had anything to do with grade level for 4th, 8th, and 11th grade, where the NAEP tests are given?

Here's a part of a post to ARN by Jerry Bracey in 2009, which offers some more detail on the impossible NAEP cut scores.  Please follow the links:

As I have shown over and over, the NAEP achievement levels are ridiculous (see for example, "Oh, those NAEP achievement levels" in the September 2005 Principal Leadership or "A test everyone will fail" in the May, 3, 2007 Washington Post, also at www.huffingtonpost.com/gerald-bracey).

This is not the paranoid position of a lone crank. I am in the company of the National Academy of Sciences, the National Academy of Education, the Government Accounting Office and the Center for Research in Evaluation, Standards and Student Testing as well as individual psychometricians. Before the Bush administration declared war on science, the NAEP reports themselves carried disclaimers including quotes from the National Academy of Sciences like this: "NAEP's current achievment setting procedures remain fundamentally flawed. The judgment tasks are difficult and confusing; raters' judgments of different item types are internally inconsistent, appropriate validity for the cut scores is lacking; and the process has produced unreasonable results."

VALIDITY lacking? UNREASONABLE results? Fundamentally flawed? Sounds worse to me than credit default swaps. Why didn't IES at USDOE ban them as not being derived from "scientifically based research?" (It actually might be a good thing to make Reid Lyon head of the National Assessment Governing Board, the agency produced these monstrosities). The achievement levels were supposed to be used only until something better could be produced. That was almost 20 years ago. But, of course, no one is working on anything better. Why? Because the results from the current procedures build such wonderful cudgels to bash schools with.

Among the "unreasonable results" are that American students scored high on some international tests of reading, math, and science, but only a small percentage ever reaches the "proficient" level on NAEP In the most recent NAEP reading assessment, only 33% of 4th graders were deemed proficient or better. Richard Rothstein and colleagues, in a perfectly straightforward procedure estimated that the highest scoring nation in one reading assessment, Sweden, would have about 33% of its kids proficient in reading if Swedish kids had to sit for NAEP (at the time of Rothstein's study, 31% of American kids reached this level).

So states will or will not become eligible for stimulus money based on invalid criteria. This is not change we can believe in. But don't look for them to change soon. In his speech to the Hispanic Chamber of Commerce, President Obama said "just a third of our 13- and 14-year-olds can read as well as they should." For extra credit, guess where that statistic came from.

There is so much Diane could share with her fans and critics if she were less preoccupied with providing Randi and Lily a venue for proposing their own high stakes testing solutions to the problem of high stakes testing.  No doubt Diane's revisionism will have to wait.  In the meantime, we have to look at how we got this point from those who refused to ignore or forget the past.

by Jerry Bracey at Huffington Post:

Looking at Diane Ravitch's 1991 article, "U. S. Schools: The Bad News is Right," and her contemporary defense on the Huffington Post of her actions concerning the Sandia Report, it is clear that Ravitch missed her true calling: conservative talk show host. In delivering self-serving distortions, simplistic generalizations, and outright lies, she can out-Rush Rush and out-Bill Bill. She lacks the ball-biting blood lust of Ann, but maybe that would have come in time.

She writes that the report claimed "that the scores really were not falling." That it said "Just take out the black and Hispanic and inner-city kids, they suggest, and the picture was rosy." What horse hockey. The report showed (page 273) that NAEP math scores had been steady for whites and rising for blacks and Hispanics although both groups trailed whites substantially. An analysis by community type (rural, advantaged metro, disadvantaged metro) showed NAEP performance steady or rising.

We're talking NAEP here, not some dinky test the Sandia engineers made up.

The report noted that blacks and Hispanics were becoming an ever larger proportion of the overall population. The engineers wrote, "these data reinforce the dropout findings indicating that the performance of racial/ethnic minorities continues to lag behind the performance of whites." They said nothing about "taking out" the black and Hispanic scores. These are engineers for God's sake.

Systems engineers, in fact. Engineers, more than any other professional group in the world would know that you don't ignore a substantial part of the system you're working with.

"If anything, today's students are performing better than previous students. We are not implying that these performance levels are adequate for today's or tomorrow's society or that these levels are comparable to those in other countries." Does that sound like painting a rosy picture to you?

As for the suppression, she writes, "David Kearns knew, as did all the senators in that room, that there is no way that a report, an opinion, an essay, or any other expression of one's views can be suppressed." Lord, lord, Diane, have you learned nothing from the seven years of thought control by the Bush administration? If it wasn't suppressed, why does former Sandia vice president, Lee Bray, insist it was?

"The report was not published by the government, but it was published and received far broader attention and a far larger audience than most of the reports published by the government." What chutzpah! Maybe most reports, but not reports about public schools. She then goes on to discuss the PISA results, which if what she said about government reports were true, she wouldn't even know about. But PISA results, along with PIRLS results were published by virtually every newspaper in the country. The publication of the Sandia Report in the Journal of Educational Research received zero media attention.

Actually, the Sandia Report was first published a few months before it appeared in the Journal of Educational Research as an appendix to a book Exploding the Myths by Paul Houston, then superintendent in Riverside, CA and Joe Schneider, then with the Southwest Regional Lab. Joe had presented the report at a SWRL board meeting and Paul invited him to present it at back-to-school meeting for administrators and parent leaders.

Within hours of sending a note advising people of the opportunity to see a government-suppressed report, Houston received a call from Peggy Dufour, Secretary Watkins' chief education adviser blasting him for presenting it and saying he had lied about it by calling it suppressed. Houston asked when the report would be released. Dufour said "we haven't and we aren't going to let it be released."

Houston suggested she check the definition of "suppress." She called Schneider and implied that SWRL risked losing some of its federal funding if he made the presentation. Exploding the Myths is not the kind of book Ravitch reads.

Ravitch claims I have "achieved certain notoriety for his insistence over many years that American schools are better than ever and that anyone who dares to criticize them is wrong, misguided, and/or part of an evil cabal to destroy public education." This sentence clearly proves one thing: Ms. Ravitch-Limbaugh-O'Reilly hasn't read a word I've written.

NAEP, Common Core, and the Continued Production of Failure

For those who never knew Gerald Bracey's work, all I can say is you have a lot of catching up to do.  It was his writing in the Kappan in the 1990s that caught my attention, and then came his books.  

He was a master at using his steel trap logic and his delightfully sharp pen to pop all the gas bags floating over the education policy centers of Washington, DC.  

One of his favorite subjects was NAEP and the ridiculous cut scores that were developed and maintained under the watchful eye of people like Chester Finn and his pal, Diane Ravitch (prior to her conversion to corporate unionism).  

Here is a clip from a piece that is available online from 2008, where he offers a brief yet enlightening history of the NAEP cut scores, which have been used ever since the late 80s by corporate ed reformers to remind us all how horrible our schools are.  

In 1988, though, Congress created the National Assessment Governing Board and charged it with establishing standards. NAEP now became prescriptive, reporting not only what people did know but also laying claim to what they should know. The attempt to establish achievement levels in terms of the proportion of students at the basic, proficient and advanced levels failed.

The governing board hired a team of three well-known evaluators and psychometricians to evaluate the process — Daniel Stufflebeam of Western Michigan University, Richard Jaeger of the University of North Carolina at Greensboro and Michael Scriven of NOVA Southeastern University. The team delivered its final report on Aug. 23, 1991. This process does not work, the team averred, saying: “[T]he technical difficulties are extremely serious … these standards and the results obtained from them should under no circumstances be used as a baseline or benchmark … the procedures used in the exercise should under no circumstances be used as a model.”

NAGB, led by Chester E. Finn Jr., summarily fired the team, or at least tried to. Because the researchers already had delivered the final report, the contract required payment.

Flawed Uses
The inappropriate use of these levels continues today. The achievement levels have been rejected by the Government Accountability Office, the National Academy of Sciences, the National Academy of Education, the Center for Research in Evaluation, Student Standards and Testing and the Brookings Institution, as well as by individual psychometricians.

I have repeatedly observed that the NAEP results do not mesh with those from international comparisons. In the 1995 Trends in International Mathematics and Science Study, or TIMSS, assessment, American 4th graders finished third among 26 participating nations in science, but the NAEP science results from the same year stated that only 31 percent of them were proficient or better.

The National Academy of Sciences put it this way: “NAEP’s current achievement-setting procedures remain fundamentally flawed. The judgment tasks are difficult and confusing; raters’ judgments of different item types are internally inconsistent; appropriate validity evidence for the cut scores is lacking; and the process has produced unreasonable results.”

The academy recommended use of the levels on a “developmental” basis (whatever that means) until something better could be developed. In 1996, the National Academy of Education recommended the current achievement levels “be abandoned by the end of the century and replaced by new standards … .”

Continuing Mischief
Here we are almost a decade into a new century and the old standards remain, causing a great deal of mischief every time a new NAEP assessment is released to the news media. No one is working to create new standards. Why? The use of the NAEP standards fits into the current zeitgeist of school reform as all stick and no carrot.

When the U.S. Chamber of Commerce and the Center for American Progress rolled out its jointly developed “Leaders and Laggards” in February 2007, the report lamented: “[T]he measures of our educational shortcomings are stark indeed; most 4th and 8th graders are not proficient in either reading or mathematics … .”

At the press conference announcing the report, an incensed John Podesta, president and CEO of the Center for American Progress, declared: “It is unconscionable to me that there is not a single state in the country where a majority of 4th and 8th graders are proficient in math and reading.” He based his claim on the 2005 NAEP assessments.

Podesta could have saved himself some embarrassment had he read the recent study by Gary Phillips, formerly the acting commissioner of statistics at the National Center for Education Statistics. Phillips, now at the American Institutes for Research, had asked: “If students in other nations sat for NAEP assessments in reading, mathematics and science, how many of them would be proficient?”

Because we have scores for American students on NAEP and TIMSS and scores for students in other countries on TIMSS, it is possible to estimate the performance of other nations if their students took NAEP assessments.

How many of the 45 countries in TIMSS have a majority of their students proficient in reading? Zero, said Phillips. Sweden, the highest scoring nation, would show about one-third of its students proficient while the United States had 31 percent. In science, only two nations would have a majority of their students labeled proficient or better while six countries would cross that threshold in mathematics. . . .

We're still using that same NAEP to make the case for how our teachers and schools are sending our economy down the drain and creating a national security threat.  And Chester Finn's Fordham Institute always takes the lead on hammering the schools every time the NAEP scores are released.

Well, something new is on the horizon that will make NAEP scores look positively great.  According to a report by the Carnegie Corporation last year, recently brought to my attention by Valerie Strauss, Common Core promises to not only run down the score but to run off large numbers of students from high school.  Here is a most disturbing chart with projections for the next few years, from p. 3, which I hope you will share (click to enlarge):

Economic Performance and Its Zero Correlation to Test Scores

The next time you hear one of the pencil-necked corporatists like Kevin Carey spew about our low test scores and competing economically, point him to this one chart and explain it to him.

Common Core Will Take 24 Years to Show Improvement That Will Fall Way Short of NAEP Results from 1992 to Now

In a research report issued last week by Brookings, researchers found that states with CCSS-like standards have not scored as well as other states on the National Assessment for Educational Progress (NAEP).  

Brookings found “states with standards most different from the CCSS . . . gained the most on NAEP (p. 29).  Offering this dismal assessment of prospects for states adopting Common Core, the Brookings research estimates

it will take 24 years for a noticeable improvement to unfold. And that improvement would add up to 7.62 NAEP scale score points, a gain in 24 years that falls far short of the 22 point gain that NAEP registered in its first 23 years [without Common Core] (p. 32).

TREE Sponsors Elaine Weiss to Talk about What NAEP Means

Elaine Weiss offers a primer on NAEP and recent gains in Tennessee.  Make a cup of tea and enjoy.

MIT Researchers: Higher Test Scores Do Not Translate into Higher Levels of Thinking

MIT Researchers: Higher Test Scores Do Not Translate into Higher Levels of Thinking http://t.co/edCbAClW1g

— Diane Ravitch (@DianeRavitch) December 17, 2013

MIT Researchers: Higher Test Scores Do Not Translate into Higher Levels of Thinking via Professor Diane Ravitch

How's that RttT Working for You, Massachusetts?

Massachusetts has been on the corporate education bandwagon since 1993 with the coming of MCAS, and in recent years the corporate PR machines have made the world aware of how much further ahead MA is in the international testing derbies and on NAEP.  

Last week's appearance of new NAEP scores gave us a chance to have a fresh look at where MA stands in relation to the other states.  Even though the CorpEd pols in MA do not acknowledge the fact, we should point out that MA started far ahead of the national average in reading and math (both 4th and 8th grade) in 1992, when the last generation of testing accountability got underway.  Since the 1990s, in fact, MA has gained nothing on the national average score in 4th and 8th grade reading and math, while gaining 11 points in 8th grad math and 3 points against the national average in 8th grade reading.

The latest NAEP results come just a month after the CorpEd's official organ in MA, the Boston Globe, printed an op-ed by Tom Payzant and Elaine Weiss entitled "Race to the Top: Good for Mass, but a Mismatch Overall."  If the 2013 NAEP scores provide at least a tiny indicator of how good RttT has been for MA, we'd have to put aside Payzant and Weiss on this one.  Massachusetts lost ground between 20011 and 2013 in 4th grade (-4) and 8th grade (-2) reading, held its place in 4th grade math, and lost 2 points in 8th grade math.  In fact, MA lost more ground than any other state since 2011.

Two days ago, the MA Secretary of Education, Matt Malone, gave a brief talk followed by a Q-A at UMass Dartmouth.  I was there to present some of the findings from TMoE, and since I had been engrossed in the NAEP data last week, I came with a question, which I had the opportunity to ask:

This year there is a new contender for the top NAEP honors--the Department of Defense schools.  They are nearly half minority, over half qualify for free or reduced priced lunch, they have a 35% annual mobility rate, over 90 percent of parents have only a high school degree.  

DoD schools were exempted from NCLB and RttT, and they have had no MCAS equivalent used to close schools or keep students from graduating. Without NCLB or RttT, they have had

• no corporate tutoring,
• no segregated charter schools,
• no AYP,
• no school closure and staff firing,
• no coerced adoption of Common Core,
• no unscientific growth models,
• no teacher evaluation based on test scores.

Even without all these corporate ed "advantages," DoD schools have a smaller percentage of "below basic" students than any other state, including MA, and they have a higher percentage of students at "proficiency."  Plus their minority-white achievement gaps are smaller than MA.  

My question, then: what are the benefits of MA's 20 years of corporate education reform that could not have been achieved without it and its accompanying pain, punishment, disruption, and divisiveness?  And can we expect another 20 years of more of the same corporate reforms?

Mr. Malone's did what most good politicians do by answering a question that he, doubtless, would prefer to have been asked, rather than the one I put to him.  He began his response, however, with a claim that DoD schools depended largely upon charter schools to educate our DoD employee's children.  And then he asked me where my accent was from, and told me that he thought I must be nice guy.  

I can live with the patronizing attitude--I have become accustomed to parochial and pedestrian yankee yokels being surprised that Southerners wear shoes, but I can't let the charter claim go without pointing that there are exactly seven (7) charter schools among DoD's 224 schools.  As a good Southerner, I thanked Mr. Malone for his answer and wished him well.  My impression: he ain't no Horace Mann.

Here is an important study of DoD schools from 2001.

DoDEA: What Can Happen without Corporate Education Reform, NCLB, RttT, etc.

When NCLBA exempted Department of Defense schools, some whined that it was unfair: after all.  Rather than protesting the grandiose stupidity of the law to begin with, the National School Boards Association contended that DoDEA schools should be put into the same vice as the the rest of America's schools that receive federal funds.  Misery truly loves company.

So what do we see as a result at the DoDEA schools (see earlier post here) after all these years of living without corporate charter schools, teacher evaluation based on test scores, value-added assessment, corporate missionaries for TFA, AYP, Reading First, scripted parrot lessons, DIBELs, segregation based on test scores, corporate tutoring, venture philanthropy, required testing to graduate or be promoted to the next grade, total compliance classroom, test and punish, test prep, withdrawal of funds from those who need it most, rewarding schools that don't need it, nervous breakdowns, vomiting students, nosebleeds, suicides by principals, school children dying from toothaches as billions are spent on testing, and on and on....How have the DoDEA schools survived without all this?

This chart can be found here.  By the way, can anyone find TN or DC???  And look Massachusetts--and DoDEA did it without MCAS!!

Part 3: Has Corporate Education "Reform" Earned Another 25 Years to Complete the Resegregation and Privatization Agendas?

Part 1 here and Part 2 here:

The continued hydroplaning of NAEP scores nationally allows the world to see what the corporate education losers will not admit: the 3rd generation of test based accountability is not working. 

Rather than moving beyond corporate ed reform's status quo (oxymoron alert), however, Duncan and the shrinking elements of the corporate media that still print the CorpEd news releases point, instead, to the RTTT grant winners where scores rose in math and reading for both 4th and 8th graders: DC and TN.  Ignored are the 17 RTTT grant-winning states where they did not (full list here).  

We might expect a bit more for three and half billion public dollars, but with losing corporate welfare schemes put in place with NCLB's and RTTT, we have to wonder if Duncan did not experience a sigh of relief when scores did not go down.  Yet.  

And even though Jane Roberts wrote for the Memphis Corporate Commercial Appeal that TN's Gov. Haslam and Kevin Huffman were taking victory laps last week, they have probably not forgotten the NAEP story from 2011:

The state dropped from 45 to 46 in the nation in fourth-grade math; 39 to 41 in fourth-grade reading; 43 to 45 in eighth-grade math; and 34 to 41 in eighth-grade reading. Twenty-six percent of fourth-grade students are proficient in reading, and 30 percent are proficient in math. Twenty-seven percent of eighth-grade students are proficient in reading, and 24 percent are proficient in math.

When 2013 scores are compared to 2009, we see, in fact, that Tennessee actually lost ground this year in 4th grade reading (3 points) and 8th grade math (3 points).  Easy come, easy go. 

We should not forget, either, Tennessee's overarching First to the Top goal that it announced in 2010, when TN won the $501 million in RTTT cash: to have all students college and career ready.  I don't know about career ready, but if we use the metric that TN uses for college readiness, the ACT, then Tennessee has a long way to go.  In 2013, Tennessee ranked 47th in college readiness, just behind Mississippi, Louisiana, and North Carolina:

One more chart from ACT that shows 5 year trends for average scores and percentage of students ready for college:

The big NAEP story this year that no one is paying attention to is the continuing progress made by the Dept. of Defense schools (DOD), where there is no RTTT teacher evaluation system based on test scores, where standardized tests don't dominate the schools, where teachers are respected and provided real professional developement, where resources are plentiful and classes small, and where minorities aren't segregated in corporate charter schools.  The DOD school students, in fact, are 40% minority and 50% qualify for free or reduced-price lunch.   

Below is a summary comparison of TN, DC, and DOD, with an added category that averages the black-white achievement gaps for 4th and 8th grade Math and Reading:

Note that DOD has higher average scores for both math and reading in 4th and 8th grades.  Also note the black-white average score gaps are much lower at DOD schools than either TN or DC (same is true for Hispanic-White achievement gaps, too).

Michael Winerip did a story on DOD schools for the NYTimes in 2011 NAEP scores were released.  Here is a clip.  Please do read it:

. . . .It has become fashionable for American educators to fly off to Helsinki to investigate how schools there produce such high-achieving Finns. But for just $69.95 a night, they can stay at the Days Inn in Jacksonville, N.C., and investigate how the schools here on the Camp Lejeune Marine base produce such high-achieving Americans — both black and white.

They would find that the schools on base are not subject to former President George W. Bush’s signature education program, No Child Left Behind, or to President Obama’s Race to the Top. They would find that standardized tests do not dominate and are not used to rate teachers, principals or schools.

They would find Leigh Anne Kapiko, the principal at Tarawa Terrace Elementary, one of seven schools here.

Test preparation? “No,” Ms. Kapiko said. “That’s not done in Department of Defense schools. We don’t even have test prep materials.”

At schools here, standardized tests are used as originally intended, to identify a child’s academic weaknesses and assess the effectiveness of the curriculum.

Ms. Kapiko has been a principal both inside and outside the gates and believes that military base schools are more nurturing than public schools. “We don’t have to be so regimented, since we’re not worried about a child’s ability to bubble on a test,” she said.

Military children are not put through test prep drills. “For us,” Ms. Kapiko said, “children are children; they’re not little Marines.”

Under Mr. Obama’s education agenda, state governments can now dictate to principals how to run their schools. In Tennessee — which is ranked 41st in NAEP scores and has made no significant progress in closing the black-white achievement gap on those tests in 20 years — the state now requires four formal observations a year for all teachers, regardless of whether the principal thinks they are excellent or weak. The state has declared that half of a teacher’s rating must be based on student test scores.

Ms. Kapiko, on the other hand, has discretion in how to evaluate her teachers. For the most effective, she does one observation a year. That gives her and her assistant principal time for walk-through visits in every classroom every day.

“We don’t micromanage,” said Marilee Fitzgerald, director of the Department of Defense Education Activity, the agency that supervises the military base schools and their 87,000 students. “Individual schools decide what to focus on.”

The average class in New York City in kindergarten through the third grade has 24 students. At military base schools, the average is 18, which is almost as good as it is in the private schools where leaders of the education reform movement — Mayor Michael R. Bloomberg of New York; the former education chancellor in New York City, Joel I. Klein; and Bill Gates of Microsoft — have sent their children.

A 2001 study on the success of the military base schools by researchers at Vanderbilt University cites the importance of the smooth relations between the teachers’ union and management, and Ms. Fitzgerald said that continued to be true.

Helping children succeed academically is about a lot more than what goes on inside the schools. Military parents do not have to worry about securing health care coverage for their children or adequate housing. At least one parent in the family has a job.

The military command also puts a priority on education. Bryant Anderson, a petty officer who is stationed at the Naval Surface Warfare Center in Dahlgren, Va., is given time off from work to serve as president of the base’s school board and coach middle school basketball and track teams.

Parents with children at the civilian schools where Ms. Kapiko has been the principal have not received that kind of support from their employers. “If Dad works in a factory, he gets three absences and he’s fired,” she said.

A family’s economic well-being has considerable impact on how students score on standardized tests, and it is hard to make exact comparisons between military and public school families. But by one indicator, families at military base schools and public schools have similar earnings: the percentage of students who qualify for federally subsidized lunches is virtually identical at both, about 46 percent.

What is clear is that the base schools have made impressive progress in narrowing the achievement gap.

In the last decade, the gap in reading between black and white fourth graders at base schools has decreased to 11 points this year (233 compared with 222), down from a 16-point difference in 2003 (230 compared with 214), a 31 percent reduction. In public schools, there has been a much smaller decrease, to a 26-point gap this year (231 compared with 205) from 30 points in 2002 (227 compared with 197), a 13 percent reduction.

The military has a far better record of integration than most institutions. Almost all of the 69 base schools are in the South. They were opened in the 1950s and ’60s because the military was racially integrated and did not want the children of black soldiers to attend racially segregated schools off base. . .