Economic Performance and Its Zero Correlation to Test Scores

The next time you hear one of the pencil-necked corporatists like Kevin Carey spew about our low test scores and competing economically, point him to this one chart and explain it to him.

Guest Post: Why Reformers Misunderstand Their Own Research

By John Thompson

According to the latest Gates Foundation study, "Gathering Feedback for Teaching," student surveys are more reliable than outside observers in evaluating teachers. I agree with the researchers that such a finding should be no surprise. Even though I trust my students' judgments more than those of principals, or even trained outside evaluators, however, I would think twice about incorporating those surveys into high-stakes evaluations. Do we want to institutionalize a conflict of interest on educators, tempting some to lower their standards in the hope that their students would reciprocate when evaluating them? Do we want to take the risk that popularity issues will undercut collaboration among teachers?

The New Teacher Project (TNTP) makes a big issue of the Gates finding that value-added results are the better predictor of future value-added results. Well duh! Since they are mostly studying the students' trajectories within their school, it is no surprise that "Gathering Feedback for Teaching" also concludes that 2/3rds of the variance it found was due to factors other than the teacher. (Emphasis is their's) But, why doesn't the TNTP recognize that such a finding contradicts its faith in teacher quality? The students current test score trajectories are based on the sum of their in-school and out-of-school experiences, not just that teacher's skill. The human observers are just trying to evaluate that teacher's instruction, and it constitutes only about 15 to 20% of the students' outcomes. The question which the TNTP should ask is whether the test score trajectories of students from the entire district provide evidence for evaluating the value-added of teachers in schools where it is harder to raise student performance.

Secondly, why would the TNTP think that such a finding has anything to do with the policies they advocate? The question is whether test score growth models are valid enough for high stakes decisions. More precisely, the question is how many teachers would be wrongly indicted as ineffective? Why would an educator commit to the toughest schools if he or she faced a 20% or a 10% or 25% chance EACH YEAR of being subject to humiliation, constant stress, and perhaps dismissal due to circumstances beyond their control? How many teachers per hall need to be inaccurately charged as ineffective before the morale of the entire building is compromised?

Why can the TNTP not see the three prime lessons of the new Gates research?

1. The MET's findings "suggest that the classroom practices of the majority of teachers, as many as 85 percent, are remarkably similar." If the goal is not the mass scapegoating of teachers, then stakes do not need to be attached to data for professional development.
2. Teachers score the lowest on the all-important factors of emotional connections, communication, and teaching analysis, problem-solving, and inquiry. But the MET does nothing but ask districts to stop imposing counter-productive test prep and abusive evaluations by untrained, overworked, stressed-out administrators.
3. The most "vexing question" is whether value-added experiments are biased due to the "infinite number of additional student and peer characteristics." The MET promises to address a few of the easiest of those issues in the next report.

Typically, I am less annoyed by Gates researchers than by the true believing teacher-bashers at the TNTP or extremists like it's founder, Michelle Rhee, but I want to challenge their single most illogical soundbite. As usual, "Gathering Feedback for Teaching" cites the TNTP's polemical "The Widget Effect," and concludes that it's expensive and dangerous experiment "significantly outperforms traditional measures." No! They outperform the NON-USE of traditional measures. Nowhere in the MET literature do they ask what it would take to reform more normative procedures or why principals have not even terminated ineffective probationary teachers who have no due process rights. The MET shows how easy it can be to identify the bottom performers, but it does not mention how hard it is to find qualified replacements.

The TNTP has done its damage, however, by starting the "teacher quality" movement down a disgusting road, so now we should concentrate on non-ideological educators, such as those who have advised the MET. We could then build on two other findings.

Firstly, both evaluations and professional development require the training of objective evaluators. We should make the investments necessary to nurture principals and others so they can gain the ability to recognize good teaching and mentor others. We should welcome dissent and heed the Gates' call for considering a robust variety of perceptions. Secondly, we must invest in creating learning cultures and professional development so that teachers can better communicate, challenge, engage, and nurture their students. The best way to do that, of course, is to use diagnostic data, which is more accurate because it has not been tarnished by gamesmanship created by high stakes.

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.

Guest Post: NCLB's Faith-Based Reforms, a Post Mortem

By John Thompson

No Child Left Behind, like the War in Iraq and "Voo Doo" economics is a part of the American tradition of "the Power of Positive Thinking." If teachers really believed that "every child can learn," said the law's authors, revitalized schools would undo the legacies of Jim Crow and poverty. It is thus fitting that we celebrate NCLB's tenth anniversary with the words of Adlai Stevenson, "I find Saint Paul appealing and Saint Peale appalling."

Educationally, NCLB was a faith-based law that attributed schools' problems to "the benign bigotry of low expectations." It proclaimed that "No Excuses!," "High Expectations!," and an attitude of "Whatever It Takes!" could undo concentrations of poverty. An unholy coalition (or should I say an overly holy coalition?) of the left and the right agreed that the horrific conditions of the educational "status quo" were "no accident." Schools were failing in precisely the way they were designed to fail. Unions, because they defended teachers with the wrong mindset, were "courthouse sitters," standing in the way of the civil rights movement of the 21th century.

Politically, NCLB was presented with a bipartisan happy face. It did not need huge investments, such as the costs of refighting the War on Poverty in order to get kids ready and able to learn. NCLB was essentially based on the hypothesis that the way to make a hog bigger is to repeatedly weigh it. Economist Eric Hanushek, for instance, has estimated that great gains could be achieved by tests that cost as little as $50 per child, per year. And Hanushek subsequently estimated that testing's minor investments, that were only modestly sucessful, data-driven accountability produced gains worth $14 trillion.

Hanushek acknowledged that we do not know how to systemically overcome the educational legacy of generational poverty. The economist supported NCLB, however, because it created, "a new attitude along the lines of 'we might not know what to do, but we've got to do something.'"

As educators predicted, however, some experiments worked while others resulted in the narrowing of the curriculum, rote instruction, nonstop test prep, fabricating test score gains, and pushing out difficult-to-educate students. Those unintended consequences, said liberal NCLB supporter Dianne Piche, should not be blamed on NCLB because, "The law is not a person ... it's a piece of paper."

And that led to another paradox. Education bureaucrats who designed NCLB worked with pieces of paper. Because of their disconnect with actual schools, they can claim innocence. But, a decade ago, educators predicted precisely how and why NCLB would backfire. Actual practitioners should have explained that the phenomenon that Piche et. el calls "student performance" is not necessarily something real. It is just bits of electrical bursts in computer systems. That data may or may not reflect actual learning. What gets measured, however, gets done, meaning that, predictably, teaching and learning took a back seat to gaming the numbers.

On the other hand, a retrospective by the economist Mark Schneider, claimed that NCLB actually worked, even though improvements on the most reliable assessment, the federal NAEP, slowed after its enactment. Schneider postulated the force known as the era of "consequential accountability." So, he argued the 2002 law deserves credit for performance gains in 1998!?!?

But now, Schneider has agreed that, "overall gains in performance during the NCLB era are not large." Schneider has thus reclaimed his inner materialist, and has distanced himself from the idealistic roots of NCLB. which said that all it takes to transform schools is to really believe. He has now spilled the beans on what other "reformers" really believe is the key to improving schools. "We had the screws on the states," he said, but politics interfered.

So, retrospectives on NCLB bring to mind other idealists like Billy Sunday, who served a loving God who sometimes was very, very ANGRY. To improve schools, we must first proclaim, "something good is going to happen to you TODAY!" But more vengeful "reformers" know that faith alone has not been transformational. Perhaps they should have reminded their allies that it is not enough to proclaim, "Heal!" Hands also need to be placed on the radio, before we can expect the faith healer to do his work.

Consequently, retrospectives defending give a glimpse of the two sides of the contemporary "reform" movement, that embraces both the liberating power of choice, but also command and control micromanaging of school systems. Some "reformers" are the good cops who love poor kids, and their paradigm is, "I think uplifting thoughts, therefore an educational reformer I am." The bad cops are the social engineers who impose data-driven dictates and litmus tests on educators. Their NCLB was a crusade to wipe out "the status quo."

These postmortems also recall the theories of Thomas Berkeley and Samuel Johnson's counter-argument. Berkeley's positivism denied the existence of material reality. Johnson replied by striking a table ad proclaiming, "I refute it thus!"

For the last decade, teachers have demonstrated Johnson's realism and passed on actual examples of how the spirit of NCLB has backfired. I was one of the naive educators who hoped that our stories might convince the true believers that reality is more than the old paradigm. I never recognized the power of the simpler explanation. Teachers' experiences might be the result of a mass hallucination, so it is understandable that they have not dampened "reformers" faith in data and NCLB.

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.