Guest Post: What was DiCarlo thinking?

By John Thompson

As conservative school "reformers" bemoan their defeat in the 2012 election, and as some seem to admit the failure of their "reforms," some accountability hawks are doing some self-criticism. For instance, Fordham's Kathleen Porter-Magee says that the reform movement suffers from "group think," and it could be heading for its educational Bay of Pigs. Porter-Magee criticizes her allies' demonization of educators, but she still suggests there is an equivalency in this educational civil war. She still seems to believe that teachers who are fighting back against an unprovoked assault launched against us share complicity in the venous politics of school reform.

Exhibit A in demonstrating that educators' struggle to defend our profession is different is Matt DiCarlo's "Value-Added for the Record." At a time when test-driven evaluations are reeling from political defeats, when many or most teachers want to drive a stake through their heart, DiCarlo, argues that "value-added should be given a try in new teacher evaluations." He is uncomfortable with counting test score growth as 40-50% of a teacher's evaluation, but he seemed willing to count it as 10 to 15% of an evaluation.

When I first read DiCarlo's position, I was stunned. The last thing our schools need is another rationale for standardized testing. It was not shocking, however, that a researcher who writes for the union-affiliated Shanker Institute would take such an unlikely position. Teachers and our representatives come from a tradition which defends dissent. I think he is dead wrong on this one. But, I read DiCarlo's take on value-added as an eloquent testimony regarding the difference between the two sides of the school reform wars. Teachers and unions remain open to the clash of differing ideas. We do not impose litmus tests. So, I offer this critique in that spirit.

DiCarlo is "frequently taken aback by the unadulterated certainty" about "this completely untested policy" known as value-added evaluations. His analysis tends to focus on things he is qualified to discuss, research design details and their policy implications. DiCarlo does not try to be a "armchair policy general when it's not my job or working conditions that might be on the line."

One problem with DiCarlo's position is that there is no offer on the table to reduce the bubble-in component of evaluations to 10 to 15%. If such a compromise were enacted, the harm done by value-added would be diminished. On the other hand, under NCLB graduation and attendance rates often count for about 10%, but few metrics has been subject to as much gamesmanship as those two. And, even though they account for a seemingly small percentage of a school's report card, they have prompted a series of destructive policies, from so-called "credit recovery" to tricks for making absences disappear, that have done real harm to schools.

That leads to the biggest problem with DiCarlo's logic regarding this issue. He sees that "value-added is the front line soldier in that larger war" and, correctly, seeks to deescalate. Then, he makes the leap to viewing value-added for high-stakes decisions as "a related yet in many respects separate question, and a largely empirical one at that." So, "That's the whole idea of giving something a try - to see how it works out."

DiCarlo's proposal ignores the most important half of the equation - students. He seems to be saying that teachers should not fight against efforts to use our students as lab rates. If the most likely outcome occurs and value-added evaluations drive more talent from the schools where it is harder to raise test scores, how long will it take to undo the damage? In districts where value-added prompts more bubble-in malpractice, how long will students pay for that experiment?

But, here's the key point in my disagreement with DiCarlo on his post. If we respect differences in opinions and maintain our faith in the honest clash of ideas, we and our students will win. Especially in regard to high stakes uses of value-added, if policy-makers would also respect the rules of evidence, educators would come out fine. So, education must remain true to the scientific method. We must continue to revere the principle of the peer review of evidence, and that means we respect our colleagues who reach differing conclusions. Nobody is immune to "group think." As long as we welcome opinions such as the one that DiCarlo presented, however, we can make the case that educators obey a set of values that are not equivalent to those of the data-driven crowd.

I still have to ask, however, what was DiCarlo thinking ...?

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.

Guest Post: Tough Educational Challenges Make for Bad 'Reforms'

By John Thompson

There is an old saying that, "Tough cases make bad law." Similarly, the New York Department of Education, due to its size, is a tough educational challenge. Ordinarily, Eric Nadelstern's "The Evolution of School Support Networks in New York City" would primarily interest policy wonks, but since the architects of New York City's "reforms" have tried to impose them on the rest of the country, this acronym-packed account of governance squabbles holds lessons for all educators. It also helps explain why so many bad educational policies are being imposed on our nation's schools.

According to Nadelstern, the bad old "status quo," which dominated the NYC schools from 1968 to 2003, was a bunch of "fiefdoms," that rewarded loyal constituencies and perpetuated a culture of compliance. Nadelstern sought to replace those bad fiefdoms with good "networks" (as in Networks for School Renewal) and good "zones" (as in the Learning Zone.) But, Chancellor Rudy Crew co-opted the idea of devolution and supposedly created a bad "district" (as in the Chancellor's District) that micromanaged.

Then, Crew's bad version of decentralization was replaced by the Mayor Mike Bloomberg's good form of centralization, known as "mayoral control." The good Bloomberg dissolved the Board of Education, and created the Education Priorities Panel. Concurrently, the bad Bloomberg "ran roughshod over" the panel and imposed good policies (such as ending social promotion) and bad micromanaging (such as mandating 150 minutes of tutoring a week.)

Nadelstern's describes a Manichean division between righteous crusaders for "Children First," as opposed to "vested interest groups" who "preyed on the school system." The battle between good and evil became even sharper when Joel Klein became the chancellor. Klein, a litigator not an educator, came to the rescue not by addressing educational substance but by creating ten regional superintendents. To keep his people from becoming an "imperial superintendency," he stripped them of financial power. Klein gave the power of the purse to six regional operations centers (ROCs). But, the ROCs recreated the same "dysfunctional, top-down culture," and became "districts on steroids." So, Klein created an Office of New Schools. Klein used that office to staff schools with principals who had been trained in his philosophy. By the way, Klein had a "genius" for defeating the bad micromanagers. For instance, he defeated some of them by micromanaging the number of boxes (two) they could move from their old offices to their new ones.

If this narrative sounds arcane, please be patient because the really good stuff begins in 2004 when Nadelstern was promoted. Unfortunately, his boss had a different priority, the stress of the political conflict took its toll, and Nadelstern underwent two back surgeries. Then, the Autonomy Zone was created as "an antidote to regional mismanagement." The Zone was rebranded as Empowerment Schools and "we created the first integrated service center (ISC)." But Klein had a couple of different priorities, and he created the learning support centers (LSCs) and the partnership school organizations (PSOs). The presumably bad side of Klein mistakenly allowed the Division of Instruction to become "a safe haven" for dissenters. The good Klein later authorized Nadelstern to dismantle the bad division. But, the good PSOs created a balance between supporting schools and creating a "culture of accountability," while the bad PSOs made excuses. Also, the ISC "foundered," and the ROCs were reorganized along the lines of school support organizations (SSOs).

Nadelstern writes with unmistakable pride that his Empowerment Zone grew to 535 schools and 22 networks as he became "completely responsible" for the "day-to-day operations" of 1,700 schools. Because he met weekly with subordinates who made weekly visits to those schools, presumably Nadelstern could always divine who was using and who was misusing their power. On the other hand, the SSOs were not as prescient, and their "diffused reporting structure" was "problematic" when resolving problems.

Getting back the alphabet soup of "reform," the ISCs were aligned into clusters, but it took a high-profile battle with a deputy chancellor before it was determined that the right way to organize clusters was around function, not geography. Seven members of the chancellor's cabinet had worked for Nadelstern, but "in retrospect, it is easy to see that our work began to unravel the summer before Joel Klein's departure and my retirement." So, once again, the district is squandering millions of dollars by micromanaging schools.

For some reason, Nadelstern does not take the story full circle and he does not mention the emails that the Klein administration was forced to release due to the Freedom of Information Act. He did not ask whether the old de facto "office of constituent service" has been reconstituted for the benefit of charter schools and Klein's other allies (such as his new boss Rupert Murdoch.)

Instead, Nadelstern outlines recommendations for the top down destruction of top down governance. Centralized power should "nurture successful networks" while protecting them from the networks that would destroy them. That way, the opponents of the superintendents' opponents would be empowered, as their enemies are driven out of the system. Then, good principals would reign supreme, unfettered by teachers' representatives or, for that matter, the administrators with the best knowledge of how schools actually operate - assistant principals. The good superintendents, using aggregate student data, would reward and punish principals and networks. And, none of the palace intrigue would influence the objective evaluations of principals or teachers...

Seriously, why is Nadelstern so confident that he can tell the difference between the top down, centralized punishment of adults (in service to children,) as opposed the micromanaging that damages students? Given the time that Nadelstern and Klein devoted to the DOE's battles over ROCs, ISCs, LSCs, PSOs, and SSOs, how could they had time to figure out whether their data had a connection to the realities inside schools? How could they ensure that their good principals and superintendents, when unchecked by other peoples' power, conveyed reliable information to the righteous few who held them accountable? And, even if Nadelstern could staff every school with administrators with the sterling moral character required to report the whole truth, given the damage done by the stress of the political combat that felled him, why would he think that school leaders could survive the unrelenting pressure that was imposed on them?

Nadelstern worked for a tough litigator whose policies damaged some students in order to help others, in the belief that a final victory over the forces of darkness would liberate them all. Since Klein created a system where it was unlikely that accurate information would be conveyed up the chain of command, it would have been nice if his deputy had cited research on how their reforms were actually implemented or journalism documenting the damage that was done to schools that they left behind. It would have also been refreshing for a person in Nadelstern's position to balance his reports of big increases in the graduation rate with an acknowledgement that "credit recovery" programs jacked up those numbers.

Given the enormous challenge of reforming central offices, should he have not urged novices like Klein to stick with the big enough task of reforming the district's administration? Why would they have possibly thought that they could train an entire system to think the way that they think? Why were they so confident in their ability to identify which schools deserved to be rewarded and punished?

Nadelstern is so convinced that a "culture of accountability" produced the improvements that occurred in his school, he ignores the damage that it did to other schools. He seems obsessed with the political combat that it took before he was "permitted to commander the chancellor's conference room at Tweed" so that he could enact the constructive part of his agenda. I have no doubt that Nadelstern benefitted some students when he worked with 14 network leaders to develop a "strong common culture of service to schools." But, his own words seem to indicate that his boss' "reforms" did more harm than good, and that their accomplishments are transitory. And, why did he adopt a risky bank shot of engaging in brutal bureaucratic combat in order to achieve the opposite - a culture where only some favored schools could treat educators and students with respect?

The answer, I bet, is linked to the enormity of the challenge of gaining control, in order to relinquish control. My theory was that the obsession with fighting New York-sized political battles blinded people to the opportunities to draw upon the Big Apple's strengths. Rather than rooting out all dissent, they could have better served students by making compromises and building on the city's diversity. And the same applies to Klein's and Nadelstern's acolytes who feel so entitled to use the tough and unique case of New York City in order to micromanage the way that districts across our diverse nation run their own classrooms.

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.

Guest Post: The Education Sector Attempts to Head Off a Common Core Trainwreck

By John Thompson

The Education Sector's new report, Getting to 2014 (and Beyond): The Choices and the Challenges Ahead, could have been entitled "Whistling Past the Graveyard." The report starts with the standard proclamation that "new accountability systems will reflect the expectation that tremendous progress in students outcomes across the country could occur through the unprecedented efforts unleashed by Common Core." Yeah, and perhaps a train wreck could be avoided if accountability hawks repudiate their tactics of the last generation, while dealing with the nine issues described by the Ed Sector. So, let's take a look at what would have to happen to avoid one disaster when value-added meets Common Core.

I have been pestering policy types for a scenario where value-added evaluations could co-exist with the transition to Common Core Assessments. I have usually been met with a sputter, sentence fragments, and an admission that we will have to choose between one policy or another. Rarely do these reformers get past the most obvious roadblock. Many or most states will soon replicate the experience of Tennessee where proficiency on eighth-grade math fell from 90% to 26% in two years when standards were raised. Needless to say, the panic and the retribution that these drops will prompt will not be conducive to rational decision-making.

The transition from bubble-in accountability to its opposite, the assessment of college readiness skills, will require teachers and principals to buy into a mixed message. During the next two years, educators will receive training for what Craig Jerald calls a "massively difficult, change in instructional practices." In the meantime, they will have to teach to, and be subject to termination by, primitive standardized tests of basic skills. That might be hypothetically possible if, to borrow from Michael Cohen, accountability is transformed into "something meaningful," if everyone recognizes that Common Core could produce "tremendous progress in student outcomes," and if we all rally toward this vision.

To his credit, Bill Tucker is, to my knowledge, the first "reformer" to articulate a scenario where value-added evaluations do not need to be abandoned in order to implement Common Core. Tucker maintains a brave front, but I suspect that he, like Jerald, does not really believe that such a two front war is possible.

Tucker cites testing expert Drew Gitomer who explains that we can't assume that the "patterns of growth from one test to a dramatically different test the next year" will be comparable. "If the new standards and tests meet their expectations and really are measuring different learning outcomes," Tucker notes, "'then a whole lot more needs to be understood before we simply apply value-added models to these data.'" The most likely outcome, I would add, is that it will prove much harder to raise Common Core scores in high-poverty schools.

Tucker then crafts a metaphor that nails the dilemma:

If today's value-added measures are like calculating a sprinter's improvement in the 100-meter dash across different tracks, then in spring 2015, we'll be attempting the equivalent of calculating value-added across not only different tracks, but also racing events. And while we may be able to infer a number of things about a 100-meter sprinter's improvement by her performance the following year in the halfmile, calculating a precise measure will be extremely difficult, at best.

If we accept Tucker's professed assumption that data-driven evaluation systems must remain in effect, then he presents the only plausible roadmap. Reformers would have to make a "bold statement" about the importance of Common Core and announce a "hiatus" in test-driven evaluations. I can't believe that Tucker really thinks it would be possible to stop value-added evaluation for only one year, but he keeps a brave face in calling for that brief lapse in test-driven accountability.

For the hiatus to remain a hiatus, and not the abandoning of value-added evaluations for multiple year, or forever, all the stars would have to align perfectly. Fields trials would have to remain on schedule and produce reliable results. Statistical models that have been developed over two decades for one type of test would have to prove reliable for a completely different test, as risky innovations are fast tracked. Even when using seven years of data, and using the latest in a series of experimental models, the Los Angeles VAM was only 27% accurate in placing teachers' performance in the correct evaluation category. But, educators under pressure to completely transform the way that they do their jobs, especially in the poorest schools, would then have to submit to high stakes evaluations using one year of real-world data.

Value-added models (VAMs) using old-fashioned test scores have yet to control for concentrations of special education students, English Language Learners, and other peer effects, or for large percentages of top students who already score highly. I am most concerned that value-added will produce an exodus of the top teachers from low-performing schools where it is more difficult to raise test scores. For that reason, Tucker's article should be read in context with the other great piece, by Robert Balfanz. Balfanz asks how the 15% of high schools with the greatest problems with chronic absenteeism and the highest dropout rates could continue to improve graduation rates while biting off the completely different challenge of Common Core. Balfanz writes, "if we continue to concentrate the neediest students in a subset of schools that are not designed for success, we will neither be able to raise performance levels nor graduation rates for these students." Balfanz then recommends three more promising approaches, using completely different strategies, to help the poorest schools.

But Balfanz's astute recommendations are the topic for another post. The point in regard to Getting to 2014 (and Beyond) is that educators already have far too much on their plates, and that is even more true in high-poverty schools. In a rational world, value-added evaluations would be the first thing we would take off teachers' and principals' plates.

Tucker and the rest of the Education Sector contributors focus on "a new expectation of accountability systems" with the assumption that they are the only way "to continuously improve their ability to drive students toward college and career readiness." If we really want to improve our poorest schools, we need to listen to Balfanz and others while not assuming that their research-based policies can only be implemented when shoe-horned into the failed ideology of "accountability."

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.

Guest Post: Fight Club Fantasies of the Corporate Education Deformers

By John Thompson

Political scientist Patrick McGuinn wrote a sympathetic account of education reform advocacy organizations (ERAOs) that gather in Washington, D.C. The ERAOs, "compare notes and plot strategy in what is (half in jest) referred to as 'fight club.'" I wonder if they would be so enthusiastic about political bloodletting if more of their members had been covered in the blood of actual students.

Viewing schools from 30,000 feet, it is easy for "reformers" to simply blame the teachers unions that they "derisively called the 'blob.'" Glorying in rhetorical violence, accountability hawks seek to destroy the members of the educational "status quo," under the assumption that disruptive innovation will somehow create better alternatives. It is easier to kick down a barn, however, than to build one. And anyone who has labored to turnaround a school that accepts every child who walks through the door knows that it is harder to improve the toughest neighborhood schools than it is to start a charter that can kick out the most challenging kids.

McGuinn is one of many political scientists who are intrigued by the "brass knuckle" school of reform that has been funded by corporate reformers and edu-philanthropists. He identifies two strands of reform, "system refiners," who embrace accountability, and "system disrupters." He then documents how "it is clear that a new club of reform organizations is itching for a fight and that politicians in both parties are increasingly willing to join them in the ring."

What McGuinn does not mention, however, is evidence that this political combat can benefit kids. McGuinn cites another political scientist, E. E. Schattschneider, who observed that "new policies create new politics." He does not mention the large body of social and cognitive science which explains why test-driven policies have largely failed. Neither does he cite the educational research which explains why it is unlikely that such testing will improve schools that serve intense concentrations of trauma and generational poverty.

For non-educators and non-students, a coalition that unites (former?) Democrats like Joel Klein and Michelle Rhee with Republicans like Jeb Bush and Mitch Daniels is an object of fascination. In a man-bites-dog twist, the Democrats for Education Reform (DFER), for instance, has taken the lead in attacking teachers unions and firing teachers based on primitive metrics that may or may not say anything about their actual performance. For scholars who do not have any skin in the schooling game, it must be fun to listen to the ERAO's trash talk, and how they want a "Republican counterpart to DFER--mdash;which insiders jokingly refer to as ReeFER."

This social engineering experiment known as "reform" took off in the early 1990s, as "New Democrats" sought replacements for the old New Deal/Fair Deal safety nets. Presumably these bright and good-hearted reformers would have been equally happy to fix welfare or health care for the old fogies in those fields, but they ended up setting education practitioners straight. The era's reform de jour was the "instructional triangle" where "teachers engage with students on subject matter." So, "reformers" leaped to a quick and cheap solution. Teachers were "deputized" as the point of the spear in a battle against poverty.

Nearly a generation of new research, however, has confirmed decades of previous social science which explains why the instructional triangle is inherently incapable of turning around our toughest schools, and why socio-emotional interventions and trusting relationships are needed. We now have even more evidence how and why test-driven accountability leads to gimmicks that encourage educational malpractice, while undermining the trust and the teamwork necessary for overcoming the legacy of poverty. In fact, many or most reformers who teach in charter schools could explain the same thing. Something tells me, though, that few classroom teachers of any type make their soirees.

When I was an academic, I loved to study grand and dramatic concepts like "creative destruction." So, I see the allure of big ideas like "disruptive innovation," and I see why new activists want to show that the scholarly canon is wrong. Political tsunamis are much more thrilling to witness, even if failed cataclysms are not so enjoyable for those on the ground.

My views changed as I came to know and love students. So, I wonder about the total number of hospital visits that have been made by the attendees of ERAO get-togethers, or how many students' funerals they have attended. I wonder how many unconscious kids they have held after students were assaulted at school or succumbed to an untreated chronic condition. Similarly, if "reformers" believe that their primitive bubble-in tests are a valid measure of students' learning, I doubt they have had much experience sharing the joys of teaching and learning. Had ERAO members shared enough deep relationships with students, they would show more respect to poor kids' minds. If they are still willing to use standardized testing to defeat their adult enemies, I question whether they grasp the insults their metrics are imposing on the children who they want to help.

I suspect that most of these "reformers" are oblivious to out-of-school factors that hinder classroom performance because they have little concrete understanding of communities suffering from extreme poverty. They would have to be supermen to expend so much effort defeating their adult enemies and to still be able to produce good for schools. I also find it hard to believe that the ERAOs would revel in the "Fight Club" imagery if they had spent much time in the ring fighting the real fight.

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.

Guest Post: Why Reformers Misunderstand Their Own Research

By John Thompson

According to the latest Gates Foundation study, "Gathering Feedback for Teaching," student surveys are more reliable than outside observers in evaluating teachers. I agree with the researchers that such a finding should be no surprise. Even though I trust my students' judgments more than those of principals, or even trained outside evaluators, however, I would think twice about incorporating those surveys into high-stakes evaluations. Do we want to institutionalize a conflict of interest on educators, tempting some to lower their standards in the hope that their students would reciprocate when evaluating them? Do we want to take the risk that popularity issues will undercut collaboration among teachers?

The New Teacher Project (TNTP) makes a big issue of the Gates finding that value-added results are the better predictor of future value-added results. Well duh! Since they are mostly studying the students' trajectories within their school, it is no surprise that "Gathering Feedback for Teaching" also concludes that 2/3rds of the variance it found was due to factors other than the teacher. (Emphasis is their's) But, why doesn't the TNTP recognize that such a finding contradicts its faith in teacher quality? The students current test score trajectories are based on the sum of their in-school and out-of-school experiences, not just that teacher's skill. The human observers are just trying to evaluate that teacher's instruction, and it constitutes only about 15 to 20% of the students' outcomes. The question which the TNTP should ask is whether the test score trajectories of students from the entire district provide evidence for evaluating the value-added of teachers in schools where it is harder to raise student performance.

Secondly, why would the TNTP think that such a finding has anything to do with the policies they advocate? The question is whether test score growth models are valid enough for high stakes decisions. More precisely, the question is how many teachers would be wrongly indicted as ineffective? Why would an educator commit to the toughest schools if he or she faced a 20% or a 10% or 25% chance EACH YEAR of being subject to humiliation, constant stress, and perhaps dismissal due to circumstances beyond their control? How many teachers per hall need to be inaccurately charged as ineffective before the morale of the entire building is compromised?

Why can the TNTP not see the three prime lessons of the new Gates research?

1. The MET's findings "suggest that the classroom practices of the majority of teachers, as many as 85 percent, are remarkably similar." If the goal is not the mass scapegoating of teachers, then stakes do not need to be attached to data for professional development.
2. Teachers score the lowest on the all-important factors of emotional connections, communication, and teaching analysis, problem-solving, and inquiry. But the MET does nothing but ask districts to stop imposing counter-productive test prep and abusive evaluations by untrained, overworked, stressed-out administrators.
3. The most "vexing question" is whether value-added experiments are biased due to the "infinite number of additional student and peer characteristics." The MET promises to address a few of the easiest of those issues in the next report.

Typically, I am less annoyed by Gates researchers than by the true believing teacher-bashers at the TNTP or extremists like it's founder, Michelle Rhee, but I want to challenge their single most illogical soundbite. As usual, "Gathering Feedback for Teaching" cites the TNTP's polemical "The Widget Effect," and concludes that it's expensive and dangerous experiment "significantly outperforms traditional measures." No! They outperform the NON-USE of traditional measures. Nowhere in the MET literature do they ask what it would take to reform more normative procedures or why principals have not even terminated ineffective probationary teachers who have no due process rights. The MET shows how easy it can be to identify the bottom performers, but it does not mention how hard it is to find qualified replacements.

The TNTP has done its damage, however, by starting the "teacher quality" movement down a disgusting road, so now we should concentrate on non-ideological educators, such as those who have advised the MET. We could then build on two other findings.

Firstly, both evaluations and professional development require the training of objective evaluators. We should make the investments necessary to nurture principals and others so they can gain the ability to recognize good teaching and mentor others. We should welcome dissent and heed the Gates' call for considering a robust variety of perceptions. Secondly, we must invest in creating learning cultures and professional development so that teachers can better communicate, challenge, engage, and nurture their students. The best way to do that, of course, is to use diagnostic data, which is more accurate because it has not been tarnished by gamesmanship created by high stakes.

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.

Guest Post: NCLB's Faith-Based Reforms, a Post Mortem

By John Thompson

No Child Left Behind, like the War in Iraq and "Voo Doo" economics is a part of the American tradition of "the Power of Positive Thinking." If teachers really believed that "every child can learn," said the law's authors, revitalized schools would undo the legacies of Jim Crow and poverty. It is thus fitting that we celebrate NCLB's tenth anniversary with the words of Adlai Stevenson, "I find Saint Paul appealing and Saint Peale appalling."

Educationally, NCLB was a faith-based law that attributed schools' problems to "the benign bigotry of low expectations." It proclaimed that "No Excuses!," "High Expectations!," and an attitude of "Whatever It Takes!" could undo concentrations of poverty. An unholy coalition (or should I say an overly holy coalition?) of the left and the right agreed that the horrific conditions of the educational "status quo" were "no accident." Schools were failing in precisely the way they were designed to fail. Unions, because they defended teachers with the wrong mindset, were "courthouse sitters," standing in the way of the civil rights movement of the 21th century.

Politically, NCLB was presented with a bipartisan happy face. It did not need huge investments, such as the costs of refighting the War on Poverty in order to get kids ready and able to learn. NCLB was essentially based on the hypothesis that the way to make a hog bigger is to repeatedly weigh it. Economist Eric Hanushek, for instance, has estimated that great gains could be achieved by tests that cost as little as $50 per child, per year. And Hanushek subsequently estimated that testing's minor investments, that were only modestly sucessful, data-driven accountability produced gains worth $14 trillion.

Hanushek acknowledged that we do not know how to systemically overcome the educational legacy of generational poverty. The economist supported NCLB, however, because it created, "a new attitude along the lines of 'we might not know what to do, but we've got to do something.'"

As educators predicted, however, some experiments worked while others resulted in the narrowing of the curriculum, rote instruction, nonstop test prep, fabricating test score gains, and pushing out difficult-to-educate students. Those unintended consequences, said liberal NCLB supporter Dianne Piche, should not be blamed on NCLB because, "The law is not a person ... it's a piece of paper."

And that led to another paradox. Education bureaucrats who designed NCLB worked with pieces of paper. Because of their disconnect with actual schools, they can claim innocence. But, a decade ago, educators predicted precisely how and why NCLB would backfire. Actual practitioners should have explained that the phenomenon that Piche et. el calls "student performance" is not necessarily something real. It is just bits of electrical bursts in computer systems. That data may or may not reflect actual learning. What gets measured, however, gets done, meaning that, predictably, teaching and learning took a back seat to gaming the numbers.

On the other hand, a retrospective by the economist Mark Schneider, claimed that NCLB actually worked, even though improvements on the most reliable assessment, the federal NAEP, slowed after its enactment. Schneider postulated the force known as the era of "consequential accountability." So, he argued the 2002 law deserves credit for performance gains in 1998!?!?

But now, Schneider has agreed that, "overall gains in performance during the NCLB era are not large." Schneider has thus reclaimed his inner materialist, and has distanced himself from the idealistic roots of NCLB. which said that all it takes to transform schools is to really believe. He has now spilled the beans on what other "reformers" really believe is the key to improving schools. "We had the screws on the states," he said, but politics interfered.

So, retrospectives on NCLB bring to mind other idealists like Billy Sunday, who served a loving God who sometimes was very, very ANGRY. To improve schools, we must first proclaim, "something good is going to happen to you TODAY!" But more vengeful "reformers" know that faith alone has not been transformational. Perhaps they should have reminded their allies that it is not enough to proclaim, "Heal!" Hands also need to be placed on the radio, before we can expect the faith healer to do his work.

Consequently, retrospectives defending give a glimpse of the two sides of the contemporary "reform" movement, that embraces both the liberating power of choice, but also command and control micromanaging of school systems. Some "reformers" are the good cops who love poor kids, and their paradigm is, "I think uplifting thoughts, therefore an educational reformer I am." The bad cops are the social engineers who impose data-driven dictates and litmus tests on educators. Their NCLB was a crusade to wipe out "the status quo."

These postmortems also recall the theories of Thomas Berkeley and Samuel Johnson's counter-argument. Berkeley's positivism denied the existence of material reality. Johnson replied by striking a table ad proclaiming, "I refute it thus!"

For the last decade, teachers have demonstrated Johnson's realism and passed on actual examples of how the spirit of NCLB has backfired. I was one of the naive educators who hoped that our stories might convince the true believers that reality is more than the old paradigm. I never recognized the power of the simpler explanation. Teachers' experiences might be the result of a mass hallucination, so it is understandable that they have not dampened "reformers" faith in data and NCLB.

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.

Guest Post: No Excuses for obfuscating abysmal attrition data with complex calculations

By John Thompson

MacArthur Genius grant recipient Roland Fryer did some complex calculations in "Creating 'No Excuses' (Traditional) Public Schools" to estimate student performance gains during the first year of Houston's "Apollo 20." The massive pilot program extends "No Excuses" instruction to neighborhood schools. Fryer, however, omitted a simple statement of its most important data for that type of experiment. In contrast to "No Excuses" charters, low-income students were not going out of their way to volunteer for 21% more hours of instruction and much more rigorous enforcement of academic and behavioral standards. Some of those charters have had attrition rates of 40% or more. So, an evaluation of whether Fryer's model could be scaled up for neighborhood schools requires an estimate of how many students opted out during the program.

Fryer reported that over 7,000 students were enrolled in nine schools, and that all results were based on the sample of students who remained in school long enough to take the spring tests. He did not reveal how many students took those tests. Or if Fryer did, I could not find it.

Fryer found large gains in math for the two grades where Houston could afford more than $2,500 per student for personalized tutoring, but overall results were disappointing. Middle school students largely declined in reading while high school kids increased for a total effect on reading that was statistically insignificant. Results for "double-dozing" math and reading classes also were discouraging.

Fryer acknowledged, however, that even those modest increases are questionable if his statistical models could not account for challenging students opting out. But he was not very transparent in reporting the students' attrition rate. Back when I was in school, we could turn to the tables at the back of an academic paper and discern that sort of essential information. So, I checked out Table Two on page 71 and looked for something like, "N = x." All I found, however, was that Fryer reported 8,693 "observations" on students who began the school year with Apollo and took two spring tests.

So, I checked out the district's interim report, dated January 2011, which said that 7,385 students were in the program. The Houston Chronicle reported in the fall of 2011, only 6,156 enrolled at the beginning of the second year. But still, it was impossible to tell from Fryer's work whether attrition affected his results. The district's interim report, however, said that the students who entered the Apollo program were 86.6% economically disadvantaged. According to Fryer's report, the sample of students who remained until testing was 61% economically disadvantaged. In other words, Fryer started with nine high-poverty schools, but his conclusions seem to be based on a sample that were not far from average poverty for public schools.

Fryer's report is preliminary and it is categorized as a "working paper," but his "back of the envelope" estimates of return on investment are just as perplexing. He worries that the most effective treatment, where each highly-recruited tutor worked intensively with two students, would be difficult to scale up. Even so, Fryer asserted that the return on investment would be 2-1/2 times as great as the returns that James Heckman found for high-quality pre-school. Heckman, however, evaluated gains that were sustained over time, and clearly Fryer did not. Apparently, Fryer's estimate was based on test score increases, which he compared to increases studied by Alan Krueger due to reducing class size. Even if Fryer is a true believer in data-driven instruction, however, it is hard to understand that how anyone could assume that one-year boosts in math scores, after a diet of interim tests every three weeks, as well as benchmark tests, would be life-transforming.

That gets us to back to the key issue. Fryer claims that if all five characteristics of "No Excuses" schools are implemented with fidelity, then the legacy of poverty can be overcome. What else would it take to scale up his model? "Apollo 20" required the removal of 310 teachers and nine principals. The district replaced 100 instructors with Teach for America teachers, and 60 veterans with a history of increasing test scores. Those transfers received two annual $10,000 bonuses. Replacing nine principals required 200 interviews and one third of those who were hired have since been replaced. A national search attracted 1,200 applicants for tutors. Of the 287 who were hired, thirty left. All three groups were subjected to intense training in "No Excuses" values and techniques and were subjected to detailed and expensive oversight.

It is hard to imagine where Houston, or any other district, could find enough of the type of educators that their experiment requires. The key metric, however, was not the attrition of those carefully-selected adults, but that of the students. The big question is how many high-poverty neighborhood school students stuck out the year, and whether test score gains were inflated by the loss If anyone can find that answer in Fryer's work, or explain why he did not volunteer that information so that everyone could evaluate his evidence for themselves, I'm all ears.

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.

Guest Post: Assumptions that 'Just Seem Obvious' to Steve Brill

By John Thompson

There is a long line of education experts exposing the factual errors in Steve Brill's Class Warfare so, instead, I set out to illuminate the assumptions underneath his brand of "reform." I was overwhelmed, however, by the number of times that Brill, and the non-educators he fawned over, discovered findings that "just seemed obvious." So, I only want to recount Brill's tale of the naive assumptions, opinions, and assertions, based on numbers from one district, that supposedly propelled Bill Gates into the experiment that Gates said was, "the riskiest thing we have ever done."

Brill begins with "Identifying Effective Teaching Using Performance on the Job," by Tom Kane, Robert Gordon, and Douglas Staiger. Kane et. al wrote, "the current credential-centered regime is built upon two questionable premises ..." They then presented evidence that was solid enough to contribute to the minor league debate over teacher certification. But then these economists made another leap, "the second premise is that school districts learn nothing more about teacher effectiveness." Without making any effort to link their data (and that assertion) to reality, Kane and company called for dangerous and revolutionary changes to the entire nation's educational systems. When I first read their opinions, I chalked them up to being academic theory.

If Brill's account is trustworthy, though, I was naive. Either Brill was loose with his words, or he was celebrating a classic "bait and switch." According to Brill, the authors asserted that the need for "paper qualifications" is a "bedrock of public education." Worse, Kane et. al supposedly claimed that the second core assumption of our system is that "districts can learn nothing more of teaching effectiveness after the initial hire," and "this was why almost all teachers initially receive satisfactory evaluations." (emphasis mine.) Even worse, they successfully convinced Bill Gates that a "surprising" lack of volatility in teachers' performances, measured by standardized tests, was grounds for overturning decades of social science research on teaching and learning, and the way that organizations change.

Rereading the paper, I fear that Brill accurately reported the authors' true intentions. Now I see the report as a grab bag of political soundbites. Kane et. al predicted an impending shortage of teachers, which would be most damaging for high-poverty schools. Their solution - based on the presumption that the lowest scoring teachers should be fired using a methodology that is systematically biased against teachers in high-poverty schools - would supposedly attract talent to those schools!?!?! To their credit, Kane and company recognized the need to adjust for circumstances beyond the teachers' control. For example, if construction was going on outside the window on testing day... But, they argued that test score growth models that would be used for evaluations should not be controlled for poverty.

Fortunately, the Gates team has subsequently distanced themselves from their initial overreach, but it still raises a huge question. What did they not know about peer effects and when did they not know it?

If Brill is accurate, having assumed that there is a single simple explanation for the sorry state of evaluations, the theorists did not need to search for other reasons why inner city schools have a hard time attracting and retaining talent. Having reached the shocking discovery that, "some teachers are better than others," Gates and company thus decided that the path to school improvement must go through their version of instruction-driven, standardized test-driven reforms.

According to Brill, Bill Gates bought the hypothesis that the lack of volatility in test scores meant that their version of teacher quality had to be the focus of reform. After a two-hour discussion, supposedly, Gates became increasingly frustrated that educators ignored a solution that "seemed so obvious."

I will leave it to the experts to analyze how stable or volatile test scores are. Subsequent research by Gates' scholars has been used to argue that teacher performance is no more unstable than major league baseball players' performance - as if that has much to do with fixing schools. My point is that their assumption could only have been made by people with no knowledge of the logistics of schooling. As I have explained, if Brill, Kane, or Gates had had a rudimentary knowledge of how schools are actually run, they could not have jumped to such a weird conclusion.

And that gets to other assumptions. When I was in school, Kane would not have begun such a paper with a mischaracterization of his opponents' positions. A reader used to assume that a scholar would start his argument by honestly summarizing what his evidence was and was not able to prove. In polite discourse, a summary of opposing arguments was always presented in a fair manner. Kane would have been honor-bound to summarize the record of longstanding evaluation methods such as the Toledo Plan, the Teacher Advancement Program, and other successful peer review and mentoring efforts, as well as National Board certification. Had Gates been briefed on the strengths and weaknesses of those union-endorsed methods, perhaps he would have chosen to fund efforts to better incorporate data into those holistic systems, or perhaps he would have financed a scaling up on those methods of improving teacher quality.

The bad news is that education, and educators, are held in such low esteem that revolutionary reforms are mandated by non-educators, based on what seems obvious after being exposed to a sliver of information. Too often, it presented only by self-interested parties who represent a small minority of scholars. The good news is that snap judgments can be reversed. Smart people have changed their minds when introduce to better evidence.

This era of primitive test-driven hypotheses reminds me of the rookie mistakes made by first generation econometricians such as Robert Fogel, Stanley Engerman, Alfred Conrad, and John Meyer. One even earned a Nobel Prize, showing that a bone-headed error, born of inexperience need not be a mark of shame. Back then, however, there were more checks and balances in regard to academic assertions. When I was in school, these economists made the daring, and soon-to-be-proven-silly proclamations that the building of the railroad system did not accelerate economic growth and that slavery was profitable. Those mistakes are humorous now, but then again, we did not cite their speculations as reasons to tear apart our railroads or reinstate slavery.

And we should remember a key reason why an econometric model, based on data from one state, reached such inaccurate conclusions about slavery across the nation. To the Harvard economists, it seemed obvious that half of Kentucky's mules must be momma mules and half must be daddy mules. Not knowing that mules are sterile, they assumed the profits due to mule reproduction.

If Gates and Kane would be more open about what they do not know, and listen to actual practitioners, they could still be a powerful force for good. But they need to listen to educators with concrete knowledge of how schools are run, and understand the power of peer pressure.

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.

Guest Post: The Facts Not Included in Steve Brill's Tell All

By John Thompson

Steve Brill’s Class Warfare is full of breathless accounts of the rich and powerful that are reminiscent of the biographies of Kitty Kelly. Ironically, the most valuable part of his Class Warfare may be his accounts of dramatic decisions to undertake risky and revolutionary "reforms" based on minimal evidence and some strange assumptions.

According to Brill, "Identifying Effective Teaching Using Performance on the Job" by Tom Kane, Robert Gordon, and Douglas Staiger was used to convince Bill Gates and President Obama that "teacher quality," measured by standardized tests, must be the focus of reform. According to Brill, they argued that tests showed, "remarkable consistency among teachers," and that "obviously" this suggested that the person in front of the classroom was the key to closing the achievement gap. Gates was supposedly told that the "surprising" lack of volatility in the test scores of one district meant we could trust algorithms to drive the firing of teachers. If we believe Brill, because of the stability in the charts that Kane and Gordon presented to Gates, we should presume that 25% of teachers with the lowest test score growth should be fired.

Perhaps I should make no assumptions, and use my 90% low-income district to explain Urban Education 101 for reformers. In our six magnet and charter high schools, the attendance rates never drop below 95%, while our seven neighborhood schools struggle to reach an attendance rate of 80%. When you add in the far lower enrollment rates and the far higher suspension rates in the neighborhood schools, their students attend class at a rate that is 1/4th to 1/3rd less than their peers in selective schools (and far less than their previous three years in middle school.) It would take a superhuman effort for neighborhood school teachers, with kids who are five to six years behind grade level in reading, to meet growth targets that were not controlled for school selectivity.

By the way, those patterns are all stable. Should we thus assume that the teachers in the magnet schools are better than the teachers in neighborhood schools with deplorable scores?

Neither would anyone with firsthand knowledge of actual classrooms assume that a relative lack of test score volatility within schools justifies the "reformers’" extraordinary conclusions. I would assume that Kane, at least, would know that even when student assignment policies are stable, students are assigned differently to tested and non-tested classes. I wonder how he would address a stable pattern in my district where middle school "Science" classes are taught to the names, dates, facts, and figures of the benchmark tests, producing pass rates ranging from 70 to 73%. Then freshmen Biology students have a pass rate that fluctuates from 30% to 51% to 41% on tests designed to measure scientific thinking. Since the Biology teachers’ growth targets would be based on middle school tests, would Kane support the mass firing of those teachers for failure to meet targets generated from the results of different tests?

Recognizing that districts differ, let us take some more obvious examples of stable patterns from my school system. As 5th graders, the kids of Zip Code 73120 attend schools with poverty rates ranging from 23%, 50%, 69%, and 76%, with the special education populations ranging from 5.5% to 14%. The number of suspensions are 0, 14, 33, and 86. The pass rates for 5th grade Reading range from 81% to 95%.

Between 5th and 6th grade, the district's student population drops by nearly 1/5th as the numbers of special education and ELL students increase. In 73120, the decline is more remarkable as the majority of their students move to magnet, enterprise, and charter schools, including four that have gained national recognition. So, the 6th graders arrive at a school with 2,020 disciplinary actions, which lost its middle class students after the campus policeman was hospitalized in a riot, and where 1/4th of their class are on IEPs. Should the 6th grade teachers to be blamed for a Reading pass rate of 44%? Is it reasonable to expect those teachers to meet test score targets based on the three previous years of performance in very different types of classrooms, with incomparably different peer effects?

Similarly, a non-educator might be unaware that the challenges facing an Algebra II class are radically different to those of an Algebra I class next door. The demographic differences between the two math classes, however, are just as stark as the unbelievable differences between our 5th and 6th graders. My school was the lowest-ranked secondary school in Oklahoma, but the percentages of poor and at-risk juniors in Algebra II were not dramatically different to the percentages in many suburban schools. After all, 1/3rd or more of the freshmen Algebra I students drop out before reaching their 11th grade year. (By the way, 9^th grade is the year when the enrollment rate reaches rock bottom, with the average freshmen being in school for less than 2/3rds of a year, and when the truancy and suspension rates explode. Should we fire most, if not all inner city Algebra I teachers for not raising the performance of students who do not attend class?)

Worst of all, the Gates shop has given no indication that they have become aware of the difference between a student with a math or a reading disability, as opposed to a conduct disorder or serious emotional disturbances. Neither have they dealt with the effect of federal policies on how schools are allowed to handle those students, and the implication for schools where 35 to 40% of their students are on Individual Education Plans.

In other words, they do not have the background knowledge to comprehend the scenario described by Paul Tough where an inner city class of thirty kids, because it has 8 to 10 kids who have been traumatized to the point where their brain circuitry has been altered, degenerates into a culture of hitting and fighting. Otherwise, how could Kane and Gates assume that teachers with those sorts of classes would have a chance of hitting their growth targets?

Unless these theorists assumed that Seriously Emotionally Disturbed students are distributed evenly throughout our diverse nation's classrooms, they betrayed a basic misunderstanding of peer effects in urban schools. A class where a third of the students are well- behaved kids with cognitive differences, can not be compared to a class with a critical mass of children who have been victimized to the point where they can not control their behavior. And if the theorists have that level of ignorance, perhaps they also are unaware that schools facing an impossible load of the most difficult-to-educate students tend to turn certain classes into dumping grounds.

Brill’s Class Warfare included 19 pages of sources and endnotes, but there is no indication that he read any of the immense body of social science on teaching, learning, and schooling. According to Brill, Gates gave the go-ahead for his Measuring Effective Teaching effort after two hours of discussion. There was no indication that Gates was exposed to scholarship that contradicted the hypothesis that teacher quality must drive reform.

Had Gates been briefed on the way schools are organized, and the need for early education, a concentration on reading comprehension and socio-emotional skills before 3rd grade, the need for high-quality curriculum, and the promise of community schools that turn education into a team effort, I doubt this smart man would have assumed that there was only one way to fix urban schools.

If Gates, or for that matter, if President Obama (who later adopted the Gates agenda) had been provided a full and fair briefing by educators as well as economists, it would be no skin off Brill's nose. There are still plenty of Kitty Kelly-type books out there begging to be written by a person with Brill's confidence that he obviously knows all of the answers.

Dr. John Thompson was an award-winning historian, lobbyist, and guerilla-gardener who became an award-winning inner city teacher after crack and gangs hit his neighborhood. He blogs at thisweekineducation.com, and huffingtonpost.com, and is writing a book on 18 years of idealistic politics in the classroom and realistic politics outside.