Due Diligence & the LA Times

Derek Briggs and Ben Domingue at the University of Colorado at Boulder released a review of the research used in the LA Times value-added project today. It's definitely worth reading and sharing with others.

The authors looked at the work cited in the 2010 LA Times release of value-added scores for LAUSD teachers. The review disputes the finding that teacher qualifications have no association with student outcomes, looks at the issue of non-random assignment , and discusses the (im)precision of the model used.

The authors conclude:

Causal inference may well be the holy grail of quantitative research in the social sciences, but it should not be proclaimed lightly.When the causal language of teacher "effects" or "effectiveness" is casually applied to the estimates from a value-added model simply because it conditions on a prior year test score, it trivializes the entire enterprise. And instead of promoting discussion among parents, teachers and school administrators about what students are and are not learning in their classrooms, it seems much more likely to shut them down.

You can read the full research report here.

Remarkably, the LA Times' Jason Felch read this review and concluded that it confirmed much of the previous analysis. His story ran with the headline (chosen by either Felch or an editor) "Separate study confirms many Los Angeles Times findings on teacher effectiveness." A fact sheet later published by NEPC notes the Times also broke the embargo, a move that reflects poorly on the newspaper. Briggs and the Times assistant managing editor both appeared on 89.1 KPCC to explain their work, the latter sounding more like an advocate/PR man than an editor.

Take a break

Sent to the Washington Post, Feb 8, 2011

Columnist Jay Mathews and some school officials are disturbed because Fairfax County School District high students get 90 minutes a week recess ("High-schoolers' 'recess': Benefit or brain drain?" Feb. 6). Some students, they claim, are simply taking it easy during this time, not studying.

That's only 18 minutes a day, probably much less than these critics take for coffee and unscheduled breaks whenever they feel like it, an option high school students don't have.

These short breaks, time when we can take it easy, are important. Research on creativity tells us that true learning requires some downtime. When Einstein had a problem in his work, he would take some time off for music. After a little relaxation, the solution would often present itself.

We should probably discuss whether 18 minutes a day is enough, not whether it should be eliminated.

Stephen Krashen

Source: Krashen, S. 2001. Incubation: A neglected aspect of the composing process? ESL Journal, 4(2): 10-11. Available at http://sdkrashen.com/articles/incubation/all.html

Musical Interlude: Tom Chapin

New York Charter School Solution Worse than the Problem

Sharon Otterman for the Times has another nail today for the Bloomberg/Black edu-coffin.  The first graph has this:

The new statistics, part of a push to realign state standards with college performance, show that only 23 percent of students in New York City graduated ready for college or careers in 2009, not counting special-education students. That is well under half the current graduation rate of 64 percent, a number often promoted by Mayor Michael R. Bloomberg as evidence that his education policies are working. 

And buried in the story is this detail that sent me to the State pdf. file to have a closer look:

The data also cast new doubt on the ability of charter schools to outperform their traditional school peers. Statewide, only 10 percent of students at charters graduated in 2009 at college-ready standards, though 49 percent received diplomas. The state has not yet calculated results for every district and school. 

So as Cathie thrashes around the City getting booed for blowing up a fresh supply of 12 community public schools for the benefit of corporate-run charters, one must wonder, as Sarah Palin says, WTF??

Here's the dope.  Note that the State somehow could not figure a way to keep the charter data on the same page with the other school data.  Click each chart if you need to enlarge:

 

We must congratulate Meryl Tisch and the corporate foundations running the State education enterprise for coming up with a novel term for the "poor."  The new preferred term by the oligarchs: "high need."  And the "low need?"  Not to worry--they already have it all.

The 2010 Bunkum Awards Have Been Announced

Congratulations, Arne!  From the NEPC (ht to Marilyn Segal):

Bunkum Awards 2010

2010 was a banner year for bunk, leading to a new record of 24 reviews. Yet our awards can recognize only the true crème de la crème of bunkiness. This year’s victors include contributions from perennial frontrunners like Fordham, Heartland, and Heritage, but surprise winners included the South Carolina Policy Council Education Foundation, which until now had toiled unrecognized in the minor leagues. And we also were compelled to open the contest to an important new competitor – the U.S. Department of Education – which seems poised to contribute exceptional bunk for the foreseeable future.

The 'Good Enough for Government Work' Award

Think Tank(s) U.S. Department of Education

 

Review of A Complete Education (U.S. Department of Education)

Review of College- and Career-Ready Students (U.S. Department of Education)

Review of Fostering Innovation and Excellence (U.S. Department of Education)

Review of Great Teachers and Great Leaders (U.S. Department of Education)

Review of Meeting the Needs of English Learners and Other Diverse Learners (U.S. Department of Education)

Review of Successful, Safe, and Healthy Students (U.S. Department of Education)

 

This year’s Grand Prize goes to Secretary Arne Duncan and his U.S. Department of Education staff for the exceptionally disappointing low quality of their research reviews supporting their plans for the reauthorization of ESEA (aka, the Blueprint). Our esteemed panel of judges solemnly considered whether the federal government was even eligible for such an award. With so many resources at its disposal, the government seems to have an unfair advantage. But the Blueprint research summaries stood out in two ways that we felt needed recognition. First, they almost religiously avoided acknowledging or using the large body of high-quality research that the federal government itself had commissioned and published over the years. Second, they first raised our expectations with repeated assurances that recommended policies would be solidly grounded in research – only to then dash those hopes in research summary after research summary.

The issues addressed in the Blueprint and the research summaries are certainly vital to the nation’s education system – standards, teacher quality, comprehensive education, special needs, safe and healthy students, and charter schools. But across the board, our reviewers found the work to be of inadequate quality. One reviewer was astounded that the administration did not mount a comprehensive defense of its central education policies. The research summary reviewed by another was described as a “political text that starts with a conclusion and then finds evidence to support it.” Then, there was the question of critical omissions – such as the complete absence of a rationale for the chosen accountability system and intervention models. In terms of sources, the research summaries were often found to be summarizing non-research. For example, only 10% of the 80 or so citations in the “Great Teachers, Great Leaders” summary referred to peer-reviewed research sources.

The 'If I Say It Enough, Will It Still Be Untrue?' Award

Think Tank(s) The Heritage Foundation

 

Review of Closing the Racial Achievement Gap (Heritage Foundation )

 

While the Heritage Foundation receives this award for publishing Closing the Racial Achievement Gap, byMatthew Ladner and Lindsey Burke, it should really share this award with the think tanks in Colorado, New Mexico, Arizona, Wisconsin, Oklahoma, Utah, and Indiana, as well as the Hoover Institution and the Pacific Research Institute, all of which also published essentially the same report. Perhaps even the derivative articles by Dr. Ladner at National Review Online and Foxnews.com deserve some recognition.

But Ladner’s fecundity isn’t really what sets this work apart. It’s his willingness to smash through walls of basic research standards in his dogged pursuit of his policy agenda. In this case, the agenda is a passel of Florida reforms: vouchers funded by tax credits, charter schools, online education, performance-based teacher pay, test-score grading of schools and districts, test-based grade retention, and alternative teacher certification. He likes them. A lot. And he contends that other states should adopt the same package because, in his vision, they have clearly caused an improvement in Florida schools.

The problem, alas, is with the “caused” part. Nothing in the data or analyses of Dr. Ladner or the Heritage Foundation comes even close to allowing for a causal inference. Instead, the report offers uncontrolled descriptive data focused on NAEP fourth-grade reading scores. Those scores increased, so Ladner’s favored policies must be working. Our reviewer threw a few pails of cold water on that conclusion, noting first that other policies were also in effect during the time period studied. Florida has an excellent supportive reading program aimed at early grades, its accountability system strongly targets resources at lower-performing schools, and it has one of the nation’s most aggressive class-size reduction reforms. Ladner has steadfastly forsworn the possibility that these factors may have played a major role in any achievement improvements.

The reviewer also pointed out the reason why Ladner focused on fourth-grade reading and why NAEP growth in other subjects and grades wasn’t nearly as impressive: the researchers failed to properly address the state’s policy of retaining third-graders with low reading scores. This gave the bottom-scoring students another year to grow before they took the fourth-grade reading test – voilà. By analogy, consider growth in height instead of growth in test scores. If two states wanted to measure the average height of their fourth-graders, but one state (Florida) first identified the shortest 20% of third-graders and held them back to grow an additional year before measurement, the study’s results might be considered biased.

But Dr. Ladner has found his bone, and he’s not letting go. When confronted with the reviewer’s critique, he responded, "The change averse may wish to quibble over the details, or agonize over just what reform did how much of what," but the bottom line is that Florida’s fourth-grade scores increased dramatically. Indeed they did. And no research-bound egghead is going to mess up his good causal story.

The ‘F Double Minus’ Award

Think Tank(s) Heartland Institute & Education Trust

 

Review of 2010 State School Report Card (Heartland Institute)

 

Review of Stuck Schools: A framework for identifying schools where students need change—now (Education Trust )

 

There is a crowded industry devoted to rating and ranking schools and states. The cool part about these ratings is that the imperious think tank raters get to assign a grade of F Double Minus to states and schools that have not adopted their group’s cherished reforms.

The competition is especially plentiful and fierce in this category, but Herbert Walberg and Mark Oestreich of the Heartland Institute vanquished all contenders with their 2010 State School Report Card. Our honorees ranked states (and the District of Columbia) on student achievement, low education expenditures, and “adherence to learning standards,” as well as a ranking based on an average of the first three. They created the indices and validated them by a rigorous examination of their pre-conceived proclivities and then highlighted the top- and lowest-performing states for each of the indices. Although the report explains how the indices were devised, it does not cite any research or rationale to support the methods, other than the data sources themselves plus reports from “several think tanks.” As the report acknowledges, the researchers also don’t even try to control for state variations in demographics or other factors. The report then assigns letter grades to each of the states (plus DC), with a forced distribution: 10 states are assigned A's, B's, C's, and D's, and 11 states must get F's. The beauty of this normative system is that it guarantees it will always be raining – there will always be failing states. Even setting aside problems with the criteria chosen, this seems a bit problematic. Anyway, after acknowledging that there is “very little empirical evidence” as to why some states do better than others, the authors stroll along to their conclusion that education quality is poor and that school choice is the obvious remedy. Assuming there was an acceptable method in this pile of subjective accretion, it is completely unconnected with these final recommendations.

The runner up in this category is a report from the Education Trust, called Stuck Schools, which suggests a framework for identifying chronically low-performing schools in need of turnaround. Among four main problems identified by the reviewer was the following:the norm-referenced methodology guarantees "failed" schools independent of any true performance or improvement level by the school. Sound familiar?

The ‘Plural of Anecdote is Not Data’ Award

Think Tank(s) The Reason Foundation & The Fordham Institute

 

Review of Charter School Autonomy: A Half-Broken Promise (The Thomas B. Fordham Institute, with Public Impact)

 

Review of Fix the City Schools: Moving All Schools to Charter-Like Autonomy (Reason Foundation)

 

Fittingly, we have a double winner in this plural category.

In Fix the City Schools, the Reason Foundation argues for "portfolio" school districts focused on closing low-performing schools and opening new ones under the management of autonomous corporations and sings Hosanna praises to the improvements in student achievement in New Orleans in the post-Katrina era. However, assuming real gains were made, there are myriad reasons unrelated to the portfolio approach that could explain some or all of them, including the massive exodus of low-income children from the city, plus a significant increase in resources. Miraculously, the findings from New Orleans, supplemented by examples from other cities, all evangelically testify to the healing powers of the portfolio school approach.

Not to be outdone, the Fordham Institute, in Charter School Autonomy,contends that charter schools have been deprived of the autonomy necessary for them to deliver on the innovative practices they promised. How could charters truly be innovative if they are still laboring under the yoke of bureaucratic control? It’s an interesting excuse for the not-particularly-impressive results of charter schools. But outside of anecdotes and rhetoric, there is nothing in this report that addresses autonomy in relation to financial performance, resource allocation, academic results, or other key school characteristics and outcomes. The authors simply fail to address their research question – whether and how authorizers’ constraints have had an adverse impact upon charter school autonomy and success.

Now, if we were being completely fair, this award would also be shared by the Blueprint research summaries published by the U. S. Department of Education. All six reviews noted the paucity of research evidence, and the department’s documents were replete with shadowed boxes providing war stories and anecdotes of “success.” In some cases, the boxed anecdotes were actually longer than the accompanying text. But in the “share the wealth” spirit of the administration, we decided that others should be allowed to bask in the glory of this Bunkum Award.

The ‘Remedial Arithmetic’ Award

Think Tank(s) The Cato Institute

 

Review of They Spend WHAT? The Real Cost of Public Schools (The Cato Institute)

 

The goal of this CATO Institute report was to calculate a comprehensive amount spent by k-12 public school districts. Yet, not being content with the actual legally required definitions and conventions used by trained and experienced finance specialists, researcher Adam Schaeffer calculated the “real” costs of public education using his own, extraordinarily creative formula. He found schools cost a lot more than he thought they should. In fact, his estimates of “real” costs vary from 3% to 151% of the government’s numbers.

The CATO breakthrough that we most honor with this award was the decision by Schaeffer to double-count expenditures by adding both capital construction and debt service to his calculation. As our reviewer explained, “most capital construction expenditures are not paid with taxpayer dollars. They are paid with proceeds from bonds. Taxpayer dollars then service the debt. This is key. The cost of a house bought with a loan is not the purchase price plus the cost of the loan. For a school district, what taxpayers are paying for in any year is debt service in that year.” Interestingly, Schaeffer then infers that it’s the National Center for Education Statistics method of cost-calculation that’s designed to mislead the public.

The ‘Magic Potion’ Award

Think Tank(s) The South Carolina Policy Council Education Foundation

 

Review of How School Choice Can Create Jobs for South Carolina (The South Carolina Policy Council Education Foundation )

 

This South Carolina Policy Council Education Foundation report, authored by Sven Larson, wins this year’s prize for the most fatuous cause-and-effect claim. The claim is that school choice will miraculously (our word, not theirs) decrease the unemployment problem of five poor, rural South Carolina counties. Reading this report, one feels transported to an old-time traveling medicine show peddling magic potions: That’s right, ladies and gents, just by being exposed to school choice programs, students will be given a dose of the elixir of private entrepreneurialism that will result in 329 student jobs in 123 new businesses!

As our reviewer explains, the claims in the report are based overwhelmingly on the “tuitioning” programs established in Vermont and Maine back in the 19^th century, whereby very small towns that do not operate schools pay tuition to schools in other towns. Milwaukee’s choice adventures are also cited, as children there are said to be more entrepreneurial. But none of the source documents were peer-reviewed and the data are simply cross-sectional; no causal inferences are (sensibly) possible. Moreover, while ascribing New England small town characteristics to South Carolina counties may seem fanciful, it is not too far a reach for Mr. Larson. Unencumbered with traditional citations, the author announces that the benefits of vouchers are “widely documented.”

Corporate Primitivism as Education Reform

The corporate education futurologists (Gates, Broad, etc.) who appear hellbent to define tomorrow's education systems represent, to borrow a phrase from Marquard, antiquarians turned to face in the opposite direction.  Let me briefly explain.

In Choosing Equality: The Case for Democratic Schooling, the authors offer this synopsis of the “neo-conservative position” on education reform:

Reform efforts should concentrate on a return to academic basics and more rigorous demands on student and teacher performance.  The program calls for standardizing pedagogy and curriculum, mandating more time on more core curriculum . . . increasing the use of standardized and competitive testing . . . lengthening the school day and year, legislating more stringent promotion and graduation requirements.  Proposals from improving teaching quality center on merit pay, master teacher programs, and test-based competency measures.  A number of proposals also promote corporate models of school management and an emphasis on economic development goals (p. 19).

A summary of “bold” thinking with not a whole lot left out, yes?  You may be surprised to learn, then, that the above summary of "new" reform ideas is from a book published a quarter-century ago.  

How time flies and how little education reform ideas change.  What has changed about this short list of 1985 reform notions advanced by neo-conservatives Ronald Reagan and Bill Bennett is that the “liberal” Obama Administration now promotes all these same priority items as new and bold.  Who knew that when Obama came into office decrying the “same tired battles” on education policy that ending those battles would require the Administration to re-initiate the same tired Republican initiatives from a quarter century ago, which offered then as they do now an education plan grounded in 19^th Century psychology and early 20^th Century social theory.  And what passed as pedagogical theory then as now was and is a stripped-down version of Frederick Taylor’s time and motion studies for making widgets, then and now applied to children and teachers. 

Perhaps this time around the corporate re-reformers will recapture those learning glory days that remain elusive even as the persistent fiction that fuels the quest provides a constant and ready source of inspiration for a continuing folly continues to block credible federal education policy.

Not just Cleveland

Poverty is the major factor everywhere, not just Cleveland.

Sent to the Cleveland Plain Dealer, Feb. 5, 2010

When discussing the Cleveland schools, Brent Larkin understands that it is hard to improve school achievement in the presence of "abject poverty" ("Peter Raskind starts atop bottomed-out Cleveland public schools," Feb. 5). But when Larkin discusses American students' performance on international tests, he appears to be unaware that the same factor is at work.

On the recent PISA tests, American 15-year-olds attending schools with less than 10% of students living in poverty averaged 551 on the reading test, second in the world. Our overall scores are unspectacular (tied for 10^th out of 60 countries on the reading test) because we have a high percentage of children living in poverty, over 20%. This is the highest among all industrialized countries. In contrast, child poverty in high-scoring Finland is less than 4%.

For Cleveland and for the US as a whole, the major problem is poverty. Before we worry about teacher quality, institute longer school days, and increase testing, we need to make sure that all children are protected from the effects of poverty: This means adequate health care and nutrition, and access to books. When we do this, American test scores will be at the top of the world.

Stephen Krashen

Original article at: http://www.cleveland.com/opinion/index.ssf/2011/02/raskind_starts_atop_bottomed-o.html

Kress the Moneyman

Sandy Kress is busy these days advocating for, among others, Pearson, TFA, News Corp-owned Wireless Generation, and money managers. Here's the most recent list of companies Kress is lobbying for in the great state of Texas:

Abraham Trading Company 
     Moody Building / Second & Main Street P.O. Box 7  Canadian, TX 79014
     Type of Compensation: Prospective
     $25,000 - $49,999.99
     Client Start Date: 01/19/2011
     Client Term Date: 12/31/2011

Akin Gump Strauss Hauer & Feld LLP 
     300 West 6th Street Suite 2100  Austin, TX 78701
     Type of Compensation: Prospective
     Less Than $10,000.00
     Client Start Date: 01/19/2011
     Client Term Date: 12/31/2011
Citizen Schools Inc. 
     Museum Wharf 308 Congress Street  Boston, MA 02210
     Type of Compensation: Prospective
     $10,000 - $24,999.99
     Client Start Date: 01/19/2011
     Client Term Date: 12/31/2011
Edvance Research Inc. 
     9901 IH-10 West Suite 1000  San Antonio, TX 78230
     Type of Compensation: Prospective
     Less Than $10,000.00
     Client Start Date: 01/19/2011
     Client Term Date: 12/31/2011
Global Financial Aid Services Inc. 
     10467 Corporate Drive  Gulfport, MS 39503
     Type of Compensation: Prospective
     Less Than $10,000.00
     Client Start Date: 01/19/2011
     Client Term Date: 12/31/2011
Governor's Business Council 
     515 Congress Avenue Suite 1780  Austin, TX 78701
     Type of Compensation: Prospective
     $25,000 - $49,999.99
     Client Start Date: 01/19/2011
     Client Term Date: 12/31/2011
Pearson Education 
     1 Lake Street  Upper Saddle River, NJ 07458
     Type of Compensation: Prospective
     $25,000 - $49,999.99
     Client Start Date: 01/19/2011
     Client Term Date: 12/31/2011
Teach For America 
     315 West 36th Street 7th Floor  New York, NY 10018
     Type of Compensation: Prospective
     $10,000 - $24,999.99
     Client Start Date: 01/19/2011
     Client Term Date: 12/31/2011
Texas Instruments Incorporated 
     12500 TI Boulevard  Dallas, TX 75243
     Type of Compensation: Prospective
     $25,000 - $49,999.99
     Client Start Date: 01/19/2011
     Client Term Date: 12/31/2011
Wireless Generation Inc. 
     55 Washington Street 9th Floor  Brooklyn, NY 11201
     Type of Compensation: Prospective
     $10,000 - $24,999.99
     Client Start Date: 01/19/2011
     Client Term Date: 12/31/2011

Rahm's Too Big to Fail

Please share this.  This is really too good to miss.  JH

Join NAACP President Ben Jealous and Thousands of Marchers in Raleigh Feb. 12

 

From IndyWeek.com:

National NAACP President Ben Jealous will be back in Raleigh a week from Saturday. His presence is meant to focus the 5th annual HK on J march on the issue of school resegregation, with the Wake County battle over student assignments continuing to be an enormously important test case.

The march — which calls for Historic Thousands at the General Assembly on W. Jones Street (HK on J) — is scheduled to kick off at 10:30 a.m. on Saturday, Feb. 12.

From the NAACP national office:

Thousands Will Rally in Raleigh To Fight Segregation
NAACP Pledges “Forward Ever, Backwards Never” As Tea Party-Backed School Board Advances Re-Segregation Agenda in Wake County

WHO: NAACP President and CEO Benjamin Todd Jealous, NAACP North Carolina State Conference President Reverend William J. Barber II, will lead thousands of NAACP members and over 90 organizations in demonstration against “separate and unequal.”

WHAT: On Saturday, February 12, thousands of activists will march to the North Carolina State Legislature building to oppose attempts to re-segregate the schools and to march for jobs. NAACP President and CEO Benjamin Todd Jealous and NAACP North Carolina State Conference President Reverend William J. Barber II will be leading the march of over 90 organizations.

“Forward Ever, Backwards Never” will serve as the march’s clarion call. Local Tea Party-backed school board officials have advanced an agenda of “neighborhood schooling” that would drastically reduce school diversity and roll back years of progress. Like Arizona and immigration policy, Wake County is being watched by other states where efforts to restore segregationist educational systems are being considered.

“Our fight in Wake County against resegreation should be our fight throughout the nation,” stated Reverend Barber. “We will never back away from our struggle to ensure that every child has access to a high quality, constitutional, well funded and diverse public education.”

“Separate but equal was wrong then, and it’s wrong now,” stated Jealous. “We cannot in good moral conscience separate the struggle for diverse and superior education from the struggle for jobs and economic solutions. We’ve got to fight for our children, for good schools, for good jobs, and for a great future for all people in our nation.”

WHERE: Raleigh, North Carolina

Marchers will depart from Estey Hall (100 block W. South St., Shaw University campus) and march to the North Carolina State Legislature building (16 West Jones Street)

WHEN: Saturday, February 12, 2011

Marchers gather at Estey Hall: 9:30 AM

March begins: 10:30 AM

Program at Jones Street: 11:00 AM — 12:00 PM

Press availability: 12:30 PM

Founded in 1909, the NAACP is the nation's oldest and largest civil rights organization. Its members throughout the United States and the world are the premier advocates for civil rights in their communities, conducting voter mobilization and monitoring equal opportunity in the public and private sectors.