"A child's learning is the function more of the characteristics of his classmates than those of the teacher." James Coleman, 1972
Showing posts with label National Research Council. Show all posts
Showing posts with label National Research Council. Show all posts

Friday, May 27, 2011

National Research Council Rebukes NCLB (Duncan-Gates-Broad) Policies Based on "Primitive Intuition"

After ten years of test, punish, and push out policies and hundreds of billions of tax dollars spent for the benefit of the education industry and corporate foundations, American schools have been blown up and children are less educated than their counterparts in countries with sane education policies.  Even as measured by their gains on American tests, comparisons show students' scores are just .08 of a standard deviation higher than they were 10 years ago.  That amounts to moving from the 50th percentile to the 53rd percentile.  Rod, Maggie, Arne, Bill, and Eli: heckuva job. 

This last dose of reality therapy comes from another report issued by the National Research Council.  Here is a good introduction from HuffPo, even though there is much more to say:

NEW YORK -- Education policies pushing more tests haven't necessarily led to more learning, according to a new National Research Council report.

"We went ahead, implementing this incredibly expensive and elaborate strategy for changing the education system without creating enough ways to test whether what we are doing is useful or not," said Dan Ariely, a professor of behavioral economics at the Massachusetts Institute of Technology and member of the committee that produced the report.

Heavily testing students and relying on their scores in order to hold schools -- and in some cases teachers -- accountable has become the norm in education policy. The No Child Left Behind Act, the largest piece of education legislation on the federal level, for example, uses performance on math and reading exams to gauge whether schools are failing or succeeding -- and which schools are closed or phased out.

"Incentives are powerful, which means they don't always do what they want them to do," said Kevin Lang, a committee member who also chairs Boston University's economics department. "As applied so far, they have not registered the type of improvements that everyone has hoped for despite the fact that it's been a major thrust of education reform for the last 40 years."

The tests educators rely on are often too narrow to measure student progress, according to the study. The testing system also failed to adequately safeguard itself, the study added, providing ways for teachers and students to produce results that seemed to reflect performance without actually teaching much.

"We're relying on some primitive intuition about how to structure the education system without thinking deeply about it," Ariely said.

Increasing test scores do not always correlate to more learning or achievement, the study authors said. For example, Lang mentioned that high school exit test scores have been found to rise while high school graduation rates stagnate.

"None of the studies that we looked at found large effects on learning, anything approaching the rhetoric of being at the top of the international scale," Lang said. He added that the must successful effects the report calculated showed that NCLB programs moved student performance by eight hundredths of the standard deviation, or from the 50th to the 53rd percentile.

The report, released Thursday and sponsored by the Carnegie Corporation of New York and the William and Flora Hewlett Foundation, recommends more rigorous testing of reforms before their implementation. "Before we did welfare reform, we did a lot of experiments at the state level," Lang said.

"We tried different ways of doing it and we learned a lot, instead of deciding that on the basis of rather casual theorizing that one set of reforms was obviously the way to go," Lang added. "There has not at this point been as much experimentation at the state level in education."

The 17-member committee responsible for the study, according to Education Week, is a "veritable who's who of national experts in education law, economics and sciences." The National Academies -- a group of four institutions chartered by Congress to consult on various issues -- launched the committee in 2002, and since then, it has tracked the effects of 15 programs that use tests as teaching incentives.

The report comes as congress works to reauthorize and overhaul No Child Left Behind, and as states countrywide pass laws that link the hiring and firing of teachers to their students' performance on standardized tests.

"It raises a red flag for education," Ariely said. "These policies are treating humans like rats in a maze. We keep thinking about how to reorganize the cheese to get the rats to do what we want. People do so much more than that."

This reductive thinking, Ariely said, is also responsible for spreading the notion that teachers are in the profession for the money. "That's one of the worst ideas out there," he said. "In the process of creating No Child Left Behind, as people thought about these strategies and rewards, they actually undermined teachers' motivations. They got teachers to care less, rather than more," he added, because "they took away a sense of personal achievement and autonomy."

The report's findings have implications for developing teacher evaluations, said Henry Braun, a committee member who teaches education and public policy at Boston College. When "we're thinking about using test-based accountability for teachers, the particular tests we're using are important," he said. "But just as important is the way it's embedded into the broader framework. The system as a whole, as it plays out, determines whether we end up with better teachers."

Friday, October 09, 2009

NYTimes, WaPo, Business Roundtable Remain Mum on NRC Smackdown of Duncan Plan

So far we may count zero print or non-print corporate media outlets to cover the National Research Council rebuke of the anti-teacher Broad-Walton-Gates pay-per-score evaluation plan for U.S. teachers. If the NRC had been laudatory of the RTT bribery plan, wonder if that would be a non-story, too.

Thursday, October 08, 2009

The National Research Council Waves Duncan Off RTT Plan for Teacher Evaluation

If there is opinion that is representative of the best that science has to offer policy makers, it is from the National Academies. In a press release today that coincides with a new report that evaluates strategies proposed by the corporate oligarchs in charge at the U.S. Department of Education, the authors conclude that value-added assessment is not appropriate for high stakes decisions such as teacher pay. Also, the NRC report warned against using NAEP data to assess teacher or school effectiveness:
Date: Oct. 7, 2009

Contacts: Sara Frueh, Media Relations Officer

Alison Burnette, Media Relations Assistant

Office of News and Public Information

202-334-2138; e-mail


FOR IMMEDIATE RELEASE

Education Innovations Funded by 'Race to the Top' Should Be Rigorously Evaluated; Value-Added Methods to Assess Teachers Not Ready for Use in High-Stakes Decisions


WASHINGTON -- The Race to the Top initiative -- a $4.35 billion grant program included in the American Recovery and Reinvestment Act to encourage state-level education reforms -- should require rigorous evaluations of the reform efforts it funds, says a new report from the National Research Council. The initiative should support research based on data that links student test scores with their teachers, but should not prematurely promote the use of value-added approaches -- which evaluate teachers based on gains in their students' performance -- to reward or punish teachers. Too little is known about the accuracy of these methods to base high-stakes decisions on them right now, the report says.


The U.S. Department of Education is developing regulations that explain how the $4.35 billion will be awarded. The National Research Council's report offers recommendations to help the department revise these guidelines.


The report strongly supports rigorous evaluations of programs funded by the Race to the Top initiative. Only with careful evaluations -- which allow effective reforms to be identified and perhaps used elsewhere -- can the initiative have a lasting impact. Without them, any benefits of this one-time expenditure on innovation are likely to end when the funding ends, the report says.


Evaluations must be appropriate to the specific program being assessed and will be easier to design if grantees provide a "theory of action" for any proposed reform -- a logical chain of reasoning explaining how the innovation will lead to improved student learning. Evaluations should be designed before programs begin so baseline data can be collected; they should also provide short-term feedback to aid midcourse adjustments and long-term data to judge the program's impact. While standardized tests are helpful in measuring a reform's effects, evaluations should rely on multiple indicators of what students know and can do, not just a single test score, the report adds.


The Department of Education's proposed guidelines encourage states to create systems that link data on student achievement to teachers. The report applauds this step, arguing that linking this data is essential to conducting research about the best ways to evaluate teachers.

One way of evaluating teachers, currently the subject of intense interest and research, are value-added approaches, which typically compare a student's scores going into a grade with his or her scores coming out of it, in order to assess how much "value" a year with a particular teacher added to the student's educational experience. The report expresses concern that the department's proposed regulations place excessive emphasis on value-added approaches. Too little research has been done on these methods' validity to base high-stakes decisions about teachers on them. A student's scores may be affected by many factors other than a teacher -- his or her motivation, for example, or the amount of parental support -- and value-added techniques have not yet found a good way to account for these other elements.


The report also cautions against using the National Assessment of Educational Progress, a federal assessment that helps measure overall U.S. progress in education, to evaluate programs funded by the Race to the Top initiative. NAEP surveys the knowledge of students across the nation in three grades with respect to a broad range of content and skills and is not aligned with the curriculum of any particular state. Although effective at monitoring broad trends, it is not designed to detect the specific effects of targeted interventions like those to be funded by Race to the Top.


The study was sponsored by the National Research Council. The National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and National Research Council are private, nonprofit institutions that provide science, technology, and health policy advice under a congressional charter. The Research Council is the principal operating agency of the National Academy of Sciences and the National Academy of Engineering. A committee roster follows.

Copies of Letter Report to the U.S. Department of Education on the Race to the Top Fund are available from the National Academies Press; tel. 202-334-3313 or 1-800-624-6242 or on the Internet at http://www.nap.edu. Reporters may obtain a copy from the Office of News and Public Information (contacts listed above).