"A child's learning is the function more of the characteristics of his classmates than those of the teacher." James Coleman, 1972

Friday, September 04, 2009

Two teachers on using test scores to evaluate teachers

One of the more controversial aspects of the Obama Education Department's approach has been its insistence upon using student test scores as a means of evaluating teachers for merit pay. This is in fact something Sec. Duncan has posed as a non-negotiable requirement for a state to be eligible for $4.5 billion in grants that are part of ARRA (stimulus). These funds, a part of the badly named Race to the Top (RtTP - as if the purpose of education is a race) have led Gov. Schwarzeneggar to try to change current law which that keeps test scores from being used to evaluate teachers.

I want to share an op ed in the Sacramento Bee by 2 teachers who are part of the Accomplished California Teachers Network. David Cohen, who teaches in upscale Palo Alto, is like me a National Board Certified Teacher and a member of the Teacher Leaders Network. Alex Kajitani is California's current Teacher of the Year, and teaches at an inner city middle school in San Diego. And they clearly make the case in their title: Test scores poor tool for teacher evaluation.

Cohen and Kajitani note that while on the surface such linkage might seem obvious, such appearances are misleading. Addressing their remarks to Gov. Schwarzeneggar, whom they urge to go back to school on the subject, they write
experts in education, testing and even economics have argued that state tests are not designed for teacher evaluation and will not yield reliable results. You are taking in us in a direction that will harm our schools and our students.
They note that funding is temporary, but would lead to a permanent and destructive change to
California's thoughtful, research-based and consensus-driven state education policy
in the process of pursuing the funds.

Let me digress briefly to reinforce one point already made - that experts in education and testing disagree with such an approach. There are three principal professional organizations that deal with psychological measurement in schools, the American Educational Research Association, the National Council for Measurement in Education, and the American Psychological Association. In 1999 they jointly reissued The Standards for Educational and Psychological Testing. That document makes clear that tests developed to allow valid inferences for student performance usually cannot be used to draw valid inferences about either teachers or schools. And anyone who understands testing recognizes that most state tests at best measure what a student can give back at the time of testing, in no way controlling for any knowledge or skill developed prior to the current school year.

Returning to the op ed: let me share one very blunt paragraph:
The overemphasis on testing does not enhance educational quality, but instead will promote schooling that leaves too many of our children underprepared for higher education, unskilled at critical thinking and less engaged in their communities. Parents and business leaders consistently say they want us to develop in students the types of skills least valued in a test-driven educational atmosphere.
Neither teacher is afraid of evaluation. However, they believe that the only information the tests provide is how students perform on those tests. They are blunt in asserting that they do not believe the tests either fairly evaluate their students - from two very different environments - or provide an accurate indication of their teaching.

Let me quote the heart of the piece. This will be an extensive selection, but it is necessary to demonstrate what they are trying to communicate:
Like English teachers across California, Cohen works with a set of standards requiring instruction in a range of language arts skills: reading, writing, listening and speaking. Two of these four standards areas are entirely ignored by state tests that offer no listening or speaking components. The tests mostly measure writing skills by checking some basic proofreading skills, but usually, no actual writing.

The all-important reading assessments are similarly narrow and are further suspect because test-savvy students work backward from the questions and don't have to read the passages, and then rely on a variety of outside knowledge to eliminate obvious wrong answers; meanwhile, test-averse students often post scores masking their true abilities. How then can the practice of an English teacher be accurately measured with tests that hardly overlap with the teaching expected of us?

Kajitani, a math teacher, knows that before each test period it is time to pause the teaching of true problem-solving and conceptual reasoning to be sure that students have memorized the operations on which they will be tested and to refresh their test-taking skills. Effective teachers may know how to squeeze in both "teaching to the test" and teaching real, in-depth critical thinking, but this begs the question of where the teacher's time is best spent, for the true benefit of the children they are educating. We sacrifice better learning for better test scores.

All good teachers want to be able to properly assess how their students are doing, in order that we can adjust our instruction to meet their needs. And yet:
Respect for our students and respect for our teaching both demand evaluation based on a broad range of information and multiple measures of performance. Test-driven policies notoriously push in the opposite direction.

As members of the aforementioned Accomplished Teachers Network, Cohen and Kajitani
support efforts to create more effective evaluations, with greater focus on actual teaching practices, including robust and varied evidence such as student and teacher portfolios.

Here, since like David Cohen I am an NBCT, I note that the National Board process is focused on actual teaching practices, and requires the candidate for certification to reflect upon various aspects of her/his teaching practice in terms of how it assists the students. That is something far more valuable than merely prepping students for tests that do not even fairly assess either the knowledge and skill in the domain or ascertain how much the student has learned.

The authors conclude that
evaluating individual teachers based on test scores, in a reactionary effort to compete for Race to the Top grant money, is not the answer. It would be a travesty of education reform for the teachers and students of our state.
And yet as states and schools are desperate for money, education will be distorted in its pursuit, to the detriment of meaningful learning by our students.

This is as perverse as schools, in need of money, selling naming rights to stadiums, or allowing soft drink and junk food machines in the building - in the latter case the desire for money outweighs the medical and dental health implications of encouraging students to consume such products.

The pursuit of money from Race to the Top funds is similar - it is the consumption of non-nutritious educational practices. It is selling the soul of meaningful education. It is as damaging to the minds of our students as the junk food and soda are to their bodies and teeth.

Just my thoughts at the end of the second week with my students.



  1. I agree. I'm concerned that the “Race to the Top” plan has the potential to penalize teachers and schools for circumstances beyond their control, even though the intention is to motivate them to succeed. Uncooperative parents, unmotivated students, and unsupportive administrators can adversely affect student performance, despite the teacher’s best efforts. I encourage you to read my July 29 blog post on this issue at http://parentteacherpartnerships.blogspot.com/

    Natalie Schwartz
    Author, "The Teacher Chronicles: Confronting the Demands of Students, Parents, Administrators and Society."

  2. Merriam-Webster definition of the word "bribe" (Etymology: Middle English, morsel given to a beggar, bribe, from Anglo-French, morsel):

    1. money or favor given or promised in order to influence the judgment or conduct of a person in a position of trust
    2. something that serves to induce or influence

    This is exactly what Duncan and his DoE staff buddies are doing -- all with Obama's stamp of approval.

    It's unbelievable that it's come to this.

  3. You need to read a book, Master Teachers making a difference on the edge of chaos, by Dexter Chapin. It is the best discussion of what it means to be a teacher.

  4. "Pay for performance" is nothing less than equating teachers to prostitutes.

    I am begging your pardon. For the last time, I am NOT A WHORE; I am a public school nursery school teacher.

  5. "Pay for performance" guarantees that teachers with connections and sway will use their persuasive powers in order to stack their classrooms nicely and assure that they are deemed “good teachers.” Students will be further viewed from a deficit-model. Based on a number, many children will be viewed negatively as bringing down scores and paychecks before they ever set foot in the classroom. Pay for performance also guarantees that teachers will be pit against one another in shuffling around students who are more likely to score poorly, such as low income students and English language learners. This will occur at precisely the time when we need to collaborate in order to become strong teachers and to reassert the primacy of the child’s mind as the center and purpose of education.

  6. Anonymous9:18 PM

    Our governor in Wisconsin (Doyle) is all set to jump right in and compete for the "stimulus" money. He wants to modify current state law that prohibits tying teacher salary to test scores.
    Doyle eliminated the qualified economic offer that was used by the state for 15 years to keep the lid on salary increases. Many thought this was a good thing not realizing Doyle was doing it to make the move toward merit pay.

    Doyle is also gung ho on more testing and is apparently ignoring all the evidence cited in this post.
    When I met him in 2002 when he first ran for governor I warned him about pushing testing. I informed him about the problems with standardized testing and described better measures of accountability. He was responsive to my message then, but now seems to be caught up in the pro testing mind set that pervades mainstream media discourse on accountability.

  7. This is an easy problem to avoid. Just come up with a better more reasonable way of determining who is performing well as a teacher. We all know great teachers, good teachers and bad teachers. We know that not all teachers are well above average. So how do we deterimine who is doing well, who needs help and who needs to find anohter job? There are very few jobs in the world where people are not judged on performance. Why shouldn't teachers be evaluated? Teachers are practically asking for test scores to be used by not coming up with something better.