"A child's learning is the function more of the characteristics of his classmates than those of the teacher." James Coleman, 1972

. . .a pupil attitude factor, which appears to have a stronger relationship to achievement than do all the “school” factors together, is the extent to which an individual feels that he has some control over his own destiny. James Coleman, 1966

Sunday, February 19, 2012

Rethinking Teacher Quality and Metrics-mania

Once President Barack Obama referred to the claim that teacher quality translates directly into increasing lifetime earnings of students, that assertion and connected study are destined to remain in the public domain in the same way that claims about three quality teachers in a row has persisted. These claims share something other than being quickly and broadly embraced, however; they are also misleading, incomplete, and harmful to our understanding and pursuit of higher teacher quality (see DiCarlo on three great teachers in a row claims and NEPC's review of NBER study connecting teacher quality and student lifetime earnings).

Part of the problem with building education policy on flawed research is that the debate tends to focus on the quality of the study in question, and as a result, we fail to ask broader questions about the process. For example, even critics of some of these studies and policies coming from the studies remain steadfast in pursuing metrics-based teacher quality research:
"I've long argued that the worthwhile debate over value-added accountability is not whether it's 'good' or 'bad' but how to do it smart." (From Rick Hess's Straight Up blog at Education Week) 
"Most of the controversy surrounding value-added and other test-based models of teacher productivity centers on the high-stakes use of these estimates. This is unfortunate – no matter what you think about these methods in the high-stakes context, they have a great deal of potential to improve instruction....Hopefully, these productive low/no-stakes uses for value-added have not been drowned out by all the controversy over its high-stakes use. Research and policy should start focusing on the former as well." (From Matthew DiCarlo at Shanker Blog)
The pursuit of better metrics for identifying high-quality teachers is just as flawed as the recurring pursuit of better/higher standards and better tests—all of which have failed to produce the reform they are intended to support.
Reforming the Pursuit of Teacher Quality: Rejecting Metrics

If teacher quality matters, and it does, and if all students regardless of their life circumstances deserve high-quality teachers (or at least should never be subjected to years of inexperienced and un-/under-certified teachers, as high-poverty, minority, and ELL students currently experience), we must rethink how we determine teacher quality, specifically by rejecting our traditional mania with metrics. This rethinking must include the following:

• Reject the rising culture of high-stakes testing by embracing appropriate uses of test data, which include understanding what data can and cannot explain (test data is a pale metric of learning, but it isn't learning) and avoiding using test data for purposes other than what tests were designed to measure (don't use student test scores to identify teacher quality, for example). [See the work of Bracey and Popham.]

• Shift away from focusing on teacher quality's correlations with test scores, drop-out rates, lifetime earning, and other metrics, and toward teachers' use of authentic assessment and teacher feedback to support student learning and development of the whole child. Expand, then, the evidence of high-quality teaching to include a wide range of artifacts, and not mere quantitative data.

• Decrease and then eliminate credentialing, certification, and accreditation bureaucracy that dilutes the power inherent in education as a rich and vibrant field of study. As someone who certified to teach (and taught high school for 18 years) and now is a teacher educator, I firmly believe certification bureaucracy to be the weakest link in how we move students into the field of teaching. Teachers must first be engaged and challenged students who experience and examine the most effective teaching practices that combine content knowledge with pedagogy.

• While majoring in education must combine sophisticated examinations of content in the context of teaching that content as well as placing future teachers often in the field, the first few years of every teacher's career should be heavily mentored to insure that no students find themselves in classrooms with inexperienced teachers alone. Teaching must be supported as a communal profession, one in which all teachers support and share in the teaching of all children. Current moves to implement merit pay and other forms of competition among teachers destroys the essential need for experienced and high-quality teachers to mentor new and struggling teachers.

• Professional organizations must be allowed to assume their rightful role as organizations that provide professional support and resources for autonomous educators (and not be reduced to extensions of the bureaucracy of certification and accreditation).

• Teacher salaries, advancement, and accountability must be linked to only that which is within those teachers' ability to control: (1) the learning conditions provided by the teachers, (2) the teachers' content knowledge as that is couched in pedagogy, (3) teaching experience, (4) levels of education attained related to content and/or teaching, and (5) teachers' scholarship. Teachers being held accountable for student outcomes is not equitable accountability since student outcomes are not within the control of any teacher.

• The use of data as descriptive must be distinguished from the use of data as predictive. The former is often powerful, and the latter is of little value and often more harmful than helpful.
For over a century, measuring, labeling, and ranking students has been at the center of our education system. During the past three decades, that process has been intensified. We sit in 2012 knowing that these metrics-based paradigms have failed miserably.

Applying that same metrics-mania (within punitive and competition-based policies) to our pursuit of high-quality teachers is more insanity and the only outcome possible is predictable failure—most notably diluting the pool of people willing to teach, discouraging high-quality teachers from working with the students having the greatest needs, and reducing an honorable profession to a service industry.

Suggesting that we need better metrics for identifying high-quality teachers is the same mistake as suggesting we need better standards and better tests. U.S. public education is struggling under the weight of problems that have nothing to do with these seemingly common-sense arguments. There is nothing worse than continuing to seek this fruitless array of "better" that masks what we should be doing instead.


Bracey, G. W. (2006). Reading educational research: How to avoid getting statistically snookered. Portsmouth, NH: Heinemann.

Popham, W. J. (2003). Test better, teach better: The instructional role of assessment. Alexandria, VA: Association for Supervision and Curriculum Development.

Popham, W. J. (2001a) Teaching to the Test? Educational Leadership, 58(6), 16–20.

Popham, W. J. (2001b). The Truth about testing: An educator’s call to action. Alexandria, VA: Association for Supervision and Curriculum Development.

Popham, W. J. (1999) Why Standardized Tests Don’t Measure Educational Quality. Educational Leadership, 56(6), 8-15.

Sawchuk, S. (2011). EWA research brief: What studies say about teacher effectiveness. Washington DC: Education Writers Association. Retrieved 7 July 2011 from http://www.ewa.org/...

Related Reading

Thomas, P. L. (2012, January 30). Further confessions of an outlier. Daily Kos.

-----. (2012, January 15). Accountability without autonomy is tyranny. Daily Kos.

1 comment:

  1. As usual, Paul Thomas nails it. But to deal with the great teachers increase earnings and three teachers in a row myths, we have to understand its reason for being and why it's pushed. These kinds of memes sound good, even appearing to lift up teachers. But what they actually about is the foundation of the "it's really teachers who are the problem" meme. Which not only drives and justifies policy, but undermines the relationship among students, parents, and learning. so now Johnny is not learning because he doesn't have a great teacher. Which means the greatest impact is removing responsibility for learner from the student. When you take down learning, the consequences are enormous. And I'm not certain these are unintended consequences.