"A child's learning is the function more of the characteristics of his classmates than those of the teacher." James Coleman, 1972

Monday, November 22, 2010

Simplistic manipulation of data: Paul Thomas responds to Thomas Friedman

Submitted to the NY Times

Paul Thomas, Furman University

Nov 21, 2010

Thomas L. Friedman's commentary on Secretary of Education Arne Duncan's November 4 speech (The New York Times, November 20, 2010) reveals an ironic lesson that many people have failed to learn from Mark Twain's apt quip from the turn of the twentieth century:

"Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: "'There are three kinds of lies: lies, damned lies and statistics.'"1

Like Secretary Duncan's August 25, 2010, speech in Little Rock, Arkansas, Friedman proceeds to reinforce Duncan's claims about education in the U.S. without mentioning poverty a single time. As well, Friedman emphasizes a "few data points" offered by Duncan as evidence.

Before examining the charges offered by both Friedman and Duncan, let's look briefly at a state-to-state comparison here in the U.S. to illustrate the folly of making sweeping claims about educational quality with a "few data points."

Two Southern states, Mississippi and South Carolina share both a long history of high poverty rates (Mississippi at over 30% and SC at over 25%) and reputations as poor schools systems. Yet, when we compare the SAT scores from Mississippi in 2010 (CR 566, M 548, W 552 for a 1666 total) to SAT scores in SC (CR 484, 495, 468 for a 1447 total), we may be compelled to charge that Mississippi has overcome a higher poverty rate than SC to achieve, on average, 219 points higher.

This conclusion, based on a "few data points," is factually accurate, but ultimate false once we add just one more data point, the percentage of students taking the exam: 3% of Mississippi seniors took the exam while 66% of SC took the exam. A fact of statistics tells us that SC's large percentage is much closer to the normal distribution of the all seniors in the state, thus must be lower than a uniquely superior population, such as in Mississippi.

In short, the SAT averages used to compare Mississippi and SC tell us little of value—and thus, the claims made by Friedman based on the speech by Duncan.

The data points praised by Friedman and used by Duncan—drop-out rates, the relationship between education and economic success, and the comparison of teacher pools among countries—appear like SAT scores to be valid data points to draw conclusions about the quality of nations' schools. But the full picture proves that assumption to be false.

One of the most damning failures of the argument is that rigorous research by Gerald Bracey has shown us little positive correlations between measured educational quality and the strength of any nation's economy. This is good political discourse, but the evidence isn't there.

The call to recruit the best and brightest students (as other top countries do, they always add) is also a compelling charge that falls apart when placed against evidence. Studies, again, have failed to show that such a simple process in fact achieves what we would expect.

Finally, the persistent refrain praising Finland and Denmark as "countries leading the pack in the tests that measure these skills" offers yet another simplistic conclusion (such as the one above about Mississippi) that is compelling but incomplete.

Finland and Denmark (according to studies from UNICEF in 2005 and 2007) have childhood poverty rates of 2.8% and 2.4% respectively, while the U.S. childhood poverty rate is 21.9%.

Further, Finland's entire population is only 5 million people, while the U.S. school system educates 50 million children with 3.2 million teachers. In short, as with Mississippi and SC, the full picture about populations reveals a "few data points" as being more about misleading than illuminating.

Do we need better schools and do our children deserve the best teachers in every classroom possible? Of course. No one refutes either of these statements.

But these lofty goals cannot be attained as long as we refuse to acknowledge the historical pattern of social failures that are reflected in (and too often exacerbated by) our schools, such as high drop-out rates for racial minorities and children living in poverty.

Throughout the world, the full picture of any nation's schools reflect the social realities of that country; when schools appear to be failures, the facts show that social failures (the conditions of children's lives outside of school) are driving the educational data.

And we certainly will never address these social failures of the truth about our schools if our political leaders and media voices refuse even to say the word "poverty" while languishing in the simplistic manipulation of data that condemns statistics to Twain's bitter assessment over a century ago.

1 Twain's ascribing the quote to Disraeli appears to be inaccurate, but Twain's actual quote is accurate

Paul Thomas, EdD, Associate Professor

Furman University

No comments:

Post a Comment