"A child's learning is the function more of the characteristics of his classmates than those of the teacher." James Coleman, 1972
Showing posts with label grading. Show all posts
Showing posts with label grading. Show all posts

Sunday, October 20, 2019

When Artificial Intelligence Equals Real Stupidity

Algorithms grading student essays?  What could go wrong?
“The algorithms are prone to a couple of flaws. One is that they can be fooled by any kind of nonsense gibberish sophisticated words. It looks good from afar but it doesn’t actually mean anything. And the other problem is that some of the algorithms have been proven by the testing vendors themselves to be biased against people from certain language backgrounds.”

Tuesday, October 01, 2013

ASD in TN Makes State Test the Only Grade that Counts

We have known for a long time that school grades are better indicators of later student success than standardized test scores, and we have known, too, that many parents have complained and continue to complain when their children get passing marks and are held back for failing the state test.

Well, both of those possibilities have just been eliminated in Memphis schools under the control of Supt. Chris Barbic (TFAer, BA in English) and the Achievement School District in Tennessee.  Without consulting parents or anyone else, Barbic and his data expert, the former TFAer and KIPP teacher, Josh Czupryk (BS in public relations and BA in Political Science), have come up with, "after research," a new grading scheme that is pegged to cut scores on the state test, known as the TCAP.  Gone: grades for participation, effort, homework completion, or even attendance.  Details from the ASD website:


. . . standards based grading separates evidence of learning from motivational indicators.

•  Evidence of learning: test, quiz, project, assignment, paper, conversations between students-teachers, student modeling a concept or skill

•  Motivational indicators: homework completion, class participation, attendance

Standards based grading seeks to communicate where a student is performing against grade level standards for learning to the student and parent/guardian 

Standards based grading can help teachers, students, and parents hone in on areas for growth where a student is not demonstrating mastery of a grade level skill or concept and accelerate areas where the student is performing at or above grade level expectations. 

Ultimately, meeting grade level expectation is determined by each student's performance on the TCAP assessment and not grades provided by the ASD.  The grades are a way for us to inform parents/guardians, students, and instructors about each student's progress towards standards.

Let me see if I can translate this for those unfamiliar with school as year-round test prep. The TCAP is the only thing now that counts, no matter how much you participate and no matter how hard you try or how much homework you do or how often you come to school.  If pre-tests show you deficient, then you will practice those elements until you are proficient.  If you don't get there, or if you have language issues or disabilities, then you don't pass--or perhaps you would be interested in finding one of those other schools to attend, where the reputation of the ASD empire does not depend upon getting the bottom five percent into the top 25 percent in five years.

This new grading policy is headed toward Gates' ultimate "competency-based" efficiency model for educating the poor, whereby students will be hooked up to a laptop to do the same kind of mindless remediation that the efficiency zealots of previous decades had children doing, minus the fast computers and web access.

Wednesday, October 03, 2012

To Dissect a Polemic: First, Examine the Polemicist

Dissecting The Atlantic's Reform Manifesto: Part IV
Go to Part I; Part II; Part III

"Why Kids Should Grade Their Teachers," by Amanda Ripley

I had an instructive moment in my education well after I left the educational institutions at which I'd been vacationing ("all work and no play," right?): I was working at a bookstore, naturally, as I was "between" my eight or nine Clinton-approved careers (there are no right-left divides when it comes to power, people), where one of the managers was a great reader of review journals (as befits a bookstore manager as one who might find it expedient and an aid to "talking" books rather than reading them).  One day he pointed out a review of a book on Whitman.  He was excited by the review as it was by David Reynolds and he had liked Reynolds' book Beneath the American Renaissance.  His wisdom was that we really should be aware of WHO was writing the review as much (or more than) as what the review actually stated.

I had a flashback to a history course I took in college (pass/fail primarily because I was there under duress to be with my then-girlfriend who was a history major).  I was passing until the final paper for which I actually received an "F."  This had never occurred to me in all my school days.  I approached the professor during office hours..."how, why, change it?!"  He explained that my sources were shit.  This baffled me.  (The paper was written well, I knew, as I had the aforementioned girlfriend serving for the time as editor.)  What?  He clarified without clarifying--these were untrustworthy books I'd read and used.  How in the world should I know that, I asked, and added, these are in our library!  He spoke slowly to me as he explained further as it was clear I would need remedial help.  These are not reputable authors, and that you might not know, but these are also not reputable publishers.  Still I was a bit baffled.  But these are in our library, I muttered, why did they buy them?  He agreed to let me rewrite it using other sources.

Readers of any kind of polemic should always be on guard: there is, as some folks say of Jesus, "a reason for the season."  A polemic works by offering a proposition that we might term a solution to an issue or problem and in order to bolster the perception of viability for the answer proposed the polemicist offers a kind of exemplary narrative.  The "Why" of the "What" or "How," so to speak.  The "truth function" of this narrative is often, if not always, specious.

Morse Peckham, whose work I am obsessed with lately and likely misusing--you should not trust me as I am often a polemicist--, might simply call this an Explanation and an Example.  Thus, as with the reason for the season, any god is an Explanation...examples, however, are hard to come by, and when examined (tested, assessed, measured...) the cookie crumbles.

Amanda Ripley, the author of several "reform" polemics including her most recent featured in The Atlantic's "Special Report" on school reform, has proposed a problem, Teachers are not evaluated properly or, to use a polemical commonplace term, "truly."  This is really a "sub-problem."  The fact of this article's appearance in this magazine within its pamphlet presentation makes its primary problem more easily restated as, What are the possible mechanisms to terminate existing educational labor?

Teachers of literature and "language arts" (I'm not exactly sure what that term means--perhaps "linguistic strategies" might be more instructive) will be very familiar with one rather necessary tactic to model in order for the children (and adults) to read well: noticing context clues.  This is a tactic used most often in schoolrooms to help students make "educated" guesses regarding unfamiliar words.  That is to seek out definitional parameters of unknown terms using the words around the term.  This is not an easy task, actually.  One tends to understand the kind of meaning a term "should" relate, but it's very easy to be wrong, and often very wrong.  One might also not know the meaning of several words in a text thus making it impossible for the text to stimulate a response other than bafflement.

But, that is not our first tactic "deployed" in "defending" ourselves against polemical attack.  (A polemic can, I think, normally be conceived of as an aggressive mode of persuasion and the military metaphor seems apt to me.)

Our first task rather is to try to "situate" the item we're reading contextually.  For example, I have, via "pre-reading" (described in Part I of this dissection) determined that the primary meaning of the pages contained in this "Special Report" will be "schools need reform."  Reform often first must "deform" (polemically speaking) one's current conceptions of a subject.  The primary subject here is "School."  In order for "School" to mean something "new" or different, or rather, in order to change a common understanding of "School" as a term that subsumes many other terms, the polemicist will have to devalue current cultural assumptions regarding the "parts" that make up the "whole" of the subject.

Ripley is herself a context clue and we must pre-read her before we read her polemic.  First, her bio at the New America Foundation (I wrote a little concerning the polemical bent of the NAF in a post on Minecraft at the Errant) lets us know she writes a lot for magazines controlled by powerful corporate interests--Time and The Atlantic seem the biggest ones to me: the "everyman" rag and the "collegiate" class rag.  But I love the first line most: "Ripley is an investigative journalist who writes about human behavior and public policy."  First, I think we should stress that one cannot be a "journalist" when "think-tank" funded.  This is rather, our word of the day, a polemicist.  Next, as Amanda Ripley has zero in her bio about how she comes to be "knowledgable" regarding human "behavior" we might assume she thinks she "studies" people and creates narratives to describe her understanding of behavior.  But there is no indication that she does this in any kind of scholarly or scientific way.  She does this in an "interested" way--as the bio and her list of publications AND soon-to-be released book on "education" called The Smart Kids Club (autobiographical?).  Her "interest" is then put to work in "policy" creation based on her "study" of behavior.  She then popularizes these "social science" fictions via national outlets of extensive reach.

Here are some titles to articles she promote at the NAF bio site to fill out our understanding of her work: Training Teachers to Embrace Reform (WSJ); Boot Camp for Teachers (Atlantic); Teacher, Leave Those Kids Alone (Time); Your Child Left Behind (Atlantic).  I think you get the idea.

Not included at NAF but listed on her personal promotional site:

“What Makes a Great Teacher?” (The Atlantic, Jan/Feb 2010)
Can Michelle Rhee Save Our Schools?” (Time, Nov 26, 2008)

Perhaps these two are not proper examples of the NAF explanation for reform.

Seems plausible that we really don't need to read a word of Amanda Ripley's writing to understand her polemic.  But, one supposes one should, just this once, so as to dismiss her name and her work forever after.  And having read any of these I'm sure you would be ready to pronounce on her impending tome.

***

Ripley sets the polemical tone in the abstract that precedes the article proper.  This is actually the whole argument in just five sweet sentences.

A decade ago, an economist at Harvard, Ronald Ferguson, wondered what would happen if teachers were evaluated by the people who see them every day—their students. The idea—as simple as it sounds, and as familiar as it is on college campuses—was revolutionary. And the results seemed to be, too: remarkable consistency from grade to grade, and across racial divides. Even among kindergarten students. A growing number of school systems are administering the surveys—and might be able to overcome teacher resistance in order to link results to salaries and promotions.

Economist, Harvard, Evaluation, Students, Revolutionary (though familiar and simple), Results (seeming), No Racial Bias (poverty), Systemic Adoption, UNION-resistance, "MERIT" Rewards (not stated--punishments, i.e., termination).  Hitting the agitprop highlights.

Note too: even kids who are nearly babies can pick the wheat from the chaff!

Now that I'm prepared for revolutionary findings (primed for belief) I can read this revealing description of human behavior as it leads to policy.

But, let's play another writer/reader game: Let's look at the set-up and the closing.

Opening:

Nubia Baptiste had spent some 665 days at her Washington, D.C., public school by the time she walked into second period on March 27, 2012. She was an authority on McKinley Technology High School. She knew which security guards to befriend and where to hide out to skip class (try the bleachers). She knew which teachers stayed late to write college recommendation letters for students; she knew which ones patrolled the halls like guards in a prison yard, barking at kids to disperse.
If someone had asked, she could have revealed things about her school that no adult could have known. Once Nubia got talking, she had plenty to say. But until that morning of her senior spring, no one had ever asked.

Would you believe me if I told you someone was about to ask Nubia some very important and revealing questions?

Do you believe Nubia Baptiste revealed all of this to our intrepid investigator?  What sorts of questions must Star Journalist ask to elicit appropriate (useable in magazine polemic) responses?

Closing:

When I called Nubia Baptiste over the summer with the survey results, she was not surprised. “Everybody knows the good teachers from the ones who don’t really want to be in the job,” she said. When I started describing the huge variation between teachers, she interrupted me. “I lived the dynamic,” she said. 
Nubia was on her way to Temple University, where she was considering studying science or engineering. Having personally witnessed many of the recent reforms in D.C., she was wise to what mattered most. 
“I don’t care about the results,” she said. “I care about the change the results bring. If I come back in five years and some crappy teacher is still sitting at that crappy desk, then what was the point of the survey?”

Indeed, Nubia, indeed.  Everybody knows, and Nubia is our fictional "exemplar" who will represent this proffered explanation for a way to introduce an absolutely useless and biased "measure" into a labor environment that is increasingly harried by the bullies of state and power.  But Nubia tells Amanda's/David G. Bradley's/NAF's story and gives her instruction, "what matters most:" getting rid of crappy teachers who sit at crappy desks.  (Do the good ones sit at nice desks, or even sit at all?)

I don't really have the energy to take you through the 4,000 words.  But I do have some favorite admissions.
In towns around the country this past school year, a quarter-­million students took a special survey designed to capture what they thought of their teachers and their classroom culture. Unlike the vast majority of surveys in human history, this one had been carefully field-tested. That research had shown something remarkable: if you asked kids the right questions, they could identify, with uncanny accuracy, their most—and least—effective teachers.
So, we started with already "graded" instructors--what's the "effective/ineffective" benchmark that the kids are "accurate" to reflect?  Doesn't this already beg to many questions to list?
“There are some students, knuckleheads who will just mess the survey up and not take it seriously,” Ferguson [the Harvard Economist] says, “but they are very rare.” Students who don’t read the questions might give the same response to every item. But when Ferguson recently examined 199,000 surveys, he found that less than one-half of 1 percent of students did so in the first 10 questions. Kids, he believes, find the questions interesting, so they tend to pay attention. And the “right” answer is not always apparent, so even kids who want to skew the results would not necessarily know how to do it.
There it is again...polling questions.  Haven't the innumerable horserace election programs taught us all this...polling questions are already ANSWERS in the guise of questions?

***
Just an fyi:
The Atlantic web page for this article tells me there are no Google+ recommendations but 3,000 Facebook recommendations.
***
Oh, I suppose it's not a surprise that this is Gates-Funded research.
***
But, in truth, there does seem to be common sense here (not science, I know) from which we might learn something.

Of the 36 items included in the Gates Foundation study, the five that most correlated with student learning were very straightforward:
1. Students in this class treat the teacher with respect.2. My classmates behave the way my teacher wants them to.3. Our class stays busy and doesn’t waste time.4. In this class, we learn a lot almost every day.5. In this class, we learn to correct our mistakes.
When Ferguson and Kane shared these five statements at conferences, teachers were surprised. They had typically thought it most important to care about kids, but what mattered more, according to the study, was whether teachers had control over the classroom and made it a challenging place to be. As most of us remember from our own school days, those two conditions did not always coexist: some teachers had high levels of control, but low levels of rigor.

How do we not agree with this?  However, the reasons that a class "fails" the test of these five "super-correlations" must be innumerable.  Here are three: bad teachers, bad students, bad schools.

Now all we have to do is figure out how to correlate out the right ones and we're headed toward managerial utopia.
***
A brief closing aside: as Ripley notes in her abstract, student-grading of professors is common-place.  Here is an article by Mark Edmundson (who I must admit is the author of a favorite study on Emerson included in his book Towards Reading Freud) on that very thing back in 1997: "On the uses of a liberal education: as lite entertainment for bored college students."
A college student getting a liberal arts education ponders filling out a questionnaire that includes an opportunity for him to evaluate his instructor. At times it appears that the purpose of his education is just to entertain him.
***
Finally, what does letting the "consumer review"decide the quality of education and the quality of instructor tell us about our culture?


Photo: New America Foundation