More Bogus Research from UofArkansas's Walmart School of Ed Reform

What a shame that a great state university like Arkansas would allow billionaire bullshit "research" to sully its good name!

NEPC has reviewed the recent charter school "research" study by Jay Greene and his team and found it severely lacking in most everything that research requires to be legitimate:

BOULDER, CO (August 19, 2014) – A recent report from the University of Arkansas Department of Education Reform (DER) on charter school productivity asserts charter schools are more effective in producing achievement on standardized tests and are also less costly per pupil than traditional public schools. A new review released today finds the report’s claims suffer from multiple sources of invalidity, rendering the report useless.

Gene V Glass, Regents' Professor Emeritus at Arizona State University, reviewed The Productivity of Public Charter Schools for the Think Twice think tank review project of the National Education Policy Center (NEPC).

The report uses findings from the National Assessment of Educational Progress (NAEP) and “revenues received” to support its claim that charter schools spend less per pupil than traditional public schools and produce achievement as good as or superior to that of traditional public schools.

In his review, however, Glass points out that  the report inaccurately employs NAEP test results, and that its calculation of expenditures in charter and traditional public schools relies on questionable data. The report, meanwhile, also discounts the fact that demographic differences between the two sectors are highly correlated with NAEP performance. In short, Glass says, “The sector with the higher percentage of poor pupils scores lower on the NAEP test.”

Taken together, the report’s flaws leave readers with little evidence on which to base any valid conclusions, Glass concludes. He predicts, however, that despite its many shortcomings, charter school supporters will attempt to use the findings to advocate expanded funding for charter schools. In that respect, he writes, “The report continues a program of advocacy research that will be cited by supporters of the charter school movement.” 

The Facts About KIPP That Kevin Chavous Ignores

The other day at Huffington Post, school privatization advocate, Kevin Chavous, gave a “shout out” to KIPP, Inc., the nation’s largest corporate chain of charter schools and certainly the most popular high-rolling philanthro-capitalists who would never send their own children to one of these total compliance lockdown schools.  As is customary for KIPP promoters, Chavous tries to portray the KIPP chain gang model as an outgrowth of the late Harriet Ball’s teaching method. 

We know, however, from Jay Mathews’ saccharine tribute (Work Hard, Be Nice) to KIPP founders, Mike Feinberg and David Levin, Harriet Ball did not want her name associated with KIPP’s penal version of her own tough, yet loving, approach to teaching.  In fact, a former KIPP teacher whom I interviewed for an extended research essay on KIPP teaching, had traveled to Texas to get to know Harriet Ball; he found her distraught about KIPP’s misuse of her name for practices that she did not endorse and did not use.

The other piece of Chavous’s pep piece for KIPP that requires correction is the claim that “80 percent of KIPP alumni have gone on to college.”  This is an old bit of propaganda first circulated by Jonathan Alter for a Newsweek pep piece for corporate education reform four years ago:

The irony is, we know what works to close the achievement gap. At the 60 KIPP (Knowledge Is Power Program) schools, more than 80 percent of 16,000 randomly selected low-income students go to col­lege, four times the national average for poor kids.

Suspecting hyperbole gone wild or just old-fashioned making it up, reporter Carolyn Grannan called KIPP’s home office in San Francisco to check the numbers and found that when Alter presented his outrageous “fact” in 2008, a total of 447 former KIPPsters were attending college.  

As busy as he is during this election season, it is easy for me to imagine how Mr. Chavous could come to depend on discredited stories from old Newsweek’s for his “facts.”  As senior adviser and Board member of American Federation for Children, America’s leading school voucher advocacy group, Chavous is teaming this summer and fall with Rhee’s Michelle Students First to dole out millions to political candidates who sign on to support school privatization by voucher and/or charter. 

If Chavous had checked with the KIPP home office or even done a Google search, he would have found this from KIPP’s own research:

As of March 2011, 33 percent of students who completed a KIPP middle school 10 or more years ago have graduated from a four-year college.

Now this figure would be more impressive if 33 percent of KIPP students who started at KIPP in 5^th grade ended up graduating from college, especially since the completion rate for low-income children is less than 9 percent.  However, consider these facts:

From 40 to 60 percent of children who begin KIPP in 5^th grade leave before completing 8^th grade, with the largest percentage of leavers being low-test performers and students who do not conform to KIPP’s total compliance demands.  From a 2011 study headed by Gary Miron: 

 . . . .The departure of low-performing students helps KIPP improve its aggregate results. Unlike local school districts, KIPP is not replacing the students who are leaving. When a student returns to a traditional public school after the autumn head count, KIPP retains most or all of the money (the amount depends on the particular state) allocated for educating that student during that school year.

. . . KIPP schools enrolled a lower percentage of students with disabilities (5.9%) than did their local school districts (12.1%).

. . . KIPP enrolled a lower percentage of students classified as English Language Learners (11.5%) than did their local school districts (19.2%).

. . . Combining public and private sources of revenue, KIPP received, on average, $18,491 per pupil in 2007-08. This is $6,500 more per pupil than what the local school districts received.

Even with these massive advantages, KIPP is able, then, to see approximately 17 percent of the children who begin KIPP in 5^th grade end up with a college diploma.  That would still be impressive, if it were not for the horrific fact that Chavous does not mention.  The thousands of children who do NOT make through KIPP, to college, or through college have all absorbed one important lesson before they are discarded, and that is that their failure to make it is no one’s responsibility except their own.  And so in the end, the “No Excuses” ideology applies universally to the children, as well as teachers, who are psychologically maimed by KIPP, even if it never applies to KIPP itself.  And for that atrocity, Mr. Chavous, there is no excuse.

Tennessee Celebrates Charter Growth But Ignores Horrid State Report Cards

Jane Roberts at the Commercial Appeal has a gushing piece this morning on the growth of the segregated corporate charter schools in Tennessee, particularly in Memphis, Tennessee's Gates Hub.  Here's how she begins:

Tennessee is popping up on education reform maps for its innovation, including that it now has the fastest-growing charter school movement in the nation.

Eleven charters opened in the state last year, a 37 percent increase. The number of students attending charter schools -- about 9,500 -- jumped 38 percent over the 2009-2010 school year, down several percentage points from the last year but still the fastest growth in the nation, according to National Alliance for Public Charter Schools.

"Tennessee went from having some of the poorest legislation in support of charters to some of the most favorable. That was the precursor to making Tennessee a hotbed for charter growth," said Jamal McCall, executive director of KIPP Memphis charter school.. . .

Growth, indeed.  Too bad Jane doesn't seem at all concerned about the abysmal performance of these new child chain gangs, where learning takes a back seat to behavioral control and cultural sterilization.

Below are the test results for the Memphis charters in Jane Roberts's back door. Released December 6.  If anyone is curious, here's the link. And here is a link to a previous post for Memphis charter data on value-added scores.

What jumps out in looking at these reports is that, while the state is creeping forward, the Memphis charters are moving backwards since 2009 in terms of grades.

No Achievement Scores were posted on the State website for New Consortium of Law and Business, Memphis College Prep, and City University School of Liberal Arts.

(Please excuse the weird spacing).

 KIPP Diamond Academy

Academic Achievement Grades

This page displays the 2009 re-start of all comparison reporting for achievement data for
Tennessee’s annual comprehensive educational report card.

Only 2009 converted data are reported and no trend data are available.

Scores and grades may only be compared for 2008-09 and 2009-10. (See Note.)

Grades 3-8: TCAP Criterion Referenced Academic Achievement									View Chart ?
(3 year average)	2009		2010		2011			2011 State
CRT	Score	Grade	Score	Grade	Score	Grade	Trend	Score	Grade	Trend
Math	-	-	50	B	51	B	NC	50	B	+
Reading/Language	-	-	42	D	40	D	NC	49	C	NC
Social Studies	-	-	47	C	49	C	NC	52	B	NC
Science	-	-	39	F	37	F	NC	49	C	NC
Note:   The 2009 baseline provided a fixed transition point prior to the 2009-10 school year implementations of the new curriculum standards and assessments more reflective of national and international student performance in the 21st Century. The 2009 achievement scores and all grades connected with these scores are considered the new baseline for future public reporting. These converted achievement scores and grades are based on restructured calculations and a redefined grade scale that are updated to reflect the current status of educational attainment in the state in 2009. The 2009 change has prohibited comparisons to previous years’ data prior to 2009 for achievement reporting including state, district, and school-level scores and grades. For 2010, the most appropriate and meaningful comparsion would be to 2008-09 and the 2010 State level data.

 Memphis Business Academy

									Grades 3-8: TCAP Criterion Referenced Academic Achievement View Chart ? (3 year average) 2009 2010 2011 2011 State CRT Score Grade Score Grade Score Grade Trend Score Grade Trend Math 43 D 40 D 39 F - 50 B + Reading/Language 43 D 42 D 39 F - 49 C NC Social Studies 44 D 47 C 45 C NC 52 B NC Science 41 D 42 D 37 F - 49 C NC

Memphis Academy of Science and Engineering

Grades 3-8: TCAP Criterion Referenced Academic Achievement									View Chart ?
(3 year average)	2009		2010		2011			2011 State
CRT	Score	Grade	Score	Grade	Score	Grade	Trend	Score	Grade	Trend
Math	39	F	36	F	34	F	NC	50	B	+
Reading/Language	43	D	39	F	35	F	NC	49	C	NC
Social Studies	42	D	41	D	38	F	-	52	B	NC
Science	46	C	41	D	36	F	-	49	C	NC

Memphis Academy of Health Science

Grades 3-8: TCAP Criterion Referenced Academic Achievement									View Chart ?
(3 year average)	2009		2010		2011			2011 State
CRT	Score	Grade	Score	Grade	Score	Grade	Trend	Score	Grade	Trend
Math	43	D	43	D	46	C	+	50	B	+
Reading/Language	45	C	44	D	42	D	NC	49	C	NC
Social Studies	48	C	50	B	50	B	NC	52	B	NC
Science	51	B	48	C	45	C	NC	49	C	NC

 Southern Avenue Charter School

Grades 3-8: TCAP Criterion Referenced Academic Achievement View Chart ? (3 year average) 2009 2010 2011 2011 State CRT Score Grade Score Grade Score Grade Trend Score Grade Trend Math 37 F 36 F 37 F NC 50 B + Reading/Language 37 F 37 F 39 F NC 49 C NC Social Studies 37 F 39 F 40 D + 52 B NC Science 36 F 36 F 36 F NC 49 C NC
Star Academy
Grades 3-8: TCAP Criterion Referenced Academic Achievement View Chart ? (3 year average) 2009 2010 2011 2011 State CRT Score Grade Score Grade Score Grade Trend Score Grade Trend Math 51 B 50 B 52 B NC 50 B + Reading/Language 47 C 47 C 49 C NC 49 C NC Social Studies 41 D 40 D 40 D NC 52 B NC Science 41 D 41 D 39 F - 49 C NC

Promise Academy

Grades 3-8: TCAP Criterion Referenced Academic Achievement									View Chart ?
(3 year average)	2009		2010		2011			2011 State
CRT	Score	Grade	Score	Grade	Score	Grade	Trend	Score	Grade	Trend
Math	-	-	45	C	46	C	NC	50	B	+
Reading/Language	-	-	42	D	42	D	NC	49	C	NC
Social Studies	-	-	36	F	36	F	NC	52	B	NC
Science	-	-	36	F	35	F	NC	49	C	NC

Circles of Success Learning Academy

Grades 3-8: TCAP Criterion Referenced Academic Achievement									View Chart ?
(3 year average)	2009		2010		2011			2011 State
CRT	Score	Grade	Score	Grade	Score	Grade	Trend	Score	Grade	Trend
Math	46	C	40	D	42	D	NC	50	B	+
Reading/Language	45	C	41	D	40	D	NC	49	C	NC
Social Studies	41	D	40	D	38	F	-	52	B	NC
Science	36	F	34	F	32	F	NC	49	C	NC

Soulsville Charter School

Grades 3-8: TCAP Criterion Referenced Academic Achievement									View Chart ?
(3 year average)	2009		2010		2011			2011 State
CRT	Score	Grade	Score	Grade	Score	Grade	Trend	Score	Grade	Trend
Math	45	C	41	D	39	F	-	50	B	+
Reading/Language	45	C	41	D	37	F	-	49	C	NC
Social Studies	44	D	42	D	42	D	NC	52	B	NC
Science	42	D	40	D	37	F	-	49	C	NC

New Charter Study by Mathematica With More Bad News for Corporate Ed Reform

The first national charter school study was conducted in 2009 by CREDO at Stanford, and the co-funders of the study (the Walton Foundation and Pearson) were not enamored by the results. So bad were they for charter school fans that the study, though given skimpy coverage by the LA Times, was never reported by WaPo or the NYTimes, and received minimal coverage from one news magazine, U. S. News and World Report, which obviously did not get the memo:

June 17, 2009 12:58 PM ET | Zach Miners | Permanent Link | Print

On average, charter schools are not performing as well as their traditional public-school peers, according to a new study that is being called the first national assessment of these school-choice options. The study, conducted by the Center for Research on Education Outcomes at Stanford University, compared the reading and math state achievement test scores of students in charter schools in 15 states and the District of Columbia—amounting to 70 percent of U.S. charter school students—to those of their virtual "twins" in regular schools who shared with them certain characteristics. The research found that 37 percent of charter schools posted math gains that were significantly below what students would have seen if they had enrolled in local traditional public schools. And 46 percent of charter schools posted math gains that were statistically indistinguishable from the average growth among their traditional public-school companions. That means that only 17 percent of charter schools have growth in math scores that exceeds that of their traditional public-school equivalents by a significant amount.

In reading, charter students on average realized a growth that was less than their public-school counterparts but was not as statistically significant as differences in math achievement, researchers said.

"We are worried by these results," Margaret Raymond, director of CREDO and lead author of the report, Multiple Choice: Charter School Performance in 16 States, said at a news conference. "This study shows that we've got a 2-to-1 margin of bad charters to good charters."  . . . .

This new study released in Friday's news dump, entitled "Charter-School Management Organization: Diverse Strategies and Diverse Student Impacts," has more bad news for school privatizers who prefer the charter route.  Even though a swarm of urban school colonizers from Gates, Walton, and the New Schools Venture Fund helped set up the parameters for this study in order to get the most favorable outcome, and even though the Gates "research" hothouse, the Center for Reinventing Public Education co-authored the study,  there's enough bad news for charter proponents that mirrors years of previous research on charters that this study, too, has been ignored by the corporate media.  Ed Week had a piece on the new study entitled "Academic Gains Vary Widely for Charter Networks," and Time had a pre-release gloss by corporate spinner extraordinaire, Andy Rotherham.  That was it for coverage, except for a misleading and dissembling press release by Jim Peyser at the New Schools Venture Fund.  And only one of the national charter school associations offered a press release on this big event.  And most telling, the Gates "research" hothouse that co-authored the study, the Center for Reinventing Public Education, does not even mention it anywhere on its website.  Shhhh.

Mathematica led the study, and as their Press Release indicates, the study "was commissioned by NewSchools Venture Fund, with the generous support of the Bill  Melinda Gates Foundation and the Walton Family Foundation." An undisclosed number of the sludge-tank "thought leaders," including Andy Rotherham, carefully set up the parameters for the sample to pump the corporate welfare Charter Management Organizations (CMOs).  These are the corporate non-profit tax sponges preferred by the vulture philanthropy movement.

Even though the names of the CMOs are not listed in the Report, Jim Peyser, insider and hovering point man for the NSVF's involvement in the study, mentions these well-funded total compliance testing camps as representative of the CMOs that were part of the study:  KIPP, Aspire, Achievement First, Noble Network, Uncommon Schools.

Rotherham, now writing for Time, was given, in fact, exclusive access to the Report in order to spin the story the best way possible before the release.  And as lead spinner, Rotherham gave it the ole' college try.  A  few clips with comments:

Rotherham spinning in Time:

The study found that, in general, students at charter-network schools outperform similar students at traditional public schools, although sometimes not by very much.

Findings from the Study's Executive Summary:

Test score impact estimates for the average CMO after two to three years in middle school are positive in all four subjects, but they are not statistically significant.

The overall average impacts mask a great deal of variation among CMOs. Two years after students enroll in the CMOs covered by the impact analysis, they experience significantly positive math impacts in half of these CMOs (11 of 22), while students in about one-third of the CMOs (7 of 22) do significantly worse in math. Similarly, students in nearly half of the CMOs (10 of 22) experience significantly positive impacts in reading, while students in about a quarter of CMOs (6 of 22) experience reading impacts that are significantly negative.  Table 3 shows that half of the CMOs (11 of 22) have significantly positive impacts in math or reading and nine have significantly negative impacts in one or both subjects; 10 of the 22 CMOs have significantly positive impacts in both subjects while only four have significantly negative impacts in both subjects (p. xxvii).

That is, even with all the advantages that charter schools enjoy, and even with the selective culling that took place to create the sample for this study, charters are, on average, doing no better than the public schools that the charterites want to shut down. 

Table 1 from the Report illustrates two of the primary reason that charters have a test performance advantage over public school: charters regularly have fewer students who are English language learners, and fewer students with special needs and disabilities. 


Notice, too, that this study compares charters to the host district average, rather than the schools in the immediate vicinity.  Gary Miron and others have noted elsewhere that these district comparisons often mask even larger percentages of ELL and SPED children in the poorest communities where charters replace public schools.

And how about class size differences between charters and their public counterparts in the host districts? 

Class sizes and pupil-to-instructor ratios are also smaller in CMO schools than in their host districts. The average pupil-to-instructor ratios in math and reading are about 20.9 students per instructor; by contrast, in comparison schools the ratios are 23.5 in math and 23.2 in reading (p. xxiv)

And how about that "creaming" reputation that charters have, drawing as they do students with higher achievement to begin with, thus making any subsequent comparisons to public school students skewed? Remember that we know from Table 1 above that the sample for this study was disproportionately African American and Hispanic when compared to the host district.  From the Report:

Students Entering CMO (Middle) Schools Typically Have Prior Achievement Levels That Are Similar to the Local Average and Somewhat Higher Than the Local Average For Black And Hispanic Students

. . . most CMOs attract somewhat higher achieving students of color relative to those served by their host districts.  Thirteen of 22 CMOs in our sample serve black students who had significantly higher average pre-entry reading test scores than the averages for their black peers in the host district; only two CMOs served black students with scores significantly lower than those of black students locally. Likewise, the pre-entry reading scores of Hispanic students in 13 of 23 CMOs were significantly higher than Hispanic averages locally, and only three CMOs served Hispanic students with significantly lower baseline reading achievement than that of other Hispanic students in their districts. The percentages are similar for reading test scores.  Thus, while CMOs attract a disproportionate number of black and Hispanic students, these students tend to have higher test scores on average when they enter the CMO than their black and Hispanic peers in the host districts (pp. 19-20). 

The concluding section of the study takes up this subject again, in discussing peer effects (my bolds):

. . .because CMOs operate schools of choice, the families they attract are different in both measurable and unmeasurable ways, which may give rise to peer effects. The selection process of students is driven in part by who learns about and chooses to apply to CMO schools. It is possible that the parents or students who end up enrolling in some CMO schools are more motivated or have other assets. In addition, CMOs can encourage certain families to apply or enroll in their school; even those with random lotteries can target their recruitment efforts and ask students to sign agreements to attend regularly and do their homework. An individual student may benefit from being in the same school and classroom with other students with higher levels of motivation or parental support. If peer effects are contributing to CMO impacts, this does not mean that our impacts are improperly measured. Indeed, our experimental results suggest the impacts are accurate. But it could affect our understanding of the mechanisms behind the impacts: Peer effects may explain why CMO students do better than they would have had they been placed in a school or classroom where there are fewer students like themselves. If that turns out to be true, it would also have important implications for policy: Similar effects might not be achieved, for example, if CMO practices were directly applied to conventional public schools that are not schools of choice. While peer effects can be challenging to estimate, future research should explore their importance (p. 75). 

Besides Rotherham, the other propaganda spin comes from Jim Peyser of the venture philanthropy outfit, NewSchools City Funds, a spinoff of NSVF.  Peyser is intent on making the claim that larger is better (volume, volume, volume!) and that TFA is better (cheaper, cheaper, cheaper!):

There is a statistically significant association between math achievement in CMOs and the percentage of new teachers coming from Teach For America and Teaching Fellows teachers. This finding not only demonstrates the value of TFA to the charter school sector, but it underscores the importance of alternative teacher preparations programs in general to addressing public education’s human capital challenges. Other staffing decisions (including opportunities for tenure) were not associated with positive impacts.

This is what the Report actually says (my bolds):

Math impacts are higher among CMOs that rely more heavily on TFA and the Teaching Fellows programs as sources of new teachers. Specifically there is a statistically significant association between math impacts and the percentage of new teachers from these two sources, both of which tend to recruit and provide some training to recent graduates of highly selective colleges. One should be cautious about placing substantial weight on this finding because this is one of the many secondary hypotheses tested and the positive association could be due to random chance (p. 69).

Another of Peyser's misleading conclusion that he would like to see in the Report has to do with size of CMOs (Peyser's bolds):

The strongest CMOs tend to be larger than the lower performing ones, countering a long-held hypothesis that scale and quality are incompatible. 

And yet the Report makes a specific warning against drawing the conclusion that Peyser draws (my bolds):

Large CMOs in our sample tend to have positive impacts, while small CMOs are more likely to have negative impacts. This might indicate that funders have had some success in supporting the expansion of CMOs that are more effective.  In particular, eight of the 12 large CMOs (those operating more than 8 schools in 2009-10) have significant positive impacts in at least one subject, while only 3 of the 10 small CMOs (those operating 8 or fewer schools in 2009-10) have significant positive impacts in at least one subject. Meanwhile, only 2 of 12 large CMOs have significantly negative impacts in at least one subject, while 7 of 10 small CMOs have significantly negative impacts in at least one subject. CMOs that have positive impacts in both reading and math operate an average of 12 schools, while those with negative impacts in both subjects operate an average of 6 schools. Despite this pattern effectiveness is not related to size in a linear way: Correlations between math and reading CMO impacts and CMO size are not statistically significant (p. 58). 

. . . .

We also looked at whether absolute CMO growth (change in the number of schools operated by the CMO between fall 2004 and fall 2009) and relative CMO growth (the number of schools operated by the CMO in fall 2009 divided by the number of schools operated by the CMO in fall 2004) are associated with two-year impacts in math and reading. In both of these cross-sectional analyses, we found no statistically significant associations (p. 59).

Finally, there are other findings of this study that you would never see mentioned by Rotherham, Peyser, or Arne Duncan:  no positive impact could be attributed to performance-based teacher compensation, singular curricular or instructional approaches (think Common Core), or the constant use of "formative" testing to prepare for more testing:

Several other notable CMO-level characteristics do not show significant relationships with impacts.

We found no significant relationship between impacts and three other factors that we posited might contribute to student achievement. Specifically, impacts are not correlated with (1) the extent to which CMOs define a consistent educational approach through the selection of curricula and instructional materials, (2) performance-based teacher compensation, or (3) frequent formative student assessments (although impacts are larger when teachers frequently use student test results to modify lesson plans). Nor are impacts significantly associated with school or class sizes.  Math impacts are positively correlated with more hours of annual instruction, but this relationship appears to be largely due to the association of instructional time with behavior policies and coaching. We ran multivariate regressions of impacts on key practices that were significantly associated with impacts in bivariate regressions. In the multivariate regressions, the association between impacts and instructional time declined substantially and became not statistically significant (p. xxx). 

One has to wonder what it will take for this latest Gates/Walton/Broad failure to become too obvious to ignore.  The elephant trumpets, the corporate spinners and scammers double down, and the politicians concur that In God We Trust.

Tennessee Charter School Data Gives Good Reasons to Go Slow or Begin Backing Up

On Monday the Commercial Appeal in Memphis carried a story on efforts by local officials to take a deep breath and assess the charter school situation as these same officials work to create a Metro school system out of separate city and county systems.  This seems to be a reasonable proposal, but, of course, the hedge funders, vulture philanthropists, Koch Brothers puppet, Gov. Haslam, and other corporate welfare bottom feeders are in an instant lather to block any effort to call a timeout on the spread of the intensely segregated (by race and class) exclusionary charter schools in Memphis, with their punishing protocols that no middle class parent would consider for their own children.

Greg Thompson, former investment banking analyst and new chief of the Tennessee Charter School Incubator, is leading the charge for the uninterrupted flow of public money into these corporate charters.  He and the front office of KIPP, Inc. and TFA, Inc. argue that the spread of these urban chain gangs should remain unhindered by planning, evaluation, or just a little common sense. To bolster their insistent whine, they refer to "a report issued this month on behalf of the Center for Research on Education Outcomes (CREDO) at Stanford University," even though the State has yet to produce such a report despite repeated requests to various offices within Commissioner Kevin "TFA" Huffman's Department of Education.

So far, all that we have is a joint press release from TN's Achievement School District/New Schools for New Orleans, both of whom are supposedly contracting with CREDO for some kind of services to help determine which schools get the federal "innovation" grants.  The Press Release has never even been posted to the Web, even though it is dated September 30, 2011 (email me if you want a copy of it).  From the Press Release:

   So if a school does not have three years of data, then its "effect size" cannot be determined, right?   Again, from the Press Release:

Now here is the first oddity in this.  Of the 15 Memphis charter schools on the list of high flyers from 2008-2011, there are 5 schools with only one year of data, having opened only in the Fall of 2010:

Memphis School of Excellence

New Consortium of Law and Business

Circles of Success Learning Academy

Omni Prep Academy

Veritas College Preparatory Charter School  

Two others, Freedom Preparatory Academy and Promise Academy, have only been in operation for two years and, thus, have only 2 years of data.  Promise Academy is listed as the school with the highest math growth, and Veritas, just opened last year, is listed the school with highest reading growth.

Second oddity: Three of the schools on the "CREDO" list have simply awful results on the state TCAP tests, which are supposedly used by "CREDO" to arrive at the list of schools that are outperforming public schools:

Memphis Academy of Health Sciences in Memphis

Soulsville Charter School

Memphis Business Academy

The TCAP Composite Scores for these three schools are below, as they appear on the State's public site:

2011 School Value Added

Memphis Academy Of Health Sciences in Memphis

TCAP Composite

In 2010, the Tennessee Department of Education implemented significant changes in state testing. For the purposes of these analyses, the new testing has been equated so it can be reported on the same scale. The Growth Standard reflects these changes and is based on statewide student achievement in 2009.

Estimated School Mean NCE Gain
Grade	6	7	8	Mean NCE Gain over Grades Relative to
Growth Standard	0.0	0.0	0.0
State 3-Yr-Avg	0.2	-0.4	-0.6	Growth Standard	State
2009 Mean NCE Gain	12.1 G*	1.6 G*	-4.4 R*	3.1	3.4
Std Error	1.1	0.8	0.8	0.5	0.5
2010 Mean NCE Gain	8.8 G*	-5.5 R*	-7.3 R*	-1.3	-1.1
Std Error	1.0	0.8	0.7	0.5	0.5
2011 Mean NCE Gain	11.1 G*	-0.9 R	1.6 G*	3.9	4.2
Std Error	1.2	0.8	0.8	0.5	0.5
3-Yr-Avg NCE Gain	10.6 G*	-1.6 R*	-3.4 R*	1.9	2.2
Std Error	0.6	0.5	0.4	0.3	0.3
Estimated School Mean NCE Scores
Grade	6	7	8
State Base Year (2009)	50.0	50.0	50.0
State 3-Yr-Avg	47.1	47.0	46.7
2008 Mean
2009 Mean	48.6	48.4	41.6
2010 Mean	44.1	43.1	41.1
2011 Mean	46.0	43.2	44.7

G*	Estimated mean NCE gain is above the growth standard by at least 1 standard error.
G	Estimated mean NCE gain is equal to or greater than growth standard but by less than 1 standard error.
Y	Estimated mean NCE gain is below the growth standard by 1 standard error or less.
R	Estimated mean NCE gain is more than 1 standard error below the growth standard but by 2 standard errors or less.
R*	Estimated mean NCE gain is below the growth standard by more than 2 standard errors.

Soulsville Charter School in Memphis

TCAP Composite

In 2010, the Tennessee Department of Education implemented significant changes in state testing. For the purposes of these analyses, the new testing has been equated so it can be reported on the same scale. The Growth Standard reflects these changes and is based on statewide student achievement in 2009.

Estimated School Mean NCE Gain
Grade	6	7	8	Mean NCE Gain over Grades Relative to
Growth Standard	0.0	0.0	0.0
State 3-Yr-Avg	0.2	-0.4	-0.6	Growth Standard	State
2009 Mean NCE Gain	2.6 G*	-0.5 Y	-0.4 Y	0.5	0.8
Std Error	1.3	0.9	0.9	0.6	0.6
2010 Mean NCE Gain	-8.4 R*	1.1 G*	-3.5 R*	-3.6	-3.3
Std Error	1.9	1.0	0.9	0.8	0.8
2011 Mean NCE Gain	5.7 G*	-2.9 R*	-2.3 R*	0.1	0.4
Std Error	1.7	1.0	0.9	0.7	0.7
3-Yr-Avg NCE Gain	-0.1 Y	-0.8 R	-2.1 R*	-1.0	-0.7
Std Error	0.9	0.6	0.5	0.4	0.4
Estimated School Mean NCE Scores
Grade	6	7	8
State Base Year (2009)	50.0	50.0	50.0
State 3-Yr-Avg	47.1	47.0	46.7
2008 Mean
2009 Mean	35.6	40.5	44.7
2010 Mean	31.3	36.7	37.0
2011 Mean	44.7	28.4	34.4

G*	Estimated mean NCE gain is above the growth standard by at least 1 standard error.
G	Estimated mean NCE gain is equal to or greater than growth standard but by less than 1 standard error.
Y	Estimated mean NCE gain is below the growth standard by 1 standard error or less.
R	Estimated mean NCE gain is more than 1 standard error below the growth standard but by 2 standard errors or less.
R*	Estimated mean NCE gain is below the growth standard by more than 2 standard errors.

2011 School Value Added

Memphis Business Academy in Memphis

TCAP Composite

Estimated School Mean NCE Gain
Grade	6	7	8	Mean NCE Gain over Grades Relative to
Growth Standard	0.0	0.0	0.0
State 3-Yr-Avg	0.2	-0.4	-0.6	Growth Standard	State
2009 Mean NCE Gain	6.6 G*	-2.9 R*	-2.1 R	0.5	0.8
Std Error	0.9	0.9	1.2	0.6	0.6
2010 Mean NCE Gain	7.0 G*	-5.4 R*	-10.0 R*	-2.8	-2.6
Std Error	1.1	0.8	0.8	0.5	0.5
2011 Mean NCE Gain	8.9 G*	-3.4 R*	-2.5 R*	1.0	1.2
Std Error	1.2	0.8	0.7	0.5	0.5
3-Yr-Avg NCE Gain	7.5 G*	-3.9 R*	-4.9 R*	-0.4	-0.2
Std Error	0.6	0.5	0.5	0.3	0.3
Estimated School Mean NCE Scores
Grade	6	7	8
State Base Year (2009)	50.0	50.0	50.0
State 3-Yr-Avg	47.1	47.0	46.7
2008 Mean
2009 Mean	45.2	42.6	37.2
2010 Mean	43.0	36.6	31.7
2011 Mean	42.1	32.4	31.4

G*	Estimated mean NCE gain is above the growth standard by at least 1 standard error.
G	Estimated mean NCE gain is equal to or greater than growth standard but by less than 1 standard error.
Y	Estimated mean NCE gain is below the growth standard by 1 standard error or less.
R	Estimated mean NCE gain is more than 1 standard error below the growth standard but by 2 standard errors or less.
R*	Estimated mean NCE gain is below the growth standard by more than 2 standard errors.

Finally, consider the case of KIPP, which is listed as #2 on the list of "CREDO" high flyers.  It is hard to say which public schools they were matched with, but when matched with the state average, they did not do so hot, or at least there were no detectable differences (NDD) between KIPP's Explore Math scores or their End of Course Algebra I scores when compared with the State Averages.  Again, from the State's public site: