State department educational policy advocacy: “Evidence-based” or puffery?
One of the Iowa Department of Education’s favorite phrases is ‘evidence-based,’ as in “schools and educators should be adopting ‘evidence-based’ practices.” That makes sense on its surface, right? So when the DE issued a press release today touting two Iowa districts that had adopted the National Institute for Excellence in Teaching’s (NIET) Teacher Advancement Program (TAP) – which focuses on teacher leadership and performance-based teacher compensation – I started digging around in Google Scholar for research about the program. I don’t live in the world of teacher leadership/compensation and thus don’t know much about TAP other than that it appears to support the DE’s policy proposals this legislative session (and that Dr. Brad Buck, an amazing school leader that I greatly respect, is the superintendent of one of the two districts). For all I know, TAP could be amazing or it could be puffery…
What did I find?
- A study from Stanford University finding small positive student learning impacts in math and none in reading
- A study from Vanderbilt University finding positive student learning impacts in math at the elementary level and negative student learning impacts in math in grades 6-10 (when controlling for selection bias)
- A dissertation in Louisiana that found no student learning impact in elementary English / Language Arts
- A study by Mathematica finding no student learning impact in math, reading, or science in the Chicago Public Schools (after 4 years)
And, of course, on its web site I found research from NIET itself claiming that TAP is awesome. Indeed, there’s a plethora of publications on TAP’s web site from NIET; agencies such as the Milken Family Foundation, the Joyce Foundation, or the Gates Foundation that have funded TAP; and ideologically-oriented educational organizations. But despite TAP’s claims of ‘a decade-long track record of growth and success in raising student achievement in high-need schools‘ (page 19), the evidence from independent researchers is decidedly more mixed and leans much more pessimistic when it comes to student learning outcomes.
So is TAP ‘evidence-based’ or puffery? Given the greater preponderance of negative findings by outside, independent scholars, it appears to be extremely arguable. Given that uncertainty and its own call for ‘evidence-based’ practices, should DE be basing its 2013 education reform package on the TAP model? (and should it be touting the model via press releases to the Iowa public and media, which know even less about all of this than I do?)
We already know that DE is willing to ignore decades worth of peer-reviewed research when it conflicts with its policy advocacy (see, e.g., recent incidents regarding 3rd grade retention and cutting elementary school recess). I don’t know if TAP falls into this category of willful blindness or not. But I am wondering if the studies listed above were ever presented to those in decision-making positions so that they could make a truly informed decision.
I hope that the initiatives go well for the two Iowa districts that are trying TAP. Their educators are going to invest a great deal of time, energy, and money in the model. Hopefully they will see the results for which they are striving. Until I see further independent research supporting TAP (send it if you’ve got it; maybe I missed some!), right now I’m less sanguine about whether we should be basing tens of millions of dollars statewide to implement the model here in Iowa.
Got any thoughts on this?
[Note that there's also a bigger question here that's always worth considering: What should we do if/when our own governmental agencies fail to apply the same standards to themselves that they wish to apply to us?]
Image credit: What’s your superpower?