One of the Iowa Department of Education’s favorite phrases is ‘evidence-based,’ as in “schools and educators should be adopting ‘evidence-based’ practices.” That makes sense on its surface, right? So when the DE issued a press release today touting two Iowa districts that had adopted the National Institute for Excellence in Teaching’s (NIET) Teacher Advancement Program (TAP) – which focuses on teacher leadership and performance-based teacher compensation – I started digging around in Google Scholar for research about the program. I don’t live in the world of teacher leadership/compensation and thus don’t know much about TAP other than that it appears to support the DE’s policy proposals this legislative session (and that Dr. Brad Buck, an amazing school leader that I greatly respect, is the superintendent of one of the two districts). For all I know, TAP could be amazing or it could be puffery…
What did I find?
- A study from Stanford University finding small positive student learning impacts in math and none in reading
- A study from Vanderbilt University finding positive student learning impacts in math at the elementary level and negative student learning impacts in math in grades 6-10 (when controlling for selection bias)
- A dissertation in Louisiana that found no student learning impact in elementary English / Language Arts
- A study by Mathematica finding no student learning impact in math, reading, or science in the Chicago Public Schools (after 4 years)
And, of course, on its web site I found research from NIET itself claiming that TAP is awesome. Indeed, there’s a plethora of publications on TAP’s web site from NIET; agencies such as the Milken Family Foundation, the Joyce Foundation, or the Gates Foundation that have funded TAP; and ideologically-oriented educational organizations. But despite TAP’s claims of ‘a decade-long track record of growth and success in raising student achievement in high-need schools‘ (page 19), the evidence from independent researchers is decidedly more mixed and leans much more pessimistic when it comes to student learning outcomes.
So is TAP ‘evidence-based’ or puffery? Given the greater preponderance of negative findings by outside, independent scholars, it appears to be extremely arguable. Given that uncertainty and its own call for ‘evidence-based’ practices, should DE be basing its 2013 education reform package on the TAP model? (and should it be touting the model via press releases to the Iowa public and media, which know even less about all of this than I do?)
We already know that DE is willing to ignore decades worth of peer-reviewed research when it conflicts with its policy advocacy (see, e.g., recent incidents regarding 3rd grade retention and cutting elementary school recess). I don’t know if TAP falls into this category of willful blindness or not. But I am wondering if the studies listed above were ever presented to those in decision-making positions so that they could make a truly informed decision.
I hope that the initiatives go well for the two Iowa districts that are trying TAP. Their educators are going to invest a great deal of time, energy, and money in the model. Hopefully they will see the results for which they are striving. Until I see further independent research supporting TAP (send it if you’ve got it; maybe I missed some!), right now I’m less sanguine about whether we should be basing tens of millions of dollars statewide to implement the model here in Iowa.
Got any thoughts on this?
[Note that there’s also a bigger question here that’s always worth considering: What should we do if/when our own governmental agencies fail to apply the same standards to themselves that they wish to apply to us?]
Image credit: What’s your superpower?
In Saydel we are very interested in TAP because of the system of supports that it will provide for our teachers as well as the evidence that does exist that when done well, student achievement results are improved (you mention NIET data, Gates, and others).
For us the ability to bring in Master Teachers to provide weekly, job-embedded professional development is a high point. The PD that they will facilitate and lead is collaborative, follows a continuous improvement cycle process (review of data, field tested strategies taught to teachers, etc.) and there is on-site coaching and feedback – with the help of mentor teachers and building leadership. There is also a rubric that is used to guide the feedback that does a pretty decent job of delineating the skills, behaviors, and attributes of effective instruction and teachers.
In addition to those supports, we went on a number of site visits to TAP schools and one of the things we heard repeatedly was the change in the culture of the building that occurs with TAP. In the hallways and at passing times the teachers were talking about how to further improve lessons, instructional practices, and student thinking. Essentially what we hard was a transition from a focus on teaching to a focus on learning as the prized outcome in the system. As importantly, there was a positive energy about what a group of committed people can do with the resources for improvement.
In addition to the support for teachers, there are weekly leadership team meetings (building admin., master and mentor teachers) in the buildings. In those meetings the building principal facilitates and leads discussions, using the same continuous improvement cycle as occurs with the teachers, on what is being done regarding inter-rater reliability of use of the rubrics, and much more.
Hopefully what people hear in all of this is that we will have a systemic and systematic approach to improving student learning. This will be accomplished through frequent, focused, collaborative, job-embedded, and data-guided professional development. It is also supported in implementation through frequent, formative feedback guided by solid descriptors of quality practice. That sounds like a lot of education jargon in re-reading it, except that I believe it is pretty powerful stuff (how’s that for a shift from the jargon!).
Your concerns related to TAP are certainly reasonable Scott (thank you for the kind sentiments – they are more than generous and I have the good fortune of working with some great people and a very supporting and visionary Board) and there are instances where TAP has not been as successful as the leadership at NIET would prefer.
Having said all of that, we believe that TAP, through a focused system of supports, will allow us to translate the hard work that is already occurring in most instances in the district, into tangible student outcomes.
I’ve spent a lot of time researching the TAP model after the two districts were awarded the grant and completely agree with you, Scott. Every TAP school is supported by huge amounts of grant money and when the money runs out the program can no longer be supported. Both districts have their own reasons for getting into the program, but one of the main factors is that both districts had not been receiving adequate funding to support their existing programs. Other schools using the program around the country speak highly of the quality of the professional development, but are less enamel red of the teacher pay plan and have indicated it did little to improve student achievement. I guess time will tell how this plays out.