Today the Iowa Department of Education (DE) released a report on achievement levels in Iowa compared to other states. The report also focuses heavily on closing the significant achievement gaps that exist in our state. Here are some very quick reactions that I have to the report…
- The emphasis on better meeting the learning needs of traditionally-underserved student populations is absolutely necessary. Educationally and otherwise, we often have neglected students of color, students in poverty, students who are English language learners, and students with disabilities.
- It’s hard to argue with proposed educational solutions that are focused on instruction, proven effective, and scalable, but I think that there is an accompanying, unstated concern: How should we think about educational initiatives that need to occur but don’t have ‘significant bodies of evidence’ behind them yet? For example, we live in a digital world and we know that students need to be fluent with the technologically-transformed information spaces of our time. And yet the peer-reviewed research to support this move isn’t there yet. It’s just sort of common sense: all we have to do is look around and realize that this is a need. Given the lack of ‘research,’ however, does that mean we don’t do it?
- I wish that the report’s initial framing of the issues focused on the substantial changes that are occurring in the ways that we learn, citizenship needs in an increasingly-complex democracy, and other concerns related to life success beyond just economy/workforce issues. The latter are definitely important, but preparing future employees is not schools’ primary societal function.
If we’re going to work on raising scores and closing achievement gaps, let’s do our best to focus on assessments that matter. Right now we seem to be concerned mostly about average scores on assessments of primarily lower-level thinking. It’s also worth noting that our own National Research Council has found that decades of test-based incentives have done nothing to improve student learning outcomes. In fact, high school exit exams as configured in many states actually decrease graduation rates without concurrent increases in achievement.- Despite the sturm und drang around Iowa’s NAEP scores, we must recognize that there are no objective criteria and/or research-based evidence behind the cut scores for the different NAEP proficiency levels. The cut scores are set by committee and thus are inherently political. The NAEP benchmarks have been vociferously criticized by the National Academy of Sciences, the Government Accounting Office, the National Academy of Education, and many, many others. The designers of NAEP freely admit that the cut scores and levels are arbitrary.
- Is our concern merely about raising Iowa students’ academic performance levels or is it necessary that we also BEAT OTHER STATES AND NATIONS? The rhetoric that’s flying around about Iowa ‘slipping to the middle of the pack’ seems very concerned about the latter. It’s also worth noting that most of the countries to which we negatively compare Iowa also wouldn’t do very well on NAEP.
- ‘Rapid iteration,’ ‘living in perpetual beta,’ and other ideas related to quickly trying things, getting feedback to see if they worked, and adjusting course accordingly are all extremely important, particularly in a rapidly-changing world. As such, Response to Intervention (RTI) is a great process, particularly if feedback loops are short in time. But the RTI process also traditionally has been deeply rooted in notions of low-level cognitive work. Terms like ‘progress monitoring’ and ‘data-based decision-making’ are typically employed by educators in service of factual recall and procedural knowledge regurgitation. Turning those ideas toward higher-order thinking outcomes is going to be a lot of work in most school systems.
- We need to be careful that we don’t turn ‘fidelity of implementation’ and ‘best practices’ into cookie-cutter instructional recipes and/or scripted lessons (as has occurred in many districts across the country). The report says that we need to ‘eliminate variability in instruction.’ I understand the sentiment behind that phrase but we need to be very wary of simplistic, stupid solutions to this issue.
- The underlying premise of the report (and its accompanying policy proposals that we’ll see in the near future) is that education is a system amenable to fairly mechanistic solutions: put in place the right inputs, processes, and feedback loops and we’ll get the desired outcomes. Classic systems theory stuff. Learning and teaching are inherently messy domains, however, that often defeat externally-imposed procedures and expectations. As other nations show, we can improve student learning outcomes with thoughtful, purposeful changes, but we should be prepared for a lot of messiness along the way.
- There’s a difference between ‘differentiation’ (as proposed in the report’s description of RTI) and ‘personalization’: see McClaskey & Bray’s chart on this. Differentiation is good, but a move away from primarily teacher-directed learning environments also is needed.
Final thoughts
Will teacher quality initiatives, the Iowa Core, and better deployment of RTI improve student learning outcomes in Iowa? Probably, at least somewhat. Are we going to see massive shifts in student learning outcomes in Iowa as a result of these? Probably not. These are school-focused interventions promulgated by the state department of education, and they’re all likely to have some positive impact. But they’re not enough. The research is very clear that roughly 80% of student learning outcomes is a result of NON-school factors. If we’re truly concerned as Iowa citizens and policymakers about improving student learning outcomes and closing achievement gaps, we’ll pay attention to the 80%, not just the 20%, just as most other ‘higher-performing’ nations have done. That means looking beyond the Department of Education for solutions.
Take some time to read over the report. What are your reactions?
Well stated, and yes, proven effective and scalable is a good start. But it must be more than taking something “research based”, “implement with fidelity”, and whaaala, you have the desired outcome. We often take a quantitative approach to a qualitative problem as it assumes we have controlled for the variables. The fact is, we don’t control for them, we just ignore them because we see them as out of our control. The end result is policy that is less than effective.
Scott, excellent commentary as always, but I find portions of the report totally absurd and self-serving.
Look at the #1 Objective – 1. Teacher quality (“the who”): Focuses energy on ensuring that the best and
brightest teaching candidates are recruited and supported, and ensuring that those
who do enter the profession have the highest-quality learning experiences that result
in highly skilled professionals.
Oh Really?
98% of the public school teachers in Iowa are Unionized. Firing rates are 1.3% vs 9.8% for the private school sector. IOWA’s state teacher’s union collects $14.4 mil in dues annually and contributes $1.8mil to in-state political campaigns to ensure status quo. The National Council on Teacher Quality Report Card: Iowa Teacher Policy gave out the following grades:
• Delivering well prepared teachers: D
• Expanding the pool of teachers: D
• Identifying effective teachers: D
• Retaining effective teachers: C-
• Exiting ineffective teachers: D+
Your point # I The underlying premise of the report (and its accompanying policy proposals that we’ll see in the near future) is that education is a system amenable to fairly mechanistic solutions… isn’t this back to the future? Where we created our education framework based on industrialized ideas? Taught children in batches? Same system…same outcomes? That is 180% from what many of our educational thought leaders are saying today.
I applaud you for your point about innovation and scalability. So we will only implement programs that have a proven track record? Ask the former employees at KODAK how that philosophy is working for them.
Hi Scott,
As always, appreciate your insight. Some reactions to your writing…
1. Your point on innovation is well taken. We should always be working toward developing innovative education practices, empirically validating them, and (if they work) trying to take them to scale. It is important to note that the RtI model doesn’t discourage innovation, nor does relying on “evidence-based” approaches. There is absolutely an important place for innovation – but for those things we try to scale across the entire system, we want to have a high degree of certainty, or confidence, they are going to work and work in the direction we expect. RtI is a proven high reliability instructional approach. It’s exactly the kind of instructional approach you want to double down on.
2. Regarding the purpose of education, you state that education’s primary purpose is not preparing employees. This is a normative statement, but I’ll cut you some slack since this is YOUR blog! There are a number of important purposes to education including citizenship development, personal growth, social justice and equity, pure academic exploration … and I could go on. It’s difficult to argue that any of these aren’t important – but so is the economic-driver aspect of our work.
3. When it comes to the assessments used for RtI we are talking about formative measures that are customized to the task at hand. You lump all assessments into the tired box of ‘low-level-fact-recall-regurgitation-standardized-measures’ yada yada yada… When it comes to RtI, many of the formative measures relate to basic literacy and numeracy. While I certainly share your point about the need for better and more sophisticated measures, these are at a very micro-instructional level and it’s critically important kids get this stuff so they can access the higher level work. You go on to riff about “test-based-incentives” and the like – note again that in the context of RtI these are FORMATIVE instructional measures. Let’s not cast too-wide a net when we are blowing off steam about assessments. The assessments used for RtI are very different in form, function, and use than those used for high stakes accountability. Here, we are talking about the former, not the latter.
4. On NAEP, this is a high level system measure designed to give an indication of overall performance across states. Because of that use, it doesn’t really matter so much what the cut point is so long as it’s stable across time and across the systems being compared. The main goal is system comparability here – a minor but important point.
5. I’d like to make a few points about the 80/20 split you continue to cite regarding outside/inside factors that relate to education which is becoming a common element of your shtick. Could you cite specific research that supports this statistic? I’ve never seen any statistical model that predicts 100% of the variance in student achievement so I’m skeptical. Further, to get anywhere near this level of sophistication, you’d need to be relying on a standardized measure with some kind of value-added analysis technique (multiple statistical controls including prior student performance and longitudinal data). Given your preponderance to cast harsh aspersions on both standardized-measures and value added, it seems a little self-serving a duplicitous to use it here to advance your point. Clarify?
To be clear, I take absolutely no issue with the point you are harping on, that poverty has an impact on education and we need to be doing all we can to mitigate its deleterious effects. You are preaching to the choir on this! That work is important and in our country (and our state) schools are put on the front-line in our effort to address child poverty. This point granted, the existence of poverty does nothing to mitigate the importance of making sure we (as educators) provide a quality, high expectation, and caring education system for every child. I think you do under-estimate how efficacious our educators and our efforts can be in making significant and meaningful differences in the lives of children. I don’t think this is what’s in your heart, but let’s make sure we don’t sell our educators short in terms of how important they can be. A system that works to mitigate poverty but dumps kids in a low expectation and low quality school is no answer either. Let’s watch out for these sort of false dichotomies of saying it’s poverty or it’s education as that debate wastes a lot of energy that we should be directing at how we get much better at both of these.
I look forward to your response, and best holiday wishes to you and your family.
JG