[this blog post is a follow-up response to the Twitter exchange with Van Schoales posted below]
Hi Van.
Although I’ve admired your work for years, you and I have never met, which means that we don’t have a relationship to lean into. Seth Godin reminds me that ‘if your audience isn’t listening, it’s your fault, not theirs.’ So please take this post as a very-public apology for whatever anger or defensiveness I sparked by my tweet. I’m sure that it could have been worded more artfully, and I regret not phrasing it in a way that maybe would have been received better. I offer this longer-form blog post as an attempt to bypass the lack of nuance available in 280 Twitter characters. I tend to do much of my thinking out loud in front of others, because their feedback makes me smarter.
I don’t know if you had a chance to read my previous blog post, Much ado about NAEP, but I tried to make two key points. The first was that the timing of the March 2022 NAEP tests should give us pause when interpreting the results. Most educators in America probably would tell us that, as difficult as the 2020-2021 school year was, the 2021-2022 year was even tougher. Kids and families still were dealing with incredible trauma, children showed us daily in classrooms (or by their absences) that they needed more support, and we didn’t do a great job of effectively serving large numbers of our young people. Then we had another big COVID spike right before the NAEP administration, and we have absolutely no idea how that impacted student test-takers except that probably a whole bunch of them (and their families) were ill, absent, struggling, etc. during the months that immediately preceded the test. We also know that these things were most true for our least-resourced children. All of this together feels like a reason to take the 2022 NAEP results not just with a grain but a giant boulder of salt. It’s difficult enough in normal times to help our students feel motivated for standardized tests. It’s even tougher given the March 2022 context and when NAEP results don’t mean anything to students personally.
The second point that I tried to make was that ‘Proficient’ on NAEP doesn’t mean what most folks think it does. To quote Tom Loveless, former director of the Brown Center on Education Policy at the Brookings Institution, in most states the Proficient label is “significantly above” what most state policymakers and assessments deem as ‘grade level,’ and the more accurate NAEP proficiency level for most states would be closer to Basic. This is rarely if ever mentioned in media coverage of NAEP results. Accordingly, most people tend to interpret ‘proficient’ in the ordinary sense of the word rather than in an aspirational sense, which is why I mention it whenever I think it is relevant (e.g., in my tweet). As I said in my blog post post, we can have some interesting discussions about whether Basic or Proficient is the right NAEP target for states, but we should at least recognize that Proficient is very aspirational in most parts of the country.
Here in Colorado, about 75% of 4th graders met the Basic NAEP standard in math for 2022, and about 36% met the Proficient standard. In 8th grade, about 63% of Colorado students met the Basic NAEP standard in math in 2022, and about 28% of students met the Proficient standard. Those results are essentially equivalent to the 2022 national NAEP averages for math. For reading, about 68% of Colorado 4th graders met the Basic NAEP standard in 2022, and about 38% met the Proficient standard. For 8th grade, about 73% of students met the Basic NAEP standard for 2022 in reading, and about 34% met the Proficient standard. Colorado students’ NAEP results in reading were a few percentage points higher than the 2022 national averages. As many have noted, all of the scores for both Colorado and the nation are down from 2019, which is to be expected.
In my blog post, I also quoted Loveless’ statement that, because the NAEP Proficient standard is so aspirational, “If high school students are required to meet NAEP proficient to graduate from high school, large numbers will fail. If middle and elementary school students are forced to repeat grades because they fall short of a standard anchored to NAEP proficient, vast numbers will repeat grades.” [emphasis added]. We also have evidence that similar percentages of students in nominally higher-performing countries also would have trouble meeting the NAEP Proficient mark. NCES has done the work of mapping Colorado’s state standards for proficiency to NAEP equivalent scores. For math, Colorado’s standard is well above NAEP Proficient in 4th grade and close to Proficient in 8th grade. For reading, Colorado’s standard is much closer to Proficient than Basic in both 4th and 8th grade. Colorado’s standards clearly are more aspirational than those of most other states. Accordingly, fewer Colorado students will be deemed ‘at grade level’ than if our benchmarks were set closer to those elsewhere.
School lunch not eligible: 76% Basic, 42% Proficient
School lunch eligible: 51% Basic, 16% Proficient
Yikes! Those performance gaps are both troubling and persistent! They’re also similar to the other large city districts that were sampled, better than some and worse than others. Denver basically is in the middle of the pack for the 2022 NAEP sample of large city school districts.
To quote my own tweet, these performance “divides continue to be of concern.” Like you, I believe that DPS should be extremely transparent about those performance gaps. Also like you, I hope that DPS identifies publicly some concrete plans and actions to remedy its existing equity issues. Additionally, I’m cognizant of the difference between NAEP performance gaps and NAEP performance levels (which is what I was trying to say, albeit artlessly, in my tweet). If Colorado makes it harder for students to be deemed ‘proficient’ than in most other states, of course we’re going to say, “look, fewer kids are proficient!” That’s how we set up the system in the first place (and, once again, we can have a rich discussion about where the line should be set for proficiency). We’re also probably going to say that fewer students are ‘proficient’ in a large urban school system because, sadly, that’s basically the pattern that we see in big city school districts all across the country. Equity gaps are large and persistent in America for students of color, who live in poverty, whose primary language isn’t English, or who have a disability, and the past couple of decades of school reform haven’t done much to alter those. DPS isn’t doing great on these fronts, but it’s not an outlier either.
You said in your tweet that I should be “outraged by Black and Latinx proficiency levels.” Am I – and, without speaking for them, probably my colleagues at the University of Colorado Denver – outraged about proficiency gaps? Yes, of course. Just like you, we also care about equity and we all are fighting for historically-marginalized children across a variety of fronts. Am I personally outraged about proficiency levels? Less so, given the fact that Colorado decided to set a much higher bar than most other states. ‘Proficiency’ is a politically-determined label, not a context-free indicator. If DPS was in most states in America, we would say that 41% of its Black students were ‘at grade level’ in 8th grade math instead of 11%. That number is still terrible, particularly compared to their White student peers, but it’s not “1 in 10” either. Again, if Colorado sets the bar higher, by definition fewer students will be proficient. For me, the gaps are much more alarming than whatever level we apply to children’s performance. I think that the concern is in the inequity, not the label?
Van, this is a long post. You may disagree with much of what I said here, and who knows if you even read through to the end or not. But if you did, let me close with this: I think that you and I both have a similar passion for equity in schools, and I also think that we both have a passion for making school different, particularly for historically-marginalized children and families. I might be wrong, but I don’t think so. Given your work with DSST and The Odyssey School of Denver and my work around instructional redesign and leadership for deeper learning, I think that we might have a really interesting and productive conversation together. Let me know if you ever want to have a meetup. I’m happy to join you for lunch or coffee at whatever location is easy for you.
Thanks in advance if you actually read through all of this. Hope we get a chance to talk sometime.
2. Always remember that the labels for NAEP ‘proficiency’ levels are confusing. Journalists (and others) are failing us when they don’t report out what NAEP levels mean. For instance, the New York Times reported this graph today from NCES:
BUT… ‘Proficient’ on NAEP doesn’t mean what most folks assume it does. NAEP itself says that ‘Proficient’ does not mean ‘at grade level.’ Instead, the label Proficient is more aspirational. Indeed, it’s so aspirational that most states are not trying to reach that level with their annual assessments. See the map below from NCES (or make your own), which shows that most states are trying for their children to achieve NAEP’s Basic level, not Proficient:
Once again, in the words of Tom Loveless, former director of the Brown Center on Education Policy at the Brookings Institution, “Proficient on NAEP does not mean grade level performance. It’s significantly above that.” So essentially the New York Times and others are reporting that “only one-fourth of 8th graders performed significantly above grade level in math.” Does that result surprise anyone?
Equating NAEP proficiency with grade level is bogus. Indeed, the validity of the achievement levels themselves is questionable. They immediately came under fire in reviews by the U.S. Government Accountability Office, the National Academy of Sciences, and the National Academy of Education. The National Academy of Sciences report was particularly scathing, labeling NAEP’s achievement levels as “fundamentally flawed.”
Loveless also stated:
The National Center for Education Statistics warns that federal law requires that NAEP achievement levels be used on a trial basis until the Commissioner of Education Statistics determines that the achievement levels are “reasonable, valid, and informative to the public.” As the NCES website states, “So far, no Commissioner has made such a determination, and the achievement levels remain in a trial status. The achievement levels should continue to be interpreted and used with caution.”
Confounding NAEP proficient with grade-level is uninformed. Designating NAEP proficient as the achievement benchmark for accountability systems is certainly not cautious use. If high school students are required to meet NAEP proficient to graduate from high school, large numbers will fail. If middle and elementary school students are forced to repeat grades because they fall short of a standard anchored to NAEP proficient, vast numbers will repeat grades. [emphasis added]
In its prescriptive aspect, the NAEP reports the percentage of students reaching various achievement levels—Basic, Proficient, and Advanced. The achievement levels have been roundly criticized by many, including the U.S. Government Accounting Office (1993), the National Academy of Sciences (Pellegrino, Jones, & Mitchell, 1999); and the National Academy of Education (Shepard, 1993). These critiques point out that the methods for constructing the levels are flawed, that the levels demand unreasonably high performance, and that they yield results that are not corroborated by other measures.
In spite of the criticisms, the U.S. Department of Education permitted the flawed levels to be used until something better was developed. Unfortunately, no one has ever worked on developing anything better—perhaps because the apparently low student performance indicated by the small percentage of test-takers reaching Proficient has proven too politically useful to school critics.
For instance, education reformers and politicians have lamented that only about one-third of 8th graders read at the Proficient level. On the surface, this does seem awful. Yet, if students in other nations took the NAEP, only about one-third of them would also score Proficient—even in the nations scoring highest on international reading comparisons (Rothstein, Jacobsen, & Wilder, 2006).
The NAEP benchmarks might be more convincing if most students elsewhere could handily meet them. But that’s a hard case to make, judging by a 2007 analysis from Gary Phillips, former acting commissioner of NCES. Phillips set out to map NAEP benchmarks onto international assessments in science and mathematics.
Only Taipei and Singapore have a significantly higher percentage of “proficient” students in eighth grade science (by the NAEP benchmark) than the United States. In math, the average performance of eighth-grade students could be classified as “proficient” in [only] six jurisdictions: Singapore, Korea, Taipei, Hong Kong, Japan, and Flemish Belgium. It seems that when average results by jurisdiction place typical students at the NAEP proficient level, the jurisdictions involved are typically wealthy.
We can argue whether the correct benchmark is Basic or we should be striving for Proficient, and we all can agree that more kids need more support to reach desired academic benchmarks. But let’s don’t pretend that ‘Proficient’ on NAEP aligns with most people’s common understandings of that term. We should be especially wary of those educational ‘reformers’ who use the NAEP Proficient benchmark to cudgel schools and educators.
4. The pundits already are chiming in on the 2022 NAEP results. They’re blaming overly-cautious superintendents and school boards, “woke” educators, teacher unions, parents, online learning, video games, social media, screen addiction, “kids these days who don’t want to work,” state governors, and anything else they can point a finger at. As I said yesterday, it’s fascinating how many people were prescient and omniscient during unprecedented times, when extremely challenging decisions needed to be made with little historical guidance, in an environment of conflicting opinions about what was right. Despite the massive swirl of disagreement about what should have occurred during the pandemic, many folks are righteously certain that they have the correct answer and everyone else is wrong. The lack of grace, understanding, and humility is staggering.
Also, look again at the graph above. One way for journalists, commentators, and policymakers to frame those results is to call them ‘appalling.’ Another way is to say:
Scores are down but, even during a deadly global pandemic that shut down schools and traumatized families, the math and reading achievement of about two-thirds of our students stayed at grade level or above. How do we help the rest?
Always consider how an issue is framed and whose interests it serves to frame it that way (and why).
We can whirl ourselves into a tizzy of righteous finger-pointing, which is what many folks will do because it serves their agenda to do so. Or we can
interpret these NAEP outcomes with deservable caution and an understanding of past results,
recognize that a decline in assessment scores is a naturally-expected outcome of an incredibly difficult two years of trauma during a deadly global pandemic, and then
I think that it’s unlikely that many states, schools, and communities will actually do this because of the fragility and brittleness of our school structures. But I’m pretty sure that the path forward is not simply doubling down on more math, reading, and testing, and it sure isn’t uncritically accepting NAEP results.
Most schools here in the U.S. now have been back for a month or two. And I’m hearing from educators that things are … ‘better.’ Which has me wondering, “How are we defining better?”
As we all know, the end of the 2020 school year and the entire 2020-21 school year were an incredible challenge. Schools shut down. People died. Everything was disrupted, and everyone was scared and anxious. Then, over the summer of 2021, we were much too optimistic about an allegedly ‘normal’ return to school. And it wasn’t. In many (most?) schools, the 2021-22 school year was somehow even tougher than the previous one as we experienced extremely high levels of student refusal and absenteeism, educator stress and burnout, and so on.
In a conversation with Catlin Tucker, I wondered how much better last school year could have been if we had leaned more into relationships and care. There was so much policy rhetoric around students’ ‘learning loss.’ Accordingly, many schools jumped much too fast into their traditional instructional processes without really addressing the trauma that children (and educators) still were carrying with them at the beginning of the school year. And it didn’t work.
I hypothesized in that discussion that if we had started the first few weeks with a significant focus on relationships and care and getting students and families the supports that they needed (say, 80% of our time and energy) and a lesser emphasis on the academic stuff (say, 20%), we could have laid the groundwork for a much smoother school year as we created a stable foundation that allowed us to transition back to ‘normal’ expectations. But many schools didn’t do that, at least not sufficiently to remedy the problem. It was as if we knew that our young people still were traumatized but didn’t want to address it genuinely, at the levels that our children deserved. Sure, we recognized and paid lip service to the issue, and maybe even halfheartedly implemented some new socio-emotional learning (SEL) program, but we didn’t really meet kids’ needs. The proof was obvious as we mostly tried to return to regular learning-teaching practices and then wondered why kids’ behavior, attendance, and academic performance were so terrible and why teachers were incredibly stressed and leaving the profession.
The past few years have shown that the rigidity of our school systems is also a brittle fragility, particularly during a time of dire need for young people and their families. The saddest part of last school year may have been that we could have hit the reset button at any time. We could have taken a pause from school as we know it, invested more deeply into kids rather than content, and built, together, to where we needed to be. But we chose not to. We just kept on with the things that weren’t working, and children and educators paid the price.
All of which brings us to this school year, which supposedly is ‘better.’ And I’m wondering why. Did we finally transform how we interact with our children? Did we finally center their emotional and trauma needs and establish foundational structures of relationship and care that allow us to learn together in functional community? Or, as I suspect from the many educator discussion areas that I’m in, at the beginning of this year did we just lean more heavily into ‘expectations’ and ‘consequences’ that ignore underlying root causes and instead emphasize control and compliance? In other words, if one end of a continuum might be framed as ‘Kids are struggling so they need care’ and the other end might be framed as ‘Kids are struggling so they need control,’ which end of the continuum did our schools lean into? Did we create new, effective systems of care or did we just socialize and force our young people into submission (as we always seem to do)?
How about your school? What did it lean into this year?
I do a basic introductory workshop on Monday. Gina and Robbi have created a fabulous workshop and I can’t wait to see their session in action on Tuesday. Then I will try and extend all of this work even further during my Wednesday workshop. Descriptions are below…
Many schools have created future-ready vision statements and college- and career-ready profiles of a graduate. But most schools still are struggling to transition their day-to-day classroom instruction to include more critical thinking, problem-solving, creativity, and other ‘future-ready’ student competencies in ways that are substantive, meaningful, and aligned to those vision statements and graduate profiles.
This workshop focuses on how to redesign classroom instruction for future-ready learning. We will use the free 4 Shifts Protocol to redesign lessons, units, and other instructional activities together for deeper learning, greater student agency, more authentic work, and rich technology infusion. The protocol contains concrete, specific ‘look fors’ and ‘think abouts’ that allow educators, coaches, and instructional leaders to shift students’ instructional work in deeper, more robust directions. The protocol is a useful complement to SAMR, TPACK, Triple E, and other frameworks that schools may be using, and also is an excellent capacity-building bridge to more complex inquiry and PBL projects.
This active, hands-on workshop is intended for teachers, instructional / technology coaches, and school leaders who are prepared to roll up their sleeves and dive into this important instructional redesign work!
This session incorporates Dr. Scott McLeod’s 4 Shifts Protocol and is designed to introduce teachers to practical skills to (re)design lessons focusing on deeper learning, greater student agency, more authentic work, and rich technology infusion. Teachers will learn to recognize the four shifts, evaluate ways to personalize the four shifts, (re)design a lesson, and use the four shifts to permanently pivot to incorporating the shifts into future.
This session is targeted for teachers PK-12, special education, literacy programs, gifted and talented classrooms, instructional coaches, and administrators.
New technologies give us new possibilities. In this workshop we will identify several different blended learning structures and how they might be used to facilitate students’ deeper learning, greater student agency, and more authentic, real world work. Station rotations, genius hours, flipped classrooms, flex models, and other blended learning strategies can create powerful pathways for our children. Bring a computer and come prepared to roll up your sleeves and engage in some active (re)design discussions!
This active, hands-on workshop is intended for teachers, instructional / technology coaches, and school leaders who are prepared to roll up their sleeves and dive into this important instructional redesign work!
Hope you’ll join us for one or all of these sessions!
A school leader in one of my Facebook groups asked if anyone had a discussion guide for the next time their teachers held vertical discussions across grade levels. Here was my response:
I’ve done this with schools before. Not exactly sure what the desired outcome of your conversations is, but I’ve seen really powerful discussion arise from the simple questions of “What do you expect students to know and be able to do by the end of their school year with you?” (to the lower grade team) and “What do you expect students to know and be able to do when they enter your classroom at the beginning of the school year?” (to the higher grade team)…
Small group conversation around those two questions can easily fill most of an hour (be sure to have them take notes!). Also helpful to have some debrief time at the end where you just ask folks “What did you hear today? What does that mean for our practice? How can I be of support?“
Good luck and have fun!
What do you like your educators to talk about in their vertical discussions?
ISTE Certification has kept me busy! Despite my familiarity with all of the ISTE Standards, I have found that I am thinking much more deeply about the ISTE Standards for Educators as I go through this process with my cohort (which I appreciate)…
One of our activities asked us to reflect on the Universal Design for Learning (UDL) guidelines. Here’s some of what I wrote:
ENGAGEMENT UDL Guideline(s): Recruiting Interest, Sustaining Effort & Persistence, Self Regulation Tool(s): Blogging platforms such as WordPress or Squarespace
I believe in technology tools that have low floors, high ceilings, and wide walls (i.e., they are relatively easy to learn but have extraordinary, open-ended, and wide-ranging power). One such tool for me when it comes to the UDL principle of Engagement is a blog. Blogs allow learners and creators to do almost anything, particularly if they use a more powerful, self-hosted platform such as WordPress or Squarespace. The ability of blogs to host almost any kind of media that we wish (text, audio, video, images, charts, tables, diagrams, hyperlinks) in almost any configuration that we wish (see, e.g., the wide variety of blog templates) means that they are infinitely customizable. Accordingly, learners and creators can make their blog anything that they wish. This capacity taps directly into the Engagement guideline of Recruiting Interest because it ‘optimizes individual choice and autonomy.’ Similarly, the interactive nature of blogs (e.g., hyperlinks, pingbacks, comments, embedding of social media feeds, RSS subscription) highlights the Engagement guideline of Sustaining Effort and Persistence because it ‘fosters collaboration and community.’ Blogs can be deeply reflective tools that also foster visibility, sharing, contribution, and connection, which aligns directly with the Engagement guideline of Self Regulation and its emphasis on self-assessment, reflection, and motivation.
REPRESENTATION UDL Guideline(s): Perception, Language & Symbols, and Comprehension Tool(s): Presentation software such as Apple Keynote, Microsoft PowerPoint, or Google Slides
I believe in technology tools that have low floors, high ceilings, and wide walls (i.e., they are relatively easy to learn but have extraordinary, open-ended, and wide-ranging power). An underutilized tool for the UDL principle of Representation is presentation software such as Keynote, PowerPoint, or Google Slides. As I work with my graduate students – some of whom are differently abled than their educator peers – I have found that presentation software creates an open ‘green field’ of possibility. Students can use text. They can use images. They can create lines, diagrams, charts, tables, timelines, and concept maps. They can embed audio or video. They can tap into various color schemes, fonts, and transparency. They can tap into the power of layering, grouping, animations, and transitions. Together, these simple-to-learn capabilities that we often take for granted in presentation software can be used in incredibly diverse ways to represent any topic, idea, or concept that we wish with as much complexity as we wish. Recent examples of this in our own cohort include our introductions and our ISTE Standards for Educators jigsaw activity. These examples illustrate how a simple set of tools can create phenomenally-powerful and divergent opportunities to share what we know, can do, and have learned. The capabilities inherent in presentation software allow us to check off the boxes in essentially every subcategory of Representation (i.e., Perception, Language & Symbols, and Comprehension).
ACTION & EXPRESSION Guideline(s): Executive Functions Tool(s): Google Sheets
I believe in technology tools that have low floors, high ceilings, and wide walls (i.e., they are relatively easy to learn but have extraordinary, open-ended, and wide-ranging power). For the UDL Principle of Action & Expression, I chose to focus on one particular guideline, Executive Functions. For the past few years, I have been making my own interactive templates in Google Sheets. My students – or workshop participants – can go directly to a template that I have made and interact in a variety of ways with content, questions, or each other. I like that Google Sheets creates a different URL for each tab, and I can configure and merge the rows, columns, cell entries, formulae, auto-calculations, and conditional formatting into almost any format I wish. My principal licensure students and I use them routinely to work on thorny leadership problems of practice and systemic school and district redesign concerns. I also like that with a quick mouse click students can see each others’ responses as well if they are on a different tab rather than a shared one. I can even hide sections of the template and reveal them later for additional consideration. The possibilities are nearly endless for individual or small group, collaborative work, and the shared, online nature of the tool allows for easy access and easy archiving of our thinking work together. All of this connects directly to the Executive Functions checkpoints related to goal-setting, planning, strategy development, managing information and resources, and monitoring progress.
Nothing earth-shattering here, but I enjoyed the opportunity to think a little more about this!
