I often get asked by administrators for some recommended reading. Here are some of my favorite books on data-driven decision-making. If the Amazon widget doesn’t load in a few seconds, here’s a static picture of the list.
[Transparency disclosure: If you buy a book using this list, CASTLE gets 4% of the proceeds. Your cost doesn’t go up any. Amazon just pays us a little for the referral through its Associates Program.]
I just finished Dr. Paul Collier's award-winning book, The Bottom Billion. An economics professor at Oxford University, Collier notes that traditionally we have thought of the world's population as being made up of '€˜a rich world of one billion people facing a poor world of five billion people'€™ (p. 3).
Instead, we need to recognize that the developing world should be divided into two groups, the middle four billion and the bottom billion. The countries in the middle group have economies that are growing, life expectancies that are rising, infant mortality rates that are dropping, and malnutrition rates that are declining. They still have a long way to go compared to the developed world, but they are making progress. India and China are both part of the middle four billion, despite once being poorer than many of the countries in the bottom billion.
In contrast, the bottom billion people continue to live in '€˜the fourteenth century, [doomed by] civil war, plague, and ignorance'€™ (p. 3). Their economies are not growing and, '€˜given the power of compound growth rates, [the] differences between the bottom billion and the rest of the developing world will rapidly cumulate into two different worlds'€™ (p. 10). The bottom billion not only are '€˜falling behind … [they are] falling apart'€™ (p. 3).
Below is a graph of the growth rates of the bottom five billion over the past few decades [note that the actual growth rates were not this linear; Collier reported his figures by decades so I had to approximate]. Growth rates were as follows:
When these growth rates are plotted cumulatively, the chart looks something like this:
As you can see, the bottom billion are barely better off than they were 35 years ago. About 70 percent of the bottom billion lives in Africa. The bottom billion includes countries such as Somalia, Afghanistan, Haiti, Yemen, and North Korea.
I have greatly appreciated how this book has reoriented my thinking about global economic development challenges. Being a data guy, I have jumped into some of Collier'€™s numbers in order to help me envision what the challenges are that face the bottom billion. As we look toward the plight of our own disadvantaged student populations, we would be well-advised to look for data that help us reorient our thinking so that we may appropriately address our learning challenges. What data do you have that might shake up the thinking of your educators, students, and families?
[I learned a lot from this book. I give it 4 highlighters.]
State and federal accountability schemes require that students master low-level academic content. Our decisions regarding how we structure our instruction to facilitate student mastery of that content strike to the very heart of what we believe about teaching and learning. To facilitate conversations about this issue, I made a short video:
What do you believe is the best way to structure instruction to ensure student content mastery?
When I moved to Iowa from Minnesota, the Iowa Motor Vehicle Division
(MVD) didn’t test me before it issued me a driver’s license. It took into
account my long history of driving and my clean record and determined that I did
not need to take either a written or driving test. I did a quick vision check,
took one of those goofy photos, and I was all set.
Imagine, however, if the MVD, before it would issue me a license, wanted me
to sit through a series of classes intended to ‘teach’ me how to operate a car
and drive safely. I would have been completely annoyed. ‘Test me now!’ I would
have exclaimed vociferously. ‘I already know how to do this! Stop wasting my
time!’ By now you’re probably nodding your head in agreement, knowing that you’d
do the same thing in my situation. Although you’d rather not have to do the
written and driving tests again, you’d definitely rather be tested than sit
through hours of instruction on material you already know.
Unfortunately this is exactly what happens to our nation’s schoolchildren on
a daily basis. Millions of students regularly experience curricula and lessons
that address content and concepts with which they’re already familiar. It’s not
just the ‘talented and gifted’ kids; there are plenty of students who know the
material in a particular learning unit before they even start. They’re just
never given the chance to demonstrate their knowledge ahead of time. Nor do they
have the opportunity to request to be pre-tested.
What a colossal waste of time this is. Rather than
the joy of wrestling with and thinking about new material, students suffer
through yet another hour ‘learning’ old information. Rather than working with
children who are eager, interested learners, teachers suffer through yet another
group of disengaged students.
I wonder why we don’t care more about this? It’s one thing to
cover the required curriculum. It’s quite another to have students cover the
curriculum despite the fact that they already know it. As a former
eighth grade teacher, I know how difficult it is to differentiate instruction.
But it’s relatively easy to do some simple pre-testing and at least make an
attempt at altering a ‘one size fits all’ lesson plan. If more teachers did this
on a regular basis, they might be surprised at how much instructional time they
gained back during the year. And of course they’d also have better baseline data
with which to assess student learning growth for each curricular unit. And did I
mention the message of respect for students that accompanies the practice of
pre-testing?
Why don’t we do more pre-testing? Why is it so hard
to get teachers to buy into this?
I’m a strong proponent of data-driven practice, particularly for classroom- and student-level progress monitoring of essential learning outcomes. But, as Morville notes, there are diminishing (and even negligible) returns if one goes too far. We have to know when to say when.
I received this message recently from someone who works in a school district
department of research, evaluation, and assessment:
Do any of you know of
research on effective strategies for maximizing performance on
computer-delivered assessments? I think we have some common-sense ideas, but I
am wondering if there is anything out there that may be more substantive or
research-based.
Do any of you have
documents that discuss tips and strategies (for computer-delivered assessment)
that you share with your teachers?
When eduwonkette asked me to guest blog about data-driven decision-making in schools, I eagerly agreed. Why? Because in my work with numerous school organizations in multiple states, I have seen the power of data firsthand. When done right, data-driven education can have powerful impacts on the learning outcomes of students.
Unfortunately, most school districts still are struggling with their data-driven practice. Much of this is because they continue to think about using data from a compliance mindset rather than using data for meaningful school improvement. An uninformed model of data-driven decision-making looks something like this:
This is the NCLB model. Schools are expected to collect data once a year, slice and dice them in various ways, set some goals based on the analyses, do some things differently, and then wait another whole year to see if their efforts were successful. Somehow, this model is supposed to get schools to 100% proficiency on key learning outcomes. This is dumb. It’s like trying to lose weight but only weighing yourself once a year to see if you’re making progress. Compounding the problem is the fact that student learning data often are collected near the end of the year and given back to educators months later, which of course is helpful to no one.
A better model looks something like this:
The key difference in this model is an emphasis on ongoing progress monitoring and continuous, useful data flow to teachers. Under this approach, schools have good baseline data available to them, which means that the data are useful for diagnostic purposes in the classroom and thus relevant to instruction. The data also are timely, meaning that teachers rarely have to wait more than a few days to get results. In an effective data-driven school, educators also are very clear about what essential instructional outcomes they are trying to achieve (this is actually much rarer than one would suppose) and set both short- and long-term measurable instructional goals from their data.
Armed with clarity of purpose and clarity of goals, effective data-driven educators then monitor student progress during the year on those essential outcomes by checking in periodically with short, strategic formative assessments. They get together with role-alike peers on a regular basis to go over the data from those formative assessments, and they work as a team, not as isolated individuals, to formulate instructional interventions for the students who are still struggling to achieve mastery on those essential outcomes. After a short period of time, typically three to six weeks, they check in again with new assessments to see if their interventions have worked and to see which students still need help. The more this part of the model occurs during the year, the more chances teachers have to make changes for the benefit of students.
It is this middle part of the model that often is missing in school organizations. When it is in place and functioning well, schools are much more likely to achieve their short- and long-term instructional goals and students are much more likely to achieve proficiency on accountability-oriented standardized tests. Teachers in schools that have this part of the model mastered rarely, if ever, complain about assessment because the data they are getting are helpful to their classroom practice.
NCLB did us no favors. It could’ve stressed powerful formative assessment, which is the driving engine for student learning and growth on whatever outcomes one chooses. Instead, it went another direction and we lost an opportunity to truly understand the power of data-driven practice. There are hundreds, and probably thousands, of schools across the country that have figured out the middle part of the model despite NCLB. It is these schools that are profiled in books such as Whatever It Takes and It’s Being Done (both recommended reads) and by organizations such as The Education Trust.
When done right, data-driven decision-making is about helping educators make informed decisions to benefit students. It is about helping schools know whether what they are doing is working or not. I have seen effective data-driven practice take root and it is empowering for both teachers and students. We shouldn’t unilaterally reject the idea of data-driven education just because we hate NCLB. If we do, we lose out on the potential of informed practice.
Everyone in a school knows that some teachers are effective and some aren’t, but in most schools there is no organized way to ensure that students who get weak or bad teachers still learn what they need to learn. That’s not his fault, or the fault of any individual teachers who work hard; it is the fault of the way schools have been organized for generations.
and
[T]he essential issue at the heart of the controversy is whether schools should be asked to organize themselves to have student learning be at the center of all of their activities. I say they should. But the fact is that relatively few schools have done it.
