Archive | Assessment RSS feed for this section

Adaptive learning

Unit 1

Teacher 1:

In the past I have mapped out my school year ahead of time. I’ve planned how long each unit is going to take; identified the resources, activities, and assessments that I’ll use for each unit; and then marched students through the content. But this year, I’ve got an amazing idea! Before school starts I’m going to print off all of the worksheets, quizzes, and tests that the publisher sends with the textbook. I’ll also add in a few of my own supplemental activities, and put everything into numbered folders. Since kids like videos, for some units I’ve even got some VHS tapes on which I’ll place Post-It notes with time-marked segments for them to watch. Students will have access to a printed checklist for each unit that shows what they need to read, watch, and do, and they’ll also get an overview checklist of all of the units for the entire year. This way, instead of students marching to my pace, they can go as fast or as slow as they need to. They can even bounce around different units as desired, focusing on whatever they want to work on that day, and can skip stuff if they can prove mastery! I’ll also put some stickers into each folder. As students complete each reading, worksheet, quiz, test, activity, or video, they can put a sticker on their checklist showing that they’ve completed it. It will be just like getting points and leveling up in a video game! We’ll also have tracking posters stapled to the bulletin board so that I can monitor overall task and unit completion for each student, and intervene as necessary if students are moving too slow, need extra help, or are ready for enrichment activities. The system will be entirely student-driven, freeing me up to be a facilitator of learning instead of a ‘sage on the stage.’ I’m so excited to set up this system of personalized learning!

Teacher 2:

In the past I have mapped out my school year ahead of time. I’ve planned how long each unit is going to take; identified the resources, activities, and assessments that I’ll use for each unit; and then marched students through the content. But this year, my school has an amazing idea! Before school starts I’m going to have access to an online adaptive learning system that includes all of the worksheet, quiz, and test items that the publisher sends with the digital textbook. There also are some supplemental activities, and everything is organized into numbered units. Since kids like videos, for some units the system even has some digital tutorials for them to watch. Students will have access to an online checklist for each unit that shows what they need to read, watch, and do, and they’ll also get an overview checklist of all of the units for the entire year. This way, instead of students marching to my pace, they can go as fast or as slow as they need to. They can even bounce around different units as desired, focusing on whatever they want to work on that day, and can skip stuff if they can prove mastery! The system also has digital badges for each unit. As students complete each reading, worksheet, quiz, test, activity, or video item, they get a digital badge for their checklist showing that they’ve completed it. It will be just like getting points and leveling up in a video game! We’ll also have access to an online data analytics system so that I can monitor overall task and unit completion for each student, and intervene as necessary if students are moving too slow, need extra help, or are ready for enrichment activities. The system will be entirely student-driven, freeing me up to be a facilitator of learning instead of a ‘sage on the stage.’ I’m so excited we have this system of personalized learning!

12 education guidelines from Alfie Kohn

Alfie Kohn says:

  1. Learning should be organized around problems, projects, and (students’) questions – not around lists of facts or skills, or separate disciplines.
  2. Thinking is messy; deep thinking is really messy. Therefore beware prescriptive standards and outcomes that are too specific and orderly.
  3. The primary criterion for what we do in schools: How will this affect kids’ interest in the topic (and their excitement about learning more generally)?
  4. If students are “off task,” the problem may be with the task, not with the kids.
  5. In outstanding classrooms, teachers do more listening than talking, and students do more talking than listening. Terrific teachers often have teeth marks on their tongues.
  6. Children learn how to make good decisions by making decisions, not by following directions.
  7. When we aren’t sure how to solve a problem relating to curriculum, pedagogy, or classroom conflict, the best response is often to ask the kids.
  8. The more focused we are on kids’ “behaviors,” the more we end up missing the kids themselves – along with the needs, motives, and reasons that underlie their actions.
  9. If students are rewarded or praised for doing something (e.g., reading, solving problems, being kind), they’ll likely lose interest in whatever they had to do to get the reward.
  10. The more that students are led to focus on how well they’re doing in school, the less engaged they’ll tend to be with what they’re doing in school.
  11. All learning can be assessed, but the most important kinds of learning are very difficult to measure – and the quality of that learning may diminish if we try to reduce it to numbers.
  12. Standardized tests assess the proficiencies that matter least. Such tests serve mostly to make unimpressive forms of instruction appear successful.

via http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/10/30/a-dozen-basic-guidelines-for-educators

We’ve got extra funds! Let’s buy test-taking devices!

Amy Prime says:

I know of another administrator who sent out a happy email to her staff because the school had some extra funds and was able to purchase a classroom set of tablets for the teachers to use for classroom testing. How many of you parents out there, upon discovering your child’s school had extra funds available, would want that money used to buy test-taking devices?

via http://www.desmoinesregister.com/article/20131020/OPINION01/310200071/Iowa-View-Opt-Out-Movement-growing-schools-test-test-test

Want students to look worse? Change the cut scores.

In a must-read article explaining the politics behind cut scores on state-level tests, principal Carol Burris notes:

In 2011, the College Board created a College Readiness index.   It was a combined index of 1550, which only 43 percent of all SAT test takers achieved. You can find it here. Now add up New York’s chosen index.  It is 1630, significantly higher than the 2011 College Board’s index associated with a B- in college.

The above illustrates how one can manipulate the percentage of college readiness by … changing the definition of “college ready” to suit oneself. . . . In the end, [the New York State Education Department] chose values that are extraordinarily high, producing an index that exceeds the College Board’s index for achieving a B- average.

Next time you read or hear about someone saying that more difficult tests and/or higher cut scores will ensure college readiness, remember this and ask some tough questions.

Performance assessments may not be ‘reliable’ or ‘valid.’ So what?

Meh

In a comment on Dan Willingham’s recent post, I said

we have plenty of alternatives that have been offered, over and over again, to counteract our current over-reliance on – and unfounded belief in – the ‘magic’ of bubble sheet test scores. Such alternatives include portfolios, embedded assessments, essays, performance assessments, public exhibitions, greater use of formative assessments (in the sense of Black & Wiliam, not benchmark testing) instead of summative assessments, and so on. . . . We know how to do assessment better than low-level, fixed-response items. We just don’t want to pay for it…

Dan replied

I don’t think money is the problem. These alternatives are not, to my knowledge, reliable or valid, with the exception of essays.

And therein lies the problem… (with this issue in general, not with Dan in particular)

Most of us recognize that more of our students need to be doing deeper, more complex thinking work more often. But if we want students to be critical thinkers and problem solvers and effective communicators and collaborators, that cognitively-complex work is usually more divergent rather than convergent. It is more amorphous and fuzzy and personal. It is often multi-stage and multimodal. It is not easily reduced to a number or rating or score. However, this does NOT mean that kind of work is incapable of being assessed. When a student creates something – digital or physical (or both) – we have ways of determining the quality and contribution of that product or project. When a student gives a presentation that compels others to laugh, cry, and/or take action, we have ways of identifying what made that an excellent talk. When a student makes and exhibits a work of art – or sings, plays, or composes a musical selection – or displays athletic skill – or writes a computer program – we have ways of telling whether it was done well. When a student engages in a service learning project that benefits the community, we have ways of knowing whether that work is meaningful and worthwhile. When a student presents a portfolio of work over time, we have ways of judging that. And so on…

If there is anything that we’ve learned (often to our great dismay) over the last decade, it’s that assessment is the tail that wags the instructional, curricular, and educational dogs. If we continue to insist on judging performance assessments with the ‘validity’ and ‘reliability’ criteria traditionally used by statisticians and psychometricians, we never – NEVER – will move much beyond factual recall and procedural regurgitation to achieve the kinds of higher-level student work that we need more of.

The upper ends of Bloom’s taxonomy and/or Webb’s Depth of Knowledge levels probably can not – and likely SHOULD not – be reduced to a scaled score, effect size, or regression model without sucking the very soul out of that work. As I said in another comment on Dan’s post, “What score should we give the Mona Lisa? And what would the ‘objective’ rating criteria be?” I’m willing to confess that I am unconcerned about the lack of statistical ‘validity’ and ‘reliability’ of authentic performance assessments if we are thoughtful assessors of those activities.

How about you? Dan (or others), what are your thoughts on this?

Image credit: Meh, Ken Murphy

We can no longer claim to believe in the individual student

Ryan Bretag says:

We can no longer claim to believe in the individual student if we continue supporting a system of assessment driven by seeing them as a collective unit meant to be mass produced in the same way at the same time instead of seeing them as a separate learner meant to be uniquely fostered.

via http://www.ryanbretag.com/blog/?p=4141

Teacher ‘accountability’ [VIDEO]

I don’t get to attend the meetings between educators and policymakers when they talk about teacher ‘accountability,’ but this is how I envision the conversation often plays out…

Happy viewing! (with captions!)

No one is listening to the students

Diane Ravitch says:

[The] data-driven focus [of Houston Independent School District’s Apollo Program] contains the seed of its own destruction. Talking about tests all the time, doing test prep all the time, making kids take tests that they are not relevant to them and that they are not prepared for. . . .

I was not surprised by the emotional and physical reactions of these kids as staff kept trying to get them and keep them in school. The kids keep saying that the learning is irrelevant. They keep saying that school is boring. They keep saying that no one understands them and their plight. Telling them, “No Excuses!” is disrespectful.

via http://dianeravitch.net/2013/05/17/pbs-in-houston-watch-the-faces-of-the-students

What does it mean to be ‘aligned to the Common Core?’

Now Common Core Aligned!

Did you know that…

As expected, with the advent of the Common Core we are seeing a lot of labeling and re-labeling of instructional materials, resources, and activities. Publishers are adding the Common Core designation to existing textbooks, resources, assessments, and professional development opportunities just as fast as they can. Educators are unpacking the Common Core and affirming to themselves that they’re already doing what the standards expect. Lots of Common Core hoopla. Lots of Common Core assurances. Lots of old educational wine in new Common Core bottles…

Plus, of course, lots of gratuitous Common Core labeling and hucksterism. Because if it’s not stamped ‘Common Core’ these days, hardly anyone’s going to look at it. 

We have the standards. And publishers’ criteria. And state and school district certification efforts. But we also have lots of confusion, including whether or not teachers are prepared or unprepared to implement the standards.

As we sort out that confusion – and as we work together to become better prepared for implementation of the Common Core juggernaut – we need to be critical consumers of both our own lessons and the vendor pitches that accompany the standards. Because if there’s anything that policy-level folks agree on, it’s usually that the Common Core is supposed to be different. Very different.

Of course if we absorb the Common Core into what we’ve always done without substantially changing anything – and this is extremely likely given our history – then things won’t be different at all. We know from past experience that standards usually don’t change instruction much. Neither do they change the day-to-day learning experiences of most children. Implementation always trumps wishes. Regardless of the rhetoric accompanying the Common Core, our historically high rates of reform assimilation indicate that what kids do in school on a daily basis is unlikely to be very different in most places. As Richard Elmore notes,

Internal accountability precedes external accountability and is a precondition for any process of improvement.

What does it mean to you for things to be ‘Common Core aligned?’ [Although Common Core chief architect / circus barker David Coleman believes that “people really don’t give a sh*t about what you feel or what you think“, I do.] Perhaps more importantly, what are you and your fellow educators doing to avoid old wine in new bottles?

P.S. Never fear. This blog post is Common Core-aligned℠. See ELA-Literacy.CCRA.R.8.

In Iowa, our Department of Education brags about elementary schools that cut recess

Everyone wants children to be able to read. But unpacking that educational goal – and the political rhetoric that often surrounds it – may require a bit more digging and critical analysis. Here’s an example…

In the 2004-2005 school year, 18 4th graders took the state reading test at Charter Oak-Ute Elementary. Only 14 were deemed proficient, for an AYP percentage of 78%. That apparently sparked a 7-year quest to raise test scores.

2005 Charter Oak Ute Elementary Reading

Today the Iowa Department of Education (DE) touted Charter Oak-Ute Elementary as one of the 5 schools (out of 1,409 in the state) that’s supposedly proving that poverty does not equal destiny. In fact, DE boldly said on its home page:

It may be well known that high-poverty schools will have lower proficiency rates than their more affluent counterparts. Sure, it’s well known. But it is wrong.*  [yes, that was our Department of Education dismissing decades of peer-reviewed research on student learning outcomes in high-poverty schools]

 DE screenshot 01

What did Charter Oak-Ute Elementary do to warrant DE’s publicity? Well, in 2011-2012, 19 of its 21 3rd grade students passed the reading test – for an AYP percentage of 90%** – despite 58% of its students receiving free/reduced price lunch. [for reference, the average statewide reading proficiency for 3rd graders is 76%]

2012 Charter Oak Ute Elementary Reading

From 14 of 18 students to 19 of 21 students. If Charter Oak-Ute Elementary had kept its reading proficiency percentage steady, only 16 3rd graders would have passed the state reading test last year. So it essentially moved the needle for 3 students. In seven years.***

By now many of you may be wondering, “What did this elementary school do to bump up these 3 kids’ reading scores?” Well, according to its principal:

[Teachers and students] weren’t happy with some of the things we had to drop, such as morning recess time because we really don’t need that.

That’s right. Among other interventions, the school cut recess. For 7- and 8-year-olds.

Never mind statements against cutting recess from the National Association for the Education of Young Children and the American Academy of Pediatrics. Never mind the research that shows how recess breaks maximize children’s cognitive performance or shows recess is important for children’s learning, social development, and health (“no research clearly supports not having recess”) or connects recess to good classroom behavior. Never mind children’s needs for breaks, exercise, and play. Never mind our childhood obesity epidemic, particularly for low-income kids.

And, apparently, never mind DE’s own admonitions for schools to adopt ‘evidence-based practices.’ Whether proposing 3rd grade retention or cutting recess (FYI, for both the research is heavily AGAINST them), DE is beginning to show that is willing to hold up and/or advocate for practices that are anything BUT ‘evidence-based.’

A high-poverty school that gets rid of elementary school recess to feed the always-hungry maw of ever-increasing test score goals should raise concerns for us. Because it’s yet another example of the kinds of dehumanizing microaggressions that happen all too often to children who are in poverty and/or of color. And it’s not what we in Iowa should be encouraging. Because if DE is willing to tout this recess-cutting school as doing what it needed to raise reading scores, the writing is potentially on the wall for ‘whole child’-oriented practices in larger school districts that have even greater concentrations of children in poverty. Yes, that means you, Des Moines, Waterloo, Sioux City, and Davenport (and others)…

I’m concerned that we’re becoming one of THOSE states. In Iowa we always have prided ourselves as being more enlightened than many of those states in which districts were cutting art, music, recess, physical education, foreign language, and other aspects of school necessary to provide well-rounded schooling experiences for children. We took pride in doing our best to attend to the needs of the whole child – for every child. But that commitment to children – and our recognition of decades of child development research – appears to be waning.

So put February 25, 2013 down on your calendar as the day when not only did Iowans learn that one of our own schools cut recess to improve test scores but also that our own Department of Education was willing to brag about it. Welcome to the new #edreform in Iowa.

 

* At least it’s ‘wrong’ for the 5 schools out of 1,409 that DE cherry-picked [please ignore the other 1,404]
** DE said it was 92%?
*** Of course this ignores ordinary year-to-year variation, differences between cohorts of students, random measurement error, etc.