Archive | Research and Evaluation RSS feed for this section

Iowa wants to fail 3rd graders (and other thoughts on the Governor’s Education Blueprint)

Over the past month I've been reading and thinking about the new Education Blueprint proposed by the Iowa Governor and the Iowa Department of Education (DE) as well as various reactions to that document. If you haven't yet read Trace Pickering's insightful (and also lengthy) response to the Blueprint, be sure to do so. Another important read is school change guru Michael Fullan's recent paper, Choosing the Wrong Drivers for Whole System Reform.

Here are some additional thoughts of my own. These are not all-encompassing - I have additional questions and concerns - but they do constitute a few important issues that caught my attention. I'm also intentionally not commenting on topics for which I'm fairly ambivalent (e.g., charter schools) or don't know enough (e.g., teacher salary schedules and compensation tiers) and instead will leave those to others who care or know more than I do.

Failing 3rd graders fails our 3rd graders

I'll pick the low-hanging fruit first. Failing 3rd graders who can't pass some reading assessment is a really, really bad idea. It doesn't matter how many safeguards and second chances there are and I understand why the policy is being proposed (both educationally and politically). The bottom line is that, regardless of the 'social promotion' rhetoric and whatever gut intuition parents or policymakers may have, the research evidence is overwhelmingly unidirectional that in-grade retention does far more harm than good. Desired test score increases often never materialize and, even if they do, they usually don't persist past a few years. One of the stronger and consistent findings in educational research is that, in the long run, in-grade retention is at best a long-term wash score-wise and the resultant negative impact on students' psyches and their likelihood to graduate is horrific. The Governor and DE don't get to advocate for research-driven practices in other parts of the Blueprint but ignore that requirement here.

Input-Process-Output

We can visualize a box that represents the day-to-day occurrences within a classroom or other learning environment. That box is the most important aspect of schooling: if what students and teachers do on a daily basis in their learning-teaching interactions doesn't change substantially, all hope of achieving 'world class schools' in Iowa vanishes. WE LEARN WHAT WE DO. There are a variety of inputs (e.g., standards, curricula, teacher quality, funding and resources, school structures, technology infrastructure, laws and policies) that hopefully impact what occurs inside the box. We also look at what comes out of the box (e.g., student knowledge, skills, and dispositions) to see if what we wanted to happen actually did happen. This is a classic Input-Process-Output systems model (that hopefully is accompanied by a recursive feedback loop that informs the system).

IowaBluePrintSystem

There are 85 main bullet points, or action ideas, in the Blueprint. As you can see in my annotated version of the Blueprint, I tried to place each action idea into one of three categories: Input, Process, or Output (coded I, P, & O in the document). You are welcome to disagree with my categorizations (and I admit I struggled with some of them), but the evidence is quite stark that the Blueprint is overwhelmingly focused on inputs and outputs and gives very little attention to the day-to-day learning and teaching processes that occur between students and teachers.

IowaBluePrintPieChart

This is unsurprising. This is traditional school reform stuff:

We'll change some inputs; let's try better teachers and higher standards. Oh, and we'll also change some structures around. How about reallocating some monies, reorganizing traditional schools a bit, and allowing for charter and online schools? On the back end, we'll assess like crazy by changing our tests or using new and/or additional ones.

In the end, we change only a little and, if we're lucky, we see a little change in results. This is the way most states do it, but it's neither the only way nor the required way. Where in the Blueprint is the recognition that we need to do something DIFFERENT in our classrooms? Where's the acknowledgment, for example, that we need to invest heavily in teachers' ability to facilitate learning environments that foster higher-order thinking skills (an increasing necessity these days)? Where's extensive language about better facilitating student engagement in their courses? There's virtually nothing about students' interest in what they're supposedly learning. There's nary a bullet point about student hands-on or applied or problem-based learning or authentic intellectual work (a great program already being piloted by DE, by the way). To the extent that PBL and AIW and similar issues are addressed at all, the Blueprint does so indirectly; all hopes lie with effective implementation of the Iowa/Common Core and the Smarter Balanced assessments. Instead of just holding educators 'accountable' on the front and back ends of the process, how about directly investing in them so that they actually can be successful? The overwhelming emphasis of the Blueprint is on accountability rather than capacity-building. Go ahead and do a search in the Blueprint for the terms training or professional development or capacity; you won't find anything. If DE and the Governor are truly serious about 'world class schooling' in Iowa, they should be focusing heavily on the Process box - the day-to-day learning and teaching processes occurring in classrooms all across the state - and right now they're not.

Low-level testing

Much of the Governor's education concerns appear to be driven by NAEP scores and proficiency levels, despite the fact that most of the items are predominantly factual recall and low-level procedural knowledge AND despite the fact that the designers of NAEP freely admit that the level designations are arbitrary AND despite the fact that the American Institutes of Research notes that most of the nations to which we are comparing Iowa also wouldn't score well on NAEP. If we want our students to be gaining higher- rather than lower-order thinking skills, end-of-course assessments appear to offer us nothing better. So there's a lot of new and/or additional testing in the Blueprint that's focused on stuff you can easily find using Google - or that can be done cheaper by people elsewhere in the world - instead of on the skills and capacities necessary to really foster a world-class citizenry and workforce. We're not talking about assessments like the College and Work Readiness Assessment or what they do in Singapore. Again, when it comes to higher-order thinking skills, there's virtually no proposed investment in the Blueprint for the instructional side and all of our hopes rest on the Smarter Balanced assessments, for which right now we have no idea what they will look like and no idea how they will operate. The Blueprint essentially validates and tweaks and expands current testing schemes, despite significant warnings to the contrary from our very own National Research Council.

Digital, global world. Analog, local schools.

It's a globally-connected world out there, but the Blueprint primarily focuses on globalization as an economic force to which we must respond, not a societal / learning / citizenship issue to which we should attend for mutual benefit and empowerment. The Blueprint also says that Iowa students and graduates need to be internationally competitive but most of what it proposes is vastly different from what other countries are doing to achieve better results. The Blueprint contains no significant investment in teacher capacity-building, no emphasis on early childhood education, no amelioration of the impacts of family and neighborhood poverty on learning, and no recognition of the importance of strategic foreign language learning (particularly at younger ages), just to name a few.

It's also a digital world out there, but you wouldn't know it given the lack of emphasis placed on technology in the Blueprint. For example, only nominal attention is paid to online learning, despite the fact that it's booming nationwide and despite Iowa's meager offerings compared to other states. Even though Iowa ranks abysmally low when it comes to Internet speed and access, there's nothing regarding the importance of universal statewide broadband Internet access for both educational and economic development purposes. Most damning, there's absolutely no recognition of the power and potential of digital technologies to transform learning, teaching, and schooling, despite the rapid and radical reshaping of every other information-oriented societal sector by digital tools and the Web. In the world of the Blueprint, it's as if computers and the Internet essentially didn't exist. Go ahead and do a search in the Blueprint for the terms Internet or digital or technology; the omissions are quite alarming, actually. There's one meager shout-out to the rapid growth in 1:1 laptop initiatives across the state, but no support for giving every Iowa child a powerful digital learning device, for providing technology integration assistance for educators, for upgrading woeful infrastructures, for rethinking policies, or for anything else of substance when it comes to educational technology. It's 2011. Personal computers have been around for three decades and the Internet has been around for at least a dozen years for most of us. Digital technologies are transforming how Iowans and the world connect, collaborate, and LEARN; this omission is both sad and shameful.

A lost opportunity

There are a few things that I'm glad the Blueprint included. Although there is only a single bullet point referencing competency-based (rather than age-based) student progression, if done well that one thing alone has the potential to significantly and positively reshape much of how we do education in Iowa. I also like the willingness to invest in district-level innovation and to give districts some flexibility. The proof of most of this, like everything else, will depend on the legislative language and the resources committed.

As I think about the Blueprint as a whole (and we are encouraged by the document to treat it as 'a set of changes designed to work together'), it feels like a lost opportunity. The Governor and DE had the chance to dream big and swing for the fences. They had the chance to propose impactful, sweeping changes to the current system. They had the chance to create learning and teaching environments that prepare students for the next 50 years rather than the last 50 and to educate the public as to why those changes are necessary. The Blueprint rhetoric is right but the action items fall far short. I don't know if it's a lack of knowledge or vision or courage that's holding them back, and of course there are political considerations with all of this. But the result is a a tweak of the current system, a tinkering at the edges rather than a rethinking of the core. Perhaps it's foolish of me to wish for more.

I welcome all feedback. Thanks.

A new resource: CASTLE Briefs

CASTLE Brief 03b.pngI'm pleased to announce a new resource today: CASTLE Briefs.

As our web site notes:

CASTLE briefs are intended to help practicing and preservice school administrators with various technology leadership issues. Between 500 and 2,000 words in length, CASTLE briefs attempt to answer the question, "What do school administrators need to know about this technology leadership topic?" Some CASTLE briefs are classic research or policy briefs; others may be more practice-oriented or focus on thought leadership in a particular area.

ANYONE may write a CASTLE brief. Sometimes we will extend invitations to authors but we also accept at-large submissions. We are open to your ideas about content, format, and style but please note that we frown upon commercial advertisements disguised as briefs. Images, audio, video, and other multimedia are welcome inclusions in a brief. We would prefer APA citation style for your references section. All CASTLE briefs will be made available under a Creative Commons attribution-share alike copyright license.

Our first brief is titled  for consideration.

I hope that you will consider contributing to the CASTLE Brief series, either by submitting a brief yourself or at least adding some ideas to the list of potential topics. If you're a professor, note that writing a CASTLE brief would be a great assignment for your students! (hint, hint)

I'm looking forward to seeing how this develops!

Does hyper-texting lead to stress and depression? Or vice versa?

TrappedWe know that teens text a LOT: the average teenager sends 3,339 texts a month. Many adults are worried about the potential negative impacts upon youth of all of this texting. Common concerns cited include lack of face-to-face interpersonal skills, repetitive stress injuries, and an inability to focus.

Like others, I think the texting phenomenon is worth paying attention to and studying. But I’m not sure this recent article has it quite right. Here’s an excerpt:

Dr. Scott Frank, director of the Master of Public Health Program at Case Western Reserve University School of Medicine, has seen the effects of all that late-night texting. In a 2009 study, he found that “hyper-texting’’ teens - those who texted 120 times or more on an average school day - were more than 60 percent more likely to sleep less than seven hours per night and to doze off in class than those who were not hyper-texters.

“These teens were also more than 60 percent more likely to miss more school because of illness and to have poor academic performance,’’ he wrote in an e-mail. “Teens were also 25 percent more likely to report high levels of stress and 40 percent more likely to have symptoms of depression.’’

The way the article reads, the teens’ stress and depression and illness-related truancy and poor academic performance all seem to stem from the act of ‘hyper-texting.’ But this may be a classic case of correlation versus causation. In other words, perhaps teens that already are stressed, depressed, truant, and/or doing poorly in school are turning to their phones and their friends for validation and support more often than other teens do. As the article itself notes:

But many teens said feeling popular and connected to friends is more important than a good night’s rest.

“When I’m texting someone I don’t feel alone,’’ said A.J. Shaughnessy, a ninth-grader at Boston College High School. “When you don’t have your phone, you feel incomplete.’’

Michael Joyce, 16, a sophomore at the school, said the sound of his phone vibrating on his night table makes him happy. “Oh, good,’’ he thinks as he’s awakened, “someone’s texting me. Maybe someone needs me.’’

Sometimes teens answer late-night calls and messages less out of excitement than fear. In focus groups convened by the Pew Research Center, some teens related stories of friends or acquaintances who became angry or insulted when text messages or phone calls weren’t immediately returned. “As a result, many teens we heard from said they feel obligated to return texts and calls as quickly as possible, to avoid such tensions and misunderstandings,’’ the report said.

It’s hard for me to read that second excerpt and not believe that the underlying root problems are something other (and bigger) than hypertexting.

I think we have to be careful about what we infer and what causal directions we imply. Texting isn’t going away anytime soon. Whatever negative consequences accompany frequent texting, it’s far better for us to be accurately informed than it is to draw mistaken conclusions.

Image credit: The Stig texting IMG_0609

HELP WANTED – Research study of alumni of 1:1 laptop schools

TrappedMy doctoral student, Trent Grundmeyer, wants to study alumni of 1:1 laptop schools for his dissertation. More specifically, he’s interested in those students’ perceptions of their college readiness, college learning, and professors’ integration of learning technologies.

For the study, we need high school alumni who spent their final two years of high school in a 1:1 laptop program and are now completing their first year of college this spring. Accordingly, we hope to find at least three high schools that 1) have had laptop programs for at least 3 years, and 2) are willing to help us identify 10 of their alumni for potential participation in the study.

If this describes your high school, please get in touch with me and Trent!

Struggling with your dissertation?

Dissertations are difficult things. There are multiple reasons why most folks don’t have one. Here are some words of wisdom that I’ve heard from others and now pass along to my own doctoral students…

  1. Big ideas are good but you’re not going to save the world with your dissertation. Scale it down. Bite off something manageable and doable and save the rest of it for future work – by yourself or others.
  2. Along those lines… the best dissertation is a done dissertation. Get it done. Put the ‘Dr.’ in front of your name. Celebrate yourself for completing a large, hopefully-worthwhile task. Move on with your life. Go do great things!
  3. Fake diplomaphoto © 2007 dkl | more info(via: Wylio)
  4. The key to the successful completion of a dissertation is to treat it like a regular course that you might take. Block off the time that you'd ordinarily take to a) drive to and from class, b) be in the class, and c) read and/or do outside work for the class and then make that time REGULAR AND INVIOLATE (to yourself, your family, your friends, your employer) just as you would a traditional course. Pretend you have face-to-face accountability even though you don’t. Otherwise too many more-immediate and less-amorphous events pop up and you'll never, ever finish. The percentage of students who are ABD is appalling…
  5. Chunk it. The idea of writing a 150– to 250–page book is awfully intimidating. The idea of writing 2 pages is much more manageable. Set small, achievable goals for yourself (in the next hour I will write 2 paragraphs…; during this session I will find 5 new sources…).
  6. A dissertation always takes longer than you think it will. Get used to that idea now.
  7. Pick a topic that really interests YOU, not someone else (like your employer or your advisor), because no one’s with you at 1:00am on a Saturday night when you’re ready to tear your hair out. Pick something that hopefully will sustain you through the tough times.
  8. And, finally… I always tell my advisees that at some point in the process they will hate me. It all will be good in the end, but there will be moments when they curse my name. I’m okay with that: my job is to get them successfully past the entire committee.

What other words of wisdom do you have for educators working on their dissertations?

Should a ’21st century leadership’ book ignore technology?

In September 2007 I profiled a book that I had purchased called The 21st Century Principal. In that post, I noted the extreme paucity of content related to digital technologies, particularly given its purpose of examining

the future of public schooling in the United States – and what it will mean to be a leader in public schools – by focusing on the issues that are most likely to have an impact on American society within the next 20 years.

newprincipals21stcenturyToday I received notice of a new book, The Challenges for New Principals in the 21st Century. Here are the chapter titles:

  1. Translational Leadership: New Principals and the Theory and Practice of School Leadership in the Twenty-First Century
  2. New Headteachers in Schools in England and Their Approaches to Leadership
  3. So You Want to be a Headteacher?: “Liabilities of Newness,” Challenges, and Strategies of New Headteachers in Uganda
  4. Problems Reported by Novice High School Principals
  5. Accelerating New Principal Development Through Leadership Coaching
  6. From Mentoring to Coaching: Finding the Path to Support for Beginning Principals

I know many of the editors and authors for this book, and I’m sure it’s going to have some good, research-based information for new principals. But I confess I’m highly skeptical of the book title…

I understand that the role of school principals encompasses many things besides digital technologies. That said, should school leadership books be allowed to include phrases like ‘challenges’ and ‘21st century’ in their titles if they substantially ignore the leadership and learning issues related to digital technologies? I’m not sure they should…

HELP WANTED – Questions for interviews of ‘technology-savvy’ superintendents

For an upcoming CASTLE research project, we’re going to do interviews of multiple years’ worth of the winners of eSchoolNews’ technology-savvy superintendents awards. We are soliciting input regarding our interview protocol. These superintendent interviews will be done over the telephone, recorded as podcasts, and made available online to the public as a free resource.

What changes / additions / deletions would you suggest to our draft interview protocol? We’ll take a look at anything left in the comments area by Sunday, September 19!

YellowphoneInterview protocol

1. As we get started here, would you confirm verbally that you received the consent form that was sent to you and that you also recognize that this interview will be publicly available on the Internet? [pause] Thank you.

2. Where are you currently a superintendent and how long have you served in that role?

  • eSchoolNews identified you as one of the nation’s most technology-savvy superintendents. Are you still in the district for which you won the award?
  • Would you share with us your previous administrative and teaching experiences?

3. Why do you think you were selected for the eSchoolNews award?

  • Can you describe some of the most important technology initiatives that your district implemented?
  • What do you think was the most important initiative you implemented (and why)?

4. In general, what do you think is the difference between a regular superintendent and a ‘technology-savvy’ superintendent?

  • Are there particular leadership mindsets or behaviors that would help outsiders identify a superintendent as one that was ‘technology-savvy?’

5. How do you ensure that digital technologies are used as tools to enhance learning and not just as ‘technology for the sake of doing technology?’

6. What are some of the challenges to being a ‘technology-savvy’ superintendent? What gets in the way of doing this well? [ask follow-up probes about any/all of the items below as necessary]

  • Staff?
  • Resources?
  • Parents / community members?
  • Time?
  • Professional development / training?
  • Lack of organizational vision?
  • Other?

7. Why do you think most superintendents struggle when it comes to being effective technology leaders?

  • What can be done to help superintendents in this area?

8. Did your superintendent licensure program - or any of your other administrative licensure programs - help prepare you to be an effective technology leader?

  • Did your administrative licensure programs incorporate coverage of technology leadership issues into their curricula?
  • Did your administrative licensure programs help you develop your own personal technology proficiencies?
  • What recommendations do you have for preservice preparation of superintendents and principals when it comes to the technology side of leadership?

9. What kind of learning opportunities do you have to enhance your own development as a technology leader? What, if anything, is being done by each of these entities to help you become a better technology leader?

  • School district
  • State and/or national leadership associations
  • State department of education and/or regional service agencies
  • Universities
  • Other entities

10. What are some ways that you personally use technology?

  • What do these technology tools do for you?
  • How do you keep up with the current trends in education as a whole and, more specifically, the ever-changing world of technology in education and society?
  • What are some must-have technology tools for superintendents?

11. We’ve been focusing on the superintendent side of all of this. What do building-level school leaders need to be doing when it comes to technology implementation and integration in their schools? What role do they play in all of this?

12. What kinds of professional development do you think are most effective when it comes to technology? Let’s start with teachers. What training do they need and how should it be structured?

  • Okay, and now how about for administrators? What training do they need and how should it be structured?
  • Should technology-related training for administrators be different than what is given to teachers? Why or why not?

13. So far we’ve been talking a lot about effective technology leadership. What does ineffective, or bad, technology leadership look like?

14. Here’s our next-to-last question: What advice do you have for school leaders who want to be more ‘technology savvy?’

15. Thanks so much. Is there anything else you want to say on the topic of effective technology leadership? Anything that we should have discussed but didn’t?

Image credit: Do you remember?……Call

Don’t give too much weight to student test scores for teacher evaluation [Report]

2010epireportThe Economic Policy Institute’s new report, Problems with the Use of Student Test Scores to Evaluate Teachers, cautions against heavy reliance on the use of test scores in teacher evaluation.

Authors of the report include four former presidents of the American Educational Research Association; two former presidents of the National Council on Measurement in Education; the current and two former chairs of the Board of Testing and Assessment of the National Research Council of the National Academy of Sciences; the president-elect of the Association for Public Policy Analysis and Management; the former director of the Educational Testing Service's Policy Information Center and a former associate director of the National Assessment of Educational Progress; a former assistant U.S. Secretary of Education; a former and current member of the National Assessment Governing Board; and the current vice-president, a former president, and three other members of the National Academy of Education.

“School is really, really boring. I hate coming here.”

Civic Enterprises has released its latest study, Raising Their Voices, concerning America’s dropout crisis. What resonated with me the most was the voices of the students in the report. Here are some samples:

“To me, high school is like elementary and middle school. It’s all the same. We’ve been doing the same thing over and over again.”

“If you just fight your way through it now and get through school ... eventually it will be interesting when you get into your career field.”

“I’m going to be honest: school is really, really boring. I hate coming here.”

Issue 1: Student boredom

I hate coming here. If you just fight your way through it. The same thing over and over again. These are pretty damning words. They also are pretty common. As the report noted, many students view high school as something that must be tolerated as a stepping-stone to [something] better (emphasis added). 

When’s the last time your school organization asked its students how interesting and engaging their classes were (and then took their responses seriously)?

Issue 2: Meaningful community discussion

The researchers brought together students, parents, and teachers in four different communities to collaboratively discuss the high school dropout program in their local area. In each case, individuals remarked that this was the first time that teachers, parents, and students had been brought together to talk about any issue, including the dropout crisis (emphasis added).

When’s the last time your school organization had teachers, parents, and students (and, yes, administrators) in the same room talking candidly and safely about important issues?

Issue 3: Disconnects between groups

The report noted that:

while dropouts cited boredom as the leading cause for dropping out, many educators we surveyed did not see this as the central cause. In fact, only 20 percent of teachers saw a student’s lack of interest in school as a major factor in most cases of dropout. More than twice as many believed students were making excuses for their failure to graduate. . . .

Additionally, although students said that higher expectations would have mitigated the factors leading to their dropping out, only 32 percent of teachers agreed that we should expect all students to meet high academic standards and graduate with the skills that would enable them to do college-level work, and that we should provide extra support to struggling students to help them meet those standards.

These disconnects exist everywhere, of course. No organization is immune from them. But perception shapes reality. If students say they’re bored and teachers just think students are making excuses and don’t reflect on their own instructional practices, the problem never gets solved.

When’s the last time your school organization intentionally worked to uncover and then meaningfully address existing cognitive, emotional, and perceptual disconnects between groups?

Wrap-up

The Raising Their Voices study was conducted on behalf of the AT&T Foundation and the America’s Promise Alliance. The report illustrates the kind of conversations that can occur when you bring disparate groups of school stakeholders together. It also shows that disconnects between groups can be effectively bridged through structured dialogue and a spirit of mutual respect. The report includes recruiting instructions and a sample discussion guide to help schools set up their own local focus groups. As school leaders, we should do this more…

Happy reading!

[cross-posted at LeaderTalk]

Single-media schools, multimedia world

If a picture tells a thousand words, then the two images below from a recent report by the Global Information Industry Center at the University of California, San Diego are of interest. The first image shows the average American’s hourly information consumption per day. Note that the small yellow wedge represents printed text, which of course is the overwhelmingly dominant information medium in P-12 schools.

Hourlyinformationconsumption

The second image shows the decreasing prevalence of printed text in our lives since 1960:

Hourlyinformationconsumption2

These data represent average Americans. I’m sure they would look different if we just looked at our younger generations.

It’s simple, really:

Singlemediaschools

How long are American schools going to get away with these kinds of expansive disconnects between how we consume information in schools and in our daily lives?

Switch to our mobile site