Dangerously Irrelevant

The Economic Policy Institute’s new report, Problems with the Use of Student Test Scores to Evaluate Teachers, cautions against heavy reliance on the use of test scores in teacher evaluation.

Authors of the report include four former presidents of the American Educational Research Association; two former presidents of the National Council on Measurement in Education; the current and two former chairs of the Board of Testing and Assessment of the National Research Council of the National Academy of Sciences; the president-elect of the Association for Public Policy Analysis and Management; the former director of the Educational Testing Service's Policy Information Center and a former associate director of the National Assessment of Educational Progress; a former assistant U.S. Secretary of Education; a former and current member of the National Assessment Governing Board; and the current vice-president, a former president, and three other members of the National Academy of Education.

Civic Enterprises has released its latest study, Raising Their Voices, concerning America’s dropout crisis. What resonated with me the most was the voices of the students in the report. Here are some samples:

“To me, high school is like elementary and middle school. It’s all the same. We’ve been doing the same thing over and over again.”

“If you just fight your way through it now and get through school ... eventually it will be interesting when you get into your career field.”

“I’m going to be honest: school is really, really boring. I hate coming here.”

Issue 1: Student boredom

I hate coming here. If you just fight your way through it. The same thing over and over again. These are pretty damning words. They also are pretty common. As the report noted, many students view high school as something that must be tolerated as a stepping-stone to [something] better (emphasis added). 

When’s the last time your school organization asked its students how interesting and engaging their classes were (and then took their responses seriously)?

Issue 2: Meaningful community discussion

The researchers brought together students, parents, and teachers in four different communities to collaboratively discuss the high school dropout program in their local area. In each case, individuals remarked that this was the first time that teachers, parents, and students had been brought together to talk about any issue, including the dropout crisis (emphasis added).

When’s the last time your school organization had teachers, parents, and students (and, yes, administrators) in the same room talking candidly and safely about important issues?

Issue 3: Disconnects between groups

The report noted that:

while dropouts cited boredom as the leading cause for dropping out, many educators we surveyed did not see this as the central cause. In fact, only 20 percent of teachers saw a student’s lack of interest in school as a major factor in most cases of dropout. More than twice as many believed students were making excuses for their failure to graduate. . . .

Additionally, although students said that higher expectations would have mitigated the factors leading to their dropping out, only 32 percent of teachers agreed that we should expect all students to meet high academic standards and graduate with the skills that would enable them to do college-level work, and that we should provide extra support to struggling students to help them meet those standards.

These disconnects exist everywhere, of course. No organization is immune from them. But perception shapes reality. If students say they’re bored and teachers just think students are making excuses and don’t reflect on their own instructional practices, the problem never gets solved.

When’s the last time your school organization intentionally worked to uncover and then meaningfully address existing cognitive, emotional, and perceptual disconnects between groups?

Wrap-up

The Raising Their Voices study was conducted on behalf of the AT&T Foundation and the America’s Promise Alliance. The report illustrates the kind of conversations that can occur when you bring disparate groups of school stakeholders together. It also shows that disconnects between groups can be effectively bridged through structured dialogue and a spirit of mutual respect. The report includes recruiting instructions and a sample discussion guide to help schools set up their own local focus groups. As school leaders, we should do this more…

Happy reading!

[cross-posted at LeaderTalk]

If a picture tells a thousand words, then the two images below from a recent report by the Global Information Industry Center at the University of California, San Diego are of interest. The first image shows the average American’s hourly information consumption per day. Note that the small yellow wedge represents printed text, which of course is the overwhelmingly dominant information medium in P-12 schools.

The second image shows the decreasing prevalence of printed text in our lives since 1960:

These data represent average Americans. I’m sure they would look different if we just looked at our younger generations.

It’s simple, really:

How long are American schools going to get away with these kinds of expansive disconnects between how we consume information in schools and in our daily lives?

The University Council for Educational Administration (UCEA) has started a new podcast channel and will be interviewing top educational leadership scholars from around the world about school administrator research and/or preparation.

The first live podcast will occur tomorrow and will feature Dr. Karen Seashore, who holds the Rodney Wallace Professorship for the Advancement of Teaching and Learning at the University of Minnesota. Karen will be discussing the linkages between P-12 school leadership and student academic learning outcomes. As a former faculty colleague of Karen’s, I know her well and anticipate that this will be a great conversation.

• Listen live to the interview with Dr. Karen Seashore
• Download the interview afterward
• Learn more about UCEA’s interview series

We here at CASTLE have been working with UCEA to get this up and running. CASTLE is one of several UCEA centers; I also serve as UCEA’s Associate Director of Communications. Stay tuned to the podcast channel to hear future interviews and, while you’re at it, sign up for UCEA’s Twitter feed if you’re interested in more school leadership resources.

Happy listening!

Here are two quotes from Education and Learning to Think, an interesting little research-based book published by the National Research Council way back in 1987!

Higher order thinking is nonalgorithmic. That is, the path of action is not fully specified in advance.

Higher order thinking tends to be complex. The total path is not “visible” (mentally speaking) from any single vantage point.

Higher order thinking often yields multiple solutions, each with costs and benefits, rather than unique solutions.

Higher order thinking involves nuanced judgment and interpretation.

Higher order thinking involves the application of multiple criteria, which sometimes conflict with one another.

Higher order thinking often involves uncertainty. Not everything that bears on the task at hand is known.

Higher order thinking involves self-regulation of the thinking process. We do not recognize higher order thinking in an individual when someone else “calls the plays” at every step.

Higher order thinking involves imposing meaning, finding structure in apparent disorder.

Higher order thinking is effortful. There is considerable mental work involved in the kinds of elaborations and judgments required. (p. 3)

The seventh item on the list, self-regulation, is one that I think is particularly lacking in many K-12 schools because the teachers “call the plays” so much of the time…

Here’s what I think is the money quote:

The goals of increasing thinking and reasoning ability are old ones for educators. . . . But these goals were part of the high literacy tradition; they did not, by and large, apply to the more recent schools for the masses. Although it is not new to include thinking, problem solving, and reasoning in someone’s school curriculum, it is new to include it in everyone’s curriculum. It is new to take seriously the aspiration of making thinking and problem solving a regular part of a school program for all of the population . . . It is a new challenge to develop educational programs that assume that all individuals, not just an elite, can become competent thinkers. (p. 7)

I liked this book. It's very short, but it made me think. I give it 4 highlighters.

Here are my notes from Dr. Yong Zhao’s presentation, Catching Up or Leading the Way: American Education in the Age of Globalization, at the 2009 School Administrators of Iowa (SAI) conference…

• Dr. Zhao’s book will be out soon from ASCD.
• Thomas Friedman: “When I was growing up, my parents used to say to me: ‘Finish your dinner — people in China are starving.’ I, by contrast, find myself wanting to say to my daughters: ‘Finish your homework — people in China and India are starving for your job.’”
• When something bad happens in America, it has to do with education. When something good happens in America, it has to do with some politician.
• March 24, 1958 article in LIFE magazine comparing Russian students’ work with American students’ was very similar to the rhetoric of 2 Million Minutes.
• 1983, A Nation at Risk: “a rising tide of mediocrity” and “we are raising a new generation of Americans that is scientifically and technologically illiterate.”
• Achieve, the College Board, and ACT are writing the upcoming national standards. Ask yourself who stands to benefit from the new standards?
• 2 Million Minutes: now the ‘enemy’ is China and India. Dr. Zhao grew up in China and is back there almost every month. He disagrees with Bob Compton, the director.
• The USA continues to be the most economically competitive country in the world. We continue to be the most innovative, as measured by patents issued. And of course we are the most open, democratic.
• No other country comes close to the US when it comes to exports of intellectual property / knowledge (patents, royalties, copyrights, license fees). China dominates toy exports, not knowledge exports. China is a country built on cheap labor, not knowledge.
• If the US educational system is so bad, why are other countries (like China) trying to emulate us (see, e.g., China’s 2002 and 2005 curriculum and assessment reforms).
• Singapore is emphasizing the explicit teaching of critical and creative thinking skills.
• The correlation between the 1964 First International Math Study test scores (FIMS) and economic output, hourly productivity, quality of life, etc. 40 hours later are all negative. Democracy, creativity, livability all have no relationship or a negative relationship with the FIMS scores.
• Recommends reading Day of Empire: How HyperPowers Rise to Global Dominance – And Why They Fail, by Amy Chua, and The Rise of the Creative Class, by Richard Florida. Diversity of talents, creativity, entrepeneurship, and passion are what allow nations to thrive.
• We are busy closing the achievement gap. Asian countries are busy closing the creativity gap.
• The strengths of American education: school talent shows (value individual talents, inspires passion and responsbility, tolerate deviation, cultivate entrepeneurship) and children are popcorn (some pop early and some pop late; respect individual differences; have faith in every child; second, third, and fourth chances)
• Creativity is fundamentally to be different. America is a society that tolerates, values, and celebrates difference.
• We do face new challenges. For example, globalization (i.e., the death of distance). Columbus took about 3 months to get from Spain to the Bahamas. Now it takes 13 hours on airplane. Electronic information, money, voice phone calls, etc. now get there instantly.
• Global supply chains: corporations can fragment their production and distribute it wherever it makes sense (outsourcing of labor). Products are now made from parts that come from a multitude of different countries.
• New Commission on the Skills of the American Workforce: “Today, Indian engineers make $7,500 a year against $45,000 for an American engineer with the same qualifications. If we succeed in matching the very high levels of mastery of mathematics and science of these Indian engineers - an enormous challenge for this country - why would the world’s employers pay us more than they have to pay the Indians to do their work? They would be willing to do that only if we could offer something that the Chinese and Indians, and others, cannot.”
• Equalling China’s or India’s performance on standardized tests doesn’t differentiate ourselves, doesn’t allow us to offer what others cannot.
• Problems in other countries also affect everyone now (e.g., swine flu, poverty, violence, financial meltdowns). We no longer can keep ourselves isolated.
• Where does the new hope lie? With Madonna and eBay! Many people don’t like Madonna. But across the globe, there are enough who do for her to be successful. Globalization allows us to find the other crazy people across the world who find value in what we offer. eBay allows us to find others outside our local area who will buy our trash. Globalization expands our audiences and allows our skills, talents, products, etc. to find new places that they can thrive. What do you have that others don’t? What can you contribute to other markets? What’s your niche?
• Societal changes create new job opportunities. Right now, for example, we need people who understand other countries’ cultures, languages, politics, etc.
• Technology demands new skills (e.g., virtual designers). Virtual “gold farming” is now a $2 billion industry worldwide.
• The Partnership for the 21st Century Skills framework is too long and complex. Our problem in America is that we keep adding, we never take away.
• See Jenifer Fox’s www.strengthsmovement.com.
• We need to take technology and the digital world SERIOUSLY.
• Every child should have a personalized curriculum. This is happening in other countries (e.g., United Kingdom).
• We should think of schools as global enterprises, not local entities, and draw on global resources.
• Never send a man to do a machine’s job. Let people and computers each do what they do best.
• Slides available at http://zhao.educ.msu.edu

The New York Times reports today on a study of MySpace users from the United Kingdom. Here is a quote from the article:

The MySpace study asked social networking users between the ages of 14 and 21 (aka "Generation Y") questions about their interactions both on social networks and in their real life, too. Some 36% of the respondents said they found it easier to talk about themselves online than in the real world, leading them to share more about themselves using technology. This group also felt that their online friends knew more about them, and so, in a sense, were closer than offline friends because they all knew what was going on in each other's lives.

Outside of the social networking sites, the survey respondents overwhelmingly felt ill-at-ease in social groups. A whopping 72% said they felt "left out" and didn't think they fit into any particular group. More than four-fifths (82%) said they moved between four or more different groups of friends in an effort to find acceptance.

So is this…

1. Proof that Internet naysayers are correct that the Web is eroding young people’s ability to communicate effectively face-to-face?, or
2. Proof that Internet advocates are correct that young people are breaking free of local, geographical constraints to find affinity groups that matter to them?

In other words, are these results something about which to be concerned or to celebrate? Both? Neither? Thoughts?

I tweeted:

And Barry Dahl replied:

Barry’s right and I’m wrong. I failed my own information literacy test. Why? Because even though I had access to (and linked to) the original report, I didn’t critically consume it the way I should have. Instead I relied on this report from CNN:

And because I did, I made an incorrect statement that then got retweeted by others. Shame on CNN for being misleading and/or inaccurate, but shame on me too for not doing my homework the way I should have. Just because CNN is a traditional, reputable news organization doesn’t mean that I don’t need to be a critical consumer of the information it provides.

Thanks, Barry.

In my never-ending quest to wrap my head around workforce data despite no background or training whatsoever, I’ve been playing around with the Bureau of Labor Statistics (BLS) web site. But first a quick look at General Motors (GM)!  [click on all images for larger versions]

General Motors has a shrinkage issue

As many of you know, GM has been in the news lately as it faces possible bankruptcy proceedings. The image below shows the shrinkage of GM’s workforce over a generation.

Combine this image with all of the other news on the U.S. automobile industry and it’s easy to see that automotive jobs in America, at least as they’ve traditionally been configured, often are a loser’s game due to lower costs and, often, higher quality overseas.

Hey, how are we supposed to make a living?

Below are two charts that I made after diving deep into the BLS Industries At A Glance data, particularly the historical trend data. The first chart shows that the number of employees in the professional and business services, financial activities, and education and health services supersectors grew substantially over the past three decades. In contrast, the manufacturing supersector has lost over a third of its employees and those job losses show no signs of slowing down any time soon. Of course the education and skills needed for these growth sectors of the American economy are different and/or higher than those needed for most manufacturing jobs. FYI, the data points are from the month of April for each year.

The second chart shows the average increase in real earnings since 1980, broken out by labor supersector and adjusted for inflation. As you can see, not only are manufacturing jobs disappearing, those that are left actually have seen a decline in inflation-adjusted earnings over the past three decades. In other words, the purchasing power of your average manufacturing employee is less than it was three decades ago. Not so for the other three supersectors in the chart. I’m no workforce expert but this doesn’t seem to make a strong argument for the manufacturing industry here in America until our companies figure out how to effectively navigate overseas competition despite higher wages, corporate health care and other legacy costs, Americans’ expectations regarding standard of living, and other issues.

I’m not completely sure what to make of all of this. Right now I’m trying to locate data and present them in ways that make sense to me because I have a sense that this stuff is pretty important. As I share this out, your thoughts and expertise are welcome!

One last thing

FYI, despite my best efforts with it, Wolfram Alpha was of no help whatsoever with this investigation. Maybe down the road as it gets more sophisticated, increases its store of data, etc.

Related posts

• Immunizing your graduates from economic downturns
• Iowa - 21st century curricula

Here are two presentations by Dr. Richard Miller, Chair of the English Department at Rutgers University, that are well worth any university instructor's time to watch.

The Future Is Now: Presentation to the RU Board of Governors 

This Is How We Dream, Part 1 and Part 2