Archive | Research and Evaluation RSS feed for this section

“Time on task” is less important for higher-level understanding and creativity

the nature of the task helps to determine the relationship between time and achievement. It turns out that more hours are least likely to produce better outcomes when understanding or creativity is involved. “How much is learned by rote is a direct function of time and effort,” acknowledges literacy expert Frank Smith. “But when the learning is meaningful we learn much faster. . . . Having to spend long periods of time in repetitive efforts to learn specific things is a sign that learning is not taking place, that we are not in a productive learning situation.”

Sure enough, researchers have found that when children are taught to read by focusing on the meaning of the text rather than primarily on phonetic skills, their learning does “not depend on amount of instructional time.” In math, too, even the new-and-improved concept of “engaged” [time on task] is directly correlated to achievement only if both the activities and the measure of achievement are focused on rote recall. By contrast, there is no “linear positive relationship for higher level mathematics activities, including mathematical applications and problem solving.”

Alfie Kohn via http://www.joebower.org/2012/08/limits-of-time-on-task.html

Teachers are only part of the student learning equation

Here’s a slightly-modified version of a comment I left over at The Des Moines Register regarding teachers’ impacts on student learning outcomes…

Sure, what happens in the classroom matters. But peer-reviewed research shows over and over again that between 2/3 and 4/5 of student achievement is based on non-school factors. Schools only contribute about 20% to 33% to students’ overall learning outcomes.

In addition, teachers are only part of the school equation. They’re the most important part, but non-teacher factors such as administrators, curriculum, other students in the school, available learning resources, and so on also impact student achievement. So teachers are responsible for about half (or so) of the school impact, but the rest lies outside their domain.

When you add all of this up, good teachers clearly are absolutely critical to student academic success. But their overall impact on student learning falls around 10% to 17%. Other in-school and out-of-school factors account for the rest. What this means is that – the occasional tale of heroic, exceptional teachers and schools aside – we should be making state and national policy based on what the research shows generally occurs, not exceptions, anecdotes, personal intuition, or unsubstantiated policy/political claims. And we definitely should not be holding teachers 100% accountable for outcomes for which they’re only 1/6 to 1/10 influential.

We need a much broader (and smarter) conversation about what it means to educate our nation’s children.

Image credit: Bigstock, Teacher and students

About those PISA scores…

Here are PISA scores for 15-year-olds as typically presented by politicians. When you see these, it’s easier for Americans to be more alarmist (oh no, others are ahead of us!).

Average scores

AveragePISAscores 

Percentage of high scorers

 PercentagePISAhighscorers

But we also should look at the data another way.

Number of high scorers

NumberPISAhighscorers

Our size benefits us, and we need to remember that too. [of course we have many more LOW scorers as well…]

More data at American Achievement in International Perspective.

What does the research say about school one-to-one computing initiatives?

CastleLogo 300dpi 400pixels

Today’s resource is a re-issue of an older version. Last November we rolled out our first CASTLE brief. We’ve now got a new, much better template for our briefs so here’s the revised edition, authored by Dr. Nick Sauers, 1to1 Schools, and myself:

If you downloaded the old one, please use this one instead! Our second brief is in the works and, as the CASTLE Briefs web page notes, we’re actively soliciting ideas and authors. If you’re interested, get in touch!

[Continuing what I hope will be a month-long wave of resources for school leaders and the programs that prepare them…]

Focusing on superintendents: 5 technology leadership articles from AASA

AASA logo

Kicking off what I hope is an awesome, seemingly-endless month of resources for school leaders and the programs that prepare them, today I thought I’d share 5 technology leadership articles from AASA’s School Administrator magazine. All 5 focus on superintendents and feature either my thinking or my research.

  1. Blocking the future (May 2008). Superintendents may not have all the answers but they should at least have the right mindset. Are your leaders’ primary orientations toward enabling or blocking? 
  2. Rethinking technology restrictions in school (April 2012). Prohibition (i.e., overly-restrictive technology filtering and blocking) doesn’t work, whether for alcohol or school technology. It’s also inconsistent with how administrators approach non-technology-related school discipline issues.
  3. Responsibility for asking the right questions (November 2007). Superintendents may not be technology-savvy themselves, but that doesn’t mean they can’t ask better questions.
  4. The most important tool you probably don’t know (September 2011). RSS readers can be incredibly powerful tools for superintendents’ professional and personal learning.
  5. Online credentials: A state of wariness (September 2010). More teachers are getting their principal credentials from online Educational Leadership programs. But are they able to get jobs? This article highlights some of my research and was authored by my primary research partner and CASTLE Co-Director, Dr. Jayson Richardson.

Happy reading!

The promise of the digital

The promise of the digital is not in the way it allows us to ask new questions because of digital tools or because of new methodologies made possible by those tools. The promise is in the way the digital reshapes the representation, sharing, and discussion of knowledge.

Mark Sample via http://journalofdigitalhumanities.org/1-1/conversations/building-and-sharing-when-youre-supposed-to-be-teaching-by-mark-sample/.

What should I ask principals of problem-based learning schools? [HELP WANTED]

I want to interview principals of schools that are dedicated to problem-, inquiry-, and/or challenge-based learning. You know, schools like those in the New Tech, Expeditionary Learning, Big Picture, and other similar networks. In particular, I’m interested in how they’re balancing their hands-on, student-directed learning missions with the demands of NCLB, AYP, and other external accountability policy mechanisms.

Do they care about accountability assessments? Do they even pay attention to them? How do their students do on those exams? And so on…

We’re creating our interview protocol right now. What would YOU ask them related to implementation, instruction, curriculum, and/or assessment? Thanks for any suggestions you can provide!

Connected learning resources and infographic

Fresh from the Digital Media & Learning conference in San Francisco are two new web resources, Connected Learning and the Connected Learning Research Network. The work is centered around three learning principles and three design principles:

Learning principles

  • Interest-powered. Interests power the drive to acquire knowledge and expertise. Research shows that learners who are interested in what they are learning, achieve higher order learning outcomes. Connected learning does not just rely on the innate interests of the individual learner, but views interests and passions as something to be actively developed in the context of personalized learning pathways that allow for specialized and diverse identities and interests.
  • Peer-supported. Learning in the context of peer interaction is engaging and participatory. Research shows that among friends and peers, young people fluidly contribute, share, and give feedback to one another, producing powerful learning. Connected learning research demonstrates that peer learning need not be peer-isolated. In the context of interest-driven activity, adult participation is welcomed by young people. Although expertise and roles in peer learning can differ based on age and experience, everyone gives feedback to one another and can contribute and share their knowledge and views.
  • Academically oriented. Educational institutions are centered on the principle that intellectual growth thrives when learning is directed towards academic achievement and excellence. Connected learning recognizes the importance of academic success for intellectual growth and as an avenue towards economic and political opportunity. Peer culture and interest-driven activity needs to be connected to academic subjects, institutions, and credentials for diverse young people to realize these opportunities. Connected learning mines and translates popular peer culture and community-based knowledge for academic relevance.

Design principles

  • Shared purpose. Connected learning environments are populated with adults and peers who share interests and are contributing to a common purpose. Today’s social media and web-based communities provide exceptional opportunities for learners, parents, caring adults, teachers, and peers in diverse and specialized areas of interest to engage in shared projects and inquiry. Cross-generational learning and connection thrives when centered on common interests and goals.
  • Production-centered. Connected learning environments are designed around production, providing tools and opportunities for learners to produce, circulate, curate, and comment on media. Learning that comes from actively creating, making, producing, experimenting, remixing, decoding, and designing, fosters skills and dispositions for lifelong learning and productive contributions to today’s rapidly changing work and political conditions.
  • Openly networked. Connected learning environments are designed around networks that link together institutions and groups across various sectors, including popular culture, educational institutions, home, and interest communities. Learning resources, tools, and materials are abundant, accessible and visible across these settings and available through open, networked platforms and public-interest policies that protect our collective rights to circulate and access knowledge and culture. Learning is most resilient when it is linked and reinforced across settings of home, school, peer culture and community.

Infographic

Below is an infographic made by Dachis Group that highlights these essential components of connected learning. What if every learning environment was centered around these principles?

Connected Learning

Where does 21st Century teaching begin? [Guest Blog]


startingpoint

The 21st Century Teaching Project Findings (Part 2)

Seann Dikkers  3/1/12

This post is part of an ongoing series abridged from the 21st Century Teaching Project (21CTP) – a study of expert professional development trajectories and digital age practice. 

Let’s assume that the goal of teacher training and professional development (PD) is to prepare teachers with powerful models, tools, and pedagogies that will inform expert practice over a career. If so, the 21CTP is designed to help us as a community, 1) hear from 39 award winning teachers, and 2) ask relevant questions about how to study and design teacher training and PD in the coming years.

When over half of these teachers say they completely changed their practice mid-career, I’m particularly interested in what, who, and how those trajectories started. In the first part of this series, I shared one data point on what wasn’t working. The following posts will highlight what was working, who did support these teachers, and how they did grow into expert practitioners.

21CTP Theme 2: Narrated Beginnings

A beginning narrative explains ‘what started it all?’ or ‘where did you first start thinking about?’ practice in the classroom. These questions inform essential beliefs, experiences, and exposure that is relevant to expert practitioners.

Immediately, ‘best practice’ studies are designed to give indications for expanded inquiry. Ideally, we can be given new insights toward recreating similar narratives with a similar end. (By the way, thanks for the clear and helpful feedback from the last post! You are all a gift and the comments are largely being integrated into the final write up of the study.) So we begin simply by asking those that are doing what we hope to see more of, ‘How did you do that?’ – then we listen.

My own assumption was that award winning teachers were going to be those that entered their professional life with a sort of ‘gift’. Their spark of life, talent, and refinement would eventually lead them to promotion and recognition – they were just gifted. They would have begun with a clear vision for expert practice and simply grow towards it. These teachers would have a ‘positive predisposition’ towards expert practice.

Upon completing the first phase of interviews, not one of the preliminary teachers fit this model. Instead I found teachers that claimed to have actually started teaching with faulty predispositions requiring change before they tapped into digital resources, paradigm shifts, and other teachers with great ideas to copy. Instead of having an internal compass, these teachers grew in a community of practice, looked for new tools, and laughed about epic failures as they learned and grew. They weren’t ‘gifted’ as much as they were ‘growing’ the way the rest of us do.

In fact, data from the 21CTP revealed four distinct ‘beginning narratives’’. For a full reading of all 39 stories, check here

Positive Predisposition

In the expanded round of interviews there were in fact teachers that had a great model of teaching they were seeking to resemble, and simply worked towards it. 15% of the teachers fit this beginning narrative I’ll call a ‘positive predisposition’ toward expert practice.

These narratives generally agreed:

“I have always taught the way I do now but I try to constantly try to find new ways and innovative ways to teach so, I’m a constant learner myself. I like to try new things.”

These teachers often had examples that they were trying to follow.

“I remember another elementary teacher who was very active and action oriented. She would act something out every day… I think that is the person I am trying to emulate.”

Those with a positive predisposition shared similar accounting of where they started on a path toward expert practice. They claimed to have always taught they way they did and often had a clear role-model they were trying to emulate.

Progressive Predisposition/Change

Not all teachers shared a positive role-model. On the contrary some entered the profession itching to change things or re-create their practice to look different from their past experiences. 28% of the 21CTP teachers fit this profile:

“Even in my early teaching, I was looking for a different approach towards teaching and learning.”

A progressive predisposition is equally powerful as a starting point for PD on the part of these teachers. However, lacking actual models, they often feel pressure from ‘the system’ and often reported looking outside the profession for new models.

“Again, because there is still a lot of pressure for the test and just getting things done.”

In year three, for instance, one teacher was “exhausted” and took a leave of absence. Upon returning, he reported re-connecting to, “The stuff I enjoy doing outside of school…” Refreshed, he was “always learning something new.”

For both predispositions, teachers were always looking for new ideas and tools to help them grow in the classroom. They held a constant idea of what they wanted and grew over time towards these mental models. 43% of 21CTP participants had a predisposed vision for teaching they continually worked toward. 
 

External Influence

Theme 1 noted what wasn’t necessarily working for expert teachers. From here forward this study turns to what was working for these teachers. For 57% of teachers, they changed their practice mid-career. It can’t be understated how relevant ongoing PD is for expert practice for these teachers.

The first narrative that experienced a change in disposition fit a profile where they experienced a person, tool, or PD program that they report was the start of a new way to practice their craft. Like those with a positive predisposition, these teachers identified a model of practice, through external influence, that became a driving goal. For instance, in the preliminary phase, one teacher credited their social network:

“Developing networking early on… Just sharing ideas, the basic web 2.0 type practices, ideas, tips, software with other educators within my state and increasing abroad. Shortly thereafter, within a year or so, I began to look at integration of video games and video technology into the classroom.” 

There was a laundry list of external influences that seemed unique to each person. In Theme 3, I’ll break down traditional and non-traditional PD assets and the degree to which the teachers were influenced by them. 23% of the teachers named these programs, people, and tools as the starting point for their changed practice.

Sudden Realization

The largest percentage of the 21CTP teachers reported a “sudden realization” or specific moment they could recall. Much like remembering where they were when they heard a major news story, these teachers had a moment when they perceived their own practice as deficient and in need of change. For progressive change narratives, they didn’t yet have a positive model, but recognized what they couldn’t do anymore. For example: 

“I remember crashing and burning real bad on what I would consider traditional lectures.”

“We all love our field, it’s so horrible to feel like you are torturing someone with the things you are passionate about.”

“I wasn’t bold and brazen, I was naive.”

“I had the moment where I realized I was teaching the same way my teachers taught me in high school and I was bored then and I was looking at some of my students who I knew were bright and energetic, lively kids and I could tell they were bored.”

These teachers (33%) did not consider themselves experts initially. They reported a simple realization that what they were doing wasn’t going to work anymore. They changed as a reaction and began doing anything else to garner better classroom results – starting a PD journey from a ‘sudden realization’.

Untitledtheme 2 table

So what?

Our best practitioners have told us that there are at least four beginning narratives toward award winning practice; it is a stretch to claim there is a predominant beginning narrative. Teachers can have positive models of practice, or a negative one. Teachers can enter the profession with a predisposed vision of practice, or not. Sources of change can be internal processes, or externally affected. As with our students, there are multiple types/paths for learning among adults. School leadership cannot afford to think that there is only one way to build expert practice. There is no ‘one size fits all’ that actually works for all.

I spend more time in the larger write up on this section noting that these beginnings don’t appear to be exclusive. Teachers noted one as primary, but often shared the importance of others.  

Also, among calls for reform, this data reconfirms past research that teacher beliefs about practice are significant PD components (see lit sources below). Some traditional models of training and PD (especially ones that provide models of practice – good or bad) should be clung to instead of thrown out with the bathwater primarily because, for some of the teachers, they work.

Finally, much of the PD field claims one progression for change: 1) Teacher learns, 2) Teacher changes practice, and 3) Student learning increases. For a significant portion of our sample, this was not what they claimed happened. For these teachers, they claimed they: 1) Got frustrated, 2) Changed practice, 3) Learned over time, 4) Student motivation increased, then 5) Student learning was enriched. Knowing does not necessarily preclude doing for these teachers.

In Theme 3, I’ll share a closer look at traditional and emergent resources reported as essential, or not so much, to these teachers. What worked, what didn’t. Among the leading PD assets: Effective Leadership, a Community of Practice, and New Media digital tools and resources. More to come…

Blog Discussion:

  1. Do all teachers with positive role-models progress toward them over time? Really a larger study of a random sample of teachers could gather this and more about predispositions. In schools, talk to teachers and find out if there are predispositions that drive their PD, vise versa, or both.
  2. There isn’t a clear ‘best’ beginning narrative, which means many types of beginnings can work towards expertise – not just “gifted” people. I find this encouraging to the rest of us! The data actually slants just a bit toward narratives where the teacher was “crashing and burning”, then resolved to be better. Never give up on teachers willing to grow, today’s worst teachers may be winning awards tomorrow if they are ready to try new things.
  3. Buffet style PD is a growing technique for district and building level training. Are these models intuitively accepted more easily because they are addressing actual adult learning styles more effectively? Teacher selected PD opportunities should at least be targeted for further expanded study, at best these should be the default for district level leaders.
  4. For all four beginning narratives, teachers had or sought a better way to teach. What they called the traditional model of ‘sage on the stage‘ or ‘grill and drill‘ was obsolete – which is expected of these participants – but not to be understated.

– 

Seann Dikkers is a researcher and dissertator in educational technologies at the University of Wisconsin – Madison. Dikkers spent fourteen years as in the public schools as a teacher, principal, and consultant. Dikkers has presented nationwide as a designer and consultant in new media integration strategies for educational leadership, teaching, and learning. His design and research bridges education leadership and curriculum and instruction scholarship – including CivWorld, ParkQuest, History in our Hands, Mobile Media Learning, Augmented Reality and Interactive Storytelling editor (ARIS), the Comprehensive Assessment for Leadership in Learning (CALL), and the Teacher’s Toolbox. Dikkers edited the recent release of Real-Time Research: Improvisational Game Scholarship and is the founder/president of GamingMatter. Currently, Dikkers is in the process of interviewing award winning teachers across the country to find out strategies for professional development growth in digital media use.

– 

Addenda:

Beliefs of practice aren’t conclusive, but they can be informative for a field of study. For those that are interested in more detail, this study is built to complement current evidence being gathered on PD (Desimone, 2011), revisiting teachers as case units (Borko, 2004), accomplished examples of practice (Sheingold & Hadley, 1990), and beliefs that affect practice (Calderhead, 1996; Pajares, 1992; Ertmer, 2005), in a time of emergent digital skills and ‘literacies, (Trilling & Fadel, 2009; Lankshear & Knobel, 2008; Collins & Halverson, 2009). That’s the short version. Though done and IRB approved, the full lit review will be approved for posting in the next couple weeks at the project home site – along with detailed descriptions of selection, collection and analysis methods. Look for it at: 21 Century Teaching Project.

Borko, H. (2004). Professional Development and Teacher Learning: Mapping the Terrain. Educational Researcher, 33(8), 3-15.

Calderhead, J. (1996). Teachers: Beliefs and knowledge. In D. Berliner & R. Calfee (Eds.), Handbook of Educational Psycology (pp. 709-725). New York: Macmillan Library Reference.

Collins, A., & Halverson, R. (2009). Rethinking education in the age of technology : the digital revolution and schooling in America. New York: Teachers College Press.

Desimone, L. M. (2011). A Primer on Effective Professional Development. Phi Delta Kappan, 92(6), 68-71.

Ertmer, P. A. (2005). Teacher Pedagogical Beliefs: The Final Frontier in Our Quest for Technology Integration? . Education Technology Research and Development, 53(4), 25-39.

Lankshear, C., & Knobel, M. (Eds.). (2008). Digital Literacies: Concepts, Policies, and Practices (Vol. 30). New York, NY: Peter Lang.

Pajares, M. F. (1992). Teachers’ beliefs and educational research: Cleaning up a messy construct. Review of Educational Research, 62(3), 307-332.

Sheingold, K., & Hadley, M. (1990). Accomplished Teachers: Integrating computers into
classroom practice. New York: Centre for Technology in Educaiton.

Trilling, B., & Fadel, C. (2009). 21 Century Skills: Learning for Life in our Times. San Francisco, CA: Jossey-Bass.

Switch to our mobile site