[Warning: Long post ahead]
Yesterday I took the ACT college entrance exam for the first time. At age 44.
It all started with Ira Socol’s blog post, which argued that if politicians think that the standardized tests they are espousing are so important, they had better be able to pass those tests themselves. I then sent these tweets:
Iowa legislators & @terrybranstad: Are you brave enough to do this? bit.ly/xFGZNz #iaedfuture #didn‘tthinkso
— Scott McLeod (@mcleod) February 8, 2012
Iowa legislators & @terrybranstad: Could you pass the very tests you espouse? bit.ly/xFGZNz #iaedfuture #didn‘tthinkso
— Scott McLeod (@mcleod) February 9, 2012
Neither Iowa Governor Terry Branstad nor any state legislators responded (surprise!) but Dr. Jason Glass, Director of the Iowa Department of Education, said that he would take the ACT if I would too. The good folks at ACT said that they would be happy to administer a retired test. And that’s how I ended up in a small room at Roosevelt High School in Des Moines with Jason and 8 students who took the exam for practice.
How was the experience?
I took the SAT when I was a kid so the ACT was new territory for me. I’ll break out my thoughts by test area…
English. The English test was primarily a test of grammar, sentence formation, and paragraph flow and structure. I was asked many questions about punctuation and phrasing and word choice (e.g., who or whom or whose or who’s). Occasionally I was asked about spelling (e.g., its v. it’s) or whether a particular sentence or paragraph should be inserted, deleted, and/or moved elsewhere in a reading passage. The focus was primarily on writing composition. At times I felt like I do when I’m helping edit one of my students’ doctoral dissertations! With the caveat that I’m not a writing expert, I felt that this test did a fairly decent job of assessing whether students could identify grammar errors, poor wording, stilted sentence flow, and other technical mistakes in written passages.
Math. I know how to do a number of advanced statistical procedures, including linear regression and hierarchical linear modeling. However, I still thought that this would be the hardest test for me since I haven’t done any geometry or trigonometry since I took those high school courses almost 30 years ago. Most of the test focused on algebraic and geometric concepts. There were a handful of trigonometry questions, plus I was surprised to see a question on logarithms. I correctly completed more problems than I thought I would but – probably due to lack of day-to-day immersion and practice – was correct about the pacing. I had to pencil in last-minute random guesses for several questions because I simply ran out of time. My biggest concerns about the math test relate to the fact that much of what is assessed is math that – and I think I’m safe saying this – most of us will never use again (how many of you have needed to calculate the cosine of an angle recently? how many of you have needed to determine the formula of a circle on a standard coordinate plane?). This is a curricular issue more than an assessment issue since the ACT draws off of the math courses that most high school students take. Smarter people than I have weighed in on what math courses high schoolers should take and I’ll defer to them before I reveal too much of my ignorance. I know some of the arguments about ‘inculcating habits of mind’ and ‘more students might be turned onto higher-level math and science’ and so on. It just bothers me a great deal that we’re herding many, many students through math classes that are largely irrelevant to their future life success (most high school students don’t get much probability and statistics, for example, even though that’s what I think they’ll need most often beyond foundational numeracy). My other big concern about the math test was that the problems generally were either decontextualized pure math problems (students: who cares?) or pseudocontextual word problems (students: who cares?) of the type that Dan Meyer rails against regularly. There wasn’t much on the math test that I think would be of interest to typical high school students outside of the artificial environments of classes and testing. [Note: I’m happy to be proven wrong on any of these concerns, so have at it in the comment area.]
Reading. 4 multi-paragraph reading passages pulled from 1 fictional novel and 3 non-fiction essays or books; 10 questions per passage. Could I pull out essential details from what I read? Could I infer authors’ intent? Could I decipher meaning and voice? Could I make reasonable conclusions based on the text? A classic test of reading comprehension. Yes, I could do these things. This was the test on which I scored best.
Science. This test had 7 passages, each of which contained one or more often-interconnected tables, charts, graphs, maps, or diagrams. Most of the passages described various scientific experiments and most included additional narrative text. This was the hardest test for me, despite having taken numerous advanced science courses in both high school and college. I am comfortable with electrical, genetic, geological, kinetic, chemical, and other scientific terms and concepts and, as a professor, regularly spend time deciphering research studies and policy reports that present information in complex ways. The challenge for me was not understanding the material but rather navigating the sheer amount of information presented and answering the questions within the time given. I easily could have used another 10 to 15 minutes. Like for the math test, I had to hastily pencil in some random guesses and did worst on this test. I am very impressed by any high school students that score well on this test. I’ll also note that ACT admits that the science test doesn’t directly assess scientific knowledge or skills (although some familiarity with scientific concepts and terminology helps for comprehension purposes). Instead, what the test assesses is the ability to decipher various ways of presenting scientific information and to then make appropriate inferences and conclusions. That’s a worthy goal but I wonder if renaming the test to something like ‘Information and Data Analysis’ might be more accurate.
Writing. We didn’t get to take the writing assessment, primarily because of ACT’s desire to do same-day scoring of our results.
How do I now think about the ACT?
Jason and Governor Branstad have proposed legislation requiring every Iowa high school student to take the ACT. Here are some of my thoughts about the exam and its desirability as a statewide mandate…
The issue of time. Time is an issue for any assessment. Students shouldn’t have unlimited time to finish but neither should they have inadequate time. The ACT is intentionally designed to be an assessment that sorts, sifts, selects, and ranks participants. Having now taken it, I wonder how much of that sorting and ranking function is accomplished by benchmarking time of completion to those students who are quicker at computation or faster readers. This is different than benchmarking to difficulty of task. I have a feeling that many students might be more successful if they simply had more time to navigate the assessment and show their understanding.
Cognitive complexity. I confess that the exam often was more difficult than I thought it would be. Despite being exposed beforehand to some practice questions, the overall experience was more demanding and draining than I expected. I didn’t take any practice exams (which might have helped with my pacing) but did spend a few hours reviewing some math formulas and familiarizing myself with the other tests.
Content of the tests. With the exception of the math concerns that I expressed above, it’s hard for me to contend that the skills tested on the ACT aren’t worth knowing. Students and citizens need to be technically-competent writers. Students and citizens need to have knowledge of at least some more-than-basic mathematical concepts. Students and citizens need to be able to comprehend complex texts and information displays. And so on.
I think we can do better. Despite being more impressed with the ACT than I anticipated, I still left the exam wanting more. Although my overall experience was positive and I learned a lot about the exam, I still have the same disposition toward it that I had before. Unlike many school and university assessments, the ACT doesn’t assess too much factual recall. I think that’s good. The exam does, however, focus heavily on procedural knowledge (and provides a variety of contexts in which students can show that knowledge). Occasionally it assesses – in a fairly-limited bubble test way – some application, synthesis, analytical, or inferential skills. But for the most part, the exam does not get at higher-order thinking skills in any substantive, applied, hands-on, performance-based way. Even the writing assessment (from what I can tell from ACT’s materials and what I’ve read about it) can be successfully completed in fairly rote, formulaic ways. If we’re going to ask every student in the state to take a college- and career-readiness exam, it should be an exam worth taking. Despite its long history and deep roots, I’m not convinced that the ACT is it.
What might be some alternatives? Although I’m fairly statistics-savvy, I’m not a psychometrician. And although I know a lot about incorporating data-informed practices into schools, I’m not an expert in large-scale and/or high-stakes assessments, particularly those used for college admissions. I have a generalized interest in these topics but do not live in this space on a day-to-day basis. All that said, I look at learning environments like those provided by the New Tech Network or the Big Picture Schools or the Expeditionary Learning schools or the Science Leadership Academy and I want to see more of that for our students. Those schools – and states like New Hampshire – are working hard to assess students with performance-based assessments rather than (or at least along with) bubble tests. Students have the chance to be innovative and creative. They have the chance to do inquiry-based, interest-based, hands-on work. They are able to show that they are critical thinkers and problem solvers. And then I hear about tests like PISA, the College and Work Readiness Assessment (CWRA), and those given by other countries - tests that purportedly focus on higher-level cognitive skills and give students opportunities to show not just what they know but what they can do with what they know. And I want that for our students too. As it’s currently designed, does the ACT get us there? Nope. Might the ACT be part of a more holistic, multiple-assessment strategy for getting at college- and career-readiness? Perhaps. But it’s not enough by itself. Many hopes are riding on the PARCC and Smarter Balanced assessments that are being created right now. We’ll see if they can fulfill their promise.
I’ll close with a few additional thoughts about the experience of taking the ACT as an adult several decades removed from initial college entry and the proposed participation mandate for all students…
Try it (again). I think it behooves us as adults – particularly those of us involved in education delivery, educator preparation, or educational policy- and decision-making – to be fairly familiar with what we’re asking our youth to do. Many of us still think that schooling and testing and learning and being a child or adolescent are like they were when we were young. They’re not. Despite our general inclinations to believe that our own time in school was the educational golden age worth returning to, we must recognize that the scale, scope, complexity, and demands of course content, curricula, assessments, and culture all have increased dramatically over the years/decades. If you’ve never shadowed a high school student throughout her entire school day, try it. I think you’ll be surprised. Similarly, I encourage you to retake a college entrance exam like Jason and I did. I believe you’ll find it worthwhile and illuminating. And then see if what we’re asking of our youth is what we really want as parents, communities, and citizens.
You’re so brave! I was struck by the sheer number of comments that Jason and I received that expressed disbelief that we would do something like this. Typical statements included variations of ‘You’re so brave! I could never do that!’ and ‘You’re willing to report your score publicly? Really?’ and ‘There’s no way in hell I’d ever take that exam again!’ and so on. I’m still mulling over what it says about us, our schools, and our society when we’re willing and even eager to have our children submit to experiences that we’re not willing to engage in ourselves as adults.
Mandated participation. Like some states, the Des Moines Public Schools has all of its students take the ACT. I asked Roosevelt High School’s principal what that experience was like for lower-achieving students. She said that a great many of them left the exam utterly humiliated. Jason was less concerned about that statement than I was, saying to the press something along the lines of ‘Far worse things will happen in life to those students than sitting through a 4-hour exam for which they’re unprepared.’ I, however, am greatly empathetic toward those adolescents. Do academically-disengaged students really need yet another formal reminder – this one with state and/or national, not just local, weight behind it – that they’re not up to snuff? How is kicking them while they’re down an incentive toward college or career readiness?
Still searching. Jason and I received a very kind offer from the Council for Aid to Education to also take the CWRA. I’m going to take them up on it and am excited about the opportunity. I wonder if I also can take PISA? I’m on an assessment quest. What else should I investigate or take?
The media. Lesson learned: No matter how much you emphasize that you’re focused on learning about and better understanding the substance and process of the college entrance exam that you’re publicly taking, the media and others will inevitably focus on your results. Jason and I both scored better than we anticipated. If you really must know, Google it.
If you’ve read all the way down to here, I appreciate your engagement and will await your comments and feedback. Thanks!
Image credits: Iowa Department of Education
I have always lived by my own belief of “never ask anyone to do that which you are not willing to do yourself”. I agree that understanding through hands-on experience makes for a very fair and educated opinion. Unfortunately I am not sure those are widely shared beliefs. Here’s to hoping that the understanding of others through first-hand experience becomes a trend… very soon.
I applaud your effort in taking the ACT. I think it would be rather fun to retake the assessment again as an adult to compare how I did in high school. I feel that I have learned quite bit more in my adulthood than in high school because I care more about my education. A typical high school student may not have the interest in learning all the material or saving the information for later recall. The analysis provided sounds very familiar to how we assess students in grades 2-11.
Very interesting. It makes me want to take the ACT again. (I took the ACT, SAT twice, and the GRE in my day.)
I always performed less well on the English portions of the exams than the math/analytical/”science” portions. I’m guessing that it was the opposite for you. I’m thinking the individual sections of the tests are biased as to learning styles.
Moreover, as you allude, they do not necessarily assess true skill and understanding. e.g. Just because I haven’t memorized the formula for cosine doesn’t mean I don’t understand the relationship in the sides of a triangle and what that means for “real-world” applications of it. Just because I take the time I need to critically analyze some graphs, text, and numbers, doesn’t mean that I can’t come to an intellectual decision.
You aren’t brave, you are *simply* expanding your learning, as all folks that support education should.
My SAT splits on Verbal-Math were pretty even. So hard to say!
A student at the recently complete 1:1 conference said that if he could change anything about his education it would be that teachers complete the projects or tasks that they ask their students to complete. His thinking was that if teachers did this, it might result in them raising their expectations for their students. Interesting thought that aligns well with your point about adults experiencing first hand what they are asking of their students and children.
Way to go Scott! I couldn’t agree more that those proposing mandates should experience them themselves. I applaud both you and director Glass for taking the ACT and for candidly discussing your experience. As a science teacher I’ve always felt a little uneasy about how standardized science tests are more about reading of texts, graphs, and takes than about any content or process skills. again, great post. Thank you. @dave_glenn
I wish folks would actually walk into a Big Picture Company, see what goes on, and talk to their recent graduates before promoting the model. My experience across three states is virtually all of these schools fail to deliver an adequate education and their graduates and teachers know it. Recent graduates are very impressive on soft-skills; they shake hands, maintain eye contact, generally converse comfortably with adults, and are quite self-aware. This is no small task in an urban environment. However, universally they complain they found they were severely lacking in basic literacy and numeracy skills required for college or advancement beyond entry or low-middle level positions. It’s a perfect example of throwing out the baby with the bath water. What’s frustrating is that teachers at nearby schools, local researchers, and policy wonks who interact with BPC schools virtually all have the same feedback, even across states. Yet legislatures and more casual followers of education all seem to fall in love with the place. It’s like a dirty little secret except almost everyone is in on it.
In our district in Wisconsin we have started to roll out pieces of the Common Core to our teachers. One of the activities we did was to give teachers a middle school math test with CCSS expectations. It was an eye opener, to say the least. It really helped them understand the importance of making sure students can apply the skills they learn. Knowing isn’t enough anymore. Application with creativity and persistence is key.
Thanks, Scott. I have a little, very little, increased appreciation for the ACT. I remain a stauch opponent of all the bubble tests, including ACT and SAT. They do not predict likely future performance, do not serve kids now or their futures, and do not represent anything close to what we know is best assessment practices in education. The obsession with bubble tests is dumbing down and narrowing curriculum.
Yes, it’s amazing that the school where I teach [Urban, Title 1, significant minority population and poverty] has set the state record for number of students advancing to the state level for Junior Academy of Science three years running, yet we are “failing” our students because not enough of them score well on multiple choice questions.
Kudos for doing that Scott. And congrats on scoring well.
I wonder how you would have done had you not prepped at all. And I wonder if that would have been a more accurate reflection of what the ACT assesses. Either way, however, it’s good to hear that there is less of an emphasis on trivia and more on skills.
I agree, however, that the Common Core tests will be interesting to see…
I only spent a few hours reviewing and familiarizing. I didn’t want to go all-out like many students/families do with multiple practice tests, paid preparation services or tutors, etc. But neither did I want to go in cold. Hopefully all students who take a fairly high-stakes exam like this do at least a little bit of preparing/familiarizing. Maybe that’s wishful thinking…
I’m absolutely sure I would have done worse on the math section if I had done no preparation at all.
The problem is that most actual Skills are not easily or accurately tested by machine scored tests, and it’s extremely expensive (and time consuming) to score more authentic assessments. Fortunately we already pay people to do this, they are called Teachers. Unfortunately all current school reforms seem to assume that Teachers are incapable of either doing that job, or of accurately reporting the results.
Bill, given the enormous grade inflation that has happened over the last couple of decades, I think it is pretty clear that schools and colleges *are* incapable of accurately reporting the results of their assessments. Whether this is the fault of the teachers or the administrators is unclear.
It’s rather amusing that you mention “Grade Inflation” when No Child Left Behind will be requiring 100% of students score “Proficient” on the standardized tests. This gets more entertaining when you consider that one of the “Identified Subgroups” is “Limited English Proficiency”, so yes, they expect 100% of students identified as “limited proficiency” to be proficient. Pardon me while I go bang my head into the wall….
Bigger question is how does one go about teaching these skills on ACT?
Scott, congratulations on taking the ACT before talking about it. Two points that you made are,in my opinion, most important. First, we need more higher level thinking problems on any test we give the kids. I discussed that in my latest post on my blog, where I also emphasized that this process must start in middle schools.
The second point – and perhaps the most important one – is the reactions you got from the “public”: What, me take an academic test again? To me, this points to a societal or cultural problem – we do not in this country hold academics in very high esteem. It is a deep structural problem and I am not sure how this will change.
I really enjoyed this, but I have one concern. I am a mathematician, so you can probably guess what I am going to take issue with: your sentence about not needing mathematics.
I will start by agreeing with you. Most people will not use cosines in their lives. However, I think that you are holding mathematics to a different standard than everything else. Most people will not read science charts in their daily lives, and (much to my chagrin) most people do not feel they need to know the difference between “its” and “it’s.”
With all due respect, I would caution a person in your position from suggesting that (at least portions of) one subject is not necessary. As I rant about in my blog (http://symmetricblog.wordpress.com/wp-admin/post.php?post=565&action=edit&message=6&postpost=v2), almost all of high school subjects are unnecessary. Of the ACT topics, I would argue that most people would only use “reading comprehension” and “writing” in their daily lives. Even grammar is unnecessary: my two-year old does not need to think about verbs and nouns when he says “I want that.” It is a triumph of evolution that language comes so easily, so most people will never think about it again.
In summary, I liked the post, I agree with you that few people will use trig after high school, but you seem to selectively choose “utility” as a reason why we should teach something.
Thanks again—I really am happy that you challenged them, took the ACT, and blogged about it.
I required my staff to take a practice portion of the ACT (not the entire test) in the fall of 2010. It was amazing to see the fear in people’s eyes and the test anxiety that returned for people, even though they knew that the test wasn’t going to be officially scored and nobody was going to look at their results. It was a great reminder for our staff what are kids go through. I thought it was an invaluable experience for my staff.
That’s a great idea, Vince. Even a few practice problems can remind folks of the need to be critical thinkers about what we’re asking our students to do…
I enjoyed reading this blog, Scott, and especially the discussion posts it has sparked in the comments section.
Especially interesting to me is Bret’s post. I would follow it up with another article asking, “When Will I Use Math?”: http://math-blog.com/2011/05/21/when-will-i-use-math/ The short article links to a great video about the ways in which we will use math in our lives and careers, and then links to their website WeUseMath.com.
So there’s a joke where a student is whining to his math prof, “Why do I need to learn this stuff? I’ll never use it!” The prof responds calmly, “You’re quite right, Joe, if you only aspire to flip burgers throughout your career.”
Many high school students are not inspired to aim higher, and they don’t realize math will get them further until they are already falling behind (and losing interest).
Even if you don’t use the math principles themselves, at least you can think more logically with mathematical practice.
I’m a 27 year old stay at home mom. Now that my kids are older, me and my husband have our little “nest” completed and own a home – I can go to college.
In 2006 when I first took the ACT,I received an 18. In fact,if you look at what the test taking “generation” (mine) VS today’s…the scores have slumped.
As I reluctantly cracked open my practice ACT book yesterday – realized why I failed so hard and why adults (my age) know nothing.
Growing up in a poor area, with dismal educational goals – I was taught “How to take a test”. I wasn’t taught how to actually DO math, or actual grammar rules. We were taught process of elimination, how to “cheat the test” and other silly buzz words found in the ACT practice book.
One of my practice ACT questions – “If 600 students attended a high school, and 54 were seniors, what percent are seniors?” Then given A, B, C, D, E. You know what? Knew the answer was B, because of “test taking strategies” I learned from the book, and my High School “education”. But I didn’t know HOW to solve the problem.
I cried. I cried because this is easy, because I felt dumb, and most of all…because no one taught me.
My husband (who is an engineer) knows my background, has seen my school, and has also survived not getting mugged while visiting my parents; kindly explained to me…I simply divide.
This is why so many people aren’t prepared for college, and this is why standardized testing is so dangerous. I’m just glad my husband is understanding, loving, and kind enough to solve the problem legitimately without teasing.
PS- The school system I attended is rated as “educationally unacceptable” by today’s standards.