Jay McTighe: Beware of the Test Prep Trap

Image uploaded from iOS

In this era of accountability, educators throughout the nation are under pressure. Administrators are held accountable for student achievement in their schools as gauged by standardized tests. Increasingly, teachers’ evaluations include a percentage based on the results of test scores (at least in the tested grades and subjects). In some states, a school can be “reconstituted” if standardized assessment results do not improve over time. And in many communities, the test scores for a district or a school affect real estate values within their boundaries. Not surprisingly, these factors lead teachers and administrators to pay close attention to the results of external tests and strive to improve them. One consequence of this high-stakes accountability system is the increased use of “test prep” in the classroom; i.e., where teachers spend time focusing primarily on the tested content while giving students lots of practice with the test format (primarily multiple choice).

While mindful of the pressures associated with the high-stakes of accountability testing that lead to test preparation actions, excessive test prep can narrow the curriculum, undermine meaningful learning, and negatively affect student interest and motivation. At best, test prep can yield modest, short-term gains in test scores, especially if students are unfamiliar with standardized test formats and protocols.  However, I contend that the practice itself, while well intentioned, is grounded in misconceptions that may, ultimately, undermine the learning that students need to perform well on standardized tests. Let’s explore these points further.

What is Test Prep?

The practice of test prep in the U.S. has several distinguishing characteristics. Students typically engage in exercises and worksheets that mimic the format of standardized tests. Since accountability tests are generally constructed around sets of selected-response items, test prep involves lots of practice on decontextualized, multiple-choice questions. Sometimes, test prep includes timed, on-demand, assessments to simulate test-day conditions. In states that employ computer-based testing, students are often given opportunities to practice using a laptop or tablet device. Many schools and districts have institutionalized test prep by mandating the use of interim or benchmark assessments modeled after their state tests. Not surprisingly, we have witnessed the growth of an entire cottage industry of commercial “test prep” materials to address this perceived need.

A second characteristic of test prep relates to the content that is practiced. Typically, teachers are exhorted to focus only on tested knowledge and skills. The logic is understandable; i.e., since we are being held accountable for student achievement on standards A, B, and C, then we don’t have to worry about standards X, Y, and Z since they are not tested. Yet this logic has problematic consequences for learning. For example, in English/Language Arts, most standardized tests do not assess Listening, Speaking, or extended writing even though they are listed in all E/LA standards. Accordingly, teachers rarely spend any “test prep” time on listening and speaking skills or essay writing, even though these skills are fundamental to literacy development.

A third aspect of test prep involves the explicit teaching of test taking strategies and common “trigger” words used in test prompts. Examples of trigger words include compare, critical, distinguish, differentiate, key, major, significant, solve. Here are common test-taking strategies that are taught:

  • Read the question completely before you look at the answer options.
  • Read all the choices before choosing your answer.
  • Cross out any choices that you know are wrong.
  • Answer the questions that you know first; then tackle the remaining questions.
  • Usually the correct answer is the choice with the most information.
  • Always take an educated guess if you do not know the answer.
  • In a question with an “All of the above” choice, if you see that at least two correct statements, then “All of the above” is probably the answer.
  • Be aware of the time and pace yourself. Don’t spend too much time on any one item.
  • If time is about to expire, just select an answer for as many of the items as possible, even if you haven’t had time to read the questions.

It is not uncommon to see such trigger words and test-taking tips posted on classroom walls prior to test days as reminders to students.

Of course, it makes sense to familiarize students with standardized test formats since it is a genre that they will see throughout their school lives. Similarly, there is nothing wrong with imparting test taking strategies in advance of a high stakes assessment. However, such actions can be accomplished quickly and should not divert valuable instructional time from more substantive learning.

Test Prep Practice is Rooted in Misconceptions

The logic of test prep is plausible and rooted in experience from other domains. For example, if you want to improve your performance in dribbling a basketball or piano playing, then you must practice those activities. Shouldn’t the same apply to test taking?  Perhaps, but I contend that excessive test prep practices reflect two fundamental misconceptions that deserve to be critically examined:

Misconception #1 – The best (and only) way to improve test scores is to practice the test.

While this statement seems to makes sense on the surface, Wiggins and McTighe (2001) offer the following analogy to expose this misconception:

“To begin to uncover the flaw in this reasoning, consider an analogy. Once per year, we go to the doctor for a physical exam. No one particularly relishes the thought of such an exam, but we go with the understanding that it is in our long-term interest to get an objective (yet superficial) measure of our health. The doctor performs a few tests in a short span of time (e.g. blood pressure, pulse, temperature, blood work for cholesterol, etc.). The ‘physical’ is a small sample of tests, yielding a few useful indicators of one’s health status. Its validity and value stem from the fact that the results suggest our state of health, not because the physical defines healthfulness.

Now suppose we are terribly concerned about the final numbers (weight, blood pressure, cholesterol, etc.) and that these ‘scores’ ultimately link to our personal health insurance costs. What we might do, in our panicky state prior to each annual physical, would be to ‘practice’ for the test – focus all our energy on doing well on the physical exam (as opposed to what its indicators suggest). If our doctor knew of our actions, her response would surely be: ‘Whoa! You’re confused: you have mixed up causality and correlation here. The best way to prepare for your physical exam is to live a healthful life on a regular basis –exercising, watching weight, lowering intake of fats, eating more fiber, getting sufficient sleep, avoiding tobacco, etc.’

It would be thought silly to practice the physical exam as a way to improve one’s health. But this confusion is precisely what we see in schools all over North America. Local educators, fearful of results, focus on the indicators, not their causes. The format of the test misleads us, in other words.”

Misconception #2 – Standardized test items involve primarily recall and recognition, and thus drill and practice will be the most effective method to prepare students for them.

Given the predominant use of the multiple-choice format, there may be the assumption that these items primarily test factual knowledge, basic skills and “low-level” thinking. After all, there is a “right” answer, and all a student has to do is select that answer from a set of given alternatives. What follows from this assumption is the belief that covering lots of factual information and drilling and practicing multiple-choice items will provide adequate preparation for the accountability tests. Moreover, one might conclude that there is no need for, nor benefit from, in-depth learning involving extended thinking or the use of more authentic assessments.

Grant Wiggins (2013) points out the flaw in this reasoning: “Even though the test format requires a selected response, it does not mean that the tested knowledge is necessarily simple. The [format] deceives you into thinking that since you are mimicking the format of the test, you are therefore mimicking the rigor of the test. But data show the opposite conclusively: local tests are often less rigorous than state and national tests even when they mimic the format.”

Item analyses of consortium and state test results validate Grant’s point. Indeed, the pattern is remarkably consistent – the most widely missed items on standardized tests are not those assessing simple recall of factual knowledge or basic skills as referenced at Level 1 on the Depth of Knowledge scale (Webb, 2006). Instead, they require inference and interpretation in reading, analysis and reasoning in mathematics and science. While knowledge and skills are needed, the more difficult items demand “higher-order” thinking and involve transfer at Depth of Knowledge (DOK) Levels 2 and 3. Such items often include distractors that present typical misconceptions, common errors, and flawed reasoning that will trip up test takers who only have learned by rote. Accordingly, low-level, drill and practice is not the optimal instructional method for improving test scores.

Too often, the information revealed by test prep exercises identifies whether students have chosen the “correct” answer rather than helping teachers determine if they have a conceptual understanding of the underlying concepts and skills and can apply (transfer) those.

Casualties of Test Prep                                                                                

There are opportunity costs to consider when precious classroom time and energy are devoted to test prep, and excessive test prep can have significant negative consequences. When classroom instruction and assessment fixates excessively on the multiple-choice format, meaningful learning is sacrificed and students are likely to become bored and disengaged by repeated drills on decontextualized items that lack relevance. Judy Willis, MD, a board-certified neurologist who left her medical practice to become a teacher, has written extensively on the brain and learning. She addresses the negative consequences of test prep in a recent article (Willis, 2012):

Boredom, frustration, negativity, apathy, self-doubt, and the behavioral manifestations of these brain stressors have increased in the past decade. As facts increase, as over-packed curriculum expands, and as demands for rote memorization for high-stakes testing intensify, the brains of our students have reacted to the increased stress. Stress, including that provoked by sustained or frequent boredom or frustration, detours brain processing away from the higher, rational, prefrontal cortex. In the stress state, the lower, reactive brain is in control. Retrievable memory is not formed, and behavioral responses are limited to involuntary fight/flight/freeze-seen in the classroom as acting out, zoning out, or dropping out.

In short, it doesn’t matter how many practice tests we give; if the learners are not engaged or fail to see the purpose, their learning will not be optimized and performance on high-stakes tests will not be bolstered.

Don’t take my word; ask yourself:  Teachers – To what extent are your students motivated and genuinely engaged by test prep?  Administrators – Do your best teachers claim that test prep is their favorite or most effective teaching practice?  Parents – Do your children rave about the joys of test prep at the dinner table?

The pressures to improve accountability test scores can result in a narrowing of the curriculum. It is often the case that the tested subjects receive greater attention compared to those not tested. Indeed, we have witnessed schools and districts that have doubled up on reading and mathematics instructional time while reducing or eliminating the arts and/or health and physical education. Sadly, for many students, these are the most engaging subjects in their school day.

The use of precious classroom time for test prep can distort students’ perception of the nature of schooling. They could easily conclude that a primary mission of schools is to improve test taking savvy and raise test scores rather than to strive for meaningful learning. Moreover, a focus on multiple-choice teaching and testing can convey the fallacious idea that navigating school and life is simply a matter of choosing the “correct” answer from 4 or 5 alternatives!

Ironically, the widespread use of test preparation practices based on narrow, inauthentic assessments can unwittingly undermine the very “college and career” readiness competencies identified in national and state standards and for the development of 21st century skills. Many educators and policy makers worry that important educational goals (e.g., discussion and debate, extended writing for real audiences, research, teamwork, creative problem solving, expression in the arts, or substantive research and experimental inquiry) that are not easily and cheaply tested are likely to “fall through the cracks.” To be blunt, students will not be equipped to handle the sophisticated work expected in colleges and much of the workforce if teachers simply march through a superficial “coverage” of discrete knowledge and skills in grade-level standards and assess learning primarily through multiple-choice tests of de-contextualized items.

So, What Should be Done?                                                                          

It would be naïve, indeed irresponsible, to dismiss the reality of high-stakes accountability tests by imploring educators to ignore them or suggesting that if teachers simply “teach well and love the children” the test scores will take care of themselves. As noted, it is prudent to introduce students to the test format. However, beware of mistaking the measures for the goals. Excessive “multiple-choice” teaching and practice testing are not the best long-term strategies for developing a well-rounded, educated person or realizing significant improvements in scores on annual accountability tests.

I contend that the best way to raise test scores over the long haul is to: 1) teach the key concepts and processes contained in standards (the content that is purportedly tested) in rich and engaging ways for deep learning; 2) collect evidence of student understanding of that content via more authentic local assessments; and 3) regularly review student work on authentic tasks in Professional Learning Communities (McTighe, 2008).

To summarize, I offer the following set of Do’s and Don’ts as more effective alternatives to test prep.

DO DON’T
• Teach to the standards that are being tested. • Ignore those elements of the standards that are not assessed (e.g., listening, speaking, research, extended writing, genuine problem solving).
• Give students opportunities to become familiar with the test formats (selected- and brief-constructed response; timed writing). • Use the standardized test formats exclusively. Students need to experience a variety of assessment types, including performance tasks, extended writing, open-ended problem solving, and discussion/debate.
• Engage students in deep and meaningful learning by using engaging instructional strategies, primary sources and authentic tasks. • Engage in excessive “test prep” by only practicing de-contextualized items that mimic the test format.
• Teach for understanding and transfer by engaging students in “higher order” thinking. • Dwell on drill and practice (rote learning) focused on factual recall.
• Regularly use formative assessments to give students specific feedback on the important performances called for by the standards. • Use assessments solely for the purpose of giving grades. (Grades are not feedback, and are unlikely to improve performance.)
• Regularly review student work on authentic tasks in Professional Learning Communities and plan instructional and curricular improvements based on more genuine and informative performance data. • Rely on a once-a-year test score reports as the primary metric to determine how well students are learning or what improvements are needed.


Jay McTighe is an accomplished author, having co-authored 14 books, including the award-winning and best-selling
Understanding by Design series with Grant Wiggins. His books have been translated into fourteen languages. Jay has also written more than 35 articles and book chapters, and been published in leading journals, including Educational Leadership (ASCD) and Education Week. Jay has an extensive background in professional development and is a regular speaker at national, state, and district conferences and workshops. He has made presentations in 47 states within the United States, in 7 Canadian provinces, and internationally in 36 countries on six continents. 

References

McTighe, Jay.  Measuring what matters: Part 1 – The case for an assessment overhaul. In What’s Working in Schools Newsletter, December 2010. Bloomington, IN: The Hope Foundation.

McTighe, Jay.  (2013). Core Learning: Assessing What Matters Most. Midvale, UT: School Improvement Network

McTighe, J., Seif, E., and Wiggins, G.  You can teach for meaning. In Educational Leadership. Alexandria, VA: Association for Supervision and Curriculum Development. Volume 62, Number 1, September, 2004.

McTighe, Jay. Making the most of professional learning communities. Oxford, OH: National Staff Development Council. Volume 3, Number 8, May 2008.

Webb, N. L. (2006). Identifying content for assessing student achievement. Chapter 8. In S. M. Downing & T. M. Haladyna (Eds.), Handbook of test development. Mahwah, NJ: Lawrence Erlbaum Associates, 2006.

Wiggins, Grant. Radical test prep. In Granted, and… blogpost. Authentic Education, April 2013.

See: https://grantwiggins.wordpress.com/2013/04/02/radical-test-prep/

Wiggins, G. and McTighe, J. Yes, but… Misconceptions about standards-based reforms. Unpublished article. October 2001.

Willis, Judy.  Bad for the brain: Goodbye to unsustainable education models. In  Edutopia. San Rafael, CA: The George Lucas Educational Foundation. June 2012.

See: http://www.edutopia.org/blog/bad-for-brain-unsustainable-models-judy-willis-md

Willis, J., MD. Success on standardized tests without sacrificing authentic learning. In STEM Magazine, January-February 2013, 10-15.

Reach Higher With First Lady Michelle Obama

First Lady Michelle Obama poses with members of her staff at the White House on College Signing Day, 2015. (Official White House Photo by Pete Souza)
First Lady Michelle Obama poses with members of her staff at the White House on College Signing Day, 2015.
(Official White House Photo by Pete Souza)

Guest post from Eric Waldo, executive director of first lady Michelle Obama’s Reach Higher Initiative

Education after high school is a critical part of a successful future and the most valuable asset a person can have. It helps people understand the world around them and gives them the tools they need to overcome life’s challenges. In addition, according to Georgetown’s Center on Education and Workforce, college graduates are set to make 84 percent more over their lifetimes than high school graduates.

That is why the first lady launched the Reach Higher initiative to help achieve the president’s “North Star” goal: by 2020, America will once again have the highest proportion of college graduates in the world. Reach Higher is focused on providing encouragement and activating educators and other adults to help students see college as an option.

Reach Higher is excited to partner with Newsela to help inspire students to take charge of their future through education. With Newsela, students can get ready for college-level reading while reading real-life stories about how high education has transformed people’s lives. The stories run the gamut from unique scholarships to special challenges facing students applying to college, all at a reading level that is just right for them.

Newsela and Reach Higher have put together these resources to help you bring this initiative into your school:

  1. Reach Higher Text Set, created with Michelle Obama.
  2. An inspiring op-ed from the first lady, My college story can be yours.
  3. Classroom resources including paired Text Sets, an op-ed writing assignment and more.

For senior educators, we hope you join us in launching a #ReachHigher College Signing Day in your school on April 26th. These events offer a unique opportunity to celebrate graduating seniors, showcase our student leaders, and inspire more young people to take the first steps on their journey to higher education. Download the College Signing Day Kit.

We can only begin to imagine what America’s students can achieve – especially given the gift of a college education. Encourage them to #ReachHigher.