Anatomy of a Quiz Question, Part 3: Why Did So Many Students Choose The Same Wrong Answer?

Need to catch up? Here are Part 1 and Part 2 of the Anatomy of a Quiz Question series.

As you may tell your students, understanding why you got a question wrong is critical. When we understand our mistakes, we know how to do better next time. For this reason, it’s often helpful for you as a teacher to look at the most commonly chosen wrong answer for a particular question, and ask yourself why students were fooled by this option. Here’s an example, from the article on the migrant crisis, titled “People run to Europe to escape war, but some are stuck at train station.”

A is the correct answer because it names the central problem of the article. The introduction explains that the Syrians are looking for “a safe place to live,” and the second and third sections focus on why the migrants are “having trouble” by giving details about the difficult and dangerous journey they face.

aaap3-1

So then why did 27% of students choose D? Choice D does partially address the main idea of the article, stating that Syria itself is “dangerous” and Syrians are leaving home. However, all three sections focus not on the danger in Syria, but on the danger Syrians face as they search for safer areas. Because D does not address this dangerous journey, it does not fully capture the main idea.

Here are some potential teaching points for students who chose D:

  • Read the entire article before choosing an answer.
  • Make sure to read all four answer options all the way through.
  • Remember that if only part of an answer seems correct, there is probably a better answer option.
  • Before answering quiz questions, read each section of the article and annotate the text with the main idea of each section.

Finally, here are three more articles you might assign students to help them develop their skills. The quizzes for these articles all include central idea questions, and each also relates to a topic covered in the migrants article above.

  1. Read about the experience of Chinese-American migrant railroad workers and compare the experience of these migrants to that of the Syrian refugees in the present day. (Here’s a Newsela Learning and Support lesson that incorporates this article.)
  2. Read about the sale of Syrian artifacts and gather background information on the civil war in Syria.
  3. Read about U.S. immigration policy and compare it to the policies of European countries regarding the Syrian refugees.

We hope our “Anatomy Of A Quiz Question” series was helpful to you and your students. Tweet us @newsela to let us know your takeaways or suggestions for future topics.

Anatomy of a Quiz Question, Part 2: Why Was That Question So Hard?

As we mentioned in Part 1 of our “Anatomy Of A Quiz Question” series, Newsela quiz questions are meant to be challenging enough to match the items your students may see on Common Core standardized tests. So what makes a question stand out as particularly tough, and how can you help your students perform well on even the trickiest items? Below is an article excerpt along with a question we’ve identified as especially challenging. This question is from an article on the migrant crisis, titled “People run to Europe to escape war, but some are stuck at train station.”

Countries Say They Need More Help

Europe cannot decide what to do. More than 332,000 migrants have entered Europe this year. Greece, Italy and Hungary are seeing the most people coming in. They have asked for more help. Germany has demanded that other countries take in more migrants. Germany expects to let 800,000 migrants stay this year. That is the most in Europe.

The people coming to Europe are desperate. They just want to be safe. They cross the sea in boats that are badly built. Others ride on the tops of trains. Many migrants have died. 

This week, 11 migrants drowned trying to cross the sea to Greece. Also, trains between London and Paris were stopped. Migrants were running on the tracks. They tried to climb on top of the trains.

Screen Shot 2015-10-22 at 12.19.01 PM

Only 37% of students answered this question correctly, compared to the average score of 50% for third grade anchor 2 questions. Those who chose A likely understood the main idea of the section, which is that European countries have asked for support with managing the flood of migrants. Choice A gives a detail about a country that has asked for this support, and explains the exact demand that the country has made.

aqqp2-3

This question was challenging because it required several different reading comprehension skills, including the following:

  • Attending to the question wording — it asked about a section, not the whole article.
  • Identifying the main idea of the section, potentially using the section title as a clue.
  • Separating key details from unimportant details.

If your students need support with main idea questions, check out some of the teacher-created lesson plans in Newsela’s Learning and Support section. This lesson teaches students to identify central idea using the 5Ws (who, what, when, where and why) using an article on a young lemonade entrepreneur. Another lesson assesses central idea and understanding of people and events through a text set on Malala Yousafzai.

How do you help students break down tricky questions, and what strategies do you use to teach main idea? Tweet us @newsela and stay tuned for Part 3 of our series.

Anatomy of a Quiz Question, Part 1: How Do Newsela Questions Compare to Standardized Test Items?

In this three-part series, we’ll break down Newsela quiz questions from several different angles. You’ll learn how Newsela items stack up against Common Core standardized test items. Next, we’ll explain what makes some quiz questions particularly challenging for students. Finally, we’ll shed light on why students get tricked by certain wrong answers. In each post, you’ll get classroom-ready tips on how to best use Newsela quizzes to continue to unlock the written word for your students.

PART 1 How Do Newsela Questions Compare To Standardized Test Items?

Recently, the New York Times invited readers to play the role of a third grader on exam day. The Times published a passage from this year’s third-grade reading standardized test, along with six Common-Core aligned questions that assessed students’ understanding of the short text. Then, readers could answer the questions for themselves and they could even see the percentage of other readers who answered correctly. Here at Newsela, we couldn’t resist the Times’ challenge. We took the quiz and compared our scores. We were inspired to dive deeper into the full set of this year’s test questions released by EngageNY and ask ourselves: How do our questions measure up?

As we looked through the released questions, we immediately realized that these items were tough. Really tough. Some questions had a pass rate as low as 29%, and the average pass rate for released grade five multiple choice questions was just 65%. When New York Times readers tried answering a third-grade main idea question, only 58% responded correctly. Based on these results, many students (and perhaps adults) are not yet prepared to succeed with these items — so as a teacher, you’re being asked to step it up in terms of rigor. That’s where Newsela can help. Check out how a Newsela grade five central idea question stacks up against the same type of question from EngageNY:

Newsela question:

aqq1

Student pass rate: 58%

EngageNY question:

aqq2

Student pass rate: 59%

Note that both questions ask students to identify key details from a short section of the text, requiring them to read closely and draw conclusions. The questions also had a near-identical pass rate, suggesting they are similar in difficulty level.

Across the board, our questions mirror the Common Core questions in standard alignment, in text dependency, in student pass rate, and in level of rigor. When we noticed this, we were thrilled. Why? Because our comparison confirmed that Newsela provides assessments that truly help assess student progress — not only toward success on exams, but toward the college and career readiness that the Common Core measures as well.

If your students take just two Newsela quiz weekly between now and springtime exam season, they’ll get valuable exposure to the types of items they’ll see on the big tests. They’ll go into the test confident in their abilities, and so will you. In fact, our data shows that completing two Newsela quizzes per week leads to measurable reading gains.

Do you use Newsela to prepare your students for Common Core standardized tests, or other Common Core aligned assessments? If so, we hope that this comparison gives you extra assurance that you’re doing right by your class. Tweet us @Newsela to let us know how you use Newsela to help your students succeed, and look out for parts 2 and 3 of our series coming up soon.

Students Rise to the Summer Reading Challenge

Newsela’s Summer Reading Challenge aims to help parents, teachers, and students combat summer slide — the phenomenon where, in the absence of homework and class time, kids lose gains made over the school year. Last year’s Challenge was a stride in the right direction: thousands of articles were read by students all around the country. But this year’s Summer Reading Challenge really blew us away.

More than a million articles were read by over 100,000 students in all 50 states and abroad. The cities with the most SRC participants were New York, Chicago, San Jose, Newark, and Indianapolis, respectively. New York had the most by far with more than 15,000 students participating.

Check out the following infographic to get an inside look at the 2015 Summer Reading Challenge. Be sure to scroll to the bottom to see which states and grades topped the SRC.

SRC-Infographic-v23

Congratulations to all our readers. We’re proud of you! 

Did your kids participate in the Summer Reading Challenge? Share your SRC story with us on Twitter @Newsela.

Summer Reading Challenge: Toughest Quiz Questions

Untitled design (10)

Last month, we launched our second annual Summer Reading Challenge for students. They’ve proven equal to the task. So far, more than 91,000 students have signed up and have read 1,127,900 articles, earning a total of 28,533 badges and 432,977,845 points.

We’ve posted previously on the most popular content on Newsela (including Who’s King in Newsela’s Animal Kingdom?) and engaging with many of these articles now has the added benefit of earning badges. If your readers love articles on animals, have them peruse our favorite dog stories to earn the Canine Chronicles badge. For inquiring young minds, take on the Mini-Mythbusters badge, where commonly accepted theories are put to the test. Budding moguls in your midst? Inspire them to action with the articles in the Junior Entrepreneurs badge.

We found that some badges were harder to earn than others. The Power of the Pen badge, which contains articles about writing with impact, contained 4 of the 5 toughest recent quiz questions – most of them from our article on teens using poetry as an outlet for creativity and problem-solving.

Of our toughest questions, 3 out of 5 of them were “BEST” questions, meaning they asked students to look for the best answer out of multiple plausible responses. We had previously learned through research that these questions asking students to not only find a possible answer, but the best of all possible answers, were difficult for students.

Screen Shot 2015-07-28 at 3.15.49 PM

Questions from fourth grade, one of our lower reading levels, had the highest incorrect response rate. More surprising was that most of our difficult questions were actually from Anchor Standard 1, which asks students to look at what the text says. Though many of our teachers tell us that they try to get students to focus on finding the main idea (Anchor Standard 2), this tells us that paying attention to small details and gaining a full comprehension of what happened in each story is something our readers should work on as well. Good news: we have over 675 Anchor Standard 1-aligned articles for teachers and students to choose from.

Have your students or children been participating in the Summer Reading Challenge? If so, how did you motivate them to keep reading? Let us know your thoughts @Newsela.

Articles in Power of the Pen:

https://newsela.com/articles/writing-therapy/id/10381/

https://newsela.com/articles/teen-poetry/id/4501/

https://newsela.com/articles/juvenilehall-writers/id/5430/

https://newsela.com/articles/malala-sanjose/id/11053/

The BEST Choice (Part 2): Are Questions With Negative Words Extra Tricky for Students?

Last week, we found that CCSS Anchor 2 (finding the main idea) questions were often a challenge for our readers. Often, these questions used qualifying words such as “except”, “most”, “best”, “does”, and “does not”. These questions might ask students to find the BEST answer for which sentence contains the main idea of the article when other answers contain important or interesting ideas.

Our next question was whether or not some of these words were more tricky than others for students. We decided to put the words into two categories: positive and negative. Questions that asked students to find the “best” answer or locate a sentence that “does” include a main idea from the story were considered positive. Questions that asked students things like “All of the following sentences include key ideas from the article EXCEPT” were considered negative.

We found that, overall, students performed better on the positively worded questions.

With that said, we wanted to see which qualifying words are the hardest and easiest for each grade. The table below provides a list of the toughest and easiest qualifiers for students, by grade. For example, if you teach eighth grade, your students will tend to find “DOES NOT” questions the most difficult, while performing relatively better on “BEST” questions.

 

So do your students do best on “BEST” questions? Most likely. Except for 11th graders, students perform best on “BEST” questions or questions with a positive qualifying word. So now the questions become:

  • What is the BEST technique to improve outcomes on negative qualifying questions?
  • What DOES NOT help students in answering questions with a qualifying word?

Again, we asked our Manager of Assessment what a good teaching strategy would be to help students struggling with the “EXCEPT” or “DOES NOT” questions. A lot of the time, she says, not taking the time to read questions carefully is the main culprit. As simple as it sounds, asking students to slow down and to read every word of the question can help them improve their scores.

Have you noticed any trends with qualifier questions in your classroom? Tweet us @Newsela and let us know what is working BEST and what is NOT working.

The BEST Choice (Part 1): How “Best Answer” and “Except” Questions Affect Student Scores

You’re probably familiar with the following scenario: students are taking a test and read the multiple-choice question “Which of the following sentences BEST illustrates the main idea in the story?” Answer A includes a sentence that relates to a key idea from the article. But then again, so does Answer C.

To assess students’ understanding of the main idea, many standardized tests — including PARCC and Smarter Balanced — ask students to provide the BEST answer. These questions typically have more than one answer that works, but one of the answers works better than the other. Quizzes on Newsela also feature similar questions, and we wondered whether the structure of the question could negatively affect a student’s performance. Specifically, we looked into CCSS R.I. Anchor 2 questions that included a qualifier word like BEST, EXCEPT, DOES NOT. Examples of these questions include the following:

  • Which detail DOES NOT support the main idea?
  • Which of the following is LEAST important to include in a summary of the article?
  • Which sentence is MOST important to support the main idea of the article?
  • Which of the following BEST describes two main ideas in the article?
  • All of the following are main ideas of the article EXCEPT:

These qualifying words can then be bucketed into two categories — positive and negative words, as described below:

Positive – BEST, MOST, MOST CLEARLY

Negative – DOES NOT, EXCEPT, LEAST, NOT

So how do students do on tests and quizzes when a question contains one of the above words? We decided to look at average quiz scores for questions with qualifying words. As many teachers might guess, we found that these questions tend to be more difficult for students. Overall students demonstrated proficiency* in only 20 percent of all questions with a qualifying word.

Student-Proficency-v1

We asked our Manager of Assessment what she would recommend for students who were “Not Proficient” or “Nearly Proficient”. She said one technique for dealing with Anchor 2 questions is to have students provide a short, 50 word summary of the article. They can even provide a summary that is under 160 characters (Tweet length) if Twitter is used in the classroom. Using their brief summary, they should be able to separate the main idea from the key ideas or interesting points made in the article.

Newsela also looked into which qualifying words (BEST, EXCEPT, DOES NOT, etc.) were especially confusing to students. We’ll let you know which words trip students up on our quiz questions next week.

*Proficient is defined as achieving an average of 70 percent or above, nearly proficient is defined as an average between 50 and 70 percent, and not proficient is defined as below 50 percent.

Follow Up: The 15-Second Rule of Test-Taking

A few weeks ago we learned that students tend to do better on quizzes when they read a little before looking at quiz questions. Unfortunately, we also found that this wasn’t what our readers were doing. According to our data, nearly two-thirds of our students have opened at least one quiz to look at questions within the first 15 seconds of reading. Naturally, teachers were curious about this and some even struck up conversations on Twitter.

The teachers wanted to know whether this was the case across all students or if reading ability affected when a student opened the quiz. To understand how various types of readers open the quiz panel, we used the Newsela percentile. The Newsela percentile provides a percentile to all students after they have submitted 5-8 quizzes. Using the same cut scores provided by SBACC, we then group students into four levels — Level 1 to Level 4 with 4 being our strongest readers.

We initially looked at all levels of students to see if a certain level of a student was more likely to open the quiz right away. We found that across all levels, there were similar numbers of students opening the quiz within the first 15 seconds.
15sec1

Still, this was specific to opening the quiz within 15 seconds. To answer whether reading level affected how long students spent on articles before opening quizzes, we needed to look beyond 15 seconds. We examined the average time students spent on an article before they opened quizzes to see whether that had more of an effect.

This time, the results were surprising. We noticed that strong readers in Level 4 spend nearly 40 more seconds on an article before opening up the quiz. In other words, the stronger readers wait to open the quiz until they’ve read a bit of the article or done some pre-reading exercises. The weaker readers were much more likely to open the quiz right away.
15sec2

Next, we looked beyond the groups of students to their scores. We reconfirmed that spending more time doing pre-reading exercises or reading parts of the article prior to looking at quiz questions correlated with better quiz scores. On their worst scoring quizzes (25% or less), students in Level 1 only spent 40 seconds on average looking at the article before opening the quiz. However, on their best scoring quizzes (70% or higher), students in Level 1 avoided opening the quiz for around 60 seconds.

15sec3

25% or below 26 to 69% Above 70%
Level 1 42.6 52.3 61.2
Level 2 50.6 66.3 73.3
Level 3 57.5 79.9 87.3
Level 4 51.2 100.4 91.9

Although it surprises some teachers, we found that for the most part, stronger readers wait longer before looking at quiz questions. This does not necessarily mean that students should read the entire article prior to looking at a single quiz question. However, we would recommend having students do some pre-reading exercises or read the first section of an article before they open the quiz.

Teachers – what do you make of these results? Tweet us @Newsela, we’d love to hear from you.

Back to newsela.com

Most Read on Newsela Last Week

Here are the top three most read articles on newsela.com from last week:

3. Higher minimum wage sparks hope in workers, fear in some businesses

Should a $15-an-hour minimum wage become federal policy? This article presents the viewpoints of activists, politicians, economists, and business owners.

minwage-effect-3d226681.jpg.885x497_q90_box-0,239,3000,1924_crop_detail

Ask your students to consider each side of the story with these discussion questions:

  • Whose point of view did you agree with most? Why?
  • Why do you think the article included the age, gender, and race/ethnicity breakdown of paid workers earning less than $15 an hour?

2. Supreme Court makes history with 5-4 gay marriage equality ruling

Last week, the Supreme Court handed down a historic decision to allow same-sex marriage across the United States. This article was the second most read on Newsela last week.

mctscotus-gaymarriage-420d7c0f.jpg.885x496_q90_box-576,584,4099,2561_crop_detail

Get the discussion going on this timely issue with the following questions:

  • What is meant by the phrase, “equal dignity in the eyes of the law”?
  • How does the author convey that this is an important milestone in U.S. history?

1. Hardcover yearbooks remain popular among today’s tech-savvy students

Teens may love documenting the weekend on Instagram or Snapchat, but when it comes to capturing the high school experience, hardcover yearbooks still have their place.

paper-yearbooks-a9cd40b2.jpg.885x497_q90_box-0,155,3000,1840_crop_detail

Here are a couple questions to get kids thinking critically about this article:

  • Which part of this article was most surprising to you?
  • Would you want a digital yearbook? Why or why not?

As always, we’d love to hear what you and your students had to say about our most read articles. Tweet us @Newsela and let us know which articles piqued the most interest in your classrooms.

Most Read on Newsela Last Week

Here are the top three most read articles on newsela.com from last week:

3. The art of the selfie is a big draw at many museums

Smile and say “selfieee!” There’s a cultural shift under way at many museums as they try to attract a social media-savvy generation.

museums-selfies-8fe54cd8

What did your students have to say about this article? Fuel the conversation with these questions:

  • Do you think these changes are a good idea? Who would be most excited about them, and who would be most opposed?
  • What did you find most surprising about this article?

2. Pop culture’s K-wave sparks rise in Korean language college courses

South Korean pop music, or K-wave, now enjoys a global audience. On many U.S. college campuses, that interest has led to record enrollment in Korean language classes.

korean-language-5bd75ee5

Here are a couple discussion questions you can ask your students:

  • Why did the author include detailed numbers on Korean language students, as well as the number of colleges that teach Korean?
  • What do you think the popularity of K-wave means for other genres of world music?

1. Are students struggling to read because they can’t see?

Our mission at Newsela is to unlock the written word for everyone. This article exploring the link between vision and literacy through new research is of particular interest to us – and apparently readers agreed.

vision-literacy-767a2794

Were the findings surprising to your students? Try asking them the following questions:

  • What are some challenges the researchers faced in conducting this study?
  • How could the results of the study aid students learning to read?

Did your class raise any other interesting points? We’d love to hear them. Tweet us at @Newsela with your insights.