Multiple Choice Questions
Multiple choice questions are often called fixed choice, selected response or multiple choice items because they are not always questions, and they require students to select from among various options that are presented to them. The options are fixed.
These items remain important because they can be scored rapidly, providing quick feedback to students. Also, they are efficient when assessing large numbers of students over broad content.
One drawback is that constructing multiple choice items well requires plenty of time for writing, review, and revision. A time-saving tip is to write a few items each day while preparing for class or after class, so that the material is fresh in your mind. The items will then most likely reflect what you emphasized in class, which is fairer for the students. If you construct the items so that they can be easily shuffled, like on index cards or software with easy cut and paste, you can simply shuffle items around to build quizzes and tests later.
An important consideration in constructing multiple choice items is to make them measure learning rather than test-taking skills of “test wise” students. The suggestions here are designed to help you with this, but first some vocabulary needs to be introduced.
The following vocabulary will be used in the rest of this discussion.
The prompt or first part is the stem. For example:
Frequent use of sprays, oils, and antiseptics in the nose during a bad cold may result in:
Students select from among the options, which include the correct response and the incorrect responses or distractors. In our example, a is the correct response and b,c, and d are the distractors.
A. the spreading of the infection to the sinuses.
B. damage to the olfactory nerve.
C. destruction of white blood cells.
D. congestion of the mucous membrane in the nose
- High diagnostic power if distractors are constructed to address common mistakes or misconceptions.
Student responses can be scored objectively and analyzed statistically for impartial, reliable and valid diagnostic information about student learning. Reliability refers to the consistency with which a learning outcome is measured. This concept is mostly applied to sets of items or entire tests when considering multiple choice items. Validity is the degree to which an item effectively accomplishes the task for which it was designed. A test cannot have validity unless each of its items is valid. The discussion here only addresses validity of single multiple choice items. Regarding a newly constructed item, answers to the following questions should be yes for the item to be considered valid:
- Is it reasonable to expect that the students have prior knowledge needed to respond to the item?
- Does the item address an important concept that students should have learned from instruction?
- Is the item stated clearly, using the appropriate vocabulary for the students?
Is the level of thinking addressed by the item appropriate for the
- content addressed?
- level of the student on the novice to expert continuum?
- learning objectives of the course?
- Difficult and time-consuming to write good items that address thinking skills above the factual level
- The items are difficult to phrase so that all students interpret them in the same way
- When students study for multiple choice tests, they focus on recognition, not recall. Recent learning theories indicate that students need to process information to really learn it, so time spent studying for recognition is not as effective as time spent working with information.
- By guessing, students who don’t know the answer have a 25% chance of correctly selecting the correct response in a multiple choice item with 4 options. This decreases to 20% for items with 5 options, etc.
WRITING EFFECTIVE MULTIPLE CHOICE ITEMS
The following tips can help you create multiple choice items to most effectively measure student learning.
- Write the stem first, then the correct answer, then the distractors to match the correct answer in terms of length, complexity, phrasing, and style
- Base each item on a learning outcome for the course
- Ask a peer to review items if possible
- Allow time for editing and revising
- Minimize the amount of reading required for each item
- Be sensitive to cultural and gender issues
- Keep vocabulary consistent with student level of understanding
- Avoid convoluted stems and options
- Avoid language in the options and stems that clues the correct answer
Writing effective multiple choice item stems:
- Format stems as clearly, concisely phrased questions, problems, or tasks if possible
- If phrasing the stem as a question requires extra words, make the stem into an incomplete statement
- Include most information in the stem so that the options can be short
- When making the stem an incomplete statement, make sure the options follow the stem in a grammatically correct manner
- Avoid using negatives in stems when possible
Writing effective multiple choice item options:
- Make sure there is only one best or correct answer
- Keep options parallel in format (if all options cannot be constructed in a parallel way, make 2 options parallel to each other and the rest of the options parallel to each other…the key is to construct options that do not stand apart from each other purely because of style)
- Make options mutually exclusive (ex. Avoid 1-4, 2-5, 3-6 etc. as options because they overlap)
- Make options of similar length and make sure the longest answer is only correct some of the time
- Avoid “all of the above” or “none of the above”
- Avoid repeating the same words in all of the options by moving the words to the stem
- Arrange options in logical order if possible
- Avoid using specific language like “all,” “never,” or “always”
- Keep options plausible for students who do not know the correct option
- Options selected by very few students should be altered if the item is reused
Sample items that exemplify the information above:
Which of the following philosophical schools was most identified with the Greek Philosopher Aristotle?
D. Empiricism (correct)
A blood pressure reading is 120/82. What information does this reading provide?
A. Force needed to open a heart valve
B. Force of atrial contractions
C. Pressure of the blood against artery walls (correct)
D. Pressure of the blood as it flows in veins
Multiple Choice Items Addressing Complex Thinking Skills
To write, multiple choice items addressing complex thinking skills are more difficult than those intended to survey factual knowledge. However, the effort can be highly rewarded because of the valuable information they can yield quickly about your student’s competencies, especially when distractors are skillfully designed to target key weaknesses in novice thinking.
Stems of these items often present a problem, scenario, or reading upon which students reflect to answer the question. Ideas for writing these items include:
- present a problem that requires application of course principles,
- require analysis of a problem,
- require evaluation of alternatives
- design problems that students to combine several concepts or ideas
- designing options that require a high level of discrimination can also contribute to multiple choice items that test higher-order thinking
The following multiple choice questions are examples with formats that you can adapt to fit most disciplines.
EXAMPLE 1: This item addresses higher level thinking skills because students must evaluate multiple pieces of evidence, then apply that evidence to solve a problem. Evidence is presented, then the student must select the best action to take with the evidence.
Tim’s second grade teacher is concerned because of the following observations about Tim’s behavior in class:
- Withdraws from peers on the playground and during groupwork
- Often confuses syllables in words (ex: says mazagine instead of magazine)
- Often confuses b and d, p and q, etc. when writing or recognizing letters
- Loses his place when reading
The teacher has arranged a meeting with Tim’s mother to discuss these concerns. Which of the following statements is best for the teacher to say to Tim’s mother?
A. Tim needs extra practice reading and writing problematic letters and words at home at least 30 minutes per day.
B. Please discuss the importance of schoolwork to Tim so that he will increase his efforts in classwork.
C. These are possible symptoms of dyslexia so I would like to refer him to a specialist for diagnosis.
D. Please adjust Tim’s diet because he is most likely showing symptoms of ADHD due to food allergies.
Explanation: C is the best answer because the behaviors could be symptoms of dyslexia. The other options are plausible, but not the best. Tim is already showing signs of social anxiety, so extra practice, or increasing effort could worsen the anxiety. The collection of “symptoms” does not indicate ADHD as much as a learning disability.
EXAMPLE 2: This item was released from an AP biology test and was written to address an understanding of a hypothesis about the natural origin of life on Earth with supporting scientific evidence as well as evaluate scientific questions and hypotheses.
By discharging electric sparks into a laboratory chamber atmosphere that consisted of water vapor, hydrogen gas, methane, and ammonia, Stanley Miller obtained data that showed that a number of organic molecules, including many amino acids, could be synthesized. Miller was attempting to model early Earth conditions as understood in the 1950s. The results of Miller’s experiments best support which of the following hypotheses?
A. The molecules essential to life today did not exist at the time Earth was first formed.
B. The molecules essential to life today could not have been carried to the primordial Earth by a comet or meteorite.
C. The molecules essential to life today could have formed under early Earth conditions.
D. The molecules essential to life today were initially self-replicating proteins that were synthesized approximately four billion years ago.
Explanation: Notice that C is the correct options because the other options require the students to infer information that has not been provided.
Source: AP Central
EXAMPLE 3: This item was released from an AP history test and was written to address an understanding of the development and interaction of cultures during state-building, expansion, and conflict. It also addresses the skills of argumentation, contextualization, interpretation, and synthesis. It requires students to compare two passages, one from a modern historian and one from a person living during the historical time frame.
Whenever I visited Jerusalem, I always entered the al-Aqsa Mosque, beside which stood a small mosque which the Franks had converted into a church.... [T]he Templars, ... who were my friends, would evacuate the little adjoining mosque so that I could pray in it.
Usamah ibn Munqidh, Muslim historian, Jerusalem, circa 1138
The Crusader states were able to cling to survival only through frequent delivery of supplies and manpower from Europe.
[They] were defended primarily by three semi-monastic military orders: the Templars, the Hospitallers, and the Teutonic Knights. Combining monasticism and militarism, these orders served to protect pilgrims and to wage perpetual war against the Muslims.
Palmira Brummett, world historian, 2007
The second passage does not support the first passage because the second passage
A. shows that an influx of manpower from Europe was not critical for the survival of the Crusader states
B. shows that Muslims vastly outnumbered Europeans in the Crusader states
C. minimizes the importance of Hospitallers and Teutonic Knights in the administration of the Crusader states
D. presents an incident in which a military order supported a Muslim traveler
Explanation: D is the best response.
Source: AP Central
Haldyna, T. M. and Rodriguez, M.C. (2013). Developing and Validating Test Items. Routledge, New York. ISBN-10: 0415876052.
Haldyna, T. M. (1996). Writing Test Items to Evaluate Higher Order Thinking. Routledge, New York. ISBN-13: 978-0205178759