Writing Good Test Questions

I have been thinking and writing a lot about assessment lately. My last post was basically a rant against unthinking assessment practices, and a challenge to all teachers (pointing the finger at myself here too) to be thoughtful in how they assess.

So how do we go about assessing thoughtfully?

In the curriculum and pedagogy courses I teach, we always spend some time talking about how to best assess students' learning. There are, of course, lots of ways to assess students, and the assessment vehicle you choose should match your goals of the assessment. Here are a few options we often discuss:

• Sometimes observation is all you need--especially while you are teaching. Reading students' body English and facial expressions, keeping aware of the the kinds of questions they are asking, and noting the kind of responses they are giving in response to your questions are all good ways to assess students' thinking while teaching is ongoing, and gives you the opportunity to change course if needed.
• Projects and performances are often valuable ways of having students apply their learning and demonstrate their proficiency at specific tasks or skills.
• Conferences, interviews, and small group meetings can allow the teacher the chance to talk with students in an individual or comfortable group setting. This takes some planning and management; what is the rest of the class doing while you meet with the individual or small group? But I've found that hearing students explain their understanding firsthand is often one of the best ways to know what they know!
• And, of course, tests and quizzes are still a key part of teachers' assessment strategies. In our current high-stakes, high-accountability school culture, outside testing pressures are often pretty significant. Some teachers argue that students need to take tests and quizzes just to practice, so they will feel comfortable and prepared for the high-stakes, state-mandated testing.

Image by COCOEN Daily Photos CC BY-NC-SA 2.0

I always want to remind my students that tests and quizzes aren't the only way to assess, so I try to assign a variety of different kinds of assignments and projects to them in addition to tests and quizzes.

It's important to remember that all assessment vehicles have pros and cons. The benefit of tests and quizzes is that they are relatively straightforward to assign, and relatively straightforward to mark. (Essay questions may or may not be so easy, either for the students or the teachers.) The biggest downside for tests and quizzes is that they often do not assess students' higher order thinking abilities. So it's important to think through what you are hoping to assess with any given assessment vehicle. Are you just assessing students' ability to remember and understand basic facts? A multiple choice test is probably a fine choice. Are you assessing their ability to deconstruct an argument and evaluate the evidence provided? A multiple choice test is probably a poor way to do this.

It's not that objective questions (multiple choice, true-or-false?, matching, etc.) are bad questions. They can be a valid assessment choice, depending on the intent of the assessment vehicle. But knowing that I've written some poor tests over the years (e.g., assessing what is easiest to measure, rather than what I most highly value), I see that teachers may sometimes need guidance in how to construct questions. 

In this vein, I offer the pre-service teachers I serve some advice for writing test questions. This is mostly based on my own experiences, and hopefully this helps to prevent them making some of the same mistakes I have made. (My sincere apologies to the kids who had me for their teacher, especially those first 4 or 5 years. I got better at this as time went on...) As I've learned more about educational psychology over the years this has also informed the way I think about writing test questions.

So, without further ado...

Mulder’s Guide to Writing Test Questions

How I write multiple-choice questions:

• I generally try to make the questions as straightforward as possible!  (Cynically, I sometimes refer to these questions as multiple-guess questions.)
• I try to have the same number of choices for each question.  (This doesn’t always work.)
• I try to make the stem of the question longer than the choices. (This doesn’t always work.)
• I try to phrase the choices in similar format.  
• I try to place the choices in some logical order (usually alphabetical, if that is reasonable.)  I do this to keep myself from falling into patterns.
• I generally try to avoid “all of the above” or “none of the above”.  (ESPECIALLY for younger students—these sort of questions are beyond their cognitive developmental level!)
• I try to avoid “trick” questions in which two of the choices are equally valid.
• I generally try to have students apply critical thinking to at least 1/3 of the questions.  This is not easy!  Writing “good” multiple choice questions requires getting students to think beyond simple regurgitation of knowledge.  (When I write critical thinking questions, I usually try to find ways to make connections between two separate concepts I want students to understand.)
• I usually have students write their answer on a blank rather than circle their choice.  This can lead to problems with the “a” looking like a “d”, etc., but it is quicker to score at a glance if there are a lot of questions to be marked.

How I write true-or-false questions:

• I generally try to make the questions as straightforward as possible! 
• I try to avoid using “not” in questions.  (Instead of, “If I go to the moon my mass will not change.” try, “If I go to the moon, my mass with remain the same.”)
• I write the letters “T” and “F” on the paper and have students circle their choice.  (T’s and F’s can look a lot alike.)
• I try to have about an equal number of “true” and “false” statements, but not always.  (Though I have been known to make them ALL true and watch them scratch their heads. “They can’t ALL be true, can they?”  What a horrible person I am...  J)
• As a twist, I've occasionally done only "false" statements. I turn these into a constructed response question: students articulate what is false about the statement. (I would not do this with younger students, but I did use this technique with middle schoolers to some success.)

How I write fill-in-the-blank questions:

• I usually DON’T use fill-in-the-blank questions.  (Mostly because I don’t like them!  I would usually make them multiple-choice questions instead.)
• When I DO use f-i-t-b questions on a test, I try to keep it to ONLY the MOST important vocabulary from the unit, and I inform students ahead of time!
• When I do use f-i-t-b questions, I do usually give a “word bank" listing the word choices.  (Especially with younger students, or if there are a LOT of words.) Without a word bank, I think you have to be much more open to whatever words students choose to fill in, and so long as the word they use adequately completes the sentence, they deserve credit for their response.

How I write matching questions:

• Note:  Basically, matching questions are multiple-choice questions with LOTS of possible choices.  This makes them useful for vocabulary lists in particular.
• I generally try to limit the number of items to match to 10 or less.  (More than that takes a LONG time for students to think through the list.)  If you have more than 10 items, break the list into two sections of questions.
• I try to use definitions similar to but not exactly like the definitions we used in class.  (I do this to push my junior high students’ thinking beyond simple memorization; I probably would not do this with younger students.)
• I have students match the words from the list to the definitions/descriptions rather than match the definitions/descriptions to the words.  (Like making the stem of a multiple-choice question longer and the choices shorter.)  This is a subtle difference, but many students have an easier time answering questions this way!

How I write constructed-response (short-answer or essay) questions:

• Constructed-response questions give students a prompt, and then demand a written answer. This makes them a better choice than objective questions (like the ones described above) for getting at students' higher-order thinking skills--applying, analyzing, evaluating, and creating. 
• I like to give students choices in the questions they answer when possible.  Often times I will write several questions that get at similar/related topics and give students the option of which question(s) they would like to answer.
• For larger “essay” questions, I sometimes break the question into parts and ask the parts separately.  (This can make it easier to evaluate too, especially if they get part of it but not all of it.)
• I personally like constructed-response questions the best in tests; I think they give the students the most opportunity to explain what they are thinking than other types of test questions.  This is a trade-off however:  more objective (multiple choice, T or F, matching) tests usually take longer to write, but are faster to mark, while more subjective (short-answer) tests are often quicker to write but require more time to mark accurately.