Monday, September 25, 2017

Learning to Teach Again: Writing Tests

It's time for our first test of the semester in World Regional Geography. I've spent quite a bit of time over the past few days working on writing it. This perhaps something the non-teachers out there don't realize: writing a (good) test is actually a lot of work!

Yes, I know there are lots of pre-fab tests that come with curriculum materials. In my experience, these vary in quality quite a lot. Some of them are pretty good. Some of them are pretty awful. Most are somewhere in between, perhaps with some great questions and some...less great questions.

I generally prefer to write my own tests though, and in this course in particular--while I do have a great text that we're using--I don't have a teacher's manual the way I did when I taught in K-12. And, honestly, I really prefer to write my own test questions anyway.

I've shared my strategies for writing test questions before on this blog, and I'm putting them into practice as I've been working on this test. It's been a timely reminder for me about the challenges of writing good questions: questions that get at what I most highly value, and not just what is easy to measure.

Image by Alberto G. [CC BY 2.0]
(Funny, because I never use Scantron sheets for tests I write...)

Here's the thing: I really want to know what my students know, and understand what they understand. It's easier to write questions that just demand basic factual recall, such as, "Which of these is the correct definition for 'military dictatorship?'" It's more difficult to write test items that gets at deeper understanding. Here's one I'm trying out: "Where in the world are you most likely to find military dictatorships? Based on demographic, social, and economic factors, why are military dictatorships more likely there?"

Of course, students would probably prefer to just answer the former type of question, because they are less demanding for them. And, honestly, as a teacher, it's easier to grade tests made up of the former type of question, because they are more objective and straightforward. For the latter question, is it equally correct to answer "Africa," or "Southeast Asia," or "in developing nations?" And what if they talk about demographic factors, but not social and economic factors in their response?

What I'm getting at here is my belief that tests--and all forms of assessment, really--should be opportunities for students to demonstrate what they know, understand, and are able to do. Unfortunately, I know I've written tests in the past that have more like a game of "gotcha!" than a real opportunity to demonstrate learning.

If I'm serious about students' learning, I believe I need to also be serious about the kind of tests I write. I need to be mindful of my own biases in test-writing, balancing what is easy and what I value. I need to ensure that the content included on the test matches what I've identified as the goals of the unit. I need to be careful that the test I write is responsive to the needs of the students as well. (Note that I did not say the desires of the students.) Ideally, the test I write should embody my philosophy of education; what I believe about teaching and learning, about my role as a teacher, about the identity and integrity of my students, about creating an environment conducive to learning...all of this ought to be reflected in the test that I write.

A tall order? Yep. But it's what I'm aiming for, at least.


  1. I struggle with the idea of open book vs closed book tests. Open book leads to more indepth and thoughtful answers. How important is it for students to "memorize" data rather then interpret the data they are given? Can both closed and open book tests achieve this?

    1. Very recently I saw a conversation on Twitter about "open Internet" tests. The idea that if the questions we are asking on a test can be answered via Google search might mean we aren't asking questions that matter. I found that an interesting argument...but I'm not sold on it. I believe that being able to recall specific information is still important and valuable. But being able to *apply* that information is more valuable, overall, in my estimation. Something to think about!

  2. For me it's about teaching critical thinking. On our report cards it divides many of our social areas (including science) into knowledge of terms and concepts (the memorizing part) and applies (or understands) knowledge - this is the critical thinking. The balance is important and the transparency for the students is important. What exactly is being assessed?

    1. The question you end on is the key to me! Am I assessing what is easily measured? Or what I really value? It's not that these can't be the same thing. (I think they can.) But I think your point about transparency is key. A test shouldn't be a game of "gotcha!" for the students. We should be clear with students about what we want them to know, understand, and be able to do.