Creating exams…it’s not an easy task. From crafting exam questions to trying to make your exams more accessible while also trying to promote academic integrity – there are numerous elements to consider. In this episode we talk about exam accessibility including time restraints, how you can write better exam questions, and what you can do about online exam security.
Although there’s not a one-size-fits-all approach to designing online exams or in-person exams, there are numerous factors you can take into consideration to more effectively asses your students and to be more conscious when creating your exams.
Recommended Resources:
20-Minute Mentor: How Can I Make My Exams More Accessible?
20-Minute Mentor: How Can I Write Better Exam Questions to Measure Student Performance and Learning?
20-Minute Mentor: How Can I Increase Exam Security with Custom Question Banks
Magna Online Seminar: Writing Better Multiple-Choice Questions
20-Minute Mentor: How Can Students Use Self-Compassion to Reduce Test Anxiety?
Faculty Focus article: Fourteen Simple Strategies to Reduce Cheating on Online Examinations
Creating exams…it’s not an easy task. From crafting exam questions to trying to make your exams more accessible while also trying to promote academic integrity – there are numerous elements to consider. In this episode we talk about exam accessibility including time restraints, how you can write better exam questions, and what you can do about online exam security.
Although there’s not a one-size-fits-all approach to designing online exams or in-person exams, there are numerous factors you can take into consideration to more effectively asses your students and to be more conscious when creating your exams.
Recommended Resources:
20-Minute Mentor: How Can I Make My Exams More Accessible?
20-Minute Mentor: How Can I Write Better Exam Questions to Measure Student Performance and Learning?
20-Minute Mentor: How Can I Increase Exam Security with Custom Question Banks
Magna Online Seminar: Writing Better Multiple-Choice Questions
20-Minute Mentor: How Can Students Use Self-Compassion to Reduce Test Anxiety?
Faculty Focus article: Fourteen Simple Strategies to Reduce Cheating on Online Examinations
This is the Faculty Focus Live podcast sponsored by the Teaching Professor. I'm your host, Tierney King, and I'm here to bring you inspiration, energy, and creative strategies that you can utilize in your everyday teaching. Creating exams, it's not an easy task. From crafting exam questions to trying to make your exams more accessible, while also trying to promote academic integrity, there are numerous elements to consider. So today we're going to talk about exam accessibility, how you can write better exam questions, and finally, what you can do about online exam security. To start in Elizabeth Harrison's 20-Minute-Mentor, How Can I Make My Exams More Accessible?, she explains how important it is that everyone has an equitable opportunity to learn, and in terms of assessment, an equitable opportunity to demonstrate what they've learned.
Elizabeth Harrison:What I want to suggest to you today is that it's very important for us to provide equitable opportunities for students to demonstrate what they've learned. Why is this so important? It's because on the one hand, we know that all learners differ, everyone learns differently from everyone else, for many different reasons. Biology, genetics is certainly one, socio economic background is another that involves the preparation that students had available to them, or took advantage of perhaps. We also know that we have a wider variety of students in our colleges and universities today than we ever had before. And that includes students with disabilities who in the past would not have been able to come to college, or a university. And we need to think about all of these, the great variety of learners and provide for them, make sure that they have an equitable opportunity to learn and to demonstrate what they've learned. Now, this is not saying that the rigor of our courses should be changed. We do not guarantee that all students will succeed in college, but we should guarantee that everyone has an equitable opportunity to learn. And then in terms of assessment, an equitable opportunity to demonstrate what they've learned. So three things to consider when you're designing an exam. First, your goal? What do you want students to demonstrate? What are you looking for, what will you be looking for as you grade these exams? And then what is essential in relation to the goals of the course?
Tierney King:As you create your exams and keep these overarching ideas in mind, Harrison also recommends alleviating some of the stress exams can have on students by taking the concept of time out of the equation. But this only works if it's compatible with your course content and what you're assessing. Sometimes, Harrison says time is necessary to incorporate into exams. But if you do have the opportunity to take time out of the equation, Harrison recommends trying it.
Elizabeth Harrison:So here's the most controversial piece of this, and that is the question of time and testing. Is testing under pressure of time essential to your course or to your discipline? My background, my disciplinary training is in history, I teach Buddhism and about Asian religions. I've decided as I thought about this over the years that I don't care how quickly students can share their ideas with me or their understanding of a concept with me, I want to see what their understanding is. I want to see how they're relating that to themselves or to other concepts in the class. A way to take time out of this idea of assessment is to allow all students to take as long as they want. You can almost do that with a take home exam where students have three days or a week, whatever - whatever period you set. There is a time limit there. But it's so long that it may as well not be there. If students choose to do their exam the night before that's on them. It's not on you or on us. I choose to schedule my class at the end of the day in the evening. I know no one else follows me in the classroom. So I tell my students they can all take as long as they want on an exam, and I bring a book or other work and I sit and wait until everyone is finished. It never, it rarely takes more than half an hour beyond the usual testing time and everyone has a chance. The stress is off. And they can think about what they want to say instead of thinking about whether they have enough time left to say it. You could consider constructing shorter exams. What do you really need to have students show in order to assess whether they know their stuff? Maybe three problems in a math exam would work just as well as seven. And you could give them more often. It wouldn't take you as long, the grading is less, and it might cumulatively add up to the same amount of time. But it's quick if you only have a few problems. So you might think about that as another kind of solution.
Tierney King:As you start to become more conscious of creating accessible exams, you also have to take into consideration the craft of writing exam questions, and how you can create well written multiple choice questions or open ended questions that don't cause confusion. Your goal is never to confuse the students, but measure their learning through well crafted exam questions. Here, Ken Alford addresses some of these issues in his 20-Minute Mentor, How Can I Write Better Exam Questions to Measure Student Performance and Learning?
Ken Alford:Exams are difficult to write. We just fool ourselves if we think they're not. A well written exam is difficult to put together. And the challenge is, is once you get it put together, it's not going to be with you forever. You're going to have to keep working on it and changing it. The first suggestion I would give you is that you should be on the lookout pretty much all the time. Just have it in the back of your mind that if a topic comes into your mind, and you think of a way to approach that with a test question, then write it down right then. Keep a running account of test questions that you can use and tweak and add to your exams. Because it's very difficult. If you sit down and try to write the exam the day before it has to be submitted to the testing center, it's much more effective to think of good ideas along the way. Here are a few suggestions that I would give you regarding the writing of multiple choice questions. The first is, you need the grammar to be as unambiguous as possible. English is a tough language. It's very easy to be ambiguous in English. It's so easy for us to misplace a modifier or to to have an implied pronoun, which was not the pronoun we intended to imply. Or to have a phrase, a modifying phrase misplaced. Grammar that is imprecise in a multiple choice question can really caused students difficulty, because they won't be able to hear the tone, the intonation, the emphasis, and the way you wrote and read that question. They'll read it cold, most likely in a testing center. And it can sound very different than when it came off your fingers on the on the keyboard. And so it's helpful if you have your questions read by someone else. But I would suggest for the second item, avoid incomplete statements. For example, I would avoid a question like this, Which of the following is the state capital of California? and then a colon and four choices. I learned and was quite surprised to learn that that kind of incomplete structure can sometimes give students that are not native English speakers, some extra difficulty in trying to understand that question. A small way to modify that would be to take that question and say, Which of the following items listed below is the...and then explain what it is you're looking for. Or in the case of the state capitol, Which of the items in the list below is the capital of the state of California? It's a very small change. But it can make a big difference, especially for non-native English speakers. Another suggestion I would give is to avoid extremes. All, none, never just cause confusion. A student can really understand the concept, or the principle that you are testing and miss it because they didn't quite catch the impact that the superlative had on the question. So I would just avoid them. At the same time, there's two other distractors or possible answers that I would just completely removed from your tests. I've, for about 10 years or so, I've just completely taken these off. I don't have any answers that say all the above. And I don't have any answers that say none of the above. And I can't tell you the number of students that have thanked me for that small little change. Because I remember as a student, I hated all of the above and none of the above because I could understand the principle - I understood it. And I might forget one really arcane, little exception that would cause it to be missed, which always struck me more as a trick rather than an evaluation of my knowledge. So I just recommend you remove those.
Tierney King:As you're thinking about multiple choice questions, tou've had to be creative in terms of establishing strategies to reduce cheating, especially this year with online exams. This concept of online exams and security has been a looming question of concern, and in Ken Alford's 20-Minute Mentor, How Can I Increase Exam Security with Custom Question Banks? he explains what question banks are, and how you can use them to help reduce cheating on online exams.
Ken Alford:What is a question bank? Simply a question bank is just a collection of questions. You might think of, oh, you know, on the televised lotteries, or bingo parlor games, they have those big metal cages and they'll put balls in with numbers or letters and things and then they turn the cage and out, we'll drop one of the balls. Well, that's kind of the way a question bank works. Think of it as if you've got one of those metal cages and into that cage, you drop your questions. And then the questions can come out in a random order. So think of it this way, if we have that metal cage, and we drop, you know, questions into that cage, how many ways are there to pick four? If we have four questions, and we want to select two questions out of that metal cage, how many ways are there to do that? Well, it turns out there are six different ways to select four different questions that are unique, we could select the first and second questions. And that's the same as if we pick the second and then the first. So what order they're in doesn't matter. We want the unique questions. But we could pick one and two, and one and three, and one and four, and two and three, and two and four, and three and four, we have six different options. Now let's modify that just a little bit. Let's imagine that we have our 25 questions, and we put them in that circular ball to choose from. But let's imagine that we write just for more questions. So we've got 29 questions now. Now, out of that 29 questions, each student will receive 25. So there will be four questions out of the question bank that students won't receive. Just making that little change of adding four questions to the testing bank changes the number of available and possible test combinations and question combinations that students could receive from one on the printed test to 23,751 different combinations that students could receive different question numbers in their test. Now, if we select a few questions, oh, my goodness, this is where the power of math really starts to kick in. If we have 29 questions still in our question set, and instead of picking 25, we decide to pick just five, if we do that, it turns out there are over 118,000 ways to pick five questions from 29. What happens is if students talk to each other, they'll say such and such as on the test, and if you give it to them even in a little bit different way, I found it's enough to disincentivize them from improperly sharing. Also, you need to be careful with student perceptions. Because they will, they can perceive that it's unfair they're not receiving the exact same questions. And so some considerations for you are, I would adjust the exam in stages. I wouldn't go completely to a test bank environment, I would introduce a test bank one semester and then check the question, see how they're doing make sure that they're fair. You need to make sure the test stays the same level of difficulty.
Tierney King:In addition to a custom question bank, the Faculty Focus article, Fourteen Simple Strategies to Reduce Chea ing on Online Examinations, the author offers insight into simp e ways to decrease cheating during online exams. This i cludes creating questions that equire higher order thinking, uch as requiring students to e plain, analyze, infer, create, ompose or even evaluate their ma tery of course content thr ugh the question. Using varied uestion types such as op n ended questions intermi ed with multiple choice can make it difficult for students to give the same response s their peers. This method also equires students to use pecific responses and su porting narratives that are u ique to their understandin of the content. You co ld also creatively remind stu ents of academic integrity poli ies. By creating a short v deo and explaining the guidel nes for the online exam. Ther may be some psychological i pact on students after v ewing a mandatory short ideo on integrity policies. You ay also want to delay score avai ability. Set a later date fter the testing window ends So this way, students can't a vise other students if their a swers were correct, or incorrec. Although there's not a one-si e-fits-all approach to desig ing online exams or in-person e ams, there are numerous fact rs you can take into considerat on to more effectively assess yo r students and to be more con cious when creating your exam questions. Whether you're driving to work, or you just need a 15-minute think session, we hope the Faculty Focus Live podcast will inspire your teaching, and offer ideas that you can integrate into your own course. For more information on the resources included in this episode, please check out the links provided in the episode description.