Multiple choice answer formatting

[jmillner1]

Would it be possible in the future to have the multiple choice answer items be shown in a 2 by 2 format vs. the 1 by 4 it is currently in? For students having to choose from a set of graphs, it would be helpful to see them together vs scrolling. 

[James · Accepted solution]

@jmillner1 

I don't think is a good idea from a usability perspective. It may work well on large screens where graphs can be seen side by side, but it creates issues for students using small screens. It also creates an inconsistent pattern where some questions are side-by-side but most are vertical. That can lead to students missing those questions off to the side because they're not used to looking there for it.

While you might want only a 2x2 grid, other people might want to use a horizontal list for all four, which makes the matter worse.

I've had issues with online math homework systems that put them side-by-side (Webwork comes to mind) is that the label appears at the bottom left of the image and looks like it's the label for the graph below it rather than the one above it).

Canvas has put a lot of thought and testing into creating a system that works for the majority of users on a large range of devices. Instructors may lack the training in accessibility and usability and create what looks good for us on our screens, but end up making matters worse for students. That's what happened with the WebWork questions -- the instructors were content experts, but not design experts. Then questions that made perfect sense to the instructor that wrote the question ended up being shared as part of a national library and given to lots of students without that instructor and so it makes no sense. Canvas doesn't share your questions (unless you put them in Commons), but I'm making the point there is a lot to making a good question that works well for all students than most people consider.

Having a horizontal list of answers might be a good idea. The LaTeX tasks package creates horizontally columned lists rather than vertical lists and says the reason for its creation was an unwritten agreement in German math textbooks to organize columns horizontally. I use this on some of my written exams that get printed before giving to the students. With the paper copy, I have complete control over the layout and don't have to worry about different devices. It's even great for graphs since I don't have to mess with minipages.

But, I don't think it's right to use in an online format like Canvas.

That doesn't mean we're stuck with nothing. Here are some alternatives approaches. I'm not saying any of these are perfect, just hoping to encourage thinking about it in a different manner. Sometimes the answer to "why am I doing this" is "because I've always done it this way" and those are circumstances where rethinking things can lead to a better way to do things.

Sometimes we're forced to change the way we ask things because of the software we use. I teach math and we have to often change how we have always done things when creating online question. Canvas does not support the requirements of math or science very well and we often have to rely on external systems. We all feel that technology should not drive what we can ask, but sometimes it can inspire us to create better questions.

• Add the graphs to the prompt and label them as A, B, C, D. Then ask the student for the letter (fill in the blank rather than multiple choice if you like). This eliminates the randomization and potentially still has the same issues with getting things that look good on your screen vs good on student screens. The randomization could be resolved by creating multiple versions of the same question and then pulling one of them for the quiz.
• Include a single graph in the question prompt and then ask them for the equation that is graphed as the responses. You've just created up to four times the work for student.
• Include a single graph in the question prompt and then ask the students to describe the graph.
• Instead of having the student pick the correct graph, give them a single graph and then ask them why that is not correct. I use something like this in the series chapter of calculus 2. I may start "A Maclaurin series polynomial begins p(x)=2+x-x^2+x^3/4+..." and then ask "Why can this graph not be the graph of the polynomial?" Choices involve y-intercepts, increasing/decreasing at x=0, concavity at x=0, etc. One of the prompts may be that there is nothing wrong and that is the correct graph, but then they had to think about all of the things that could be wrong before picking that one.

You can always create different versions of the question and then offer one of them in the quiz. This helps reduce cheating but requires more work than just checking the box to shuffle the answers.

You didn't say what kind of graphs you were using. What I've seen most are the "here's an equation, pick the right graph from the four options" questions that show up in online homework/testing systems. I feel those are some of the most worthless questions out there. Students just go to Desmos or their graphing calculator and graph it and then pick the one that matches. Even if the question is to first identify the critical points, then the intervals where the function increasing or decreasing, find the maximums and minimums, find the inflection points and the intervals where the function is concave up and concave down. At the end, the student ignores all that and just uses technology to graph it. Worse, they don't check the graph they pick against the answers they gave. The student doesn't learn anything. Then they get 100% on the homework and fail the exams.

I know the person that wrote those homework question is trying to mimic the paper-and-pencil approach where you would do all those things and then manually make a sketch. That's not what students do when they have access to technology. They just graph it.

After the TI-92 came out in 1995, I had to decide whether to ban them during tests or change the questions I was asking. I changed my tests to not ask the straight-forward skill questions you see because students have calculators that can do those things. Instead, I focus on understanding and applying rather than mere computation. That decision is not unanimous among math teachers. Some instructors / schools take different approaches -- I have a trig student who took trig at another school that didn't allow them to use calculators at all (or at least on the test). But if you're doing an online quiz, students have technology available. Even if you have locked down the browser, unless you're monitoring them, they could use their phone or another computer to look up the content.

[jmillner1]

Thank you James for that very detailed response to my question. I too teach Math and completely understand your views on everything you stated above.  Though this type of question may be "worthless" they are still asked on our state tests, and our teachers still incorporate them occasionally. I was asking a question, not demanding that Canvas change its system just for a teacher.  We are all working on our teaching practices and will continue to do so. 

[LaMercer]

That's a brilliant point. Many students tend to scroll down on tiny screens to find answer choices. This is especially true when dealing with content like images of correct food chains/webs, data tables, political cartoons, or anything that necessitates visual representation. Thanks for highlighting that, jmillner1!