cancel
Showing results for 
Search instead for 
Did you mean: 

Add Regular Expressions to Quizzes

Add Regular Expressions to Quizzes

This idea has been developed and deployed to Canvas

 

          

Currently in quizzes, for fill-in-the-blank or short-answer questions, you have to come up with every possible phrase combinations to match what students might type that would be correct. Previous recommendations asked that the answer "contain" keywords rather than match.

I would like to see regular expressions, which have been in every major programming language since the 1950's and every LMS until Canvas. Regular expressions are also known as regex, pattern matching, and wildcards.

Here is a good example:

    Who was the major African-American civil rights leader of the 1960's assassinated in 1968?

The official answer is "Dr. Martin Luther King, Jr.", but what if they forget to put "Dr.", abbreviate Luther as "L.", or forget "Jr.". Is it wrong? How many permutations of the right answer could there be? Is punctuation included?

Using regex, the answer would be /.*Martin.*L.*King.*/i, meaning anything before Martin, must contain an L
between Martin and King, can contain anything after King, and "i" means case insensitive. If you use "g" with the "i", you
can apply multiple phrases across an entire document so you can check papers for content automatically.

Please add this to quizzes. Most programming languages (including JavaScript) accept this as a phrase or function.

This extends upon Allow fill-in-the-blank answers that can *contain* a term vs having to match a term  and Improve Regular Expression Tool (pattern matching).

 

Comments from Instructure

This idea was completed with general availability of Quizzes.Next.  You can find more information about the overall project in the Quizzes.Next User Group.‌

  • Quizzes.Next is now available for all paid accounts.
  • Quizzes.Next documentation will live in the Quizzes.Next User Group until July 14, 2018. On July 14, 2018, all of the documentation will be moved to the Canvas Guides in the Canvas Instructor Guide or Canvas Student Guide
  • Quizzes.Next updates will be included in the Canvas Release Notes. This includes new features, updated features, and relevant fixed bugs. New and updated features will be deployed to Quizzes.Next near the same time as the Canvas release. Fixed bugs may be deployed at any time.
42 Comments
GregoryBeyrer
Adventurer III

This is an excellent idea. Another use of this tool would be to require that the answer to a quiz question be a certain length. For example, I might require a 50-word answer to a question and I do not want to read any answer that is fewer than 50 words. I want to use a regular expression to give a preliminary score to each answer and thereby let me know which ones meet the word requirement, else I have to do lots of copying and pasting or fiddling with browser extensions.

I could also use the same tool to give a *maximum* length and give students a question like writing Twitter-style comments.

scottdennis
Community Team
Community Team

Hello  @rwatkins ​,

Thank you for submitting this feature idea.  If I am understanding correctly it is similar to   which was voted on previously but did not receive sufficient votes necessary to move forward in the process.  Your idea submission will open for vote with the next cohort starting on July 6th.

For more information on the voting process please see How does the voting process work for feature ideas?

ronmarx
Community Member

Oh, I'll be waiting to vote on this idea!!!

This is really a request for some artificial intelligence to be built into machine gradable assessment questions. I'd settle for keywords alone, but while Instructure is adding this type of feature, they might as well just enable full regex and get as close to AI for assessment evaluation as possible.

Thanx for posting this. Please cross-reference my name for the two ideas that will be up for voting in the same cohort in July.

Cheers,

iRon_Mrx

biray
Community Member

This idea has moved to the next stage and will be open for voting among the Canvas Community, from Wed. July 6, 2016 - Wed. October 5, 2016.

Check out this doc for additional details about how the voting process works! Smiley Wink

rake_9
Learner II

We are in the process of moving to Canvas and have instructors whose quizzes in the old LMS rely on regular expressions.  Assuming regular expressions are included in the new quiz tool, I would also hope that the import process for that tool is able to correctly consume and make use of regular expressions.

James
Navigator

Along with great power comes great confusion. Despite being awesome, regular expressions are rarely understood by the masses and when poorly constructed, they can lead to wrong answers being accepted as right.

Consider the regular expression offered in the original request as a good example: /.*Martin.*L.*King.*/i

The following response, although not a statement of my entertainment habits nor the answer to the question, would be considered correct by that regular expression.

I like to play Martingale - the king of roulette systems

So would

Martin Short was on the Larry King show December 13, 2012.

and

Steve Martin plays in the horn section of B.B. King's "In the Midnight Hour" music video.

     credit to IMDB

and

I went to Martinique and found it lacking.

     No offense to Martinique intended.

It does not accept the presumably correct answers of M.L. King, Jr. or King, Martin L.

Now, these are admittedly contrived answers and students probably won't put them into the answer box, but they go to show how much care needs to go into developing good regular expressions. Even in the example given in the referenced ​​, the "regular expression" given ^c[aou]t& isn't a valid regular expression and if the & was a $, it still wouldn't do what the do what the poster claimed (it would not match clout).

Part of the beauty of using regular expressions over some other system of pattern matching is that regular expressions are clearly defined. Yes, there are variations in them, but the basic level is the same for everyone.


Still, even with that level of uniformity, these examples show that they are difficult for people to get right and to understand. By the way, since you don't include ^ or $ anchors, the .* or .* at the beginning and the end are not necessary and /Martin.*L.*King/i is equivalent to /.*Martin.*L.*King.*/i

I am not saying there shouldn't be regular expression support, I'm kind of a big fan of them myself. I use them in most of my code to make sure it runs on the correct page within Canvas. I like text editors that have support for them and wish my browser did in the find command (I didn't like the extension that added them). I even love regular expressions that capture portions for reuse (wouldn't that be truly awesome to capture parts of the student responses to use in a custom response?)

But I am anticipating a lot of questions generated by faculty who will think they know what they're doing when they don't. That's to be expected. You are asking humans without training to speak a computer language. Regular expressions get a lot of questions from computer programmers, so I can't even imagine how much harder it will be for non-coders.

I also know that our Canvas Admin knows nothing about them (she could figure it out, but would probably ask me instead) and I imagine there are a lot of other schools in the same situation except they may not have anyone on-staff who knows them.

Like I said, though, that's not an argument against them. Just an anticipation. And perhaps a statement that sometimes regular expressions are overkill.

If they are implemented, then the regular expressions should be checked for validity before being accepted and there should be a tool that would take a bunch of expressions, one per line, and run them through the validator to let people know what will match or not match.

karen_yip
Surveyor

seriously I have too many things to do than to sit and type out multiple cases of "law" "laws" "policy" "policies"

I would be a user of this function if we could use *

(my quizzes that have been imported from Moodle do have terms with * programmed as answers and now Canvas marks the answer WRONG because I have an * and the student's answer does not. grrrrr)

ronmarx
Community Member

I agree, Karen. James is getting a reputation here of over-thinking suggestions that will make Canvas a more intuitive work environment, especially for the designers creating courses. If you haven't voted up on , please check out the idea and comments. In addition, there's another designer time saver idea up for vote at .

Thanks!

tbunag
Community Member

James's post didn't strike me as saying that this shouldn't be done, just that there are important and serious implications for doing so.  It may be a bit of a devil's advocate approach, but it's really important to address these issues.

What I'd love to see is a regular expression editor of some kind, somewhat like the math expression editor, though I'm not sure if that's practical.  It would be great to be able to do some basic things with a friendly interface (something like "must include: x, y, z" in order: yes/no, and other options), along with an actual regular expression editor for more advanced users.

kmeeusen
Community Coach
Community Coach

Ronald:

I think, but am probably overthinking, that many of  us do not consider Jame's responses as "overthinking". Rather, James provides very complete answers that help better inform our decisions.

A previous LMS that shall forever remain unnamed provided this capability, and as an instructor I made use of it. I learned very quickly that I had to be very careful using regular expressions, and found myself hand-grading FIB answers way more often than I liked. I learned, of course, but if I had read something like  @jjones9 ​ response earlier, I could have saved myself considerable grief.

I too am a fan of regular expressions, and do miss them. So I will support this idea when it opens for voting.

KLM

ronmarx
Community Member

Kelley,

I totally get this. I strive to see the big picture, however, and be as inclusive of ideas that make the tools we use to build content better. I felt James's analysis shut down this idea to simply maintain the status quo. The machines we use—including the Canvas LMS—are only as intelligent as we make them. There are basic tools that are nearly perfect because there are no practical improvements to them that wouldn't transform them into another tool (think construction tools like hammers and hand saws).

If a designer or teacher doesn't need a particular tool offered in Canvas, then don't use it. James himself mentioned the still-increasing library of work-arounds to get Canvas to work more effectively and efficiently. For example, the idea is pretty widely used, especially by teachers in K-12 school districts, and especially when they're migrating content from another LMS to Canvas. Bulk creation of content doesn't seem to be treated by the Canvas work environment, and teachers/designers have to resort to repetitive tasks that suck the joy out of creating new courses. Being able to select multiple items in Modules, multiple pages in Pages, multiple assignments, etc. in order to move/manipulate them seems obvious.

The grief you experienced using regular expressions in FIB or SA questions may have simply been a common symptom of a learning curve. Had James proffered a better solution, then my responses to him and elsewhere would have been different. Having regular expressions in the Quizzes layer of Canvas seems obvious too, especially when you remember in the first place why we're using digital tools to grade assessments.

We all want to make Canvas better, and that usually means change. That's the type of conversation I want to drive here, and why I don't entertain ideas that do nothing more than shut down the conversation. That being said, I've learned a few little-documented procedures and work-arounds that solved issues I was experiencing. I wouldn't have found them if I retreated into a position of not having the conversation to begin with.

Best regards,

Ron M

rwatkins
Community Member

Kelley,

Add Regular Expressions to Quizzes in open until October 5th.

https://community.canvaslms.com/ideas/5826

Randy Watkins
Contra Costa College

rafael_chavez
Community Member

I see James Jones's point that not everybody knows how to write regex, and that writing the wrong regex can lead to mismatched answers (wrong or right);  but the same is also true now. I ask in one question to write an equation; if the student adds extra spaces between the signs to make it more legible, it will be marked incorrect even though it is correct. Anticipating every possible correct answer is very difficult, but having regex as an option would reduce the possibilities for those who know them or are willing to learn them.

mhspico
Community Member

As part of this feature, maybe adding a regex tester would be a good idea. There are plenty on the web, but putting one directly into the Canvas Quiz engine would probably reduce the regex mismatching that James is describing. A simple "Test my regex" button after the regex input field, where users could input text to see if Canvas marked it right or wrong, should work perfectly.

And yes, adding regex importing to the course import tool would probably be a good idea, assuming this gets approved.

CraigOgden
Explorer II

I know how to use Regular Expressions so this would be a nice feature.  I don't think it needs to be "regular expressions", but at least something like it.

Maybe have a dialog box with different option like:

ignore caps, exact, contains, etc...

Regular expressions are nice but can be complicated for non-programmers.

jlmiller
Community Member

YES - and yes to this feature... I use many fill-in-the-blank questions and it would great to even be able to give credit for spelling variations... "s" or no "s" etc.

jsparks
Instructure
Instructure

Hi friends.  We are thinking about this and a lot of other features for quizzes.  We will consider this for implementation. Please take a look at my InstCon presentation for more information.

Jason

CraigOgden
Explorer II

I am liking what I am seeing in that presentation.  Do you know when Quizzes 2 will be coming out?

jsparks
Instructure
Instructure

I don't have a timeline to share at the moment, but we are working on it!

jrowe
Surveyor

Moodle has simple wildcard characters for answers to fill in the blank questions. 

I am requesting that Canvas staff start by implementing Moodle simple wildcards as a feature request.  
example of Moodle simple wildcards here>  Short-Answer question type - MoodleDocs

http://metadatace.cci.drexel.edu/omeka/items/show/2242" title="http://metadatace.cci.drexel.edu/omek...

I'm not really fond of these quotes, as they feel less than polite.

"rarely understood by the masses"  and "humans without training" and " faculty who will think they know what they're doing when they don't."