Using Calculated Values in the Text of Formula Questions

James
Community Champion
15
5896

The examples give a technical explanation and some knowledge of the browser's developer tools, HTML, and JavaScript are beneficial for understanding. There's a not-too-technical way explained before the examples that will work for the simple cases.

When generating formula questions with legacy quizzes, the values displayed in the question text are limited to the random values that are generated. The answers can be calculated, but Canvas does not allow you to use these calculated values in the text of the question itself. Some people call this calculated questions‌.

Let's say that you want to ask a relatively simple question, "find the square root of [x]." But you want x to be a perfect square so that the solution is a whole number between 1 and 10. This is currently not possible using the web interface. You can generate random numbers between 1 and 10 and make them whole numbers, but you cannot square that number and use it in the question.

Understanding the way that Canvas generates random numbers can help. The Variable Definitions and the Formula Definition are calculated in the browser and sent to the server. They are stored, but not computed on the server. Later, when they are displayed to the student, the stored values are used.

What this means is that if we manipulate the values in the browser before they are sent to the server, that they will be delivered to the student the way that we want them to be, not the way that Canvas generated them. They will persist unless we regenerate the questions.

The place where we need to manipulate them is in the table underneath in the Generate Possible Solutions.

The trick is to generate the possible solutions, edit the page, and then update the question.

Easy Way

It turns out that there is a contenteditable attribute that you can add to any element on a page and allow you to edit that element. If simple editing, like in example 1 below, is all that you want to do this, this way is certainly easier than what I originally described. If you want to have the computer do the calculations for you, as in example 2, then you may not need this approach.

If you use this approach, you'll want to be careful. The values for the variables and the solutions are editable, but do not assume that you can change anything on the page this way and have it work. 


This should only need to be done once per editing session. The next time you reload the page you will need to tell it to make the page editable again.

You will only need to do one of the following.

Make page editable with the browser's developer tools

This is the slightly more technical way, but it doesn't require a browser extension.

For the impatient, here are some instructions for Chrome.

  1. Right click the mouse button somewhere near the top of the Questions panel
  2. Choose Inspect.
  3. Find an element in the document object model (DOM) that is a parent of the questions. The div with id=questions_tab is a good place to add this.
  4. Right click and choose Add attribute.
  5. Type: contenteditable="true"
  6. Click outside of what you're editing and you're good. Go back to your page and now you can just type in the values you want.

There are other ways to get to the developer tools, but that's not really what this blog is about.

Browser extension

If you don't feel comfortable editing the HTML on the page, I found a browser extension for Chrome called Content Editable. It adds the contenteditable="true" attribute to the body element, which makes your entire page editable. Then you just click in the table and type the new values directly and then update the results.

Example 1: Generate Perfect Squares

This is the example I've been using so far. I want the answer to be an integer value between 1 and 10. I'm going to call it x. I create a variable definition for x and then make x the answer in the Formula Definition section.

Set up the problem

307715_pastedImage_3.png

Generate the possible solutions

Now we ask Canvas to generate some answers. I'm going to generate 3 combinations, just because that's all that will show up in their table anyway without scrolling.

307716_pastedImage_4.png

Manually edit the results

Do not update the question just yet. This is the step where have to manually edit the values in that table.

If you chose the easy way listed earlier and made the page editable, then you can skip this section and just type in the values you want.

Most people are using a browser that has a developer tools built in. There are various ways to get to it. It's probably enabled for Firefox and Chrome and activated using F12, but you may have to enable it with Safari. Once you do this, click the inspector tool and then click on the 8 in the x column of the table.

307718_pastedImage_9.png

Another way to get there is with Firefox or Chrome is to mouse over the 8 in the x column and click the right mouse button. There you can select Inspect with Chrome or Inspect Element with Firefox. The screen shots I'm showing are with Chrome.

Should see something like this in the developer tools once you click on the 8 with the inspector.

307719_pastedImage_10.png

What we have is a standard HTML table (it helps if you know what those are). Here's what the entire table looks like if you were to expand it.

<table class="combinations">
<thead>
<tr>
<th>x</th>
<th class="final_answer">Final Answer</th>
</tr>
</thead>
<tbody>
<tr>
<td>8</td>
<td class="final_answer">8</td>
</tr>
<tr>
<td>3</td>
<td class="final_answer">3</td>
</tr>
<tr>
<td>10</td>
<td class="final_answer">10</td>
</tr>
</tbody>
</table>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

If you know how to read HTML, you'll see 2 columns, identified by the <td> element. Probably easier for people to see is the 8, 3, and 10 in a plain <td> and the the (hopefully) obvious td with the class of final_answer.

It's the first td element that we need to change.

In both Chrome and Firefox developer tools, you can double click on the 8 and edit it right there. Do this and make it 64.

Expand the second row by clicking on the <tr> element so that you can find the 3. Then double click on the 3 and make it 9. Then double click on the 10 and make it 100. Again, you are doing this in the developer tools, not on the web page itself.

After doing this, the HTML page looks like this

307720_pastedImage_11.png

Update the question

Now you can update the question.

When you preview the quiz, you get this:

307721_pastedImage_13.png

And the correct answer is 3

307722_pastedImage_14.png

Success! As long as you don't regenerate the possible combinations, the values you put in will persist. If you regenerate the possible combinations, you'll need to re-edit the page.

Example 2: Enforcing Constraints

In statistics, I have some questions about probabilities. They normally go something like this. "[a]% of people feel the full Mueller report should be released, [b]% feel the summary should be released, and the rest said it should not be released. If [n] people are randomly selected, find the probability that [x] said the report should be fully released, [y] said the summary should be released, and the rest said it should not be released."

The reason for the rest is because I cannot force Canvas to pick three random numbers that add up to 100, which is what I really desire.

The variable definition section normally looks like this, making sure that a+b cannot exceed 100 and x+y cannot exceed n.

307723_pastedImage_18.png

What I would really like to ask is this. "[a]% of people feel the full Mueller report should be released, [b]% feel the summary should be released, and [c]% said it should not be released. If [n] people are randomly selected, find the probability that [x] said the report should be fully released, [y] said the summary should be released, and [z] said it should not be released." 

And then enforce the conditions that a+b+c=100 and x+y+z=n

Set up the problem

Here is what my problem set up looks like. Note that I've put place holders in for [c] and [z], but the values I put in them are totally irrelevant, I'm going to override them in a later step.

307724_pastedImage_20.png

When I create the formula definitions, I do not refer to c or z directly because they're bogus. I need Canvas to calculate the correct answer for me without using those variables. You can assign them the correct value as I did with z, or you can just use the formula for them as I did in finding p3, which is really c=100-a-b and then c/100.

307725_pastedImage_21.png

Generate the possible solutions

This time, let's generate 20 possible solutions. I've discussed in other places how that is overkill and likely to generate duplicates and not give you the randomization that you really want. But for this problem, I want to make it difficult to manually edit and show you how to automatically compute the values. I also want to show you how to delete rows that may contain values you don't want, such as the 0 in the first row.

307726_pastedImage_23.png

Manually edit the HTML

As explained in the first example, open up the developer tools. 

Here's what it looks like in the elements panel.

307727_pastedImage_25.png

Deleting unwanted answers

Notice that the final_answer for this one is 0. That's unacceptable to me. It would be too easy for a student to enter 0 without actually knowing what they're doing and get the question right.

I want to delete all the rows that have 0 as a final answer.

The easiest way to do this is to manually find the rows that have a final answer of 0. For each one, right click the mouse on the <tr> element that holds that row and choose "Delete element."

307728_pastedImage_26.png

That will completely delete the row from the table and you'll now only have 19 solutions instead of 20.

My visible page within Canvas now looks like this:

307729_pastedImage_27.png

If I scroll down some more, I find additional rows with final answers of 0. I can manually delete them as well.

If you want some JavaScript to check and see if you caught them all, then you can switch to the console panel and paste this code in.

document.querySelectorAll('table.combinations td.final_answer').forEach(a => {
if (a.textContent === '0') {
console.log(a);
}
});‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

It doesn't tell you which items are 0, but when I run it, I see that I have 4 more.


If you really trust your JavaScript skills, then this will remove all rows that have a final_answer of 0 automatically.

document.querySelectorAll('table.combinations tbody tr').forEach(row => {
const fa = row.querySelector('td.final_answer');
if (fa.textContent === '0') {
row.remove();
}
});‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Adjusting the remaining values

At this point, I've removed all the rows I don't want and I'm down to 15 that I do want. I could manually go through and edit them as explained in the first example, but that's kind of a pain, especially if you have lots of values.

For each row, I need c to be 100-a-b and I need z to b n-x-y. For the first row, c should be 100-63-19=18 and z should be 22-12-3=7.

307731_pastedImage_31.png

For this to work, realize that JavaScript indexes arrays starting with 0, not 1. So column a is really column 0 and column c is column 2. Column z is column 6.

I paste this code into the console panel and execute it by pressing enter.

document.querySelectorAll('table.combinations tbody tr').forEach(row => {
const col = row.querySelectorAll('td');
col[2].textContent = 100 - col[0].textContent - col[1].textContent;
col[6].textContent = col[3].textContent - col[4].textContent - col[5].textContent;
});‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This is what my browser looks like now.

307732_pastedImage_32.png

Success! Woohoo! Yippee!

Update the question

After updating the question and previewing it, I get the following question.

307733_pastedImage_34.png

JavaScript Notes

The JavaScript that I included is powerful, but also dangerous and you should make sure you understand what's going on if you're going to use it.

Here's a deeper dive

document.querySelectorAll('table.combinations tbody tr').forEach(row => {
const col = row.querySelectorAll('td');
col[2].textContent = 100 - col[0].textContent - col[1].textContent;
col[6].textContent = col[3].textContent - col[4].textContent - col[5].textContent;
});‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

In line 1, it finds all rows inside a body of a table with a class of combinations. There are two tables with the class of combinations, but that seemed to limit it to the correct table. The forEach then iterates through each row of the table.

In line 2, I grab all of the columns from that table. The columns are in td elements.

In line 3, I assign the value for the c variable, which is the third column, but because JavaScript starts indexing at 0, it's col[2].

In line 4, I compute the value for the z variable in the 7th column (col[6]).

The values in the table are text. Their value is contained within the textContent property. You may have seen the value property before, but that is for inputs and these are not inputs.

Since they are text, you need to be careful with the math. JavaScript tries to coerce values when necessary. Since I'm doing subtraction, it tries to convert the text into a number so that it can subtract the values.

However, for addition, the + operator is also used for string concatenation. The numbers 10 + 13 are 23, but the strings '10' + '13' are '1013'. And if you mix integers with strings, you can get really confusing values until you understand what's going on. 100-'10'+'15' is '9015' (It coerced the '10' into 10 and got 100-10=90, but then appended the string '15' to '90' to get '9015').

To work around this, you may need to force the text into numbers using parseInt() or parseFloat(). parseInt('10') + parseInt('13') is 10+13=23. Another way to force the conversion is to multiply by 1: 1*'10'+1*'13'=23.

Again, if you don't understand the JavaScript you are using, you may want to stick to manually editing the HTML using the developer tools.

Example 3: Using variable text (not possible)

Let's say you wanted to have a question like "What position in the English alphabet is [x] in?" where [x] is a letter such as "A" or "B". You might think that since you can edit the HTML directly that you can pass anything you want to Canvas and it will show up.

307734_pastedImage_2.png

If you try putting in a value that is not numeric, Canvas sanitizes it and makes it 0 if it cannot be recognized. If you go back later and try to edit the question, you get this:

307735_pastedImage_3.png

You cannot do text substitutions in formula questions.

15 Comments
barrybrown
Community Participant

This is just begging for a Chrome extension to automate the process.

James
Community Champion
Author

Which process are you talking about automating? I listed a browser extension for Chrome in the "Browser Extension" subsection of the "Easy Way" section. It makes the page editable. Are you talking about a browser extension that adds all the functionality to formula questions that is missing?

barrybrown
Community Participant

The latter. Since all of the variable values are generated by the browser, the extension could provide a place to upload a file or a spot to write code to generate the values (similar to what you showed with the developer tools). It could even remember the code used to generate the values so the next time the quiz is edited it will re-run the same code.

James
Community Champion
Author

I was thinking of a different approach a few years ago.

I was thinking that if you used a variable name in a definition that was the same as one of the random variables you had, that it would calculate the rule. That way people could install a userscript and save everything as part of Canvas. That way it would persist across browsers and even work if you came back another semester and didn't have the values saved because you had cleared your browser storage. I don't know how to create an extension, so I hadn't thought of doing it that way. Most of my stuff is done with user scripts and Tampermonkey. The user script means it's up to the user to enable or not, so they could have the default actions of Canvas or they could choose to do it the new way.

For example, if [a] is a random integer between 1 and 10, but you wanted to show a² to people and ask for the square root, then you could create formulas like this:

  1. x=a
  2. a=x*x
  3. x

The "a" on the left hand side of the equation in formula 2 would store the value to a, overwriting any value that was randomly generated.

The storage to x was because I needed the original value for the answer, although I suppose I could have done

  1. a=a*a
  2. sqrt(a)

Or if you needed a constant, for example: What is the significance level when the confidence level is [cl]% ?  Let [cl] be a random integer between 0 and 2 and then you could either have 90%, 95%, or 99% for the confidence level using this approach:

  1. n=cl
  2. cl=at(reverse(99,95,90),n)
  3. at(reverse(0.01,0.05,0.1),n)

One problem I saw was that Canvas didn't save the value of the formulas, especially the intermediate steps, only the answers to the final step. In other words, the a=x*x isn't saved to an accessible location. That means I would have to write code that would duplicate the functionality that is already built into Canvas. I suppose I could borrow their code, but that seems dishonest -- even though it is open source.

uwe_zimmermann
Community Participant

I was actually looking for something else, when I now stumbled over this entry. But yes, I have been wondering as well, why the table of calculated results is just not editable in Canvas as it is in Blackboard - this should be pretty easy to set up from the developers' side and would make it so much easier for us to use!

I will try to edit the HTML-table manually next time, but this is not a nice way to do it.

cgaudreau
Community Contributor

 @James ‌, since you mentioned contenteditable‌, I'm taking a chance to run something by you:

I am looking for a way to prevent faculty from editing certain aspects of a template, in so much than one erroneous <div> being removed can result in a big mess, and lots of reactive clean up.

Taking a chance, I added the following code to my instance:

<div class="non-editable" contenteditable="false">

However, not being on the HTML Whitelist, the code is stripped on save and returns:

<div class="non-editable">

This is a perfect use-case for us, but we're kind of stuck with the strip of the second half of the code. Any thoughts on how to keep it going through to the save state?

Thanks in advance!

James
Community Champion
Author

cgaudreau 

I'm not sure how your templates are set up, but if you're entering it through the RCE, then it's inside an iframe where the body is marked contenteditable=true. The only way I can think (I've got class in 2 minutes so not a lot of time to think), is to run custom JavaScript that watches for the RCE to pop up, then look for a class called "non-editable" and then apply add the contenteditable=false to that.

That's off the top of my head, completely untested.

cgaudreau
Community Contributor

Thanks for the tip. We don't use JavaScript in our instance, but this might be a solution worth investigating.

Hope you have a good class, and thanks for the in-the-moment feedback!

James
Community Champion
Author

You would definitely have a problem if they clicked on the HTML Editor as it will all be inside one element and you wouldn't be able to apply that.

Another way I can think to do it is to handle it outside of Canvas. Have a form that people fill out with the information and then take the information from the form and write it to Canvas through the API. That may not be acceptable, but what you're wanting to do isn't a core functionality so you have to look for creative alternatives.

James
Community Champion
Author

 @uwe_zimmermann  wrote a nice blog about Using Python to create Calculated Questions. It provides some additional information about how things work.

yyuen
Community Novice

All,

I have created a chrome extension to do everything this page talks about and then some. Download it here:

https://chrome.google.com/webstore/detail/canvas-quiz%20/jbpanaoeongimfdcfbnhfaagblakgpca?hl=en&auth...

Click on "Help" next to the new selection button for more info.

The extension page has my email, email me if you have any questions.

yyuen
Community Novice
ardavan_asef-va
Community Member

Hi

Thanks for great information. I generate my quizzes - both multiple choice and fill multiple blanks- outside of CANVAS. They are configurable- usually I generate 26 sets of each problem (using excel functions and macros)  and then take them to canvas. This is much better than formula questions since beside providing 5 choices for each question (in the problem statement I provide 5 answers marked A to E, and the students enter a letter even in fill multi blank questions), I can also use all excel functions (including Normal Distribution etc ) and also I have control on the numbers (like the example you have used for sqrt.

But there is one serious shortcoming- I Can not place this group of questions within another group. That means, unlike formula questions that I can randomize their appearance in the exams, I cannot do the same for the quiz groups that I generate. Formula quizzes are not considered group, but my quizzes are groups.

Is here a way to put several groups of questions (say each containing 26 questions) within a master group (say containing 15 groups)?

Best Ardavan

YushYuen
Community Member

Updated the chrome extension. More power:

  • Now work on all domains! (used to be just *.instructure.com)
  • Upload CSV file! Just create your solution sets in your favorite spreadsheet program. Import the CSV file. DONE!
  • The solution table is directly editable! Just edit away!
  • Create any custom javascript function as your formula.

Here is the link to the extension:

https://chrome.google.com/webstore/detail/canvas-quiz%20/jbpanaoeongimfdcfbnhfaagblakgpca?hl=en

ardavan_asef-va
Community Member

Is there a not-difficult way to copy some quizzes (multiple choice, fill the blank, formula) into new quizzes?

Thanks