Allow Global Account setting for "Treat Ungraded as Zero"

0 Likes
(5)

My role is both an administrator and assessor/teacher in our organisation.

We have a need to be able provide "Treat Ungraded as Zero" as a Global Account setting for both Teachers and Students.

As our organisation is a VET provider where students need to have a score of 100% in order to gain their qualification, both students and teachers need to always be able to view where they are at by percentage of the course correctly completed.

Without this set, it appears that the student may have completed all of their work correctly but they don't realise that they still have assignments to submit. Teachers also have the same problem. This becomes even more important as students near the completion of their course. As it stands they see 100% but don't realise that it is only based on graded assignments.

If this idea was implemented:

  • For Teachers, this means we could set it at an account level so that all of our teachers would automatically set that way as soon as they enter the Gradebook, instead of us having to continually ask and explain to our teacher how to do so.
  • For Students, this means that we could "Calculate based only on marked assignments" unticked as a Global Setting. They could still choose to tick it if they wished but no matter how many times they are asked to untick the box, they do not do so. It is also not helpful that if they untick the box and then leave that screen and return to it, they have to untick the box again, over and over. 
13 Comments
Naomi
Community Team
Community Team
Status changed to: Open
 
rohan
Community Novice

essential change

laura_weatheral
Community Novice

The number of times I have had a student tell me they have finished because they have 100% when half the course isn't yet completed; is ridiculous! This would make it clearer and simpler for assessors, teachers and students. 

jerry_nguyen
Community Contributor

Having that option would be great. But for now, there is a workaround using Custom Javascript (uploaded via Theme Editor) with the following script

$(window).on("load", function(){ //Load the script when the page is "loaded"
if (window.location.href.match('/grades/')) { //only run this script on Grades
$("#only_consider_graded_assignments").click() //Click the checkbox
// $("#only_consider_graded_assignments_wrapper").css("display", "none"); // remove '//' before this line if you want to remove the checkbox from the page
}
})
James
Community Champion

Faculty who want to force students to start off with 0 points should go through and use the set default grade to put in 0's ahead of time for all assignments. This should probably only be used in a non-weighted gradebook, points-based system.

If that is too onerous, then you could make one assignment "Course Completed" that is graded as complete/incomplete and worth enough points that it will keep the student from ever reaching 100% unless it is checked. Once the student meets the requirements, the teacher checks that box. This could even be automated (we have one course where we scan the submissions every 20 minutes and check for completion to do further processing)

Along that same line of thought, you could educate the students and instructors that completion of the course is not 100%, but once the instructor checks that "Course Completed" box. I'm a big proponent of explaining things whenever you do something outside the normal usage.

James
Community Champion

Jerry ( @jerry_nguyen ), thanks for the JavaScript. I've noticed how you've been stepping up in the Community recently and providing lots of technical solutions; I appreciate it.

There are some issues people should know about before they implement this.

The biggest is that it won't fix the issue on the mobile apps where that checkbox doesn't exist, it always provides the "ungraded only" experience.

There's also a slight typo in your check for running on the grades page. It only works if there is a trailing slash, which isn't there by default. Once that is fixed, it will also run on the view all current grades that is linked from the Dashboard. That will fail gracefully because you're using jQuery when it doesn't find the id=only_consider_graded_assignments.

I would use a regular expression like this to make sure it's on a course grade.

 /^\/courses\/\d+\/grades\/?$/.test(window.location.pathname)

 

The original poster had mentioned they wanted it to be this way for teachers as well.

That is a bit more challenging, but more enforceable -- at least when looking at the individual view. It used to be that you could go to the individual view, turn on the "treat ungraded as 0" and then go back to the main page and it would stick. With the 2021-02-20 release, they added an account feature that must be enabled for instructors to turn it on for the main gradebook (our institution does not have it turned on).

The URL for the gradebook is the same whether it is the class view or the individual view. However, in the individual view, there is a input#ungraded that can be checked or unchecked.

For the main page, the option is inside of a span that is dynamically generated using ReactJS and has all those ugly non-friendly class names that may change across releases. That means adding mutation observers, then clicking on the View menu, waiting for it to appear to see if it's checked. That is really bad accessibility.

The value isn't settable through the API. It's a value that is saved in the browser's local storage with a key of _userid_course_courseid_include_ungraded_assignments and is either true or false. True means to treat the ungraded assignments as 0.

The good news is that if you could write to the local storage in the browser, you could force it on without having to mess with mutation observers and breaking the accessibility. I do not know when it is read, but changing it in the local storage does not trigger a recalculation on the page.

I don't recommend that anyone take the JavaScript approach to solve this problem, but to incorporate a Canvas-based approach like the extra assignment or putting in zeros ahead of time so that it works on all devices for all users.

jerry_nguyen
Community Contributor

@James 

Thanks for your in-depth insight and correction as always 👍

I must agree with you that having to use Javascript should be the last resort as users can disable the custom JS easily and it doesn't always work cross-browser and devices (E.g. Mobile app). 

The option to "View ungraded as 0" (Trainer's grade book) can be changed through an undocumented API endpoint '/api/v1/courses/course_id/gradebook_settings?gradebook_settings[view_ungraded_as_zero]'

Off topic, James, I'm curious how do you set up the periodic scan for submissions? is it through a cron-job to send API requests or using the "Live Events" provided by Canvas Data?

 

James
Community Champion

@jerry_nguyen 

Nice to know about the undocumented API call. I don't have that feature enabled, so I was watching what happened on the individual grade page and it wrote to the local storage. There were no network calls at all made there, which is why I started looking at local storage. I did not take the time to go into beta, enable the feature, and then check ... mostly because I think it's a bad approach.

Regarding the scanning for submissions ...

Live Events might make it faster, especially if you set up a queue that only handled submissions. You still need a fall-back to the API, though, since the live events are a best-effort attempt and might not get delivered. I have an LTI that students can click to say "I'm all done" that is immediate, but students don't always click that.

Live Events weren't around when I developed the code. I use a cron job to run a script on a Linux server that uses the List submissions for multiple assignments endpoint. It has graded_since and submitted_since parameters that can limit the information to just those since the last scan. This can speed up the query, especially for large courses. For us, we are using it in an orientation course where students are removed after completion and so I use the workflow_state=submitted parameter and don't mess with graded_since or submitted_since.

I could use GraphQL, but that came along to Canvas after I had written the other and what I wrote is still working so I didn't see a need to change it. It would be faster in my case, but as many things with Canvas and GraphQL, they don't allow filtering, so I would have to download the the submissions for the 2170 students in it -- so maybe it's not faster with GraphQL 🙂

I've got other scripts that automate things grades. I've got an H5P installation running on WordPress. I couldn't get the LTI from Canvas to work, but WordPress is under the same CAS login that Canvas is, so I grabbed the user's login ID that way. H5P writes the events to a MongoDB XAPI data store. I query it once an hour, looking for students who have done the activity in H5P. Then I update their grade in Canvas based on that. I know H5P has a hosted solution, but it's cost prohibitive since I'm the only user and it's for about 6 assignments a semester.

jerry_nguyen
Community Contributor

@James 

You can use that gradebook_settings endpoint for all sorts of things. However, it's only applicable to the person who made the call

gradebook_settings.JPG

I didn't know that Live Events are only best-effort attempts. Do you use it at your institution? if so what is the main purpose? 

With that amount of API calls you're making, have you ever reached the limit as mentioned here: https://canvas.instructure.com/doc/api/file.throttling.html ?

I like to use GraphQL as it can greatly reduce the number of API calls. I use it to generate a full grading todo list for our trainer with one single query. However, as you said, Canvas GraphQL filtering ability is so limited. Having to write a loop to filter and simplify the result can be painful because of the way they structure the JSON result 🙄

James
Community Champion

@jerry_nguyen 

If you have an admin token, or someone with the right privileges, you can masquerade as that user and force it. That introduces the question whether you keep forcing people to do it the way that you want it done or run it once (and whenever a new account is created) and then let them change it if they want to. If you keep forcing it, the end-user will eventually give up -- or come to the Community to complain that they change it but it never sticks and ask if there's a way to make it remember their choice.

Some information is only available through the API, some only through GraphQL, some only through Canvas Data, and some only through Live Events. Obviously, there is some overlap as well.

Of those, only Live Events are event driven and provide near-real-time, but they still requiring polling the Amazon SQS (you can set it up to send to your website instead of SQS) and those aren't immediate because of the way the SQS works. The documentation says that they will not attempt to resend an event if it does not make it through the first time: "If service is unavailable and Canvas cannot reach it, the events will not be delivered for the duration of the service outage."

The Event Data Identifiers page gives some additional warnings: "Event data is not aggregated over time; therefore, it should not be used as a single source for your data warehouse. Use it in conjunction with Canvas Data Extracts and Canvas APIs to ensure the integrity of your LMS analytics dataset. ... Events are delivered in a "best-effort" fashion. In order to not slow down web requests, events are sent asynchronously. That means that there's a window where an event may happen, but the process responsible for sending the request to the queue is not able to queue it. ... In event of endpoint downtime, all events will be lost."

The introduction to Live Events says "Live Events are well suited for analytics and data collection applications, but should not be used for applications that need their data immediately and as up-to-date as possible. If you find your application needs the most up-to-date information possible, you should use the regular Canvas API instead of Live Events." That page gives some examples of questions that might be answered using Live Events. One example is knowing when the content of a page changes. If you were running your own index service, this could give you a much faster update than overnight processing and downloading all of the content pages.

I set Live Events up for my uses in Fall 2020 to track student activity. Every time a student does something (view a page, attempt a quiz, etc), I have an event queue written. I was primarily using it to gather information for my stats class. I wrote that script to download the access report data for an entire class since it's not available through the API and comes only one student at a time ... but the information is mostly available through Live Events (and no where else unless you want to go through the requests table and try to figure it out). It's also nice because it gives the User Agent string from the browser, so I can tell who is using what device and if they're using an outdated browser that will likely have problems with other things. Again, you could use the requests table, but it's huge and difficult to work with -- and Canvas Data is stale before you get it.

The Amazon SQS is fairly cheap, but we're a small school. I spend under $2 a month for the queue (usually about $1).

Robert Carroll ( @robotcars ) has written a lot about Live Events in the Community. One use he had was to develop a solution to update Canvas Data so that it had live information rather than the 24-36 hour delay you might see. His school district is huge and his sharing how little he paid in SQS fees finally encouraged me to dabble with it.

I prefer the REST API because it's easier for me to merge objects (without having to resort to LoDash or similar library). In the REST API, you get an array of just one time. In GraphQL, it's buried under data and consolidated, so if you're trying to normalize your tables according to data structure, GraphQL doesn't make it as easy.

I need to share my thoughts on this about the New Quizzes Public API: Rest or GraphQL thread, but I've not taken the time to put my thoughts together completely on it. I'm open to GraphQL, but it's harder to write, the documentation is poor, and filtering (one of the great upsides of it) sucks.

As for reaching the API limit -- yes, I have reached that. I started using the Bottleneck JS library to keep from doing so. My intragroup peer reviews had that issue, the resetting of missing or late status also hit the threshold. My access report script should probably use it, but I'm not sure if I added it there or not, yet. My script that downloads all of the essentials each night for our early alert system uses it heavily. Because I parallelize all the requests that I can (something that cannot be done as easily with GraphQL), I had to go through and set different timings for different types of requests. The key is the number of concurrent requests and the minimum time between requests. I may allow 30 concurrent requests, but make sure there is 50ms between them so I don't hit the x-rate-limit-remaining threshold. It varies widely depending on the API call (enrollments are really expensive). That initial limit of 700 is even worse on Canvas Studio. Studio starts at 70 and anything above 1 call per second will eventually trigger it. Unfortunately, the Bottleneck library doesn't allow you to dynamically change the timings once the items are in the queue so I cannot react to the remaining amount, only set it before I make the calls.

imerricks
Community Participant

My institution also has a critical need to have "View ungraded as O" selected by default. In our competency-based course, a student must have completed each written assignment to be eligible to take the final assessment. We want to make it as easy as possible for teachers to confirm all assignments have completed, but the default view showing 100% completion for students who have yet to submit assignments is very misleading.

colleen
Community Participant

If you look further up, jerry_nguyen has a solution. If you don't understand it, contact him directly and he usually can help directly. He solved our problem.

ProductPanda
Instructure
Instructure
Status changed to: Archived
Comments from Instructure

As part of the new Ideas & Themes process, all ideas in Idea Conversations were reviewed by the Product Team. Any Idea that was associated with an identified theme was moved to the new Idea & Themes space. Any Idea that was not part of the move is being marked as Archived. This will preserve the history of the conversations while also letting Community members know that Instructure will not explore the request at this time.