SpeedGrader Should Allow Grading of Large Classes

(2)

TLDR: SpeedGrader will be rendered unusable when there are “too many” submissions (2500+). Canvas needs to implement a solution so that SpeedGrader can still be used in this case. The solution does not necessarily have to increase SpeedGrader’s capacity. An alternative would be to limit how many submissions SpeedGrader loads at once. Partitioning by the course’s sections is a possibility and the functionality is already there in SpeedGrader (in the screenshot). However, the option does not appear because the SpeedGrader UI does not load before the error is thrown. More suggestions are offered in the "Suggestions" section below.

Overview: Canvas recently added an error message that is visible to the instructor when SpeedGrader takes too long to load the submissions. This message was necessary because the browser eventually timeouts, ceasing any further loading. The error (in the screenshot) informs the client to try again or that there may be too many records to load in SpeedGrader. Although this error illuminates a bit more on the situation, Canvas does not give a viable solution or workaround, effectively rendering SpeedGrader useless for a large class.  Yes, it is possible that an instructor can download the submissions, manually grade each, and then input the grades back into Canvas.  However, for 2500+ students, this is not practical. We would really like to have some functionality to use SpeedGrader to grade large classes on Canvas.

Suggestions:

        Change the way SpeedGrader loads such that the UI still loads even if the submissions do not load.

        Add a setting to load a single section by default.

        Add a dropdown menu that can select which section (set to All by default) to load in SpeedGrader.

        The section that the Gradebook is set to filter to will be the inherited filter for SpeedGrader.

Steps to Reproduce:

  • *Prerequisite Canvas course must have 2500+ students*
  • Create an assignment (issue is most apparent in a file submission assignment).
  • Load SpeedGrader for the assignment. 

Expected: Provide a workaround when SpeedGrader fails to load after the browser times out (suggestions offered above) and describe the workaround in the displayed error message.

Actual: SpeedGrader is stuck on the “loading…” white screen and displays an error to try again. Despite trying again, the same outcome will occur and instructors will still be unable to grade using SpeedGrader.

20 Comments
Renee_Carney
Community Team
Community Team

canhelp Please take a look at the thread here, https://community.canvaslms.com/ideas/10875 

canhelp
Community Explorer

Upon further investigation, we discovered that graded discussion assignments do seem to load normally in SpeedGrader (though may take awhile to load). From our limited test cases, it seems like only quizzes and assignments with submissions are affected. However, take this information with a grain of salt because we were only take to test it in our single example course.

Renee_Carney
Community Team
Community Team

Thanks for returning to post more information canhelp 

Should I archive this thread?

canhelp
Community Explorer

We do not believe that this feature request should be archived just yet. In the other feature request https://community.canvaslms.com/ideas/10875" modifiedtitle="true" title="Increase SpeedGrader capaci..., you stated, "Eng identified some inefficient queries that  sometimes cause SpeedGrader to max out at submission numbers lower than the 2,500 threshold established this summer." However, we have a course that has almost 5000 enrollments. Will the hotfix resolve this SpeedGrader error in this case?

Renee_Carney
Community Team
Community Team

It will not.  It will stay at the threshold of 2500.  Let me do some additional checking to see if we plan to increase above 2500 entries.  I know there were some technical limitations in the past that dictated the 2500.

Renee_Carney
Community Team
Community Team

Great news.  We're going to open this idea for vote because we continue looking for ways to improve how canvas works for large enrollment courses and will continue to improve its performance with every opportunity we get.  So it's great to hear your feedback!

The limitation is still necessary, but we're striving to improve it in the future.

There is a workaround in the How do I use SpeedGrader? guide, that will help with the current status.  You might give sections a try.

To improve SpeedGrader performance, large courses should be separated into sections, which allows SpeedGrader to display submissions for a specific section and decrease the overall loading time for an assignment's data.

canhelp
Community Explorer

That is awesome news! We appreciate the reconsideration of this feature request. 

Also, we are aware of that workaround. However, it does not seem to apply in our case, which was the premise of this feature request. Our issue was predicated from the fact that you cannot use that the specific section filter in SpeedGrader since the UI does not load before the browser timeouts. This is displayed in the screenshots as well.

rake_9
Community Champion

We are trying a work-around of having the instructor create a separate assignment for each section.  This is fairly awkward and annoying, but we're hoping it actually works.

davwoods
Community Novice

The suggestion to " Add a setting to load a single section by default." should would be improved by saying that the setting should be at the course level. Most of our cross-listed courses are under 2500 students, but there is a significant number of them that are above 2500 students, some by a lot. In the under-2500-student courses, they could continue to see all the students in all the sections at one time, without a problem (except maybe speed), while the over-2500-student courses could choose the single section option in order to actually use speedgrader for their course.

yosha_vandaele
Community Novice

I would like to have a search box added to the speedgrader drop-down, this would make it easier to quickly find the submissions you require in a long list.

jayde_colquhoun
Instructure
Instructure

We have this same problem in our institution. Large courses are expected here due to common core offerings, and our integration into Canvas will mean there are some sections with over 1000 students. 

 

A few of these courses reach the 'threshold' in Speedgrader (though often it doesn't even reach the advised threshold and fails to load at much lower numbers than documented - despite the fix mentioned in 2018). We have generally been advised from support it is either due to student numbers or other times it is due to submission numbers. 

 

The workarounds offered have failed for us also. We have filtered the gradebook by section into student numbers lower than 500, however, it still fails to load or the delay is exceptionally long. We have also downloaded all assignments, however, this download also often fails, and when it does download, it creates greater stress for staff needing to then sort through sometimes 1,000+ plus files named subjectively to find their students. 

 

Please consider that large courses are a reality in larger institutions and we need these fantastic tools offered to be able to support our staff. 

colin_lowe
Community Explorer

Hi Jayde, we too are facing the same issue, we have many classes with 1,000+ students and five with close to 2,000 students.  The work around we have put into place is to break the students up into tutorial/marker based sections (usually less than 100 per section) then you put each tutor into their corresponding section AND go to the tutors profile from the People menu and set their privileges to 'limit this user to only see fellow section users'

Speedgrader will then load very quickly for the tutors, the issue is the course coordinator wants to check a tutors marking can't because Speedgrader won't load for them.  We had to create one or two more login's for the coordinator and break the class up into two or three sections for the coordinator who now has to log in up to three times to review tutors grading.

I don't consider our request to have this issue resolved an improvement request, the product is not working and this should be considered a bug and addressed.  We certainly never faced this issue with our last LMS.

jayde_colquhoun
Instructure
Instructure

Thanks, Colin - what a workaround (multiple logins!) - we are going to look at this and see if we can sustain a similar solution for our common core courses - initially we weren't filtering our sections small enough. Our affected course coordinators and course managers have come up with this workaround which is getting them through the thick of it at the moment - but is not sustainable and disappointing given the loss of 'speed' that speedgrader could provide:

  • Open Gradebook (New Gradebook enabled)
  • Filter by tutorial section (these are small sections of a few hundred students)
  • Right click on student’s name and open in new tab
  • This will bring up the grades/assignment for this student
  • Then select the assignment, select rubric, mark and enter grade and comment
  • Repeat for each student
gemma_sinclair
Community Participant

We had a case that a course with 800 submissions was not able to load in speedgrader with error: 

"Something went wrong. Please try refreshing the page. If the problem persists, there may be too many records on "First Contact - Your Future" to load SpeedGrader."
The workaround was accessing via sections only but we would like an interim solution as well as a long term solution for this issue. We are using both the old and new integrations with TurnItIn.
mvandevelde
Community Contributor

I’d like to add to this idea.

Problem:

Another issue with loading all this data at once is that the system is unaware if two teachers looking at the same assignment column and that might want to grade the same student assignment.

There is a recommendation by Canvas in the documentation to use sections to prevent this. But this only works if you want to work in this way with sections and mark assignments separately between sections.

This is the same case with Rubrics, if each teacher is responsible for different parts of a rubric, it’s easy for them to overwrite each other’s entry.

Solution:

Warn teachers with a simple message, as they go to mark an assignment in the SpeedGrader if another teacher is already viewing the same student’s assignment.

---

I’ve suggested a warning message rather that a lockout/read-only mechanism, as this would likely lead to more issues.

Mark.

Stef_retired
Instructure Alumni
Instructure Alumni

mvandevelde‌, rather than bundling this new request into an existing idea, we encourage you to add your vote and use case description to this separate idea open for voting: https://community.canvaslms.com/ideas/14247-notify-graders-when-multiple-graders-is-present-in-speed... 

mvandevelde
Community Contributor

Thanks stefaniesanders,

That’s a good point, I’ve added my vote to Notify Graders When Multiple Graders is Present in Speedgrader

I also think it’s very unlikely the warning idea on its own is enough. It would only be possible if Canvas changes how it loads data to accommodate such an idea. That’s why I’ve mentioned it here.

warren_goetzel
Community Participant

There was an update on the R1 call today.  Now in Beta.....

 
Mikee
Community Participant

Astonished that this is still live and very much a cause of frustration for our leading academics who have significant numbers of enrolments in their subjects.

 

This absolutely belies the capability of canvas to be used in any large institution.

KristinL
Community Team
Community Team
Status changed to: Archived

Thank you for sharing this idea with the Instructure Community!

The Product Team reviewed all feature proposals recently, and unfortunately, this thread was identified as one that they would not be able to include in their current or future plans. While we appreciate your proposal, we also want to be transparent about the likelihood of something like this making it to production.

Thank you for collaborating, and we hope that you submit another idea in the future!