API for getting custom data for a given user in a course

[maguire]

Today there is an API to store custom data for a given user:

PUT /v1/courses/{course_id}/custom_gradebook_columns/{id}/data/{user_id}

However, there is a need for the equivalent GET/v1/courses/{course_id}/custom_gradebook_columns/{id}/data/{user_id}

Background at https://community.canvaslms.com/ideas/10952-api-needed-for-getting-custom-data-for-a-given-user-in-a... 

Is it possible to have such an API or is there another way to get the data (without having to get the entire column)?

[stuart_ryan]

Hi  @maguire ,

I have done some sleuthing and indeed you are correct. I was hoping there might be another way through the gradebook history APIs, but alas there is not.

Upvoted for this!

Cheers,
Stuart

[maguire]

Hi  @stuart_ryan ‌,

Speaking of history, it seems that there is no API to access the history of custom columns in the gradebook. This seems like another shortcoming (especially as I have a lot of use of these entries in several courses with a large number of teachers).

Thanks for your earlier upvote.

[stuart_ryan]

No worries at all  @maguire , we are looking at Gradebook and the APIs at the moment, I am starting to formulate some ideas in my mind about some additional APIs that would assist (not necessarily related to history specifically).

For example, the ability to extract an entire Gradebook 'row' for a single student in a single course, by ID, passing a date and time to give a point in time snapshot.

The ability to pull a Gradebook (basically what you would see on screen) for a course, in JSON format, again at a point in time. I have yet to find an easy way to do this without looping through submissions and so on.

It might be good to collaborate on some ideas of how things could be enhanced and come up with a series of related feature requests?

Cheers,
Stuart

[James]

I wonder if more requests like this will be met by the GraphQL API rather than the REST API? I only heard of GraphQL when this thread appeared in the Community: Experimental GraphQL API?  but a quick look at it sounds like it holds a lot of potential.

[stuart_ryan]

OOOH thank you for that  @James , I had not seen that before! That is really interesting to note they are playing around with it. Might be a good time for us to get our wish list in

[maguire]

Yes, I am open to this. Perhaps this is best done via e-mail until we have something to report. Using GraphQL (Thanks  jjones for the pointer to this) might be a nice way to access a row of the gradebook - as one could specify the student's user_id and just the columns one wanted, and as you say even specify a time point (so that one can examine changes over time).

Because there is no functionality for getting e-mail about student submissions at the session level, instructors have to regularly look at their subset of students in the degree project course (with >340 students and >30 teachers) or ask the students to send them e-mail when the submit something new. I would like to be able to provide the missing functionality.

Speaking of pulling the gradebook's contents as you see it, has anyone else encountered a problem with the web view of the gradebook not matching the results when calling the APIs to get the contents? For example, of 9 students that I am working with (who show up in the gradebook with me listed as their examiner in a custom field) only 5 are found when I get the information using the APIs. The custom field seems to be empty for these students according to the API. [A separate implement done by the central IT unit of the university also gets the same results as I do.]

[James]

 @maguire ,

I haven't done any serious work with the custom gradebook columns (just testing things for people in the Community). Since they have to be added through the API, that means that no one else at our school has either, so I don't have any data to debug the situation with; sorry.

You're pretty on top of things, so I doubt either of these is the problem, but I'll mention them because I don't have anything else to offer.

For clarification, are you saying that the 5 that were found did not have any data in them? The documentation says that it only returns students who have associated data. Is it possible that you got the wrong column ID? Another question would be if you added the include_hidden query parameter? In getting both the list of columns and the data, it defaults to not showing hidden columns. Beyond that, I'm not going to be able to help out on this one.

Finally, you tagged the wrong James Jones. A trick I've learned when there are common names is to right click on the person's name from the header of a previous post and copy the link. Then paste it into the message and Jive will convert it to the person's name.

[maguire]

James,

I am sorry for tagging the wrong  @James  !

No, the students that were found and the students that are missing both have data in the custom column (in fact, in the case of the 9 - these students have the same identical string as the value), the problem is that one set of students have this data in the custom column when I query it with GET /api/v1/courses/:course_id/custom_gradebook_columns/:id/data

but the others do not (i.e., these student's user_id and the value that should be stored) are not returned. I've got the correct column ID (19) (as this data is being returned for other students).I get 13 pages of results returned for this column with a total of 125 results, but it seems that they do not contain all of the data. Curiously, I have now found a case for another teacher where the value of another column for the student does appear, but not the value for the examiner column (19).

[James · Accepted solution]

Have you tried specifying the per_page query parameter? It sounds like you're using the default of 10. Normally you can go up to 100.

I'm note sure if this the same thing, but it reminds me of a problem a couple of years ago where the correct number of results were being returned but some were duplicated while others were omitted. Changing the per_page made a difference.  I think the underlying issue was no sort order being specified and the PostgreSQL order is not deterministic when that doesn't happen.

[maguire]

In an experiment with per_page set to 100, I get 125 responses, just as I did before. However, they are a different 125!

Computing the intersection of the two reveals that there are 38 that are not in the per_page set to 10 that are in the per_page set to 100. I'm not sure why the DB query should produce duplicates, but it seems to.

[James]

Here is the discussion that I was talking about: https://community.canvaslms.com/thread/1557 I mentioned it a couple of other places in the Community. I think the ultimate feedback from Canvas was to open a support ticket.

There have also been some issues with pagination and cross-shard tables. Not completely sure what that means or if it applies, but thought I would mention it.

When Canvas loads data, it seems they often use a per_page of either 33 or 50 at a time. That cuts down on the number of requests (over 10 at a time) while letting them return quickly enough to be acted on right away. The per_page=100 can be a little slower.

[maguire]

Yes, this appears to be the same type of problem that you referred to in 2015. I too looked at the github Canvas source code, but did not find where the query is actually taking place. Too bad that it still has not been fixed.