How to use the REST API to get students from a course?

[DOVKRUGER]

I wrote some quick perl and python code to get the list of courses from canvas.

I can get user data for a single course like this:

https://canvas.instructure.com/doc/api/v1/courses/$course/users?per_page=250

where $course is my course number in perl.

However, the max number the API will support appears to be 100. So I need to get the first 100 and link to get the next group. The only docs I can find on paging:

https://canvas.instructure.com/doc/api/file.pagination.html

seems to be more for interactive documents. I don't see anything in the JSON coming back that indicates how to get the next bunch. Any help?

[chriscas · Accepted solution]

Hi @DOVKRUGER,

The pagination info will not be in the JSON, but rather the html response header.  In python, if you use the request library, you can use the .link property of the response to get the current and next links Canvas sends.  I could send you a function I've made to get all pages of a Canvas API request if you have interest and are using python.  If this is part of your perl code, perhaps someone with experience in that language will come in and give a suggestion.

-Chris

[DOVKRUGER]

Please post your python code. I got the perl code working, found the link, parsed and got the next link. But I'm considering switching to python anyway, and I want to see the logic of how to check for the end.
It's really awkward, the way I stop is if the next is the same as the current link then you are done, but since they don't publish a decent example, it's difficult to know whether that is the correct logic or whether there is some weird case that would create problems.

I haven't done REST programming, but this seems like really poor design, and worse documentation. If you are providing a REST API for something that is too big to do in a single go, then surely you return JSON that has the link, or a boolean flag with whether to continue or not. To store it in the header, force me to parse it out, and to check some condition to know whether there is more?

And the idea that if you send a malformed URL, instead of getting back JSON with an error code, you get a huge HTML file is also pretty bizarre.

So instructure should clean up their mess, and yes, I would really appreciate an example, in any language.

[chriscas]

Here is my python function.  There are probably better (more "pythonic" ways to approach this, but as most python projects are really more side-projects for me, I can't afford to spend a lot of time on it).

# Paginated API call to Canvas
# Returns a tuple including list of all items from all pages of a paginated API call and the http status code and reason returned from the call(s)
# Last updated 2023-12-15
def canvas_get_allpages(url, headers):
    if not 'per_page=' in url:
        if '?' in url:
            url=url+'&per_page=100'
        else:
            url=url+'?per_page=100'
    return_data=[]
    return_key=None
    repeat=True
    while repeat:  
        canvas_request=requestswithretry().get(url,headers=headers)
        if canvas_request.status_code!=200:
            return_data=None
            repeat=False
        else:
            canvas_request_responsedata=canvas_request.json()
            # if Canvas returned a single item dictionary instead of a list, try using the value of that dictionary as the list
            if type(canvas_request_responsedata) is dict:
                if len(canvas_request_responsedata.keys())==1:
                    return_key=list(canvas_request_responsedata.keys())[0]
                    canvas_request_responsedata=next(iter(canvas_request_responsedata.values()))
                else:
                    return_data=canvas_request_responsedata
                    repeat=False
            if type(canvas_request_responsedata) is list:
                # if a list was returned, add it to the existing list
                return_data=return_data+canvas_request_responsedata
                url=canvas_request.links.get('current',{}).get('url',url)
                last_url=canvas_request.links.get('last',{}).get('url','')
                # if not on the last page of results, set up retrieval of next page
                if (url != last_url) and ('next' in canvas_request.links.keys()):
                    url=canvas_request.links['next']['url']
                else:
                    repeat=False
    return_data_namedtuple = collections.namedtuple('return_data_namedtuple', ['data', 'reason', 'status_code'])
    return return_data_namedtuple(return_data if (return_key==None) else {return_key:return_data},str(url)+': '+str(canvas_request.reason),canvas_request.status_code)

[DOVKRUGER]

I would also appreciate some answers on finding some commands.

If you need me to post as a separate request, I can, but I can't find how to:

1. get a final grade for a user. Since I can manually download a spreadsheet, it's possible that this is the only way, but I suspect not. I searched around, but the API is quite big. It's very difficult to prove that something does not exist. For a grade for a particular assignment I suppose I am supposed to go to the assignment, get the list of all users who have completed it, and find the one with the user I want, or perhaps there is a call where I send the user id and it just gives me the one I want

Is there a call to give me grades for the columns that are not assignments, like categories (ie tests) and the final grade?
Can I get the final letter grade? I saw there is a way to get the grading scheme letter by letter, but I just want to get the grade for the student.

2. I'd like to set a grade using a put, and would just like an example

[melodyc_lam]

@DOVKRUGER 

Assuming you're a Canvas Admin for at least a subaccount --

You'll need to use the ""Get a course" endpoint and in the include[] parameter include current_grading_period_scores for the final grade.

Hope this helps somewhat. I know it doesn't answer all the questions, but I've never really needed to work with non-final scores.

[matthew_buckett]

It's a long time since I've done any perl, but if you're using HTTP::Client (https://metacpan.org/pod/HTTP::Client), then after doing a get() request, you should be able to use the response_headers (https://metacpan.org/pod/HTTP::Client#$client-%3Eresponse_headers) to get the Link URL values.

Some API endpoints still use the `&page=1` parameters which aren't guaranteed to work in the long run, but as a quick fix may allow you to check that your solution is going to work and you're getting the data back you expect. 

[dgrobani]

Here's the Python function I use to return a list from the API:

def get_list(url, r_data=None):
    """ compile a paginated list up to 100 at a time (instead of default 10) and return the entire list """
    r = get(f'{url}{"&" if "?" in url else "?"}per_page=100', r_data)
    paginated = r.json()
    while 'next' in r.links:
        r = get(r.links['next']['url'], r_data)
        paginated.extend(r.json())
    return paginated

[chriscas]

Yeah, I think this is about how mine started (with stylistic differences, the formatted strings in python just break my brain visually for whatever reason).   I found a few specific calls (I don't remember which ones) that were a little weird with the next urls, and there are some paginated calls where Canvas doesn't just return an array list for whatever reason.  Couple that with my code running as part of a few larger scripts that we don't want to "ungracefully" error out, and things explode a bit.  @DOVKRUGER, Depending what your long term goals are, this might be better to start with.

-Chris

[DOVKRUGER]

No, if you have more error checking, I want that. This is the biggest problem with the API: huge numbers of details left out, semantics of what happens when...

The best thing would be to design a better API. The second best is at least to present working code that navigates the gotchas. In any language, but commented so I can see what is being handled. And even if not commented, at least the code is showing me there's an error case.

I appreciate your time and answers

May I ask why this thing always asking for captchas? I log in. This seems unnecessarily paranoid.

[DOVKRUGER]

This looks very elegant, would you like to share a library such functions if you have one?

[dgrobani]

Thanks, Dov. I put my Python scripts on GitHub seven years ago and never updated them there, but they might offer some ideas as they are: dgrobani/py3-canvaslms-api: Python 3 API wrapper for Instructure's Canvas LMS with real-world exampl....

You should also definitely look at ucfopen/canvasapi: Python API wrapper for Instructure's Canvas LMS. Easily manage courses, users, gr... This library is much more extensive, robust, and maintained than mine.