API Next bookmark returning no results

[ColinOBrien]

Hello,

I've put together a script to draw long lists of student page view logs. Its using the pagination rel values as suggested and this was previously working, however I tried it recently and it will start with the first page of results correctly, however the next page bookmark link always returns no values even through there are values there.

I thought I might not be taking out the bookmark link correctly, so I opened the link in a browser just to see that the link returns values,  but it doesn't, just an empty array. I could understand if the array was returning but not populating in my end result, but I'm not sure what's happening here. Has this happened to anyone else before?

[ColinOBrien · Accepted solution]

Thanks Chris, I think I'm using the REST API document correctly. 

The way I currently have it working is that it'll make the API call and correctly return the first 10 page view results. From the response headers, I cut out the text for the Next url.

I start a loop that makes a get url call with the next result, but it just returns an empty array. 

Things get weirder though, so the first call lists what would be expected in terms of response headers, so First, Next, Current. But I figured I'd try to at least see the response headers from the second calls empty array, but it only lists First, Current and Last, no next is listed which given the results makes sense I guess. If its an empty array those headers seem right, but there is thousand of pages to be listed for the user that should be getting listed. 

I dont think this has anything to do with the API in of itself since no-one else seems to be having the issue. Thought it might be a caching issue of some sort since for my test user its the same next page bookmark link returned but tried a few new users with the same results. Its just weird behaviour thats blocking me and I cant figure out what exactly is happening!

[chriscas]

Hi @ColinOBrien,

Are you using the  User Page Views API, as described at Users - Canvas LMS REST API Documentation?  I have been working with that API pretty extensively over the past few days, and have not encountered the issue you mention.  I think web browsers might filter that link info out from display even in the developer tools, so I wouldn't necessarily rely on that as confirmation of what's going on...

Are you able to tell us what programming language you made your script in, and perhaps post the entire thing, or at least a snippet where you're handling the pagination links so someone could ensure your methodology looks correct?

Look forward to hearing back from you!

-Chris

[ColinOBrien · Accepted solution]

Thanks Chris, I think I'm using the REST API document correctly. 

The way I currently have it working is that it'll make the API call and correctly return the first 10 page view results. From the response headers, I cut out the text for the Next url.

I start a loop that makes a get url call with the next result, but it just returns an empty array. 

Things get weirder though, so the first call lists what would be expected in terms of response headers, so First, Next, Current. But I figured I'd try to at least see the response headers from the second calls empty array, but it only lists First, Current and Last, no next is listed which given the results makes sense I guess. If its an empty array those headers seem right, but there is thousand of pages to be listed for the user that should be getting listed. 

I dont think this has anything to do with the API in of itself since no-one else seems to be having the issue. Thought it might be a caching issue of some sort since for my test user its the same next page bookmark link returned but tried a few new users with the same results. Its just weird behaviour thats blocking me and I cant figure out what exactly is happening!

[chriscas · Accepted solution]

Hi @ColinOBrien,

What scripting language are you using?  Someone here could probably share some working code for pagination to help you out if we knew what language you were working in.

Your general approach seems correct, so I'll throw out a couple details that could be impacting what you're doing.  First, when you're using the pagination next links, you don't need to include any additional parameters as JSON, form data, etc, as the next link will have all of your parameters URL-encoded as part of the link.  Second, when you're parsing for the next link, make sure you're getting the whole thing including the entire querystring, as that is where all your parameters and the pagination info is included.

For debugging purposes, you could probably print the next url your script is generating and examine it to make sure the format looks correct (for this specific API, you should see at least your start_time, end_time, page paramaters in the querystring, and the base API url should stay the same).  As an example, you should see something like this progression:

First call url:
https://institution.instructure.com/api/v1/users/zzzzz/page_views?end_time=2025-03-22T03:59:59Z&start_time=2025-03-21T04:00:00Z&per_page=100 

Next call url:
https://institution.instructure.com/api/v1/users/zzzzz/page_views?end_time=2025-03-22T03%3A59%3A59Z&start_time=2025-03-21T04%3A00%3A00Z&page=bookmark:bunchofrandomcharactershere&per_page=100 

Next call url:
https://institution.instructure.com/api/v1/users/zzzzz/page_views?end_time=2025-03-22T03%3A59%3A59Z&start_time=2025-03-21T04%3A00%3A00Z&page=bookmark:bunchofdifferentrandomcharactershere&per_page=100  

Now as I was investigating the flow of this exact API call in my working scripts to create this post, I noticed a bit of an anomaly.  Fro some reason, Canvas appears to be repeating the start_time and end_time parameters for the next link, so you actually end up with:

Next call url:
https://institution.instructure.com/api/v1/users/zzzzz/page_views?end_time=2025-03-22T03%3A59%3A59Z&start_time=2025-03-21T04%3A00%3A00Z&end_time=2025-03-22T03%3A59%3A59Z&start_time=2025-03-21T04%3A00%3A00Z&page=bookmark:bunchofrandomcharactershere&per_page=100  

That repetition hasn't affected the function I created to get all pages of a Canvas API result, but I thought it was interesting because I don't think the parameters really *should* be getting repeated in the URL. Depending on your exact code, maybe it's throwing something off though.

Let us know if you're able to share your programming language or any of your existing code at all.

-Chris 

[ColinOBrien]

Thanks again Chris, ok so here's the URL headers, the followings sanatised but you'll get the idea,

First URL call:     
https://example.instructure.com/api/v1/users/123/page_views?per_page=50

Response headers returned: 
<https://example.instructure.com/api/v1/users/123/page_views?page=first&per_page=50>; rel="current",<https://example.instructure.com/api/v1/users/123/page_views?page=bookmark:-random characters-&per_page=50>; rel="next",<https://example.instructure.com/api/v1/users/123/page_views?page=first&per_page=50>; rel="first"

URL passed through for next loop:
https://example.instructure.com/api/v1/users/123/page_views?page=bookmark:-random characters-&per_page=50

Loop 1 response headers returned
<https://example.instructure.com/api/v1/users/145/page_views?page=bookmark:-random characters-&per_page=50>; rel="current",<https://example.instructure.com/api/v1/users/145/page_views?page=first&per_page=50>; rel="first",<https://example.instructure.com/api/v1/users/145/page_views?page=bookmark:-random characters-&per_page=50>; rel="last"

In my code I'm separating out the next url by slicing out the text from the from the response headers, it takes a few steps but it does list the URL as needed. But I don't understand why loop 1 doesn't return a next header, as a result the program basically keeps repeating the same first page of information over again.

I've checked dozens of times that its passing through the initial next url and it does, but the url just returns nothing!

[ColinOBrien]

Thanks again @chriscas , I've completely rebuilt the piece of code that pulls the next bookmark from the response headers to double confirm its pulling the next link correctly. This is what's happening, (I removed the https:// from the start of the links as I think it making them links is preventing me from posting this previously)

Initial call
example.instructure.com/api/v1/users/123/page_views?per_page=50

First response headers
<example.instructure.com/api/v1/users/123/page_views?page=first&per_page=50>; rel="current",
<example.instructure.com/api/v1/users/123/page_views?page=bookmark: - random characters - &per_page=50>; rel="next",
<https://example.instructure.com/api/v1/users/123/page_views?page=first&per_page=50>; rel="first"

first generated call
example.instructure.com/api/v1/users/123/page_views?page=bookmark: - random characters - &per_page=50

response headers
<example.instructure.com/api/v1/users/123/page_views?page=bookmark: - random characters - &per_page=50>; rel="current",
<example.instructure.com/api/v1/users/123/page_views?page=first&per_page=50>; rel="first",
<example.instructure.com/api/v1/users/123/page_views?page=bookmark: - random characters - &per_page=50>; rel="last"

The initial next page doesn't lead anywhere, it just goes to an empty array without a next page listed in its response headers.