cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Surveyor

Am I mistaken or is pagination in the "List user page views" API broken?

I have been playing around with the "List user page views" API endpoint and pagination appears to be broken...or I'm an idiot.  Either possibility is equally likely.

Normally, when you go and get a page of data you will get a "next" in the header as explained here:
Pagination - Canvas LMS REST API Documentation 

I've written lots of API calls and this always works fine.  HOWEVER, when I try to bring back page views it *ALWAYS* shows that there is a "next" page even when there is not.  Obviously this is a problem because my code keeps going back for more data.

I would love some feedback.  I'm hoping I'm just missing something, but I'm afraid that this API endpoint just does not work like all of the others I've used. 

Labels (1)
19 Replies
Highlighted

If this is your literal Link header, then it does appear that something is getting stripped out.

"Link": "; rel=\"current\",; rel=\"next\",; rel=\"first\"",

My Link headers look something like this:

Link: <https://richland.instructure.com/api/v1/courses?page=1&per_page=10>; rel="current",<https://richland.instructure.com/api/v1/courses?page=2&per_page=10>; rel="next",<https://richland.instructure.com/api/v1/courses?page=1&per_page=10>; rel="first",<https://richland.instructure.com/api/v1/courses?page=5&per_page=10>; rel="last"

I hope this isn't just another Jive stripping out things when you post them in the community but it's really there.

Highlighted

Ha!  Thanks for questioning me on this.  I was writing the contents of the header to the browser screen and that was not displaying.  I dumped it to a text file and it *IS* coming back. I've just always looked for "next" to be there and I go and grab the next page if it is.  In this case, "next" always seems to be there but it doesn't work correctly.

Anyway, this is what is coming back in the header:

rel="next",<https://zzzzzzzzzz.instructure.com/api/v1/users/XXXXXXXXX/page_views?end_time=2018-09-04&start_time=...>

But this looks REALLY strange to me.  Why are there TWO end_time and start_time parameters in this "next" link?  This is really confusing.  My original URL that I sent looked like this:

https://zzzzzzzzzzzz.instructure.com/api/v1/users/XXXXXXX/page_views?start_time=2018-07-01&end_time=... 

Highlighted

Note that that isn't the next link.  The next link is the one before the ;rel="next".

The date repetition does look odd, but it doesn't seem to affect the pagination working.  I've tried with three different users and the next link disappears at the end.

Highlighted

I've seen that duplication myself and consider it a bug in Canvas code, but Peter is right about it shouldn't be affecting things. He's also right that you want the one before the "rel". It's a comma separated list of links and the semicolon separates the link from the relationship.

You can split on the comma and iterate through array. This is what I do with my PHP code, but it's not absolutely necessary to split it. Here's a fairly lazy check for a "next" link in PHP. It assumes that the link headers are in $link. If it succeeds, then $matches[1] will contain the URL.

preg_match( '/<(.*?)>; rel="next"/', $link, $matches )

There's a lengthy discussion on the Canvas Developer's group about Handling Pagination that has contributions from several people and code examples in multiple languages.

Highlighted

I see--thanks for the clarification. Since I have not actually used those URLs coming back in the header before (I always use the page=#, incrementing the number myself) I'll need to play around with it a bit.  But you guys have put me on the right track.  Many thanks, pklove‌ and james@richland.edu‌.

Highlighted
Surveyor

I just wanted to thank everyone again for the help.  Once I started dumping the header text to a text file instead of trying to print to the browser screen for debugging I was able to see the links being passed back and everything began to make much more sense.  I was able to get things working this morning.  You guys really saved me a ton of time by pointing me in the right direction!

Highlighted

david_taylor  james@richland.edu I've seen the duplicate parameters on the when I resend the parameters on the returned next href. So instead I hard code the parameters into the initial request_url instead of in the request body. Below is an example of a request in Ruby.

request_url = "#{canvas_url}#{api_endpoint}" #include parameters for API call in url
count = 0
more_data = true
while more_data
    request = Typhoeus::Request.new(
        request_url,    #we need a variable here because we need the api url to change
        method: :get,
        headers: { authorization: "Bearer #{canvas_token}" }
        )

    request.on_complete do |response|
        #get next link
            links = LinkHeader.parse(response.headers['link']).links
            next_link = links.find { |link| link['rel'] == 'next' }
            request_url = next_link.href if next_link
            if next_link && "#{response.body}" != "[]"
                more_data = true
            else
                more_data = false
            end
        #ends next link code

        if response.code == 200
            data = JSON.parse(response.body)
            #do stuff here
        else
            puts "Something went wrong! Response code was #{response.code}"
        end
    end

    request.run
end
puts "Script done running"‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Hope that helps!

Highlighted

That's  not the case here.  Making the call with the request parameters as a query string does result in them being duplicated.

BTW, as the page views endpoint call is a GET, I think you would expect the request parameters to be passed as a query string and not in the body anyway.

Highlighted

David - Maybe give Insomnia a whirl...it's very useful for testing out calls before writing a script: https://insomnia.rest/