cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Community Member

prevent making thousands of calls to the API from web page? Is there a better way?

Jump to solution

I want to display the analytics for observees to the observers. On the observers page >

```

/profile/observees/

 ```  in order to do so the api calls that I would need to call are:

```

//api/v1/users/self/ --to get the user id
/api/v1/users/:user_id/observees/ - -using the initial api call to substitute for user id to get an array of the ids
//api/v1/users/:user_id/page_views -- pass in each id history to display to parents (repeat api call for each student)

```

I am using the fetch api to get the information:

```

const apiTest = () => {
  const url1 = "/api/v1/";
  const options = {
    'credentials''same-origin',
    'headers': {
      'accept''application/json'
    },
    'timeout'5000
  };
  fetch("/api/v1/users/self/"options)
  beep boop
        })
        ).then(res => {
          blah blah

        }).etcetera);
}
apiTest();

```

and then innerHTML to display information fetched.

Is there a better approach to this?

I feel like I am going to breeze through our API cap or something.

1 Solution

Accepted Solutions
Highlighted
Learner II

You seem enthusiastic about trying to solve this so I'm going to lay out what I can and just state, I'm always open to collaborate, especially with like-minded K12 folks.

Years ago when I started looking to provide tools for our district, which is not small, the amount of endpoints and code and limited staff (me) necessary to bring in the data we wanted to use was just overwhelming. Then Canvas Data came out it solved much of our reporting and data needs, and we focused on using the API to send data to Canvas and manage background tasks. One of our purposes for CD was an LTI for teachers to discern between submissions/activity and just course access, which is page views or Requests, so I get the need for showing access vs course progress stats.

Then, Live Events was released and we no longer have a delay in data, and can use Canvas Data or Live Events, or combine them based on purpose. We currently use LE to provide a 1 page attendance screen of last submission by attendance week, number of submissions for that attendance week, and most recent activity, with a count of students current online for the course. This gives teachers a much faster way to do SIS attendance than bouncing around Canvas to collect that info on each student, and we're eventually going to use this data to pass a boolean course attendance to the SIS.

Both of those solutions require some technical architecture and deployment to ingest the data, and a secure web server to host the LTI. If you're interested in that, we can go down that rabbit hole.

However, an LTI for your purposes could be much simpler start using a web server and server side LTI as james@richland.edu suggests. The Awesome CanvasLMS has tons of open source resources, including a single page Node LTI examples from pklove‌ that might be a decent starting point. With that, you could use a developer token to collect page views when the user accesses the LTI and provide the information you want to the parent, or other endpoints only available through admin privileges.

Additionally, I tried to look into what API and GraphQL endpoints are available for observers but was directed to look at the permissions by role in Canvas, this revealed some really absurd options.

Analytics - Page Views is not available, cannot be enabled for an Observer. Parent should have access to this. Alternatively, a parent should have a view of /users/:kid profile page by default, which would provide page view list.

Really absurd, permissions page has the option to allow an observer to:

Add/Edit/Delete:

  • Course Pages
  • Learning Outcomes
  • Course Files
  • Course Calendar events
  • Course Content*

But for some odd reason, we can't let them Manage the Course State, and Add/Edit or Delete course sections.

We can give them the option to Import Learning Outcomes, though. /sarcasm :thumbsdown

Most of this, I'd hope isn't actually possible through the views/routes, but it shows some not-thought-out-to-well purpose of the Permissions page. Some testing will be done and a conversation with our CSM.

edit: nope!

In test instance, I allowed observers to set course content. I deleted assignments, modules, and edited the syllabus... but can't set page_view rights to their kids.

Smiley Sad

View solution in original post

16 Replies
Highlighted
Learner II

I don't think you'd be in danger of hitting the cap limits.

Can you use the ENV* variables to get the user id, or better yet, use self on endpoint #2

/api/v1/users/self/observees/

* console.log(ENV)

> ENV.current_user {id:}

> ENV.current_user_id

Tagging Canvas Developers‌ for additional visibility.

Highlighted
Community Member

ok, I can eliminate the first call completely using the self on the second, thanks.

but am I approaching this in the right way?

if a parent has 6 kids that is 7 api calls...

and OMG never new this ENV variable existed

where is documentation of this witchcraft?

0 Kudos
Highlighted
Learner II

OMG never new this ENV variable existed

where is documentation of this witchcraft?

I don't think there's documentation, not even really sure it's there for us/developers, I think we just found it and started using it?

The rate limiting question is hard to evaluate without more details, and honestly I've never worried about it, and I've done some pretty terrible things against production, but not as crazy as james@richland.edu and his concurrent data pulls. He definitely has more insight here than I do.

One thing to remember is that each user is using their own quota.

For more info read,

API Rate Limiting

Dynamic Throttling Implementation using rate limit header

Is there any limits on the number concurrent calls to canvas REST API?

If you are writing something for use within Canvas Themes those requests are limited as the user. Never use your own token in a theme hack. If you are using an LTI or something that uses a developer token, have the user issue the token, not your own.

if a parent has 6 kids that is 7 api calls...

More than that :smileygrin:

// when I first read that, I saw it as a student with 6 courses, leaving this bit for detail.

The user page_views endpoint is by user, not by course. If you want to split the data by course, you'll have to code it.

{
"url": ".../courses/1234/files/6789/download?wrap=1",
"context_type": "Course",
...
"links": {
"user": 4567,
"context": 1234,
...
},
...
}‍‍‍‍‍‍‍‍‍‍‍

// 6 kids is 6 requests + pagination

By default you will get 10 rows from the end point, with pagination you can get 100 rows at a time, collecting additional page views with additional requests.

Pagination - Canvas LMS REST API Documentation 

Handling Pagination 

but am I approaching this in the right way?

With the API you always have to start with what's available, but after that, the options are in your imagination and desired product.

Scope

How many users are you expecting, how many of your observers log in each week/month/term?

What are you trying to show the observers, what do they want to see?

Is it a theme hack or an LTI?

Do your observers use the Canvas Parent app, or do they use the desktop/browser?

User Experience

What does the user see when they get to your feature?

Will they get just a long list of page views? Can you load the most recent 100, and build in something like an infinite scroll that collects/shows more if they browse that far?

The following get considerably more complicated, because you have to do enough collection to gather the data and split by course or range...

Can you give them a date picker and let them set a range? Then break the results up by course?

Can the observer click into each course instead of seeing hundreds of page views per course on 1 page...

maybe they don't want to see every course all at once, but are checking into activity of just 1?

Since the endpoint has a date range param, maybe you could default them to the current week, and then give them a month filter or the ability to scroll back to see previous weeks?

Highlighted
Community Member

Mr. Carroll, just plain thank you. Your terrific insight and additional references that now have taken refuge in a new bookmark in my browser.

Is it a theme hack or an LTI?

Do your observers use the Canvas Parent app, or do they use the desktop/browser?

This is a theme hack I guess you could say.

the observers use both but it seems they tend to utilize the browser a bit more

User Experience

What does the user see when they get to your feature?

Will they get just a long list of page views? Can you load the most recent 100, and build in something like an infinite scroll that collects/shows more if they browse that far?

-Honestly I would just lock it down to the past 20 things that the student has done.

   this was just to give the parents a since of when or if they students where doing their work

-Is there something I am missing that I could pass in to share with parents that would give them a since to know when their kids are or if they are doing things(not just in one class but any classes)?

-If you could so happen to infinitely scroll through your wisdom to point me in the right direction for this little feature I would be ecstatically pleased.

0 Kudos
Highlighted
Learner II

You seem enthusiastic about trying to solve this so I'm going to lay out what I can and just state, I'm always open to collaborate, especially with like-minded K12 folks.

Years ago when I started looking to provide tools for our district, which is not small, the amount of endpoints and code and limited staff (me) necessary to bring in the data we wanted to use was just overwhelming. Then Canvas Data came out it solved much of our reporting and data needs, and we focused on using the API to send data to Canvas and manage background tasks. One of our purposes for CD was an LTI for teachers to discern between submissions/activity and just course access, which is page views or Requests, so I get the need for showing access vs course progress stats.

Then, Live Events was released and we no longer have a delay in data, and can use Canvas Data or Live Events, or combine them based on purpose. We currently use LE to provide a 1 page attendance screen of last submission by attendance week, number of submissions for that attendance week, and most recent activity, with a count of students current online for the course. This gives teachers a much faster way to do SIS attendance than bouncing around Canvas to collect that info on each student, and we're eventually going to use this data to pass a boolean course attendance to the SIS.

Both of those solutions require some technical architecture and deployment to ingest the data, and a secure web server to host the LTI. If you're interested in that, we can go down that rabbit hole.

However, an LTI for your purposes could be much simpler start using a web server and server side LTI as james@richland.edu suggests. The Awesome CanvasLMS has tons of open source resources, including a single page Node LTI examples from pklove‌ that might be a decent starting point. With that, you could use a developer token to collect page views when the user accesses the LTI and provide the information you want to the parent, or other endpoints only available through admin privileges.

Additionally, I tried to look into what API and GraphQL endpoints are available for observers but was directed to look at the permissions by role in Canvas, this revealed some really absurd options.

Analytics - Page Views is not available, cannot be enabled for an Observer. Parent should have access to this. Alternatively, a parent should have a view of /users/:kid profile page by default, which would provide page view list.

Really absurd, permissions page has the option to allow an observer to:

Add/Edit/Delete:

  • Course Pages
  • Learning Outcomes
  • Course Files
  • Course Calendar events
  • Course Content*

But for some odd reason, we can't let them Manage the Course State, and Add/Edit or Delete course sections.

We can give them the option to Import Learning Outcomes, though. /sarcasm :thumbsdown

Most of this, I'd hope isn't actually possible through the views/routes, but it shows some not-thought-out-to-well purpose of the Permissions page. Some testing will be done and a conversation with our CSM.

edit: nope!

In test instance, I allowed observers to set course content. I deleted assignments, modules, and edited the syllabus... but can't set page_view rights to their kids.

Smiley Sad

View solution in original post

Highlighted
Community Member

Sorry about the late reply handling other issues that popped up yesterday.

I also have a huge team to deal with all of the shenanigans that may arise(Me, myself & and a former student of mine working as an intern). We have written a Library in C# and hosting on a NuGet server to handle most back end things and reports. I would love to work with you to see how you have approached things in the past and moving forward, I just started in September and are, as they say, "building the plane as we fly it". I would like to share our libraries and everything I done so far as well. We are also utilizing doxygen to document as we go. 

ATM I don't have good latch of how to process the information and pass the info to the parents.

0 Kudos
Highlighted
Navigator

It's more likely that you'll bore the person to death while you wait for the results to load. Determining anything from page_views in real-time is slow. The page_views uses bookmarks for pagination, not the page= notation, so you cannot take advantage of concurrent requests. It took my computer 2.4 seconds to fetch 100 results and then 1.7 seconds to fetch the next link supplied in the headers. The next one after that was 4.9 seconds. Normal API calls are much, much quicker.

However, there is an even bigger issue. The page views is limited to the admin users and a user cannot even fetch page_view information about themselves unless they're an admin. 

GET /api/v1/users/self/page_views

{"status":"unauthorized","errors":[{"message":"user not authorized to perform that action"}]}

That means that you're not going to be able to do it completely within the browser since including an admin account token would be a huge security no-no. One could write a server-side script that contains an admin token so you can download the page view information or go to an LTI.

0 Kudos
Highlighted
Community Member

However, there is an even bigger issue. The page views is limited to the admin users and a user cannot even fetch page_view information about themselves unless they're an admin. 

Thank you for crushing my dreams sir. But analytics is locked to self so I thought that page views would be my next best bet.

Problem (K-12 BTW) : I would love to show parents their students activity logs or something of that nature to when the last time they interacted in classes.

As I dig through the API I am becoming more discouraged.

Would you with all of your Canvas wisdom be able to point me in the right direction kind sir? 

0 Kudos
Highlighted
Learner II

With 320k students, 95k Canvas Enrollments, and 44k unique students... the API is discouraging when trying to do large collections and analysis. Posting ideas, replying to your comment above.

0 Kudos