Automatic RollCall Attendance Exporting

[GuyMor]

Hello,

In lieu of an API call to get attendance into Student Information Systems, is it possible for us to receive a scheduled attendance report and have it automatically mailed to us by Canvas or by the Instructure customer support team on a daily basis? 

Thanks,

Guy Mor

[Chris_Hofer]

@GuyMor ...

I have not heard of anything like this, but that's not to say it doesn't exist.  I'd recommend reaching out to your assigned Customer Success Manager team at Instructure to ask about this.  If you are not a Canvas administrator, talk to someone who is at your school...such as their Online Learning / eLearning / Distance Education team.  Good luck!

[GuyMor]

Thanks Chris, I will try this route.

[James · Accepted solution]

@GuyMor 

It is possible, but not trivial, to tell Canvas to automatically generate and email the attendance report.

When you launch the attendance report from the Admin > Attendance > Generate Report tab, it launches the RollCall Attendance LTI. That means that if you can get to that page through automation, you can schedule it yourself.

There are a couple of ways to get there.

• Use a headless browser such as Puppeteer or Selenium.
• Use a program that accesses the API and launches the LTI and then completes the form.

I used the second method (used past tense -- the server that contains the code decided not to recognize its network connection this summer and I cannot reach that server except by logging in at the console. Thankfully, I had the code backed up, but haven't found time to set it up on another system). I would use the API call to get a sessionless launch url for the rollcall LTI, launch it, navigate through the redirects until I got the page with the form. Then I would fill out the form and submit it three times (once for each of the three previous weeks to give faculty a chance to catch up with a backlog of entry), and then wait for the emails to come. I had the email setup so that it went to a special account that intercepted it and processed the file automatically, adding the information to a database that was available locally for analysis.

The first way may be easier if you can get logged into Canvas through the browser in an automatic way. You would still go to the form, but it would be inside Canvas. I've done some things with Puppeteer and could probably automate it if I hadn't already written the script to do it through the API.

Whether you intercept the mail and automatically process it or you just send it to someone's email and they manually process it is separate from generating the email.

Getting the information into the SIS is a completely different task than getting it out of Canvas. Our SIS doesn't support attendance, but I was gathering the information for an early alert system.

[GuyMor]

Thank you so much James,

This is definitely the direction I was hoping for. I figured out the part where I can extract a file from the HTML link in a similar way to what you describe. Once its a csv on my end, I can figure out the API into the SIS. 

Could you clarify the workflow you used once you launched the sessionless LTI please:

What I understood is that you (ingeniously) spun up a server with just the RollCall, and then basically went in and manually scheduled the reports in bulk, and that you did so after the attendance was taken (please correct me if I misunderstood).

1) Could you have scheduled it in advance of the attendance? Say, at the beginning of a semester coulf you have scheduled a report that would trigger at a future date, the end of every week for 15 weeks?

2) How could I account for multiple courses with the seassionless LTI? I see the call gets the courses and the account, does the interface then give you a choice of these, essentially allowing you to take attendance of any course and by extension report for any course? could it report an all the courses too?

3) Could you see an RPA like MS Power Automate or Uipath automating this part as well, perhaps daily for each course?

4) Is the source code to run the LTI outside of Canvas something you are willing to share? of course, taking out any keys or URLs that are related to your instance

[James]

I think your statement before (1) is about 179° off from what I do. I didn't spin up a server. Roll Call hasn't (as far as I know) had the source code released. I don't manually schedule any reports in bulk - I have code that does it form me (I'm all about eliminating repetitive keystrokes or mouse clicks)., so nothing is manual. You can only download information that is there, but you can schedule the program to run whenever you want. I use Linux, so there are cronjobs for scheduling regular events. I think Windows has a scheduler, but I don't use Windows to do backend work.

I'm using Node to execute JavaScript, although you could use Python or whatever language you're familiar, with to make network calls to Canvas servers. The main thing to understand is that I am filling out the form that is available under Admin > Attendance > Generate Report. I am limited by whatever it can do. I don't have any special access to the backend of Roll Call. People have been asking for API access to the data for a while, but mostly, we need a different attendance tool.

I scheduled my job to run nightly so you can schedule it in advance. For some things,  you need to get it sooner rather than later as Canvas only makes the information available for a certain amount of time. I don't know about Roll Call, but web logs in Canvas Data 2 are only available for 30 days and some of the log data in other parts of Canvas disappear after a year. I went into and asked for attendance from August 2022 (14 months ago) and it delivered it, so it may not matter with Roll Call.

That email with the link said the data is available for the next 6 hours. The expires in the URL says 86400, which is 24 hours. I'm not sure which to believe, but if you run this in the middle of night and don't automatically process it, you'll want to make sure that someone is there to handle it.

A low-tech solution if you're only wanting to do this at the end of a 15 week term is to have someone fill the form out 15 times a few times a year. It will take a couple of minutes and then you wait for the emails to show up. If you were doing this every week or every day like I was, then you would want to automate it.

I fetch data for the entire institution, which limits me to one week at a time. That's why I complete the form multiple times to get the last three weeks of attendance records. That also eliminates the need to worry about multiple courses, you get everything, just for a limited amount of time.

The Roll Call attendance is launched using an account LTI code not from within a course. Use the account version of list external tools to get the ID. Store that ID somewhere. Hardcode it if you like. It's not changing for your account unless something major happens and then you can save the lookup in the future.

Then you get a sessionless launch url for an external tool. Use the account version with the ID you got in the last step.

This will do some redirects that you will need to follow and save the cookies. Eventually it will stop and you will be in a Roll Call session and have a form.

I then create a virtual DOM and parse the form, populating it with the values you need. Then you submit the form. This is why it would be easier to do with Puppeteer or Selenium. They work within the browser so you don't have to create the DOM, it's already there. The redirects are automatically followed. It's just an all-around easier process since you don't have automate as much.

I have no idea what RPA, MS Power Automate, or UIPath are to answer that question. All I can surmise from that is you're running on a Windows system. I've used Puppeteer to scrape some websites and it wasn't too bad. Whenever I mention headless browsers, most people say "Selenium" but I've not used it myself (I'm not a Python programmer). Puppeteer uses JavaScript, which I've written a bunch of code for, so it works for me. Node uses JavaScript. If I was to do this on a Windows system, I would use AutoHotKey because that's what I'm familiar with although I wouldn't be able to schedule that. Those are all personal preferences and it sounds like your tools don't overlap with mine.

Although my code has very little hardcoded into it (most of that is in a config file), it's not really packaged for me to release. I do a lot of hack programming that works for me but isn't usable by anyone else.

[GuyMor]

Thanks James, 

This is valuable information, it's encouraging that the export is for the entire institution. This also shows me that it's possible, though convoluted.

Uipath is a SaaS service that creates robots that click and execute actions on screen. A local version would be something like Keyboard Maestro on a mac or Macro Express on Windows. 

I'll report if I make any breakthrough on my end.