[Reports] Enhanced LTI Report

[shane_ohara]

Hello, I would like to request an enhancement to the LTI report based on the archived idea Report for LTIs (External Apps) Usage and a concern that account-level LTI configurations are not being reported by course. For example, we have configured many partner apps at the account-level because of high demand/use needs; however, the LTI report only shows them tied to the account. Here's a report filtered on Cengage, Macmillan, and McGraw Hill to use as examples for no listing of courses where deployed:

The report would be more useful if it also showed the courses that they are deployed in. This means we can then view total use of LTI to see which are being used, or not being used, and to then have a course count per term. Right now there is no way to identify courses where various LTIs have been deployed.

Thank you!

Shane

[fosterl]

This is still an important need at our institution. Having already voted for it, I can only hope that this moves forward some day now that the Radar stage no longer exists.

[davids]

Unfortunately, Canvas Data isn't really a viable solution for alot of us.  I've got it enabled on our account and have tried to use it, but to make any effective use out of it, you've got to have programmers and systems analysts on staff with substantial time on their hands to write custom reports to leverage those flat files into something useful.  My institution is a small private college with NO analysts, so it's just me as an LMS admin trying to export useful data analytics to justify budgeting expenditures and analyze usage metrics.

But I also know quite a few others in my roles at other larger institutions who have on-staff systems analysts, and although they have the in-house talent to write those reports, their time is always focused on far more pressing issues.  And even for those who know what they're doing, it's still not a simple process.  A colleague of mine did a presentation at a pre-conference for a regional instructional technology conference on leveraging the flat files from Canvas Data, and although it was useful and informative, I came away realizing that there's no way we could take advantage of it anytime soon and questioning how many other schools can realistically use it either.

Big data analytics are great when your institution's got the resources to write the reports and utilize it along with the VP & administrative support for research analytics, but I would venture to guess that a sizable portion of the Canvas admin population just needs interpreted data...simple, enhanced reports like this feature request calls for.  Seriously, a report how on what LTI integrations are in use on our site and which courses those integrations are deployed into ought to be a fundamental basic report for any LMS admin.  It is the starting point for justifying purchase decisions, estimating cost per user, renewing contracts with vendors, and knowing which integrations are most popular and need more support and training.  We really shouldn't even have to ask for it; if you use LTI's, it's something you need to know, and I would imagine that most admins get asked these questions all the time by those they report to.

OK, setting my soapbox aside now so that I can get back to manually hand-counting which courses of ours are using TurnItin and trying to determine how many of them use the legacy API versus how many use the newer LTI.  I can't believe there's not already a simple report created for that!

[shane_ohara]

Thank you for articulating your need for a similar report. We as an institution manage 9 "SaaS" application with 3 administrators and a manager. We do not have the overhead to use Canvas Data to find something simple as LTI's in use in a course.

[jsowalsk]

This is a very important need for our institution since we are looking at creating a third-party tool request process and reviewing the current integrations by our security and compliance team of what is already integrated into canvas.

[tvannorman]

Unfortunately, the documentation Canvas provides at: https://community.canvaslms.com/ideas/7420-enhanced-lti-report states: 

I would think "usage information" would indicate "where" something is "used".  I just had this document provided to me with the explanation the "usage information" indicates "where" something is "installed".  

I absolutely agree adding "where the LTI is used" is necessary to the LTI Report.

Hopefully Canvas will re-visit this report to provide this useful information.

[ward_michael]

I agree with Tim Van Norman that the current Canvas LTI Report is only a listing of the LTI tools "installed" and not necessarily providing information about tool "usage" which our colleges are interested in analyzing.  Having a report for a list of "installed" tools at the account levels and at the course levels would be nice to have but also need some activity hit counts on which tools are actually being used within each course (also would like to see overall account usage as well as the listing of each course showing activity usage on the tools).  The "usage" reports can show which courses are benefiting the most from a tool as well as how widespread the adoption of a particular tool is being used.

Also, it would be nice to view teacher usage versus student usage on tools too.  And root account level tool usage for example we use NameCoach that is root level installed and would like to see user usage.  Other tools may only be Sub-Account level installed due to separate licensing (maybe one college licensed and using but other is not).

[mahola]

This report would be very helpful for us.

[kholt12]

I'm proposing that Canvas be able to pull data and analytics on which instructors in an institution use the extra "tools" that can be added into the course. This could help Deans and Committees gather information about current usage and potential outreach for greater usage. It would also help to confirm the demand for a tool based on cost: which tools are we paying for that aren't being used? Which tools are being requested that we haven't purchased? Which tools seem to be used most often? This data could be extremely valuable for Canvas and any of their tool partners. 

DISCLAIMER: There would also need to be conversations about the privacy of this data and how it can be used. The last thing we'd want is for this to turn into a tool for advertising programs to faculty without their consent. Or worse, for information to be collected about faculty's use WITHIN the tool. 

[c_carrillo1]

Here it's 5 years later and this still hasn't been implemented.  Canvas really needs good reporting on LTI tool usage...

[chriscas]

I'll say that I would be a proponent of a true "usage" report, instead of the "installed" report we have had for years.  That being said, I can somewhat understand the reason that the only report we have available is for installations...  LTIs are great and very flexible, but that could cause a lot of confusion in what is considered a "used" LTI...  Some examples:

• Let's say an LTI is installed on the root account level, and it adds a course navigation menu called Homework"  How would that be treated in a usage report?  Since it's installed by default, all courses have would have it.  At the same time, I'm sure not all courses truly use it.  Looking at clicks isn't particularly helpful, since someone could click and end up at an empty page.  Courses where the navigation item is hidden could be excluded from the report, which would help, but would not produce the report I think most people want.
• Let's say an LTI is installed in a subaccount, and adds a tool to the RCE.  How would this be counted?  The tool would be available to all users in the course any time they go to the RCE, but do we know that anyone used it?  Again counting clicks wouldn't necessarily help, and that would seem to be something very computationally costly to track anyways.
• With the two examples above in mind, what about tools that perhaps add a course navigation button, an editor button, and maybe even an assignment type (Kaltura is an example of this kind of LTI that comes to mind).  That sort of thing would be even trickier to figure out how to report...

I think a lot of people might just be considering LTI assignments, which could have utilization tracked by whether an assignment tied to the LTI is actually created and available in the course.  Unfortunately, having to consider all the different LTI types and placements seems to make tracking usage almost impossible without really taxing systems trying to check events like clicks (and even that, as I mentioned earlier, being somewhat unreliable).

Again, I'd be all for a report that could accurately report on LTI usage, but I fear if Instructure did try to develop anything like that, we'd never come to a consensus on what "usage" meant.

-Chris