Treat LTI Launch as a Participation

[paul_fynn]

The absence of LTI based participation from both New Analytics and Admin Analytics is pretty much a showstopper for courses that make intensive use of New Quizzes and other integrations. The result is to substantially underreport student activity.

As an interim measure, could Canvas find/implement a mechanism which allows either

• An LTI based activity to trigger a participation record instead of a page view for every launch OR
• An LTI based activity to trigger a participation record for the FIRST launch

There may be LIT categories that we may wish to exclude.

[carlycurran · Accepted solution]

Thanks, all, for the details on this! I'm taking both of these things into account for discovery:

- how to represent LTI tool use when looking at student participation
- how to make financial aid reporting easier 

[chriscas]

Hi @paul_fynn,

As a Canvas admin for my institution, I would agree that extensive use of LTI tools can make activity harder to track in those courses.  Some LTI providers do have their own analytics system, but it's very messy to have to collect all of that data.  I know some schools attempt to create a central learning record store and data from every source possible into it, and then create their own dashboards and reports.  We're a mid-size public university, so that's really not an option for us with our staffing and budget, but it may be something you could look in to.

With all that being said, I would unfortunately be very much against counting an LTI launch as participation in our data.  At least in the US, participation has a somewhat narrow definition for the federal government reporting, and just viewing something in a course unfortunately does not count.  For something to count as participation online, the student basically needs to make some kind of submission (discussion post, assignment submission, quiz, etc).  While some LTI tools do have submissions, there are many that are just for content delivery.

The new analytics attendance feature does let institution admins define what counts as attendance, so maybe we need something similar for participation which could be used in New Analytics, Admin Analytics, and other areas of Canvas (like the green participation checkmarks in the user page view history).  With my coding hat on, I realize the amount of complexity that would add to all of these areas and reports though, and it really may not be feasible for Instructure to do.

Hopefully someone form Instructure will be willing to comment on this from their perspective.

-Chris

[DecoyLex]

For what it's worth, Canvas does have some mechanism for tools to post interaction data back to the LMS using xAPI. However, I've never actually seen any tools that implement it, and it's not clear whether that'd count as an interaction or if it'd just log time in the student's Total Activity counter.

[paul_fynn]

Thanks DecoyLex - could you expand on that a little please; we are comfortable looking around the RESTAPI documentation and that's not something that we have come across.

Certainly that was the point of Tin Can - to report event data ....

Sorry - just spotted the link - I guess the challenge here would be to get iParadigms and other LTi suppliers to code their end in - or is it already coded in and not being picked up on the Canvas side ?

[DecoyLex]

My understanding is that the tools need to actually post this data to Canvas; all that Canvas has implemented is a way for tools to report that interaction data as page views.

It's pretty safe to assume that a tool doesn't use this endpoint if you can't find a custom field (in the tool settings for LTI 1.1, or developer key settings for LTI 1.3) that uses the $Canvas.xapi.url substitution.

Unfortunately, digging into the code for Canvas, it looks like only a page view event is posted, and tools can't log these views as participations even though xAPI differentiates between seeing content and interacting with content. For a page view to count as participation, the participated field on the page view needs to be set to true [application_controller.rb:1669]. However, when the LTI::AnalyticsService creates a new PageView, participated is not set [analytics_service.rb:55].

[paul_fynn]

Thanks, that's very helpful. The problem here is that we have at least one provider (working through API) who isn't prepared to go through all the hoops required to get the authorisations required from institutions to effectively be given the authority to change the institutions' gradebook. (in the context of passing back a result) 

I think on that basis the buck goes back to Instructure to find a way that would allow institutions to opt in or out of an xAPI event push (or an API grade push) so that data pushes are effectively 'authorised' at that point.

Interesting that your federal requirement is for 'participations'; I guess the parallel in the UK is our HESA returns which take the final assessment as the last date of attendance, but in terms of useful learning analytics/engagement I think that clicks are just vanity numbers and participations are the gold standard.

[ProfessorBeyrer]

Thank you all for this discussion. It makes it especially challenging for our financial aid departments to track down date of last attendance for online classes (following the federal guidelines mentioned by @chriscas). When a student is dropped from a class their financial aid is prorated to the date of last participation, which means a deep dive into each course via the browser for the instructor (or likely the sysadmin) to look for the most recent academically relevant participation. Ugh!

[carlycurran · Accepted solution]

Thanks, all, for the details on this! I'm taking both of these things into account for discovery:

- how to represent LTI tool use when looking at student participation
- how to make financial aid reporting easier 

[SHEBENE]

If, during this discover phase, you could get to a point that LTI assignments trigger the "new" gradebook features that are triggered when an assignment is submitted. That would be amazing!

[IanGoh]

we're currently going down the path of Canvas Data 2 for financial compliance reporting.  Looking at data points like:

• enrollments.last_activity_at
• submissions.submitted_at (for assignments and classic quizzes)
• and perhaps discussion activity (still researching).

[Justin_Reynolds]

Hi!  Has there been any update on this?  We are very disappointed there isn't visibility into the publishers' systems that integrate via LTIs.  I don't know where the responsibility lies for the development of such analytics but I would hope Instructure would champion the task.  Thanks!

[pwhitmer8]

Similarly to @Justin_Reynolds , any thoughts @carlycurran on a future for LTI providers data integration into the Data Access Platform? We're an LTI content and course provider, and have used Canvas' API, now transitioning to LTI 1.3. We're sending scores, but have so much more (time on task, most common wrong answer - good stuff that instructors ask for), and having a sense of the data integration future would be much appreciated! 

[paul_fynn]

Hi All, 

I haven't fully got to grips with the recent changes, but this came up during our discussions on Canvas Impact, and there is evidence that participations are now showing up in New Analytics, and by the looks of things this may be retrospctive (ie quiz participations prior to the update may be showing. 

I'm not clear that they are showing up in the sub-account course activity report, though .... @carlycurran are you abel to confirm at all please ?

many thanks

Paul

[carlycurran · Accepted solution]

Hi, Paul! Thanks for the question. 
We are now showing New Quiz participation in New Analytics (and Admin Analytics) when a student _submits_ a New Quiz. 
These New Quiz submissions as participations were backfilled, so you should see them in past data. 

Does that answer your question?

[paul_fynn]

Hi @carlycurran,

Further to your update on this we are now finding what seem to be anomalies between the data reported in 

i) The sub account Course Access Log - current report shows New Quiz Views, does not appear to show New Quiz participations

ii) The detailed  Student Profile - Student Page views (detailed links) shows New Quiz Views but does not appear to show New Quiz Participations

iii) Overall New Analytics per course/cohort appears to show both New Quiz Participations and Views in the Weekly Online Activity Participations list, however where a student completed a New Quiz last year, and revisited the New Quiz in this year, the latter does not show up as a View - this may correspond to the student accessing the quiz via the Grades view.

This gives us some problems, particularly given the increased emphasis on Student LMS access and participation as part of our student retention and achievement policies.

Is anyone else aware of / able to validate this issue ?

Many thanks

Paul