Leveraging Analytics to improve course design

[hollands]

It seems that my original post has disappeared with the launch of the much more awesome community site, so if it is somewhere already my apologies for the double post.

One of the things my team at Rowan University is getting into is Analytics, not necessarily to solve problems but to identify patterns to generate questions about improving course design. Devlin Daley once said when talking about Analytics "Good Analytics make good questions, not good answers," and that has always stuck with me. By looking at course analytics you can come up with all kinds of questions and possibilities to improve the design of the course.

At Rowan Online (I promise this isn't a plug), we're using both Canvas and Kaltura (media platform) to deliver content to students. Both provide Analytics and they're both proving to be very helpful when discussing course design to faculty. Often as a designer I find I have to sort of pitch my case to an instructor who has been doing things in a face to face environment for a long time. They don't necessarily know the online space is a different environment. By showing them some analytical patterns you can really easily help them with come evidence based course design.

Here is a short talk I did with my colleague at one of our open houses. This was a dry run for a conference presentation at Drexel University, so my apologies if we spoke too fast on some of the info points.

Rowan Online Open House - Session 1 - March 2015​

The TL;DW version of this:

1. If your LMS has Analytics take a look!

2. Gather up your data

3. Start to look at patterns.

4. Generate some questions (Am I comfortable with {Insert Analytic} showing what is is showing? Can I improve the pattern by doing X? What are other courses showing

Some of the things we saw already

1. Students watch about 10 minutes of a lecture video on average (Might be an indicator to faculty that shorter lectures might be more effective).

2. At our institution Discussions is the most popular area to be in outside of Modules

3. Grades was all the way down at 8 (which I found very very shocking).

Anyway, just thought I'd pass it along. 

[Stef_retired]

 @hollands ​, I believe my reply to your original post also got lost in the launch! Fortunately, I was able to find it in my email notifications (love those!), so here it is again (and apologies to anyone who is seeing this for the second time). My personal experiences in social media and in my own coursework caused me to implement some design changes, which correspond with your interpretation of the analytics, in my courses:

1. A year or so ago, I realized that when I'm on Facebook, if I see a video that's more than three minutes long, I won't bother clicking on it. So I redesigned my courses to accommodate what I'm sure is a similar tendency among my students by breaking up my videos into two- and three-minute chunks, each of which focuses on a single topic.
2. In keeping with that, I also shortened the pages in my course progression. Now, each page presents a quick, focused paragraph on a single topic.
3. I live in an asynchronous world, and as such, I view Discussions as the critical element in fostering student engagement. Anything we can do to promote improvements in the functionality of Canvas Discussions (<== a plug for your new feature idea) is fine by me!

[hollands]

Thanks stefaniesanders​ for finding and reposting! We recently had a meeting with a faculty member who had a 3.5 hour lecture (not kidding), and having some data to back up the idea of cutting it up was so beneficial. It not only made our point have more weight behind it but the faculty member got to actually see that their lecture wasn't being watched 100%.

[kroeninm]

Wonderfully concise webinar  @hollands !  I like the approach you took with your focus on the application and questions that may arise from Analytics instead of the button-clicks.  I've done a few Analytics training sessions in past years but I think this approach is much for valuable as I was struggling with the idea of how to convey "how to use" the Analytics to faculty and not just "how to find" them   We have Kaltura as well so very helpful to combine the two.

Thanks!

- Melanie

[hollands]

You're most welcome  @kroeninm ​. As a person with a music background it's still weird to look at math and analytics as a positive thing (I have never gotten along very well with numbers), but I have to say it's really changed my design approach and more importantly my approach with the faculty. It's a lot less of "well let's put it there just because" and a lot more of "Well when we do X this happens. How do we feel about that?" I find it really useful especially when you have a faculty member who might be resistant to the idea of teaching with a LMS.

[morris_admin]

Hi  @hollands ​​,

I attended your session at Drexel's elearning conference, so it was nice to come across this post as I am preparing to give my own presentation for faculty on using Canvas analytics to inform instruction. The idea for this session was definitely inspired by your talk, so thanks for that!

- Jessica

[hollands]

Hi morris_admin! It's great to "see you again"! I'm glad the presentation was enjoyable and at least some help! Best of luck with your presentation!