A New Look for the Studio Quiz Results Page

AkosFarago
Instructure
Instructure
3
1226

Canvas.png

After the Item Analysis improvements announced last year, the Studio Quiz Results page will adopt the same layout and metrics from Jan 16. Here are the changes applicable for Studio video quizzes:

 

 

Old Report

New Report

Report Generation

Generated every 24 hours

On-demand generation

Quiz attempts

All attempts are included

Last attempt included for each students
(Individual attempts can be still tracked in the Student Analysis report)

Quiz Analysis - Median Score

Not available

Included

Quiz Analysis - Standard Deviation

Standard deviation represents a points-based value.

The standard deviation represents a percentage-based value to align with the mean score. This allows for a better understanding of the dispersion or spread of scores relative to the average performance.

Quiz Analysis - Cronbach’s Alpha

Not available

Random items are excluded from Cronbach’s Alpha calculation to ensure proper measurement of internal consistency

Item Analysis - Median Earned Points

Not available

Included

Item Analysis - Discrimination Index

Not available

Students who haven’t received an item are excluded when calculating the discrimination index.

Item Analysis - Pearson Correlation Coefficient (rpb)

The point-biserial (rpb) correlation coefficient is used.

Corrected item-total correlation coefficient is used.

Performance by Quintile

Focused on individual question answers rather than reflecting students' overall quiz performance.

Not available yet.

 

As you notice, one of the biggest improvements is that the report will need to be generated on-demand although the guaranteed no delay in the results will make up for the additional click. Also, keep in mind that similarly to the previous IA improvements, there need to be 3 students who fill out the quiz before the report could be generated. While there are some minor differences in the new metric definitions, we removed the Performance by Quintile section due to inconsistencies in those results. We intend to reintroduce it in the future however, there is no set timeline at the moment.

 

AkosFarago_0-1704803499607.png

 

We are gearing up to invest considerable efforts in enhancing the video quiz experience. Some of the areas we will focus on is fostering collaboration between Teachers to share their video quizzes with each other, gauging student understanding with more question types (pst… open-ended is coming!) and giving more controls to Teachers over their video quizzes such as setting limits on quiz attempts and disabling the ability to seek forward. Send me a direct message or add a comment below if you want to participate in the research! We are grateful for all the feedback!

 

Best,

Akos

3 Comments