Kathy Doig

Can someone answer questions about quiz item analysis - the help documents are inconsistent

Discussion created by Kathy Doig on Oct 15, 2019
Latest reply on Oct 25, 2019 by Chad Scott

I am conducting a training for college instructors on the use of item statistics to improve grading fairness and improve test quality.  When possible, I am referring to a variety of LMSs since audience members will vary in the products they use.  I wanted to include Canvas as one of those LMSs.

I have found two sources of information on the quiz item analyses provided in Canvas.  One is Canvas 2019 | guides.canvaslms.com | updated 2018-10-06 Test Quiz Item Analysis.  The second is  the following webpage https://community.canvaslms.com/docs/DOC-15054-4152780620  I have attached these.

These two items would appear to discuss the same content; that is, the test item statistics calculated in Canvas and their interpretation.  I first read the guide document which made no sense to me based on my knowledge of item analysis.   I could not believe that what it described was correct.  So I explored the website and found the webpage referenced above.   Here too, I find confusing explanations as well as inconsistencies with the guide document. So I believe that these two sources of support for users are inconsistent and that each has different errors.

I have attached each of the sources with my questions/comments.  I would appreciate some help from someone knowledgeable about item analysis, perhaps a psychometrician, to help me understand what is really calculated and provided in Canvas.    Thanks for your assistance.

Kathy Doig PhD

Professor Emeritus

Michigan State University