I am conducting a training for college instructors on the use of item statistics to improve grading fairness and improve test quality. When possible, I am referring to a variety of LMSs since audience members will vary in the products they use. I wanted to include Canvas as one of those LMSs.
I have found two sources of information on the quiz item analyses provided in Canvas. One is Canvas 2019 | guides.canvaslms.com | updated 2018-10-06 Test Quiz Item Analysis. The second is the following webpage https://community.canvaslms.com/docs/DOC-15054-4152780620 I have attached these.
These two items would appear to discuss the same content; that is, the test item statistics calculated in Canvas and their interpretation. I first read the guide document which made no sense to me based on my knowledge of item analysis. I could not believe that what it described was correct. So I explored the website and found the webpage referenced above. Here too, I find confusing explanations as well as inconsistencies with the guide document. So I believe that these two sources of support for users are inconsistent and that each has different errors.
I have attached each of the sources with my questions/comments. I would appreciate some help from someone knowledgeable about item analysis, perhaps a psychometrician, to help me understand what is really calculated and provided in Canvas. Thanks for your assistance.
Kathy Doig PhD
Michigan State University