Hi @lapeavey ...
Have you tried using these option within your quiz?
You can find more information about these settings within this Guide: What options can I set in a quiz? - Canvas Community.
I hope this will be of help to you. Sing out if you have any other questions...thanks! Take care, stay safe, and be well.