As near as I can tell, it's not possible to get to Speedgrader from within the quiz page. The documentation only talks about accessing a New Quiz Speedgrader from the Gradebook.
Is it possible to get to Speedgrader from the Quiz itself?
If Not, Is there an existing Idea requesting this feature?
You can open the individual quiz attempts from the moderate page, and that will show the same ability to update scores as the speedgrader, including adding fudge points. Go to the moderate tab, and click the attempt link next to each student. You can regrade, adjust scores on individual questions, add fudge points, and add comments to individual questions. It is essentially the same view that displays inside the speedgrader.
What is missing is the ability to add a general comment, and the easy navigation from one student to the next. General comments can be added in the gradebook,without using speedgrader, and navigation from student to student is fairly simple in the gradebook.
Getting everything in one place for manual grading with easy navigation between students, and an opportunity for general comments, still requires speedgrader. I am not aware of a path to speedgrader from within the quiz itself. However there is an update underway to open to the assignment setting page before going to the quiz. Currently that page is only accessible by clicking edit from the assignment tab. Once it opens directly without clicking edit, it should display the speedgrader link.
There is also an open idea conversation about this: https://community.canvaslms.com/t5/Idea-Conversations/New-Quizzes-Add-location-for-SpeedGrader-Acces...