Hi @AshleyColnett,
I don't know of an easy way you'd be able to do this with a course shell that's reused year after year. Are you using a paid school/institution instance of Canvas, or the Free for Teacher one?
As a Canvas admin, I think the most common practice is to have a new course shell generated every unique time the course is offered (by term, section, cohort, etc). It's certainly not the only way to do things, as you are doing something different right now, but if you get a new course shell each time the course is run if eliminates the possibility for the situation you're running in to right now. In addition, it helps keep records and grades straight each time the student registered, which may be preferred by your school/institution. If you get a new course shell, it's very easy to import materials from a prior course, so don't worry about having to completely start over or anything.
Do you think this might be a possible workaround for you? Let us know here.
-Chris