Hi @irizarryl-4,
I don't think there is really a way to customize what's on the coming up list. On the student side, items from unpublished courses won't show, but on the teacher side I think it's different because the expectation is that the course will be published before whatever the assignment is will be due. I *think* but am not 100% sure, that if the assignments themselves are unpublished (not just the course) that they might not show up for teachers, but then the teacher will need to remember to go back and publish all of those for student, which can be a lot of work in large courses.
I hope this info helps!
-Chris