@TFraser2 Your suggestion sounds like an idea that was attached to a theme that was selected for development in 2023. You might subscribe to the theme and contribute to the discussion. It seems that the updated Assign To features that arrived 7/20/24 grew out of that theme.
In the meantime, I want to suggest a workaround that may save time. Rather than selecting all students to include in order to exclude a couple of students, you could specify the students to exclude and set an available until date in the past. This should lock the assignment for the excluded students. In the gradebook, you could select Excused as the status for their submissions to these assignments. It is a less elegant solution than genuinely not assigning to the excluded students, but it may be good enough for the time being.