Enhancements don't end when the student leaves the school

We turned it on for one school which is not using Canvas this year. So, it seems that if a student or teacher ever had a course at that school, they are seeing the new dashboard. Than means if they moved on to Jr. High or High School, it was still showing that dashboard. I'm not sure if there is a way to prevent that other than deleting old courses?