This could be a couple things. First make sure they're using their school account to log into the class notebook.
Second, the first time you use OneNote you are on the homepage. There should be a menu that says My Notebooks, Shared with Me, and Class Notebooks. The students need to click on class notebooks to see the list of notebooks.