Hi All,
You may have also noticed that the "Homework Help" button only displays when visiting sites like Canvas (very sneaky Google!). However, you can click into the URL address bar on any website for options to "Ask Google about this page" using Google Lens.

I did some research and discovered the following:
Chrome - Google Lens
Google Workspace
Google Workspace Admins who manage Chrome browser or ChromeOS devices for schools may be able to disable the Google Lens feature with the LensOverlaySettings policy. Otherwise, the GenAiDefaultSettings policy is used as the default.
Use of Personal Google Accounts
Keep in mind, a Google Workspace Admin cannot manage Chrome browser settings for users on personal computers if they are not signed into a Chrome browser profile using their school managed account. Management is tied to the Chrome profile, not just the Google Workspace account itself.
I believe schools can enforce a policy that requires a separate, managed Chrome profile whenever a user signs into their managed Google Workspace account on an unmanaged (personal) device. This would ensure that school or work-related browsing data is kept separate and managed, protecting both personal data and the organization's data.
Google's Term's of Service for Chrome AI Features
Students signed into personal Google accounts are not managed by their school's organization and are subject to different terms of service. In other words, data is sent to Google and used to train their AI model. This includes any of the built-in AI features in Chrome such as Text capture or Help me write.
Use of Gen AI features in Chrome "are not covered by Chrome’s enterprise data guarantees for generative AI as they are subject to their own guarantees and/or terms of service." See the Generative AI features and policies help page for Chrome Enterprise and Education. This page includes a list of built-in and additional Google generative AI features available through Chrome (including a breakdown of which terms of service are subject for each feature).
How can school's respond?
If you are a school that uses Google Workspace, find out who manages your Google Workspace for Education account. Ask them to contact your Google Workspace for Edu account rep and let them know that capturing data from password protected sites like Canvas is unethical.