Nutrition Facts and PII

[lbeach]

I really appreciate that Instructure is posting "Nutrition Facts" for all of its AI-connected tools. I am wondering, however, about this statement in all of them: 

When we asked ChatGPT what this means, the response is:

1. Canvas does not intentionally send structured PII fields (like names, emails, or IDs from the Canvas user profile) to the AI model.
• This suggests that fields such as "student name" are excluded by design from the prompt sent to the model.
2. However, if a user types PII into the discussion reply itself, that text content may be sent to the model, because it’s part of the reply being summarized.
• For example, if a student writes: “Hi, I’m Jordan Smith, and my ID number is 1234567…”, that whole message is part of the source text, and thus may be passed to the LLM for summarization.
3. “PII is not exposed” likely refers to not displaying that PII in the final generated summary, but not necessarily excluding it from the model’s processing pipeline.

Summary

Statement	Meaning
"PII is not exposed"	Likely means PII is not shown in the final summary output
"PII in discussion replies may be sent"	If students include PII in their own messages, that full content could be sent to the LLM
"No PII is intentionally sent"	Canvas isn’t explicitly sending PII fields from Canvas accounts to the model

I'm wondering if someone at Instructure could clarify. I think this is an important consideration in decisions regarding the use of these tools.

Thank you,

Lisa Beach

[emilyk23 · Accepted solution]

Hi @lbeach - thanks for inquiring about this! It looks like your interpretation is mostly spot on.

"PII in discussion replies may be sent"	If students include PII in their own messages, that full content could be sent to the LLM
"No PII is intentionally sent"	Canvas isn’t explicitly sending PII fields from Canvas accounts to the model

The only thing to clarify is that "pii is not exposed" does not necessarily mean "pii is not shown in the final summary output." There aren't guardrails in place (outside of the prompt) to prevent that. But, PII isn't at risk of being shown to anyone who doesn't already have access.  The feature is only available for educators and only runs agains the data in the course that they teach.  

Hope this clarifies that statement!