Assessing Anthropic’s Claude’s capability to create engaging and safe learning material in a highly sensitive and highly regulated school environment

Our recent investigation and review of Anthropic’s Claude revealed how it attempted to incorporate empathetic reasoning behind its design of a learning module. We found that in seeking to design for emotion, Claude drew on pathos and sought to evoke tense emotions, including fear and insecurity, in order to “engage” learners.  

The importance of utilizing empathy for design is emphasized in the human computer interaction (HCI), user experience (UX) and design literature. That empathy is considered crucial to design can be seen from the broaders attempts from Gen AI tools to incorporate such into their outputs. 

Our findings draw attention to the potential links between pathos and trauma and how Claude’s strategies risk compounding and re-traumatizing survivors of child abuse.  

Our investigation identified that whilst Claude can be useful for identifying some relevant content for inclusion within a module on Child Safety in schools that takes into account state specific regulations, it cannot be relied upon to meet safety, accessibility and engagement needs. 

We found that Claude’s ability to review its own products cannot be relied upon to ensure the learning modules it creates are safe for learners, and that the manner in which the content is presented can reach diverse learners. In this trial, we found that whilst Claude delivers its output with statements of finality, our review demonstrates both the potential for improvement and the need for critical appraisal from learning designers.

Critical appraisal from learning designers is especially important when Claude claims that it has all the information it needs, when it claims that its content is “accurate”, when it claims that it understands the profile and needs of users based on general information and when it cannot identify contradictions across intention and the strategies employed.

In response:  Our designs acknowledge that emotional risk is a potential obstacle to learning, and our briefs are informed by designing for emotional capacity and safety.