Ahead of the new school year, a handful of tech companies issued grim warnings to American educators embracing artificial intelligence in the classroom: Chatbots, they said, could endanger students and lead to self- harm. Vigilance was paramount. “The risks of students using AI can literally be deadly,” one company cautioned. Another noted: “Student lives depend on it.”
The “it” is the software those companies are selling — tools that themselves use AI to scan students’ conversations with chatbots and alert adults to potential danger. Across the US, teachers and administrators are increasingly turning to companies such as GoGuardian and Lightspeed Systems for real-time monitoring of student-bot conversations, according to interviews with more than a dozen educators. The goal is to catch early warning signs of severe outcomes, including teen suicides.
“I sleep better knowing that we have this tool for our students,” says Ian Haight, director of technology systems and services for Kalamazoo Public Schools in Michigan.
