Students Want AI Companions — Not to Be Watched

📌 Original paper title

AI Sensing and Intervention in Higher Education: Student Perceptions of Learning Impacts, Affective Responses, and Ethical Priorities

👥 Authors and affiliations

Bingyi Han, Ying Ma, Simon Coghlan, Dana McKay, George Buchanan, and Wally Smith. The first author is affiliated with the School of Computing and Information Systems at the University of Melbourne and the Department of Computer Science at Saarland University; collaborators are from the University of Melbourne and RMIT University.

🔗 Paper link

https://doi.org/10.1145/3772318.3790360

🧠 What this paper really asks

A common narrative for educational AI is that if a system can detect distraction, confusion, or frustration, it can prompt sooner or alert a teacher to intervene. The technical path sounds reasonable, but this CHI 2026 paper turns the question around: Do students actually want to be observed by AI while learning, then intervened upon by a system or a teacher?

The authors focus not on whether algorithms can recognize gaze or expression, but on how students experience such sense-and-intervene systems. That angle matters because education is not only an efficiency problem. Whether students feel respected, can control their learning pace, and fear classmates seeing that they “need help” all shape acceptance of AI—and learning itself.

As shown in Figure 1, the study split the system along two dimensions: sensing—whether AI sensing is used—and intervention—whether prompts come from the system or a teacher steps in. The authors further compared two sensing modes: gaze-based attention detection and face-based emotion detection.

🔬 How the authors ran the study

The study used a two-stage online mixed-methods experiment with 132 Australian university students. In stage one, participants watched six video scenarios and imagined using an educational AI system. Scenario combinations included: no clear disclosure of AI sensing, with either system prompts or teacher help; and clear disclosure of gaze detection or facial emotion detection, each triggering system prompts or teacher help.

After each scenario, students rated two kinds of questions on five-point scales: whether they believed the system would improve learning—personalization, efficiency, grades, and control of learning pace—and their affective responses—pleasantness, confidence, anxiety, discomfort, worry about being labeled as struggling, and fear of making mistakes. Scale reliability was high: Cronbach’s alpha was 0.948 for learning beliefs and 0.957 for affective responses.

In stage two, participants rated and ranked six ethical concerns: privacy, autonomy, fairness, accuracy, transparency, and learning benefit. Beyond item ratings, the authors ran 15 pairwise comparisons and used a Bradley–Terry model to estimate which value risks students prioritized.

📊 Key findings

First, students did not reject AI prompts, but clearly rejected being monitored. In scenarios without disclosed sensing, evaluations of learning value were relatively positive—students believed the system could support personalization, raise efficiency, and improve grades. Once the system was explicitly said to be detecting gaze or facial emotion, evaluations turned negative.

The numbers are direct. Overall learning-belief scores fell from 3.28 without sensing to 2.51 with sensing; affective-response totals fell from 2.85 to 1.96. Perceived efficiency dropped from 3.60 to 2.56; anxiety rose from 3.03 to 4.22; discomfort rose from 3.26 to 4.33. Sensing itself erased much of the goodwill that personalized intervention might otherwise earn.

Second, gaze detection and expression detection looked much the same to students. The authors had hypothesized that students might accept gaze-based attention detection more readily, as less of a private emotional intrusion than facial emotion detection. Results showed no significant differences on learning beliefs or affective responses. What students disliked was not one sensor type, but the experience of “being observed and judged while learning.”

Third, students preferred system prompts over automatically summoning a teacher. With or without sensing, system-generated prompts were more welcome than teacher intervention. Without sensing, mean learning belief was 3.47 for system prompts versus 3.09 for teacher help; affective response was 3.47 versus 2.26. With sensing, both worsened, but system prompts still beat teacher intervention.

That is not because students want no teachers. Automatically calling a teacher changes classroom social relations. Students worried about classmates seeing them singled out for help, about being labeled as “struggling,” and about teachers intervening before they were ready. For university students, help that bypasses voluntary choice easily becomes pressure.

🧩 Qualitative feedback explains why

In open responses, the most common concern was anxiety and discomfort from continuous monitoring, mentioned by 78 participants. Sixty-six mentioned interrupted learning autonomy and control. Students described beginning to “perform focus,” fearing that gaze, expression, or momentary distraction would be misread. Attention then shifted from the learning task to looking like a good student.

Twenty-five mentioned peer gaze, judgment, and embarrassment. If AI tells a teacher “this student needs help,” and the teacher walks over, that is a social signal in class. It can make students feel ashamed and can reveal to peers who is struggling.

Thirty-two participants worried that systems cannot adapt to different learners—including neurodiversity, disability, cultural difference, and everyday mood variation. Some noted that autistic or ADHD students’ eye contact and expression may not match system expectations; others worried about cultural bias in facial recognition. The core issue: once educational AI treats bodily performance as learning state, it easily misreads difference as a problem.

⚖️ The ethical issues students care about most

In ethical rankings, autonomy and privacy came first. Across 18 ethical items, mean privacy was 4.14 and mean autonomy 4.10, clearly above fairness and related dimensions. Across 1,980 pairwise comparisons, autonomy won 515 times and privacy 462; the Bradley–Terry model likewise put autonomy’s priority probability at 0.39 and privacy at 0.27—above transparency, learning benefit, accuracy, and fairness.

The result is instructive. Students did not put “maximize learning outcomes” first. They asked first: Can I choose? Can I opt out? Will my struggles and emotions be visible to others? How will the data be used?

Fairness ranking lower does not mean fairness is unimportant. The authors note that students may have weaker awareness of algorithmic bias, or may first want to avoid being pulled into a sensing system at all. When a system already feels invasive, students prioritize exit and control over debating internal fairness.

✅ How strong is the evidence

The paper’s strength is that it cleanly separates system features—whether sensing is used, sensing mode, and intervention form—while collecting both quantitative ratings and open responses. The 132 participants spanned majors and degree levels; scale reliability was high; ethical ranking used pairwise comparisons that forced trade-offs, not only average ratings.

Its evidence boundary is also clear. The study used video scenarios, not long-term real use. Scenario experiments suit capturing first reactions and value judgments, but cannot fully simulate being stuck on a problem and urgently needing help. Conclusions also come mainly from Australian university students and should not be generalized directly to K–12, other countries, or different regulatory cultures.

🚦 Implications for educational AI design

The article’s advice for educational AI is direct: do not treat sensing capability as automatic progress. Stronger cameras, gaze tracking, and expression recognition do not equal better teaching. If students become anxious, perform focus, and fear mistakes because they are watched, the system is already harming the learning experience.

The authors propose five design directions. First, minimize invasive sensing; prefer low-intrusion signals such as task time and platform interaction. Second, system prompts should be light, customizable, and refuseable—ideally asking first whether help is wanted. Third, teacher intervention should not be summoned directly by the system; students should confirm, or first receive a private notice. Fourth, consent should not be a one-time checkbox at first use, but dynamically asked at key tasks. Fifth, before notifying a teacher, the system should give students a private feedback channel so they can adjust on their own.

What the paper truly challenges is a technology-centered story: if AI can sense more accurately, it can intervene better. Student feedback shows that “good intervention” in education must be useful, private, and controllable. Without the last two, even highly personalized help can become surveillance.

📚 Key terms

👁️ AI sensing: inferring attention, emotion, or learning state from signals such as gaze, expression, and behavior.

🛟 AI intervention: providing help from those inferences—screen prompts, or alerting a teacher to step in.

🧭 Learning autonomy: whether students can decide when to accept help, whether to be sensed, whether data are shared, and how to pace their own learning.

🔒 Social privacy: not only whether data leak, but whether struggles and emotional states become visible to teachers or classmates.

⚖️ Value trade-offs: educational AI often balances personalization, accuracy, privacy, autonomy, and fairness. This paper shows how students rank those values.

📖 Reference

Han, B., Ma, Y., Coghlan, S., McKay, D., Buchanan, G., & Smith, W. (2026). AI sensing and intervention in higher education: Student perceptions of learning impacts, affective responses, and ethical priorities. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. ACM. https://doi.org/10.1145/3772318.3790360