CHI 2024 | If AI Knows Less, Do Students Explain More Deeply?

If a student’s task is to teach an AI, how smart should that AI be?

Intuitively, stronger model ability should mean a better educational experience. But once learners reverse roles and teach the AI, trouble appears: a model that already knows everything and can fill in mistakes will quickly take over the conversation. Students only need to tell it conclusions, not why — and then rarely expose where their own thinking is incomplete.

The CHI 2024 paper Teach AI How to Code turns this tension into a testable system design. Instead of adding more knowledge to the large model, the authors make it answer only from an editable knowledge state, then schedule it to regularly ask why and how.

What they isolate is a variable often ignored in educational AI: to help people learn, how should an AI’s capability be constrained, exposed, and scheduled?

01 When AI answers too well, learners have nothing left to teach

Learning by teaching others is usually called Learning by Teaching. Explaining concepts, answering follow-ups, and correcting the other party’s misunderstandings force the teacher to reorganize knowledge and make it easier to notice what they have not made clear. Traditional teachable agents use exactly this path, putting students in front of a virtual learner who needs help.

Large models lower the cost of building teachable agents, but bring a new problem: they already know the answer. The authors first had 15 participants unfamiliar with binary search talk with a baseline AlgoBo played by GPT-4. Across 31 dialogues totaling 546 messages, 244 were knowledge-telling; only 15 involved reasoning, extension, or co-construction.

Those numbers do not finish the problem. Two-thirds of participants directly said AlgoBo seemed too competent, making the teaching process awkward; the number of comprehension statements it produced was about twice that of participants. Students thought they were teaching a beginner, but faced an expert who casually filled every gap.

This formative study also exposed three teaching anti-patterns: students directly ordered the agent to change code, fed it complete answers, or gave conclusions without explaining reasons. From that, the authors set concrete design goals: the agent needs controlled unknowns and misconceptions, should actively ask why and how, and should remind students during dialogue to check their own teaching.

02 TeachYou does not make the model play dumb — it takes its capability apart

The full system combines the teachable agent AlgoBo with the learning environment TeachYou. Learners first read binary-search materials and complete exercises, then teach AlgoBo in sequence to understand the concept, write code, and discuss algorithmic efficiency and applications. Figure 2’s interface puts learning goals, agent dialogue, learner code, agent code, and test results on one screen.

The intervention behind it has three parts. Reflect-Respond manages what AlgoBo knows right now; Mode-shifting switches it between receiving help and actively probing; Teaching Helper checks how the learner is teaching. The three constrain knowledge, dialogue pace, and teaching behavior respectively.

The authors did not pin hopes on a single role prompt to play a novice. Models easily call pretrained knowledge and may also forget, in long dialogues, what they are supposed not to know. So the teachable agent’s capability is split into an external state and interaction rules, and learners get a chance to see and change this student’s knowledge boundary.

03 An external knowledge state controls what the model knows

The Reflect-Respond pipeline maintains a JSON knowledge state for AlgoBo, recording factual knowledge and code implementation separately. After each dialogue turn, the Reflection stage extracts information from what the learner just said and updates that state; the Response stage retrieves only state relevant to the current question and generates an answer from it. If the state has no answer, AlgoBo asks the learner to keep explaining.

As Figure 3 shows, the authors did not modify model parameters. They placed a readable, editable, continually updated cognitive boundary outside the model. Teachers can swap in a different knowledge state to simulate learners at different levels, or inject specific misconceptions; the same large-model instance only handles understanding and expression.

Technical evaluation tested three questions: whether knowledge level can be reconfigured, whether it holds after unrelated dialogue, and whether it updates after receiving teaching. On three topics — binary search, merge sort, and breadth-first search — the authors prepared 9 multiple-choice questions per topic covering understanding, implementation, and analysis; four initial states ranged from fully blank to gradually adding facts and correct code.

Results largely matched expectations. An empty state missed every question; with facts only, the agent could answer some concept questions but not implementation ones; after code was added, it began solving implementation questions, and correct code outperformed incorrect code. Random unrelated dialogue largely left knowledge level unchanged; correct teaching raised scores on all three question types.

But control was incomplete. Incorrect teaching sometimes also raised scores, because partially correct information embedded in it was enough to trigger model inference. The authors also found that tiny prompt changes affected whether the agent could answer a given question. An external state can lower the probability that the model calls prior knowledge, but cannot lock it out with program-like privileges.

Figure 6 shows how technical evaluation writes teaching content back into the knowledge state, then uses questions to test whether the agent truly changed. It verifies whether the agent can be configured and taught — not whether students have learned.

04 What really drives deeper explanation is when AI starts probing

Keeping the agent ignorant is not enough. Formative work already showed that students easily stop at telling answers. Mode-shifting therefore has AlgoBo switch into questioner mode every three messages: in concept and programming stages it asks why; in discussion stages it asks how or requests related examples, and keeps probing until explanations are sufficient.

In Figure 4, the agent no longer only says it understands; it pushes the teacher back into the reasoning process. Why must binary search update the bounds? How does that operation affect efficiency? The value of the questions is not to stump students, but to stop dialogue from ending too early.

Teaching Helper intervenes from the other side. It labels recent dialogue as imperative correction, direct answer-feeding, insufficient explanation, or normal teaching, and gives matching prompts. For the first two behaviors, red feedback temporarily blocks sending so the learner first chooses a more suitable teaching strategy; to avoid frequent interruption, feedback appears at most once every six messages.

Figure 5 also exposes limits of this mechanism. Feedback depends on predefined dialogue types; green suggestions sometimes only remind the learner to keep explaining and may not fit the current content. It can pull people off obvious shortcuts, but still cannot read the teaching strategy each student is using.

05 A 40-person study shows dialogue deepened — not that grades rose

The formal user study recruited 40 algorithm beginners, mean age 24, 25 men and 15 women. Researchers randomly assigned 20 to a baseline version and 20 to full TeachYou. Both groups had Reflect-Respond; the difference was that the full version also provided Mode-shifting and Teaching Helper. Groups did not differ significantly in initial self-ratings or practice time.

The clearest result appeared in the problem-solving stage. Knowledge-building message density was 3.5% ± 6.6% in the baseline group and 8.4% ± 7.1% in TeachYou, two-tailed p=0.03, Cohen’s d=0.71. Ratings for discovering new knowledge also rose from 3.25 ± 1.71 to 4.95 ± 1.70, p<0.01, d=1.00.

That shows that after adding probing and teaching feedback, participants more often discussed boundary conditions, efficiency, related algorithms, and practical uses. It also changed the agent’s role sense: rating AlgoBo as a struggling student rose from 3.15 to 4.60, p=0.01, d=0.93.

Dialogue also grew markedly longer. Baseline exchanges averaged 17 ± 7.7 messages; TeachYou reached 43 ± 18.5; cognitive load did not rise significantly. For a system that asks students to explain, longer is not necessarily less efficient — explanation and probing need space. But the paper does not show that this extra dialogue ultimately converted into higher exam scores.

That evidence boundary must be held. The study measured dialogue quality, system logs, and self-report; it did not set pre/post tests to compare learning gains directly. The paper itself lists this as a limitation. So TeachYou supports the claim that students explained more deeply and felt they discovered more new knowledge — not that students had already learned better.

06 Good educational AI does not have to be smartest at every moment

By this point in the paper, whether AlgoBo “knows” binary search is no longer the main takeaway. More worth keeping is a design principle: model capability in educational settings needs calibration. Whether an agent is useful depends not only on how many questions it can answer, but on when it should not know, when it should probe, and whether its knowledge changes can be understood by the learner.

The paper also leaves several open questions. Probing every three messages is only a heuristic from the pilot; some participants felt it interrupted an otherwise teaching rhythm. Teaching Helper felt repetitive for people who already taught well. AlgoBo’s response latency was about 30 seconds — hard to ignore in a real classroom.

Generalization is similarly limited. The user study centered only on binary search; technical evaluation covered only three algorithm topics. Prompts and classifiers tuned for procedural knowledge may not suit math, physics, or declarative knowledge. The sample is only 40 people, individual differences are large, and real-classroom and long-term use data are missing.

Next steps need the system to judge when to step back and when to press harder, and to let learners adjust agent knowledge and intervention strength from their own baseline. Making fixed probe frequency more aggressive does not solve that. An AI that can teach certainly matters; an AI that leaves people room to teach may be harder to design.

Paper information

Original title: Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Education

Authors: Hyoungwook Jin, Seonghee Lee, Hyungyu Shin, Juho Kim

Affiliations: School of Computing, KAIST; Stanford University

Venue: CHI 2024, 11–16 May 2024, Honolulu, USA

Paper link: https://doi.org/10.1145/3613904.3642349

Key terms

Learning by Teaching: Learners organize their own knowledge by explaining to others, answering follow-ups, and correcting misunderstandings. This paper has students teach AI programming to elicit deeper self-explanation.

Teachable Agent: A virtual agent that learners can teach. Its point is not to answer for students, but to play a learning object that needs to be taught.

Knowledge-building: Going beyond restating existing conclusions to explain reasons, derive relations, compare methods, or form new understanding. This paper measures its density via dialogue coding.

References

Jin, H., Lee, S., Shin, H., & Kim, J. (2024). Teach AI how to code: Using large language models as teachable agents for programming education. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. ACM. https://doi.org/10.1145/3613904.3642349