Worked Examples Are Not Better Just Because They Are More Interactive

📌 Original paper title

Exploring the Design and Impact of Interactive Worked Examples for Learners with Varying Prior Knowledge

👥 Authors and affiliations

Sutapa Dey Tithi, Xiaoyi Tian, Ally Limke, Min Chi, and Tiffany Barnes, all at the Department of Computer Science, North Carolina State University.

🔗 Paper link

https://doi.org/10.1145/3772318.3791631

🧠 What this paper really asks

This CHI 2026 paper addresses a practical question in intelligent tutoring systems: how should worked examples be designed so they keep scaffolding without turning students into spectators?

The strength of traditional worked examples is clear: they present a complete solution and lower cognitive load for novices. The weakness is equally clear. For students with stronger foundations, too much demonstration can become redundant; for weaker students, full problem-solving may be too hard. Instructional effects interact with prior knowledge—what the paper calls aptitude–treatment interaction.

The authors bring the ICAP learning framework into propositional-logic proof tutoring. ICAP’s core claim is that learning engagement can deepen from passive reception, through active manipulation and constructive generation, to interactive co-construction. Dissatisfied with letting students merely “watch examples,” the authors designed two interactive forms: one asks students to repair buggy solutions; the other asks them to complete proof structure with guided prompts.

As shown in Figure 1, the logic tutor used a three-panel interface: a student workspace for constructing proofs on the left, available logic rules in the center, and contextual instructions on the right. The design let the system log each choice, help request, error, and correction for fine-grained behavioral analysis.

🔬 How the authors ran the study

The study was deployed in an undergraduate discrete-mathematics course at a U.S. public research university and included 155 students who completed all seven levels of tasks. Students first took a pretest; the system then used stratified random assignment by pretest score to balance high- and low-prior-knowledge students across conditions.

Training had three conditions. The control group alternated between problem solving and traditional worked examples; the Buggy group alternated between problem solving and buggy examples; the Guided group alternated between problem solving and guided-completion examples. In the posttest, all students independently completed six problems; the system no longer gave next-step hints and provided only immediate feedback on errors.

Figure 2 shows how the four problem types differ. Traditional problem solving asks students to construct proofs independently; traditional examples let students watch a complete solution step by step; Guided examples provide partial proof structure and subgoals for students to fill missing links and rules; Buggy examples provide a partial solution with expert-designed errors for students to judge what is wrong and how to fix it.

The critical design point is not interface novelty but the allocation of cognitive load. Guided examples partly offload generating a full proof onto the system so students can focus on rule relations; Buggy examples ask students to understand correct principles, identify errors, and repair them at once—raising the demand on prior knowledge.

📊 Key findings

First, both interactive-example groups outperformed the control group on the posttest. Mean problem scores were 67.8 for control, 72.3 for Buggy, and 72.4 for Guided. Mixed-effects regression confirmed significant positive associations for both training conditions relative to control: Buggy averaged 4.44 points higher, Guided 4.56.

Figure 3 compares posttest problem scores across groups. The point is not that any interaction works, but that both forms pulled students out of merely watching examples and required them to judge or complete proof steps.

Second, Guided better suited low-prior-knowledge students; Buggy better suited high-prior-knowledge students. Among low-pretest students, Guided learners reached 78.7% posttest rule-application accuracy—significantly above the control group’s 71.8%—and problem time fell from 12.3 minutes in control to 8.6 minutes. Buggy showed no significant advantage for low-prior students.

Among high-pretest students, Buggy’s advantage was clearer. High-prior students under Buggy reached 82.6% posttest rule accuracy, significantly above the control group’s 74.4%; posttest time also fell from 11.2 minutes to 6.3. The paper treats this as a classic instructional-fit effect: students with stable knowledge structures can gain higher-order consolidation from finding and fixing errors, while novices facing buggy examples may lack the capacity to judge what is wrong.

Third, extra training time bought faster, more accurate posttest performance. Buggy and Guided groups spent longer in training than control—about 1.12 and 1.17 hours versus 0.79—yet total tutor time was similar, and both groups finished posttest problems more quickly.

🧩 Behavioral data add a deeper explanation

The authors did not stop at scores. They used first-order Markov models to analyze behavioral transitions across problem types. Buggy examples showed a clear “struggle loop”: students repeatedly switched between erroneous attempts and viewing rules. For high-prior students, that struggle may be productive struggle—difficulty that advances understanding; for low-prior students, it more readily becomes getting stuck.

Guided examples showed another pattern. After errors, students more often requested hints rather than continuing repeated wrong attempts. The paper reports a 44% probability of requesting a hint after an erroneous attempt in Guided. Guided’s value is not merely “giving answers,” but pulling students out of unproductive trial-and-error into manageable steps.

That is the paper’s most memorable point: Interaction that shares a name can produce entirely different cognitive work. Repair trains evaluation and diagnosis; completion trains rule linking and structural understanding. What tutoring systems must do is not make interfaces more complex, but map interactive actions onto clear learning goals.

✅ How strong is the evidence

The evidence is relatively solid: a real-course deployment, 155 undergraduates, pretest-stratified random assignment, independent posttest problems, and Mann–Whitney U tests with Bonferroni correction plus mixed-effects regression. The authors also decomposed problem scores into rule accuracy, completion time, and solution length, avoiding a single-score story.

Effect sizes are not dramatic. Buggy and Guided raised average posttest scores by about 4–5 points over control—valuable instructional design improvements, not a universal transform of learning outcomes. More importantly, differences between low- and high-prior students remind us that average effects can mask fit problems.

⚠️ Where the limits are

The authors acknowledge heavy reliance on interaction logs and a lack of qualitative material on momentary emotion, confidence, frustration, and learning context. The paper can see who got stuck when, but cannot fully explain why.

Another limit is that scaffolding was not perfectly balanced across interventions. PS and Guided had hint mechanisms; Buggy did not, by design, to raise challenge—yet behavioral analysis showed clear struggle loops in Buggy. Future systems may need timely hints or self-explanation when unproductive struggle is detected in Buggy as well.

The task setting also has boundaries. Propositional-logic proofs are open yet rule-clear, closer to programming, mathematics, and science problems. Effects of Buggy and Guided may not transfer directly to writing or design, where correctness is less crisp.

🚦 Implications for educational AI design

The article offers a direct reminder for today’s AI education systems: do not equate “more AI help” with “better learning.” Systems need to judge whether a student currently lacks structure, rule understanding, or challenge.

For novices, a good AI tutor should first reduce task complexity, break problems into achievable subgoals, and let students form rule connections through completion. For students with stronger foundations, the system should offer more frictional tasks—diagnosing errors, comparing solutions, explaining why a step fails.

That also explains the authors’ caution toward generative AI in the discussion. LLMs can generate hints and personalized materials, but if buggy examples are merely randomly wrong, or hints are so fine-grained that students need not think, they undermine the “appropriate difficulty” this paper emphasizes. Educational AI’s key is not unlimited answers, but well-calibrated challenge and support.

📚 Key terms

🧱 Worked example: an example that already shows complete solution steps. It can lower novice load, but may also leave students merely watching.

🧭 ICAP framework: classifies learning engagement as passive, active, constructive, or interactive. This paper uses it to design examples at different levels of cognitive participation.

🐞 Buggy example: an example containing errors that students must find and fix. It better suits learners who already have some foundation.

🪜 Guided example: a partial solution with prompts for students to complete missing rules or links. It better suits weaker students who need structured support.

🎯 Aptitude–treatment interaction: the same instructional method has different effects for different students. The core finding here is that Buggy and Guided fit different prior-knowledge levels.

📖 Reference

Tithi, S. D., Tian, X., Limke, A., Chi, M., & Barnes, T. (2026). Exploring the design and impact of interactive worked examples for learners with varying prior knowledge. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. ACM. https://doi.org/10.1145/3772318.3791631