CHI 2026 | AI Helps Teachers Write Feedback — Drafts Improve, But Do Students Learn?

A student proposes a taxation scheme in an assignment; the teacher’s comment asks him to expand the plan and strengthen the argument. The problem is: he already wrote that he would tax — what is still missing next? One AI-assisted comment in the paper narrows the ask: how large should the tax amount be, and why would that size affect the governance of negative externalities? Both comments push him to revise; the latter lets him know where to start.

The University of Michigan brought this kind of feedback into an introductory economics course with 354 students. After teaching assistants reviewed and adopted AI suggestions, students’ revised drafts scored about 5 percentage points higher in quality than the human-only feedback group. But on the knowledge post-test that followed, the two groups showed no significant difference. Time TAs spent grading assignments also did not decrease.

Making an assignment more complete and being able to solve the next problem independently still sit a distance apart. This paper produced an effect on the former and left the latter in the discussion. When reading it, both sides deserve time.

01 Finding the error still does not mean feedback is easy to write

This course’s writing tasks ask students to analyze real problems with economics concepts. One assignment discusses market changes and policy choices after oat-milk preference rises; another asks students to identify market failure from news and propose a solution. Scoring focuses on whether concepts are used correctly and whether analysis is adequate; literary flair and narrative rhythm are not in the main assessment scope.

Before the study began, the team shadowed 5 experienced TAs across 20 one-hour think-aloud interviews. TAs usually read through first, then take the rubric item by item to find evidence, make judgments, and write comments. One student paragraph may scatter several scoring points; the same concept may also appear in different places in the essay. TAs also switch among papers, scoring sheets, past comments, and meeting notes — sometimes with three or four browser tabs open at once.

After spotting the error, the comment still has to let the student know how to revise, without writing the answer for them. Giving the correct conclusion outright is easy; writing only “please analyze further” is also easy. The hint that sits between those two is what takes work. One TA said in an interview that they also wanted to use guiding questions more often — it is just not that easy to write.

Past comments can save time wording from scratch, yet may not fit the assignment in front of you. The tax example at the opening is typical: the student already chose a policy; the next step needed is discussing policy parameters. Continuing to ask for “a persuasive solution” does not tell him which step was missing this time.

02 The final comment box starts empty — TAs decide item by item what to send

FeedbackWriter uses GPT-4o to complete three steps for each rubric item: locate sentences where the student tried to address that requirement, judge whether the rubric is met, then draft a comment. In Figure 1, the left side is the student essay; the right side arranges feedback by scoring item. Clicking a comment scrolls the essay to the corresponding evidence; clicking a highlighted sentence in the essay can also find related comments.

What TAs see at once includes the scoring requirement, original sentences, AI judgment, and candidate comments. When there is disagreement, they can change “met” to “not met,” or correct the highlight range. The final comment box and score default to empty and need explicit TA confirmation. AI suggestions and teacher-provided past comments appear side by side; TAs can adopt, edit, or write their own.

To send a suggestion to the student, the TA must first click adopt once. That arrangement at least guarantees final content is not filled in automatically; it does not guarantee every click was preceded by thorough thought. More practical help comes from the original sentences beside it: when a TA really wants to verify, they need not re-search the whole assignment.

The harder preparation happens outside the interface. The team repeatedly revised the rubric with the course instructor, writing out course background the teacher already knows by default. For example, “adequately analyze welfare changes” needs to say which welfare items, how deep the explanation should go, and which alternative phrasings are acceptable. The model has not sat through this course; a few seemingly clear scoring requirements can easily admit another reading. Both groups of TAs ultimately could use the same rubric and historical comments.

03 The comparison was trained TAs — revised drafts still improved

The classroom trial ran from January to April 2025, involving 354 students and 11 TAs. All TAs had earned an A or A+ in this course, completed a semester of feedback training, and during the term also joined weekly grading calibration. They formed a more persuasive control than “no feedback.”

Randomization happened at the level of TAs and the discussion sections they led. On the first assignment, some sections used AI-assisted feedback and others human-only feedback; on the second assignment conditions swapped, with students staying in their original sections. Figure 2 shows the flow: submit first draft, receive comments, revise and submit final draft, then take a knowledge post-test. Across the semester, 1,366 first and final drafts were processed — that number should not be treated as independent student count.

The human group used an interface matched to FeedbackWriter in layout and common features, and could also consult the rubric, historical comments, and write in-text annotations — only without AI suggestions. That comparison reduced interference from “the new system is simply more convenient” itself.

As Figure 3 shows, the AI-assisted group’s mean first-to-final draft quality score rose from 47% to 69%; the human group from 50% to 67%. Because starting points differed, subtracting the two final-draft means directly is not the intervention effect. Using a mixed-effects model controlling first-draft quality, assignment differences, and TA first-draft scores, and accounting for student and TA individual differences, the study estimated AI-assisted final-draft quality about 5 percentage points higher, Cohen’s d = 0.50. The authors interpret that as roughly meeting two more rubric requirements.

This “quality score” needs a clear caveat. It comes from GPT-4o judging rubric items one by one and computing a score — not unified human blind grading. The authors had experts spot-check 60 essays totaling 2,100 judgments; model accuracy relative to experts was 84.6%. That validates automatic scoring to a degree, but is not enough to fully rule out bias: generating feedback and assessing essays used the same model and scoring pipeline, so further checks are still needed on whether results favor revisions this feedback style elicits.

04 Comments got longer — and it became easier to specify the next step

AI-assisted feedback averaged 286 English words per essay; the human group averaged 157, and covered more rubric items. Word count alone makes it hard to judge whether students benefited. The paper then analyzed feedback ultimately sent to students, splitting it into units aimed at single issues, checking whether it could guide the next step, whether it left room for independent thinking, whether content was accurate, and whether tone supported learning.

In Figure 4, the share meeting actionability was 89.3% versus 75.8%; the share meeting independent-learning guidance requirements was 92.6% versus 82.3%. These labels were coded with GPT-4.1 assistance and verified against researcher-annotated samples. So 92.6% means the feedback text met the corresponding standard — it cannot be read as 92.6% of students improving independent learning ability.

Concrete comments illustrate the difference more easily than metric names. Some students confused a change in demand with a change in quantity demanded. Human feedback pointed directly to the correct concept; one AI-assisted comment first asked the student to distinguish the two concepts, and the TA then added a suggestion to draw a water-market tax graph and observe price and quantity changes. The student thus gained an executable checking step.

Still, TAs also deleted places where AI wrote too fully. For requirements that listing a few concepts could complete, they removed lengthy explanations; on final drafts, continuing a string of next-step suggestions may also be unfitting. In interviews, some TAs noted students are busy and overly long comments are easily skipped. Whether more text becomes more help still depends on how much information this revision truly needs.

05 TAs edit AI — and merge fragmentary comments into one

Logs show 11.3% of AI rubric-item judgments were overturned by TAs. What this measures is human–model judgment disagreement — it cannot be taken directly as the model’s true error rate. The study also recorded 806 cases of continuing to edit comments after adopting a suggestion; those suggestions included AI-generated content and historical comments.

What TAs specifically changed is more worth watching than how many they adopted. Sometimes the AI hint was too indirect — the student was only one term away but could not see where — so the TA added a clearer follow-up question. Sometimes the student misunderstood the whole analysis frame; reminding scoring points one by one instead scattered attention, so the TA merged several comments and first had them address that larger conceptual misunderstanding.

Another number easy to miss: highlight positions were adjusted only 11 times. TAs explained that after finding evidence and finishing judgment, further correcting highlight boundaries was not very useful. Judgment affects the score; comments go to the student — they were willing to spend time on those two. As for highlights that already helped them find the original sentences, being a bit off was fine.

The paper did not include a pure-AI feedback group, so it cannot answer what happens if TA review is removed entirely. What it validates is a workflow with trained TAs who have course knowledge. Even “time saved” needs restraint: grading time in the logs was similar across groups, though interviews had people feel some steps were easier. The benefit currently observed is providing fuller feedback in similar time and improving revised drafts.

06 Drafts improved — the post-test has not yet shown extra gain

The day after final drafts were submitted, students could take an optional extra-credit knowledge quiz with multiple-choice and open-ended items; graders did not know students’ condition assignment. After the two assignments, 262 and 305 people participated respectively. The AI-assisted group’s mean post-test score was 59%, the human group 56%, as Figure 5 shows. The paper’s adjusted model found no significant condition difference, p = 0.31.

It is worth another look at this model: it also includes final-draft quality as a control variable, and final-draft quality may itself be affected by feedback. So this non-significant result cannot be stretched into “AI is completely useless for learning,” nor is it proof the two groups’ learning effects are equivalent. The more careful conclusion is that this study reports improved revisions, but has not yet established sufficient evidence of extra knowledge post-test gain.

The authors propose that one revision gives too little practice opportunity per knowledge point. For example, a student rewriting a tax-policy analysis once does not mean they can already complete the same reasoning independently on another market problem. That is an explanation awaiting follow-up research; the paper does not directly prove why students failed to pull apart on the post-test.

Students’ own feelings were not collected. Students knew the course used AI tools, but did not know which specific comments had AI assistance. Whether they trusted those suggestions, were willing to read them, or how they understood the TA’s role — assignment scores cannot answer. This method also fits course tasks that can clearly list concept requirements better. For open argumentation or narrative writing, judging quality may need another approach.

I would continue trying this in such courses: with the same grading time, TAs give fuller guidance, and students’ revised drafts improve. But the next round of evaluation needs to go one step further. After correcting the assignment, switch the situation and let the student themselves judge which concept to use and why to analyze that way. That answer has a chance of telling us how much they carried away from the comments.

Paper information

Original title: AI-Mediated Feedback Improves Student Revisions: A Randomized Trial with FeedbackWriter in a Large Undergraduate Course

Authors: Xinyi Lu, Kexin Phyllis Ju, Mitchell Dudley, Larissa Sano, Xu Wang. All authors are from the University of Michigan.

Published at CHI 2026. Paper link: https://doi.org/10.1145/3772318.3791121

Key terms

AI-assisted feedback: In this paper, AI provides candidate suggestions to TAs; TAs review, edit, and decide final content; what students receive is comments confirmed by TAs.

Rubric: Breaks assignment requirements into checkable items. This paper mainly checks economics concepts and their application, not overall essay style.

Cohen’s d: An effect size expressing between-group difference in standard-deviation units. Here d = 0.50 describes revision difference; it cannot be read as scores rising 50%.

References

Lu, X., Ju, K. P., Dudley, M., Sano, L., & Wang, X. (2026). AI-mediated feedback improves student revisions: A randomized trial with FeedbackWriter in a large undergraduate course. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26). Association for Computing Machinery. https://doi.org/10.1145/3772318.3791121