A single-semester pilot at Dartmouth College suggests the right way to use LLMs in a course is not as a chatbot a student opens when stuck, but as a quiet grader sitting behind every paragraph of the textbook. According to a June 28, 2026 paper by Jonah Bard presented at the iTextbooks’26 workshop in Seoul, an AI-graded reading platform called Phosphor was associated with learning gains between 0.71 and 1.30 standard deviations on the cumulative final exam for MATH 010. The catch, and it is the only one that matters, is that the effect only showed up on the lessons where Phosphor required students to type out their own answers. When the researcher dialled the lessons down to multiple-choice format, the gains vanished.
That result lands in the middle of a live debate. Earlier work cited in the paper - a 2025 PNAS study by Bastani and colleagues - reported that unfettered GPT-4 access in a high school math class actually harmed subsequent performance by 17 percent once the tool was taken away. A 2026 survey from the Higher Education Policy Institute found 94 percent of UK university students reported using generative AI on assessed work, up from 53 percent two years earlier. Phosphor threads the needle: students did not need to choose between using AI and learning. The system did not give them a chatbot to lean on; it graded them.
What Phosphor Actually Did
Phosphor wraps a statistics curriculum as a web app. Every lesson has a 15 to 20 question quiz bank; each time a student finishes a chapter, Phosphor pulls four questions at random. Multiple-choice auto-grade. Constructed-response questions go to Claude Sonnet 4.6, which grades them against instructor-supplied rubric criteria and returns a pass-or-fail judgment with a short explanation.
The platform was deployed in three sections of MATH 010 (Introductory Statistics, a non-major course for underclassmen) in Spring 2026. Of 151 enrolled students, 143 finished the term. Phosphor was entirely optional and ungraded.
By the end of the term, 90.2 percent of enrolled students had logged in at least once. The median student reached 22 of 24 lessons and completed roughly 46 percent of all quizzes - which Bard treats as a lower bound on actual reading, since completing a quiz presupposes reading the chapter. Student- and instructor-reported baselines for the same course’s traditional reading sat at 10 to 15 percent. That gap is the most striking single number in the paper.
The Size of the Effect
The headline effect sizes are bracketed, on purpose. Using a Tobit model that handles the 27 percent of students who maxed out the Final, full dosage of Phosphor predicted a 14.7-point gap on the 0 to 100 Final Exam scale, or 1.30 SD, between the most engaged and the disengaged baseline. When Bard added controls for prior midterm performance, the gap roughly halved to 8.0 points, or 0.71 SD. The paper frames 0.71 SD as a conservative lower bound - the midterm control partly absorbs learning Phosphor itself produced earlier in the term - and 1.30 SD as a likely-inflated upper bound driven by self-selection.
The largest stand-alone contrast is the Module Review. Students who passed all three scored 7.1 points higher on the Final, with Cohen’s d of 0.66 and p below 0.0001, the only comparison that survives the paper’s Holm correction for multiple tests. A cleaner within-module check confirms the direction: in Module 2, passing the review predicted a 6.1-point advantage on Midterm 2 (d=0.39), with content, timing, and cohort held fixed.
The most informative pattern is what happens when quiz format changes. Bard designed Phosphor iteratively, and Module 2 was switched mid-term to multiple-choice-only quizzes in response to student complaints that the LLM grader was rigid. Module 1 and Module 3 kept 40 percent constructed-response questions in the mix. The dose-response math is clean. In Module 1, every additional lesson completion predicted about 1.6 additional percentage points on Midterm 1, R squared 0.123, p below 0.001. In Module 2, the within-student slope collapses (R squared 0.001). On the cumulative Final, each completion is worth roughly 0.4 additional points, R squared 0.091, and the loss lines up exactly with the format change.
The interpretation Bard offers is that students were doing the same quizzes with similar engagement; what changed was how much cognitive work each quiz demanded. The AI contribution is not a new pedagogical claim - short-answer quizzes with feedback have outperformed multiple choice in cognitive psychology for decades - but that LLM grading makes constructed-response grading at scale feasible in the first place.
What the Study Cannot Tell Us
This is an observational pilot at a single selective institution. There is no randomised control group; the comparison is between students who chose to engage with Phosphor and those who chose not to. Bard flags this directly: “the most engaged students are also the most able and motivated, we control directly for prior achievement … which attenuates the gap.” The across-modules MCQ-versus-CRQ comparison is also confounded by content domain, timing in the term, and the simultaneous introduction of the Module Review.
Phosphor’s in-platform RAG chat sidebar saw almost no use: 72 queries across 143 students, with only 14 students submitting more than one. That lines up with a Khan Academy report Bard cites that roughly 15 percent of users regularly engage with their supplementary chatbot. A controlled study that randomised students into quiz-format conditions - MCQ only, CRQ only, no platform - is the natural next step, and Bard names it explicitly as future work. Until that runs, the cleanest summary is the paper’s own: high engagement and measurable efficacy are simultaneously achievable, but only when the assessment format demands active generation. LLM grading makes that feasible. It does not, on this evidence, replace good assessment design.
What This Means
For educators, the immediate read is that the bottleneck on AI-augmented teaching has quietly moved off the model. Schools and edtech vendors have spent two years arguing about which chatbot to put in front of students; Phosphor suggests the higher-impact use of the same LLM is behind the curriculum. That shifts the procurement question from “which model do we license” to “do we have an authoring team that can write rubric criteria and a course map that can survive iterative revision.”
For parents and students, the 94 percent figure from the HEPI 2026 survey tracks how universal AI tool use has become. Bastani et al.’s warning that unguided AI tool use can degrade learning still stands. Phosphor does not contradict it; the harm Bastani measured comes from handing students a chatbot with no pedagogical scaffolding, not from the LLM itself - the same dependency pattern our coverage of cognitive offloading in March 2026 flagged, where students who quit using AI performed worse than those who never started. A tool that grades you is not the same as a tool you ask for the answer.
The Bottom Line
Phosphor, a Dartmouth-built reading platform that uses Claude Sonnet 4.6 to grade student-written answers in real time, was associated with final-exam gains of 0.71 to 1.30 standard deviations across 143 students in MATH 010 in Spring 2026. The effect only appeared on lessons that required constructed responses; multiple-choice lessons did nothing. The study is observational and self-selected, so 0.71 SD is the honest figure to quote and 1.30 SD the one to be most cautious about. The design lesson, that LLMs work better behind the textbook than in front of it, is the part that survives even if the specific effect size does not.