Economics Professor Roberto Serrano of Brown University has encountered suspicions of widespread generative AI use among students in his "Welfare Economics and Social Choice Theory" course. An anomalous discrepancy in results between homework assignments and in-person exams points to a systemic breach of academic integrity.
What Happened
During the "Welfare Economics and Social Choice Theory" course, the average score for the midterm homework was an anomalous 96%, whereas the historical norm typically ranges between 65–80%. Meanwhile, on the in-person final exam, the average score plummeted to 48.6%. Professor Serrano identified signs of AI usage in the form of unnaturally phrased logical arguments within mathematical proofs.
Context
The situation demonstrates the growing ineffectiveness of traditional assessment methods, such as homework, in the era of advanced LLM accessibility. The ability of models to generate complex mathematical text creates a gap between producing a plausible result and the student's actual understanding of the material.
Why It Matters for the Industry
For the AI industry, this case highlights the critical problem of competency verification. It creates a demand for the development of specialized evaluation tools (evals) capable of detecting patterns of "hallucinatory logic" and specific markers of unnaturally constructed (contrived) arguments. It also stimulates a shift toward "Proof of Process" architectures, where not only the final answer but also the user's reasoning log is evaluated.
Why It Matters for Users
Students and the academic community should consider that even complex mathematical proofs generated by AI leave characteristic traces of unnatural logic that can be detected by experts. This may lead to stricter knowledge control regulations and a return to more rigorous in-person exam formats.
What Is Not Yet Known / Limitations
The provided data does not specify whether official sanctions were applied to the students or how the university plans to change its assessment methodology in the long term.
Sources
Author
Look at AI, Editorial Staff