Speaker: Anudeex Shetty (U of Melbourne)

Time: Friday, November 01, 16:15 to 17:45 

Location: Keplerstraße 2, room 181  &  Zoom: zoom.us/my/lea.frermann 

Title: Who Wrote This? Text Forensics Across Human And Machine Authors

Abstract: Authorship attribution is a longstanding problem for human authors and is important for maintaining digital integrity. With the rapid rise of large language models (LLMs), the problem now extends to detecting and attributing AI-generated text. However, current detection methods struggle to generalise across languages, out-of-distribution domains, and realistic settings where humans and AI co-author text. In this talk, we will cover our recent EMNLP work on generalisation in authorship attribution and an upcoming INLG shared task. We will then discuss our future process-informed framework for AI detection in contemporary human-AI co-authorship. We argue that writing is not just a product but a process, and that the existing AI detection ecosystem is fundamentally limited because it builds benchmark datasets and detection methods without considering the writing process.

Speaker bio: Anudeex Shetty is a PhD candidate at the University of Melbourne, supervised by Jey Han Lau, Qiongkai Xu, and Olya Ohrimenko. His research focuses on text forensics: detecting and attributing LLM-generated text and watermarking LLMs. He also works on LLM safety and pluralistic alignment. His research has appeared in venues such as ACL and EMNLP. Before his PhD, he completed a Master of Computer Science at the University of Melbourne and previously worked as a Software Engineer at Uber R&D, building high-throughput, low-latency fraud-detection systems.

Speaker: Damian Curran (U of Melbourne)

Time: Friday, October 30, 10:15 to 11:45 

Location: Keplerstraße 2, room 181  &  Zoom: https://zoom.us/my/lea.frermann 

Title: A Dataset of Motion-Response Text Pairs from US District Court Cases for Legal Judgment Prediction 

Abstract: Many existing Legal Judgment Prediction (LJP) datasets are based on test conditions which do not resemble real-world contexts, in that both the input to the task and the predicted case outcome labels are drawn from the same document. In other words, the outcome was known at the time the input was written. By utilizing the chronological event logs of court cases in US district courts, called dockets, we create a 1.6k instance outcome prediction dataset called MOP-2026 where the text inputs predate the outcome. We describe the collection, filtering and labelling of the pairs of opposing argument text and define a new subtask of LJP called motion-outcome prediction with a ternary label set. We provide baseline results and measure the performance of a variety of LLMs on the task. While exceeding naive baselines, even the latest LLMs perform below most reported metrics in LJP, supporting recent commentary that valid test conditions greatly impact predictive performance, and that many earlier LJP systems appear to overstate predictive capability in real world settings. We close with a discussion of the inherent challenges of LJP. 

Speaker bio: Prior to pivoting into a career in technology, Damian practiced as a commercial lawyer in Melbourne and London. In 2024 he commenced his PhD in computer science at the University of Melbourne, exploring the challenges of using AI in the legal industry. He has honours degrees in Law and Civil Engineering and a Masters in Artificial Intelligence. He consults on AI deployment in legal institutions, works with the University of Melbourne's Centre for Digital Ethics and AI, and has taught across its computing, law and business schools.