1111. Aileen Nielsen, Stavroula Skylaki, Milda Norkute, and Alexander Stremitzer, Building a Better Lawyer: Experimental Evidence that AI Can Increase Legal Work Efficiency

Rapidly improving artificial intelligence (AI) technologies have created opportunities for humanmachine cooperation in legal practice. We provide evidence from an experiment with law students (N=206) on the causal impact of machine assistance on the efficiency of legal task completion in a private law setting with natural language inputs and multidimensional AI outputs. We tested two forms of machine assistance: AI-generated summaries of legal complaints and AI-generated text highlighting within those complaints. Compared to no AI assistance, AI-generated highlighting reduced task completion time by 30% without any reduction in measured quality indicators. AI-generated summaries produced no change in performance metrics compared to no AI assistance. AI summaries and AI highlighting together improved efficiency but not as much as AI highlighting alone. Our results show that AI support can dramatically increase the efficiency of legal task completion, but finding the optimal form of AI assistance is a fine-tuning exercise. Currently, AI-generated highlighting is not readily available from state-of-the-art, consumer-facing large language models, but our work suggests that this capability should be prioritized in the development of legal AI products.

1111. PDF

Scroll to Top