LLMs – the Good, the Bad or the Indispensable?: A Use Case on Legal Statute Prediction and Legal Judgment Prediction on Indian Court Cases (2023.findings-emnlp)
Copied to clipboard
Shaurya Vats, Atharva Zope, Somsubhra De, Anurag Sharma, Upal Bhattacharya, Shubham Nigam, Shouvik Guha, Koustav Rudra, Kripabandhu Ghosh
| Challenge: | Large Language Models have touched upon many real-life tasks. |
| Approach: | They apply Large Language Models to two popular tasks: Statute Prediction and Judgment Prediction. |
| Outcome: | The proposed model performs well in Statute Prediction and Judgment Prediction on Indian Supreme Court cases. |
Similar Papers
A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated great potential for domain-specific applications, such as the law domain. |
| Approach: | They propose a framework to investigate LLMs' competence in the law domain by using similar cases and multi-choice options. |
| Outcome: | The proposed solutions can be extended to other domains to facilitate evaluations in other domain. |
Enabling Discriminative Reasoning in LLMs for Legal Judgment Prediction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) underperform in legal judgment prediction due to challenges in understanding case facts and distinguishing between similar charges. |
| Approach: | They propose a framework that allows LLMs to discriminate among charges and a judicial reasoning framework to improve their models for effective legal judgment prediction. |
| Outcome: | The proposed framework improves accuracy and efficiency when dealing with complex and confusing charges. |
Legal Judgment Prediction: A Reflection on the State of the Art (2026.acl-long)
Copied to clipboard
| Challenge: | Legal Judgment Prediction (LJP) involves predicting judgment outcomes based on fact descriptions of cases. |
| Approach: | They propose to use argument trees to build automated legal judgment prediction systems that are trustworthy and can be used to predict cases. |
| Outcome: | The proposed model outperforms competitors on standard evaluation datasets and enables pluralistic values to be naturally expressed. |
Legal Judgment Reimagined: PredEx and the Rise of Intelligent AI Interpretation in Indian Courts (2024.findings-acl)
Copied to clipboard
| Challenge: | Prediction with Explanation is the largest expert-annotated dataset for legal judgment prediction and explanation in the Indian context . |
| Approach: | They propose to use an annotated legal judgment prediction corpus to improve models' accuracy . they employ transformer-based models tailored for both general and Indian legal contexts . |
| Outcome: | The proposed system improves the accuracy and explanatory depth of models for legal judgments. |
Neural Legal Judgment Prediction in English (P19-1)
Copied to clipboard
| Challenge: | Recent work on legal judgment prediction has focused on Chinese, but only feature-based models have been considered in English. |
| Approach: | They propose a hierarchical version of BERT which bypasses BERT’s length limitation. |
| Outcome: | The proposed model outperforms existing models in binary violation classification, multi-label classification and case importance prediction. |
Humans or LLMs as the Judge? A Study on Judgement Bias (2024.emnlp-main)
Copied to clipboard
| Challenge: | Proprietary models such as GPT-4, Claude, Gemini-Pro and others are being democratized to improve evaluations of LLMs. |
| Approach: | They propose a framework that is free from referencing groundtruth annotations for investigating **Misinformation Oversight Bias**, **Gender Bia**,**Authority Bia* and **Beauty Bia's** on LLM and human judges. |
| Outcome: | The proposed framework investigates **Misinformation Oversight Bias**, **Gender Bia**,**Authority Bia* and **Beauty Bia' on LLM and human judges. |
Are Large Language Models (LLMs) Good Social Predictors? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies suggest that Large Language Models can generate human-like responses, but it is unclear how well they work and where the plausible predictions derive from. |
| Approach: | They propose to use LLMs to generate human-like responses by mutability and accessibility of social inputs to perform a social prediction task. |
| Outcome: | The proposed model performs well in three realistic settings and a novel social prediction task. |
LLMs and Copyright Risks: Benchmarks and Mitigation Approaches (2025.naacl-tutorial)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized natural language processing, but their widespread use has raised significant copyright concerns. |
| Approach: | This tutorial will provide an overview of relevant copyright principles and their application to AI and examine specific copyright issues in LLM development and deployment. |
| Outcome: | The course will provide an overview of relevant copyright principles and their application to AI, followed by an examination of specific copyright issues in LLM development and deployment. |
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations (2024.emnlp-main)
Copied to clipboard
Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdullah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty, Jimmy Huang
| Challenge: | Large Language Models (LLMs) have gained significant attention due to their capabilities in performing diverse tasks across domains. |
| Approach: | They review the primary challenges and limitations causing inconsistencies in evaluations . early models could generate coherent text but limited to simple tasks . |
| Outcome: | The proposed evaluations are reproducible, reliable, and robust. |
LLM Agents in Law: Taxonomy, Applications, and Challenges (2026.acl-long)
Copied to clipboard
Shuang Liu, Ruijia Zhang, Ruoyun Ma, Yujia Deng, Lanyi Zhu, Jiayu Li, Zelong Li, Zhibin Shen, Mengnan Du
| Challenge: | Large language models (LLMs) have improved the legal domain, but deployment of standalone models faces significant limitations regarding hallucination, outdated information, and verifiability. |
| Approach: | They present a survey of LLM agents for legal tasks and analyze their architectures . they analyze the transition from standard legal LLMs to legal agents . |
| Outcome: | The proposed architectures bridge the gap between technical capabilities and domain-specific needs. |