Towards Explainability and Fairness in Swiss Judgement Prediction: Benchmarking on a Multilingual Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Using Swiss Judgement Prediction, we evaluate the explainability of state-of-the-art monolingual and multilingual LJP models. |
| Approach: | They propose an occlusion-based approach to evaluate the explainability performance of legal judgement prediction models using Swiss Judgement Prediction, the only available multilingual LJP dataset. |
| Outcome: | The proposed framework allows us to quantify the influence of lower court information on model predictions, exposing current models’ biases. |
Similar Papers
Legal Judgment Prediction: A Reflection on the State of the Art (2026.acl-long)
Copied to clipboard
| Challenge: | Legal Judgment Prediction (LJP) involves predicting judgment outcomes based on fact descriptions of cases. |
| Approach: | They propose to use argument trees to build automated legal judgment prediction systems that are trustworthy and can be used to predict cases. |
| Outcome: | The proposed model outperforms competitors on standard evaluation datasets and enables pluralistic values to be naturally expressed. |
An Empirical Study on Cross-X Transfer for Legal Judgment Prediction (2022.aacl-main)
Copied to clipboard
| Challenge: | Cross-lingual transfer learning is understudied in legal NLP but not in legal Judgment Prediction (LJP). |
| Approach: | They explore cross-lingual transfer learning techniques on legal JP using a trilingual Swiss-Judgment-Prediction dataset and adapter-based fine-tuning. |
| Outcome: | The proposed methods improve the model’s performance by augmenting the training dataset with machine-translated versions of the original documents, using a 3 larger training corpus. |
Towards Interactivity and Interpretability: A Rationale-based Legal Judgment Prediction Framework (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models lack interpretability due to the neglect of rationale in the prediction process. |
| Approach: | They propose a rationale-based legal judgment prediction framework that follows the judge's real trial logic and provides good interactivity and interpretability. |
| Outcome: | The proposed framework provides good interactivity and interpretability which enables practical use. |
Deconfounding Legal Judgment Prediction for European Court of Human Rights Cases Towards Better Alignment with Experts (2022.emnlp-main)
Copied to clipboard
| Challenge: | Legal Judgement Prediction systems without expert-informed adjustments can be vulnerable to shallow, distracting surface signals. |
| Approach: | They propose to use domain expertise to identify statistically predictive but legally irrelevant information and adopt adversarial training to prevent it from relying on it. |
| Outcome: | The proposed model aligns better with expert rationales than baseline models . the results are compared with an existing benchmark dataset of human rights cases . |
Towards Explainability in Legal Outcome Prediction Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Current legal outcome prediction models do not explain their reasoning in the real world, but human legal actors need to understand the model’s decisions. |
| Approach: | They propose a method for identifying the precedent employed by legal outcome prediction models and a taxonomy of legal precedent to compare human judges and neural models. |
| Outcome: | The proposed model learns to predict outcomes reasonably well, but its use of precedent is unlike that of human judges. |
Neural Legal Judgment Prediction in English (P19-1)
Copied to clipboard
| Challenge: | Recent work on legal judgment prediction has focused on Chinese, but only feature-based models have been considered in English. |
| Approach: | They propose a hierarchical version of BERT which bypasses BERT’s length limitation. |
| Outcome: | The proposed model outperforms existing models in binary violation classification, multi-label classification and case importance prediction. |
From Citations to Criticality: Predicting Legal Decision Influence in the Multilingual Swiss Jurisprudence (2025.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to evaluating the importance of legal cases are manual and resource-intensive. |
| Approach: | They propose a dataset that uses two-tier labels to evaluate case criticality . they use the LD-Label to identify cases published as Leading Decisions and the Citation-L Label to rank cases by their citation frequency and recency. |
| Outcome: | The Criticality Prediction dataset outperforms existing approaches to evaluate case criticality . the proposed model outperformed the existing models in a zero-shot setting . |
LexGLUE: A Benchmark Dataset for Legal Language Understanding in English (2022.acl-long)
Copied to clipboard
Ilias Chalkidis, Abhik Jana, Dirk Hartung, Michael Bommarito, Ion Androutsopoulos, Daniel Katz, Nikolaos Aletras
| Challenge: | Laws and their interpretations, legal arguments and agreements are typically expressed in writing. |
| Approach: | They propose a benchmark to evaluate model performance across legal NLU tasks . they also evaluate several generic and legal-oriented models . |
| Outcome: | The proposed model performs better across multiple tasks than previous models. |
Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration (2023.emnlp-main)
Copied to clipboard
Yiquan Wu, Siying Zhou, Yifei Liu, Weiming Lu, Xiaozhong Liu, Yating Zhang, Changlong Sun, Fei Wu, Kun Kuang
| Challenge: | Recent advances in deep learning have enabled a variety of techniques to be used to solve the LJP task. |
| Approach: | They propose a framework that leverages the strength of both LLMs and domain-specific models in the context of precedents. |
| Outcome: | The proposed framework leverages the strength of both LLM and domain models in the context of precedents. |
FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing (2022.acl-long)
Copied to clipboard
| Challenge: | Using pre-trained language models, we evaluate performance group disparities while none of these techniques guarantee fairness, nor consistently mitigate group disparity. |
| Approach: | They present a benchmark suite of four datasets for evaluating the fairness of pre-trained language models and the techniques used to fine-tune them for downstream tasks. |
| Outcome: | The proposed methods show that performance group disparities are vibrant in many cases, while none of these techniques guarantee fairness, nor consistently mitigate group disparity. |