Papers by Frank Schilder
Beyond Pointwise Scores: Decomposed Criteria-Based Evaluation of LLM Responses (2025.emnlp-industry)
Copied to clipboard
| Challenge: | DeCE is model-agnostic and domain-general, requiring no predefined taxonomies or handcrafted rubrics. |
| Approach: | They propose a decomposed LLM evaluation framework that separates accuracy and recall from accuracy and relevance. |
| Outcome: | The proposed framework achieves stronger correlation with expert judgments than traditional metrics and pointwise LLM scoring. |
Litigation Analytics: Extracting and querying motions and orders from US federal courts (N19-4)
Copied to clipboard
| Challenge: | a detailed manual analysis of a docket could provide valuable information for the suit and the respective judge. |
| Approach: | They applied machine learning and machine learning to extract and aggregate docket statistics . they used a search engine to query the data in real time and a question-answering interface . |
| Outcome: | The proposed method extracts information from 8 million federal dockets and keeps up with newly closed docketes. |
Exploring the Effectiveness of Prompt Engineering for Legal Reasoning Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that Chain-of-Thought (CoT) prompts improve tasks such as arithmetic and common-sense reasoning. |
| Approach: | They evaluate CoT prompts and various prompting strategies for legal reasoning tasks . they find that the best results are achieved with prompts derived from specific legal reasoning techniques . |
| Outcome: | The proposed approaches improve the COLIEE entailment task on the Japanese bar exam . the proposed approaches surpass the best system from 2022 with an accuracy of 0.789 . |