Papers by Tobias Schimanski
CHATREPORT: Democratizing Sustainability Disclosure Analysis through LLM-based Tools (2023.emnlp-demo)
Copied to clipboard
Jingwei Ni, Julia Bingler, Chiara Colesanti-Senni, Mathias Kraus, Glen Gostlow, Tobias Schimanski, Dominik Stammbach, Saeid Ashraf Vaghefi, Qian Wang, Nicolas Webersinke, Tobias Wekhof, Tingyu Yu, Markus Leippold
| Challenge: | a lack of transparency in sustainability reporting is a key challenge due to the sheer volume and complexity of sustainability reports . only a few entities worldwide have the resources to analyze these reports at scale . a novel LLM-based system to automate the analysis of corporate sustainability reports is needed . |
| Approach: | They propose a novel LLM-based system to automate the analysis of corporate sustainability reports. |
| Outcome: | The proposed system automates the analysis of corporate sustainability reports. |
ClimRetrieve: A Benchmarking Dataset for Information Retrieval from Corporate Climate Disclosures (2024.emnlp-main)
Copied to clipboard
| Challenge: | Qualitative disclosures typically include narrative descriptions of climate-related risks, opportunities, strategies, and governance. |
| Approach: | They simulate typical tasks of a sustainability analyst by examining 30 sustainability reports with 16 detailed climate-related questions. |
| Outcome: | The proposed model combines expert knowledge with embeddings in a dataset with over 8.5K unique question-source-answer pairs labeled by different levels of relevance. |
ClimateBERT-NetZero: Detecting and Assessing Net Zero and Reduction Targets (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing tools for extracting information about net zero and emission reduction targets have not been used to assess the vast amounts of information about sustainability commitments made by public and private actors. |
| Approach: | They propose a data set and train and release a natural language classifier to detect whether a text contains a net zero or reduction target. |
| Outcome: | The proposed model can be combined with conventional Q&A models to analyze the ambitions displayed in net zero and reduction targets. |
DIRAS: Efficient LLM Annotation of Document Relevance for Retrieval Augmented Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | RAG systems leave out important relevant information (low recall) and excessively related but irrelevant information (high precision) authors propose a manual annotation-free schema that can be used for RAGs with limited performance. |
| Approach: | They propose a manual annotation-free schema that annotates unseen queries with calibrated relevance scores. |
| Outcome: | Evaluators show that DIRAS can achieve GPT-4-level performance on annotating and ranking unseen (query, document) pairs. |
Towards Faithful and Robust LLM Specialists for Evidence-Based Question-Answering (2024.acl-long)
Copied to clipboard
| Challenge: | Evidence-Based QA has proved insufficiently faithful with Large Language Models . a typical application of LLMs is in Evidence-based Question Answering (QA). |
| Approach: | They propose a data generation pipeline with automated data quality filters to fine-tune LLMs for better source quality and answer attributability. |
| Outcome: | The proposed model can synthesize high-quality training and testing data at scale. |