Papers by Markus Leippold

CHATREPORT: Democratizing Sustainability Disclosure Analysis through LLM-based Tools (2023.emnlp-demo)

Copied to clipboard

Jingwei Ni, Julia Bingler, Chiara Colesanti-Senni, Mathias Kraus, Glen Gostlow, Tobias Schimanski, Dominik Stammbach, Saeid Ashraf Vaghefi, Qian Wang, Nicolas Webersinke, Tobias Wekhof, Tingyu Yu, Markus Leippold

Reasoning

Challenge:	a lack of transparency in sustainability reporting is a key challenge due to the sheer volume and complexity of sustainability reports . only a few entities worldwide have the resources to analyze these reports at scale . a novel LLM-based system to automate the analysis of corporate sustainability reports is needed .
Approach:	They propose a novel LLM-based system to automate the analysis of corporate sustainability reports.
Outcome:	The proposed system automates the analysis of corporate sustainability reports.

ClimRetrieve: A Benchmarking Dataset for Information Retrieval from Corporate Climate Disclosures (2024.emnlp-main)

Copied to clipboard

Tobias Schimanski, Jingwei Ni, Roberto Martín, Nicola Ranger, Markus Leippold

Representation Learning

Challenge:	Qualitative disclosures typically include narrative descriptions of climate-related risks, opportunities, strategies, and governance.
Approach:	They simulate typical tasks of a sustainability analyst by examining 30 sustainability reports with 16 detailed climate-related questions.
Outcome:	The proposed model combines expert knowledge with embeddings in a dataset with over 8.5K unique question-source-answer pairs labeled by different levels of relevance.

ClimateBERT-NetZero: Detecting and Assessing Net Zero and Reduction Targets (2023.emnlp-main)

Copied to clipboard

Tobias Schimanski, Julia Bingler, Mathias Kraus, Camilla Hyslop, Markus Leippold

Language Models

Challenge:	Existing tools for extracting information about net zero and emission reduction targets have not been used to assess the vast amounts of information about sustainability commitments made by public and private actors.
Approach:	They propose a data set and train and release a natural language classifier to detect whether a text contains a net zero or reduction target.
Outcome:	The proposed model can be combined with conventional Q&A models to analyze the ambitions displayed in net zero and reduction targets.

AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators (2024.acl-long)

Copied to clipboard

Jingwei Ni, Minjing Shi, Dominik Stammbach, Mrinmaya Sachan, Elliott Ash, Markus Leippold

Responsible & Trustworthy NLP

Challenge:	generative AI is a counter-measure to misinformation, but factual claim detection suffers from inconsistency in definitions and high cost of manual annotation.
Approach:	They propose a framework that assists in the annotation of factual claims with the help of large language models.
Outcome:	The proposed framework can be used to annotate factual claims with the help of large language models and can work with or without expert supervision.

DIRAS: Efficient LLM Annotation of Document Relevance for Retrieval Augmented Generation (2025.naacl-long)

Copied to clipboard

Jingwei Ni, Tobias Schimanski, Meihong Lin, Mrinmaya Sachan, Elliott Ash, Markus Leippold

Semantic Text Processing

Challenge:	RAG systems leave out important relevant information (low recall) and excessively related but irrelevant information (high precision) authors propose a manual annotation-free schema that can be used for RAGs with limited performance.
Approach:	They propose a manual annotation-free schema that annotates unseen queries with calibrated relevance scores.
Outcome:	Evaluators show that DIRAS can achieve GPT-4-level performance on annotating and ranking unseen (query, document) pairs.

Can Reasoning Help Large Language Models Capture Human Annotator Disagreement? (2026.eacl-long)

Copied to clipboard

Jingwei Ni, Yu Fan, Vilém Zouhar, Donya Rooein, Alexander Miserlis Hoyle, Mrinmaya Sachan, Markus Leippold, Dirk Hovy, Elliott Ash

Language Models

Challenge:	Variation in human annotation (i.e., disagreements) is common in NLP, but it is unclear whether it is possible to model this variation in LLMs.
Approach:	They evaluate the influence of different reasoning settings on LLM disagreement modeling . RLVR-style reasoning degrades performance in disagreement modeling, they find .
Outcome:	The proposed reasoning settings improve LLM disagreement modeling, while RLVR-style reasoning degrades it.

When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLP (2023.acl-long)

Copied to clipboard

Jingwei Ni, Zhijing Jin, Qian Wang, Mrinmaya Sachan, Markus Leippold

Language Models

Challenge:	Multi-task learning (MTL) is a machine learning paradigm where multiple learning tasks are optimized simultaneously, exploiting commonalities and differences across them.
Approach:	They propose a parameter-efficient MTL architecture to improve task aggregation and to include loosely related skills from multiple datasets.
Outcome:	The proposed architecture outperforms single-task learning (STL) and is expected to outperformed it.

Environmental Claim Detection (2023.acl-short)

Copied to clipboard

Dominik Stammbach, Nicolas Webersinke, Julia Bingler, Mathias Kraus, Markus Leippold

Responsible & Trustworthy NLP

Challenge:	a growing number of environmental claims are being made by companies in the face of climate change.
Approach:	They propose a task of environmental claim detection to detect environmental claims at scale . they use an expert-annotated dataset and models trained on this dataset to do this .
Outcome:	The proposed task detects environmental claims in quarterly earning calls . the number of environmental claims has steadily increased since the Paris Agreement in 2015 .

Co-DETECT: Collaborative Discovery of Edge Cases in Text Classification (2025.emnlp-demos)

Copied to clipboard

Chenfei Xiong, Jingwei Ni, Yu Fan, Vilém Zouhar, Donya Rooein, Lorena Calvo-Bartolomé, Alexander Miserlis Hoyle, Zhijing Jin, Mrinmaya Sachan, Markus Leippold, Dirk Hovy, Mennatallah El-Assady, Elliott Ash

Information Extraction & Text Mining

Challenge:	Social scientists often need to develop codebooks that can be reliable but require significant human effort.
Approach:	They propose a mixed-initiative annotation framework that integrates human expertise with automatic annotation guided by large language models.
Outcome:	The proposed framework integrates human expertise with automatic annotation guided by large language models.

Towards Faithful and Robust LLM Specialists for Evidence-Based Question-Answering (2024.acl-long)

Copied to clipboard

Tobias Schimanski, Jingwei Ni, Mathias Kraus, Elliott Ash, Markus Leippold

Question Answering

Challenge:	Evidence-Based QA has proved insufficiently faithful with Large Language Models . a typical application of LLMs is in Evidence-based Question Answering (QA).
Approach:	They propose a data generation pipeline with automated data quality filters to fine-tune LLMs for better source quality and answer attributability.
Outcome:	The proposed model can synthesize high-quality training and testing data at scale.

AI for Climate Finance: Agentic Retrieval and Multi-Step Reasoning for Early Warning System Investments (2026.eacl-demo)

Copied to clipboard

Ario Saeid Vaghefi, Aymane Hachcham, Veronica Grasso, Nakiete Msemo, Chiara Colesanti Senni, Markus Leippold

Information Retrieval

Challenge:	EWS financial flows are opaque and lack standardized labels, structures, and terminology for EWS-related spending.
Approach:	They propose an agent-based Retrieval-Augmented Generation system that uses hybrid retrieval and internal chain-of-thought reasoning to extract relevant financial data and classify EWS investments.
Outcome:	The proposed system outperforms four alternatives on multi-label classification and budget allocation on an annotated CREWS Fund corpus.

Towards Climate Awareness in NLP Research (2022.emnlp-main)

Copied to clipboard

Daniel Hershcovich, Nicolas Webersinke, Mathias Kraus, Julia Bingler, Markus Leippold

Responsible & Trustworthy NLP

Challenge:	Increasing focus on efficient AI and NLP research lacks systematic climate reporting guidelines . a proposed model card would be practical with limited information about experiments and the underlying computer hardware.
Approach:	They propose a model card that is practically usable with limited information about experiments and the underlying computer hardware.
Outcome:	The proposed model card would be usable with limited information about experiments and the underlying computer hardware.

Efficient Test-Time Scaling of Multi-Step Reasoning by Probing Internal States of Large Language Models (2026.acl-long)

Copied to clipboard

Jingwei Ni, Ekaterina Fadeeva, Tianyi Wu, Mubashara Akhtar, Jiaheng Zhang, Elliott Ash, Markus Leippold, Timothy Baldwin, See-Kiong Ng, Artem Shelmanov, Mrinmaya Sachan

Responsible & Trustworthy NLP

Challenge:	Existing verification approaches, such as Process Reward Models, are computationally expensive and limited to specific domains.
Approach:	They propose a transformer-based probe that uses internal states of frozen LLMs to estimate credibility of reasoning steps during generation.
Outcome:	The proposed probes match or exceed PRMs that are up to 810 larger.

Actors, Frames and Arguments: A Multi-Decade Computational Analysis of Climate Discourse in Financial News using Large Language Models (2026.findings-eacl)

Copied to clipboard

Ruiran Su, Markus Leippold, Janet B. Pierrehumbert

Semantic Text Processing

Challenge:	a new study examines how financial news media portrays climate change . financial news is the nervous system of the global economy .
Approach:	They propose a three-stage Actor–Frame–Argument pipeline that uses large language models to extract actors, stances, frames, and argumentative structures from a 980,061-article corpus.
Outcome:	The proposed pipeline extracts actors, stances, frames, and argumentative structures from a 980,061-article corpus of climate-related financial news from the Dow Jones Newswire (2000–2023) it is based on a human-annotated gold standard and a Decompositional Verification Framework (DVF) that decomposes evaluation into completeness, faithfulness, coherence, and relevance, with multi-judge scoring calibrated against human ratings.

Automated Evidence Extraction and Scoring for Corporate Climate Policy Engagement: A Multilingual RAG Approach (2025.emnlp-demos)

Copied to clipboard

Imene Kolli, Saeid Vaghefi, Chiara Colesanti Senni, Shantam Raj, Markus Leippold

Multilinguality

Challenge:	InfluenceMap's LobbyMap Platform monitors the climate policy engagement of over 500 companies and 250 industry associations . however, a significant portion of the analysis remains manual, making it time- and labor-intensive and susceptible to human error.
Approach:	They propose an AI-assisted framework that leverages Retrieval-Augmented Generation to accelerate the monitoring of corporate climate policy engagement.
Outcome:	The proposed framework accelerates the monitoring of corporate climate policy engagement by leveraging Retrieval-Augmented Generation to extract and classify evidence from multilingual corporate documents.