Papers with macro-F1
Employing Text Matching Network to Recognise Nuclearity in Chinese Discourse (C18-1)
Copied to clipboard
| Challenge: | Experimental results show that nuclearity recognition is a challenging task in Chinese discourse parsing due to the need for more deep semantic information. |
| Approach: | They propose a text matching network that encodes discourse units and paragraphs by combining Bi-LSTM and CNN to capture global dependency information and local n-gram information. |
| Outcome: | The proposed model outperforms baselines on the Chinese Discourse TreeBank . the proposed model is based on a novel text matching network . |
Sentiment Reasoning for Healthcare (2025.acl-industry)
Copied to clipboard
| Challenge: | Sentiment Reasoning is an auxiliary task in sentiment analysis where the model predicts both the sentiment label and generates the rationale behind it based on the input transcript. |
| Approach: | They propose a task - Sentiment Reasoning - for both speech and textmodalities and propose 'multimodal multitask framework' . they propose to use a model that generates the rationale behind each predicted label and provides a rationale for model prediction with quality semantically comparable to humans. |
| Outcome: | The proposed task improves model transparency by providing rationale for model prediction with quality semantically comparable to humans while improving model’s classification performance. |
Measuring What Counts: The Case of Rumour Stance Classification (2020.aacl-main)
Copied to clipboard
| Challenge: | Numerous methods have been proposed to predict the stance of replies towards a given rumour, but their performance is not optimal for the four-class imbalanced task of rumor stance classification. |
| Approach: | They propose to use a four-class problem to predict the stance of replies towards a given rumour to help identify the most informative minority classes. |
| Outcome: | The proposed methods are robust to imbalanced data and score higher systems capable of recognising the two most informative minority classes (support and deny). |
An Empirical Study of Synthetic Data Generation for Implicit Discourse Relation Recognition (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to recognize implicit discourse relations are limited by the lack of training data. |
| Approach: | They propose a method to generate synthetic data for IDRR using a large language model . they extract confused discourse relation pairs based on false negative rate and use two-stage prompting to generate effective synthetic data. |
| Outcome: | The proposed method achieves state-of-the-art macro-F1 performance without sacrificing micro-F1. |
Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have great potential for synthetic data generation. |
| Approach: | They show that large language models can generate useful data even for complex tasks . they use a symmetric task difficulty asymmetry to prompt an LLM to generate plausible input text for a target output structure. |
| Outcome: | The proposed approach outperforms existing models by a substantial margin on closed information extraction tasks with 1.8M data points and 770M parameters. |
Predicting Clinical Trial Results by Implicit Evidence Integration (2020.emnlp-main)
Copied to clipboard
| Challenge: | Clinical trials are expensive and time-consuming, and inappropriately designed studies can be devastating in a pandemic. |
| Approach: | They propose a model that takes a PICO-formatted clinical trial proposal and predicts the outcome from it. |
| Outcome: | The proposed model outperforms baseline models on a benchmark dataset with 10.7% relative gain over BioBERT. |
Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset (2021.acl-long)
Copied to clipboard
| Challenge: | Using supervised machine learning, we assess how solidarity discourses changed before and during the COVID-19 crisis. |
| Approach: | They use social scientific concept of solidarity and its contestation, anti-solidarity, as problem setting to assess how European solidarity discourses changed before and during COVID-19. |
| Outcome: | The proposed model outperforms the baseline classifier with expert annotations by 25 points, from 58% macro-F1 to almost 85%. |
Decoding the Market’s Pulse: Context-Enriched Agentic Retrieval Augmented Generation for Predicting Post-Earnings Price Shocks (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing methods for forecasting large stock price movements after corporate earnings calls are prone to **narrative bias** Existing approaches lack temporal-causal reasoning and are unable to predict large stock prices. |
| Approach: | They propose a retrieval-augmented framework that deploys a team of cooperative LLM agents . they retrieve structured evidence from a Causal-Temporal Knowledge Graph built from financial statements and earnings calls . |
| Outcome: | The proposed framework outperforms larger LLMs and fine-tuned models in macro-F1, MCC, and Sharpe for the same forecasting horizon. |
Generative Calibration for In-context Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning is one of the most exciting features of large language models . performance is sensitive to various configurations of the prompt, such as the choice or order of the training examples. |
| Approach: | They propose to calibrate the in-context predictive distribution by adjusting the label marginal . they find that the proposed method outperforms the ICL and state-of-the-art calibration methods . |
| Outcome: | The proposed method outperforms state-of-the-art methods by 27% absolute in macro-F1. |
Efficient Annotator Reliability Assessment and Sample Weighting for Knowledge-Based Misinformation Detection on Social Media (2025.findings-naacl)
Copied to clipboard
Owen Cook, Charlie Grimshaw, Ben Peng Wu, Sophie Dillon, Jack Hicks, Luke Jones, Thomas Smith, Matyas Szert, Xingyi Song
| Challenge: | Misinformation spreads rapidly on social media, confusing the truth and targeting potentially vulnerable people. |
| Approach: | They propose to use inter- and intra-annotator agreement to understand the reliability of each annotator and influence the training of large language models based on annotators reliability. |
| Outcome: | The proposed framework utilises inter- and intra-annotator agreement to understand the reliability of each annotator and influence the training of large language models based on annotators reliability. |
ToxiGAN: Toxic Data Augmentation via LLM-Guided Directional Adversarial Generation (2026.eacl-long)
Copied to clipboard
| Challenge: | Generating toxic text for augmentation is a sensitive and challenging task . toxic language remains pervasive, often escaping automated moderation systems . |
| Approach: | They propose a class-aware text augmentation framework that combines adversarial generation with semantic guidance from large language models to provide balanced guidance. |
| Outcome: | Experiments on four hate speech benchmarks show that ToxiGAN outperforms traditional and LLM-based augmentation methods. |
Distill and Align Decomposition for Enhanced Claim Verification (2026.findings-eacl)
Copied to clipboard
Jabez Magomere, Elena Kochkina, Samuel Mensah, Simerjot Kaur, Fernando Acero, Arturo Oncevay, Charese Smiley, Xiaomo Liu, Manuela Veloso
| Challenge: | Existing methods for complex claim verification struggle to align decomposition quality with verification performance. |
| Approach: | They propose a reinforcement learning approach that optimizes decomposition quality and verifier alignment using Group Relative Policy Optimization. |
| Outcome: | The proposed method outperforms prompt-based approaches and existing methods in six evaluation settings. |
Detecting Causal Language Use in Science Findings (D19-1)
Copied to clipboard
| Challenge: | Prior studies on identifying inappropriate use of causal language relied on manual content analysis, which is not scalable for examining a large volume of science publications. |
| Approach: | They developed a prediction model that classifies conclusion sentences into “no relationship”, “correlational”, “conditional causal” and “direct causal” categories. |
| Outcome: | The proposed model can be used to identify the inappropriate use of causal language in scientific publications and news articles. |
HebID: Detecting Social Identities in Hebrew-language Political Text (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing NLP datasets focus on coarse-grained identity categories . existing datasets are mostly English-centric and focus on fine-grain categories based on cultural contexts. |
| Approach: | They introduce the first multilabel Hebrew corpus for social identity detection . they use Hebrew-tuned encoders alongside 2B-9B-parameter decoders . |
| Outcome: | The proposed classifier is based on a national public survey and uses Hebrew-tuned encoders to analyze political discourse and political speeches. |
Hierarchical Transfer Learning for Multi-label Text Classification (P19-1)
Copied to clipboard
| Challenge: | Multi-Label Hierarchical Text Classification (MLHTC) is a task of categorizing documents into one or more topics organized in an hierarchical taxonomy. |
| Approach: | They propose a transfer learning based strategy where binary classifiers at lower levels are initialized using parameters of the parent classifier and fine-tuned on the child category classification task. |
| Outcome: | The proposed method improves on micro-F1 and macro-F1, respectively, compared to binary classifiers trained from scratch. |
Automated Knowledge Graph Construction using Large Language Models and Sentence Complexity Modelling (2025.emnlp-main)
Copied to clipboard
Sydney Anuyah, Mehedi Mahmud Kaushik, Sri Rama Krishna Reddy Dwarampudi, Rakesh Shiradkar, Arjan Durresi, Sunandan Chakraborty
| Challenge: | integrating coreference and decomposition increases recall on rare relations by over 20%. |
| Approach: | They propose an open-source pipeline for extracting sentence-level knowledge graphs by combining robust coreference resolution with syntactic sentence decomposition. |
| Outcome: | The proposed pipeline achieves a 99.8% exact-match accuracy on sentence simplification. |
Interpretable Short Video Rumor Detection Based on Modality Tampering (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to detect rumors from the perspective of modality tampering are labor-intensive and time-consuming. |
| Approach: | They propose a short video rumor detection framework that integrates modality tampering detection and inter-modal matching into a model to detect modality-tampers and interpretability mechanisms to make the results more reasonable. |
| Outcome: | The proposed model improves on the short video rumor dataset by 4.6%-12% compared with other models and can explain whether the short clip is a rumour or not through the perspective of modality tampering. |
The Enemy from Within: A Study of Political Delegitimization Discourse in Israeli Political Speech (2025.emnlp-main)
Copied to clipboard
| Challenge: | a new computational model for political delegitimization discourse is proposed for analysis of democratic discourse . we identify the importance of PDD as a powerful tool in political competition . |
| Approach: | They propose a computational classification pipeline for political delegitimization discourse . they annotate a Hebrew-language corpus of 10,410 sentences from parliamentary speeches, facebook posts and leading news outlets . |
| Outcome: | The proposed model achieves an F1 of 0.74 for binary detection and a macro-F1 of 0.6 for classification of delegitimization characteristics. |
AttributionBench: How Hard is Automatic Attribution Evaluation? (2024.findings-acl)
Copied to clipboard
| Challenge: | generative search engines enhance the reliability of large language model responses by providing cited evidence. |
| Approach: | They propose to use a benchmark to evaluate whether a large language model supports the generated responses or not . |
| Outcome: | The proposed benchmark shows that even a fine-tuned GPT-3.5 only achieves around 80% macro-F1 under a binary classification formulation. |
Transition-Matrix Regularization for Next Dialogue Act Prediction in Counselling Conversations (2026.findings-acl)
Copied to clipboard
| Challenge: | Evaluated on a 60-class German counselling taxonomy, this improves macro-F1 by 9–42% relative depending on encoder and corpus-derived transition patterns. |
| Approach: | They propose to use a KL regularization term to align next dialogue act distributions with corpus-derived transition patterns to improve macro-F1 by 9–42% relative to encoders. |
| Outcome: | The proposed term improves macro-F1 by 9–42% relative to encoders and significantly improves dialogue-flow alignment. |
Dissecting Clinical Reasoning in Natural Language Inference for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies on large language models (LLMs) have demonstrated the impact of prompting strategies and fine-tuning techniques on their reasoning capabilities. |
| Approach: | They examine four classes of prompting strategies to elicit reasoning in large language models . they then construct demonstrations using a frontier model to distil multi-step reasoning capabilities into smaller models based on Low-Rank Adaptation (LoRA). |
| Outcome: | The proposed model improves in 75% of the models on MedNLI and TREC Clinical Trials. |
V-RoLoRA: RLVR-Driven MoE Routing for Steerable Pluralistic Alignment (2026.findings-acl)
Copied to clipboard
| Challenge: | Current methods for steering large language models rely on prompt engineering or reasoning-time guidance. |
| Approach: | They propose a value-controllable pluralistic alignment framework enhanced with conditioned gating that dynamically directs the flow among multiple experts based on an input value or moral vector. |
| Outcome: | The proposed method outperforms prompt-based steering and multi-task PEFT benchmarks on two 8-billion-parameter backbones. |
SMARTER: A Data-efficient Framework to Improve Toxicity Detection with Explanation via Self-augmenting Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Toxic content encompasses a wide spectrum of terminologies whose definitions vary by platform. |
| Approach: | They propose a 2-stage framework for explainable content moderation using Large Language Models (LLMs) they leverage LLMs’ own outputs to generate synthetic explanations for correct and incorrect labels . they refine explanation quality through cross-model training, allowing weaker models to align with stronger ones. |
| Outcome: | Experiments on 3 benchmarks show that the proposed framework achieves 13% macro-F1 improvement over few-shot baselines using only 6-57% of training data. |
MPCG: Multi-Round Persona-Conditioned Generation for Modeling the Evolution of Misinformation with LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Misinformation evolves as it spreads, shifting in language, framing, and moral emphasis to adapt to new audiences. |
| Approach: | They propose a multi-round, persona-conditioned framework that simulates how claims are iteratively reinterpreted by agents with distinct ideological perspectives. |
| Outcome: | The proposed framework generates persona-specific claims across multiple rounds . it is based on an uncensored large language model and is scalable to multiple tasks . |