Papers by Nikolaos Aletras
Neural Legal Judgment Prediction in English (P19-1)
Copied to clipboard
| Challenge: | Recent work on legal judgment prediction has focused on Chinese, but only feature-based models have been considered in English. |
| Approach: | They propose a hierarchical version of BERT which bypasses BERT’s length limitation. |
| Outcome: | The proposed model outperforms existing models in binary violation classification, multi-label classification and case importance prediction. |
Enhancing Linguistic Competence of Language Models through Pre-training with Language Learning Tasks (2026.acl-short)
Copied to clipboard
| Challenge: | Language models (LMs) are pre-trained on raw text datasets to generate text sequences token-by-token. |
| Approach: | They propose a framework that integrates Language Learning Tasks alongside standard next-token prediction to stimulate the acquisition of morphological, syntactic, and semantic knowledge. |
| Outcome: | The proposed framework improves performance on linguistic competence benchmarks while maintaining competitive performance on reasoning tasks. |
Improving Multimodal Classification of Social Media Posts by Leveraging Image-Text Auxiliary Tasks (2024.findings-eacl)
Copied to clipboard
| Challenge: | Prior work on multimodal content classification has not addressed these challenges. |
| Approach: | They propose to use two auxiliary tasks to fine-tune multimodal models to address hidden cross-modal semantics and weak image-text relationships when modeling text and images. |
| Outcome: | The proposed model improves by up to 2.6 F1 score across five diverse social media datasets. |
Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updates (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models underperform in languages absent or underrepresented in training data, creating barrier to equitable access for speakers worldwide. |
| Approach: | They propose a selective parameter update strategy that proactively preserves source knowledge by identifying critical parameters critical to maintaining source abilities. |
| Outcome: | Experiments in five typologically diverse languages show that SSU mitigates catastrophic forgetting. |
On the Impact of Temporal Concept Drift on Model Explanations (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Explanation faithfulness of model predictions is typically evaluated on held-out data from the same temporal distribution as the training data. |
| Approach: | They examine the impact of temporal variation on model explanations extracted by eight feature attribution methods and three select-then-predict models across six text classification tasks. |
| Outcome: | The proposed method shows the most robust faithfulness scores across datasets and in asynchronous settings. |
In Factuality: Efficient Integration of Relevant Facts for Visual Question Answering (2021.acl-short)
Copied to clipboard
| Challenge: | Current Visual Question Answering (VQA) models are trained on labelled data that may be insufficient to learn complex knowledge representations. |
| Approach: | They propose a method to integrate external knowledge into a visual pre-trained model by integrating facts extracted from a knowledge base. |
| Outcome: | The proposed method outperforms baseline models on the KVQA dataset benchmark by 19% and shows that it is weaker than previous models. |
LEGAL-BERT: The Muppets straight out of Law School (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing guidelines for pre-training and fine-tuning do not always generalize well in the legal domain. |
| Approach: | They propose to use BERT out of the box, adapt it by additional pre-training on domain-specific corpora, and pre-train it from scratch on domains. |
| Outcome: | The proposed strategies are: use the original BERT out of the box, adapt it by additional pre-training on domain-specific corpora, and pre-train it from scratch on domain specific corpors. |
Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has demonstrated that using chain of thought (CoT) on soft-reasoning tasks can yield limited or even negative performance gains. |
| Approach: | They investigate how chain of thought (CoT) is used in soft-reasoning tasks across instruction-tuned, reasoning and reasoning-distilled models. |
| Outcome: | The proposed model can steer predictions without faithfully reflecting reasoning, indicating a disconnect between CoT influence and faithfulness. |
An Empirical Study on Explanations in Out-of-Domain Settings (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work in Natural Language Processing has focused on extracting faithful explanations . yet, little is known about how post-hoc explanations perform in out-of-domain settings . |
| Approach: | They propose to use a random baseline to evaluate out-of-domain post-hoc explanation faithfulness . they suggest select-then-predict models demonstrate comparable predictive performance in out- of-domain settings to full-text trained models. |
| Outcome: | The proposed models perform better in out-of-domain settings than full-text models. |
Rethinking Semi-supervised Learning with Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a popular setting to make use of unlabelled data . Currently, there are two popular approaches to make effective use of the unlabelled datasets . |
| Approach: | They compare semi-supervised learning (SSL) and task-adaptive pre-training (TAPT) they find TAPT is a stronger and more robust SSL learner, even when using just a few hundred unlabelled samples . |
| Outcome: | The proposed methods improve model performance across different NLP tasks and data sizes. |
How does the pre-training objective affect what large language models learn about linguistic properties? (2022.acl-short)
Copied to clipboard
| Challenge: | Several pre-training objectives have been proposed to pre-train language models . but, to our knowledge, no studies have investigated how different pre- training objectives affect what BERT learns about linguistic properties. |
| Approach: | They propose to use masked language modeling to pre-train language models . they propose to optimize a mangled language modeling objective to learn linguistic information . |
| Outcome: | The proposed objectives improve BERT's learning of linguistic properties compared to non-linguistically motivated objectives. |
Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained transformer-based language models have demonstrated state-of-the-art predictive performance when adapted into a range of language understanding tasks. |
| Approach: | They propose to use salient information extracted a priori from training data to complement the task-specific information learned by the model during fine-tuning on a downstream task. |
| Outcome: | The proposed model can provide more faithful explanations across four different feature attribution methods compared to vanilla BERT. |
Active Learning Principles for In-Context Learning with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning has significantly enhanced predictive performance in few-shot learning settings. |
| Approach: | They propose to use pool-based Active Learning to identify the most informative demonstrations for few-shot learning over a single iteration to identify best demonstrations. |
| Outcome: | The proposed model outperforms all other methods, including random sampling, in the analysis of 24 classification and multi-choice tasks. |
An Empirical Study on Leveraging Position Embeddings for Target-oriented Opinion Words Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current methods for extracting opinion words for an aspect in text leverage position embeddings to capture relative position of word to the target. |
| Approach: | They propose to use pretrained word embeddings to extract opinion words for a given aspect in text. |
| Outcome: | The proposed methods outperform current methods on a task based on pre-trained word embeddings and position embedders. |
A Hierarchical N-Gram Framework for Zero-Shot Link Prediction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to zero-shot link prediction use textual features of relations as auxiliary information to improve the encoded representation. |
| Approach: | They propose a Hierarchical N-gram framework for Zero-Shot Link Prediction that leverages character n-gram information for ZSLP. |
| Outcome: | The proposed method achieves state-of-the-art on two standard ZSLP datasets. |
Regulation and NLP (RegNLP): Taming Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | polarization in AI safety and ethics debates are swaying political agendas on AI regulation and governance . regulation studies are rich source of knowledge on how to systematically deal with risk and uncertainty . |
| Approach: | They argue that NLP research can benefit from proximity to regulatory studies . they argue that regulation studies should focus on linking scientific knowledge to regulatory processes . |
| Outcome: | The proposed research space should focus on linking scientific knowledge to regulatory processes based on systematic methodologies. |
Compressing Language Models for Specialized Domains (2026.eacl-long)
Copied to clipboard
| Challenge: | Language models (LMs) excel at tasks across diverse domains, yet require substantial computational resources during inference. |
| Approach: | They propose a calibration method to improve the in-domain performance of compressed LMs in a post-training setting. |
| Outcome: | The proposed method outperforms existing methods on domain-specific tasks while preserving general performance. |
Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases (2021.naacl-main)
Copied to clipboard
Ilias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapatsanis, Nikolaos Aletras, Ion Androutsopoulos, Prodromos Malakasiotis
| Challenge: | Interpretability or explainability is an emerging field of research in NLP . experimental results indicate that the newly introduced task is very challenging . |
| Approach: | They propose to extract rationales as paragraphs in multi-paragraph structured court cases . they also propose a constraint that allows models to be more specific . |
| Outcome: | The proposed task is very challenging and there is a large scope for further research. |
Translation Error Detection as Rationale Extraction (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent Quality Estimation models rely on translation errors to predict overall sentence quality, but detecting specific errors is a more challenging task. |
| Approach: | They propose to use a semi-supervised method to detect translation errors by attribution of relevance scores to inputs to explain model predictions. |
| Outcome: | The proposed method can detect translation errors and is compared with human models using a set of feature attribution methods. |
Domain Classification-based Source-specific Term Penalization for Domain Adaptation in Hate-speech Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches for hate-speech detection exhibit poor performance in out-of-domain settings due to overemphasizing source-specific information that negatively impacts its domain invariance. |
| Approach: | They propose a domain adaptation approach that automatically extracts and penalizes source-specific terms using a classifier. |
| Outcome: | The proposed approach improves cross-domain evaluation on indomain held-out instances while preserving high performance on out-of-domain settings. |
How Private are Language Models in Abstractive Summarization? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Effective protection of private information is essential for knowledge dissemination in sensitive domains such as medical and legal. |
| Approach: | They perform a comprehensive study of privacy risks in LM-based summarization using closed- and four-weight models of different sizes and families. |
| Outcome: | The proposed models show that they leak personally identifiable information in their summaries, compared to human-generated summary summators, which show significantly higher privacy protection levels. |
Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that attention cannot be considered as a faithful explanation across encoders and tasks. |
| Approach: | They propose a new family of Task-Scaling mechanisms that scale attention weights across tasks and two attention mechanisms. |
| Outcome: | The proposed models improve explanation faithfulness across two attention mechanisms, five encoders and five text classification datasets without sacrificing predictive performance. |
Trading Syntax Trees for Wordpieces: Target-oriented Opinion Words Extraction with Wordpieces and Aspect Enhancement (2023.acl-short)
Copied to clipboard
| Challenge: | State-of-the-art target-oriented opinion word extraction methods use pre-trained BERT wordpieces and graph convolutional networks (GCNs) to encode inputs. |
| Approach: | They trade syntax trees for BERT wordpieces to eliminate the GCN component from the methods’ architectures to improve TOWE performance. |
| Outcome: | The proposed approach achieves state-of-the-art results on benchmark datasets and should serve as a strong baseline for further research. |
Understanding the Role of Input Token Characters in Language Models: How Does Information Loss Affect Performance? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models capture semantic and syntactic information, but no study has examined how information loss in input token characters affects their performance. |
| Approach: | They address this gap by pre-training language models using small subsets of token characters. |
| Outcome: | The proposed model retains 90% and 77% of the full-token model in standard NLU benchmarks and probing tasks even under extreme settings. |
Incorporating Attribution Importance for Improving Faithfulness Metrics (2023.acl-long)
Copied to clipboard
| Challenge: | Feature attribution methods (FAs) are popular for providing insights into the model reasoning process of making predictions. |
| Approach: | They propose a simple yet effective criterion that randomly masks tokens proportionately to their FA importance. |
| Outcome: | The proposed method is more faithful than hard sufficiency and comprehensiveness metrics. |
On the Limitations of Simulating Active Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Active learning (AL) is a human-and-model-in-the-loop paradigm that iteratively selects informative unlabeled data for human annotation. |
| Approach: | They propose to simulate active learning by using an already labeled dataset as the pool of unlabeled data. |
| Outcome: | The proposed model-in-the-loop paradigm can be used to perform experiments with human annotations on-the fly. |
Pit One Against Many: Leveraging Attention-head Embeddings for Parameter-efficient Multi-head Attention (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing pre-trained language models have produced performance gains in various tasks but come with large computational requirements. |
| Approach: | They propose an alternative module that uses only a single shared projection matrix and multiple head embeddings (MHE) they demonstrate that MHE attention is substantially more memory efficient compared to alternative attention mechanisms. |
| Outcome: | The proposed model is more memory efficient compared to the current model while achieving high retention ratio on several downstream tasks. |
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Cross-lingual vocabulary adaptation (CVA) methods have been proposed for adapting models to a target language . but effectiveness of these methods on increasing inference efficiency of generative large language models has not been explored. |
| Approach: | They propose to use cross-lingual vocabulary adaptation methods to adapt models to a target language to improve downstream performance. |
| Outcome: | The proposed methods significantly speed up models in four languages and four natural language understanding tasks. |
Complaint Identification in Social Media with Transformer Networks (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on identifying complaints in social media has focused on feature-based and task-specific neural network models. |
| Approach: | They evaluate a battery of neural models underpinned by transformer networks and combine them with linguistic information to predict complaints. |
| Outcome: | The proposed models outperform state-of-the-art methods on a publicly available dataset achieving a macro F1 up to 87. |
On the Ethical Limits of Natural Language Processing on Legal Text (2021.findings-acl)
Copied to clipboard
| Challenge: | Legal NLP holds the promise of improving access to justice and offers tools for empirical analysis of law on a large scale. |
| Approach: | They propose ways to think systematically about ethical limits of NLP . they place emphasis on three crucial normative parameters that have been underestimated . |
| Outcome: | The proposed methods are based on a real-life scenario that has prompted debate in the legal NLP community. |
Enhancing Data Quality through Simple De-duplication: Navigating Responsible Computational Social Science Research (2024.emnlp-main)
Copied to clipboard
| Challenge: | Social media data exhibits distinctive characteristics such as rapid and continual topic evolution. |
| Approach: | They propose new protocols and best practices for improving dataset development from social media data and its usage. |
| Outcome: | The proposed protocols and best practices improve the performance of social media datasets and their usage. |
On the Impact of Calibration Data in Post-training Quantization and Pruning (2024.acl-long)
Copied to clipboard
| Challenge: | Quantization and pruning are the foundations of compression for large language models . however, no prior work has investigated how calibration data impacts performance of compression methods. |
| Approach: | They propose an empirical study on the effect of calibration data on LLM performance. |
| Outcome: | The proposed methods improve performance in a post-training setting. |
Comparing Explanation Faithfulness between Multilingual and Monolingual Fine-tuned Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Previous studies have investigated how different factors affect faithfulness of model explanations . |
| Approach: | They find that the larger the multilingual model, the less faithful FAs are compared to its counterpart monolingual models. |
| Outcome: | The results show that the larger the multilingual model, the less faithful the FAs are compared to its counterpart monolingual models. |
Fundamental Reasoning Paradigms Induce Out-of-Domain Generalization in Language Models (2026.findings-acl)
Copied to clipboard
Mingzi Cao, Xingwei Tan, Mahmud Elahi Akhter, Marco Valentino, Maria Liakata, Xi Wang, Nikolaos Aletras
| Challenge: | Deduction, induction, and abduction are fundamental reasoning paradigms, core for human logical thinking. |
| Approach: | They propose to use a dataset of symbolic tasks to induce deductive skills into large language models (LLMs) they then use FT to fine-tune models to improve OOD generalization . |
| Outcome: | The proposed approach yields strong generalizability with substantial performance gains (up to 14.60) across realistic out-of-domain tasks. |
RISE: Robust Early-exiting Internal Classifiers for Suicide Risk Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing systems for risk assessment are prone to incorrectly predicting risk severity and have no early detection mechanisms. |
| Approach: | They propose a novel mechanism for accurate early detection of suicide risk by ensembling Hyperbolic Internal Classifiers equipped with an abstention mechanism and early exit inference capabilities. |
| Outcome: | The proposed model abstains from 84% incorrect predictions on Reddit data while out-predicting state of the art models upto 3.5x earlier. |
Unsupervised Quality Estimation for Neural Machine Translation (2020.tacl-1)
Copied to clipboard
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, Lucia Specia
| Challenge: | Existing approaches require large amounts of expert annotated data, computation, and time for training. |
| Approach: | They propose an unsupervised approach to QE where no training is required . they use a dataset that enables work on both black-box and glass-box approaches . |
| Outcome: | The proposed approach rivals state-of-the-art supervised QE models in terms of correlation with human judgments of quality. |
Self-calibration for Language Model Quantization and Pruning (2025.naacl-long)
Copied to clipboard
| Challenge: | Quantization and pruning are fundamental approaches for model compression, but they require large computational resources. |
| Approach: | They propose to use model calibration data to generate synthetic calibrations to improve model performance. |
| Outcome: | The proposed method outperforms other methods using real data in a post-training setting. |
Analyzing Online Political Advertisements (2021.findings-acl)
Copied to clipboard
| Challenge: | Online political advertising is an integral part of modern digital election campaigning. |
| Approach: | They propose to use textual and visual information from pre-trained neural models to infer the political ideology of an ad sponsor and identify whether the sponsor is an official political party or a third-party organization. |
| Outcome: | The proposed approach outperforms state-of-the-art methods for generic commercial ad classification and linguistic analysis to study the characteristics of political ads discourse. |
Journalist-in-the-Loop: Continuous Learning as a Service for Rumour Analysis (D19-3)
Copied to clipboard
| Challenge: | Existing rumour analysis tools do not scale due to the large volume and velocity of user generated content. |
| Approach: | They propose to use a web-based rumour analysis tool that can continuously learn from journalists and integrate it into existing tools and platforms. |
| Outcome: | The proposed system can be easily integrated as a service into existing tools and platforms used by journalists using a REST API. |
Point-of-Interest Type Inference from Social Media Text (2020.aacl-main)
Copied to clipboard
| Challenge: | Using a dataset of 200,000 English tweets, we can predict the type of the place from which a tweet was sent from. |
| Approach: | They propose to analyze a dataset of 200,000 tweets from 2,761 points-of-interest in the U.S. and train classifiers to predict the type of the location a tweet was sent from. |
| Outcome: | The proposed method can predict the type of the location a tweet was sent from and reach a macro F1 of 43.67 across eight classes. |
Combining Humor and Sarcasm for Improving Political Parody Detection (2022.naacl-main)
Copied to clipboard
| Challenge: | Parody is a figurative device used for mimicking entities for comedic or critical purposes. |
| Approach: | They propose a multi-encoder model that combines three parallel encoders to enrich parody-specific representations with humor and sarcasm information. |
| Outcome: | The proposed model outperforms state-of-the-art methods on a dataset of political parody tweets. |
An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels (2020.emnlp-main)
Copied to clipboard
Ilias Chalkidis, Manos Fergadiotis, Sotiris Kotitsas, Prodromos Malakasiotis, Nikolaos Aletras, Ion Androutsopoulos
| Challenge: | Large-scale Multi-label Text Classification (LMTC) is a type of classification that assigns labels to a large set of labels. |
| Approach: | They propose to use probabilistic label trees to improve frequent, few and zero-shot learning . they propose to combine a new state-of-the-art method with pre-trained Transformers . |
| Outcome: | The proposed models outperform existing models on frequent, few and zero-shot learning on three datasets from different domains. |
Automatically Identifying Complaints in Social Media (P19-1)
Copied to clipboard
| Challenge: | Complaining is a basic speech act used to express a negative mismatch between reality and expectations in a particular situation. |
| Approach: | They present a systematic analysis of complaints in computational linguistics . they collect annotated data set of written complaints expressed on Twitter . |
| Outcome: | The proposed model achieves predictive performance of up to 79 F1 using distant supervision. |
It’s about Time: Rethinking Evaluation on Rumor Detection Benchmarks using Chronological Splits (2023.findings-eacl)
Copied to clipboard
| Challenge: | Current rumor detection benchmarks use random splits as training, development and test sets which results in topical overlaps. |
| Approach: | They propose to use chronological rather than random splits for rumor classification . they propose to always use chronological splits to minimize topical overlaps . |
| Outcome: | The proposed model overestimates performance on four popular rumor detection benchmarks considering chronological instead of random splits. |
On the Importance of Effectively Adapting Pretrained Language Models for Active Learning (2022.acl-short)
Copied to clipboard
| Challenge: | Recent active learning approaches in NLP use off-the-shelf pretrained language models (LMs) . a poor training strategy can be catastrophic for AL, authors argue . |
| Approach: | They propose to first adapt the pretrained LM to the target task and then use it for AL. |
| Outcome: | The proposed approach provides substantial data efficiency improvements compared to the standard fine-tuning approach. |
Knowledge Distillation for Quality Estimation (2021.findings-acl)
Copied to clipboard
Amit Gajbhiye, Marina Fomicheva, Fernando Alva-Manchego, Frédéric Blain, Abiola Obamuyide, Nikolaos Aletras, Lucia Specia
| Challenge: | Recent success in Quality Estimation stems from the use of multilingual pre-trained models, where large models lead to impressive results. |
| Approach: | They propose to transfer knowledge from a strong QE teacher model to a much smaller model with a different, shallower architecture. |
| Outcome: | The proposed model performs better than distilled models with 8x fewer parameters. |
HashFormers: Towards Vocabulary-independent Pre-trained Transformers (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models are vocabulary-dependent, mapping by default each token to its corresponding embedding. |
| Approach: | They propose a family of vocabulary-independent pre-trained transformers that support unlimited vocabulary . they propose to map each token to its corresponding embedding by default . |
| Outcome: | The proposed models are more memory efficient than existing models while achieving comparable performance on multiple text classification tasks. |
Modeling the Severity of Complaints in Social Media (2021.naacl-main)
Copied to clipboard
| Challenge: | Complaining is a speech act used by humans to communicate a negative mismatch between reality and expectations . recent work on modeling complaints in natural language processing (NLP) has focused on distinguishing complaints from non-complaints in social media. |
| Approach: | They propose to classify complaints into various severity levels based on the face-threat that the complainer is willing to undertake and their purpose. |
| Outcome: | The proposed model achieves 55.7 macro F1 on binary complaint classification and 88.2 macro F1. |
Dynamically Refined Regularization for Improving Cross-corpora Hate Speech Detection (2022.findings-acl)
Copied to clipboard
| Challenge: | Hate speech classifiers exhibit performance degradation when evaluated on datasets different from the source. |
| Approach: | They propose to automatically identify and reduce spurious correlations using attribution methods with dynamic refinement of the list of terms that need to be regularized during training. |
| Outcome: | The proposed method improves performance across corpora and on different datasets. |
Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Science (2024.lrec-main)
Copied to clipboard
Yida Mu, Ben P. Wu, William Thorne, Ambrose Robinson, Nikolaos Aletras, Carolina Scarton, Kalina Bontcheva, Xingyi Song
| Challenge: | Existing instruction-tuned Large Language Models (LLMs) have impressive language understanding and the capacity to generate responses that follow specific prompts. |
| Approach: | They evaluate the zero-shot performance of two publicly accessible LLMs, ChatGPT and OpenAssistant, in the context of six Computational Social Science classification tasks. |
| Outcome: | The proposed LLMs perform better than state-of-the-art models on social science tasks. |
Point-of-Interest Type Prediction using Text and Images (2021.emnlp-main)
Copied to clipboard
| Challenge: | Prior efforts in POI type prediction focus on text without taking visual information into account. |
| Approach: | They propose to use multimodal information from text and images to infer the type of a place from where a social media post was shared. |
| Outcome: | The proposed method outperforms the state-of-the-art method for POI type prediction based on text-only methods and sheds light on cross-modal interactions and limitations. |
Improving Graph-Based Text Representations with Character and Word Level N-grams (2022.aacl-short)
Copied to clipboard
| Challenge: | Graph-based text representation is important in downstream natural language processing tasks. |
| Approach: | They propose a heterogeneous word-character text graph that combines word and character n-gram nodes together with document nodes. |
| Outcome: | The proposed graph outperforms baselines and state-of-the-art models in text classification and automatic summarization. |
Bayesian Prompt Ensembles: Model Uncertainty Estimation for Black-Box Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to quantify uncertainty of pre-trained large language models require specific architectures or retraining strategies. |
| Approach: | They propose a Bayesian Prompts Ensemble approach to accurately quantify LLM uncertainty . they compute output probabilities through a weighted ensemble of different task instruction prompts . |
| Outcome: | The proposed approach achieves significantly superior calibration over baselines over a range of natural language classification tasks. |
GreekBarBench: A Challenging Benchmark for Free-Text Legal Reasoning and Citations (2025.findings-emnlp)
Copied to clipboard
| Challenge: | GreekBarBench evaluates LLMs on legal questions across five different legal areas from the Greek Bar exams. |
| Approach: | They propose a three-dimensional scoring system and an LLM-as-a-judge approach to tackle the challenges of free-text evaluation. |
| Outcome: | The proposed system uses an LLM-as-a-judge approach to evaluate LLMs on legal questions across five legal areas from the Greek Bar exams. |
Active Learning by Acquiring Contrastive Examples (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using uncertainty and diversity sampling, active learning acquisition functions select difficult and diverse data points from a pool of unlabeled data. |
| Approach: | They propose an active learning acquisition function that selects contrastive examples from unlabeled data. |
| Outcome: | The proposed approach performs better or equal to the best performing baseline on all tasks, on both in-domain and out-of-domain data. |
Enhancing Logical Reasoning in Language Models via Symbolically-Guided Monte Carlo Process Supervision (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have shown strong performance in many reasoning benchmarks, but lack robust planning or symbolic abstractions. |
| Approach: | They propose to synthesize high-quality symbolic reasoning trajectories with stepwise pseudo-labels at scale via Monte Carlo estimation. |
| Outcome: | The proposed method can be trained on high-quality symbolic reasoning trajectories with stepwise pseudo-labels at scale using Monte Carlo estimation. |
Examining the Limitations of Computational Rumor Detection Models Trained on Static Datasets (2024.lrec-main)
Copied to clipboard
| Challenge: | Past research has indicated that content-based rumor detection models perform less effectively on unseen rumors. |
| Approach: | They propose to use data split strategies to minimize the effects of temporal concept drift in static datasets during the training of rumor detection methods. |
| Outcome: | The proposed model over-relys on the information derived from the rumors’ source post and overlooks the significant role that contextual information can play. |
Analyzing Political Parody in Social Media (2020.acl-main)
Copied to clipboard
| Challenge: | Parody is a figurative device used to imitate an entity for comedic or critical purposes. |
| Approach: | They propose a dataset of tweets from real politicians and their corresponding parody accounts to run supervised machine learning models for automatic classification. |
| Outcome: | The proposed models predict political parody tweets with 90% accuracy . they also identify the markers of parody through a linguistic analysis . |
Automatic Identification and Classification of Bragging in Social Media (2022.acl-long)
Copied to clipboard
| Challenge: | Bragging is a speech act employed to build a favorable self-image through positive statements about oneself. |
| Approach: | They propose to use tweets annotated for bragging to build a model that can predict bragging with macro F1 up to 72.42 and 35.95 for binary and multi-class bragging classification tasks respectively. |
| Outcome: | The proposed models predict bragging with macro F1 up to 72.42 and 35.95 in binary and multi-class classification tasks respectively. |
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling (2021.emnlp-main)
Copied to clipboard
| Challenge: | Masked language modeling (MLM) is widely used in natural language processing for self-supervised learning of text representations. |
| Approach: | They propose to use token-level classification tasks as main pretraining objectives instead of Masked language modeling (MLM) . Empirical results show that pretraining a model with 41% of the BERT-BASE’s parameters, BERT MEDIUM results in only a 1% drop in GLUE scores with their best objective. |
| Outcome: | Empirical results show that the proposed methods achieve comparable or better performance to MLM using a BERT-BASE architecture. |
Who Is Bragging More Online? A Large Scale Analysis of Bragging in Social Media (2024.lrec-main)
Copied to clipboard
| Challenge: | Social media is a natural platform for users to use bragging to gain admiration, respect, attention and followers from their audiences. |
| Approach: | They employ computational sociolinguistics methods to conduct the first large scale study of bragging behavior on Twitter by focusing on its overall prevalence, temporal dynamics and impact of demographic factors. |
| Outcome: | The proposed study shows that the prevalence of bragging decreases over time within the same population of users and younger, more educated and popular users in the U.S. are more likely to brag. |
Schema-Guided User Satisfaction Modeling for Task-Oriented Dialogues (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies on USM neglect explicitly modeling the user’s task goals fulfillment using the task schema. |
| Approach: | They propose a schema-guided user satisfaction modeling framework that explicitly models the degree to which the user’s preferences regarding task attributes are fulfilled by the system. |
| Outcome: | The proposed framework outperforms existing methods on benchmark datasets and shows that it can interpret and scale well with unseen tasks and can work in low-resource settings. |
LexGLUE: A Benchmark Dataset for Legal Language Understanding in English (2022.acl-long)
Copied to clipboard
Ilias Chalkidis, Abhik Jana, Dirk Hartung, Michael Bommarito, Ion Androutsopoulos, Daniel Katz, Nikolaos Aletras
| Challenge: | Laws and their interpretations, legal arguments and agreements are typically expressed in writing. |
| Approach: | They propose a benchmark to evaluate model performance across legal NLU tasks . they also evaluate several generic and legal-oriented models . |
| Outcome: | The proposed model performs better across multiple tasks than previous models. |
PATCH: Mitigating PII Leakage in Language Models with Privacy-Aware Targeted Circuit PatcHing (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing defense mechanisms to mitigate PII leakage are limited by existing defenses . a new approach, PATCH, identifies and edits PI I circuits to reduce leakage . |
| Approach: | They propose to use PATCH: Privacy-Aware Targeted Circuit Patching to identify PII leakage circuits in language models to reduce leakage. |
| Outcome: | The proposed approach reduces leakage by up to 65% and can reduce residual leakage to as low as 0.01%. |