Papers with regression
A Gentle Introduction to Deep Nets and Opportunities for the Future (2022.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial on deep nets will introduce a new language for fine tuning deep net programs . the tutorial will be divided into two parts: Part A will make deep net programming accessible to a broader audience . |
| Approach: | This tutorial introduces a new language for fine tuning deep nets with short (1-line) programs that are as easy to code as regression in statistics packages such as R. |
| Outcome: | This tutorial will introduce gft (general fine tuning), a new language for deep nets . glm is a "little language" similar to gslm in statistics package R . |
Systematic Evaluation of Predictive Fairness (2022.aacl-main)
Copied to clipboard
| Challenge: | Several methods have been proposed to mitigate bias in training on biased datasets. |
| Approach: | They propose to examine the effect of target class imbalance and stereotyping on model performance by analyzing binary classification, profession prediction and regression tasks. |
| Outcome: | The proposed methods show that data conditions have a strong influence on relative model performance. |
Batch-Softmax Contrastive Loss for Pairwise Sentence Scoring Tasks (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in machine learning have led to the use of contrastive loss for representation learning. |
| Approach: | They propose to use batch-softmax contrastive loss to train pairwise sentence embeddings . they propose to take a batch-softermax contrastitive loss and train it with different loss functions . |
| Outcome: | The proposed model improves on a number of datasets and pairwise sentence scoring tasks. |
The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding (2020.acl-demos)
Copied to clipboard
Xiaodong Liu, Yu Wang, Jianshu Ji, Hao Cheng, Xueyun Zhu, Emmanuel Awa, Pengcheng He, Weizhu Chen, Hoifung Poon, Guihong Cao, Jianfeng Gao
| Challenge: | MT-DNN is an open-source natural language understanding toolkit . it allows researchers and developers to train customized deep learning models . |
| Approach: | They present MT-DNN, an open-source natural language understanding toolkit . it is designed to facilitate rapid customization for a broad spectrum of NLU tasks . MT supports multi-task knowledge distillation, which can substantially compress a deep neural model without significant performance drop. |
| Outcome: | The proposed model can significantly compress a large model without significant performance drop. |
Geolocation with Attention-Based Multitask Learning Models (D19-55)
Copied to clipboard
| Challenge: | predicting the location of a social media post requires discretization of the coordinates, but results in poor performance. |
| Approach: | They propose to combine two approaches to predict location using supervised models . they evaluate a multitask convolutional neural network that predicts both discrete locations and continuous coordinates . |
| Outcome: | The proposed model outperforms singletask models and prior work on one dataset and shows that correlation between labels and coordinates has a marked impact on the effectiveness of a regression task. |
Fine-Grained Temporal Orientation and its Relationship with Psycho-Demographic Correlates (N18-1)
Copied to clipboard
| Challenge: | Temporal orientation refers to an individual’s tendency to connect to the psychological concepts of past, present or future and affects personality, motivation, emotion, decision making and stress coping processes. |
| Approach: | They propose to use a minimally supervised method to classify tweets in one of three temporal categories, past, present, and future, and a deep bi-directional long-term memory (BLSTM) to measure correlation between sentiment view of temporal orientation and different psycho-demographic factors. |
| Outcome: | The proposed method achieves 78.27% accuracy on a manually created test set. |
BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning (2024.findings-acl)
Copied to clipboard
Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang, Jinhua Zhu, Shufang Xie, Tao Qin, Rui Yan
| Challenge: | BioT5+ is an extension of the BioT5, but lacked a nuanced understanding of molecular structures. |
| Approach: | They propose a new bio-entity modeling framework, BioT5+, which integrates IUPAC names and molecule data. |
| Outcome: | The proposed model bridges the gap between molecular representations and textual descriptions and improves the grounded reasoning of bio-text and bio-sequences. |
Revisiting the Uniform Information Density Hypothesis (2021.emnlp-main)
Copied to clipboard
| Challenge: | The uniform information density hypothesis posits a preference among language users for utterances structured such that information is distributed uniformly across a signal. |
| Approach: | They propose to test the hypothesis by using reading time and acceptability data to examine the effect of surprisal on language comprehension and acceptabilities. |
| Outcome: | The proposed hypothesis makes predictions about language comprehension and linguistic acceptability . |
Deconfounded Lexicon Induction for Interpretable Social Science (N18-1)
Copied to clipboard
| Challenge: | Lexical features are useful beyond predictive performance. they can also be used to understand the subjective properties of a text. |
| Approach: | They propose two deep learning algorithms that separate the explanatory power of text from confounds. |
| Outcome: | The proposed algorithms are predictive of a set of target variables yet uncorrelated to confounds . they pick words associated with narrative persuasion and are more predictive than standard features . |
Backward Compatibility During Data Updates by Weight Interpolation (2024.eacl-long)
Copied to clipboard
| Challenge: | Retraining a model with a larger amount of training data introduces negative flips . retraining the model with the updated data introduce negative flipping . |
| Approach: | They propose a backward compatible weight interpolation method to improve model predictions without regression bugs. |
| Outcome: | The proposed method reduces negative flips without sacrificing accuracy . it is straight forward to implement and does not increase inference cost. |
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation (2022.findings-acl)
Copied to clipboard
Tianxiang Sun, Xiangyang Liu, Wei Zhu, Zhichao Geng, Lingling Wu, Yilong He, Yuan Ni, Guotong Xie, Xuanjing Huang, Xipeng Qiu
| Challenge: | Existing methods to measure instance difficulty use generalization and threshold-tuning . a new approach to learn to exit is based on hash functions to assign tokens to a fixed exiting layer. |
| Approach: | They propose a Hash-based Early Exiting approach that replaces learn-to-exit modules with hash functions to assign each token to a fixed exiting layer. |
| Outcome: | The proposed approach improves on learning to exit and predicting instance difficulty. |
BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Currently, pre-trained language model (PLM) based metrics are widely adopted in text generation tasks. |
| Approach: | They propose to use PLMs to encode stereotypical societal biases in PLM-based metrics . they show that popular metrics exhibit higher social bias than traditional metrics based on 6 attributes . |
| Outcome: | The proposed method shows that PLM-based metrics exhibit higher social bias than traditional metrics on 6 attributes. |
Making Pre-trained Language Models Better Few-shot Learners (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that the GPT-3 model can perform few-shots on language understanding tasks with a natural-language prompt and a few task demonstrations. |
| Approach: | They propose a technique for fine-tuning language models using a few examples . they propose LM-BFF, which uses prompt-based fine-uning and a pipeline for automating prompt generation . |
| Outcome: | The proposed approach outperforms standard fine-tuning procedures on a range of NLP tasks. |
A Neural Pairwise Ranking Model for Readability Assessment (2022.findings-acl)
Copied to clipboard
| Challenge: | Automatic Readability Assessment (ARA) is traditionally treated as a classification problem in NLP research. |
| Approach: | They propose a neural ranking approach to automatic readability assessment (ARA) they propose 'neural' ranking methods that can be used to rank texts by reading level . |
| Outcome: | The proposed approach performs well in monolingual single/cross corpus testing scenarios and achieves a zero-shot cross-lingual ranking accuracy of over 80% for both French and Spanish when trained on English data. |
WER-BERT: Automatic WER Estimation with BERT in a Balanced Ordinal Classification Paradigm (2021.eacl-main)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) systems are evaluated using Word Error Rate (WER) a higher WER means a lower percentage of errors between the ground truth and the transcription of the system. |
| Approach: | They propose a new balanced paradigm for automatic Word Error Rate estimation using a Librispeech dataset and a Google Cloud's Speech-to-Text API. |
| Outcome: | The proposed approach is more effective than regression in a classification setting, but suffers from heavy class imbalance. |
SciRepEval: A Multi-Format Benchmark for Scientific Document Representations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for evaluating scientific document representations fail to capture the diversity of relevant tasks. |
| Approach: | They propose a benchmark for training and evaluating scientific document representations that includes 24 challenging and realistic tasks across four formats: classification, regression, ranking and search. |
| Outcome: | The proposed model outperforms existing models by over 2 points absolute. |
Investigating Dynamic Routing in Tree-Structured LSTM for Sentiment Analysis (D19-1)
Copied to clipboard
| Challenge: | Existing deep neural network models such as LSTM and tree-LSTM have a bias problem where the words in the tail of a sentence are more heavily emphasized than those in the header. |
| Approach: | They propose a capsule tree-LSTM model that uses dynamic routing to build sentence representations by assigning different weights to nodes according to their contributions to prediction. |
| Outcome: | The proposed model improves on the Stanford Sentiment Treebank and EmoBank datasets. |
Interpretable Semantic Gradients in SSD: A PCA Sweep Approach and a Case Study on AI Discourse (2026.findings-acl)
Copied to clipboard
| Challenge: | Supervised Semantic Differential (SSD) is a mixed quantitative–interpretive method that models how text meaning varies with continuous individual-difference variables . currently no systematic method exists for choosing the number of retained components, introducing avoidable researcher degrees of freedom in the analysis pipeline. |
| Approach: | They propose a PCA sweep procedure that treats dimensionality selection as a joint criterion over representation capacity, gradient interpretability, and stability across nearby values of K. |
| Outcome: | The proposed method is based on a corpus of short posts about artificial intelligence written by Prolific participants who also completed Admiration and Rivalry narcissism scales. |
No-Worse Context-Aware Decoding: Preventing Neutral Regression in Context-Conditioned Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models can answer questions and generate summaries when given external contexts. |
| Approach: | They propose a decode-time adapter that backs off to no-context decoding when context is non-informative and uses contrastive fallback under uncertainty. |
| Outcome: | The proposed model prevents neutral regression on baseline-correct items while preserving strong context-driven accuracy on helpful contexts. |
bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark (2023.acl-long)
Copied to clipboard
Momchil Hardalov, Pepa Atanasova, Todor Mihaylov, Galia Angelova, Kiril Simov, Petya Osenova, Veselin Stoyanov, Ivan Koychev, Preslav Nakov, Dragomir Radev
| Challenge: | bgGLUE is a benchmark for evaluating language models on natural language understanding (NLU) tasks in Bulgarian. |
| Approach: | They propose to use a benchmark to evaluate language models on NLU tasks in Bulgarian. |
| Outcome: | The proposed model performs well on sequence labeling tasks, but there is room for improvement for tasks that require more complex reasoning. |
Prosody-TTS: Improving Prosody with Masked Autoencoder and Conditional Diffusion Model For Expressive Text-to-Speech (2023.findings-acl)
Copied to clipboard
| Challenge: | Expressive text-to-speech aims to generate high-quality samples with rich prosody . prosodic attributes in highly dynamic voices are difficult to capture and model without intonation . |
| Approach: | They propose a pipeline that enhances prosody modeling and sampling by introducing a self-supervised masked autoencoder and a diffusion model to sample diverse prosodic patterns within the latent space. |
| Outcome: | The proposed pipeline achieves new state-of-the-art in text-to-speech with natural and expressive synthesis. |
Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates (2021.acl-long)
Copied to clipboard
| Challenge: | Using negative flips, we quantify, reduce and analyze regression errors in deep neural networks. |
| Approach: | They propose to quantify, reduce and analyze regression errors in NLP models by negative flips. |
| Outcome: | The proposed model update regression has a prevalent presence across tasks in the GLUE benchmark. |
Quantile Regression with Large Language Models for Price Prediction (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to structured prediction tasks focus on point estimates and lack systematic comparison across different methods. |
| Approach: | They propose a novel quantile regression approach that enables LLMs to produce full predictive distributions, improving upon traditional point estimates. |
| Outcome: | The proposed model outperforms encoder architectures, embedding-based methods, and few-shot learning methods in prediction accuracy and distributional calibration. |
On Pursuit of Designing Multi-modal Transformer for Video Grounding (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for video grounding are not end-to-end, i.e., they rely on time-consuming post-processing steps to refine predictions. |
| Approach: | They propose an end-to-end multi-modal Transformer model that uses two encoders and a cross-modal decoder for grounding prediction. |
| Outcome: | The proposed model is 4.9% faster than existing models and is based on a set of encodings and decoders. |
Regression Aware Inference with LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown strong results on a range of applications, including regression and scoring tasks. |
| Approach: | They propose alternative inference strategies that estimate the Bayes-optimal solution for regression and scoring metrics in closed-form from sampled responses. |
| Outcome: | The proposed approach significantly improves over baselines across datasets and models. |
BTW: A Non-Parametric Variance Stabilization Framework for Multimodal Model Integration (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for multimodal learning are difficult to scale beyond two modalities and lack resolution for instance-level control. |
| Approach: | They propose a bi-level weighting framework that combines instance-level Kullback-Leibler divergence and modality-level mutual information to dynamically adjust modality importance during training. |
| Outcome: | The proposed method significantly improves regression performance and multiclass classification accuracy. |
FlipGuard: Defending Preference Alignment against Update Regression with Constrained Optimization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in preference alignment have significantly improved Large Language Models' ability to generate texts that align with human preferences and values. |
| Approach: | They propose a constrained optimization approach to detect and mitigate update regression with focal attention. |
| Outcome: | The proposed approach detects and mitigates update regression with focal attention while maintaining excellent overall performance. |
MentalRiskES: A New Corpus for Early Detection of Mental Disorders in Spanish (2024.lrec-main)
Copied to clipboard
Alba M. Mármol Romero, Adrián Moreno Muñoz, Flor Miriam Plaza-del-Arco, M. Dolores Molina González, María Teresa Martín Valdivia, L. Alfonso Ureña-López, Arturo Montejo Ráez
| Challenge: | Existing studies on the prevalence of mental disorders on the Web are limited to the English language. |
| Approach: | They propose to use user messages posted on Telegram groups to annotate the corpus for natural language processing and to conduct experiments on text classification and regression. |
| Outcome: | The proposed corpus contains over 1,300 subjects with more than 45,000 messages posted in different public Telegram groups. |
How do Transformer Embeddings Represent Compositions? A Functional Analysis (2025.findings-acl)
Copied to clipboard
| Challenge: | Despite the popularity of transformer-based models, little is known about how they represent compound words and whether they are compositional. |
| Approach: | They evaluate compositionality in mistral, OpenAI Large, and Google embedding models and compare them with BERT. |
| Outcome: | The proposed models perform best in addition, multiplication, dilation, regression, and the classic vector addition model performs almost as well as any other model. |
Interpretable Text Embeddings and Text Similarity Explanation: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Text embeddings are a fundamental component in many NLP tasks, but their interpretation and explanation remain challenging. |
| Approach: | They propose a framework for interpretable text embeddings and text similarity explanation . they characterize the main ideas, approaches, and trade-offs and discuss lessons learned . |
| Outcome: | The proposed methods are compared with existing models and compare them with existing ones. |
Explanation Quality Assessment as Ranking with Listwise Rewards (2026.findings-acl)
Copied to clipboard
| Challenge: | a new approach to explanation quality assessment is to rank explanations by relative quality . standard reward objectives do not preserve graded distinctions well enough for policy optimization . |
| Approach: | They reformulate explanation quality assessment as a ranking problem instead of a generation problem . they train listwise and pairwise ranking models to preserve ordinal structure . |
| Outcome: | The proposed model outperforms regression on score separation and performance on listwise and pairwise models. |