Papers with OT
Semantic Matching for Sequence-to-Sequence Learning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for sequence-to-sequence semantic matching do not match target sentences . Optimal partial transport (OPT) is a technique that partially matches semantically meaningful words between source and partial target sequences. |
| Approach: | They propose a semantic matching scheme based on the Optimal Partial Transport (OPT) to match semantically meaningful words between source and partial target sequences. |
| Outcome: | Extensive experiments show that the proposed approach improves over sequence-to-sequence models. |
Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models? (2023.findings-eacl)
Copied to clipboard
Jielin Qiu, William Han, Jiacheng Zhu, Mengdi Xu, Michael Rosenberg, Emerson Liu, Douglas Weber, Ding Zhao
| Challenge: | Recent advances in Large Language Models (LLMs) have shown powerful ability in various downstream applications. |
| Approach: | They propose an approach for cardiovascular disease diagnosis and automatic ECG diagnosis report generation. |
| Outcome: | The proposed approach generates high-quality cardiac diagnosis reports and achieves competitive zero-shot classification performance even compared with supervised baselines. |
A Methodology for Creating Question Answering Corpora Using Inverse Data Annotation (2020.acl-main)
Copied to clipboard
Jan Deriu, Katsiaryna Mlynchyk, Philippe Schläpfer, Alvaro Rodrigo, Dirk von Grünigen, Nicolas Kaiser, Kurt Stockinger, Eneko Agirre, Mark Cieliebak
| Challenge: | Existing methods to efficiently construct corpus for question answering over structured data are time-consuming and cost-intensive. |
| Approach: | They propose a method to efficiently construct a corpus for question answering over structured data. |
| Outcome: | The proposed method triples the annotation speed while maintaining complexity of queries. |
OTExtSum: Extractive Text Summarisation with Optimal Transport (2022.findings-naacl)
Copied to clipboard
| Challenge: | Extractive text summarisation aims to select salient sentences from a document to form a short yet informative summary. |
| Approach: | They propose to formulate extractive text summarisation as an Optimal Transport (OT) problem and use it to obtain an optimal summary that minimises the transportation cost to a given document. |
| Outcome: | The proposed method outperforms state-of-the-art methods and learning-based methods on multiNews, PubMed, BillSum, and CNN/DM datasets. |
Document-Level Event Argument Extraction via Optimal Transport (2022.findings-acl)
Copied to clipboard
| Challenge: | Prior work on event-level EAE models ignore syntactic structures for documents . prior work on EE is restricted to sentence-level setting where event triggers and arguments are assumed to appear in the same sentences. |
| Approach: | They propose to employ Optimal Transport to induce structures of documents based on sentence-level syntactic structures and tailored to EAE task. |
| Outcome: | The proposed model is effective in document-level EAE, with a new constraint on unrelated context words. |
MProto: Multi-Prototype Network with Denoised Optimal Transport for Distantly Supervised Named Entity Recognition (2023.emnlp-main)
Copied to clipboard
| Challenge: | Distantly supervised named entity recognition (DS-NER) aims to locate entity mentions and classify their types with knowledge bases or gazetteers and unlabeled corpus. |
| Approach: | They propose a noise-robust prototype network named MProto for a DS-NER task . they propose an optimal transport algorithm to mitigate the noise from incomplete labeling . |
| Outcome: | The proposed network achieves state-of-the-art on several DS-NER benchmarks. |
Gromov-Wasserstein Alignment of Word Embedding Spaces (D18-1)
Copied to clipboard
| Challenge: | Current unsupervised methods for learning cross-lingual correspondences involve multiple steps, including heuristic post-hoc refinement strategies. |
| Approach: | They propose to cast the correspondence problem directly as an optimal transport problem, building on the idea that word embeddings arise from metric recovery algorithms. |
| Outcome: | The proposed method can be estimated efficiently, requires little or no tuning, and performs comparable with the state-of-the-art in various unsupervised word translation tasks. |
Unbalanced Optimal Transport for Unbalanced Word Alignment (2023.acl-long)
Copied to clipboard
| Challenge: | Figure 1 illustrates the challenges of monolingual word alignment. |
| Approach: | They propose to use the family of optimal transport (OT) to achieve unbalanced word alignment that values alignment and null alignment on unsupervised datasets. |
| Outcome: | The proposed methods are competitive against the state-of-the-art methods on challenging datasets with high null alignment frequencies. |
Optimal Transport Guided Correlation Assignment for Multimodal Entity Linking (2024.findings-acl)
Copied to clipboard
Zefeng Zhang, Jiawei Sheng, Zhang Chuang, Liangyunzhi Liangyunzhi, Wenyuan Zhang, Siqi Wang, Tingwen Liu
| Challenge: | Existing methods to link ambiguous mentions to entities in multimodal knowledge graphs rely on partial correlations. |
| Approach: | They propose a framework that leverages multi-element correlations to bridge modality gap and enable fine-grained semantic matching by exploiting correlation between multimodal features and entities. |
| Outcome: | The proposed framework outperforms state-of-the-art models and confirms the effectiveness of the proposed method. |
GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models (2025.acl-long)
Copied to clipboard
Kai Yao, Zhaorui Tan, Penglei Gao, Lichun Li, Kaixin Wu, Yinggui Wang, Yuan Zhao, Yixin Ji, Jianke Zhu, Wei Wang
| Challenge: | Existing methods for offsite-tuning of large language models require high computational costs and lack theoretical analysis. |
| Approach: | They propose an offsite-tuning approach that selectively applies compression techniques such as rank compression and channel pruning to preserve the gradients of fine-tuned adapters while ensuring privacy. |
| Outcome: | The proposed method surpasses existing OT methods in privacy protection and model performance. |
Guide the Many-to-One Assignment: Open Information Extraction via IoU-aware Optimal Transport (2023.acl-long)
Copied to clipboard
Kaiwen Wei, Yiran Yang, Li Jin, Xian Sun, Zequn Zhang, Jingyuan Zhang, Xiao Li, Linhao Zhang, Jintao Liu, Guo Zhi
| Challenge: | Open Information Extraction (OIE) aims to extract structured information from text without the limitations of close ontology. |
| Approach: | They propose a method to assign ground truth labels to parallelly generated tuple proposals . they leverage intersection-over-union (IoU) as assignment quality measurement . |
| Outcome: | The proposed method outperforms the state-of-the-art models on three benchmarks. |
APLOT: Robust Reward Modeling via Adaptive Preference Learning with Optimal Transport (2025.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that RLHF improves performance of Large Language Models . BT-based RMs struggle to distinguish between similar preference responses . |
| Approach: | They propose to enhance BT-based reward models by using an adaptive margin mechanism . they use semantic similarity and reward-predicted reward differences to adjust focus . |
| Outcome: | Experimental results show that the proposed method outperforms existing methods in both in-distribution and OOD settings. |
Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies suggest different methods to improve multilingual word representations in contextualized settings including techniques that align between source and target embedding spaces. |
| Approach: | They propose to use Optimal Transport as an alignment objective during fine-tuning to improve multilingual contextualized representations for downstream cross-lingual transfer. |
| Outcome: | The proposed method achieves better performance on two tasks (XNLI and XQuAD) and is competitive with existing methods. |
Implicational Universals in Stochastic Constraint-Based Phonology (D18-1)
Copied to clipboard
| Challenge: | Optimality Theory and Harmonic Grammar are constraint-based implementations of phonological theory that do not tamper with typological structure induced by categorical frameworks. |
| Approach: | They propose to model the implicational universals of phonological theory, called T-orders, and to use stochastic constraint-based frameworks to model them. |
| Outcome: | The proposed frameworks do not tamper with typological structure induced by categorical frameworks. |
Unsupervised Sentence Textual Similarity with Compositional Phrase Semantics (2022.coling-1)
Copied to clipboard
| Challenge: | Sentence Textual Similarity (STS) is a classic task that can be applied to downstream NLP applications such as text generation and retrieval. |
| Approach: | They propose a light-weighted Expectation-Correction (EC) formulation for STS computation. |
| Outcome: | The proposed approach is more efficient and scalable than previous approaches. |
Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models that use only rationales to explain a prediction are limited by the complexity of deep neural networks. |
| Approach: | They extend selective rationalization to text matching by using optimal transport to find a minimal cost alignment between inputs. |
| Outcome: | The proposed model achieves very sparse rationale selections with high fidelity while preserving prediction accuracy compared to strong attention baseline models. |
Keyphrase Generation via Soft and Hard Semantic Corrections (2022.emnlp-main)
Copied to clipboard
| Challenge: | Extensive experiments show that CorrKG is capable of generating high-quality keyphrases. |
| Approach: | They propose a correction model CorrKG on top of the MLE pipeline to correct the biases . the adaptive adaptive mass learning scheme is designed to better fit OT and FreqFS . |
| Outcome: | The proposed model overcomes the semantic biases in keyphrase generation using OT and FreqFS techniques. |
Vocabulary Learning via Optimal Transport for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Empirical results show that VOLT beats widely-used vocabularies in diverse scenarios, including WMT-14 English-German translation, TED bilingual translation, and TED multilingual translation. |
| Approach: | They propose a token dictionary solution that can be used without trial training to find the best dictionary with a proper size. |
| Outcome: | The proposed solution beats widely-used vocabularies in English-German translation, TED bilingual translation, and TED multilingual translation. |
Imagination and Contemplation: A Balanced Framework for Semantic-Augmented Multimodal Machine Translation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal Machine Translation (MMT) is effective in resolving linguistic ambiguities, but visual information often introduces redundancy or noise, potentially impairing translation quality. |
| Approach: | They propose a semantic-augmented framework that integrates "Imagination" and "Contemplation" they first generate synthetic images from source text and align them with authentic images via an optimal transport loss . |
| Outcome: | The proposed framework outperforms baselines on translation datasets with visually ambiguous or weakly correlated content. |
Calibrating Pseudo-Labeling with Class Distribution for Semi-supervised Text Classification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies develop effective pseudo-labeling methods, but they struggle with unlabeled data that have imbalanced classes mismatched with the labeled data. |
| Approach: | They propose to use pseudo-labeling to train text classification models with few labeled data and massive unlabeled data. |
| Outcome: | Empirical results show that the proposed model outperforms state-of-the-art methods on 3 common benchmarks. |
BiMax: Bidirectional MaxSim Score for Document-Level Alignment (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Document alignment is necessary for the hierarchical mining of documents across source and target languages. |
| Approach: | They propose a cross-lingual Bidirectional Maxsim score for computing doc-to-doc similarity. |
| Outcome: | The proposed method achieves accuracy comparable to OT with an approximate 100-fold speed increase. |
Hierarchical Selection of Important Context for Generative Event Causality Identification with Optimal Transports (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for Event Causality Identification (ECI) rely on external toolkits or human annotation to obtain training signals. |
| Approach: | They propose a generative framework that leverages Optimal Transport to automatically select the most important sentences and words from full documents. |
| Outcome: | The proposed framework can predict causal relation between two events in text without external tools. |
MM-Align: Learning Optimal Transport-based Alignment Dynamics for Fast and Accurate Inference on Missing Modality Sequences (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to multimodal learning assume a complete input modality setting, i.e., each modality is either complete or completely missing in both training and test sets. |
| Approach: | They propose an alignment dynamics learning module based on the theory of optimal transport for missing data imputation and a denoising training algorithm to enhance the quality of iputation and accuracy of model predictions. |
| Outcome: | The proposed method performs faster and more accurate inferences under different missing conditions and alleviates the overfitting issue. |
Improving Text Generation with Student-Forcing Optimal Transport (2020.emnlp-main)
Copied to clipboard
Jianqiao Li, Chunyuan Li, Guoyin Wang, Hao Fu, Yuhchen Lin, Liqun Chen, Yizhe Zhang, Chenyang Tao, Ruiyi Zhang, Wenlin Wang, Dinghan Shen, Qian Yang, Lawrence Carin
| Challenge: | Maximum likelihood estimation (MLE) is used to train models, but during testing, the model is conditioned on previously generated tokens, resulting in exposure bias. |
| Approach: | They propose to use optimal transport to match the sequences generated in MLE and test modes to reduce exposure bias. |
| Outcome: | The proposed method is validated on machine translation, text summarization, and text generation tasks. |
FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning Large Language Models (LLMs) suffer from a performance bottleneck . Existing approaches like Offsite-Tuning (OT) secure the LLMs IP . |
| Approach: | They propose a framework that replaces weak adapters with a unified, powerful Proxy Small Language Model (SLM) they propose 'resource-friendly' compression and 'robust optimization' to handle data heterogeneity. |
| Outcome: | Experiments show that FedProxy outperforms OT and centralized fine-tuning methods. |
Adversarial Alignment with Anchor Dragging Drift (A3D2): Multimodal Domain Adaptation with Partially Shifted Modalities (2025.acl-long)
Copied to clipboard
| Challenge: | Domain adaptation is underexplored in multimodal learning environments due to expensive data collection and annotation. |
| Approach: | They propose a bi-alignment scheme to perform drift-drift and anchor-driving matching with partially shifting anchors. |
| Outcome: | The proposed approach achieves superior performance compared with state-of-the-art approaches. |
Safeguarding LLM Fine-tuning via Push-Pull Distributional Alignment (2026.acl-long)
Copied to clipboard
| Challenge: | Existing safety defenses for large language models fail to explicitly repel harmful patterns . Optimal transport (SOT) allows for safe fine-tuning without sacrificing safety . |
| Approach: | They propose a framework that reframes safe fine-tuning from instance-level filtering challenge to distribution-level alignment task grounded in Optimal Transport. |
| Outcome: | a new framework improves safety of large language models while maintaining competitive performance . the proposed framework reduces the risk of errors and improves model performance compared to baselines . |
Flexible-length Text Infilling for Discrete Diffusion Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing discrete diffusion models lack flexibility for text infilling without ground-truth positional data. |
| Approach: | They propose a discrete diffusion model that jointly denoises token values and token positions using a novel sample-level Optimal Transport coupling. |
| Outcome: | The proposed method outperforms existing methods on infilling benchmarks such as One-Billion-Word and Yelp. |
LLEOT: A Privacy-Enhancing Offsite Tuning Framework via Loss Landscape Elevation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to fine-tune large language models are infeasible due to privacy regulations. |
| Approach: | They propose an offsite tuning framework that secures data privacy and model parameter and capability privacy. |
| Outcome: | The proposed framework secures data privacy and model parameter and capability privacy while preserving gradient alignment. |