Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models that use only rationales to explain a prediction are limited by the complexity of deep neural networks. |
| Approach: | They extend selective rationalization to text matching by using optimal transport to find a minimal cost alignment between inputs. |
| Outcome: | The proposed model achieves very sparse rationale selections with high fidelity while preserving prediction accuracy compared to strong attention baseline models. |
Similar Papers
SPECTRA: Sparse Structured Text Rationalization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Sparse attention mechanisms are a deterministic alternative, but they lack a way to regularize rationale extraction. |
| Approach: | They propose a framework for deterministic extraction of structured explanations via constrained inference on a factor graph, forming a differentiable layer. |
| Outcome: | The proposed framework outperforms previous studies on performance and plausibility of extracted rationales. |
Unbalanced Optimal Transport for Unbalanced Word Alignment (2023.acl-long)
Copied to clipboard
| Challenge: | Figure 1 illustrates the challenges of monolingual word alignment. |
| Approach: | They propose to use the family of optimal transport (OT) to achieve unbalanced word alignment that values alignment and null alignment on unsupervised datasets. |
| Outcome: | The proposed methods are competitive against the state-of-the-art methods on challenging datasets with high null alignment frequencies. |
Optimal Partial Transport Based Sentence Selection for Long-form Document Matching (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for document matching are limited by the partial nature of the sentence-level matching signals. |
| Approach: | They propose a matching approach that equips existing document matching models with an Optimal Partial Transport component, namely OPT-Match, which selects the key sentences that play a major role in matching. |
| Outcome: | The proposed approach outperforms existing models on four publicly available datasets and the key sentences selected by it are consistent with human-provided rationales. |
SCOUT: Selective Coupling via Optimal Unbalanced Transport for Interpretable Text Classification (2026.acl-long)
Copied to clipboard
| Challenge: | Standard interpretable models often rely on scalar similarities that obscure the true evidentiary basis of a prediction. |
| Approach: | They propose a new paradigm that grounds prototype reasoning in the selective correspondence of discriminative fragments. |
| Outcome: | The proposed model outperforms rationale extraction and post-hoc attribution methods on seven benchmarks. |
SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current direct preference optimization algorithms focus on a strict set of tokens contributing signals of KL divergence and rewards to the loss function. |
| Approach: | They propose a method that automatically learns to weight the KL divergence and reward corresponding to each token during PO training. |
| Outcome: | The proposed method achieves +10% and +3% win-rate points in two PO scenarios. |
Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies suggest different methods to improve multilingual word representations in contextualized settings including techniques that align between source and target embedding spaces. |
| Approach: | They propose to use Optimal Transport as an alignment objective during fine-tuning to improve multilingual contextualized representations for downstream cross-lingual transfer. |
| Outcome: | The proposed method achieves better performance on two tasks (XNLI and XQuAD) and is competitive with existing methods. |
Optimal Transport-Based Token Weighting scheme for Enhanced Preference Optimization (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for direct preference optimization assign equal importance to all tokens while humans focus on more meaningful parts. |
| Approach: | They propose to use a transport-based token weighting scheme to enhance direct preference optimization by emphasizing meaningful token pairs and de-emphasizing less relevant ones to yield a more contrastive reward difference estimate. |
| Outcome: | Extensive experiments have validated the proposed method in improving instruction-following ability across various settings. |
CREST: A Joint Framework for Rationalization and Counterfactual Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for analyzing and training NLP models have not been integrated to combine their complementary advantages. |
| Approach: | They introduce a framework for selective rationalization and counterfactual text generation that leverages CREST to regularize selective rationales and a loss function that regularizes selective rationals. |
| Outcome: | The proposed framework generates valid counterfactuals that are more natural than those produced by previous methods and can be used for data augmentation at scale. |
A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models are pre-trained on trillions of tokens and instruction-tuned or aligned to specific preferences. |
| Approach: | They propose guidelines to help researchers perform more effective parameter-efficient LLM alignment. |
| Outcome: | The proposed methods outperform preference optimization and outperformed pre-trained models on three key axes. |
Alignment Rationale for Natural Language Inference (2021.acl-long)
Copied to clipboard
| Challenge: | Existing explanation methods pick prominent features, but alignments between words or phrases are more enlightening clues to explain the model. |
| Approach: | They propose a method to generate alignment rationale explanations for co-attention based models in NLI by feature selection. |
| Outcome: | The proposed method is more faithful and human-readable compared with existing methods. |