Papers with MAP
TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration (D19-53)
Copied to clipboard
| Challenge: | Detailed extended analyses of all submitted systems showed large relative improvements in accessing the most challenging multi-hop inference problems, while absolute performance remains low. |
| Approach: | The Shared Task on Multi-Hop Inference for Explanation Regeneration asks participants to regenerate detailed gold explanations for elementary science questions by selecting facts from a knowledge base of semi-structured tables. |
| Outcome: | The top-performing system achieved a mean average precision of 0.56 . the task combines facts from a knowledge base and supervised training data . |
ASU at TextGraphs 2019 Shared Task: Explanation ReGeneration using Language Models and Iterative Re-Ranking (D19-53)
Copied to clipboard
| Challenge: | Explanation Regeneration task is an intermediate step towards general multi-hop inference on large graphs. |
| Approach: | They propose a system that performs multi-hop inference and ranks a set of explanatory facts for a given elementary science question and correct answer pair. |
| Outcome: | The proposed system secured 2nd rank in the text graphs 2019 shared task with a mean average precision (MAP) of 41.3% on the test set. |
Team SVMrank: Leveraging Feature-rich Support Vector Machines for Ranking Explanations to Elementary Science Questions (D19-53)
Copied to clipboard
| Challenge: | TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration tackles explanation generation for elementary science questions. |
| Approach: | They propose a hybrid pipelined machine learning model and rule-based system to address MIER-19 . they use a featurerich learning-to-rank machine learning and a rule-driven system to rerank the LTR model predictions. |
| Outcome: | The proposed model was ranked fourth in the evaluation, close to the second and third ranked teams, achieving 39.4% MAP. |
Chains-of-Reasoning at TextGraphs 2019 Shared Task: Reasoning over Chains of Facts for Explainable Multi-hop Inference (D19-53)
Copied to clipboard
| Challenge: | EMNLP 2019 shared task on 'Multi-hop Inference Explanation Regeneration' identifies chains of facts relevant to explain an answer to an elementary science examination question. |
| Approach: | They propose a system that identifies chains of facts relevant to explain an answer to an elementary science examination question. |
| Outcome: | The proposed system outperforms the second best system by 14.95 points on the mean average precision (MAP) metric. |
Sememe Prediction for BabelNet Synsets using Multilingual and Multimodal Information (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing sememe KBs only cover a few languages, which hinders the wide utilization of sememes. |
| Approach: | They propose to build a multilingual sememe KB based on a dictionary called BabelNet . they use multilingual synonyms, multilingual glosses and images to encode sememes . |
| Outcome: | The proposed model outperforms previous methods in terms of MAP and F1 scores. |
CCT-Code: Cross-Consistency Training for Multilingual Clone Detection and Code Search (2025.naacl-srw)
Copied to clipboard
Nikita Sorokin, Tikhonov Anton, Dmitry Abulkhanov, Ivan Sedykh, Irina Piontkovskaya, Valentin Malykh
| Challenge: | clone detection is crucial in software development for identifying semantically similar code . clones can be found in the same language code snippets, but there is little research on multilingual clonage detection. |
| Approach: | They propose a novel training procedure leveraging cross-lingual similarity to train language models on source code in various programming languages. |
| Outcome: | The proposed method achieves state-of-the-art on C++ and Python clone detection benchmarks with comparable performance on decoder-based models. |
mbrs: A Library for Minimum Bayes Risk Decoding (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Minimum Bayes risk (MBRS) decoding is a decision rule of text generation tasks that outperforms conventional maximum a posteriori (MAP) decoders by selecting high-quality outputs based on quality or preference rather than probability. |
| Approach: | They propose to use minimum bayes risk (MBRS) decoding to determine outputs based on quality rather than probability. |
| Outcome: | MBRS is an MIT-licensed open-source project with a focus on speed, reproducibility, and extensibility. |
On Graph-based Reentrancy-free Semantic Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing graph-based approaches for semantic parsing fail on compositional generalization tasks. |
| Approach: | They propose a graph-based approach for semantic parsing that solves two problems . they propose two algorithms based on constraint smoothing and conditional gradient to approximate these problems. |
| Outcome: | The proposed graph-based approach delivers state-of-the-art results on GeoQuery, Scan, and Clevr . |
Diving into the Decoding Space of Non-Autoregressive Models via Lexically Constrained Search (2026.acl-short)
Copied to clipboard
| Challenge: | Non-autoregressive (NAR) models have been mainly developed to improve decoding efficiency. |
| Approach: | They propose a search-based decoding algorithm which is comparable to the autoregressive Grid Beam Search (GBS) method. |
| Outcome: | The proposed method does not suffer from the MAP degradation issue as the autoregressive method does. |
Quality-Aware Decoding for Neural Machine Translation (2022.naacl-main)
Copied to clipboard
Patrick Fernandes, António Farinhas, Ricardo Rei, José G. C. de Souza, Perez Ogayo, Graham Neubig, Andre Martins
| Challenge: | Despite advances in machine translation quality estimation and evaluation, decoding is mostly oblivious to this. |
| Approach: | They propose to use a decoding framework that is quality-aware for neural machine translation . they compare various methods like N-best reranking and minimum Bayes risk decoding . |
| Outcome: | The proposed quality-aware decoding outperforms MAP-based decoding on four datasets and two model classes. |
Denoising Attention for Query-aware User Modeling (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent work has proposed to build user models at query time by leveraging the Attention mechanism, which allows weighing the contribution of the user-related information w.r.t. the current query. |
| Approach: | They propose to use the Attention mechanism to build user models at query time by weighing the contribution of the user-related information w.r.t. the Attention variant adopts a robust normalization scheme and introduces . filtering mechanism to better discern among the user related data those helpful for personalization. |
| Outcome: | The proposed approach improves MAP, MRR, and NDCG above 15% w.r.t. other Attention variants at the state-of-the-art. |
If beam search is the answer, what was the question? (2020.emnlp-main)
Copied to clipboard
| Challenge: | surprisingly, beam search results on language generation tasks are low-quality . despite its high error rate, beam searches can be used to decode models with high probability . |
| Approach: | They frame beam search as the exact solution to a different decoding objective . they propose a set of decoding objectives that explicitly enforce this property . |
| Outcome: | The proposed method enforces uniform information density in text, a property motivated by cognitive science. |
Spotting Spurious Data with Neural Networks (N18-1)
Copied to clipboard
| Challenge: | Existing methods to identify spurious instances require either annotations generated by each individual annotator or both task-specific and instance-type annotations. |
| Approach: | They propose an approach that discriminates instances based on their "difficulty to learn" they use queueing theory and psychology of learning to improve annotations . |
| Outcome: | The proposed methods outperform state-of-the-art baselines and have a MAP of 0.85 and 0.22 in identifying spurious instances in synthetic and carefully-crowdsourced real-world datasets respectively. |
A Study of Latent Structured Prediction Approaches to Passage Reranking (N19-1)
Copied to clipboard
| Challenge: | a structured output framework is useful for learning to rank problems . current approaches for answer sentence reranking are mostly based on pairwise ranking signals or simple binary classification. |
| Approach: | They propose a structured output approach which regards rankings as latent variables . they propose an inference procedure to find the max-violating ranking based on decomposition of the corresponding loss. |
| Outcome: | The proposed approach solves the optimization problem on WikiQA and TREC13 datasets. |
DialogueCSE: Dialogue-based Contrastive Learning of Sentence Embeddings (2021.emnlp-main)
Copied to clipboard
| Challenge: | Conventional approaches to learning sentence embeddings from dialogues employ the siamese-network for this task, but such architecture yields a large gap between training and evaluating. |
| Approach: | They propose a dialogue-based contrastive learning approach to learn sentence embeddings from dialogues using a siamese-network. |
| Outcome: | The proposed model outperforms baseline methods on three multi-turn dialogue datasets in terms of MAP and Spearman’s correlation measures. |
Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Empirical results on machine translation suggest that DPE is effective for segmenting output sentences. |
| Approach: | They propose a new algorithm for tokenizing sentences into subword units . they propose enabling exact log marginal likelihood estimation and exact MAP inference . |
| Outcome: | The proposed algorithm improves on machine translation datasets and on a large dataset. |
Integrating Question Classification and Deep Learning for improved Answer Selection (C18-1)
Copied to clipboard
| Challenge: | Question Answering (QA) is the task of automatically generating answers to questions posed in natural language. |
| Approach: | They propose a system for Answer Selection that integrates fine-grained Question Classification with a Deep Learning model designed for Answer selection. |
| Outcome: | The proposed system outperforms the current state of the art in all variations except one . the proposed system improves QA by reducing the search space of potential answers . |
Did the Models Understand Documents? Benchmarking Models for Language Understanding in Document-Level Relation Extraction (2023.acl-long)
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) models achieve consistent performance gains in DocRE, but their underlying decision rules are still understudied. |
| Approach: | They propose to use annotations to provide rationales for document-level relation extraction (DocRE) they then propose to apply a method to evaluate models' reasoning capabilities . |
| Outcome: | The proposed models exhibit different reasoning processes in contrast to humans . the proposed models render models more trustworthy and robust to be deployed in real-world scenarios. |
Redefining Retrieval Evaluation in the Era of LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Traditional IR metrics assume that humans examine documents sequentially with diminishing attention to lower ranks. |
| Approach: | They propose a utility-based annotation schema that quantifies positive contribution of relevant passages and negative impact of distracting ones. |
| Outcome: | The proposed metric improves correlation with the end-to-end answer accuracy by up to 36% compared to traditional metrics. |
Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Recent studies have revealed a number of pathologies of neural machine translation systems. |
| Approach: | They propose to use maximum a posteriori decoding to identify the highest-scoring translation, i.e. the mode problem, to validate the model and its training algorithm. |
| Outcome: | The proposed model reproduces the statistical data well, but the beam search strays from the statistics. |
Course Concept Expansion in MOOCs with External Knowledge and Interactive Game (P19-1)
Copied to clipboard
| Challenge: | Existing methods to expand course concepts in MOOCs suffer from semantic drifts and lack of knowledge guidance. |
| Approach: | They propose to use a boundary search method to search for new concepts via external knowledge base and then use heterogeneous features to verify the results. |
| Outcome: | The proposed method improves on the datasets from Coursera and XuetangX. |
G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing domain-adaptive pre-training (DAPT) models tend to forget the general knowledge acquired by general PLMs, leading to catastrophic forgetting and sub-optimal performance. |
| Approach: | They propose a framework which augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge. |
| Outcome: | The proposed framework augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge. |
Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to extract event temporal relations from text data are limited by hard constraints and large datasets. |
| Approach: | They propose a framework that enhances deep neural network with distributional constraints constructed by probabilistic domain knowledge to improve the baseline neural network models. |
| Outcome: | The proposed framework improves baseline models with strong statistical significance on two widely used datasets in news and clinical domains. |
A Simple and Effective Method To Eliminate the Self Language Bias in Multilingual Representations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Language agnostic and semantic-language information isolation is an emerging research direction for multilingual representations models. |
| Approach: | They propose a method that factors out language identity information from semantic related components in multilingual representations pre-trained on monolingual data. |
| Outcome: | The proposed method improves cross-lingual transfer performance on weak alignment models. |
monoQA: Multi-Task Learning of Reranking and Answer Extraction for Open-Retrieval Conversational Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to the Conversational Question Answering task have used multi-task learning to solve the task. |
| Approach: | They propose to use multi-task learning to improve the ORConvQA task by sharing the reranker and reader’s learned structure in a generative model. |
| Outcome: | The proposed model outperforms baseline models on the OR-QuAC and OR-CoQA datasets and significantly outperformed existing strong baseline models. |
Dial2vec: Self-Guided Contrastive Learning of Unsupervised Dialogue Embeddings (2022.emnlp-main)
Copied to clipboard
| Challenge: | Dialogue embeddings are a critical prerequisite for semantically understanding dialogues. |
| Approach: | They propose a self-guided contrastive learning approach called dial2vec that captures interaction patterns between interlocutors and leverages them to guide the learning of the embeddings corresponding to each interlocuter. |
| Outcome: | The proposed approach achieves 8.7, 9.0, and 13.8 points absolute improvements over the strongest baseline on the three evaluation tasks respectively. |
Data Selection for Bilingual Lexicon Induction from Specialized Comparable Corpora (2020.coling-main)
Copied to clipboard
| Challenge: | Narrow specialized comparable corpora are small in size, making it difficult to build efficient models to acquire translation equivalents. |
| Approach: | They propose to use Tf-Idf and cross entropy to improve bilingual lexicon induction from specialized comparable corpora by a factor of 10 . |
| Outcome: | The proposed methods improve bilingual lexicon induction by a large margin. |
Theoretical Guarantees for Minimum Bayes Risk Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Minimum Bayes Risk (MBR) decoding is a decision rule used to generate sequences from autoregressive probability models (e.g., LLMs). |
| Approach: | They propose to use minimum bayes risk (MBR) decoding to optimize output selection by maximizing expected utility value of an underlying human distribution. |
| Outcome: | The proposed method is effective even though the language space Y is larger than the hypothesis set. |
Correlation-Aware Example Selection for In-Context Learning with Nonsymmetric Determinantal Point Processes (2025.emnlp-main)
Copied to clipboard
Qiunan Du, Zhiliang Tian, Zhen Huang, Kailun Bian, Tianlun Liu, Zhaoning Zhang, Xinwang Liu, Feng Liu, Dongsheng Li
| Challenge: | Existing studies on in-context learning (ICL) focus on the selection of individual examples and ignore correlations among examples. |
| Approach: | They propose a method to capture positive and negative correlations using the determinantal point process . they optimize the method via kernel decomposition-based MLE to fit a constructed pseudo-labeled dataset . |
| Outcome: | The proposed method outperforms baselines in ICL example selection. |
Case-Based Decision-Theoretic Decoding with Quality Memories (2025.emnlp-main)
Copied to clipboard
| Challenge: | Minimum Bayes risk (MBR) decoding is a decision rule of text generation . however, it depends on sample texts drawn from the text generation model . |
| Approach: | They propose a case-based decision-theoretic method to estimate the expected utility using examples of domain data. |
| Outcome: | The proposed method outperforms MAP decoding in translation tasks and image captioning tasks on MSCOCO and nocaps datasets. |