Papers by Zhao Tong

77 papers
Advancing Vision-Language Models with Adapter Ensemble Strategies (2024.findings-emnlp)

Copied to clipboard

Challenge: CLIP revolutes vision-language pretraining by using contrastive learning on paired web data.
Approach: They propose to combine a "adapter ensemble" with traditional machine learning techniques to augment large-scale pretrained vision-language models.
Outcome: The proposed model outperforms baselines and derives improvement when the number of ensemble parameters increases.
Generate-on-Graph: Treat LLM as both Agent and KG for Incomplete Knowledge Graph Question Answering (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods to integrate LLMs with Knowledge Graphs (KGs) however, these methods are often incomplete to cover all the knowledge required to answer questions.
Approach: They propose to integrate LLMs with Knowledge Graphs (KGs) to address insufficient knowledge and hallucination issues in Large Language Models.
Outcome: The proposed method outperforms existing methods on two datasets.
LLM as Effective Streaming Processor: Bridging Streaming-Batch Mismatches with Group Position Encoding (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods for adapting LLMs to streaming rely on expensive re-encoding or limited scalability.
Approach: They propose a group position encoding paradigm built on batch architectures to enhance consistency between streaming and batch modes.
Outcome: The proposed method outperforms existing methods on cross-lingual and cross-modal tasks.
Context-aware Information-theoretic Causal De-biasing for Interactive Sequence Labeling (2022.findings-emnlp)

Copied to clipboard

Challenge: Existing deep learning models for sequence labeling are expensive and time-consuming.
Approach: They propose an interactive sequence labeling that allows training directly with the user feedback . they identify context and feedback biases by formulating interactive sequence labels via a Structural Causal Model.
Outcome: The proposed approach can effectively alleviate the biases and can be learnt with the user feedback.
A Survey on LLM-based Conversational User Simulation (2026.eacl-long)

Copied to clipboard

Challenge: Recent advances in large language models (LLMs) have enabled high-fidelity generation of synthetic user conversation.
Approach: They propose a taxonomy covering user granularity and simulation objectives . they analyze core techniques and evaluation methodologies to help them understand the latest developments .
Outcome: The proposed model enables high-fidelity generation of synthetic user conversation.
Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning Through Trap Problems (2024.emnlp-main)

Copied to clipboard

Challenge: Current LLMs lack systematic compositionality, and therefore cannot serve as reliable cognitive models.
Approach: They propose to introduce logical traps into the original problems of MATH and GSM8K to investigate the compositionality of large language models in mathematical reasoning.
Outcome: The proposed model can generate infinite combinations from finite learned components.
R^3AG: Retriever Routing for Retrieval-Augmented Generation (2026.acl-long)

Copied to clipboard

Challenge: Retrieval-augmented generation (RAG) is often bottlenecked by the “one-size-fits-all” retrieval paradigm, as different queries exhibit distinct preferences for different retrievers.
Approach: They propose a novel routing framework that explicitly models the dynamic alignment between queries and retriever capabilities and decomposes retriever capability into two learnable dimensions: retrieval quality and generation utility.
Outcome: Experiments on knowledge-intensive tasks show that R3AG outperforms both the best individual retrievers and state-of-the-art static routing methods.
Augment before You Try: Knowledge-Enhanced Table Question Answering via Table Expansion (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods to integrate external information into a given table neglect the structured nature of the table.
Approach: They propose a simple yet effective method to integrate external information into a given table by first building an augmenting table and then generating a SQL query over the two tables to answer the question.
Outcome: The proposed method outperforms strong baselines on three table QA benchmarks.
Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts (2022.findings-acl)

Copied to clipboard

Challenge: Recent years have seen a surge of interest in improving the generation quality of commonsense reasoning tasks.
Approach: They propose a method that diversifies the generative reasoning by a mixture of expert strategy on commonsense knowledge graphs to encourage various generation outputs.
Outcome: The proposed method improves diversity while achieving on par performance on two GCR benchmarks, based on both automatic and human evaluations.
CogToM: A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Models (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks for Large Language Models (LLMs) are limited to false belief tasks, highlighting bottlenecks in specific dimensions.
Approach: They propose a benchmark to evaluate Large Language Models' Theory of Mind capabilities . they evaluate 8000 bilingual instances across 46 paradigms and validated by 49 human annotators .
Outcome: The proposed benchmark reveals performance heterogeneities and bottlenecks in 22 representative models.
Enhancing Speech Large Language Models with Prompt-Aware Mixture of Audio Encoders (2025.emnlp-main)

Copied to clipboard

Challenge: Existing work on integrating audio encoders with large language models (LLMs) has focused on semantic understanding tasks, but different tasks may require distinct features that emphasize either semantic or acoustic aspects.
Approach: They propose to use a prompt-aware mixture to enhance the Speech LLM that uses multiple audio encoders to extract different features based on the prompt.
Outcome: The proposed approach outperforms all single-encoder Speech LLMs on ASR, speaker number verification, and AC tasks.
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)

Copied to clipboard

Challenge: Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources.
Approach: They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation .
Outcome: The proposed model outperforms previous approaches by a significant margin in QA tasks over text.
Do VLMs Have a Moral Backbone? A Study on the Fragile Morality of Vision-Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Vision-Language Models (VLMs) have advanced multimodal learning, driving progress in cross-modal reasoning.
Approach: They propose to examine moral robustness of vision-language models by analyzing their moral stances under multimodal perturbations.
Outcome: The proposed model-agnostic multimodal perturbations expose VLMs to a variety of moral vulnerabilities, including a sycophancy trade-off where stronger instruction-following models are more susceptible to persuasion.
VisiPruner: Decoding Discontinuous Cross-Modal Dynamics for Efficient Multimodal LLMs (2025.emnlp-main)

Copied to clipboard

Challenge: Multimodal Large Language Models (MLLMs) suffer from significant computational overhead due to the quadratic growth of attention computations with the number of multimodal tokens.
Approach: They propose a training-free pruning framework that prunes multimodal tokens without a trained pruning method.
Outcome: The proposed pruning framework outperforms existing token pruning methods and generalizes across diverse MLLMs.
Threshold Differential Attention for Sink-Free, Ultra-Sparse, and Non-Dispersive Language Modeling (2026.acl-long)

Copied to clipboard

Challenge: a strict sum-to-one constraint forces attention sinks on irrelevant tokens, while probability mass disperses as sequence lengths increase.
Approach: They propose a sink-free attention mechanism that achieves ultra-sparsity and improved robustness at longer sequence lengths without the computational overhead of projection methods.
Outcome: The proposed mechanism produces >99 % exact zeros and eliminates attention sinks while maintaining competitive performance on standard and long-context benchmarks.
Mitigating the Alignment Tax of RLHF (2024.emnlp-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, which is also known as the alignment tax.
Approach: They propose to use a model averaging technique to find the most powerful alignment-forging Pareto front among RLHF algorithms.
Outcome: The proposed method achieves the strongest alignment-forging Pareto front among competing methods.
Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citation (2025.findings-acl)

Copied to clipboard

Challenge: citation generation and retrieval-augmented generation are still lacking in large language models due to hallucinations.
Approach: They propose a retrieval-augmented citation generation task that requires models to generate citations considering both external and internal knowledge while providing trustworthy references.
Outcome: The proposed method achieves better performance across scenarios compared to baselines . retrieval quality, question types, and model knowledge influence trustworthiness .
FedGUI: Benchmarking Federated GUI Agents across Heterogeneous Platforms, Devices, and Operating Systems (2026.findings-acl)

Copied to clipboard

Challenge: a lack of benchmarks capture real-world, cross-platform heterogeneity in GUI training . traditional methods to train GUI agents rely on centralized data collection and manual labeling .
Approach: They propose a benchmark for developing and evaluating federated GUI agents across mobile, web and desktop platforms.
Outcome: The proposed benchmarks show that cross-platform collaboration improves performance and identify platform and OS as the most influential factors.
Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding (2024.findings-emnlp)

Copied to clipboard

Challenge: Medical Information Extraction (MIE) tasks are a fundamental component of medical NLP.
Approach: They propose an alternative adaptive constraint strategy to adjust the scale and scope of contrastive tokens.
Outcome: The proposed approach selectively enhances the identification and classification capabilities while minimizing the influence of other inherent abilities in LLMs.
From Selection to Generation: A Survey of LLM-based Active Learning (2025.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have been used for selection and training of data for active learning.
Approach: They propose an intuitive taxonomy that categorizes LLM-based active learning techniques and discuss the transformative roles they can play in the active learning loop.
Outcome: The proposed model can generate entirely new data instances and provide more cost-effective annotations with fewer labeled data instances.
End-to-End Conversational Search for Online Shopping with Utterance Transfer (2021.emnlp-main)

Copied to clipboard

Challenge: a new study proposes a conversational search system that integrates product attributes and dialog with search . but it faces two real world challenges: imperfect product schema/knowledge and lack of training dialog data .
Approach: They propose an end-to-end conversational search system that integrates search with text . they propose an utterance transfer approach that generates dialogue utterations from other domains .
Outcome: The proposed system outperforms the best tested baseline in a conversational search dataset for online shopping.
MotivGraph-SoIQ: Integrating Motivational Knowledge Graphs and Socratic Dialogue for Enhanced LLM Ideation (2025.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models (LLMs) have limitations in grounding ideas and mitigating confirmation bias during refinement.
Approach: They propose a framework that integrates a Motivational Knowledge Graph with a Q-Driven Socratic Ideator to enhance LLM ideation.
Outcome: The proposed framework enhances LLM ideation by integrating a Motivational Knowledge Graph with a Q-Driven Socratic Ideator.
Alleviating Hallucinations from Knowledge Misalignment in Large Language Models via Selective Abstention Learning (2025.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) suffer from severe hallucination issues due to the knowledge misalignment between the pre-training stage and the supervised fine-tuning stage.
Approach: They propose a training objective with an abstention mechanism that selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token.
Outcome: The proposed model selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token.
Reasoning over Precedents Alongside Statutes: Case-Augmented Deliberative Alignment for LLM Safety (2026.acl-long)

Copied to clipboard

Challenge: OpenAI introduces deliberative alignment (DA) to enhance safety of its o-series models, but effectiveness of this approach in open-source LLMs is understudied.
Approach: They propose a case-augmented deliberative alignment method for large language models . they propose to use reinforcement learning on self-generated safety reasoning chains .
Outcome: The proposed method avoids narrowly enumerated rules and allows broader adaptability.
Few-Shot Dialogue Summarization via Skeleton-Assisted Prompt Transfer in Prompt Tuning (2024.eacl-long)

Copied to clipboard

Challenge: Existing prompt transfer techniques lack consideration for dialogue-specific information.
Approach: They propose a method which leverages skeleton generation as extra supervision that functions as a medium connecting the distinct source and target task.
Outcome: The proposed method significantly outperforms baselines on two dialogue summarization benchmarks.
Generate First, Then Sample: Enhancing Fake News Detection with LLM-Augmented Reinforced Sampling (2025.acl-long)

Copied to clipboard

Challenge: Existing models have a performance gap of 20% between classifying fake news and real news, making them less suitable for practical deployment.
Approach: They propose to adopt an LLM to generate fake news in three different styles, which are later incorporated into the training set to augment the representation of fake news.
Outcome: The proposed model achieves state-of-the-art performance on two benchmark datasets and improves detection accuracy by 24.02% and 11.06% respectively.
Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language (2025.acl-long)

Copied to clipboard

Challenge: Existing datasets for instruction-following are monolingual and centered on English . existing data are unable to capture linguistic and cultural subtle differences .
Approach: They propose an extension of IFEval to a localized multilingual version called Marco-Bench-MIF . their benchmark addresses linguistic constraints and cultural references via translation and verification .
Outcome: The proposed extension of IFEval to a localized multilingual version covers 30 languages with varying levels of localization.
Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddings (2026.acl-long)

Copied to clipboard

Challenge: Large language models produce powerful text embeddings, but their causal attention mechanism restricts the flow of information from later to earlier tokens, harming performance.
Approach: They propose a method that prepending a single summary token to reduce attention-level compression by partitioning the input into blocks and prepending blocks to subsequent blocks.
Outcome: The proposed method achieves consistent performance gains across 11 retrieval datasets and 30 general embedding benchmarks.
MiCRo: Mixture Modeling and Context-aware Routing for Personalized Preference Learning (2025.emnlp-main)

Copied to clipboard

Challenge: Existing reward models assume a global reward function, limiting personalization and pluralistic alignment.
Approach: They propose a framework that leverages binary preference datasets to enhance personalized preference learning.
Outcome: The proposed framework captures diverse human preferences without fine-grained annotations and significantly improves personalized preference learning on downstream tasks.
Learning How and What to Memorize: Cognition-Inspired Two-Stage Optimization for Evolving Memory (2026.acl-long)

Copied to clipboard

Challenge: Existing memory systems rely on static, hand-crafted update rules for personalization, but sparse outcome rewards provide weak supervision, resulting in unstable long-horizon optimization.
Approach: They propose a memory guideline optimization framework that learns how memory should be organized and what information to update.
Outcome: The proposed framework learns how memory should be organized and what information to update.
An Empirical Study on Neural Keyphrase Generation (2021.naacl-main)

Copied to clipboard

Challenge: Recent years have seen a flourishing of neural keyphrase generation (KPG) works, including the release of several large-scale datasets and a host of new models to tackle them.
Approach: They propose to compare the generalizability of KPG models with other models by analyzing the most crucial factors that may affect their generalizarability.
Outcome: The proposed model can be used to predict keyphrases from a set of input sequences, and it can be compared with existing models.
Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks evaluate multi-session memory in text-only conversations or assess multimodal understanding within localized contexts.
Approach: They propose a benchmark for evaluating multimodal long-term conversational memory in MLLM agents.
Outcome: The proposed framework assesses key memory capabilities along three functional dimensions: memory extraction and test-time adaptation, memory reasoning, and memory knowledge management.
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset.
Approach: They propose an industrial-friendly, expert-aligned and diversity-preserved instruction data selection method: Clustering and Ranking (CaR).
Outcome: The proposed method outperforms Alpaca's existing methods by 32.1% in GPT-4 evaluations.
Personalized Federated Learning for Text Classification with Gradient-Free Prompt Tuning (2024.findings-naacl)

Copied to clipboard

Challenge: Pretrained language models (PLMs) are used for personalized federated learning . communication costs are high with large PLMs, and local training is expensive .
Approach: They propose a framework for federated learning with pretrained language models . they propose 'discrete local search' and compression mechanism for local training .
Outcome: The proposed framework achieves superior performance compared with baselines.
CiteLab: Developing and Diagnosing LLM Citation Generation Workflows via the Human-LLM Interaction (2025.acl-demo)

Copied to clipboard

Challenge: Existing frameworks for enabling Large Language Models to generate citations are lacking . however, they can still produce hallucinated responses that are non-factual or irrelevant to the input.
Approach: They propose an open-source and modular framework for enabling LLMs to generate citations in Question-Answering tasks.
Outcome: The proposed framework is extensible and paired with a visual interface, Citefix, facilitating case study and modification of existing citation generation methods.
On the Emotion Understanding of Synthesized Speech (2026.acl-long)

Copied to clipboard

Challenge: Existing models for emotion understanding do not capture fundamental features of synthesized speech.
Approach: They evaluate emotion recognition models on synthesized speech using SER models and generative models.
Outcome: The proposed model can't generalize to synthesized speech because of speech token prediction . generative models tend to infer emotion from textual semantics while ignoring paralinguistic cues.
Thinking with Reasoning Skills: Fewer Tokens, More Accuracy (2026.acl-industry)

Copied to clipboard

Challenge: Reasoning LLMs often spend tokens on long intermediate reasoning traces when solving new problems.
Approach: They propose to store reusable reasoning skills distilled from extensive deliberation and trial-and-error exploration and retrieve these skills at inference time to guide future reasoning.
Outcome: The proposed approach reduces reasoning tokens while improving overall performance on coding and mathematical reasoning tasks.
PAPERMIND: Benchmarking Agentic Reasoning and Critique over Scientific Papers in Multimodal LLMs (2026.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks assess integrated and agent-oriented scientific reasoning in isolation . Existing systems assess integrated reasoning in isolated tasks .
Approach: They propose a benchmark to evaluate integrated and agent-oriented scientific reasoning over research papers.
Outcome: The proposed benchmark evaluates integrated and agent-oriented scientific reasoning over scientific papers.
PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering (2026.acl-long)

Copied to clipboard

Challenge: Current outcome-centric verification paradigms neglect potential errors in the derivation process.
Approach: They propose a process-aware RLVR training paradigm utilizing verifiers selected via **PRIME**.
Outcome: The proposed approach outperforms the baseline verification paradigm on AIME24, AIME25, and Beyond-AIME models.
Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods to compress context information ignore holistic contextual dependencies.
Approach: They propose a method that adjusts position encodings to minimize the distance between context tokens and special tokens.
Outcome: Enhanced Position Layout (EPL) improves compression of context information in large language models.
Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models (2024.acl-long)

Copied to clipboard

Challenge: Existing methods for retrieval augmentation work with chunked contexts, which leads to poor quality of semantic representation and incomplete retrieval of useful information.
Approach: They propose a method for retrieval augmentation of long-context language modeling using landmark embedding.
Outcome: The proposed method outperforms existing retrieval methods with a notable advantage.
An Adaptive Prompt Generation Framework for Task-oriented Dialogue System (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing black-box large language models (LLMs) have excellent performance in task-oriented dialogue (TOD) tasks, but obtaining suitable prompts for specific tasks is challenging.
Approach: They propose a black-box large language model that generates domain and slot information in the belief state, which serves as prior knowledge for subsequent prompt generation.
Outcome: The proposed framework outperforms existing prompting methods on the MultiWOZ 2.0 dataset.
Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards (2024.acl-long)

Copied to clipboard

Challenge: Reinforcement Learning from Human Feedback (RLHF) relies on scalar rewards to capture user preferences.
Approach: They propose a framework that integrates multi-objective reward modeling to represent diverse preference profiles.
Outcome: The proposed method improves performance across reward objectives and targets.
EventKE: Event-Enhanced Knowledge Graph Embedding (2021.findings-emnlp)

Copied to clipboard

Challenge: Experimental results show that events can greatly improve the quality of KG embeddings on multiple downstream tasks.
Approach: They propose an event-enhanced KG embedding model that incorporates events into KGs . they first incorporate event nodes by building a heterogeneous network with event argument links .
Outcome: The proposed model incorporates event nodes into the original knowledge graphs . it can be used to fuse event information into the KG embeddings on multiple tasks .
AMoPO: Adaptive Multi-objective Preference Optimization without Reward Models and Reference Models (2025.findings-acl)

Copied to clipboard

Challenge: Existing multi-objective preference alignment methods for large language models face limitations such as auxiliary reward/reference models and computational complexity.
Approach: They propose a framework that achieves dynamic balance across preference dimensions by using dimension-aware generation metrics as implicit rewards.
Outcome: Empirical results show that AMoPO outperforms state-of-the-art methods by 28.5% .
Double-Checker: Large Language Model as a Checker for Few-shot Named Entity Recognition (2024.findings-emnlp)

Copied to clipboard

Challenge: Recent studies have demonstrated remarkable performance on few-shot Named Entity Recognition tasks due to the high cost of obtaining high-quality labeled data.
Approach: They propose to decompose the task into entity span detection and entity type classification using a type-independent entity span detector and then classify the detected spans based on their types.
Outcome: The proposed method consistently yields improvements over two baseline approaches.
Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts (2024.findings-emnlp)

Copied to clipboard

Challenge: Reinforcement learning from human feedback (RLHF) is the primary method for aligning large language models with human preferences.
Approach: They propose to train an Absolute-Rating Multi-Objective Reward Model with multi-dimensional absolute-rating data.
Outcome: The proposed model outperforms the LLM-as-a-judge method on RewardBench . it achieves state-of-the-art performance on the benchmark .
GLIMPSE: Do Large Vision-Language Models Truly Think With Videos or Just Glimpse at Them? (2025.emnlp-main)

Copied to clipboard

Challenge: Existing video benchmarks often resemble image-based questions with scans of only a few key frames, without deep temporal reasoning.
Approach: They propose a video benchmark to assess whether large vision-language models can genuinely think with videos rather than perform superficial frame-level analysis.
Outcome: The proposed benchmark consists of 3,269 videos and over 4,342 highly visual-centric questions across 11 categories, including Trajectory Analysis, Temporal Reasoning, and Forensics Detection.
Aligning as Debiasing: Causality-Aware Alignment via Reinforcement Learning with Interventional Feedback (2024.naacl-long)

Copied to clipboard

Challenge: Existing methods to reduce LLMs' biased outputs rely on reward signals from current model outputs without considering the source of biases.
Approach: They propose to leverage the reward model in RL alignment as an instrumental variable to perform causal intervention on LLMs.
Outcome: The proposed method reduces biases by using human feedback to fine tune LLMs to human values.
Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-Context Models (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods, tasks and benchmarks to measure model’s effective memory length are limited.
Approach: They propose a method called forgetting curve to measure the memorization capability of long-context models.
Outcome: The proposed method is robust to the tested corpus and experimental settings, and can be applied to any model size.
CogMG: Collaborative Augmentation Between Large Language Model and Knowledge Graph (2024.acl-demos)

Copied to clipboard

Challenge: Large language models (LLMs) are susceptible to generating hallucinated content and often encompass factually inaccurate information.
Approach: They propose a framework that leverages knowledge graphs to address the limitations of Large Language Models (LLMs) they identify and decompose required knowledge triples that are not present in the KG, enriching them and aligning updates with real-world demands.
Outcome: The proposed framework reduces hallucinations and increases factual accuracy in QA scenarios while retaining the same quality of knowledge.
MixGR: Enhancing Retriever Generalization for Scientific Domain through Complementary Granularity (2024.emnlp-main)

Copied to clipboard

Challenge: Recent studies show the importance of document retrieval in the scientific domain.
Approach: They propose a zero-shot approach to measure query-document similarity using atomic components in queries and documents to combine them into a united score.
Outcome: The proposed approach outperforms previous document retrieval methods by 24.7%, 9.8%, and 6.9% on nDCG@5 with unsupervised, supervised, and LLM-based retrievers.
Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment (2026.acl-long)

Copied to clipboard

Challenge: Existing methods assess suitability primarily through student likelihood, favoring trajectories that align closely with the student model’s current behavior but overlooking more informative ones.
Approach: They propose a Rank–Surprisal Ratio metric that captures both alignment and informativeness to assess the suitability of a reasoning trajectory.
Outcome: The proposed metric captures both alignment and informativeness to assess the suitability of a reasoning trajectory.
Sentence-Permuted Paragraph Generation (2021.emnlp-main)

Copied to clipboard

Challenge: Existing models produce similar contents from homogenized contexts due to the fixed left-to-right sentence order.
Approach: They propose a framework permuting sentence orders to improve content diversity of multi-sentence paragraphs by permutating the sentence orders.
Outcome: The proposed framework produces more diverse outputs with higher quality than existing models.
Graph-based Multilingual Product Retrieval in E-Commerce Search (2021.naacl-industry)

Copied to clipboard

Challenge: Modern e-commerce search systems require product retrieval under multilingual scenarios.
Approach: They propose a universal multilingual retrieval system that captures interactions between search queries and items in e-commerce search.
Outcome: The proposed system outperforms state-of-the-art retrieval models on five countries and has been deployed in production for multiple countries.
CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling (2026.acl-long)

Copied to clipboard

Challenge: a novel architecture that enables LLMs to handle arbitrarily long sequences with constant memory usage and linear time complexity is a major barrier to long-context processing.
Approach: They propose a novel architecture that enables LLMs to handle arbitrarily long sequences with constant memory usage and linear time complexity.
Outcome: The proposed architecture can handle arbitrarily long sequences with constant memory usage and linear time complexity.
Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answering (2022.findings-emnlp)

Copied to clipboard

Challenge: Open-domain question answering (QA) models employ a retriever-reader pipeline . however, state-of-the-art readers fail to capture complex relationships between entities .
Approach: They propose a knowledge graph enhanced passage reader that captures entities in questions and retrieved passages.
Outcome: The proposed knowledge graph enhanced passage reader improves on open-domain QA benchmarks by up to 2.2 exact match scores.
Following Occam’s Razor: Dynamic Combination of Structured Knowledge for Multi-Hop Question Answering using LLMs (2025.findings-emnlp)

Copied to clipboard

Challenge: Multi-hop question answering is a challenging task that requires capturing information from multiple positions in multiple documents.
Approach: They propose a framework for integrating text-based and triple-based paradigms that incorporates structured knowledge into large-scale question answering.
Outcome: The proposed framework improves multi-hop question answering by incorporating structured knowledge into the models.
Discovering Low-rank Subspaces for Language-agnostic Multilingual Representations (2022.emnlp-main)

Copied to clipboard

Challenge: Existing studies show that pre-trained ML-LMs can achieve zero-shot cross-lingual transfer without explicit cross-linguistic supervision.
Approach: They propose a method to remove language-specific factors from multilingual embedding spaces by using a single value decomposition method with multiple monolingual corpora as input.
Outcome: The proposed method can boost language agnosticism without finetuning . Empirical results show that it consistently leads to improvements over existing models.
Multi-Input Multi-Output Sequence Labeling for Joint Extraction of Fact and Condition Tuples from Scientific Text (D19-1)

Copied to clipboard

Challenge: Existing methods to extract factual tuples from scientific text do not consider conditions.
Approach: They propose a new sequence labeling framework to jointly extract fact and condition tuples from scientific sentences.
Outcome: The proposed framework improves F1 score relative to existing methods by 4.2% and 6.2% on bioNLP2013.
WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling (2026.findings-acl)

Copied to clipboard

Challenge: Recent advances in training optimization for Transformer-based large language models lack systematic optimization of weight patterns during training.
Approach: They propose a Weight Scaling method that rescales weights while preserving model outputs to improve model training efficiency and model quality.
Outcome: The proposed method significantly improves convergence quality and loss reduction in LLMs with Grouped Query Attention architectures and LoRA fine-tuning tasks.
Few-Shot Class-Incremental Learning for Named Entity Recognition (2022.acl-long)

Copied to clipboard

Challenge: Existing models of Named Entity Recognition (NER) are trained on large datasets with predefined entity classes, but data of new classes arrives constantly. Existing work on NER relies on the assumption that there exists abundance of labeled data for the training of new class.
Approach: They propose a few-shot class-incremental learning problem where NER model is trained with only few labeled samples of the new classes without forgetting knowledge of the old ones.
Outcome: The proposed model improves over existing baselines by reconstructing training data of old classes and real data from the training set.
Automatic ICD Coding via Interactive Shared Representation Networks with Self-distillation Mechanism (2021.acl-long)

Copied to clipboard

Challenge: Existing methods for ICD coding ignore the long-tail of code frequency or noisy clinical notes.
Approach: They propose to use an interactive shared representation network to model code co-occurrences while focusing on the clinical note's noteworthy part and extract valuable information through a self-distillation learning mechanism to solve the long-tail problem.
Outcome: The proposed model reduces the long-tail of code frequency and noise in clinical notes and extracts valuable information through a self-distillation learning mechanism.
MemRec: Collaborative Memory-Augmented Agentic Recommender System (2026.acl-long)

Copied to clipboard

Challenge: Existing recommender systems rely on semantic user and item memories to make predictions, but these memories are kept in isolation.
Approach: They propose a framework that architecturally decouples memory management from reasoning to decouple memory management and reasoning from the user and item memories.
Outcome: The proposed framework decouples memory management from reasoning and achieves state-of-the-art performance on four benchmarks.
DTELS: Towards Dynamic Granularity of Timeline Summarization (2025.naacl-long)

Copied to clipboard

Challenge: Existing timeline summarizations lack flexibility to meet diverse granularity needs . a fine-grained timeline showing the technical details is preferred for news topics .
Approach: They propose a new paradigm to construct adaptive timelines based on user instructions or requirements.
Outcome: The proposed timelines are informative and granularly consistent, but they struggle to generate consistent timelines.
Learning Adaptive Axis Attentions in Fine-tuning: Beyond Fixed Sparse Attention Patterns (2022.findings-acl)

Copied to clipboard

Challenge: Adaptive Axis Attention learns different attention patterns for each task and model layer . sparse attention patterns do not improve the run time of the models but they reduce model memory requirements .
Approach: They propose a method that learns different attention patterns for each Transformer layer . they propose 'adaptive axis attention' method that identifies important tokens .
Outcome: The proposed method does not require pre-training to accommodate sparse attention patterns.
Towards Explainable Diagnosis: A Self-learned Explanatory Knowledge Base Approach (2026.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) have great potential to facilitate explainable diagnosis, but their effectiveness is often constrained by insufficient diagnostic expertise.
Approach: They propose a unified LLM-based framework for faithful and explainable diagnosis that builds a high-quality diagnostic knowledge base through a record-driven explanation learning paradigm.
Outcome: The proposed framework outperforms baselines on the DiReCT and JAMA benchmarks and improves the explanation completeness metric from 64.5% to 76.9% over the best existing methods.
ATIR: Towards Audio-Text Interleaved Contextual Retrieval (2026.acl-long)

Copied to clipboard

Challenge: Recent multimodal information retrieval research has focused on images, largely overlooking audio.
Approach: They propose an audio-text interleaved contextual retrieval task where queries can alternate between audio and text modalities.
Outcome: The proposed model significantly improves over baselines.
A Critical Analysis of Document Out-of-Distribution Detection (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing document understanding models focus on single-modal inputs such as images or texts.
Approach: They propose to use a spatial-aware adapter to adapt transformer-based language models to document domain to exploit multi-modal information.
Outcome: The proposed model significantly improves the OOD detection performance compared to using a standard language model and to competitive baselines.
Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation (2021.acl-long)

Copied to clipboard

Challenge: Existing approaches to incorporate bilingual dictionaries into Neural Machine Translation (NMT) models have been criticized for lack of integration of bilingual lexical information into the neural architecture.
Approach: They propose a neural architecture to incorporate bilingual dictionaries into Neural Machine Translation models by introducing three new components: Pointer, Disambiguator, and Copier.
Outcome: The proposed method achieves the following merits inherently compared with previous efforts: (1) Pointer leverages the semantic information from bilingual dictionaries, for the first time, to better locate source words whose translation in dictionary can potentially be used; (2) Disambiguator synthesizes contextual information from source view and target view, both of which contribute to distinguishing translation of a specific source word from multiple candidates in dicaries; (3) Copier systematically connects Pointer and Disambiguators based on a hierarchical
MRT: Multi-modal Short- and Long-range Temporal Convolutional Network for Time-sync Comment Video Behavior Prediction (2024.lrec-main)

Copied to clipboard

Challenge: Using time-sync comments, it is difficult to understand user behavior due to complexity of interactions between users, videos, and comments.
Approach: They propose a novel time-sync comment behavior prediction model that takes historical behavior into account and optimizes it on the basis of user preferences.
Outcome: The proposed model improves the performance of time-sync comments on visual frames and textual comments on two cats playing simultaneously.
Dense X Retrieval: What Retrieval Granularity Should We Use? (2024.emnlp-main)

Copied to clipboard

Challenge: a learned dense retrieval model is often overlooked when using a corpus for inference, resulting in a design choice of retrieval unit . granularity of retrievals is important for both retrieval and downstream tasks .
Approach: They propose a retrieval unit for dense retrieval that uses propositions to index corpus . propositions are defined as atomic expressions within text, each encapsulating a distinct factoid .
Outcome: The proposed retrieval unit outperforms passage-level units on retrieval and downstream tasks.
Federated Domain Adaptation for Named Entity Recognition via Distilling with Heterogeneous Tag Sets (2023.findings-acl)

Copied to clipboard

Challenge: a federated domain adaptation approach is used to learn with NER datasets from multiple platforms while not violating data privacy.
Approach: They propose to use a distillation approach to facilitate knowledge transfer across platforms.
Outcome: The proposed model performs better in the clinic domain.
Improving Speech Translation by Fusing Speech and Text (2023.findings-emnlp)

Copied to clipboard

Challenge: In speech translation, multimodal data to address limitations of individual modalities has shown significant effectiveness.
Approach: They propose a cross-modal model which supports three input modalities for speech, text and fused speech-text.
Outcome: The proposed model achieves an average of 34.0 BLEU on MuST-C, GigaST and newstest benchmark.
Multi-perspective Improvement of Knowledge Graph Completion with Large Language Models (2024.lrec-main)

Copied to clipboard

Challenge: Knowledge graph completion (KGC) is a widely used method to tackle incompleteness in knowledge graphs (KGs).
Approach: They propose a general framework to compensate for the deficiency of contextualized knowledge by querying large language models from various perspectives.
Outcome: The proposed framework improves knowledge graph completion (KGC) by querying large language models from various perspectives.
Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Training (2022.findings-naacl)

Copied to clipboard

Challenge: Existing studies on keyphrase generation on non-English languages haven’t been vastly investigated.
Approach: They propose a retrieval-augmented method for multilingual keyphrase generation that leverages keyphrase annotations in English datasets to facilitate generating keyphrases in low-resource languages.
Outcome: The proposed model outperforms baselines on non-English keyphrase generation datasets and the proposed model is scalable.
RLShield: Dynamic Jailbreak Detection for LLMs via Reinforced Adaptive Learning (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to detect jailbreak prompts rely on static model components or fixed decision thresholds.
Approach: They propose a dynamic jailbreak detection framework that employs reinforcement learning for adaptive threshold selection.
Outcome: Experimental results show that the framework outperforms baselines in detection performance while maintaining high computational efficiency.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations