Papers by Yu Ji
UniCodec: Unified Audio Codec with Single Domain-Adaptive Codebook (2025.acl-long)
Copied to clipboard
Yidi Jiang, Qian Chen, Shengpeng Ji, Yu Xi, Wen Wang, Chong Zhang, Xianghu Yue, ShiLiang Zhang, Haizhou Li
| Challenge: | Existing neural audio codecs are not capable of handling multi-domain audio data . et al., 2023) integrate speech modality with text-based large language models . |
| Approach: | They propose a unified audio codec with a single codebook to support multi-domain audio data . they propose combining a mix-of-experts strategy and a partitioned domain-adaptive codebook method . |
| Outcome: | The proposed codec outperforms existing codecs on acoustic and semantic representation capabilities. |
When to Trust Tools? Adaptive Tool Trust Calibration For Tool-Integrated Math Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large reasoning models have performance enhancements but still suffer from shortcomings due to limitations of the underlying language models. |
| Approach: | They propose a framework that allows the model to choose when to trust or ignore the tool results based on the confidence score of generated code blocks. |
| Outcome: | The proposed framework reduces the "Tool Ignored" issue by 4.1% to 7.5%. |
Bridging the Gap between Native Text and Translated Text through Adversarial Learning: A Case Study on Cross-Lingual Event Extraction (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent research in cross-lingual learning has found that combining large-scale pretrained multilingual language models with machine translation can yield good performance. |
| Approach: | They propose a model architecture that jointly encodes a source language input sentence with its translation to the target language during training and takes a target language sentence with it as input during evaluation. |
| Outcome: | The proposed model architecture can integrate machine translation to improve event extraction while adding machine-translated data yields unstable performance due to representational gap. |
The Art of Abstention: Selective Prediction and Error Regularization for Natural Language Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models have improved the state-of-the-art results on many NLP applications. |
| Approach: | They propose a simple error regularization trick that improves confidence estimation without substantially increasing the computation budget. |
| Outcome: | The proposed regularization improves confidence estimation without increasing computation budget. |
DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections (2026.findings-acl)
Copied to clipboard
| Challenge: | Several studies rely on additional models to optimize mixtures. |
| Approach: | They propose a method that dynamically optimizes instruction-tuning dataset mixtures by prior-scaled Boltzmann Exploration and a multi-armed bandit setup. |
| Outcome: | The proposed method improves the TÜLU-2-mixture and TÜLO-3-mixtures across 10 benchmarks while introducing minimal computational overhead over naive sampling. |
Few-shot Named Entity Recognition with Entity-level Prototypical Network Enhanced by Dispersedly Distributed Prototypes (2022.coling-1)
Copied to clipboard
| Challenge: | Existing prototypical networks for named entity recognition suffer from label dependency and tightly distributed prototypes, thus causing misclassifications. |
| Approach: | They propose an Entity-level Prototypical Network enhanced by dispersedly distributed prototypes to build entity-level prototypes and distribute them dispersionally. |
| Outcome: | The proposed system outperforms the previous models on two evaluation tasks and the Few-NERD settings in terms of overall performance. |
Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for aspect-based sentiment analysis of review text use only a few keywords describing each aspect/sentiment without using any labeled examples. |
| Approach: | They propose a weakly-supervised approach for aspect-based sentiment analysis which uses only a few keywords describing each aspect/sentiment without using any labeled examples. |
| Outcome: | The proposed method generates quality joint topics and outperforms baselines significantly on benchmark datasets. |
LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Currently, large language models (LLMs) train on short text segments due to the computational overhead quadratic in the input lengths of their Transformer architectures. |
| Approach: | They propose a method that allows LLMs pre-trained with 2K or 4K-long segments to generalize to up to 200M length inputs while retaining perplexity. |
| Outcome: | The proposed method achieves 2.7 decoding speed up and 7.5 memory saving over the original model. |
A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing surveys on scientific LLMs focus on one or two fields or a single modality. |
| Approach: | They survey 260 scientific LLMs and examine their architectures and pre-training techniques . they also discuss commonalities and differences between LLM architectures . |
| Outcome: | The proposed model architectures and evaluation techniques are used to improve scientific discovery. |
The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding (2020.acl-demos)
Copied to clipboard
Xiaodong Liu, Yu Wang, Jianshu Ji, Hao Cheng, Xueyun Zhu, Emmanuel Awa, Pengcheng He, Weizhu Chen, Hoifung Poon, Guihong Cao, Jianfeng Gao
| Challenge: | MT-DNN is an open-source natural language understanding toolkit . it allows researchers and developers to train customized deep learning models . |
| Approach: | They present MT-DNN, an open-source natural language understanding toolkit . it is designed to facilitate rapid customization for a broad spectrum of NLU tasks . MT supports multi-task knowledge distillation, which can substantially compress a deep neural model without significant performance drop. |
| Outcome: | The proposed model can significantly compress a large model without significant performance drop. |
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks focus on pairwise matching, ignoring robustness . current models exhibit frustrating degradation, with a maximum drop of 23.43 F1 score . |
| Approach: | They propose a benchmark that simulates the evaluation of open information extraction models in the real world . they perform experiments on typical models published in the last decade and a representative large language model . |
| Outcome: | The proposed model is rated robust on a knowledge-invariant clique with different syntactic and expressive forms. |
CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Models (2024.findings-acl)
Copied to clipboard
Linhao Yu, Yongqi Leng, Yufei Huang, Shang Wu, Haixin Liu, Xinmeng Ji, Jiahui Zhao, Jinwang Song, Tingting Cui, Xiaoqing Cheng, Liutao Liutao, Deyi Xiong
| Challenge: | Recent years have witnessed remarkable progress achieved by large language models in both natural language understanding and generation. |
| Approach: | They propose a large benchmark CMoralEval for moral evaluation of Chinese LLMs . they use a Chinese TV program discussing Chinese moral norms and Chinese moral anomies based on various sources . |
| Outcome: | The proposed dataset is characterized by diversity and authenticity. |
Augment before You Try: Knowledge-Enhanced Table Question Answering via Table Expansion (2025.findings-emnlp)
Copied to clipboard
Yujian Liu, Jiabao Ji, Tong Yu, Ryan A. Rossi, Sungchul Kim, Handong Zhao, Ritwik Sinha, Yang Zhang, Shiyu Chang
| Challenge: | Existing methods to integrate external information into a given table neglect the structured nature of the table. |
| Approach: | They propose a simple yet effective method to integrate external information into a given table by first building an augmenting table and then generating a SQL query over the two tables to answer the question. |
| Outcome: | The proposed method outperforms strong baselines on three table QA benchmarks. |
LORE: Continual Logit Rewriting Fosters Faithful Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using aspect-oriented summarization as a case study, we propose **LOgit REwriting**, a new controlled generation paradigm which can be faithful to external knowledge and to the LLM’s intentions. |
| Approach: | They propose a controlled generation paradigm which can be faithful to external knowledge and to the LLM's intentions. |
| Outcome: | The proposed paradigm can be faithful to external knowledge and to the LLM's intentions while balancing that with accuracy. |
A Deep Relevance Model for Zero-Shot Document Filtering (P18-1)
Copied to clipboard
| Challenge: | Existing methods for document classification do not consider document filtering . existing methods do not include document filter. |
| Approach: | They propose a deep relevance model for zero-shot document filtering called DAZER . they use word embeddings to extract the relevance signals from word embeds . |
| Outcome: | The proposed model outperforms existing models on two document collections . it estimates the relevance between a document and a category by using seed words . |
OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safety (2024.acl-demos)
Copied to clipboard
Chuang Liu, Linhao Yu, Jiaxuan Li, Renren Jin, Yufei Huang, Ling Shi, Junhui Zhang, Xinmeng Ji, Tingting Cui, Liutao Liutao, Jinwang Song, Hongying Zan, Sun Li, Deyi Xiong
| Challenge: | a rapid development of Chinese large language models poses big challenges for efficient LLM evaluation. |
| Approach: | They propose an evaluation testbed that benchmarks Chinese LLMs across capability, alignment and safety. |
| Outcome: | The evaluation platform OpenEval benchmarks Chinese LLMs across capability, alignment and safety. |
Evaluating Multimodal Large Language Models on Video Captioning via Monte Carlo Tree Search (2025.acl-long)
Copied to clipboard
Linhao Yu, Xingguang Ji, Yahui Liu, Fanheng Kong, Chenxi Sun, Jingyuan Zhang, Hongzhi Zhang, V. W., Fuzheng Zhang, Deyi Xiong
| Challenge: | Existing benchmarks and evaluation protocols suffer from inadequate or homogeneous creation of key points, exorbitant cost of data creation, and limited evaluation scopes. |
| Approach: | They propose an automatic framework which leverages Monte Carlo Tree Search to construct numerous and diverse descriptive sentences that thoroughly represent video content in an iterative way. |
| Outcome: | The proposed framework improves MCTS-VCB and DREAM-1K on video captioning tasks by 25.0% and 16.3% respectively. |
Why Does New Knowledge Create Messy Ripple Effects in LLMs? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing research has focused on post-training knowledge editing (KE) for language models to ensure that knowledge remains accurate and up-to-date. |
| Approach: | They propose to use a GradSim indicator to detect when and why updated knowledge ripples in language models. |
| Outcome: | The proposed indicator GradSim shows that LMs that fail to handle ripple effects have low GradSIM. |
Text Classification Using Label Names Only: A Language Model Self-Training Approach (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current text classification methods require a large number of labeled documents as training data. |
| Approach: | They propose a model that uses only the label name of each class to train classification models on unlabeled data without using any labeled examples. |
| Outcome: | The proposed model achieves 90% accuracy on four benchmark datasets using label names as the only supervision . |
Multi-hop Question Generation with Graph Convolutional Network (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on text-based QG focus on generating SQuAD-style questions. |
| Approach: | They propose a multi-hop question generation model that does context encoding in multiple hops with Graph Convolutional Network and encoder fusion via an Encoder Reasoning Gate. |
| Outcome: | Empirical results show that the proposed model generates fluent questions with high completeness and outperforms baselines on automatic evaluation metrics. |
SParC: Cross-Domain Semantic Parsing in Context (P19-1)
Copied to clipboard
Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, Dragomir Radev
| Challenge: | Xu et al., 2017): a dataset for cross-domain semantic parsing in context with 4,298 question sequences. |
| Approach: | They present a dataset for cross-domainSemanticParsing inContext that consists of 4,298 coherent question sequences. |
| Outcome: | The proposed dataset demonstrates that it has greater semantic diversity and can be generalized to unseen domains due to its cross-domain nature and the unseened databases at test time. |
Incorporating Global Information in Local Attention for Knowledge Representation Learning (2021.findings-acl)
Copied to clipboard
| Challenge: | Graph Attention Networks (GATs) are a promising model that takes advantage of localized attention mechanism to perform knowledge representation learning (KRL) on graph-structure data. |
| Approach: | They propose to incorporate global information into the GAT family of models by using an attention-based global random walk algorithm. |
| Outcome: | Experimental results on KG entity prediction against the state-of-the-arts demonstrate the effectiveness of the proposed model. |
Social-Group-Agnostic Bias Mitigation via the Stereotype Content Model (2023.acl-long)
Copied to clipboard
Ali Omrani, Alireza Salkhordeh Ziabari, Charles Yu, Preni Golazizian, Brendan Kennedy, Mohammad Atari, Heng Ji, Morteza Dehghani
| Challenge: | Existing methods for mitigating bias require social-group-specific word pairs for each social attribute (e.g., gender) Existing approaches require only one social attribute, rendering them impractical and costly . |
| Approach: | They propose that stereotype content models capture the underlying connection between bias and stereotypes by embedding only two psychological dimensions of warmth and competence. |
| Outcome: | The proposed method performs comparably to group-specific debiasing on multiple bias benchmarks, but has theoretical and practical advantages over existing methods. |
Generative Music Models’ Alignment with Professional and Amateur Users’ Expectations (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent years have witnessed rapid advances in text-to-music generation using large language models. |
| Approach: | They propose a task to align AI-generated music with human expressions . they use a dataset of over 1.5 million songs to analyze their content . |
| Outcome: | The proposed framework outperforms baseline models and facilitates end-to-end generation of songs audio. |
EVEDIT: Event-based Knowledge Editing for Deterministic Knowledge Propagation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge editing approaches only operate on (subject, relation, object) triple . current methods are limited to (substance, relation) triple, causing low confidence in their answers. |
| Approach: | They propose a task of event-based knowledge editing that pairs facts with event descriptions to improve model confidence. |
| Outcome: | The proposed method improves model confidence by 55.6% while maintaining the naturalness of generation. |
Can Language Models Follow Multiple Turns of Entangled Instructions? (2025.findings-emnlp)
Copied to clipboard
Chi Han, Xin Liu, Haodong Wang, Shiyang Li, Jingfeng Yang, Haoming Jiang, Zhengyang Wang, Qingyu Yin, Liang Qiu, Changlong Yu, Yifan Gao, Zheng Li, Bing Yin, Jingbo Shang, Heng Ji
| Challenge: | Despite of significant achievements in improving instruction-following capabilities of large language models, the ability to process multiple potentially entangled or conflicting instructions remains a considerable challenge. |
| Approach: | They construct multi-turn instruction with 1.1K high-quality multi-turned conversations using the human-in-the-loop approach and examine their capabilities. |
| Outcome: | The proposed model shows that it is difficult to integrate multiple turns and balance competing objectives when instructions intersect or conflict. |
Knowledge-Enriched Natural Language Generation (2021.emnlp-tutorials)
Copied to clipboard
| Challenge: | Knowledge-enriched text generation poses unique challenges in modeling and learning . a roadmap will outline the state-of-the-art methods to tackle these challenges . |
| Approach: | They propose a roadmap to tackle the challenges of knowledge-enriched text generation . they will dive deep into various technical components to illustrate how to represent knowledge . |
| Outcome: | This tutorial outlines the state-of-the-art methods to tackle the problem . it aims to show how to represent knowledge, feed knowledge into a generation model, evaluate results . |
Operator Selection and Ordering in a Pipeline Approach to Efficiency Optimizations for Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Natural language processing tasks rely on complex neural models . transformer-based models are typically slow to execute, making it a non-trivial challenge to apply them in real-world applications. |
| Approach: | They propose to consider an efficiency method as an operator applied on a model . they find that the commutativity and cumulativeness of efficiency operators are plausible . |
| Outcome: | The proposed method is commutative and cumulative, and the results are estimated by combining methods. |
Skeletons Matter: Dynamic Data Augmentation for Text-to-Query (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on a single query language, resulting in limited generalizability . a new task paradigm is proposed to unify semantic parsing tasks across different query languages . |
| Approach: | They propose a task paradigm that unifies parsing tasks across query languages . they identify query skeletons as a shared optimization target of Text-to-Query tasks . |
| Outcome: | The proposed method achieves state-of-the-art performance using only a small amount of synthesized data. |
Towards Mitigating LLM Hallucination via Self Reflection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have shown promise for generative and knowledge-intensive tasks including question-answering (QA) but the practical deployment still faces challenges, notably the issue of “hallucination”, where models generate plausible-sounding but unfaithful or nonsensical information. |
| Approach: | They propose a self-reflection methodology that incorporates knowledge acquisition and answer generation to address the issue of "hallucination" they use a set of LLMs to generate a more accurate and factually accurate answer. |
| Outcome: | The proposed approach improves factuality, consistency, and entailment of the generated answers. |
RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System (2021.naacl-demos)
Copied to clipboard
Haoyang Wen, Ying Lin, Tuan Lai, Xiaoman Pan, Sha Li, Xudong Lin, Ben Zhou, Manling Li, Haoyu Wang, Hongming Zhang, Xiaodong Yu, Alexander Dong, Zhenhailong Wang, Yi Fung, Piyush Mishra, Qing Lyu, Dídac Surís, Brian Chen, Susan Windisch Brown, Martha Palmer, Chris Callison-Burch, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Heng Ji
| Challenge: | We present a new information extraction system that can construct temporal event graphs from news documents. |
| Approach: | They propose a temporal event graph extraction system that can extract news documents . they extend the system from sentence-level event extraction to cross-document cross-media event extraction . |
| Outcome: | The proposed system can extract temporal event graphs from news documents in multiple languages and multiple data modalities. |
RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios (2022.naacl-demo)
Copied to clipboard
Xinya Du, Zixuan Zhang, Sha Li, Pengfei Yu, Hongwei Wang, Tuan Lai, Xudong Lin, Ziqi Wang, Iris Liu, Ben Zhou, Haoyang Wen, Manling Li, Darryl Hannan, Jie Lei, Hyounghun Kim, Rotem Dror, Haoyu Wang, Michael Regan, Qi Zeng, Qing Lyu, Charles Yu, Carl Edwards, Xiaomeng Jin, Yizhu Jiao, Ghazaleh Kazeminejad, Zhenhailong Wang, Chris Callison-Burch, Mohit Bansal, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Martha Palmer, Heng Ji
| Challenge: | Existing methods for event prediction are incomplete and noisy. |
| Approach: | They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework . |
| Outcome: | The proposed framework can be applied to a wide variety of newsworthy scenarios. |
NusaCrowd: Open Source Initiative for Indonesian NLP Resources (2023.findings-acl)
Copied to clipboard
Samuel Cahyawijaya, Holy Lovenia, Alham Fikri Aji, Genta Winata, Bryan Wilie, Fajri Koto, Rahmad Mahendra, Christian Wibisono, Ade Romadhony, Karissa Vincentio, Jennifer Santoso, David Moeljadi, Cahya Wirawan, Frederikus Hudi, Muhammad Satrio Wicaksono, Ivan Parmonangan, Ika Alfina, Ilham Firdausi Putra, Samsul Rahmadani, Yulianti Oenang, Ali Septiandri, James Jaya, Kaustubh Dhole, Arie Suryani, Rifki Afina Putri, Dan Su, Keith Stevens, Made Nindyatama Nityasya, Muhammad Adilazuarda, Ryan Hadiwijaya, Ryandito Diandaru, Tiezheng Yu, Vito Ghifari, Wenliang Dai, Yan Xu, Dyah Damapuspita, Haryo Wibowo, Cuk Tho, Ichwanul Karo Karo, Tirana Fatyanosa, Ziwei Ji, Graham Neubig, Timothy Baldwin, Sebastian Ruder, Pascale Fung, Herry Sujaini, Sakriani Sakti, Ayu Purwarianti
| Challenge: | Existing NLP research in Indonesian languages has been held back by factors such as language diversity, orthographic variation, resource limitation and other societal challenges. |
| Approach: | They present a collaborative initiative to collect and unify existing resources for Indonesian languages and open access to previously non-public resources. |
| Outcome: | The results show that the datasets are highly reliable and can be used to generate the first zero-shot benchmarks for natural language understanding and generation in Indonesian and the local languages of Indonesia. |
Lifelong Event Detection with Knowledge Transfer (2021.emnlp-main)
Copied to clipboard
| Challenge: | Traditional supervised Information Extraction (IE) methods can extract structured knowledge elements from unstructured data, but it is limited to a pre-defined target ontology. |
| Approach: | They propose a new lifelong event detection framework that is generalizable to other IE tasks and updates old knowledge with new event types’ mentions using a self-training loss. |
| Outcome: | The proposed framework outperforms baselines with a 5.1% gain in the F1 score and can boost the F2 score for over 30% on some new long-tail rare event types with few training instances. |
Deep Learning on Graphs for Natural Language Processing (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | Graph Neural Networks (GNNs) are powerful tools for non-Euclidean data modeling and are used in many graph-related NLP tasks. |
| Approach: | This tutorial will cover applying deep learning on graph techniques to NLP using Graph Neural Networks (GNNs) Graph4NLP is the first library for researchers and practitioners for easy use of GNNs for various NLP tasks. |
| Outcome: | This tutorial will cover the latest developments in deep learning on graph techniques and their applications in various NLP tasks. |
What Part of the Neural Network Does This? Understanding LSTMs by Measuring and Dissecting Neurons (D19-1)
Copied to clipboard
| Challenge: | Biological neural systems consist of a huge number of neurons, and can react to the environment in complicated ways. |
| Approach: | They propose a metric to quantify the sensitivity of neurons to each label and conduct experiments to prove it. |
| Outcome: | The proposed metric is based on a set of experiments that show that dropping an arbitrary neuron significantly degrades the accuracy of the model. |
Document-Level Machine Translation with Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) such as ChatGPT can produce coherent, cohesive, relevant, and fluent answers for various natural language processing tasks. |
| Approach: | They examine the impact of different prompts on document-level translation quality and discourse phenomena using figures and lines, which are invisible to GPT-4. |
| Outcome: | The proposed models outperform commercial MT systems and advanced document-level MT methods on a number of benchmarks and show potential to become a new paradigm for document- level translation. |
Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing semi-parametric language models lack the capacity to perform zero-shot tasks . large language models have shown impressive zero-shoot ability, but huge model size costs . semi-parametric language model can be used to augment a smaller language model with retrieved background knowledge . |
| Approach: | They propose a semi-parametric language model for zero-shot task generalization that augments a smaller language model with retrieved related background knowledge. |
| Outcome: | The proposed model outperforms T0-3B by 16% across seven diverse evaluation tasks while being 3.8x smaller in scale. |
Diversified Multiple Instance Learning for Document-Level Multi-Aspect Sentiment Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that D-MILN outperforms recent weakly-supervised baselines . document-level multi-aspect sentiment classification requires a lot of manual aspect-level annotations - which is time-consuming and laborious . |
| Approach: | They propose a novel Diversified Multiple Instance Learning Network to achieve DMSC with only document-level weak supervision. |
| Outcome: | The proposed method outperforms weakly-supervised baselines on TripAdvisor and BeerAdvocate datasets. |
COVID-19 Claim Radar: A Structured Claim Extraction and Tracking System (2022.acl-demo)
Copied to clipboard
Manling Li, Revanth Gangi Reddy, Ziqi Wang, Yi-shyuan Chiang, Tuan Lai, Pengfei Yu, Zixuan Zhang, Heng Ji
| Challenge: | a new system extracts supporting and refuting claims from COVID-19 related news . the system is publicly available at GitHub and DockerHub, with complete documentation. |
| Approach: | They propose a COVID-19 Claim Radar system that extracts supporting and refuting claims . the system leverages Wikidata as the hub to consolidate coreferential knowledge elements . |
| Outcome: | The system extracts supporting and refuting claims from COVID-19 pandemic information . it leverages Wikidata as the hub to merge coreferential knowledge elements . |
Unlearning Bias in Language Models by Partitioning Gradients (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent research has shown that large-scale pretrained language models exhibit issues relating to racism, sexism, religion bias, and toxicity in general. |
| Approach: | They propose a gray-box method for debiasing pretrained masked language models using partitioned contrastive gradient unlearning (PCGU) aims to optimize only the weights that contribute most to a specific domain of bias by computing a first-order approximation based on the gradients of contrastive sentence pairs. |
| Outcome: | The proposed method is low-cost and can pinpoint the sources of social bias in large pretrained language models. |
TUNA: Comprehensive Fine-grained Temporal Understanding Evaluation on Dense Dynamic Videos (2025.acl-long)
Copied to clipboard
Fanheng Kong, Jingyuan Zhang, Hongzhi Zhang, Shi Feng, Daling Wang, Linhao Yu, Xingguang Ji, Yu Tian, V. W., Fuzheng Zhang
| Challenge: | Existing benchmarks for video understanding often focus on specific aspects, overlooking the holistic nature of video content. |
| Approach: | They propose a temporal-oriented benchmark for fine-grained understanding on dense dynamic videos with two complementary tasks: captioning and QA. |
| Outcome: | The proposed model performs well on diverse video scenarios and dynamic videos, with interpretable and robust evaluation criteria. |
PUPPET: Neural-Symbolic Standardized Patients for Mental Health (2026.acl-long)
Copied to clipboard
Chen Xu, Yu ji, Zhenyu Lv, Yang Yi, Yizhe Yang, Luyao Ji, Chaoyi Chen, Xianyang Wang, Tian Lan, Zhihua Wang, Juan Wang, Xunde Dong, Fuze Tian, Qunxi Dong, Bin Hu
| Challenge: | Existing LLM-based training approaches lack faithful responses to clinical errors and explainable feedback. |
| Approach: | They propose a neural-symbolic virtual standardized patient governed by an OBSERVE-THINK-BEHAVE architecture that embeds LLM reasoning into a symbolic system where experts implant causal associations between intervention logic and patient mental states. |
| Outcome: | The proposed model outperforms baselines in faithfulness and pedagogical value. |
Improving Question Answering with External Knowledge (D19-58)
Copied to clipboard
| Challenge: | ARC-Easy, ARC Challenge, and OpenBookQA use Wikipedia to augment training data . performance degrades when additional instances exhibit higher difficulty than original training data. |
| Approach: | They propose two methods for exploiting external knowledge for QA in science . they enrich the original corpus with relevant text snippets from an open-domain resource . the second method simply increases the amount of training data by appending additional in-domain instances. |
| Outcome: | The proposed methods achieve gains in accuracy of 8.1%, 13.0%, and 12.8% on science QA tasks. |
M3AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic Lecture Dataset (2024.acl-long)
Copied to clipboard
Zhe Chen, Heyang Liu, Wenyi Yu, Guangzhi Sun, Hongcheng Liu, Ji Wu, Chao Zhang, Yu Wang, Yanfeng Wang
| Challenge: | Publishing open-source academic video recordings is an emerging approach to sharing knowledge online. |
| Approach: | They propose a multimodal, multigenre, and multipurpose audio-visual academic lecture dataset with human annotations for multimodal content recognition and understanding tasks. |
| Outcome: | The proposed dataset can be used for multiple audio-visual recognition and understanding tasks. |
BERxiT: Early Exiting for BERT with Better Fine-Tuning and Extension to Regression (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to make exiting decisions are limited to classification tasks . large-scale pre-trained language models such as BERT have brought performance gain but at the cost of heavy computational burden. |
| Approach: | They propose a fine-tuning strategy and a learning-to-exit module to accelerate BERT inference . they propose to make trade-offs between model quality and efficiency by early exiting . |
| Outcome: | The proposed approach improves early exiting for BERT, with better trade-offs . it can be combined with other acceleration methods, and the proposed strategy can be applied to regression tasks. |
UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents (2026.acl-long)
Copied to clipboard
Yifan Ji, Zhipeng Xu, Zhenghao Liu, Zulong Chen, Qian Zhang, Zhibo Yang, Junyang Lin, Yu Gu, Ge Yu, Maosong Sun
| Challenge: | Recent Large Multimodal Models (LMMs) have shown promising potential for performing end-to-end KIE directly from document images. |
| Approach: | They propose a benchmark to evaluate the performance of Large Multimodal Models (LMMs) using a constrained-category KIE track and an open-categorical KIE Track. |
| Outcome: | Experiments on 15 state-of-the-art LMMs show performance degradation under diverse schema definitions, long-tail key fields, and complex layouts, along with pronounced performance disparities across different document types and scenarios. |
Timeline Summarization based on Event Graph Compression via Time-Aware Optimal Transport (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for timeline summarization ignore the events’ intra-structures and inter-structure connections. |
| Approach: | They propose to represent news articles as an event-graph, thus compressing the whole graph to its salient sub-graph. |
| Outcome: | The proposed method significantly improves on the state-of-the-art on three real-world datasets, including two public benchmarks and a Timeline100 dataset. |
Training Long-Context LLMs Efficiently via Chunk-wise Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in long-context large language models have demonstrated superior retrieval quality compared to retrievalaugmented generation (RAG) approaches. |
| Approach: | They propose a memory-efficient training paradigm that partitions lengthy inputs into manageable chunks. |
| Outcome: | The proposed model expands maximum sequence length from 1K to 16K tokens on a single RTX 3090 GPU, while SpaCO achieves accelerated training speed. |
Defining a New NLP Playground (2023.findings-emnlp)
Copied to clipboard
Sha Li, Chi Han, Pengfei Yu, Carl Edwards, Manling Li, Xingyao Wang, Yi Fung, Charles Yu, Joel Tetreault, Eduard Hovy, Heng Ji
| Challenge: | Recent explosion of performance of large language models (LLMs) has changed the field more abruptly and seismically than any other shift in the field’s 80 year history. |
| Approach: | They propose 20+ PhD-dissertation-worthy research directions to define a new NLP playground by combining theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications. |
| Outcome: | The proposed research will cover theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications. |
Showing Your Work Doesn’t Always Work (2020.acl-main)
Copied to clipboard
| Challenge: | countless experimental papers lack empirical rigor, disregarding necessities such as statistical significance tests and computational environments. |
| Approach: | They propose to report the expected validation effectiveness of the best-tuned model with respect to the computational budget. |
| Outcome: | The proposed model favors negative errors and yields poor bootstrapped confidence intervals, the authors argue . they find that the proposed model is biased and uses error-prone assumptions . |
ReCUT: Balancing Reasoning Length and Accuracy in LLMs via Stepwise Trails and Preference Optimization (2025.findings-emnlp)
Copied to clipboard
Zhensheng Jin, Xinze Li, Yifan Ji, Chunyi Peng, Zhenghao Liu, Qi Shi, Yukun Yan, Shuo Wang, Furong Peng, Ge Yu
| Challenge: | Existing methods to train LLMs suffer from overthinking, leading to lengthy reasoning traces . Existing approaches to train large language models suffer from this problem . |
| Approach: | They propose a method to combine multiple reasoning chains for training LLMs . they use stepwise exploration and long-short switched sampling to evaluate reasoning paths . |
| Outcome: | The proposed method reduces reasoning lengths by approximately 30-50% . it also maintains or improves reasoning accuracy compared to baselines . |
Long-context Language Models Fail in Basic Retrieval Tasks Without Sufficient Reasoning Steps (2025.findings-emnlp)
Copied to clipboard
| Challenge: | despite their extensive context window, long-context language models fail in some basic cases . a recent study shows that long-cot methods are not necessary for long-constituency tasks . |
| Approach: | a new study evaluates long-context language models with a large context window . the authors propose a method that can be well addressed with arbitrary reasoning steps . |
| Outcome: | The proposed methods are well addressed with a sufficient number of reasoning steps, guided by specific CoT prompts. |
GCRC: A New Challenging MRC Dataset from Gaokao Chinese for Explainable Evaluation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing machine reading comprehension datasets lack an explainable evaluation of systems' reasoning capabilities. |
| Approach: | They propose a dataset with multi-choice questions that evaluates MRC systems' reasoning process . they use sentence-level relevant supporting facts, error reason of distractors to evaluate MRC . |
| Outcome: | The proposed dataset is more challenging and useful for identifying limitations of existing MRC systems in an explainable way. |
JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained models for knowledgegraph-to-text generation ignore graph structure during encoding and lack elaborate pre-training tasks to explicitly model graph-text alignments. |
| Approach: | They propose a graph-text joint representation learning model called JointGT which incorporates a structure-aware semantic aggregation module into each Transformer layer to preserve the graph structure. |
| Outcome: | The proposed model achieves state-of-the-art performance on various KG-to-text datasets. |
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference (2020.acl-main)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models such as BERT are notorious for being slow in both training and inference. |
| Approach: | They propose a method to accelerate BERT inference by inserting extra classification layers between each transformer layer of BERT. |
| Outcome: | The proposed method saves up to 40% inference time with minimal degradation in model quality. |
EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association (2025.acl-long)
Copied to clipboard
Weiqi Wang, Limeng Cui, Xin Liu, Sreyashi Nag, Wenju Xu, Chen Luo, Sheikh Muhammad Sarwar, Yang Li, Hansu Gu, Hui Liu, Changlong Yu, Jiaxin Bai, Yifan Gao, Haiyang Zhang, Qi He, Shuiwang Ji, Yangqiu Song
| Challenge: | Goal-oriented script planning is used by humans to plan for typical activities . however, this capability remains underexplored due to several challenges . |
| Approach: | They propose a framework that enables product-enriched scripts by associating products with each step based on the semantic similarity between the actions and their purchase intentions. |
| Outcome: | The proposed framework can generate product-enriched scripts from 2.4 million scripts . human annotations are conducted to provide gold labels for a sampled subset . |
Bacteria Biotope Relation Extraction via Lexical Chains and Dependency Graphs (D19-57)
Copied to clipboard
| Challenge: | In this paper, we describe our approach for the Bacteria Biotopes relation extraction subtask in the BioNLP Shared Task 2019 . |
| Approach: | They propose a novel approach for dependency graph construction based on lexical chains . they then propose 'neuro network' model which uses short-term memories and syntax information . |
| Outcome: | The proposed approach achieves the best F1 (66.3%) in the official evaluation participated by 7 teams. |
Reliability-aware Dynamic Feature Composition for Name Tagging (P19-1)
Copied to clipboard
| Challenge: | Word embeddings are used to encode semantic information, but their quality is not consistent across the vocabulary due to the long-tail distribution of word frequency. |
| Approach: | They propose a reliability-aware name tagging model that uses word frequency to indicate word quality . they propose to use word frequency-based reliability signals to dynamically select and compose features . |
| Outcome: | The proposed model outperforms the baseline model on OntoNotes 5.0 and up to 5% gain on cross-genre data sets. |
FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows (2022.emnlp-main)
Copied to clipboard
| Challenge: | Despite recent progress in dialogue evaluation, how to develop automatic metrics remains an open problem. |
| Approach: | They propose a consensus-based framework for dialog evaluation using segment act flows . they propose to crowdsource a large-scale dataset for it to be evaluated . |
| Outcome: | The proposed framework can reach the best or comparable correlation with human evaluation. |
WebSRC: A Dataset for Web-Based Structural Reading Comprehension (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using a web page and a question, a machine can't understand the contents of web pages. |
| Approach: | They propose a novel dataset for web-based structural reading comprehension that consists of 400K question-answer pairs and a dataset of 6.4K web pages. |
| Outcome: | The proposed dataset consists of 400K question-answer pairs, collected from 6.4K web pages with corresponding HTML source code, screenshots, and metadata. |
Shorten the Long Tail for Rare Entity and Event Extraction (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches for the long-tailed learning problem seek to manipulate the training data by re-balancing, augmentation or introducing extra prior knowledge. |
| Approach: | They propose to transform infrequent candidate mention representation with the average mention representation in the training dataset to handle the generalization challenge. |
| Outcome: | The proposed framework can generalize to rare or unseen expressions of entities or events, especially for rare types without sufficient training examples. |
RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogue systems generate accurate and informative responses, but they are prone to hallucination problems. |
| Approach: | They propose a method to generate hallucinated responses using knowledge graphs . they propose local knowledge grounding to combine textual embeddings with corresponding KG embeddments . a global knowledge ground technique is also proposed to equip RHO with multi-hop reasoning abilities . |
| Outcome: | The proposed approach outperforms state-of-the-art methods on automatic and human evaluation by a large margin. |
Enhancing Multimodal Continual Instruction Tuning with BranchLoRA (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to fine tune Multimodal Large Language Models (MLLMs) are prone to Catastrophic Forgetting (CF) existing approaches rely on the Mixture-of-Experts (MoE) LoRA framework to preserve previous instruction alignments. |
| Approach: | They propose an asymmetric tuning-freezing mechanism to mitigate parameter inefficiency . branch-specific routers are introduced to ensure optimal branch distribution over time . |
| Outcome: | The proposed framework outperforms existing frameworks on the latest MCIT benchmarks. |
Stand on The Shoulders of Giants: Building JailExpert from Previous Attack Experience (2025.emnlp-main)
Copied to clipboard
Xi Wang, Songlei Jian, Shasha Li, Xiaopeng Li, Bin Ji, Ma Jun, Xiaodong Liu, Jing Wang, Jianfeng Zhang, Jie Yu, Feilong Bao, null Wangbaosheng
| Challenge: | Existing methods to generate human-aligned content with a “jailbreak prompt” are inefficient and repetitive, causing inefficiency and a lack of experience. |
| Approach: | They propose a framework that integrates past attack experiences to aid current jailbreak attempts. |
| Outcome: | The proposed framework improves both attack effectiveness and efficiency compared to the current black-box jailbreak method. |
The Law of Knowledge Overshadowing: Towards Understanding, Predicting and Preventing LLM Hallucination (2025.findings-acl)
Copied to clipboard
Yuji Zhang, Sha Li, Cheng Qian, Jiateng Liu, Pengfei Yu, Chi Han, Yi R. Fung, Kathleen McKeown, ChengXiang Zhai, Manling Li, Heng Ji
| Challenge: | Hallucination is a persistent challenge in large language models where even with rigorous quality control, models often generate distorted facts. |
| Approach: | They propose a new framework to quantify factual hallucinations by modeling knowledge overshadowing. |
| Outcome: | The proposed framework improves model factuality on Overshadow (27.9%), MemoTrap (13.1%) and NQ-Swap (18.3%). |
Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations (2025.emnlp-main)
Copied to clipboard
Ziwei Ji, Lei Yu, Yeskendir Koishekenov, Yejin Bang, Anthony Hartshorn, Alan Schelten, Cheng Zhang, Pascale Fung, Nicola Cancedda
| Challenge: | LLMs often use assertive language when making false claims, resulting in harm and loss of trust. |
| Approach: | They find that a mismatch between semantic and verbal uncertainty is a better predictor of hallucinations than semantic uncertainty alone. |
| Outcome: | a new study shows that mismatch between semantic and verbal uncertainty is better predictor of hallucinations than semantic uncertainty alone. |
Accelerate Parallelizable Reasoning via Parallel Decoding within One Sequence (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for parallelizable reasoning tasks are inefficient, says a new study . generating lengthy reasoning sequences is computationally expensive and time-consuming, says the study authors . |
| Approach: | They propose a method that decodes multiple tokens per forward pass using a tree-like attention mask . their method achieves nearly 100% speedup in decoding while basically maintaining the answer quality . |
| Outcome: | Experimental results show that the method achieves nearly 100% speedup in decoding while maintaining the answer quality. |
Non-Sequential Graph Script Induction via Multimedia Grounding (2023.acl-long)
Copied to clipboard
| Challenge: | Existing scripts for everyday tasks are presented in a linear manner, which does not reflect the flexibility displayed by people executing tasks in real life. |
| Approach: | They propose to use loosely aligned videos to train a non-sequential graph script induction task by using a multimodal framework to ground procedural videos to WikiHow textual steps. |
| Outcome: | The proposed model outperforms the WikiHow linear baseline by 48.76% . it can predict future steps given a partial step sequence and generate explicit graph scripts . |
R3-NL2GQL: A Model Coordination and Knowledge Graph Alignment Approach for NL2GQL (2024.findings-emnlp)
Copied to clipboard
Yuhang Zhou, Yu He, Siyu Tian, Yuchen Ni, Zhangyue Yin, Xiang Liu, Chuanjun Ji, Sen Liu, Xipeng Qiu, Guangnan Ye, Hongfeng Chai
| Challenge: | Adapting existing approaches for converting natural language to SQL encounters hurdles due to distinct nature of GQL compared to SQL. |
| Approach: | They propose a method that integrates both small and large Foundation Models for ranking, rewriting, and refining tasks. |
| Outcome: | The proposed approach integrates both small and large Foundation Models for ranking, rewriting, and refining tasks while capitalizing on the superior generalization and query generation prowess of larger models for the final transformation of natural language queries into GQL formats. |
ARGUS: Policy-Adaptive Ad Governance via Evolving Reinforcement with Adversarial Umpiring (2026.acl-industry)
Copied to clipboard
Deyi Ji, Junyu Lu, Xuanyi Liu, Liqun Liu, Hailong Zhang, Peng Shu, Huan Yu, Jie Jiang, Tianrun Chen, Lanyun Zhu
| Challenge: | Existing regulatory policies create label inconsistencies and reasoning ambiguities in historical datasets. |
| Approach: | They propose a policy-adaptive governance system that enables evolving reinforcement through multi-agent adversarial umpiring. |
| Outcome: | The proposed system outperforms fine-tuning baselines on industrial and public datasets . it enables evolving reinforcement through multi-agent adversarial umpiring . |
JPU: Bridging Jailbreak Defense and Unlearning via On-Policy Path Rectification (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are vulnerable to diverse jailbreak attacks despite extensive safety alignment . |
| Approach: | They propose a method to rectify dynamic jailbreak paths towards safety anchors by dynamically mining on-policy adversarial samples to expose vulnerabilities and identify jailbreak path. |
| Outcome: | The proposed model significantly improves jailbreak resistance against dynamic attacks while maintaining its utility. |
Generating Visual Spatial Description via Holistic 3D Scene Understanding (2023.acl-long)
Copied to clipboard
| Challenge: | Existing VSD work focuses on skewed spatial understanding of target objects . Existing work merely models the 2D geometrical vision features . |
| Approach: | They propose to incorporate 3D scene features into visual spatial description tasks by sampling topologically-diverse subgraphs from Go3D-S2G. |
| Outcome: | The proposed framework outperforms baselines on two VSD datasets and produces more spatially-diversified generation. |
ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents (2026.acl-long)
Copied to clipboard
Zhuofeng Li, Yi Lu, Dongfu Jiang, Haoxiang Zhang, Yuyang Bai, Chuan Li, Yu Wang, Shuiwang Ji, Jianwen Xie, Yu Zhang
| Challenge: | Rapid rise in AI conference submissions has driven increasing exploration of large language models (LLMs) for peer review support. |
| Approach: | They propose a peer review benchmarking tool based on paper-specific rubrics and a rubric-guided framework that decomposes reviewing into drafting and grounding stages. |
| Outcome: | The proposed framework outperforms baselines with stronger/larger backbones in both alignment with human judgments and rubric-based review quality across 8 dimensions. |
Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Named entity recognition models require abundant high-quality annotations to train . distant supervision may induce incomplete and noisy labels, making supervised learning ineffective. |
| Approach: | They propose a noise-robust learning scheme for training named entity recognition models using only distantly-labeled data and a self-training method that uses contextualized augmentations created by pre-trained language models. |
| Outcome: | The proposed method outperforms existing supervised NER models on three datasets by significant margins. |
Span-based Joint Entity and Relation Extraction with Attention-based Span-specific and Contextual Semantic Representations (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods treat each span token equally important, ignoring significant features. |
| Approach: | They propose a span-based joint extraction framework with attention-based semantic representations that utilizes span-specific and contextual representations. |
| Outcome: | The proposed model outperforms existing models on ACE2005, CoNLL2004 and ADE. |