Papers by Daxin Jiang

Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder (2020.acl-main)

Challenge:	Recent large-scale vision-language pre-training relies on image-text global alignment by contrastive learning and is further boosted by fine-grained alignment in a weakly contrastive manner for cross-modal retrieval.
Approach:	They propose expansive lexicon-patch alignment (ELA) to align image patches with a vocabulary rather than only the words explicitly in the text for annotation-free alignment and information augmentation.
Outcome:	The proposed method outperforms state-of-the-art methods on cross-modal retrieval and can learn representative fine-grained information.

Copied to clipboard

Daya Guo, Duyu Tang, Nan Duan, Jian Yin, Daxin Jiang, Ming Zhou

Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks (D19-1)

Challenge:	Existing approaches for inferential text generation ignore context that is not explicitly provided . Existing models ignore background knowledge that provides crucial evidence to generate inferences .
Approach:	They propose an approach that automatically finds evidence for an event from a large text corpus and leverages it to guide the generation of inferential texts.
Outcome:	The proposed model generates inferential texts from a large text corpus and uses evidence to guide it.

Copied to clipboard

Haoyang Huang, Yaobo Liang, Nan Duan, Ming Gong, Linjun Shou, Daxin Jiang, Ming Zhou

CoSQA: 20,000+ Web Queries for Code Search and Question Answering (2021.acl-long)

Challenge:	Existing models that can handle cross-lingual tasks with limited or no training data are insensitive to different languages.
Approach:	They propose to use Unicoder to train models in one language and apply it to other languages.
Outcome:	Experiments show that Unicoder learns the mappings among different languages from more perspectives.

Copied to clipboard

Junjie Huang, Duyu Tang, Linjun Shou, Ming Gong, Ke Xu, Daxin Jiang, Ming Zhou, Nan Duan

Information Retrieval

Challenge:	Using deep neural networks to find codes is difficult . we present a dataset that includes 20,604 labels for natural language queries and codes .
Approach:	They introduce a contrastive learning method to enhance text-code matching . they find that CoSQA improves the accuracy of code question answering by 5.1% .
Outcome:	The proposed method improves the accuracy of code question answering by 5.1% and improves by 10.5% on a CodeBERT model.

Unearthing Gems from Stones: Policy Optimization with Negative Sample Augmentation for LLM Reasoning (2025.findings-emnlp)

Copied to clipboard

Zhaohui Yang, Yuxiao Ye, Shilei Jiang, Shihong Deng, Chen Hu, Linjing Li, Daxin Jiang

Empowering Dual-Encoder with Query Generator for Cross-Lingual Dense Retrieval (2022.emnlp-main)

Challenge:	Recent advances in reasoning language models have witnessed a paradigm shift from short to long CoT pattern.
Approach:	They propose a behavior-constrained policy gradient with negative sample augmented (BCPG-NSA) negative steps are valuable components in long CoT models, authors argue .
Outcome:	The proposed framework outperforms baselines on math/coding reasoning benchmarks using the same training dataset.

Copied to clipboard

Houxing Ren, Linjun Shou, Ning Wu, Ming Gong, Daxin Jiang

Bridging the Gap between Language Models and Cross-Lingual Sequence Labeling (2022.naacl-main)

Challenge:	Existing methods to distill knowledge from cross-encoder re-ranker to dual-encoding retriever are lacking in the cross-lingual setting.
Approach:	They propose to use a query generator as the teacher in the cross-lingual setting to distill knowledge to a dual-encoder retrieval model.
Outcome:	The proposed method outperforms state-of-the-art methods on two benchmark datasets.

Copied to clipboard

Nuo Chen, Linjun Shou, Ming Gong, Jian Pei, Daxin Jiang

ADAM: Dense Retrieval Distillation with Adaptive Dark Examples (2024.findings-acl)

Challenge:	Existing methods to train cross-lingual pre-trained language models have shown great success in cross-linguistic sequence labeling tasks.
Approach:	They propose a cross-lingual language informative span masking task to eliminate the objective gap between pre-training and fine-tuning stages.
Outcome:	The proposed method surpasses the state-of-the-art methods on multiple benchmarks even with limited pre-training data.

Copied to clipboard

Chongyang Tao, Chang Liu, Tao Shen, Can Xu, Xiubo Geng, Binxing Jiao, Daxin Jiang

Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension (2020.acl-main)

Challenge:	Existing methods to retrieve data from multiple encoders are too trivial for the teacher to distinguish, preventing the teacher from transferring abundant dark knowledge to the student.
Approach:	They propose a knowledge distillation framework that can better transfer the dark knowledge held in the teacher with adaptive dark examples.
Outcome:	The proposed framework can better transfer the dark knowledge held in the teacher with adaptive dark examples.

Copied to clipboard

Fei Yuan, Linjun Shou, Xuanyu Bai, Ming Gong, Yaobo Liang, Nan Duan, Yan Fu, Daxin Jiang

Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text (2022.findings-acl)

Challenge:	Existing approaches to improve machine reading comprehension performance on low resource languages are limited due to the lack of sufficient training data.
Approach:	They propose to use a mixed MRC task to translate the question to other languages and build cross-lingual question-passage pairs.
Outcome:	The proposed task improves on two cross-lingual MRC datasets.

Copied to clipboard

Siyuan Wang, Wanjun Zhong, Duyu Tang, Zhongyu Wei, Zhihao Fan, Daxin Jiang, Ming Zhou, Nan Duan

RecInDial: A Unified Framework for Conversational Recommendation with Pretrained Language Models (2022.aacl-main)

Challenge:	Existing methods for logical reasoning of text focus on contextual semantics while struggling to explicitly model the logical inference process.
Approach:	They propose a logic-driven context extension framework and a data-driven augmentation algorithm that uses contrastive learning to better capture logical information.
Outcome:	The proposed framework outperforms existing methods on two benchmark datasets, ReClor and LogiQA.

Copied to clipboard

Lingzhi Wang, Huang Hu, Lei Sha, Can Xu, Daxin Jiang, Kam-Fai Wong

Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations (2022.acl-long)

Challenge:	Existing generative methods to recommend items are shallowly integrated into the model training and have poor chit-chat ability.
Approach:	They propose a framework that integrates recommendation into the dialog generation by introducing a vocabulary pointer.
Outcome:	The proposed framework outperforms the state-of-the-art models on a benchmark dataset.

Copied to clipboard

Wei Chen, Yeyun Gong, Can Xu, Huang Hu, Bolun Yao, Zhongyu Wei, Zhihao Fan, Xiaowu Hu, Bartuer Zhou, Biao Cheng, Daxin Jiang, Nan Duan

Natural Language Interfaces

Challenge:	Existing studies focus on coarse-grained response selection in retrieval-based dialogue systems.
Approach:	They propose a Contextual Fine-to-Coarse (CFC) distilled model for coarse-grained response selection in open-domain conversations.
Outcome:	The proposed model improves over baseline methods on two datasets based on the Reddit comments dump and Twitter corpus compared with baseline methods.

Language Scaling for Universal Suggested Replies Model (2021.naacl-industry)

Copied to clipboard

Qianlan Ying, Payal Bajaj, Budhaditya Deb, Yu Yang, Wei Wang, Bojia Lin, Milad Shokouhi, Xia Song, Yang Yang, Daxin Jiang

No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension (2020.findings-emnlp)

Challenge:	We consider scaling automated suggested replies (SR) to multiple languages for a commercial email application.
Approach:	They propose a multi-lingual multi-task continual learning framework with auxiliary tasks and language adapters to train universal language representation across regions.
Outcome:	The proposed model reduces catastrophic forgetting and improves cross-lingual transfer across languages while reducing training costs.

Copied to clipboard

Xuguang Wang, Linjun Shou, Ming Gong, Nan Duan, Daxin Jiang

PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddings (2022.emnlp-main)

Challenge:	Natural Questions (NQ) benchmark sets new challenges for machine reading comprehension.
Approach:	They propose a novel approach to handle all answer types systematically using a two-step training procedure.
Outcome:	The proposed approach achieved the top 1 on both long and short answer leaderboards with F1 scores of 77.2 and 64.1.

Copied to clipboard

Qiyu Wu, Chongyang Tao, Tao Shen, Can Xu, Xiubo Geng, Daxin Jiang

Synergistic Interplay between Search and Large Language Models for Information Retrieval (2024.acl-long)

Challenge:	Existing approaches to learning sentence embeddings in unsupervised manner depend on mono-augmenting . existing approaches depend on augmenting biases and thus corrupt the quality of sentence embeds.
Approach:	They propose a method to augment a sentence with a semantically-close positive instance to construct contrastive pairs in unsupervised manner.
Outcome:	The proposed method improves performance on STS benchmarks and compares with existing methods.

Copied to clipboard

Jiazhan Feng, Chongyang Tao, Xiubo Geng, Tao Shen, Can Xu, Guodong Long, Dongyan Zhao, Daxin Jiang

Code Execution with Pre-trained Language Models (2023.findings-acl)

Challenge:	Information retrieval (IR) is an indispensable technique for locating relevant resources from vast amounts of data.
Approach:	They propose a framework that facilitates information refinement through synergy between RMs and LLMs.
Outcome:	The proposed framework improves the performance of large-scale retrieval benchmarks on web searches and low-resource retrieval tasks.

Copied to clipboard

Chenxiao Liu, Shuai Lu, Weizhu Chen, Daxin Jiang, Alexey Svyatkovskiy, Shengyu Fu, Neel Sundaresan, Nan Duan

ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification (2022.acl-long)

Challenge:	Pre-trained code intelligence models ignore the execution trace and only rely on source code and syntactic structures to understand code execution.
Approach:	They develop a mutation-based data augmentation technique to create a Python dataset and task for code execution that challenges existing models.
Outcome:	The proposed model outperforms existing models on code execution and shows its potential for zero-shot code-to-code search and text-to code generation.

Copied to clipboard

Yucheng Zhou, Tao Shen, Xiubo Geng, Guodong Long, Daxin Jiang

Information Extraction & Text Mining

Challenge:	Existing work on event-centric reasoning fails to model event-level correlations . Existing studies limit their scope to specific scenarios or overlook event- level correlations.
Approach:	They propose to pre-train a general Correlation-aware context-to-Event Transformer for event-centric reasoning by highlighting event-level correlations with effective training.
Outcome:	The proposed model is applicable to a wide range of event-centric reasoning scenarios, considering its versatility of event correlation types, application formulations, and reasoning types.

Lexicon-Enhanced Self-Supervised Training for Multilingual Dense Retrieval (2022.findings-emnlp)

Copied to clipboard

Houxing Ren, Linjun Shou, Jian Pei, Ning Wu, Ming Gong, Daxin Jiang

Beyond the First Error: Process Reward Models for Reflective Mathematical Reasoning (2025.findings-emnlp)

Challenge:	Recent multilingual pre-trained models perform poorly on multilingual retrieval tasks due to lack of multilingual training data.
Approach:	They propose to mine and generate self-supervised training data based on large-scale unlabeled corpus and introduce query generator to generate more queries in target languages for unlabed passages.
Outcome:	The proposed method performs better than baselines on a Mr. TYDI dataset and an industrial dataset from a commercial search engine.

Copied to clipboard

Zhaohui Yang, Chenghua He, Xiaowen Shi, Shihong Deng, Linjing Li, Qiyue Yin, Daxin Jiang

Reasoning over Entity-Action-Location Graph for Procedural Text Understanding (2021.acl-long)

Challenge:	Existing methods for training effective PRMs focus on the first incorrect step and all preceding steps, assuming that all subsequent steps are incorrect.
Approach:	They propose a data annotation method specifically designed to score the long CoT reasoning process by using an LLM-based judger for annotation.
Outcome:	The proposed method improves PRMs' ability to identify effective self-correction behaviors and reasoning based on erroneous steps.

Copied to clipboard

Hao Huang, Xiubo Geng, Jian Pei, Guodong Long, Daxin Jiang

Structured Data in NLP

Challenge:	Procedural text understanding aims at tracking the states and locations of entities mentioned in a paragraph.
Approach:	They propose a framework to model entities-entity, action, and location relations using a graph neural network.
Outcome:	The proposed approach outperforms strong baselines on two datasets, ProPara and Recipes.

Multi-Task Learning for Conversational Question Answering over a Large-Scale Knowledge Base (D19-1)

Copied to clipboard

Tao Shen, Xiubo Geng, Tao Qin, Daya Guo, Duyu Tang, Nan Duan, Guodong Long, Daxin Jiang

CORE: Cooperative Training of Retriever-Reranker for Effective Dialogue Response Selection (2023.acl-long)

Challenge:	Recent approaches to handle large knowledge base decompose tasks into subtasks and solve them sequentially.
Approach:	They propose a multi-task learning framework that resolves coreference in conversations . they propose enabling shared supervisions and type-aware entity detection model .
Outcome:	The proposed framework improves overall F1 score from 67% to 79% on a large-scale conversational question answering dataset.

Copied to clipboard

Chongyang Tao, Jiazhan Feng, Tao Shen, Chang Liu, Juntao Li, Xiubo Geng, Daxin Jiang

PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering (2026.acl-long)

Challenge:	Existing methods to train retrieval-based dialogue systems are suboptimal . existing methods to optimize retrieval and rerank modules are sub-optimal, causing sub-optimum performance.
Approach:	They propose a retrieval-based dialogue system with a fast retriever and a smart response reranker that combine the best of both worlds.
Outcome:	The proposed method can learn from each other and evolve together . it can be used in industrial applications and has powered industrial applications.

Copied to clipboard

Xiangfeng Wang, Hangyu Guo, Yanlin Lai, Mitt Huang, Liang Zhao, Chengyuan Yao, Yinmin Zhang, Qi Han, null Xiaoxiaoren, Chun Yuan, Tong Xu, Zheng Ge, Xiangyu Zhang, Daxin Jiang

A Graph Representation of Semi-structured Data for Web Question Answering (2020.coling-main)

Challenge:	Current outcome-centric verification paradigms neglect potential errors in the derivation process.
Approach:	They propose a process-aware RLVR training paradigm utilizing verifiers selected via PRIME.
Outcome:	The proposed approach outperforms the baseline verification paradigm on AIME24, AIME25, and Beyond-AIME models.

Copied to clipboard

Xingyao Zhang, Linjun Shou, Jian Pei, Ming Gong, Lijie Wen, Daxin Jiang

Structured Data in NLP

Challenge:	Existing studies treat semi-structured data as flat documents with pieces of text . semi-structural data is more effective to represent rich relational information . question answering is an important feature in most search engines .
Approach:	They propose a graph representation of Web tables and lists based on categorization of components and their relations . they also develop reasoning techniques on the graph model for the question answering task .
Outcome:	The proposed graph improves F1 score by 3.90 points over the state-of-the-art baselines on real datasets.

Multimodal Dialogue Response Generation (2022.acl-long)

Copied to clipboard

Qingfeng Sun, Yujing Wang, Can Xu, Kai Zheng, Yaming Yang, Huang Hu, Fei Xu, Jessica Zhang, Xiubo Geng, Daxin Jiang

Difference-aware Knowledge Selection for Knowledge-grounded Conversation Generation (2020.findings-emnlp)

Challenge:	Existing studies focus on multimodal dialogue models but neglect generation methods.
Approach:	They propose a multimodal dialogue response generation task which requires multimodal dialogs containing both texts and images which are difficult to obtain.
Outcome:	Experiments show that the proposed model can generate informative text and high-resolution image responses.

Copied to clipboard

Chujie Zheng, Yunbo Cao, Daxin Jiang, Minlie Huang

Social Norms-Grounded Machine Ethics in Complex Narrative Situation (2022.coling-1)

Challenge:	Existing knowledge selection models are limited by the context, but the difference between selected knowledge at different turns is often overlooked.
Approach:	They propose a difference-aware knowledge selection method that computes the difference between the candidate knowledge sentences provided at the current turn and those chosen in the previous turns.
Outcome:	The proposed method outperforms the state-of-the-art methods in a knowledge-grounded dialog.

Copied to clipboard

Tao Shen, Xiubo Geng, Daxin Jiang

Label-aware Multi-level Contrastive Learning for Cross-lingual Spoken Language Understanding (2022.emnlp-main)

Challenge:	Recent studies focus on data-driven methods to judge the ethics of complex real-world narratives but face two major challenges: they cannot handle dilemma situations due to a lack of basic knowledge about social norms; and they focus on sparse situation-level judgment regardless of the social norm.
Approach:	They propose to complement a complex situation with grounded social norms by a norm-supported ethical judgment model in line with neural module networks to alleviate dilemma situations and improve norm-level explainability.
Outcome:	The proposed model improves state-of-the-art performance on two narrative judgment benchmarks.

Copied to clipboard

Shining Liang, Linjun Shou, Jian Pei, Ming Gong, Wanli Zuo, Xianglin Zuo, Daxin Jiang

ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation (2021.acl-demo)

Challenge:	Existing approaches to translate spoken language understanding into low-resource languages are limited to implicit alignment and disregard the inherent semantic structure in SLU.
Approach:	They propose to model utterance-slot-word structure by a multi-level contrastive learning framework . they also propose a label-aware joint model leveraging label semantics to enhance alignment .
Outcome:	The proposed model improves performance on two zero-shot cross-lingual datasets.

Copied to clipboard

Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang, Jiusheng Chen, Ruofei Zhang, Houqiang Li, Nan Duan

Multi-View Document Representation Learning for Open-Domain Dense Retrieval (2022.acl-long)

Challenge:	Existing models for pre-training are not convenient for users to find and set them up.
Approach:	They propose to extend ProphetNet into other domains and languages by pre-training models . they pre-train a cross-lingual generation model ProphetNet-Multi and a Chinese generation model .
Outcome:	The proposed models achieve new state-of-the-art on 10 benchmarks.

Copied to clipboard

Shunyu Zhang, Yaobo Liang, Ming Gong, Daxin Jiang, Nan Duan

Representation Learning

Challenge:	Existing methods for dense retrieval are hard to match with multiple views.
Approach:	They propose a multi-view document representation learning framework to generate multiple embeddings through viewers to represent documents and enforce them to align with different queries.
Outcome:	The proposed method outperforms recent works and achieves state-of-the-art results.

K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters (2021.findings-acl)

Copied to clipboard

Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Daxin Jiang, Ming Zhou

PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks (2022.acl-long)

Challenge:	Existing methods for injecting knowledge into pre-trained models are inconsistent and can flush out knowledge when multiple kinds of knowledge are injected.
Approach:	They propose a framework that retains the original parameters of pre-trained models fixed and supports the development of versatile knowledge-infused models.
Outcome:	The proposed framework retains the original parameters of the pre-trained model fixed and supports the development of versatile knowledge-infused models.

Copied to clipboard

Yufei Wang, Can Xu, Qingfeng Sun, Huang Hu, Chongyang Tao, Xiubo Geng, Daxin Jiang

Low-Resource NLP

Challenge:	Existing approaches to build labeled training data from domain-specific data are expensive to obtain.
Approach:	They propose a Prompt-based Data Augmentation model which only trains small-scale Soft Promptes in frozen Pre-trained Language Models.
Outcome:	The proposed model outperforms several baseline models on four benchmarks and is complementary with unlabeled in-domain data.

THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption (2022.findings-acl)

Copied to clipboard

Tianyu Chen, Hangbo Bao, Shaohan Huang, Li Dong, Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, Furu Wei

FAA: Fine-grained Attention Alignment for Cascade Document Ranking (2023.acl-long)

Challenge:	enabling pre-trained models inference on ciphertext data is difficult due to the complex computations in transformer blocks.
Approach:	They propose an approximation approach for transformers which enables inference on ciphertext data.
Outcome:	The proposed approach can infer pre-trained models on encrypted data with negligible performance drop but enjoy theory-guaranteed privacy-preserving advantage.

Copied to clipboard

Zhen Li, Chongyang Tao, Jiazhan Feng, Tao Shen, Dongyan Zhao, Xiubo Geng, Daxin Jiang

Information Retrieval

Challenge:	Contemporary document ranking methods focus on transforming documents into passages to handle long inputs, but intensive query-irrelevant content may lead to harmful distraction and high query latency.
Approach:	They propose a fine-grained attention alignment approach to jointly optimize a cascade document ranking model.
Outcome:	Experiments on MS MARCO and TREC DL show that the proposed method is effective in document ranking tasks.

Towards Interpretable Reasoning over Paragraph Effects in Situation (2020.emnlp-main)

Copied to clipboard

Mucheng Ren, Xiubo Geng, Tao Qin, Heyan Huang, Daxin Jiang

Learning to Ground Visual Objects for Visual Dialog (2021.findings-emnlp)

Challenge:	Existing models ignore complex reasoning process and solve it with a one-step "black box" approach.
Approach:	They propose a sequential approach which explicitly models each step of the reasoning process with neural network modules.
Outcome:	The proposed model is more interpretable and more accurate than existing models.

Copied to clipboard

Feilong Chen, Xiuyi Chen, Can Xu, Daxin Jiang

Natural Language Interfaces

Challenge:	Existing methods to ground visual objects are inadequate for visual dialog . a posterior distribution is inferred from context and questions, while posterior distributions are used to facilitate visual objects grounding.
Approach:	They propose a method to learn to ground visual objects for visual dialog using prior and posterior distributions over visual objects to facilitate visual objects grounding.
Outcome:	The proposed approach improves the existing models in generative and discriminative settings by a significant margin.

GLGE: A New General Language Generation Evaluation Benchmark (2021.findings-acl)

Copied to clipboard

Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan

GRACE: Gradient Harmonized and Cascaded Labeling for Aspect-based Sentiment Analysis (2020.findings-emnlp)

Challenge:	Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models.
Approach:	They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks.
Outcome:	The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages.

Copied to clipboard

Huaishao Luo, Lei Ji, Tianrui Li, Daxin Jiang, Nan Duan

Sentiment Analysis

Challenge:	Existing studies ignore aspect terms interaction when labeling polarities . aspect terms extraction and aspect sentiment classification are two fundamental tasks .
Approach:	They propose a GRadient hArmonized and CascadEd labeling model to solve the imbalance issue . they extend the gradient harmonized mechanism used in object detection to aspect-based sentiment analysis .
Outcome:	The proposed model achieves consistency improvement on multiple benchmark datasets and generates state-of-the-art results.

RikiNet: Reading Wikipedia Pages for Natural Question Answering (2020.acl-main)

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Nan Duan

Natural Language Interfaces

Challenge:	Using Wikipedia pages to answer open-domain questions remains challenging in natural language understanding.
Approach:	They propose a model which reads Wikipedia pages for natural question answering . it uses a dynamic paragraph dual-attention reader and a cascaded answer predictor .
Outcome:	The proposed model outperforms the human model on the Natural Questions dataset . it achieves 74.3 F1 and 57.9 F1 on long-answer and short-answer tasks .

HeterMPC: A Heterogeneous Graph Neural Network for Response Generation in Multi-Party Conversations (2022.acl-long)

Copied to clipboard

Jia-Chen Gu, Chao-Hong Tan, Chongyang Tao, Zhen-Hua Ling, Huang Hu, Xiubo Geng, Daxin Jiang

CodeBERT: A Pre-Trained Model for Programming and Natural Languages (2020.findings-emnlp)

Challenge:	Experimental results show that HeterMPC outperforms various baseline models for response generation in multi-party conversations.
Approach:	They propose a heterogeneous graph-based neural network for response generation in multi-party conversations which models the semantics of utterances and interlocutors simultaneously with two types of nodes in a graph.
Outcome:	The proposed model outperforms baseline models on the Ubuntu Internet Relay Chat (IRC) channel.

Copied to clipboard

Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, Ming Zhou

Alleviating Over-smoothing for Unsupervised Sentence Representation (2023.acl-long)

Challenge:	Large pre-trained models have improved performance on a variety of natural language processing tasks.
Approach:	They develop a bimodal pre-trained model for programming language (PL) and natural language (NL) it incorporates a hybrid objective function that detects replaced tokens from generators.
Outcome:	The proposed model performs better on two NL-PL applications by fine-tuning model parameters.

Copied to clipboard

Nuo Chen, Linjun Shou, Jian Pei, Ming Gong, Bowen Cao, Jianhui Chang, Jia Li, Daxin Jiang

Modeling Event-Pair Relations in External Knowledge Graphs for Script Reasoning (2021.findings-acl)

Challenge:	Existing approaches to learn better unsupervised sentence representations have been successful . over-smoothing problem in unsupervised sentences reduces the capacity of powerful PLMs .
Approach:	They propose a method to solve the over-smoothing problem in unsupervised sentence representations by combining negatives from PLMs intermediate layers.
Outcome:	The proposed method improves on different strong baselines on Semantic Textual Similarity and Transfer datasets.

Copied to clipboard

Yucheng Zhou, Xiubo Geng, Tao Shen, Jian Pei, Wenqiang Zhang, Daxin Jiang

Knowledge Representation

Challenge:	Existing methods focus on graph triples with event overlap, but ignore more supportive triples . Script reasoning relies on understanding the relationship between two events .
Approach:	They propose a model to learn the inferential relations between events from the whole eventuality KG . they propose 'script adapter' to extend the model to infer the associated relations between an event chain and a subsequent event candidate.
Outcome:	The proposed model is compared with baselines using external KG or not on a script reasoning task.

TegTok: Augmenting Text Generation via Task-specific and Open-world Knowledge (2022.findings-acl)

Copied to clipboard

Chao-Hong Tan, Jia-Chen Gu, Chongyang Tao, Zhen-Hua Ling, Can Xu, Huang Hu, Xiubo Geng, Daxin Jiang

Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation (2020.coling-main)

Challenge:	Generating natural and informative texts has been a long-standing problem in NLP.
Approach:	They propose to augment TExt Generation via Task-specific and Open-world Knowledge in a unified framework.
Outcome:	The proposed model can learn what and how to generate on two text generation tasks.

Copied to clipboard

Junhao Liu, Linjun Shou, Jian Pei, Ming Gong, Min Yang, Daxin Jiang

XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation (2020.emnlp-main)

Challenge:	Cross-lingual Machine Reading Comprehension (CLMRC) is a challenging problem due to the lack of large-scale annotated datasets in low-source languages, such as Arabic, Hindi, and Vietnamese.
Approach:	They propose a novel approach to augment cross-lingual machine reading comprehension by combining knowledge from multiple language branch models into a single model for all target languages.
Outcome:	Extensive experiments on two CLMRC benchmarks show the proposed method is effective and robust to data noises.

Copied to clipboard

Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, Ming Zhou

Syntax-Enhanced Pre-trained Model (2021.acl-long)

Challenge:	XGLUE provides a benchmark dataset to train large-scale cross-lingual pre-trained models . XCLUE provides 11 diversified tasks that cover both understanding and generation scenarios .
Approach:	They introduce a new benchmark dataset to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora.
Outcome:	The proposed dataset is labeled in English and includes only natural language understanding tasks.

Copied to clipboard

Zenan Xu, Daya Guo, Duyu Tang, Qinliang Su, Linjun Shou, Ming Gong, Wanjun Zhong, Xiaojun Quan, Daxin Jiang, Nan Duan

Syntactic Text Processing

Challenge:	Existing methods that use syntax of text in pre-training and fine-tuning suffer from discrepancy between the two stages.
Approach:	They propose a model that utilizes the syntactic structure of text in pre-training and fine-tuning stages.
Outcome:	The proposed model achieves state-of-the-art on six public benchmark datasets.

Retrieval-Augmented Retrieval: Large Language Models are Strong Zero-Shot Retriever (2024.findings-acl)

Copied to clipboard

Tao Shen, Guodong Long, Xiubo Geng, Chongyang Tao, Yibin Lei, Tianyi Zhou, Michael Blumenstein, Daxin Jiang

Structural Contrastive Pretraining for Cross-Lingual Comprehension (2023.findings-acl)

Challenge:	Large-scale retrieval is indispensable in information-seeking tasks such as open-domain question answering and knowledgegrounded dialogue.
Approach:	They propose to use a large language model (LLM) to augment a query with its potential answers by prompting LLMs with a composition of the query and the query’s in-domain candidates.
Outcome:	The proposed method breaks brute-force combinations of retrievers with LLMs and lifts the performance of zero-shot retrieval to be very competitive on benchmark datasets.

Copied to clipboard

Nuo Chen, Linjun Shou, Tengtao Song, Ming Gong, Jian Pei, Jianhui Chang, Daxin Jiang, Jia Li

NeuronBlocks: Building Your NLP DNN Models Like Playing Lego (D19-3)

Challenge:	Existing methods to train multilingual language models using pretraining tasks like mask language modeling have yielded promising results on a wide range of downstream tasks.
Approach:	They propose a new task to align the structural words in a parallel sentence, enhancing models’ ability to comprehend cross-lingual representations.
Outcome:	The proposed task improves model's ability to comprehend cross-lingual representations by increasing the frequency of negative pairings.

Copied to clipboard

Ming Gong, Linjun Shou, Wutao Lin, Zhijie Sang, Quanjia Yan, Ze Yang, Feixiang Cheng, Daxin Jiang

Challenge:	Deep Neural Networks (DNN) have been widely employed in industry to address various natural language processing tasks.
Approach:	They propose an NLP toolkit that encapsulates neural network modules as building blocks to construct various DNN models with complex architecture.
Outcome:	The proposed toolkit can build, train, and test various DNN models with complex architecture.

Understand before Answer: Improve Temporal Reading Comprehension via Precise Question Understanding (2022.naacl-main)

Copied to clipboard

Hao Huang, Xiubo Geng, Guodong Long, Daxin Jiang

Coherent Entity Disambiguation via Modeling Topic and Categorical Dependency (2023.findings-emnlp)

Challenge:	Temporal reading comprehension (TRC) is a natural way to study temporal relations since natural language questions are flexible to capture divergent temporal relationships.
Approach:	They propose a reading comprehension approach that uses precise question understanding . they embed a temporal ordering question into two vectors and evaluate the temporal relation based on that .
Outcome:	The proposed approach outperforms strong baselines and achieves state-of-the-art performance on the TORQUE dataset.

Copied to clipboard

Zilin Xiao, Linjun Shou, Xingyao Zhang, Jie Wu, Ming Gong, Daxin Jiang

Joint Type Inference on Entities and Relations via Graph Convolutional Networks (P19-1)

Challenge:	Existing entity disambiguation methods struggle to capture explicit discourse-level dependencies, resulting in incoherent predictions at the abstract level.
Approach:	They propose an unsupervised variational autoencoder to extract latent topic vectors of context sentences to enhance coherence of entity predictions.
Outcome:	The proposed system achieves state-of-the-art on popular ED benchmarks with an average improvement of 1.3 F1 points.

Copied to clipboard

Changzhi Sun, Yeyun Gong, Yuanbin Wu, Ming Gong, Daxin Jiang, Man Lan, Shiliang Sun, Nan Duan

Structured Data in NLP

Challenge:	a novel graph convolutional network (GCN) is proposed for the task of joint entity relation extraction.
Approach:	They propose a graph convolutional network running on an entity-relation bipartite graph . they propose combining two different methods to perform joint entity relation extraction .
Outcome:	The proposed model outperforms existing joint models in entity performance and is competitive with the state-of-the-art in relation performance.

LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network (2020.acl-main)

Copied to clipboard

Wanjun Zhong, Duyu Tang, Zhangyin Feng, Nan Duan, Ming Zhou, Ming Gong, Linjun Shou, Daxin Jiang, Jiahai Wang, Jian Yin

Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understanding (2021.emnlp-main)

Challenge:	Existing methods for fact checking textual statements are not yet available.
Approach:	They propose a neural network approach capable of leveraging logical operations for fact checking . they use a textual statement and semi-structured tables to generate a program from it .
Outcome:	The proposed approach achieves state-of-the-art performance on TABFACT dataset . it derives a program (a.k.a. logical form) of the statement in semantic parsing manner .

Copied to clipboard

Yingmei Guo, Linjun Shou, Jian Pei, Ming Gong, Mingxing Xu, Zhiyong Wu, Daxin Jiang

SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval (2023.acl-long)

Challenge:	Existing approaches to scale out spoken language understanding to low-resource languages are noisy.
Approach:	They propose a method for mitigating noise in augmented data by training models with augmented datasets.
Outcome:	The proposed method outperforms state-of-the-art methods on two benchmark datasets.

Copied to clipboard

Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei

Re-entry Prediction for Online Conversations via Self-Supervised Learning (2021.findings-emnlp)

Challenge:	SimLM uses a simple bottleneck architecture that learns to compress the passage information into a dense vector through self-supervised pre-training.
Approach:	They propose a simple yet effective pre-training method for dense passage retrieval that learns to compress the passage information into a dense vector through self-supervised pre-tuning.
Outcome:	The proposed method outperforms multi-vector approaches on large-scale passage retrieval datasets and shows significant improvements over baselines.

Copied to clipboard

Lingzhi Wang, Xingshan Zeng, Huang Hu, Kam-Fai Wong, Daxin Jiang

CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search (2022.emnlp-main)

Challenge:	Existing work on re-entry prediction ignores conversation thread patterns and repeated engagement of target users.
Approach:	They propose to use conversation thread patterns to predict whether a user will come back to a conversation they once participated in to train a model on labels that are automatically derived from the data.
Outcome:	The proposed task outperforms the state-of-the-art models on two social media datasets with fewer parameters and faster convergence.

Copied to clipboard

Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang, Weizhu Chen, Nan Duan

Improving Zero-Shot Cross-lingual Transfer for Multilingual Question Answering over Knowledge Graph (2021.naacl-main)

Challenge:	Existing code pre-training approaches often adopt (masked) language modeling as the training objective which targets on learning to predict (macked) tokens in a given code context.
Approach:	They propose a code-text contrastive learning model which learns function-level code semantic representations through large-scale code corpus.
Outcome:	The proposed model achieves new state-of-the-art with significant improvement over existing pre-trained models on eleven domain/language-specific code search tasks with six programming languages in different code granularity.

Copied to clipboard

Yucheng Zhou, Xiubo Geng, Tao Shen, Wenqiang Zhang, Daxin Jiang

Soft-Labeled Contrastive Pre-Training for Function-Level Code Representation (2022.findings-emnlp)

Challenge:	Existing approaches to solve multilingual question answering over knowledge graph (KGQA) use of bilingual lexicon induction to map training questions into those in target language circumvents language inconsistency .
Approach:	They propose to use bilingual lexicon induction to map training questions in source and target languages as augmented training data to minimize syntax-disorder.
Outcome:	The proposed model narrows the gap in zero-shot cross-lingual transfer between source and target languages.

Copied to clipboard

Xiaonan Li, Daya Guo, Yeyun Gong, Yun Lin, Yelong Shen, Xipeng Qiu, Daxin Jiang, Weizhu Chen, Nan Duan

Maria: A Visual Experience Powered Conversational Agent (2021.acl-long)

Challenge:	Existing methods for contrastive pre-training ignore the relevance between codes in large code corpus.
Approach:	They propose a Soft-labeled contrastive pre-training framework with positive sample construction methods to learn functional-level code representation.
Outcome:	The proposed framework can obtain fine-grained soft-labels through an iterative adversarial manner and use them to learn better code representation.

Copied to clipboard

Zujie Liang, Huang Hu, Can Xu, Chongyang Tao, Xiubo Geng, Yining Chen, Fan Liang, Daxin Jiang

Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension (2020.acl-main)

Challenge:	Existing studies focus on grounding conversational agents on text-only corpora, but they lack the perception ability to our physical world.
Approach:	They propose to ground conversational agents on images retrieved from large-scale image indexes . they propose to use visual knowledge to generate informative responses based on the extracted knowledge .
Outcome:	The proposed agent outperforms state-of-the-art methods on automatic metrics and human evaluation.

Copied to clipboard

Bo Zheng, Haoyang Wen, Yaobo Liang, Nan Duan, Wanxiang Che, Daxin Jiang, Ming Zhou, Ting Liu

Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation (2020.emnlp-main)

Challenge:	Existing approaches to machine reading comprehension treat documents at their hierarchical nature, ignoring their dependencies.
Approach:	They propose a machine reading comprehension benchmark with two-grained answers . they use graph attention networks to model documents at their hierarchical nature .
Outcome:	The proposed framework outperforms existing systems at long and short answer criteria.

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Yu Yan, Jie Fu, Bo Shao, Daxin Jiang, Jiancheng Lv, Nan Duan

Text Generation

Challenge:	Existing methods for news headline generation focus on producing a single short sentence . et al., 2017; Gehrmann e.t., 2018; Zhong ee., 2019) focus on single-headline generation.
Approach:	They propose a method to generate multiple headlines with keyphrases of user interests . they propose generating multiple keyphrase-relevant headlines using a transformer decoder .
Outcome:	The proposed method achieves state-of-the-art in terms of quality and diversity.

MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding (2021.acl-long)

Copied to clipboard

Jia-Chen Gu, Chongyang Tao, Zhenhua Ling, Can Xu, Xiubo Geng, Daxin Jiang

Towards Robust Ranker for Text Retrieval (2023.findings-acl)

Challenge:	Existing models for multi-party conversation represent interlocutors and utterances individually . existing methods ignore complicated structure of MPC which may provide crucial interlocutor and tertiary semantics.
Approach:	They propose a pre-trained model for multi-party conversation that considers learning who says what to whom in a unified model with elaborated self-supervised tasks.
Outcome:	The proposed model outperforms existing models on three downstream tasks at two benchmarks.

Copied to clipboard

Yucheng Zhou, Tao Shen, Xiubo Geng, Chongyang Tao, Can Xu, Guodong Long, Binxing Jiao, Daxin Jiang

XLM-D: Decorate Cross-lingual Pre-training Model as Non-Autoregressive Neural Machine Translation (2022.emnlp-main)

Challenge:	Existing methods for text retrieval are based on a 'retrieval & rerank' pipeline, which uses a fast retriever to fetch a set of top document candidates, while a robust ranker is based upon a weak negative mining during contrastive learning.
Approach:	They propose a multi-adversarial training strategy that leverages multiple retrievers as generators to challenge a ranker.
Outcome:	The proposed model outperforms the existing de facto ranker training paradigms on the passage retrieval benchmarks using BM25-reranking, full-ranking and retriever distillation.

Copied to clipboard

Yong Wang, Shilin He, Guanhua Chen, Yun Chen, Daxin Jiang

LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation (2025.findings-emnlp)

Challenge:	Existing pre-training language models have been successful in natural language understanding and autoregressive generation tasks, but non-autoregressive models have not been sufficiently successful.
Approach:	They propose a pre-trained masked language model (MLM) and a non-autoregressive generation model with a lightweight decorator.
Outcome:	The proposed model outperforms the previous mask-predict model on translation datasets by 19.9x.

Copied to clipboard

Yiqun Shen, Song Yuan, Zhengze Zhang, Xiaoliang Wang, Daxin Jiang, Nguyen Cam-Tu

Challenge:	Existing methods for cache compression are heuristic and lack dynamic budget allocation . cnn's john mccartney and johnny mccain present a new approach for cache eviction and dynamic budgets .
Approach:	They propose a unified framework for cache compression that minimizes information loss in transformer residual streams.
Outcome:	The proposed method consistently maintains top performance across task types.

Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained Propaganda Detection (2020.emnlp-main)

Copied to clipboard

Ruize Wang, Duyu Tang, Nan Duan, Wanjun Zhong, Zhongyu Wei, Xuanjing Huang, Daxin Jiang, Ming Zhou

Challenge:	Existing methods for fine-grained propaganda detection are not based on input-output data, but instead use declarative knowledge to detect propagandistic text fragments.
Approach:	They propose a method to inject declarative knowledge of fine-grained propaganda techniques into training data to get better representations of propagandistic texts.
Outcome:	The proposed method achieves superior performance on a large dataset for propaganda detection.

Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQA (2022.findings-emnlp)

Copied to clipboard

Junjie Huang, Wanjun Zhong, Qian Liu, Ming Gong, Daxin Jiang, Nan Duan

Learning Neural Templates for Recommender Dialogue System (2021.emnlp-main)

Challenge:	Existing methods to retrieve evidences from corpus are difficult due to table-text discrepancy and data sparsity problem.
Approach:	They propose an optimized OpenQA Table-Text Retriever to retrieve tabular and textual evidences from tabular resources.
Outcome:	The proposed OpenQA Table-Text Retriever significantly outperforms existing methods on QA tasks.

Copied to clipboard

Zujie Liang, Huang Hu, Can Xu, Jian Miao, Yingying He, Yining Chen, Xiubo Geng, Fan Liang, Daxin Jiang

Allies: Prompting Large Language Model with Beam Search (2023.findings-emnlp)

Challenge:	Recent advances in neural models have shown promising progress on this task, but key challenges remain .
Approach:	They propose a framework that can decouple dialogue generation from item recommendation . they use a response template generator and item selector to generate a responses template .
Outcome:	The proposed framework outperforms the state-of-the-art methods on the benchmark ReDial.

Copied to clipboard

Hao Sun, Xiao Liu, Yeyun Gong, Yan Zhang, Daxin Jiang, Linjun Yang, Nan Duan

Instructed Language Models with Retrievers Are Powerful Entity Linkers (2023.emnlp-main)

Challenge:	Existing methods to build LLMs with stacking are limited by their information coverage and low fault tolerance.
Approach:	They propose a method that leverages large language models to iteratively generate new queries from an input query.
Outcome:	The proposed method outperforms baselines on open-domain question answering benchmarks.

Copied to clipboard

Zilin Xiao, Ming Gong, Jie Wu, Xingyao Zhang, Linjun Shou, Daxin Jiang