Papers by Chi Xiao
FlashBack: Efficient Retrieval-Augmented Language Modeling for Fast Inference (2025.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-Augmented Language Modeling (RALM) is a popular approach for large language models. |
| Approach: | They propose a modular RALM that integrates large language models with documents from an external corpus to improve inference efficiency. |
| Outcome: | The proposed method improves inference efficiency with appending context pattern while maintaining decent performance after fine-tuning by Low-Rank Adaption. |
Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models (2025.emnlp-main)
Copied to clipboard
Kaiyan Chang, Yonghao Shi, Chenglong Wang, Hang Zhou, Chi Hu, Xiaoqian Liu, Yingfeng Luo, Yuan Ge, Tong Xiao, JingBo Zhu
| Challenge: | Recent training-based TTS methods, such as continued reinforcement learning, have surged in popularity, while training-free TTS approaches are gradually fading from prominence. |
| Approach: | They propose a fine-grained sequential scaling method guided by process verification that integrates training-free TTS methods with other classical parallel scaling methods at the step level. |
| Outcome: | Experiments on five instruction-tuned large language models (LLMs) show that training-free TTS methods can extend reasoning performance boundaries. |
ET5: A Novel End-to-end Framework for Conversational Machine Reading Comprehension (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods require three steps to understand text, but span extraction and question rephrasing steps are not fully exploited. |
| Approach: | They propose a framework for conversational machine reading comprehension based on shared parameter mechanism . experimental results show the proposed framework achieves new state-of-the-art results on the ShARC leaderboard . |
| Outcome: | The proposed framework achieves state-of-the-art on the ShARC leaderboard with the BLEU-4 score of 55.2. |
Cross-Lingual Phrase Retrieval (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual phrase retrieval learn word or sentence representations in word or sentences. |
| Approach: | They propose a cross-lingual phrase retrieval model that extracts phrase representations from unlabeled example sentences. |
| Outcome: | The proposed model outperforms state-of-the-art methods on a large-scale cross-lingual phrase retrieval dataset, showing it can perform in an unseen language pair during training. |
RankPrompt: Step-by-Step Comparisons Make Language Models Better Reasoners (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing solutions to reasoning tasks require extensive human annotations or fail in scenarios with inconsistent responses. |
| Approach: | They propose a new method that enables LLMs to self-rank their responses without additional resources. |
| Outcome: | The proposed method improves reasoning performance of ChatGPT and GPT-4 with 13% improvement over existing methods. |
Bridging The Gap: Entailment Fused-T5 for Open-retrieval Conversational Machine Reading Comprehension (2023.acl-long)
Copied to clipboard
| Challenge: | Open-retrieval conversational machine reading comprehension (OCMRC) simulates real-life conversation scenes. |
| Approach: | They propose a one-stage end-to-end framework to bridge the information gap between decision-making and question generation in a global understanding manner. |
| Outcome: | The proposed framework achieves new state-of-the-art performance on the OR-ShARC benchmark. |
Bridging the Granularity Gap for Acoustic Modeling (2023.findings-acl)
Copied to clipboard
Chen Xu, Yuhao Zhang, Chengbo Jiao, Xiaoqian Liu, Chi Hu, Xin Zeng, Tong Xiao, Anxiang Ma, Huizhen Wang, Jingbo Zhu
| Challenge: | Despite the success of speech recognition, how to encode the speech features effectively remains an open problem. |
| Approach: | They propose a Progressive Down-Sampling technique which compresses acoustic features into coarser-grained units containing more complete semantic information, like text-level representation. |
| Outcome: | The proposed method yields comparable or better results on the speech recognition task and inference speedups ranging from 1.20x to 1.47x. |
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation (2024.emnlp-main)
Copied to clipboard
Yuan Ge, Yilun Liu, Chi Hu, Weibin Meng, Shimin Tao, Xiaofeng Zhao, Mahong Xia, Zhang Li, Boxing Chen, Hao Yang, Bei Li, Tong Xiao, JingBo Zhu
| Challenge: | Existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset. |
| Approach: | They propose an industrial-friendly, expert-aligned and diversity-preserved instruction data selection method: Clustering and Ranking (CaR). |
| Outcome: | The proposed method outperforms Alpaca's existing methods by 32.1% in GPT-4 evaluations. |
RankNAS: Efficient Neural Architecture Search by Pairwise Ranking (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods require training millions of architectures to estimate the accuracy of the search results. |
| Approach: | They propose a performance ranking method (RankNAS) that uses pairwise ranking and search space pruning to enlarge the search space. |
| Outcome: | The proposed method significantly accelerates NAS through pairwise ranking and search space pruning. |
Improving Autoregressive Grammatical Error Correction with Non-autoregressive Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive models assign low probabilities to tokens that need corrections . grammatical error correction (GEC) is widely applied to natural language processing tasks . |
| Approach: | They propose to use a non-autoregressive model as an auxiliary model to train GEC models to correct grammatical errors in sentences. |
| Outcome: | The proposed method outperforms baselines on English and Chinese GEC tasks significantly. |
Learning Architectures from an Extended Search Space for Language Modeling (2020.acl-main)
Copied to clipboard
Yinqiao Li, Chi Hu, Yuhao Zhang, Nuo Xu, Yufan Jiang, Tong Xiao, Jingbo Zhu, Tongran Liu, Changliang Li
| Challenge: | Neural architecture search (NAS) has advanced in recent years but most NAS systems restrict search to learning architectures of a recurrent or convolutional cell. |
| Approach: | They propose a general approach to learn both intra-cell and inter-cell architectures . they implement their approach in a differentiable architecture search system . |
| Outcome: | The proposed approach outperforms the baseline on PTB and WikiText data and shows good transferability to other systems. |
Benchmarking Automated Clinical Language Simplification: Dataset, Algorithm, and Evaluation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies to translate medical jargon into layperson-understandable language focus on accuracy and readability aspects of clinical language. |
| Approach: | They propose to construct a dataset to support automated clinical language simplification and propose a model that mimics the human annotation procedure. |
| Outcome: | The proposed model matches human annotation procedures and achieves state-of-the-art performance compared with baselines. |
Teaching Language Models to Self-Improve by Learning from Language Feedback (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) generate content that can be untruthful or harmful. |
| Approach: | They propose a method that leverages model feedback for alignment . they use a base language model to generate initial responses, critiqued and refined . |
| Outcome: | The proposed method outperforms strong baselines across diverse tasks and model sizes. |
Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition (D19-1)
Copied to clipboard
| Challenge: | Neural architecture search (NAS) is a popular approach for finding new models and freeing researchers from the hard work of designing network architectures. |
| Approach: | They propose differentiable neural architecture search methods for natural language processing . they remove the softmax-local constraint and apply it to named entity recognition . |
| Outcome: | The proposed method outperforms strong baselines on the language modeling task. |
Pi-SQL: Enhancing Text-to-SQL with Fine-Grained Guidance from Pivot Programming Languages (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing prompt-based methods craft meticulous text guidelines and examples to facilitate SQL generation, but their accuracy is hindered by the large semantic gap between the texts and the low-resource SQL programs. |
| Approach: | They propose to use Python as a pivot to bridge between natural language query and SQL program. |
| Outcome: | The proposed method improves the execution accuracy of the best-performing baseline by up to 3.20. |
Hierarchical Multi-label Text Classification with Horizontal and Vertical Category Correlations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to hierarchical multi-label text classification ignore vertical category correlations or exploit dependencies across levels without considering horizontal correlations . |
| Approach: | They propose a hierarchical multi-label text classification framework that considers both vertical and horizontal category correlations. |
| Outcome: | The proposed framework improves on real-world HMTC datasets with significant improvements over baselines. |