Papers by Xiaoling Wang
Discovering Better Model Architectures for Medical Query Understanding (2021.naacl-industry)
Copied to clipboard
| Challenge: | Neural architecture search (NAS) has attracted intense attention in computer vision and NLP. |
| Approach: | They propose to use neural architecture search to optimize model architectures for medical questions . they propose to modify the ENAS method to accelerate and stabilize the search results . |
| Outcome: | The proposed approach outperforms baseline models on two medical questions . it is compared with other NAS methods and shows that it provides the best results . |
GAML-BERT: Improving BERT Early Exiting by Gradient Aligned Mutual Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve the early exiting of natural language processing (NLP) are notoriously gigantic and slow in both training and inference. |
| Approach: | They propose a framework for improving the early exiting of BERT by asking each exit to distill knowledge from each other. |
| Outcome: | The proposed framework outperforms the state-of-the-art (SOTA) BERT early exiting methods on the GLUE benchmark. |
Length Generalization of Causal Transformers without Position Encoding (2024.findings-acl)
Copied to clipboard
| Challenge: | Besides Transformers without position encodings, the success of NoPE provides a new way to overcome the challenge of generalizing to longer sentences. |
| Approach: | They propose a parameter-efficient tuning for searching attention heads’ best temperature hyper-parameters, which substantially expands NoPE’s context size. |
| Outcome: | The proposed tuning significantly expands NoPE's context size, allowing it to generalize to longer sentences with state-of-the-art generalization algorithms. |
AutoMedEval: Harnessing Language Models for Automatic Medical Capability Evaluation (2025.acl-long)
Copied to clipboard
Xiechi Zhang, Zetian Ouyang, Linlin Wang, Gerard De Melo, Zhu Cao, Xiaoling Wang, Ya Zhang, Yanfeng Wang, Liang He
| Challenge: | Existing evaluation methods based on large language models (LLMs) are expensive and lack expertise due to limitations in human expertise. |
| Approach: | They propose an open-source automatic evaluation model with 13B parameters specifically engineered to measure the question-answering proficiency of medical LLMs. |
| Outcome: | The proposed model surpasses baselines in terms of correlation with human judgments. |
Unified Demonstration Retriever for In-Context Learning (2023.acl-long)
Copied to clipboard
Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, Xipeng Qiu
| Challenge: | In-context learning is a new learning paradigm where a language model conditions on a few input-output pairs (demonstrations) and a test input, and directly outputs the prediction. |
| Approach: | They propose a single model to retrieve demonstrations for a wide range of tasks by combining training signals from various tasks into a unified list-wise ranking formulation by language model’s feedback. |
| Outcome: | The proposed model outperforms baselines on 30+ tasks across 13 task families and multiple data domains. |
Rehearsal-free Continual Language Learning via Efficient Parameter Isolation (2023.acl-long)
Copied to clipboard
Zhicheng Wang, Yufang Liu, Tao Ji, Xiaoling Wang, Yuanbin Wu, Congcong Jiang, Ye Chao, Zhencong Han, Ling Wang, Xu Shao, Wenqiu Zeng
| Challenge: | Existing methods for learning continual tasks do not cache history data, which makes the problem more challenging. |
| Approach: | They propose a method that allocates a small portion of private parameters and learns them with a shared pre-trained model. |
| Outcome: | The proposed method is comparable to existing methods and comparable to those using historical data. |
Title2Event: Benchmarking Open Event Extraction with a Large-scale Chinese Title Dataset (2022.emnlp-main)
Copied to clipboard
Haolin Deng, Yanan Zhang, Yangfan Zhang, Wangyang Ying, Changlong Yu, Jun Gao, Wei Wang, Xiaoling Bai, Nan Yang, Jin Ma, Xiang Chen, Tianhua Zhou
| Challenge: | Existing EE datasets define fixed event types and design specific schemas for each of them, failing to cover diverse events emerging from the online text. |
| Approach: | They propose to use a sentence-level dataset to benchmark Open Event Extraction without restricting event types. |
| Outcome: | The proposed dataset contains more than 42,000 news titles in 34 topics collected from Chinese web pages. |
StreamingEval: A Unified Evaluation Framework towards Realistic Streaming Video Understanding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing research on streaming video understanding focuses on isolated aspects of visual understanding, but ignores practical deployability under realistic resource constraints. |
| Approach: | They propose a framework to evaluate streaming video understanding capabilities under realistic constraints. |
| Outcome: | StreamingEval benchmarks offline and online video models under a standardized protocol . it evaluates visual encoding efficiency, text decoding latency and task performance . |
Word Reordering for Zero-shot Cross-lingual Structured Prediction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current sentence encoders are word order sensitive, resulting in poor performance . Adapting word order from one language to another is key in cross-lingual structured prediction. |
| Approach: | They propose a new module to organize words following the source language order . they build structured prediction models with bag-of-words inputs and introduce a module to do this . |
| Outcome: | The proposed model significantly improves target language performance for languages that are distant from the source language. |
Generative Models for Automatic Medical Decision Rule Extraction from Text (2024.emnlp-main)
Copied to clipboard
| Challenge: | Medical decision rules are traditionally constructed by medical experts, which is expensive and hard to scale up. |
| Approach: | They propose to extract medical decision rules from text using generative models . their code will be open-source upon acceptance . |
| Outcome: | The proposed model outperforms state-of-the-art models on a Chinese benchmark and achieves 67% tree accuracy. |
Faster Speculative Decoding via Effective Draft Decoder with Pruned Candidate Tree (2025.acl-long)
Copied to clipboard
| Challenge: | Effective Draft Decoder (EDD) is a powerful decoding method that generates more accurate draft tokens by leveraging the encoding results as soft prompts. |
| Approach: | They propose an effective draft decoder which treats the LLM as a powerful encoder and generates more accurate draft tokens by leveraging the encoding results as soft prompts. |
| Outcome: | The proposed method significantly improves the performance of large language models and reduces inference latency. |
Typology Guided Multilingual Position Representations: Case on Dependency Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent multilingual models benefit from strong unified semantic representation models, but conflicting linguistic regularities may break the effectiveness of word position features in multilingual learning. |
| Approach: | They propose to combine prior knowledge from typology features and existing position vectors to create a position generation network which combines prior knowledge of a language's position space and typological characterization. |
| Outcome: | The proposed model can achieve the best multilingual parsing results by combining prior knowledge from typology features and existing position vectors. |
BADGE: Speeding Up BERT Inference after Deployment via Block-wise Bypasses and Divergence-based Early Exiting (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent years have witnessed the rise of many pre-trained language models (PLMs) such as GPT (Radford et al., 2019) and XLNet (Yang e.t al, 2019). |
| Approach: | They propose a framework which consists of two off-the-shelf methods for improving PLMs’ early exiting. |
| Outcome: | The proposed method can reduce the average latency of pre-trained language models and work with other inference speed-up methods like model pruning. |
Zero-Shot Event Detection Based on Ordered Contrastive Learning and Prompt-Based Prediction (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing zero-shot event detection methods do not work for unseen types . supervised methods require predefined event types or external tools . |
| Approach: | They propose a framework to detect events from unstructured text without annotating samples . they propose to use ordered contrastive learning and prompt-based prediction to identify trigger words . |
| Outcome: | The proposed model detects events more effectively and accurately than state-of-the-art methods. |
A Unified Encoding of Structures in Transition Systems (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to encode dynamic structures only encode partial information of structures. |
| Approach: | They propose a new attention-based encoder unifying all structures in a transition system. |
| Outcome: | The proposed method significantly improves the test speed and achieves the best transition-based model. |
Enhancing In-Context Learning via Implicit Demonstration Augmentation (2024.acl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a new paradigm for pre-trained language models that can make predictions for unseen inputs without updating parameters. |
| Approach: | They propose a method that enables a model to augmented copies of a demonstration by leveraging their deep feature distribution and a logit calibration mechanism. |
| Outcome: | The proposed method significantly improves the average and worst-case accuracy across diverse PLMs and tasks. |
DVAGen: Dynamic Vocabulary Augmented Generation (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Existing dynamic vocabulary approaches struggle to generalize to novel or out-of-vocabulary words, limiting their flexibility in handling diverse token combinations. |
| Approach: | They propose an open-source framework for training, evaluation, and visualization of dynamic vocabulary-augmented language models. |
| Outcome: | The proposed framework validates the effectiveness of dynamic vocabulary-augmented language models on modern LLMs and shows support for batch inference significantly improving inference throughput. |
ACE-M3: Automatic Capability Evaluator for Multimodal Medical Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing metrics for multimodal large language models only focus on token overlap and may not align with human judgment. |
| Approach: | They propose an open-source model that assesses the question answering abilities of multimodal large language models. |
| Outcome: | Experiments show that the ACE-M3 model performs better than existing models and is more reliable than existing metrics. |
IAPT: Instance-Aware Prompt Tuning for Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for prompt tuning require many soft tokens to guarantee performance . large language models still require a large amount of GPU memory and computations to fine-tune . |
| Approach: | They propose to use a parameter-efficient soft prompt generator to generate idiosyncratic soft prompts for each input instruction. |
| Outcome: | The proposed method outperforms the baselines with comparable tunable parameters and is more efficient than LoRA under the single-backbone multi-tenant setting. |
Logic-Regularized Verifier Elicits Reasoning from LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Typical verifiers require resource-intensive supervised dataset construction, which is costly and faces limitations in data diversity. |
| Approach: | They propose an unsupervised verifier regularized by logical rules that uses internal activations and logical constraints on multiple reasoning paths. |
| Outcome: | Experiments on 10 datasets show that the proposed verifier outperforms baselines and is comparable to the supervised verifier. |
SCA: Selective Compression Attention for Efficiently Extending the Context Window of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to compress the KV cache of large language models are expensive and limited in their context window and cost. |
| Approach: | They propose a method to expand the context window and reduce memory footprint by compressing the KV cache of large language models. |
| Outcome: | The proposed method can reduce memory footprint and expand context window of large language models without training. |
Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve NAT model's performance but do not fully utilize it. |
| Approach: | They propose a non-autoregressive translation method which can obtain high-quality translations while maintaining the inference speed of NAT models. |
| Outcome: | The proposed method outperforms the autoregressive translation model on three translation tasks with 7.6 speedup. |
Hierarchical Safety Realignment: Lightweight Restoration of Safety in Pruned Large Vision-Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent work has shown that pruning can reduce model performance, but it can also lead to degradation in safety performance. |
| Approach: | They propose a hierarchical safety realignment approach to prune large vision-Language Models . they quantify contribution of each attention head to safety and restore neurons . |
| Outcome: | The proposed approach achieves significant safety improvements in LVLMs pruned post pruning. |
Generation with Dynamic Vocabulary (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using static vocabulary, vocabulary is ignored in advanced generation tasks. |
| Approach: | They propose a dynamic vocabulary that can involve arbitrary text spans during generation. |
| Outcome: | The proposed vocabulary can be deployed in a plug-and-play way, thus is attractive for various downstream applications. |