Papers by Qifan Yu
Revisiting Data Reconstruction Attacks on Real-world Dataset for Federated Natural Language Understanding (2024.lrec-main)
Copied to clipboard
Zhuo Zhang, Jintao Huang, Xiangjing Hu, Jingyuan Zhang, Yating Zhang, Hui Wang, Yue Yu, Qifan Wang, Lizhen Qu, Zenglin Xu
| Challenge: | Existing DRA methods fail to accurately recover the original text of real-world privacy data. |
| Approach: | They propose to use a real-world privacy dataset to examine the performance of federated learning (FL) methods. |
| Outcome: | The proposed method improves on a real-world privacy dataset and shows that the tokens within a recovery sentence are disordered and intertwined with tokens from other sentences in the same training batch. |
Enhancing Auto-regressive Chain-of-Thought through Loop-Aligned Reasoning (2026.eacl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought prompting is a powerful technique for enhancing language model’s reasoning capabilities, but generating long and correct CoT trajectories is challenging. |
| Approach: | They propose to align the steps of Chain-of-Thought reasoning with loop iterations and apply intermediate supervision during the training of Looped Transformers. |
| Outcome: | The proposed method generates accurate reasoning chains for complex problems exceeding training length, and improves performance of the auto-regressive model. |
LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Currently, large language models (LLMs) train on short text segments due to the computational overhead quadratic in the input lengths of their Transformer architectures. |
| Approach: | They propose a method that allows LLMs pre-trained with 2K or 4K-long segments to generalize to up to 200M length inputs while retaining perplexity. |
| Outcome: | The proposed method achieves 2.7 decoding speed up and 7.5 memory saving over the original model. |
Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have highlighted severe limitations of contrastive learning models in their ability to perform compositional reasoning over objects, attributes, and relations. |
| Approach: | They propose a graph decomposition framework and negative mining techniques to improve attribute binding and relation understanding of scene graphs. |
| Outcome: | The proposed approach improves attribute binding, relation understanding, generalization, and productivity on multiple benchmarks. |
Ameli: Enhancing Multimodal Entity Linking with Fine-Grained Attributes (2024.eacl-long)
Copied to clipboard
| Challenge: | Experimental results show that understanding attributes of mentions from text descriptions and visual images plays a vital role in multimodal entity linking. |
| Approach: | They propose to integrate attributes into multimodal entity linking using a text-image-based knowledge base. |
| Outcome: | The proposed approach integrates attributes into disambiguation. |
CLOWER: A Pre-trained Language Model with Contrastive Learning over Word and Character Representations (2022.coling-1)
Copied to clipboard
Borun Chen, Hongyin Tang, Jiahao Bu, Kai Zhang, Jingang Wang, Qifan Wang, Hai-Tao Zheng, Wei Wu, Liqian Yu
| Challenge: | Pre-trained language models (PLMs) have achieved remarkable performance gains across numerous downstream tasks in natural language understanding. |
| Approach: | They propose a Chinese pre-trained language model that implicitly encodes words into characters . they propose 'contrastive learning over word' and 'character' representations to improve learning . |
| Outcome: | The proposed model can encode words into fine-grained representations without modification of production pipelines. |
FedPETuning: When Federated Learning Meets the Parameter-Efficient Tuning Methods of Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing research on federated learning (FL) for pre-trained language models (PLMs) with increasing concerns about data privacy, enterprises or institutions are not allowed to collect data from end devices or local clients to a centralized server for fine-tuning PLMs. |
| Approach: | They investigate the parameter-efficient tuning of pre-trained language models (PLMs) and develop a federated benchmark for four representative PETuning methods . |
| Outcome: | The proposed method can defend against privacy attacks and maintain acceptable performance with reducing heavy resource consumption. |
MixPAVE: Mix-Prompt Tuning for Few-shot Product Attribute Value Extraction (2023.findings-acl)
Copied to clipboard
Li Yang, Qifan Wang, Jingang Wang, Xiaojun Quan, Fuli Feng, Yu Chen, Madian Khabsa, Sinong Wang, Zenglin Xu, Dongfang Liu
| Challenge: | Existing methods for product attribute value extraction focus on extracting values for a set of known attributes with sufficient training data. |
| Approach: | They propose a prompt tuning approach to extract attributes from product information using mixed prompts. |
| Outcome: | The proposed approach improves on two product benchmarks and shows parameter-efficient training and avoids model overfitting. |
APrompt: Attention Prompt Tuning for Efficient Adaptation of Pre-trained Language Models (2023.emnlp-main)
Copied to clipboard
Qifan Wang, Yuning Mao, Jingang Wang, Hanchao Yu, Shaoliang Nie, Sinong Wang, Fuli Feng, Lifu Huang, Xiaojun Quan, Zenglin Xu, Dongfang Liu
| Challenge: | Existing prompt tuning methods only introduce prompts at the input layer, limiting performance and leaving large room for improvement. |
| Approach: | They propose a method that involves tuning a small set of soft prompts for pre-trained language models. |
| Outcome: | The proposed method outperforms state-of-the-art methods with pre-trained models on the SuperGLUE benchmark. |
Multimodal Instruction Tuning with Conditional Mixture of LoRA (2024.acl-long)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) have demonstrated proficiency in diverse tasks across different domains. |
| Approach: | They propose a method that integrates multimodal instruction tuning with Conditional Mixture-of-LoRA. |
| Outcome: | Experimental results show that MixLoRA outperforms LoRA with the same or higher ranks . MLLMs have demonstrated remarkable proficiency in diverse tasks across domains . |
Vision-Flan: Scaling Human-Labeled Tasks in Visual Instruction Tuning (2024.findings-acl)
Copied to clipboard
Zhiyang Xu, Chao Feng, Rulin Shao, Trevor Ashby, Ying Shen, Di Jin, Yu Cheng, Qifan Wang, Lifu Huang
| Challenge: | Recent vision-language models (VLMs) have shown impressive capabilities as general visual assistants, but there are two challenges to their performance: (1) lacking task diversity in pretraining and visual instruction tuning; (2) annotation error and bias in GPT-4 synthesized instruction tuning data. |
| Approach: | They propose a two-stage instruction tuning framework that fine tunes VLMs firstly and further tuned on GPT-4 synthesized data. |
| Outcome: | The proposed framework outperforms the traditional single-stage visual instruction tuning framework and achieves state-of-the-art performance across a wide range of multi-modal evaluation benchmarks. |
Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Models (2025.findings-emnlp)
Copied to clipboard
Haibo Wang, Zhiyang Xu, Yu Cheng, Shizhe Diao, Yufan Zhou, Yixin Cao, Qifan Wang, Weifeng Ge, Lifu Huang
| Challenge: | Video Large Language Models (VLMs) have been praised for their performance in coarse-grained video understanding but still face ineffective temporal grounding and inadequate timestamp representations. |
| Approach: | They propose a novel Video-LLM that senses and reasoned over specific video moments with fine-grained temporal precision. |
| Outcome: | The proposed model surpasses existing models in fine-grained video understanding tasks and exhibits strong potential as a general video understanding assistant. |
Inference Compute-Optimal Video Vision Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Using video vision language models, inference costs are often more expensive than finetuning. |
| Approach: | They investigate the optimal allocation of inference compute across three key scaling factors in video vision language models. |
| Outcome: | The proposed model configurations are based on three key scaling factors . the results can be applied to real-world tasks and tasks with fixed inference budgets. |
RoAST: Robustifying Language Models via Adversarial Perturbation with Selective Training (2023.findings-emnlp)
Copied to clipboard
Jaehyung Kim, Yuning Mao, Rui Hou, Hanchao Yu, Davis Liang, Pascale Fung, Qifan Wang, Fuli Feng, Lifu Huang, Madian Khabsa
| Challenge: | Several perspectives of robustness for pre-trained language models have been studied independently, but lacking a unified consideration in multiple perspectives. |
| Approach: | They propose a technique to enhance the multi-perspective robustness of LMs by introducing adversarial perturbation while the model parameters are selectively updated upon their relative importance. |
| Outcome: | The proposed technique improves the robustness of LMs by incorporating four perspectives on model robustness. |