Papers by Zhe Lin
Advancing Vision-Language Models with Adapter Ensemble Strategies (2024.findings-emnlp)
Copied to clipboard
| Challenge: | CLIP revolutes vision-language pretraining by using contrastive learning on paired web data. |
| Approach: | They propose to combine a "adapter ensemble" with traditional machine learning techniques to augment large-scale pretrained vision-language models. |
| Outcome: | The proposed model outperforms baselines and derives improvement when the number of ensemble parameters increases. |
MMedAgent: Learning to Use Medical Tools with Multi-modal Agent (2024.findings-emnlp)
Copied to clipboard
Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, Yixin Wang
| Challenge: | Multi-modal Large Language Models (MLLMs) exhibit limited generality and often fall short when compared to specialized models. |
| Approach: | They propose a multi-modal medical agent that picks the most suitable medical tools based on user inputs. |
| Outcome: | The proposed agent performs better than open-source models and the closed-source model, GPT-4o. |
Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive capabilities, but still suffer from inconsistency issues. |
| Approach: | They develop a ConsisEval benchmark to evaluate LLMs' inconsistency . they find that LLM models can paradoxically fail at easier problems . |
| Outcome: | The proposed model achieves highest consistency score but inconsistent to specific questions due to distraction by redundant information, misinterpretation of questions, etc. |
Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve self-correction performance of Large Language Models are based on intrinsic selfcorrectione, which allows the model to check and revise its selfgenerated answers without external feedback. |
| Approach: | They propose to decompose the self-correction capability into confidence and critique capabilities and a metric for overall self-corretion capability evaluation. |
| Outcome: | The proposed method outperforms vanilla SFT and achieves much higher accuracy after self-correction. |
An Empirical Study of Multimodal Model Merging (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown that model merging can generate a multi-task solution without synchronous training. |
| Approach: | They propose to merge vision, language, and cross-modal transformers of a modality-specific architecture to create a parameter-efficient architecture. |
| Outcome: | The proposed model merging outperforms naive models on various tasks with improvements of 3% on VQA, 7% on COCO retrieval, 25% on NLVR2, 14% on Flickr30k and 3% ADE20k. |
Visual Information Guided Zero-Shot Paraphrase Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Several studies use different information as ”pivot” such as language, semantic representation and so on. |
| Approach: | They propose to use visual information as the "pivot" of back-translation to generate paraphrases using paired image-caption data. |
| Outcome: | The proposed model generates paraphrase with good relevancy, fluency and diversity . it is based on paired image-caption data and can train a paraphrasing model . |
Pushing Paraphrase Away from Original Sentence: A Multi-Round Paraphrase Generation Approach (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent years, neural paraphrase generation models have demonstrated superior performance, but the output paraphrase still lacks diversity. |
| Approach: | They propose a back-translation guided multi-round paraphrase generation framework which leverages multi- round paraphrases to improve diversity while preserving semantic information. |
| Outcome: | The proposed model improves diversity while preserving semantic information. |
Making Better Use of Bilingual Information for Cross-Lingual AMR Parsing (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing work on meaning representations for English and other languages finds that concepts in their predicted AMR graphs are less specific. |
| Approach: | They propose a cross-lingual AMR parser that can predict more precise concepts by translating translated texts and non-English texts. |
| Outcome: | The proposed model surpasses state-of-the-art parser by 10.6 points on Smatch F1 score. |
UICOMPASS: UI Map Guided Mobile Task Automation via Adaptive Action Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Mobile task automation is an emerging technology that leverages AI to automatically execute routine tasks by users’ commands on mobile devices like Android. |
| Approach: | They propose a UI Map-guided LLM-based approach to automate mobile tasks using static analysis and LLMs. |
| Outcome: | The proposed approach achieves a 15.87% higher task execution success rate than SOTA approaches even when only APK is available. |
Expressing Visual Relationships via Language (P19-1)
Copied to clipboard
| Challenge: | Current studies on image captioning focus on single image, but there are no effective models for generating relational captions for two images. |
| Approach: | They propose a language-guided image editing dataset that contains real image pairs with corresponding editing instructions. |
| Outcome: | The proposed model outperforms baseline and existing methods on two datasets. |
On the Helpfulness of Document Context to Sentence Simplification (2020.coling-main)
Copied to clipboard
| Challenge: | Text simplification is a hot issue in the field of natural language generation (NLG). |
| Approach: | They propose to use Wikipedia context to improve sentence simplification by using neural networks to learn the effects of preceding and following sentences on current sentences. |
| Outcome: | The proposed model outperforms the best performing model on the baseline dataset by 2.46 (7.22%). |
Improving Slot Filling in Spoken Language Understanding with Joint Pointer and Attention (P18-2)
Copied to clipboard
| Challenge: | Experimental results show the effectiveness of our slot filling model at addressing the OOV problem. |
| Approach: | They propose a generative neural network model for slot filling based on a sequence-to-sequence model and a pointer network. |
| Outcome: | The proposed model is able to predict slot values on spoken language data. |
A Probabilistic Inference Scaling Theory for LLM Self-Correction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated the capability to refine their generated answers through self-correction, enabling continuous performance improvement over multiple rounds. |
| Approach: | They propose a probabilistic theory to model the dynamics of accuracy change and explain performance improvements observed in multi-round self-correction. |
| Outcome: | The proposed model can predict accuracy curves and improve accuracy over multiple rounds. |
Scene Graph Modification Based on Natural Language Commands (2020.findings-emnlp)
Copied to clipboard
Xuanli He, Quan Hung Tran, Gholamreza Haffari, Walter Chang, Zhe Lin, Trung Bui, Franck Dernoncourt, Nhan Dam
| Challenge: | Numerous parsing methods have been developed for a single sentence, while a typical human-computer interaction session or conversation is not singleturn. |
| Approach: | They propose to modify an existing scene graph given a new user's command by using graph-based sparse transformer and cross attention information fusion to improve performance. |
| Outcome: | The proposed models outperform previous systems adapted from the machine translation and graph generation literature and contribute to the research community. |
Towards Document-Level Paraphrase Generation with Sentence Rewriting and Reordering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Document-level paraphrase generation is an important task in natural language processing. |
| Approach: | They propose a coherence relationship-guided paraphrase generation model that leverages graph GRU to encode the coherency relationship graph and get the cohesion-aware representation for each sentence. |
| Outcome: | The proposed model outperforms baseline models on BERTScore and diversity scores. |
Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs (2025.findings-emnlp)
Copied to clipboard
Yu Yan, Sheng Sun, Zhe Wang, Yijun Lin, Zenghao Duan, Zhifei Zheng, Min Liu, Zhiyi Yin, Jianping Zhang
| Challenge: | Large Language Models have been developed to deal with real-world crimes, but it remains unclear whether they internalize authentic knowledge or are forced to simulate toxic language patterns. |
| Approach: | They construct knowledge-intensive Q&A to investigate misuse threats of Large Language Models in terms of dangerous knowledge possession, harmful task planning utility, and harmfulness judgment robustness. |
| Outcome: | The findings raise concerns that jailbreak success is often attributable to a hallucination loop between jailbroken LLM and judger LLM . |