Papers by Dylan Zhang
Diversification Catalyzes Language Models’ Instruction Generalization To Unseen Semantics (2025.findings-acl)
Copied to clipboard
| Challenge: | Instruction-tuned language models excel in knowledge, reasoning, and instruction-following . however, the factors enabling generalization to unseen instructions remain underexplored . |
| Approach: | They propose to model instruction-following as a computational process and design controlled experiments inspired by the Turing-complete Markov algorithm to disentangle its dynamics. |
| Outcome: | The proposed model outperforms scaling up data volumes in generalist models by combining in-domain and diverse out-of-domain tasks. |
Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Influence-based methods show promise in achieving (1), but often struggle with (2) . data selection is often biased towards high-influence tasks, harming performance on them . |
| Approach: | They propose a Balanced and Influential Data Selection algorithm that normalizes influence scores of training data and iteratively chooses the training example with the highest influence on the most underrepresented task. |
| Outcome: | The proposed model outperforms both state-of-the-art influence-based methods and non-influence-based frameworks on seven benchmarks spanning five diverse capabilities. |
Visual Prompting in LLMs for Enhancing Emotion Recognition (2024.emnlp-main)
Copied to clipboard
Qixuan Zhang, Zhifeng Wang, Dylan Zhang, Wenjia Niu, Sabrina Caldwell, Tom Gedeon, Yang Liu, Zhenyue Qin
| Challenge: | Existing methods for enhancing in-context emotion classification fail to include spatial relationships between different people and facial features within a single face. |
| Approach: | They propose a set-of-vision prompting approach that uses spatial information to mark targets precisely. |
| Outcome: | The proposed approach improves face count and emotion categorization while preserving the enriched image context. |
ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting (2025.acl-long)
Copied to clipboard
Rui Pan, Dylan Zhang, Hanning Zhang, Xingyuan Pan, Minrui Xu, Jipeng Zhang, Renjie Pi, Xiaoyu Wang, Tong Zhang
| Challenge: | Existing paradigms for bilevel optimization require second-order information, making it difficult to scale them up. |
| Approach: | They propose a scalable instantiation of a bilevel optimization paradigm for large-scale LLMs by using a memory-efficient training technique. |
| Outcome: | The proposed paradigm scales to 30B-sized LLMs on 8H100 GPUs. |
Building A Proof-Oriented Programmer That Is 64% Better Than GPT-4o Under Data Scarcity (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing proof-oriented programming languages struggle due to data scarcity . existing proof-based models struggle due a lack of sufficient corpora for proof-orientated programming languages such as F* . |
| Approach: | They propose a method that synthesizes proof-oriented programming problems for proficiency in a language and incorporates diverse coding data for reasoning capability elicitation. |
| Outcome: | The proposed method outperforms existing proof-oriented models in function- and repository-level proof-based programming by 64% relative margin and improves GPT-4o's performance by 54% by repairing outputs over GPT-4) |
LMOD: A Large Multimodal Ophthalmology Dataset and Benchmark for Large Vision-Language Models (2025.findings-naacl)
Copied to clipboard
Zhenyue Qin, Yu Yin, Dylan Campbell, Xuansheng Wu, Ke Zou, Ninghao Liu, Yih Chung Tham, Xiuzhen Zhang, Qingyu Chen
| Challenge: | Existing benchmarks for large vision-language models (LVLMs) are limited to ophthalmology-specific applications. |
| Approach: | They introduce a large-scale multimodal ophthalmology benchmark consisting of 21,993 instances across five ocular imaging modalities and 13 state-of-the-art LVLM representatives from closed-source, open-source and medical domains. |
| Outcome: | The proposed model shows significant performance drop in ophthalmology compared to other domains. |