Papers by Jingfan Chen
MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Low-rank adaptation and its mixture-of-experts (MOE) methods are highly effective but introduce significant latency in multi-tenant settings due to the LoRA modules and MOE routers added to multiple linear modules. |
| Approach: | They propose a low-rank adaptation variant that considers each LoRA module as an expert and employs a prompt-aware routing mechanism. |
| Outcome: | Extensive analysis on commonsense reasoning tasks and math reasoning tasks show that MiLoRA outperforms strong PEFT baselines with comparable tunable parameter budgets. |
Activation Steering Decoding: Mitigating Hallucination in Large Vision-Language Models through Bidirectional Hidden State Intervention (2025.acl-long)
Copied to clipboard
| Challenge: | Large Vision Language Models (LVLMs) suffer from hallucination where generated textual descriptions fail to align accurately with visual semantics. |
| Approach: | They propose a training-free approach that mitigates hallucination through targeted intervention in the model’s intermediate activations by identifying directional patterns of hallucinism in the activation space using a small calibration set. |
| Outcome: | The proposed approach reduces hallucination across multiple benchmarks while maintaining performance on general visual understanding tasks. |
AutoGUI: Scaling GUI Grounding with Automatic Functionality Annotations from LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing datasets for UI-VLMs contain large-scale context-free element annotations or contextualized functional descriptions for elements at a small scale. |
| Approach: | They propose an auto-annotation pipeline that generates massive UI element functionality annotations based on UI content changes induced by interacting with the elements. |
| Outcome: | The proposed pipeline improves accuracy and scales well with human evaluation of a high-quality AutoGUI-704k dataset. |