Papers by Fan-Yun Sun
Task Arithmetic can Mitigate Synthetic-to-Real Gap in Automatic Speech Recognition (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for speech recognition suffer from the synthetic-to-real gap . existing methods suffer from this distributional shift due to acoustic mismatches . |
| Approach: | They propose to use task arithmetic to fine-tune an ASR model on synthetic data to mitigate the synthetic-to-real gap. |
| Outcome: | The proposed method shows an improvement of 10.03% over baselines on the SLURP dataset. |
Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Large Vision-Language Models (VLMs) tend to neglect image content and over-rely on language-model priors, resulting in errors in visually grounded tasks and hallucinations. |
| Approach: | They propose a novel finetuning objective that steers the model toward capturing important visual details and aligning them with corresponding text tokens. |
| Outcome: | The proposed method achieves up to 22% reduction in hallucinations and significant gains in vision-centric and general tasks while maintaining or improving the model's general abilities. |