Papers by Yangyifan Xu
LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Long-context modeling has drawn more attention in the area of Large Language Models (LLMs). |
| Approach: | They propose a Long-context data selection framework with Attention-based Dependency Measurement which can efficiently identify high-quality long-contrast data from a large-scale, multi-domain pre-training corpus. |
| Outcome: | The proposed framework significantly boosts the performance of LLMs on multiple long-context tasks with only 1B tokens for continual training. |
Collaborative Beam Search: Enhancing LLM Reasoning via Collective Consensus (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve the reasoning capabilities of large language models (LLMs) depend on domain-specific external verifiers or self-evaluation which is brittle and prompt-sensitive. |
| Approach: | They propose a framework that harnesses the collective intelligence of multiple large language models across both generation and verification stages. |
| Outcome: | The proposed framework outperforms singlemodel scaling and multi-model ensemble baselines on six tasks by over 4 percentage points in average accuracy. |
Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to token-level adaptive training only use static word frequency information without considering the source language. |
| Approach: | They propose a bilingual mutual information based adaptive objective that assigns weights to target tokens with higher BMI . they propose to use this approach to improve token-level adaptive training . |
| Outcome: | The proposed method improves token-level adaptive training on two languages. |
Hit the Sweet Spot! Span-Level Ensemble for Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | a recent study focused on sample-level and token-level ensembles, which hinder dynamic correction and enhancement of outputs during the generation process. |
| Approach: | They propose a span-level ensemble method that balances real-time adjustments and accurate ensemble decisions. |
| Outcome: | The proposed method improves performance across language generation tasks significantly. |
Bridging the Gap between Different Vocabularies for LLM Ensemble (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to combine large language models with complete outputs have limited effectiveness . lexical gaps between different LLMs hinder dynamic correction and enhancement . |
| Approach: | They propose a method to ensemble large language models via Vocabulary Alignment (EVA) they learn mappings between vocabularies of different LLMs with overlapping tokens . |
| Outcome: | The proposed method bridges the lexical gap among various LLMs, enabling meticulous ensemble at each generation step. |
Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive performance across numerous NLP tasks, but fine-tuning them for Machine Translation (MT) often introduces catastrophic forgetting, compromising the broad general abilities of LLMs and introducing potential security risks. |
| Approach: | They propose a method that harnesses the strong generative capabilities of Large Language Models to create rationales for training data, which are then "replayed" to prevent forgetting. |
| Outcome: | The proposed approach harnesses the strong generative capabilities of LLMs to create rationales for training data, which are then “replayed” to prevent forgetting. |