Papers by Geng Zhao
Seen to Unseen: Exploring Compositional Generalization of Multi-Attribute Controllable Dialogue Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing controllable dialogue generation models focus on single attribute and lack generalization capability to out-of-distribution multiple attribute combinations. |
| Approach: | They propose a compositional generalization model that learns from seen attributes and generalizes to unseen combinations. |
| Outcome: | The proposed model can learn from seen attribute values and generalize to unseen combinations. |
Synergistic Interplay between Search and Large Language Models for Information Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | Information retrieval (IR) is an indispensable technique for locating relevant resources from vast amounts of data. |
| Approach: | They propose a framework that facilitates information refinement through synergy between RMs and LLMs. |
| Outcome: | The proposed framework improves the performance of large-scale retrieval benchmarks on web searches and low-resource retrieval tasks. |
Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language (2025.acl-long)
Copied to clipboard
Bo Zeng, Chenyang Lyu, Sinuo Liu, Mingyan Zeng, Minghao Wu, Xuanfan Ni, Tianqi Shi, Yu Zhao, Yefeng Liu, Chenyu Zhu, Ruizhe Li, Jiahui Geng, Qing Li, Yu Tong, Longyue Wang, Weihua Luo, Kaifu Zhang
| Challenge: | Existing datasets for instruction-following are monolingual and centered on English . existing data are unable to capture linguistic and cultural subtle differences . |
| Approach: | They propose an extension of IFEval to a localized multilingual version called Marco-Bench-MIF . their benchmark addresses linguistic constraints and cultural references via translation and verification . |
| Outcome: | The proposed extension of IFEval to a localized multilingual version covers 30 languages with varying levels of localization. |
Cross-lingual Social Misinformation Detector based on Hierarchical Mixture-of-Experts Adapter (2025.coling-main)
Copied to clipboard
| Challenge: | a global trend of misinformation is affecting non-native speaker users who are more susceptible to misinformation on foreign social media platforms. |
| Approach: | They propose a method to integrate sentiment analysis as an auxiliary task and a hierarchical routing strategy and expert-mask mechanism to enhance cross-lingual social misinformation detection. |
| Outcome: | The proposed method improves cross-lingual social misinformation detection in non-native speakers with only monolingual social media histories. |
FAA: Fine-grained Attention Alignment for Cascade Document Ranking (2023.acl-long)
Copied to clipboard
| Challenge: | Contemporary document ranking methods focus on transforming documents into passages to handle long inputs, but intensive query-irrelevant content may lead to harmful distraction and high query latency. |
| Approach: | They propose a fine-grained attention alignment approach to jointly optimize a cascade document ranking model. |
| Outcome: | Experiments on MS MARCO and TREC DL show that the proposed method is effective in document ranking tasks. |
Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language model (LLM)-based embedding models surpass BERT and T5 on general-purpose text embeddable tasks. |
| Approach: | They propose to adopt diffusion language models for text embeddings to overcome limitations in unidirectional attention used during autoregressive pre-training. |
| Outcome: | The proposed model outperforms the existing LLM-based embedding model on reasoning tasks by 20% and 2% on traditional embeddable benchmarks. |
Breaking the Attention Trap in Code LLMs: A Rejection Sampling Approach to Enhance Code Execution Prediction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research reveals significant performance bottlenecks in Code Execution tasks, which requires models to predict the execution results of given code snippets. |
| Approach: | They propose to use attention cracking with rejection sampling to eliminate attention traps in training data and to conduct secondary training on the outputs generated by the fine-tuned model to mitigate negative impacts from manual data intervention. |
| Outcome: | The proposed method eliminates the Attention Trap phenomenon in training data and conducts secondary training on the outputs generated by the fine-tuned model to mitigate negative impacts from manual data intervention. |
Great Memory, Shallow Reasoning: Limits of kNN-LMs (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing models trained on poor quality data have shown strong performance in language modeling and some downstream benchmarks. |
| Approach: | They evaluate kNN-LMs on a diverse set of tasks and evaluate their performance. |
| Outcome: | The proposed extension could improve on a variety of tasks, but it fails to perform on reasoning tasks that require integrating multiple pieces of information. |
Multilingual Federated Low-Rank Adaptation for Collaborative Content Anomaly Detection across Multilingual Social Media Participants (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent developments in multilingual social media platforms (SNS) exacerbate new challenges in SNS content anomaly detection due to data islands and linguistic imbalance. |
| Approach: | They propose a multilingual Federated LoRA based on SVD-based language-specific disentanglement of LoRA blocks and a local orthogonal tuning strategy to detect content anomalies. |
| Outcome: | The proposed solution is superior in multilingual content anomaly detection while reducing multilingual knowledge conflicts and communication rounds. |
CoreEval: Automatically Building Contamination-Resilient Datasets with Real-World Knowledge toward Reliable LLM Evaluation (2025.acl-long)
Copied to clipboard
| Challenge: | Publicly available datasets can be used to evaluate performance of large language models . however, contamination of test data can artificially inflate model performance . |
| Approach: | They propose a Contamination-resilient Evaluation strategy that updates data with real-world knowledge. |
| Outcome: | The proposed evaluation strategy can be used to update datasets with real-world knowledge. |