Papers by Weiran Lin
Estimating LLM Consistency: A User Baseline vs Surrogate Metrics (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are prone to hallucinations and sensitive to prompt perturbations, resulting in inconsistent or unreliable generated text. |
| Approach: | They propose a logit-based ensemble method to measure LLM consistency and propose to use it to evaluate human ratings of LLM reliability. |
| Outcome: | The proposed method matches the best-performing existing metric in estimating human ratings of LLM consistency. |
UEGP: Unified Expert-Guided Pre-training for Knowledge Rekindle (2024.findings-naacl)
Copied to clipboard
Yutao Mou, Kexiang Wang, Jianhe Lin, Dehong Ma, Jun Fan, Daiting Shi, Zhicong Cheng, Gu Simiu, Dawei Yin, Weiran Xu
| Challenge: | Existing paradigms for pre-training and fine-tuning have limitations . knowledge rekindle aims to break through performance upper bounds of experts without introducing additional annotated data. |
| Approach: | They propose a new paradigm for pre-training and fine-tuning that aims to re-incorporate the fine- tuned expert model into the training cycle and break through performance upper bounds of experts. |
| Outcome: | The proposed model breaks through performance upper bounds of experts without additional annotated data. |
SRAP-Agent: Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agent (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on the allocation of public scarce resources has limitations due to data scarcity and data scariness. |
| Approach: | They propose a framework that integrates Large Language Models into economic simulations . they conduct extensive policy simulation experiments to verify the framework's effectiveness . |
| Outcome: | The proposed framework bridges the gap between theoretical models and real-world dynamics by integrating large language models into economic simulations. |