Papers by Peiran Yao
Accurate and Nuanced Open-QA Evaluation Through Textual Entailment (2024.findings-acl)
Copied to clipboard
| Challenge: | Open-domain question answering (Open-QA) evaluations are criticized for the ambiguity in questions and the lack of semantic understanding in evaluators. |
| Approach: | They propose to examine the entailment relations of answers to identify more informative and more general system answers. |
| Outcome: | The proposed evaluations offer a much closer evaluation to human judgment on NaturalQuestions and TriviaQA while being learning-free. |
WordTies: Measuring Word Associations in Language Models via Constrained Sampling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Word associations are widely used in psychology to provide insights on how humans perceive and understand concepts. |
| Approach: | They propose an algorithm that allows an asymmetric measurement of associated words, given a cue word as input. |
| Outcome: | The proposed algorithm shares more overlap with human associations and observes the asymmetric property of human associations. |
NLP Workbench: Efficient and Extensible Integration of State-of-the-art Text Mining Tools (2023.eacl-demo)
Copied to clipboard
| Challenge: | NLP Workbench is a web-based text mining platform that allows non-expert users to obtain semantic understanding of large-scale corpora using state-of-the-art text mining models. |
| Approach: | They propose to use a microservice architecture to replace existing models or integrate a new one. |
| Outcome: | The proposed model is extensible and can be easily replaced or integrated with existing models. |
Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization (2025.emnlp-main)
Copied to clipboard
Shuo Xing, Peiran Li, Yuping Wang, Ruizheng Bai, Yueqi Wang, Chan-Wei Hu, Chengxuan Qian, Huaxiu Yao, Zhengzhong Tu
| Challenge: | emergence of large Vision Language Models (VLMs) has broadened the capabilities of single-modal Large Language Model (LLM) but VLMs are prone to significant hallucinations, especially in the form of cross-modal inconsistencies. |
| Approach: | They propose a new alignment framework that leverages image retrieval to integrate both textual and visual preference signals. |
| Outcome: | The proposed framework mitigates hallucinations more effectively than previous methods . it maintains robustness and scalability across a wide range of VLM sizes and architectures . |