Papers by Yaqi Wang
SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis (2025.findings-naacl)
Copied to clipboard
Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Wang Changxin, Zhifeng Gao, Hongshuai Wang, Li Yongge, Mujie Lin, Shuwen Yang, Jiankun Wang, Mingjun Xu, Jin Huang, Xi Fang, Jiaxi Zhuang, Yuqi Yin, Yaqi Li, Changhong Chen, Zheng Cheng, Zifeng Zhao, Linfeng Zhang, Guolin Ke
| Challenge: | Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis. |
| Approach: | They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis. |
| Outcome: | SciAssess evaluates 11 LLMs on multiple tasks across scientific fields. |
Morpheme Sense Disambiguation: A New Task Aiming for Understanding the Language at Character Level (2024.lrec-main)
Copied to clipboard
| Challenge: | Morphemes are a strong linguistic feature to capture lexical semantics, but lack of morpheme-informed resources and the expense of manual annotations hinder morphme-enhanced methods. |
| Approach: | They propose a task of Morpheme Sense Disambiguation with two subtasks in-text and in-word to generalize morpheme features on more tasks. |
| Outcome: | The proposed tasks are based on two morpheme-annotated datasets for Chinese . the best model yields a promising precision of 77.66% on in-text and 88.19% on in word . |
Long-Horizon Dialogue Understanding for Role Identification in the Game of Avalon with Large Language Models (2023.findings-emnlp)
Copied to clipboard
Simon Stepputtis, Joseph Campbell, Yaqi Xie, Zhengyang Qi, Wenxin Zhang, Ruiyi Wang, Sanketh Rangreji, Charles Lewis, Katia Sycara
| Challenge: | Deception and persuasion play a critical role in long-horizon multi-party dialogues, especially when the interests, goals, and motivations of the participants are not aligned. |
| Approach: | They propose a game in which players must determine each other’s hidden identities to complete their team’s objective. |
| Outcome: | The proposed model can be used to determine the true player identities of six human players in a cooperative-competitive game. |
SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMs (2025.emnlp-main)
Copied to clipboard
Yuanyang Yin, Yaqi Zhao, Yajie Zhang, Yuanxing Zhang, Ke Lin, Jiahao Wang, Xin Tao, Pengfei Wan, Wentao Zhang, Feng Zhao
| Challenge: | Multimodal Large Language Models (MLLMs) integrate visual and textual inputs, yet modality alignment remains one of the most challenging aspects. |
| Approach: | They propose a token-level supervision alignment method that enables more precise visual-text alignment during pretraining. |
| Outcome: | The proposed method improves performance across various model sizes, with smaller models benefiting the most. |
Chinese Morpheme-informed Evaluation of Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing evaluations of large language models focused on the perspective of various tasks or abilities. |
| Approach: | They propose to evaluate large language models from a linguistic perspective and use morpheme to measure morphology and syntax. |
| Outcome: | The proposed model outperforms ChatGPT in Chinese scenarios with a morpheme-informed benchmark and human exam questions. |
Reusing Transferable Weight Increments for Low-resource Style Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Text style transfer (TST) is crucial in natural language processing, aiming to endow text with a new style without altering its meaning. |
| Approach: | They propose a framework to use style features in weight increments to transfer low-resource styles effectively. |
| Outcome: | The proposed framework achieves remarkable performance across different backbones, achieving particularly effective results in low-resource scenarios. |
InstructPart: Task-Oriented Part Segmentation with Instruction Reasoning (2025.acl-long)
Copied to clipboard
Zifu Wan, Yaqi Xie, Ce Zhang, Zhiqiu Lin, Zihan Wang, Simon Stepputtis, Deva Ramanan, Katia P. Sycara
| Challenge: | Large multimodal foundation models perceive objects as indivisible, overlooking the components that constitute them. |
| Approach: | They propose a novel benchmark for large multimodal foundation models comprising hand-labeled part segmentation annotations and task-oriented instructions to evaluate their performance. |
| Outcome: | The proposed benchmark improves performance of current models in understanding and executing part-level tasks within everyday contexts. |