Papers by Haoran Xin
DialogQAE: N-to-N Question Answer Pair Extraction from Customer Service Chatlog (2023.findings-emnlp)
Copied to clipboard
Xin Zheng, Tianyu Liu, Haoran Meng, Xu Wang, Yufan Jiang, Mengliang Rao, Binghuai Lin, Yunbo Cao, Zhifang Sui
| Challenge: | Existing work on question-answer extraction fails to integrate incomplete utterances from dialog context for composite QA retrieval. |
| Approach: | They propose a task where questions and corresponding answers might be separated across different utterances. |
| Outcome: | The proposed methods perform well on 5 customer service datasets and set a benchmark for N-to-N DialogQAE with utterance and session level evaluation metrics. |
Temporal Scaling Law for Large Language Models (2025.emnlp-main)
Copied to clipboard
Yizhe Xiong, Xiansheng Chen, Xin Ye, Hui Chen, Zijia Lin, Haoran Lian, Zhenpeng Su, Wei Huang, Jianwei Niu, Jungong Han, Guiguang Ding
| Challenge: | Existing studies have found that the test loss of LLMs scales as power-laws with model size, computational budget, and dataset size. |
| Approach: | They propose a concept of Temporal Scaling Law to study test loss of LLMs . they break down test loss into fine-grained token positions and develop a dynamic hyperbolic-law . |
| Outcome: | The proposed model predicts the test loss of LLMs as the training steps scale up. |
On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directions (2025.findings-emnlp)
Copied to clipboard
Weiqi Wang, Tianqing Fang, Haochen Shi, Baixuan Xu, Wenxuan Ding, Liyu Zhang, Wei Fan, Jiaxin Bai, Haoran Li, Xin Liu, Yangqiu Song
| Challenge: | Conceptualization is a fundamental element of human cognition and plays a pivotal role in generalizable reasoning. |
| Approach: | They propose to categorize different types of conceptualizations into four levels based on the types of instances being conceptualized. |
| Outcome: | The proposed categorization of different types of conceptualizations into four levels based on the types of instances being conceptualized . |
A Table-to-Text Framework with Heterogeneous Multidominance Attention and Self-Evaluated Multi-Pass Deliberation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Table-to-text works have been widely applied in different domains, such as weather forecast and financial report generation. |
| Approach: | They propose a table-to-text approach on top of Self-evaluated multi-pass Generation and Heterogenous Multidominance Attention to explore the hierarchical structure. |
| Outcome: | The proposed method outperforms several SOTA methods quantitatively and qualitatively on three public datasets. |
Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have raised concerns regarding their intrinsic values. |
| Approach: | They propose a psychologically grounded five-factor value system for Large Language Models that integrates psychological principles with cutting-edge AI priorities. |
| Outcome: | The proposed value system meets standard psychological criteria, improves LLM safety prediction, and enhances Llm alignment, when compared to the canonical Schwartz’s values. |
DialogUSR: Complex Dialogue Utterance Splitting and Reformulation for Multiple Intent Detection (2022.findings-emnlp)
Copied to clipboard
Haoran Meng, Zheng Xin, Tianyu Liu, Zizhen Wang, He Feng, Binghuai Lin, Xuemin Zhao, Yunbo Cao, Zhifang Sui
| Challenge: | DialogUSR is a plug-in and domain-agnostic module that empowers multi-intent detection for chatbots . a single user query triggers inquiries on highspeed train ticket price and weather of destination. |
| Approach: | They propose a dialog utterance splitting and reformulation task that splits multi-intent user query into multiple single-intention sub-queries and recovers all coreferred and omitted information in the sub-questions. |
| Outcome: | The proposed model can be used to split multi-intent user queries into multiple sub-queries . it can be trained in two stages and perform in-depth analyses on the proposed models . |
NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding (2024.findings-emnlp)
Copied to clipboard
Chunkit Chan, Cheng Jiayang, Yauwai Yim, Zheye Deng, Wei Fan, Haoran Li, Xin Liu, Hongming Zhang, Weiqi Wang, Yangqiu Song
| Challenge: | Theory of mind evaluations currently focus on testing models using machine-generated data or game settings prone to shortcuts and spurious correlations. |
| Approach: | They propose a benchmark to stress-test machine ToM in real-world negotiation surrounding covered multi-dimensional mental states. |
| Outcome: | The proposed benchmark builds upon the Belief-Desire-Intention theory and conducts the necessary empirical experiments to evaluate large language models. |
Multimodal Language Models See Better When They Look Shallower (2025.emnlp-main)
Copied to clipboard
Haoran Chen, Junyan Lin, Xinghao Chen, Yue Fan, Jianfeng Dong, Xin Jin, Hui Su, Jinlan Fu, Xiaoyu Shen
| Challenge: | Existing studies show that multimodal large language models extract visual features from the final layers of a pretrained Vision Transformer. |
| Approach: | They propose a feature fusion method that strategically incorporates shallower layers . they propose MLLMs that extract visual features from the final layers of a pretrained Vision Transformer . |
| Outcome: | The proposed method outperforms deep layers on fine-grained visual tasks . it is the first comprehensive study of visual layer selection for MLLMs . |
InteGround: On the Evaluation of Verification and Retrieval Planning in Integrative Grounding (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing grounding approaches work well for simple queries, but many real-world information needs require synthesizing multiple pieces of evidence. |
| Approach: | They introduce "integrative grounding" to evaluate the ability to ground large language models in external knowledge sources. |
| Outcome: | The proposed approach is robust to redundant evidence, but rationalizes using internal knowledge when information is incomplete. |
ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are transforming diverse fields and gaining increasing influence as human proxies. |
| Approach: | They propose a psychometric evaluation pipeline grounded in realistic human-AI interactions to probe value orientations and novel tasks for evaluating value understanding in an open-ended value space. |
| Outcome: | The proposed evaluation pipeline is grounded in realistic human-AI interactions and performs tasks that approximate expert conclusions in value-related extraction and generation tasks. |
A Data-Centric Framework for Composable NLP Workflows (2020.emnlp-demos)
Copied to clipboard
Zhengzhong Liu, Guanxiong Ding, Avinash Bukkittu, Mansi Gupta, Pengzhi Gao, Atif Ahmed, Shikun Zhang, Xin Gao, Swapnil Singhavi, Linwei Li, Wei Wei, Zecong Hu, Haoran Shi, Xiaodan Liang, Teruko Mitamura, Eric Xing, Zhiting Hu
| Challenge: | Empirical natural language processing (NLP) systems involve interoperation among multiple components . a wealth of NLP toolkits exist ( 4), such as spaCy, DKPro, CoreNLP. |
| Approach: | They propose a unified open-source framework that supports fast development of NLP workflows . framework includes processors for NLP tasks, visualization, and annotation . |
| Outcome: | The framework offers processors for NLP tasks, visualization, and annotation, and is extensible . it is delivered through two modularized yet integratable open-source projects, Forte and Stave . |
PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models (2024.acl-long)
Copied to clipboard
Haoran Li, Dadi Guo, Donghao Li, Wei Fan, Qi Hu, Xin Liu, Chunkit Chan, Duanyi Yao, Yuan Yao, Yangqiu Song
| Challenge: | generative large language models (LLMs) exhibit surprising capability and integrate previous tasks into a unified text generation formulation. |
| Approach: | They propose a privacy evaluation benchmark to quantify the privacy leakage of language models. |
| Outcome: | The proposed benchmark compares PPLMs with different privacy implementations to find out how privacy leakage is handled. |
LexGenius: An Expert-Level Benchmark for Large Language Models in Legal General Intelligence (2026.findings-acl)
Copied to clipboard
Wenjin Liu, Haoran Luo, Xin Feng, Xiang Ji, Lijuan Zhou, Rui Mao, Jiapu Wang, Shirui Pan, Erik Cambria
| Challenge: | Existing benchmarks for legal general intelligence (GI) are result-oriented and do not evaluate the legal intelligence of large language models (LLMs). |
| Approach: | They propose a Chinese legal benchmark for evaluating legal GI in large language models . they use recent legal cases and exam questions to create multiple-choice questions . |
| Outcome: | The proposed benchmarks lack a systematic evaluation of the legal intelligence of large language models (LLMs) the results show that even the best LLMs lagging behind human legal professionals. |
DialogVCS: Robust Natural Language Understanding in Dialogue System Upgrade (2024.naacl-long)
Copied to clipboard
Zefan Cai, Xin Zheng, Tianyu Liu, Haoran Meng, Jiaqi Han, Gang Yuan, Binghuai Lin, Baobao Chang, Yunbo Cao
| Challenge: | Existing models for natural language understanding are based on a well-defined intent 1 ontology. |
| Approach: | They propose to retrain the natural language understanding model as new data from real users are merged into existing data. |
| Outcome: | The proposed model shows that the semantically entangled intents can be recognized with an automatic workflow. |