Papers by Hanqi Li
When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved impressive performance across NLP tasks. |
| Approach: | They propose to use long-context SFT to improve short-contemporary performance . they also decouple and analyze two key components, Multi-Head Attention and Feed-Forward Network . |
| Outcome: | The proposed model improves short-context performance, contrary to pretraining. |
The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis (2024.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a capability that enables large language models to excel in proficiency through demonstration examples. |
| Approach: | They present a survey on the interpretation and analysis of in-context learning . they focus on theoretical and empirical perspectives on the concept . |
| Outcome: | The proposed model can perform tasks with minimal examples without re-training and has demonstrated proficiency across various tasks with a minimal set of task-oriented examples. |
Sparsity-Accelerated Training for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated proficiency across various NLP tasks but often require additional training, such as continual pre-training and supervised fine-tuning. |
| Approach: | They propose to leverage sparsity in pre-trained LLMs to accelerate training by disregarding computations for unimportant neurons. |
| Outcome: | The proposed framework achieves comparable or superior performance to standard training while significantly accelerating the process. |
AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitor (2026.findings-acl)
Copied to clipboard
| Challenge: | AutoMonitor-Bench evaluates the reliability of LLM-based misbehavior monitors across diverse tasks and failure modes. |
| Approach: | They introduce AutoMonitor-Bench, a benchmark designed to evaluate misbehavior monitors across diverse tasks and failure modes. |
| Outcome: | The new benchmark evaluates the reliability of LLM-based misbehavior monitors across tasks and failure modes. |
NeuSym-RAG: Hybrid Neural Symbolic Retrieval with Multiview Structuring for PDF Question Answering (2025.acl-long)
Copied to clipboard
Ruisheng Cao, Hanchong Zhang, Tiancheng Huang, Zhangyi Kang, Yuxin Zhang, Liangtai Sun, Hanqi Li, Yuxun Miao, Shuai Fan, Lu Chen, Kai Yu
| Challenge: | Existing approaches to retrieval augmented generation neglect PDF structure and layout . individual PDFs often exceed prompt limits and user queries may span multiple documents. |
| Approach: | They propose a hybrid neural symbolic retrieval framework which combines both paradigms in an interactive process. |
| Outcome: | The proposed framework organizes semi-structured PDF content into relational database and vectorstore . it defeats both RAG and structured baselines on three PDF-based QA datasets . |
Decoupled Dialogue Modeling and Semantic Parsing for Multi-Turn Text-to-SQL (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work on Text-to-SQL for multi-turn dialogue has attracted great interest . current approaches mostly employ end-to end models and face data sparsity problems . |
| Approach: | They propose a decoupled multi-turn text-to-SQL framework where dialogue context is explicitly solved by an utterance rewrite model and a single-turn Text-toSQl parser are proposed. |
| Outcome: | The proposed method outperforms existing models on SParC and CoSQL datasets without annotated in-domain data. |
Drift: Enhancing LLM Faithfulness in Rationale Generation via Dual-Reward Probabilistic Inference (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improving LLM faithfulness rely on superficial calibration methods or costly retraining. |
| Approach: | They propose a probabilistic inference paradigm that leverages task-specific and lookahead rewards to ensure that LLM-generated rationales are more faithful to model decisions. |
| Outcome: | The proposed model improves both accuracy and faithfulness of Large Language Models (LLMs) on three reasoning tasks. |
Multi-Granularity Fusion Text Semantic Matching Based on WoBERT (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing text-matching methods struggle with semantic nuances in short texts . a novel approach to improve text semantic matching is being developed . |
| Approach: | They propose a multi-granularity fusion model that harnesses a pre-trained language model to capture text semantic nuances. |
| Outcome: | The proposed model improves on Chinese short text matching datasets compared to traditional methods . the proposed model captures individual text semantic nuances and improves accuracy . |
Distinguishability Calibration to In-Context Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained language models generate similar output embeddings which makes it difficult to discriminate for the prompt-based classifier. |
| Approach: | They propose a calibration method which rotates the embedding feature into a new metric space and adapts the ratio of each dimension to a uniform distribution. |
| Outcome: | The proposed method improves the distinguishability of learning embeddings on three datasets under various settings. |
Can Large Language Models Act as Ensembler for Multi-GNNs? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing graph neural networks lack the inherent semantic understanding capability of rich textual attributes, limiting their effectiveness in applications. |
| Approach: | They propose a model that integrates multiple GNNs and LLMs to provide an ensemble for multi-GNNs. |
| Outcome: | The proposed model outperforms existing models in terms of semantic understanding of graph structures and graph structures. |