Papers by Liangzhi Li
Dual-Feedback Knowledge Retrieval for Task-Oriented Dialogue Systems (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to task-oriented dialogue systems integrate knowledge retrieval and response generation, which poses scalability challenges when dealing with extensive knowledge bases. |
| Approach: | They propose a retriever-generator architecture that harnesses a retrieval and a generator to generate system responses by using feedback from the generator as pseudo-labels. |
| Outcome: | The proposed architecture shows superior performance on three benchmark datasets. |
MPrompt: Exploring Multi-level Prompt Tuning for Machine Reading Comprehension (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing soft prompt methods focus on designing the input-independent prompts that steer the model to fit the domain of the new dataset. |
| Approach: | They propose a multi-level prompt tuning method that utilizes prompts at task-specific, domain-specific and context-specific levels to enhance the comprehension of input semantics. |
| Outcome: | The proposed method improves on 12 benchmarks on various QA formats and achieves an average improvement of 1.94% over the state-of-the-art methods. |
MHGRL: An Effective Representation Learning Model for Electronic Health Records (2024.lrec-main)
Copied to clipboard
| Challenge: | Effective EHR representations are key to achieving high performance in healthcare applications. |
| Approach: | They propose a multimodal heterogeneous graph-enhanced representation learning to learn EHR representations using medical ontology and textual notes. |
| Outcome: | The proposed model outperforms baseline models on two real clinical datasets in downstream tasks. |
BlockPruner: Fine-grained Pruning for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have significant computational and memory costs associated with training and inference. |
| Approach: | They propose a training-free structured pruning approach that targets redundancies in MHA and MLP blocks. |
| Outcome: | The proposed pruning approach achieves more granular and effective pruning compared to state-of-the-art pruning methods. |
TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | ChatGPT and GPT-4 are commercial large language models (LLMs) however, they may produce vague responses or incorrect answers in certain specialized domains. |
| Approach: | They propose a token compression scheme that uses summarization and semantic compression to reduce the token size of LLMs. |
| Outcome: | The proposed method reduces token size by doing summarization and semantic compression while reducing token size with only 1.6% accuracy drop. |
Can Multiple-choice Questions Really Be Useful in Detecting the Abilities of LLMs? (2024.lrec-main)
Copied to clipboard
| Challenge: | Multiple-choice questions (MCQs) are widely used in the evaluation of large language models (LLMs) however, there are concerns about whether MCQ can truly measure LLM’s capabilities. |
| Approach: | They propose to use multiple choice questions to evaluate large language models (LLMs) to assess their capabilities. |
| Outcome: | The proposed methods show that MCQs are less reliable than LFGQs in terms of expected calibration error. |