Papers by Yiheng Shu
Distribution Shifts Are Bottlenecks: Extensive Evaluation for Grounding Language Models to Knowledge Bases (2024.eacl-srw)
Copied to clipboard
| Challenge: | Existing benchmarks fail to reflect robustness challenges and fairly evaluate models. |
| Approach: | They propose to ground language models to knowledge bases to investigate distribution shifts in language and linguistic aspects of distribution shift. |
| Outcome: | The proposed method fails to evaluate language models in large and small datasets . the proposed model fails to cope with unseen schemas and language variations . |
Logical Form Generation via Multi-task Learning for Complex Question Answering over Knowledge Bases (2022.coling-1)
Copied to clipboard
| Challenge: | Existing generation-based KBQA methods that translate natural language questions to executable logical forms are proving promising but noise introduced can lead to incorrect results. |
| Approach: | They propose a Generation-based KBQA method that uses auxiliary information to enhance logical form generation by combining unseen KB items with novel combinations. |
| Outcome: | The proposed method achieves state-of-the-art results on ComplexWebQuestions and WebQuestIONSSP datasets. |
TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Base (2022.emnlp-main)
Copied to clipboard
| Challenge: | KBQA is a challenging area for pre-trained language models due to its extensive space and complexity. |
| Approach: | They propose a model that uses multi-grained retrieval to focus on most relevant KB contexts . constrained decoding is used to control output space and reduce generation errors . |
| Outcome: | The proposed model outperforms existing models on GrailQA and WebQuestionsSP. |
Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are generalist agents capable of operating within complex environments. |
| Approach: | They propose a class of tools that can serve as a middleware layer shielding LLMs from environmental complexity. |
| Outcome: | The proposed tool can shield the LLM from environmental complexity in two representative complex environments. |