Papers by Sébastien Jean
Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In task-oriented semantic parsing, the system aims to translate users’ utterances in natural language to machine-interpretable programs (API calls) However, Large Language Models (LLMs) are known to hallucinate and therefore pose a formidable challenge in constraining generated content. |
| Approach: | They propose to use large language models to translate user's utterances to machine-interpretable programs (API calls) they identify constraints violations in task-oriented utterrances and define fine-grained metrics that complement traditional ones. |
| Outcome: | The proposed methods reduce constraints violations and improve quality of the generated API calls, but require careful consideration given their implementation complexity and latency. |