Papers by Qianchu Liu
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | XCOPA dataset provides a typologically diverse dataset for commonsense reasoning in 11 languages . current methods for evaluating commonsensible reasoning in resource-poor languages are weak compared to translation-based transfer. |
| Approach: | They propose a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages. |
| Outcome: | The proposed model performs better than current methods on a resource-poor dataset compared to translation-based transfer in the 11 languages studied . |
Towards Better Context-aware Lexical Semantics:Adjusting Contextualized Representations through Static Anchors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has shown that contextualized models generate dynamic embeddings for words in context, but static embedds are often overlooked in this trend towards contextualized modeling. |
| Approach: | They propose a method that learns a transformation through static anchors and requires only another pre-trained model. |
| Outcome: | The proposed method improves a range of benchmark tasks that test contextual variations of meaning across different usages of a word and across different words as they are used in context. |
AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual evaluation datasets that evaluate lexical semantics "in-context" have various limitations, including limited coverage of high-resource languages and superficial cues. |
| Approach: | They propose to use a set of pretrained language models to evaluate lexical semantics in context. |
| Outcome: | The proposed set shows that current models lag behind human performance in interpreting word meaning in cross-lingual contexts. |
Measuring Context-Word Biases in Lexical Semantic Datasets (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing pretrained contextualized models have been used to evaluate word-in-context representations in many lexical semantic tasks. |
| Approach: | They propose to quantify the degree of context or word biases in existing datasets by probing masked input. |
| Outcome: | The proposed model performs better when both word and context are available than with masked input. |
TN-Eval: Rubric and Evaluation Protocols for Measuring the Quality of Behavioral Therapy Notes (2025.acl-industry)
Copied to clipboard
| Challenge: | Behavioral therapy notes are important for legal compliance and patient care, but quality standards for them remain underdeveloped. |
| Approach: | They propose a rubric for evaluating therapy notes across key dimensions: completeness, conciseness, faithfulness. |
| Outcome: | The proposed evaluation framework improves on therapist-written notes and LLM-generated notes. |
Exploring the Boundaries of GPT-4 in Radiology (2023.emnlp-main)
Copied to clipboard
Qianchu Liu, Stephanie Hyland, Shruthi Bannur, Kenza Bouzid, Daniel Castro, Maria Wetscherek, Robert Tinn, Harshita Sharma, Fernando Pérez-García, Anton Schwaighofer, Pranav Rajpurkar, Sameer Khanna, Hoifung Poon, Naoto Usuyama, Anja Thieme, Aditya Nori, Matthew Lungren, Ozan Oktay, Javier Alvarez-Valle
| Challenge: | Recent success of general-domain large language models has changed the natural language processing paradigm towards a unified foundation model across domains and applications. |
| Approach: | They evaluate the performance of GPT-4 on a variety of radiology tasks . they find it outperforms or matches current SOTA radiology models . |
| Outcome: | The proposed model outperforms or matches current SOTA radiology models on a range of tasks. |
NegPar: A parallel corpus annotated for negation (L18-1)
Copied to clipboard
| Challenge: | NegPar is the first parallel corpus annotated for negation in the narrative domain. |
| Approach: | They present NegPar, a parallel corpus annotated for negation in the narrative domain . they follow the annotation guidelines in the CONANDOYLE-NEG corpus . |
| Outcome: | The proposed corpus is based on the CONANDOYLE-NEG corpus and is reannotated to ensure more consistent and interpretable representations. |
Compositional Zero-Shot Domain Transfer with Text-to-Text Models (2023.tacl-1)
Copied to clipboard
Fangyu Liu, Qianchu Liu, Shruthi Bannur, Fernando Pérez-García, Naoto Usuyama, Sheng Zhang, Tristan Naumann, Aditya Nori, Hoifung Poon, Javier Alvarez-Valle, Ozan Oktay, Stephanie L. Hyland
| Challenge: | Existing approaches to zero-shot domain transfer are limited by domain gap and lack of in-domain labels. |
| Approach: | They propose a compositional transfer learning framework (DoT51) that learns domain knowledge and task knowledge in a multi-task manner without access to in-domain labels. |
| Outcome: | The proposed framework outperforms the current state-of-the-art in zero-shot domain transfer by over 7 absolute points in accuracy on RadNLI. |