Papers by Ed Chi
CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial text generation approaches can lead to generation lacking diversity or fluency, whereas perturbing in the intermediate representation space can lead a model to generate generations that are not related to the input. |
| Approach: | They propose to generate adversarial texts through controllable attributes that are known to be invariant to task labels. |
| Outcome: | The proposed model generates more diverse and fluent adversarial examples, compared to existing approaches, and is more robust against model re-training and different model architectures. |
Can We Improve Model Robustness through Secondary Attribute Counterfactuals? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has explored how models rely on spurious correlations and how counterfactual data augmentation (CDA) can mitigate such issues. |
| Approach: | They propose a context-aware methodology which takes into account the impact of secondary attributes on the model’s predictions and increases sensitivity for secondary attributes over reweighted counterfactually augmented data. |
| Outcome: | The proposed approach improves sliced accuracy on the original dataset by 7% compared to existing methods and provides guidelines to extend this to other tasks. |
Aligning Large Language Models with Recommendation Knowledge (2024.findings-naacl)
Copied to clipboard
Yuwei Cao, Nikhil Mehta, Xinyang Yi, Raghunandan Hulikal Keshavan, Lukasz Heldt, Lichan Hong, Ed Chi, Maheswaran Sathiamoorthy
| Challenge: | Large language models (LLMs) excel at natural language reasoning, but cannot model complex user-item interactions inherent in recommendation tasks. |
| Approach: | They propose to equip large language models with recommendation-specific knowledge to address this gap by combining Masked Item Modeling and Bayesian Personalized Ranking (BPR) auxiliary task data samples are generated that encode item correlations and user preferences. |
| Outcome: | Experiments on Amazon Toys & Games, Beauty, and Sports & Outdoors show that the proposed method outperforms conventional and LLM-based baselines by significant margins in retrieval. |
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them (2023.findings-acl)
Copied to clipboard
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, Jason Wei
| Challenge: | Language models have already made good progress on this benchmark, with the best model outperforming average reported human-rater results on 65% of the BIG-Bench tasks. |
| Approach: | They propose to use chain-of-thought prompting to challenge language models on 23 challenging BIG-Bench tasks which they call BIG-Bench Hard. |
| Outcome: | The proposed language models outperform the average human-rater on 65% of the BIG-Bench tasks. |
Improving Classifier Robustness through Active Generative Counterfactual Data Augmentation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for finding meaningful counterfactuals rely on human annotation or implicit label invariance . a small amount of human-annotated counterf actual data can generate a robust dataset with learned labels. |
| Approach: | They propose a framework that generates counterfactuals by actively sampling from regions of uncertainty and automatically labeling them with a learned auxiliary classifier. |
| Outcome: | The proposed framework generates a large number of diverse counterfactuals and labels them with a learned classifier. |