Papers by Xinxi Lyu
Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for zero-shot learning are based on in-context training, but performance drops when no demonstrations are available. |
| Approach: | They propose a new method that constructs pseudo-demonstrations for a given test input using a raw text corpus and applies techniques to reduce copying. |
| Outcome: | The proposed method outperforms previous zero-shot methods on nine classification datasets and is on par with in-context learning with labeled training data in the few-shot setting. |
Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts (2022.naacl-main)
Copied to clipboard
Daniel Khashabi, Xinxi Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sean Welleck, Hannaneh Hajishirzi, Tushar Khot, Ashish Sabharwal, Sameer Singh, Yejin Choi
| Challenge: | Recent work shows the surprising power of continuous prompts to language models for controlled generation and solving a wide range of tasks. |
| Approach: | They propose to extract a discrete (textual) interpretation of continuous prompts faithful to the problem they solve. |
| Outcome: | The proposed model can find prompts that solve a task while being projected to an arbitrary text with a smaller drop in accuracy. |
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (2023.emnlp-main)
Copied to clipboard
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, Hannaneh Hajishirzi
| Challenge: | Evaluating the factuality of long-form text generated by large language models (LMs) is non-trivial because (1) generations often contain a mixture of supported and unsupported pieces of information, making binary judgments of quality inadequate and (2) human evaluation is time-consuming and costly. |
| Approach: | They introduce a new evaluation that breaks a generation into a series of atomic facts and computes the percentage of atom facts supported by a reliable knowledge source. |
| Outcome: | The proposed model breaks a generation into atomic facts and computes the percentage of atomic fact supported by a reliable knowledge source. |
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (2022.emnlp-main)
Copied to clipboard
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, Luke Zettlemoyer
| Challenge: | Large language models can in-context learn by conditioning on a few input-label pairs and making predictions for new inputs. |
| Approach: | They propose to use ground truth demonstrations to replace labels in demonstrations . they also show that other aspects of the demonstrations are key drivers of endtask performance . |
| Outcome: | The proposed model outperforms zeroshot inference on a wide range of tasks using ground truth demonstrations. |