Papers by Brandon Stewart
AutoPersuade: A Framework for Evaluating and Explaining Persuasive Arguments (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing tools for persuasion are well-equipped to identify which of a pre-existing set of messages is most persuasive, but they do not offer causal evidence on whether or how they have succeeded. |
| Approach: | They propose a framework for identifying topical components of persuasive arguments that are autopersuade. |
| Outcome: | The proposed framework validates the results through human studies and out-of-sample predictions. |
GPT Deciphering Fedspeak: Quantifying Dissent Among Hawks and Doves (2023.findings-emnlp)
Copied to clipboard
| Challenge: | We use GPT-4 to quantify dissent among members on the topic of inflation . transcripts and minutes reflect the diversity of member views in a way that is lost or omitted from the public statements. |
| Approach: | They use transcripts and minutes to quantify dissent among FOMC members . they find that transcripts reflect diversity of member views in a way that is lost or omitted . |
| Outcome: | The proposed method better captures extremes, which mirror human annotations, and suggests that Large Language Models can avoid noise in this nuanced context. |
A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors (P18-1)
Copied to clipboard
| Challenge: | Existing word2vec-based methods for learning rare or unseen words have been criticized for degrading performance in small corpus settings. |
| Approach: | They propose a la carte embedding method that relies on a linear transformation that is efficiently learnable using pretrained word vectors and linear regression. |
| Outcome: | The proposed method is based on a new dataset showing that it can be used when a word is encountered even if only a single usage example is available. |
Credible without Credit: Domain Experts Assess Generative Language Models (2023.acl-short)
Copied to clipboard
| Challenge: | ChatGPT has been criticized for its lack of accuracy and coherence . authors argue that language models could replace search engines and make college essays obsolete . |
| Approach: | a team of 10 domain experts conducts an initial assessment of language models using 100 expert-written questions. |
| Outcome: | The results show that language models are mixed in their accuracy. |
More Victories, Less Cooperation: Assessing Cicero’s Diplomacy Play (2024.acl-long)
Copied to clipboard
Wichayaporn Wongkamjan, Feng Gu, Yanze Wang, Ulf Hermjakob, Jonathan May, Brandon Stewart, Jonathan Kummerfeld, Denis Peskoff, Jordan Boyd-Graber
| Challenge: | Diplomacy is a boardgame that offers a challenge for communicative and cooperative AI. |
| Approach: | They run two dozen games with Cicero and annotate in-game communication with abstract meaning representation to separate in- game tactics from general language. |
| Outcome: | The proposed method can outperform Cicero in communicating with humans, but it's difficult to deceive and persuade AI. |