Papers by Brandon Denis
GPT Deciphering Fedspeak: Quantifying Dissent Among Hawks and Doves (2023.findings-emnlp)
Copied to clipboard
| Challenge: | We use GPT-4 to quantify dissent among members on the topic of inflation . transcripts and minutes reflect the diversity of member views in a way that is lost or omitted from the public statements. |
| Approach: | They use transcripts and minutes to quantify dissent among FOMC members . they find that transcripts reflect diversity of member views in a way that is lost or omitted . |
| Outcome: | The proposed method better captures extremes, which mirror human annotations, and suggests that Large Language Models can avoid noise in this nuanced context. |
Credible without Credit: Domain Experts Assess Generative Language Models (2023.acl-short)
Copied to clipboard
| Challenge: | ChatGPT has been criticized for its lack of accuracy and coherence . authors argue that language models could replace search engines and make college essays obsolete . |
| Approach: | a team of 10 domain experts conducts an initial assessment of language models using 100 expert-written questions. |
| Outcome: | The results show that language models are mixed in their accuracy. |
Ask Language Model to Clean Your Noisy Translation Data (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation models exhibit a noticeable decline in translation quality when exposed to noisy input. |
| Approach: | They use a dataset to evaluate the robustness of NMT models against noisy inputs. |
| Outcome: | The proposed dataset cleaners the noise from the target sentences while preserving the semantic integrity of the original sentences. |
More Victories, Less Cooperation: Assessing Cicero’s Diplomacy Play (2024.acl-long)
Copied to clipboard
Wichayaporn Wongkamjan, Feng Gu, Yanze Wang, Ulf Hermjakob, Jonathan May, Brandon Stewart, Jonathan Kummerfeld, Denis Peskoff, Jordan Boyd-Graber
| Challenge: | Diplomacy is a boardgame that offers a challenge for communicative and cooperative AI. |
| Approach: | They run two dozen games with Cicero and annotate in-game communication with abstract meaning representation to separate in- game tactics from general language. |
| Outcome: | The proposed method can outperform Cicero in communicating with humans, but it's difficult to deceive and persuade AI. |