Papers by Denis Peskoff
Good Intentions Beyond ACL: Who Does NLP for Social Good, and Where? (2025.emnlp-main)
Copied to clipboard
| Challenge: | 20% of all papers in the ACL Anthology address social good issues . authors are more likely to do work addressing social good concerns when publishing in venues outside of ACL. |
| Approach: | They use author- and venue-level perspectives to map the landscape of NLP4SG . they find authors are more likely to do work addressing social good concerns outside of ACL . |
| Outcome: | The study analyzes the literature on NLP4SG and its impact on the ACL community . 20% of all papers in the anthology address social good issues, the study finds . |
GPT Deciphering Fedspeak: Quantifying Dissent Among Hawks and Doves (2023.findings-emnlp)
Copied to clipboard
| Challenge: | We use GPT-4 to quantify dissent among members on the topic of inflation . transcripts and minutes reflect the diversity of member views in a way that is lost or omitted from the public statements. |
| Approach: | They use transcripts and minutes to quantify dissent among FOMC members . they find that transcripts reflect diversity of member views in a way that is lost or omitted . |
| Outcome: | The proposed method better captures extremes, which mirror human annotations, and suggests that Large Language Models can avoid noise in this nuanced context. |
Credible without Credit: Domain Experts Assess Generative Language Models (2023.acl-short)
Copied to clipboard
| Challenge: | ChatGPT has been criticized for its lack of accuracy and coherence . authors argue that language models could replace search engines and make college essays obsolete . |
| Approach: | a team of 10 domain experts conducts an initial assessment of language models using 100 expert-written questions. |
| Outcome: | The results show that language models are mixed in their accuracy. |
Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL (2025.findings-acl)
Copied to clipboard
Wichayaporn Wongkamjan, Yanze Wang, Feng Gu, Denis Peskoff, Jonathan K. Kummerfeld, Jonathan May, Jordan Lee Boyd-Graber
| Challenge: | Future human-AI interaction tools can build on our methods for deception detection by triggering friction to give users a chance to interrogate suspicious proposals. |
| Approach: | They propose to use CTRL-D to detect deception in a board game called Diplomacy . CTRL is a counterfactual RL that has a good recall and almost perfect precision . future tools could build on this to reevaluate trust in suspicious negotiations . |
| Outcome: | The proposed method detects human deception with a high precision when compared to a Large Language Model approach that flags many true messages as deceptive. |
Personalized Help for Optimizing Low-Skilled Users’ Strategy (2025.naacl-short)
Copied to clipboard
Feng Gu, Wichayaporn Wongkamjan, Jordan Lee Boyd-Graber, Jonathan K. Kummerfeld, Denis Peskoff, Jonathan May
| Challenge: | a natural language agent generates moves and messages based on player intentions . a dozen games with novice and experienced players generate useful advice . |
| Approach: | a team of researchers augment a natural language agent to generate move and message advice . they use a game to simulate the intentions of novice and experienced players . |
| Outcome: | The enhanced agent generates move and message advice based on player intentions . the agent helps novices compete with experienced players and even surpass them . |
More Victories, Less Cooperation: Assessing Cicero’s Diplomacy Play (2024.acl-long)
Copied to clipboard
Wichayaporn Wongkamjan, Feng Gu, Yanze Wang, Ulf Hermjakob, Jonathan May, Brandon Stewart, Jonathan Kummerfeld, Denis Peskoff, Jordan Boyd-Graber
| Challenge: | Diplomacy is a boardgame that offers a challenge for communicative and cooperative AI. |
| Approach: | They run two dozen games with Cicero and annotate in-game communication with abstract meaning representation to separate in- game tactics from general language. |
| Outcome: | The proposed method can outperform Cicero in communicating with humans, but it's difficult to deceive and persuade AI. |