Papers by Jonathan Kummerfeld
Empathy Identification Systems are not Accurately Accounting for Context (2023.eacl-main)
Copied to clipboard
| Challenge: | Empathy is a fundamental phenomenon that allows us to better communicate and relate with others. |
| Approach: | They propose a simple model that checks if an input utterance is similar to a small set of empathetic examples, but does not consider dialogue context. |
| Outcome: | The proposed model outperforms state-of-the-art models on benchmarks and empathetic rationale extraction benchmarks. |
Do Text-to-Vis Benchmarks Test Real Use of Visualisations? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for visualisations are limited and do not reflect real-world use. |
| Approach: | They analysed visualisation code from Python, R, Javascript, and Vega to find similarities and differences between real-world and benchmark datasets. |
| Outcome: | The results show that benchmark datasets do not test the same distribution of chart types, attributes, and actions as real-world examples. |
Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL (2025.findings-acl)
Copied to clipboard
Wichayaporn Wongkamjan, Yanze Wang, Feng Gu, Denis Peskoff, Jonathan K. Kummerfeld, Jonathan May, Jordan Lee Boyd-Graber
| Challenge: | Future human-AI interaction tools can build on our methods for deception detection by triggering friction to give users a chance to interrogate suspicious proposals. |
| Approach: | They propose to use CTRL-D to detect deception in a board game called Diplomacy . CTRL is a counterfactual RL that has a good recall and almost perfect precision . future tools could build on this to reevaluate trust in suspicious negotiations . |
| Outcome: | The proposed method detects human deception with a high precision when compared to a Large Language Model approach that flags many true messages as deceptive. |
A Comparative Multidimensional Analysis of Empathetic Systems (2024.eacl-long)
Copied to clipboard
| Challenge: | Empathetic dialogue systems have received significant attention, but no systematic review has verified these limitations. |
| Approach: | They analyze 21 empathetic dialogue systems using automated methods to examine their progress. |
| Outcome: | The results show that empathetic dialogue systems lack specificity, reflection levels, diversity . the results also offer guidance for developing future systems . |
Personalized Help for Optimizing Low-Skilled Users’ Strategy (2025.naacl-short)
Copied to clipboard
Feng Gu, Wichayaporn Wongkamjan, Jordan Lee Boyd-Graber, Jonathan K. Kummerfeld, Denis Peskoff, Jonathan May
| Challenge: | a natural language agent generates moves and messages based on player intentions . a dozen games with novice and experienced players generate useful advice . |
| Approach: | a team of researchers augment a natural language agent to generate move and message advice . they use a game to simulate the intentions of novice and experienced players . |
| Outcome: | The enhanced agent generates move and message advice based on player intentions . the agent helps novices compete with experienced players and even surpass them . |
More Victories, Less Cooperation: Assessing Cicero’s Diplomacy Play (2024.acl-long)
Copied to clipboard
Wichayaporn Wongkamjan, Feng Gu, Yanze Wang, Ulf Hermjakob, Jonathan May, Brandon Stewart, Jonathan Kummerfeld, Denis Peskoff, Jordan Boyd-Graber
| Challenge: | Diplomacy is a boardgame that offers a challenge for communicative and cooperative AI. |
| Approach: | They run two dozen games with Cicero and annotate in-game communication with abstract meaning representation to separate in- game tactics from general language. |
| Outcome: | The proposed method can outperform Cicero in communicating with humans, but it's difficult to deceive and persuade AI. |