Papers by Rik Koncel-Kedziorski
Knowledge Transfer from Answer Ranking to Answer Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that Question Answering (QA) based on Answer Sentence Selection (AS2) can be improved by generating an improved answer from the top-k ranked answer sentences. |
| Approach: | They propose to train a GenQA model by transferring knowledge from a trained AS2 model . they use top ranked candidate as the generation target and next k top rated candidates as context . |
| Outcome: | The proposed model outperforms existing models on public and industrial datasets. |
BizBench: A Quantitative Reasoning Benchmark for Business and Finance (2024.acl-long)
Copied to clipboard
| Challenge: | Answering questions within business and finance requires reasoning, precision, and a wide-breadth of technical knowledge. |
| Approach: | They propose a benchmark for evaluating models’ ability to reason about realistic financial problems by focusing on question-answering over financial data via program synthesis. |
| Outcome: | The proposed benchmark evaluates models' financial background knowledge, ability to parse financial documents, and capacity to solve complex problems with code. |
MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms (N19-1)
Copied to clipboard
| Challenge: | Existing datasets in this domain do not offer precise operational annotations over diverse problem types due to noise and lack of formal operation-based representations. |
| Approach: | They propose a representation language to map problems to their operation programs . they also introduce an interpretable neural math problem solver . |
| Outcome: | The proposed model outperforms baseline models and the AQUA-RAT dataset on the AQuA-rat dataset. |
DocFinQA: A Long-Context Financial Reasoning Dataset (2024.acl-short)
Copied to clipboard
| Challenge: | Existing work on automating financial numerical reasoning focuses on unrealistically specific document snippets, failing to reflect the broader and more realistic scenarios faced by analysts. |
| Approach: | They propose a long-document financial QA task that augments 7,437 questions from existing FinQA dataset with full-document context, extending the average context length from under 700 words in FinQA to 123k words in DocFinQA. |
| Outcome: | The proposed task extends the average context length from under 700 words in FinQA to 123k words in DocFinQA. |
Cross-Lingual Open-Domain Question Answering with Answer Sentence Generation (2022.aacl-main)
Copied to clipboard
| Challenge: | Open-Domain Generative Question Answering has achieved impressive performance in English . combining document-level retrieval with answer generation can generate complete sentences . |
| Approach: | They propose an open-domain approach that combines document retrieval with answer generation to generate complete sentences in English . they propose a cross-lingual generative model that exploits passages written in multiple languages . |
| Outcome: | The proposed model outperforms answer sentence selection baselines for all 5 languages and monolingual pipelines for three out of five languages. |
Text Generation from Knowledge Graphs with Graph Transformers (N19-1)
Copied to clipboard
| Challenge: | Existing methods for generating text with structured inputs are expensive and require manual annotation. |
| Approach: | They propose a graph transforming encoder which leverages relational structure of knowledge graphs without imposing linearization or hierarchical constraints. |
| Outcome: | The proposed system produces more informative texts than competing methods. |
Language Model Probabilities are Not Calibrated in Numeric Contexts (2025.acl-long)
Copied to clipboard
Charles Lovering, Michael Krumdick, Viet Dac Lai, Varshini Reddy, Seth Ebner, Nilesh Kumar, Rik Koncel-Kedziorski, Chris Tanner
| Challenge: | Using language model outputs, we find that even in simple settings, the best LMs (1) are poorly calibrated and (2) have systematic biases. |
| Approach: | They argue that language model outputs should capture natural distributions over multiple options within their textual contexts. |
| Outcome: | The proposed model outputs are calibrated to the numeric content of their contexts. |
Learning Answer Generation using Supervision from Automatic Question Answering Evaluators (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies show sentence-level extractive QA is outperformed by Generation-based QA (GenQA) models. |
| Approach: | They propose a training paradigm for GenQA using automatic QA evaluation models . they augment training data with answers generated by the GenQA model and labelled by GAVA . |
| Outcome: | The proposed training paradigm improves answering accuracy over existing models. |
PrimeX: A Dataset of Worldview, Opinion, and Explanation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that an individual's worldview -or beliefs about the overall character of the world -can explain persistent behavioral patterns and correlates with personality, well-being, political, religious, and demographic variables. |
| Approach: | They develop a dataset of public opinion survey data from 858 US residents with written explanations from the respondents for why they hold specific opinions and the Primal World Belief survey for assessing respondent worldview. |
| Outcome: | The proposed model can be used to better represent an individual's belief system and improve opinion prediction. |
Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text (2022.acl-long)
Copied to clipboard
| Challenge: | a recent study has reported that crowdsourcing cannot distinguish between machine-authored and human-authored text. |
| Approach: | They propose a framework called Scarecrow for scrutinizing machine text via crowd annotation . they use crowd annotation to identify redundancy, commonsense errors, and incoherence . |
| Outcome: | The proposed method quantifies gaps between human-authored and machine-generated text . it can detect redundancy, commonsense errors, and incoherence . |
Improving Language Model Personas via Rationalization with Psychological Scaffolds (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to building personas rely on a user’s demographic attributes and/or prior judgments, but not on any underlying reasoning behind a person’s judgments. |
| Approach: | They propose a framework that integrates rationales for why a user could have made a certain judgment into LM personas by incorporating potential rationale. |
| Outcome: | The proposed framework outperforms models conditioned on demographic attributes and/or prior judgments on public opinion and movie preference prediction tasks. |
Explaining Relationships Between Scientific Documents (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to explain relationships between scientific documents using natural language text can be useful for research efficiency. |
| Approach: | They propose a task of explaining relationships between scientific documents using natural language text. |
| Outcome: | The proposed models can be automated and humanely evaluated. |
Pyramidal Recurrent Unit for Language Modeling (D18-1)
Copied to clipboard
| Challenge: | Long short term memory units are powerful tools for language modeling, but their performance can be limited by the number of parameters. |
| Approach: | They propose a pyramidal recurrent unit which enables learning representations in high dimensional space with more generalization power and fewer parameters. |
| Outcome: | The proposed model outperforms existing models with different gating mechanisms and transformations on word-level language modeling tasks. |