When Reasoning Meets Information Aggregation: A Case Study with Sports Narratives (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using sports data, an LLM can analyze sports narratives to infer points from actions, identify related entities, attribute points accurately to players and teams, and draw conclusions. |
| Approach: | They propose a method to synthesize NBA basketball game narratives using real NBA basketball data and propose 'SportsGen' they find that most models fail to accurately aggregate basketball scores due to frequent scoring patterns and open-source models suffer from significant score hallucinations. |
| Outcome: | The proposed method can evaluate LLMs’ reasoning capabilities under complex scenarios with varying narrative lengths and density of information. |
Similar Papers
SportsMetrics: Blending Text and Numerical Data to Understand Information Fusion in LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models can handle text and data, but blending text and numerical data presents significant challenges. |
| Approach: | They propose four tasks to evaluate the numerical reasoning and information fusion capabilities of large language models in sports data analytics. |
| Outcome: | The proposed tasks evaluate the numerical reasoning and information fusion capabilities of large language models in sports data analytics. |
Moneyball with LLMs: Analyzing Tabular Summarization in Sports Narratives (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language model (LLM) approaches to tabular summarization rely on prompt engineering, decomposition pipelines, or entity-level intermediate representations to achieve strong performance. |
| Approach: | They propose a diagnostic benchmark for long-context tabular summarization using decomposition pipelines and entity-level intermediate representations. |
| Outcome: | The proposed benchmark improves accuracy and numerical fidelity, but lacks local arithmetic. |
Logical Structure as Knowledge: Enhancing LLM Reasoning via Structured Logical Knowledge Density Estimation (2026.findings-acl)
Copied to clipboard
Zhen Bi, Zhenlin Hu, Xueshu Chen, Mingyang Chen, Cheng Deng, Yida Xue, Zhen Wang, Qing Shen, Ningyu Zhang, Jungang Lou
| Challenge: | Existing data-centric paradigms equate quality with factuality or diversity and ignore the internal logical complexity of training samples. |
| Approach: | They propose a density-aware re-cognizing optimization strategy that prioritizes high-density logical samples to align training with the model's reasoning boundary. |
| Outcome: | The proposed metric outperforms existing methods and improves reasoning performance without increasing total data volume. |
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models (2024.lrec-main)
Copied to clipboard
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Tianxiang Sun, Cheng Chang, Qinyuan Cheng, Ding Wang, Xiaofeng Mou, Xipeng Qiu, Xuanjing Huang
| Challenge: | Recent advances in Chain-of-Thought prompting have facilitated significant breakthroughs for Large Language Models (LLMs) in complex reasoning tasks. |
| Approach: | They propose a hierarchical reasoning aggregation framework to address this problem . they propose dynamic sampling to adjust the number of reasoning chains . |
| Outcome: | The proposed framework outperforms existing ensemble methods on complex reasoning tasks. |
BeamAggR: Beam Aggregation Reasoning over Multi-source Knowledge for Multi-hop Question Answering (2024.acl-long)
Copied to clipboard
Zheng Chu, Jingchang Chen, Qianglong Chen, Haotian Wang, Kun Zhu, Xiyuan Du, Weijiang Yu, Ming Liu, Bing Qin
| Challenge: | Large language models (LLMs) have demonstrated strong reasoning capabilities, but they still suffer from factual errors when tackling knowledge-intensive tasks. |
| Approach: | They propose a reasoning framework for knowledge-intensive multi-hop QA that prioritizes promising answers at each hop of question. |
| Outcome: | The proposed framework outperforms SOTA methods on four open-domain multi-hop reasoning datasets by 8.5%. |
Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verification (2025.findings-acl)
Copied to clipboard
John Dougrez-Lewis, Mahmud Elahi Akhter, Federico Ruggeri, Sebastian Löbbers, Yulan He, Maria Liakata
| Challenge: | Large Language Models (LLMs) have shown remarkable proficiency in complex tasks where reasoning capabilities are paramount. |
| Approach: | They propose a framework to break down claims into atomic reasoning types needed for verification. |
| Outcome: | The proposed framework breaks down claims into atomic reasoning types needed for verification. |
Group, Embed and Reason: A Hybrid LLM and Embedding Framework for Semantic Attribute Alignment (2025.emnlp-industry)
Copied to clipboard
Shramona Chakraborty, Shashank Mujumdar, Nitin Gupta, Sameep Mehta, Ronen Kat, Itay Etelis, Mohamed Mahameed, Itai Guez, Rachel Tzoref-Brill
| Challenge: | a framework to align attributes that refer to the same concept but differ across schemas is challenging in schema only settings where no instance data is available due to ambiguous names, inconsistent descriptions, and domain-specific terminologies. |
| Approach: | They propose a framework that combines contextual reasoning and embedding-based similarity to address token limitations and hallucinations. |
| Outcome: | The proposed framework scales to large schemas and shows strong performance on healthcare schemas. |
PASTA: A Dataset for Modeling PArticipant STAtes in Narratives (2023.tacl-1)
Copied to clipboard
Sayontan Ghosh, Mahnaz Koupaee, Isabella Chen, Francis Ferraro, Nathanael Chambers, Niranjan Balasubramanian
| Challenge: | Existing models that understand narratives should infer these implicit states and their causal relationships with the narrative's explicit events. |
| Approach: | They propose a dataset that contains inferable participant states, a counterfactual perturbation to each state and the changes to the story that would be necessary if the counterfact was true. |
| Outcome: | The proposed model can reason about the impact of changes to the story that would be necessary if the counterfactual were true. |
Reason from Fallacy: Enhancing Large Language Models’ Logical Reasoning through Logical Fallacy Understanding (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated good performance in many reasoning tasks, but struggle with some more complex reasoning tasks including logical reasoning. |
| Approach: | They propose five concrete tasks from three cognitive dimensions of WHAT, WHY, and HOW to evaluate LLMs’ capability of logical fallacy understanding. |
| Outcome: | The proposed dataset can be used to evaluate LLMs’ LFU capability and to fine-tune LLM models to obtain significantly enhanced performance on logical reasoning. |
SportReason: Evaluating Retrieval-Augmented Reasoning across Tables and Text for Sports Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for retrieval-augmented reasoning on numerical sports questions focus on one or two evidence units. |
| Approach: | They propose a benchmark for retrieval-augmented reasoning on numerical sports questions . they evaluate existing retrievers and rerankers, along with agentic Retrieval-Augmented Generation systems. |
| Outcome: | The proposed benchmarks focus on the sports domain because it offers rich multi-modal resources. |