Papers by Yebowen Hu
When Reasoning Meets Information Aggregation: A Case Study with Sports Narratives (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using sports data, an LLM can analyze sports narratives to infer points from actions, identify related entities, attribute points accurately to players and teams, and draw conclusions. |
| Approach: | They propose a method to synthesize NBA basketball game narratives using real NBA basketball data and propose 'SportsGen' they find that most models fail to accurately aggregate basketball scores due to frequent scoring patterns and open-source models suffer from significant score hallucinations. |
| Outcome: | The proposed method can evaluate LLMs’ reasoning capabilities under complex scenarios with varying narrative lengths and density of information. |
DecipherPref: Analyzing Influential Factors in Human Preference Judgments via GPT-4 (2023.emnlp-main)
Copied to clipboard
| Challenge: | Human preference judgments are important in large language models to produce outputs that align with human values. |
| Approach: | They conduct an in-depth examination of pairwise human judgments released by OpenAI . they find that most favored factors vary across tasks and genres . |
| Outcome: | The proposed model reveals that most favored factors vary across tasks and genres . the findings have implications on the construction of balanced datasets in human preference evaluations - crucial step in shaping behavior of future LLMs. |
DeFine: Decision-Making with Analogical Reasoning over Factor Profiles (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models are ideal for decision-making, but they can be difficult to process when they are verbose and include repetition, hedging, and vagueness. |
| Approach: | They propose a framework that constructs probabilistic factor profiles from complex scenarios and integrates them with analogical reasoning to guide LLMs in making decisions in new situations. |
| Outcome: | The proposed framework separates the tasks of quantifying uncertainty and incorporating it into LLM decision-making. |
SportsMetrics: Blending Text and Numerical Data to Understand Information Fusion in LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models can handle text and data, but blending text and numerical data presents significant challenges. |
| Approach: | They propose four tasks to evaluate the numerical reasoning and information fusion capabilities of large language models in sports data analytics. |
| Outcome: | The proposed tasks evaluate the numerical reasoning and information fusion capabilities of large language models in sports data analytics. |
STRUX: An LLM for Decision-Making with Structured Explanations (2025.naacl-short)
Copied to clipboard
| Challenge: | a new LLM decision-making framework is designed to help users understand how and why decisions are made. |
| Approach: | They introduce a new LLM decision-making framework called STRUX that provides structured explanations for LLM decisions. |
| Outcome: | The proposed framework improves decision-making by providing structured explanations . it has been evaluated on the task of forecasting stock investment decisions based on earnings call transcripts - superior performance against strong baselines compared with previous frameworks based upon earnings call transcriptions demonstrating superior performance . |
InFoBench: Evaluating Instruction Following Ability in Large Language Models (2024.findings-acl)
Copied to clipboard
Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, Dong Yu
| Challenge: | Existing methods for evaluating Large Language Models (LLMs) ability to follow instructions have not been able to provide a detailed analysis of their compliance with instructions. |
| Approach: | They propose a new metric for evaluating Large Language Models' ability to follow instructions and a benchmark for DRFR. |
| Outcome: | The proposed metric and benchmark compared with traditional scoring methods and explores annotation sources including human experts, crowd-sourced workers, and GPT-4. |
MeetingBank: A Benchmark Dataset for Meeting Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | a lack of annotated meeting corpora hinders the development of meeting summarization technology. |
| Approach: | They present a new benchmark dataset of city council meetings over the past decade . they use a divide-and-conquer approach to divide professionally written minutes into shorter passages . |
| Outcome: | The proposed dataset provides a testbed for various meeting summarization systems and allows the public to gain insight into how council decisions are made. |