Papers by Hanlin Zhang
Evaluating Step-by-Step Reasoning through Symbolic Verification (2024.findings-naacl)
Copied to clipboard
| Challenge: | Pre-trained language models (LMs) have shown remarkable reasoning performance using explanations or chain-of-thoughts (CoT)) for in-context learning. |
| Approach: | They propose to use symbolic examples to iteratively reason over symbolic examples and to recover Prolog’s backward chaining algorithm to iterate over KBs. |
| Outcome: | The proposed model performs better on length generalization benchmarks than CoT on explanations and chain-of-thoughts (CoT) tasks. |
What Did You Refer to? Evaluating Co-References in Dialogue (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing neural end-to-end dialogue models have limitations on exactly interpreting the linguistic structures in dialogue history context. |
| Approach: | They propose to directly measure the capability of neural end-to-end dialogue models on understanding the entity-oriented structures via question answering. |
| Outcome: | The proposed model can understand large-scale English and Chinese human human dialogues using a large-format dataset. |
Not All Citations Are Equal:Entropy-Guided Citation Selection for Noise-Resistant Medical LLM (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models have demonstrated extensive potential in medical applications . however, their practical deployment in healthcare faces significant challenges . |
| Approach: | They propose a training-free multi-turn reasoning framework and a post-training methodology that provides external knowledge support for large language models. |
| Outcome: | The proposed framework elicits internal thought, external thought, and fusion thought, with an entropy-based reward that encourages selective citation of beneficial external knowledge while penalizing noisy citations. |
Optimizing Language Models with Fair and Stable Reward Composition in Reinforcement Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has developed algorithms for reinforcement learning from human feedback and AI-generated feedback. |
| Approach: | They propose a method for reinforcement learning from human feedback and AI-generated feedback that incorporates weighting, ranking, and constraining to handle disparate rewards. |
| Outcome: | The proposed method reduces disparity and enhances stability among rewards . empirical results show that the proposed method is efficient and straightforward . |
NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Queries (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate code for productive activities, but current benchmarks for code synthesis are oriented towards introductory tasks on algorithm and data science. |
| Approach: | They propose a code benchmark to mirror the complexity and variety of scenarios in real-world coding tasks. |
| Outcome: | The proposed benchmark improves on 39 large language models with close HumanEval scores and achieves an efficiency increase of more than 4 times. |
Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained large language models struggle to perform logical reasoning reliably despite advances in scale and compositionality. |
| Approach: | They propose a Differentiable Symbolic Reasoning framework that uses symbolic programming to improve LMs' logical reasoning abilities. |
| Outcome: | The proposed framework outperforms competitive baselines when faced with systematic changes in sequence length. |
A Study on the Calibration of In-context Learning (2024.naacl-long)
Copied to clipboard
Hanlin Zhang, YiFan Zhang, Yaodong Yu, Dhruv Madeka, Dean Foster, Eric Xing, Himabindu Lakkaraju, Sham Kakade
| Challenge: | Prior research has demonstrated improvements in the calibration of language models (LMs) in-context learning is a popular method for adapting static LMs to safety-critical domains. |
| Approach: | They use in-context learning to adapt static language models through tailored prompts to a wide range of tasks and find that miscalibration occurs in low-shot settings. |
| Outcome: | The proposed calibrations show that models exhibit increased miscalibration before achieving better calibration in low-shot settings. |