Papers by Mark Gerstein
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning (2024.findings-acl)
Copied to clipboard
Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, Mark Gerstein
| Challenge: | Large language models face unique challenges such as domain-specific terminologies and reasoning over specialized knowledge. |
| Approach: | They propose a multi-disciplinary collaboration framework that leverages LLM-based agents in a role-playing setting. |
| Outcome: | The proposed framework excels at mining and harnessing medical expertise within LLMs, as well as extending its reasoning abilities. |
MIMIR: A Customizable Agent Tuning Platform for Enhanced Scientific Applications (2024.emnlp-demo)
Copied to clipboard
Xiangru Tang, Chunyuan Deng, Hanminwang Hanminwang, Haoran Wang, Yilun Zhao, Wenqi Shi, Yi Fung, Wangchunshu Zhou, Jiannan Cao, Heng Ji, Arman Cohan, Mark Gerstein
| Challenge: | Large language models (LLMs) have evolved into interactive agents capable of planning, tool use, and task execution across various tasks. |
| Approach: | They propose a platform that leverages large language models to generate agent-tuning data for fine-tuneing smaller, specialized models. |
| Outcome: | MIMIR enables large models to simulate various roles and create interaction data, which can then be used to fine-tune open-source models like LLaMA2. |
Struc-Bench: Are Large Language Models Good at Generating Complex Structured Tabular Data? (2024.naacl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have advanced capabilities but produce complex structured data. |
| Approach: | They propose a structure-aware fine-tuning method to bolster LLMs' performance by crafting format-specific instructions from the intended outputs. |
| Outcome: | The proposed method outperforms LLMs on all three formats and spans text tables, HTML, and LaTeX formats. |
Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewards (2025.emnlp-main)
Copied to clipboard
Jaehoon Yun, Jiwoong Sohn, Jungwoo Park, Hyunjae Kim, Xiangru Tang, Daniel Shao, Yong Hoe Koo, Ko Minhyeok, Qingyu Chen, Mark Gerstein, Michael Moor, Jaewoo Kang
| Challenge: | Large language models have shown promise in clinical decision making, but current approaches struggle to localize and correct reasoning errors at specific steps of the reasoning process. |
| Approach: | They propose a process reward modeling framework that leverages retrieval-augmented generation to verify each reasoning step against established medical knowledge bases. |
| Outcome: | The proposed model improves on five medical QA benchmarks and two open-ended diagnostic tasks by 13.50% on MedQA. |
Investigating Data Contamination in Modern Benchmarks for Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for large language models are inflated and inconsistent with actual performance. |
| Approach: | They propose a retrieval-based system to explore potential overlaps between benchmarks and pretraining corpora and a protocol to investigate testset slot guessing. |
| Outcome: | The proposed method exploits overlaps between evaluation benchmarks and pretraining corpora and masks a wrong answer in a multiple choice question and prompts the model to fill in the gap. |