AI Agents for the Science of Science: A Survey of Tasks, Architectures, Evaluations, and Challenges (2026.findings-acl)
Copied to clipboard
| Challenge: | The Science of Science (SciSc) examines how scientific knowledge is produced, evaluated, and transformed by utilizing large-scale scholarly and bibliometric data. |
| Approach: | They propose a task-centered taxonomy for AI agents that model citations, collaborations, and community dynamics. |
| Outcome: | The proposed taxonomy distinguishes agents as simulations from tools for empirical analysis and scientific workflows. |
Similar Papers
AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation (2026.eacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial provides an overview of recent advances in AI-assisted tools and models that support and enhance the scientific research process. |
| Approach: | This tutorial provides an overview of recent advances in AI-assisted tools and models that support and enhance the scientific research process. |
| Outcome: | This tutorial provides an overview of recent advances in AI-assisted tools and models that support and enhance the scientific research process. |
From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systems (2025.findings-emnlp)
Copied to clipboard
Zekun Zhou, Xiaocheng Feng, Lei Huang, Xiachong Feng, Ziyun Song, Ruihan Chen, Liang Zhao, Weitao Ma, Yuxuan Gu, Baoxin Wang, Dayong Wu, Guoping Hu, Ting Liu, Bing Qin
| Challenge: | rapid development of artificial intelligence (AI) technologies has inspired researchers to explore how AI can accelerate and enhance research. |
| Approach: | They organize the relevant studies into three main categories: hypothesis formulation, hypothesis validation, and manuscript publication. |
| Outcome: | The authors summarize the current state of research in three main areas: hypothesis formulation, hypothesis validation, and manuscript publication. |
SciExplore: Evaluating Autonomous Agents from Scientific Navigation to Information Integration (2026.findings-acl)
Copied to clipboard
Yinhao Tang, Youqing Fang, Yanan Sun, Wenran Liu, Weiming Zhang, Bin Liu, Kuikun Liu, Wenwei Zhang, Kai Chen
| Challenge: | Existing benchmarks emphasize general-domain retrieval or static scientific question answering . SciExplore focuses on scientific database navigation, ambiguous literature retrieval, missing reference completion, and cross-source structured knowledge synthesis tasks. |
| Approach: | They propose a benchmark to evaluate scientific information-seeking and reasoning capabilities of LLMs and agents. |
| Outcome: | The new benchmark assesses the capabilities of state-of-the-art LLMs and agents in scientific research workflows. |
Datasets for Scientific Literature Understanding: A Survey (2026.findings-acl)
Copied to clipboard
| Challenge: | Empowering machines to understand scientific literature is crucial for accelerating scientific discovery and advancing the AI for Science paradigm. |
| Approach: | They propose a systematic taxonomy that organizes resources spanning structural understanding, text understanding, multimodal understanding and pre-training/instruction fine-tuning. |
| Outcome: | The proposed taxonomy organizes resources spanning structural understanding, text understanding, multimodal understanding and pre-training/instruction fine-tuning. |
ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | a new system that leverages the encyclopedic knowledge and linguistic reasoning capabilities of Large Language Models (LLMs) is proposed to enhance the productivity of researchers . a researcher's research idea generation process involves problem identification, method development, experiment design and iterative revision . |
| Approach: | They propose a system that leverages encyclopedic knowledge and linguistic reasoning capabilities of Large Language Models to assist researchers in their work. |
| Outcome: | The proposed system generates novel ideas based on human and model-based evaluations . it leverages encyclopedic knowledge and linguistic reasoning capabilities of Large Language Models based systems . |
Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Building socially-intelligent AI agents involves creating agents that can sense, perceive, reason about, learn from, and respond to affect, behavior, and cognition of other agents. |
| Approach: | They propose a set of technical challenges and open questions for researchers to advance Social-AI. |
| Outcome: | The proposed frameworks are based on the social intelligence competencies that evolved over thousands of years in Homo sapiens and are expected to be the foundations for the development of social-intelligent AI agents. |
From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are catalyzing a paradigm shift in scientific discovery, evolving from task-specific automation tools into increasingly autonomous agents. |
| Approach: | They introduce a foundational three-level taxonomy to delineate their escalating autonomy and evolving responsibilities within the research lifecycle. |
| Outcome: | The proposed frameworks provide a conceptual architecture and strategic foresight to navigate and shape the future of AI-driven scientific discovery. |
WildSci: Advancing Scientific Reasoning from In-the-Wild Literature (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language model reasoning focus on mathematics and coding domains, but scientific reasoning remains limited in other domains due to limited dataset coverage. |
| Approach: | They propose a framework for sustainable scientific reasoning QA generation by synthesizing a new dataset of domain-specific science questions from peer-reviewed literature. |
| Outcome: | The proposed framework and dataset enable scalable and sustainable research in scientific reasoning. |
BLADE: Benchmarking Language Model Agents for Data-Driven Science (2024.findings-emnlp)
Copied to clipboard
Ken Gu, Ruoxi Shang, Ruien Jiang, Keying Kuang, Richard-John Lin, Donghe Lyu, Yue Mao, Youran Pan, Teng Wu, Jiaqian Yu, Yikun Zhang, Tianmai Zhang, Lanyi Zhu, Mike Merrill, Jeffrey Heer, Tim Althoff
| Challenge: | Language model-based agents can be used to conduct and support data-driven science, but evaluating them on open-ended tasks is challenging due to multiple valid approaches, partially correct steps, and different ways to express the same decisions. |
| Approach: | They propose a benchmark to automatically evaluate agents’ multifaceted approaches to open-ended research questions. |
| Outcome: | BLADE evaluates agents’ multifaceted approaches to open-ended research questions using data from 12 datasets and research questions drawn from existing scientific literature. |
AI for Science in the Era of Large Language Models (2024.emnlp-tutorials)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have demonstrated significant prowess in tasks involving natural language, such as translating languages, constructing chatbots, and answering questions. |
| Approach: | This tutorial explores the application of large language models to three crucial categories of scientific data: 1) textual data, 2) biomedical sequences, and 3) brain signals. |
| Outcome: | This tutorial will explore the application of large language models to three crucial categories of scientific data. |