BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning

Challenge:	BioT5+ is an extension of the BioT5, but lacked a nuanced understanding of molecular structures.
Approach:	They propose a new bio-entity modeling framework, BioT5+, which integrates IUPAC names and molecule data.
Outcome:	The proposed model bridges the gap between molecular representations and textual descriptions and improves the grounded reasoning of bio-text and bio-sequences.

Similar Papers

Overview

Challenge

Approach

Outcome

BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations (2023.emnlp-main)

Copied to clipboard

Qizhi Pei, Wei Zhang, Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingce Xia, Rui Yan

Multimodality

Challenge:	et al., 2022) argue that the current models for drug discovery lack the ability to integrate molecules, proteins, and natural language.
Approach:	They propose a framework that integrates biological knowledge with chemical knowledge and natural language associations.
Outcome:	The proposed framework shows superior performance across a wide range of tasks.

Biomedical relation extraction with pre-trained language representations and minimal task-specific architecture (D19-57)

Copied to clipboard

Ashok Thillaisundaram, Theodosia Togia

Semantic Text Processing

Challenge:	Using a pre-trained BERT-Base model, we learn domain-specific language representations using biomedical text.
Approach:	They propose a system that extends BERT, a state-of-the-art language model, which learns contextual language representations from a large unlabelled corpus.
Outcome:	The proposed model outperforms a baseline model while relying on an extremely simple setup with no specially engineered features.

Integration of Deep Learning and Traditional Machine Learning for Knowledge Extraction from Biomedical Literature (D19-57)

Copied to clipboard

Jihang Mao, Wanli Liu

Information Extraction & Text Mining

Challenge:	BB system is among the top two systems in five of all six subtasks . knowledge about microbial diversity is crucial for the study of microbiome and bacteria .
Approach:	They present a system that uses word embedding and lexical features to perform entities recognition, normalization and relation extraction.
Outcome:	The proposed system achieves state-of-the-art in five of six subtasks and is among the top two in five.

Training Text-to-Molecule Models with Context-Aware Tokenization (2025.findings-emnlp)

Copied to clipboard

Seojin Kim, Hyeontae Song, Jaehyun Nam, Jinwoo Shin

Text Segmentation

Challenge:	Text-to-molecule models have shown great potential across chemical applications . however, they rely on atom-level tokenizations, which limiting the ability of models to capture global structural context within molecules.
Approach:	They propose a text-to-molecule model that uses substructure-level tokenizations to model global connectivity.
Outcome:	The proposed model outperforms state-of-the-art models using only 2% of training tokens.

CodeT5+: Open Code Large Language Models for Code Understanding and Generation (2023.emnlp-main)

Copied to clipboard

Yue Wang, Hung Le, Akhilesh Gotmare, Nghi Bui, Junnan Li, Steven Hoi

Language Models

Challenge:	Existing code LLMs adopt a specific architecture or rely on a unified encoder-decoder network for downstream tasks, lacking flexibility to operate in the optimal architecture for a particular task.
Approach:	They propose to initialize code LLMs with frozen off-the-shelf LLM and explore instruction-tuning to align with natural language instructions.
Outcome:	The proposed model outperforms open-source LLMs on 20 code-related benchmarks.

Rethinking Text-based Protein Understanding: Retrieval or LLM? (2025.emnlp-main)

Copied to clipboard

Juntong Wu, Zijing Liu, He Cao, Li Hao, Bin Feng, Zishan Shu, Ke Yu, Li Yuan, Yu Li

Information Retrieval

Challenge:	Recent studies have focused on integrating protein-related knowledge into large language models through continued pretraining and multi-modal alignment.
Approach:	They propose a retrieval-enhanced method which significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios.
Outcome:	The proposed method significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios.

BioTool: A Comprehensive Tool-Calling Dataset for Enhancing Biomedical Capabilities of Large Language Models (2026.acl-long)

Copied to clipboard

Xin Gao, Ruiyi Zhang, Meixi Du, Peijia Qin, Pengtao Xie

Language Models

Challenge:	despite the success of large language models, their performance in highly specialized domains remains unsatisfactory.
Approach:	They propose a biomedical tool-calling dataset designed for fine-tuning LLMs . the dataset contains 34 frequently used tools from the NCBI, Ensembl, and UniProt databases .
Outcome:	The proposed dataset outperforms commercial LLMs on biomedical domains.

CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation (2021.emnlp-main)

Copied to clipboard

Yue Wang, Weishi Wang, Shafiq Joty, Steven C.H. Hoi

Language Models

Challenge:	Pre-trained models for Natural Languages (NL) like BERT and GPT have been shown to transfer well to Programming Languages.
Approach:	They propose a unified pre-trained encoder-decoder Transformer model that leverages the code semantics conveyed from the developer-assigned identifiers.
Outcome:	The proposed model outperforms existing models on understanding and generation tasks and can capture semantic information from code.

BioEL: A Comprehensive Python Package for Biomedical Entity Linking (2025.findings-naacl)

Copied to clipboard

Prasanth Bathala, Christophe Ye, Batuhan Nursal, Shubham Lohiya, David Kartchner, Cassie S. Mitchell

Robustness in NLP

Challenge:	Entity Linking in biomedical literature is a critical task that enhances the extraction and integration of information from diverse scientific literature.
Approach:	They propose a Python package that allows for better Entity Linking in biomedical literature . the package includes four components: Ontology Object, Dataset Object and Evaluation Framework .
Outcome:	The proposed open-source package enables the implementation and comparison of biomedical entity linking tasks.

Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models (2025.findings-emnlp)

Copied to clipboard

Haonan He, Yuchen Ren, Yining Tang, Ziyang Xu, Junxian Li, Minghao Yang, Di Zhang, Yuan Dong, Tao Chen, Shufei Zhang, Yuqiang Li, Nanqing Dong, Wanli Ouyang, Dongzhan Zhou, Peng Ye

Language Models

Challenge:	Biology-Instructions is the first large-scale instruction-tuning dataset for multi-omics biological sequences.
Approach:	They propose a large-scale instruction-tuning dataset for multi-omics biological sequences . they propose 'chatMultiOmics' to overcome limitations of current LLMs on multi-ome tasks .
Outcome:	The proposed dataset bridges LLMs and complex biological sequence-related tasks while maintaining conversational fluency.