MolXPT: Wrapping Molecules with Text for Generative Pre-training (2023.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that Generative pre-trained Transformers (GPT) have great success in natural language processing. |
| Approach: | They propose a unified language model of text and molecules pre-trained on SMILES wrapped by text. |
| Outcome: | The proposed model outperforms strong baselines of molecular property prediction on MoleculeNet and performs comparably to the best model in text-molecule translation while using less than half of its parameters. |
Similar Papers
Translation between Molecules and Natural Language (2022.emnlp-main)
Copied to clipboard
| Challenge: | MolT5 pretrains models on unlabeled natural language text and molecule strings . bringing a new drug to market can cost over a billion dollars and take over ten years . |
| Approach: | They propose a self-supervised learning framework for pretraining models on unlabeled natural language text and molecule strings. |
| Outcome: | The proposed framework pretrains models on unlabeled natural language text and molecule strings, and it generates high quality outputs. |
MolTRES: Improving Chemical Language Representation Learning for Molecular Property Prediction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for chemical representation learning often lead to overfitting and limited scalability due to early convergence. |
| Approach: | They propose a framework to train Transformers on SMILES sequences to learn from structural examples and integrate external materials embedding to enrich molecular representations. |
| Outcome: | The proposed model outperforms state-of-the-art models on molecular property prediction tasks. |
Multilingual Molecular Representation Learning via Contrastive Pre-training (2022.acl-long)
Copied to clipboard
| Challenge: | Molecular representation learning is an essential role in cheminformatics, but language model-based approaches focus on local features, hence they may not capture global information. |
| Approach: | They propose a multilingual molecular embedding generation approach that uses two different languages to train a given molecule. |
| Outcome: | The proposed approach is pre-trained using SMILES and IUPAC as two different languages on large-scale molecules. |
MolRAG: Unlocking the Power of Large Language Models for Molecular Property Prediction (2025.acl-long)
Copied to clipboard
| Challenge: | Recent LLMs exhibit limited effectiveness on molecular property prediction task due to semantic gap between representations and natural language and lack of domain-specific knowledge. |
| Approach: | They propose a framework that integrates Chain-of-Thought reasoning for molecular property prediction. |
| Outcome: | The proposed framework outperforms pre-trained LLMs on four datasets and matches supervised methods. |
BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations (2023.emnlp-main)
Copied to clipboard
| Challenge: | et al., 2022) argue that the current models for drug discovery lack the ability to integrate molecules, proteins, and natural language. |
| Approach: | They propose a framework that integrates biological knowledge with chemical knowledge and natural language associations. |
| Outcome: | The proposed framework shows superior performance across a wide range of tasks. |
Training Text-to-Molecule Models with Context-Aware Tokenization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Text-to-molecule models have shown great potential across chemical applications . however, they rely on atom-level tokenizations, which limiting the ability of models to capture global structural context within molecules. |
| Approach: | They propose a text-to-molecule model that uses substructure-level tokenizations to model global connectivity. |
| Outcome: | The proposed model outperforms state-of-the-art models using only 2% of training tokens. |
MolTC: Towards Molecular Relational Modeling In Language Models (2024.findings-acl)
Copied to clipboard
Junfeng Fang, Shuai Zhang, Chang Wu, Zhengyi Yang, Zhiyuan Liu, Sihang Li, Kun Wang, Wenjie Du, Xiang Wang
| Challenge: | Molecular Relational Learning (MRL) is a promising way to understand interactions between molecular pairs. |
| Approach: | They propose a novel LLM-based multi-modal framework for molecular interaction modeling following Chain-of-Thought (CoT) theory which integrates graphical information of two molecules in pair. |
| Outcome: | The proposed framework integrates graphical information of two molecules in pair. |
RecGPT: Generative Pre-training for Text-based Recommendation (2024.acl-short)
Copied to clipboard
| Challenge: | Existing models for text-based recommendation lack data sparsity and flexibility to capture fluctuations in user preferences over time. |
| Approach: | They present the first domain-adapted and fully-trained large language model for text-based recommendation. |
| Outcome: | The proposed model outperforms baseline models on rating prediction and sequential recommendation tasks. |
Less for More: Enhanced Feedback-aligned Mixed LLMs for Molecule Caption Generation and Fine-Grained NLI Evaluation (2025.acl-long)
Copied to clipboard
| Challenge: | Recent trends have led to the use of multimodal models to learn molecular and linguistic representations, either in separate but coordinated spaces or in a common space. |
| Approach: | They propose a novel atomic-level evaluation method leveraging off-the-shelf Natural Language Inference (NLI) models for use in the unseen chemical domain. |
| Outcome: | The proposed method surpasses state-of-the-art models in the unseen chemical domain while relying on a granularity-based evaluation method. |
ReactXT: Understanding Molecular “Reaction-ship” via Reaction-Contextualized Molecule-Text Pretraining (2024.findings-acl)
Copied to clipboard
Zhiyuan Liu, Yaorui Shi, An Zhang, Sihang Li, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, Tat-Seng Chua
| Challenge: | Molecular-text modeling is an emerging research field that aims to facilitate molecule-relevant tasks with a textual interface and textual knowledge. |
| Approach: | They propose a new method for reaction-text modeling that uses three types of input contexts to incrementally pretrain LMs. |
| Outcome: | The proposed method improves experimental procedure prediction and molecule captioning and offers competitive results in retrosynthesis. |