MSDAFL: molecular substructure-based dual attention feature learning framework for predicting drug–drug interactions

Abstract

1 Introduction

Drug–drug interactions (DDIs) can cause unexpected adverse drug reactions, affecting treatment efficacy and patient safety (Vilar et al. 2014). DDIs refer to interactions that occur between two or more drug administration processes, including changes in drug properties and the occurrence of toxic side effects (Sun et al. 2016). Therefore, research on DDI prediction is of great practical importance. However, traditional biological or pharmacological methods are costly, time-consuming, and labor-intensive (Shao and Zhang 2013).

Machine learning offers a fresh avenue for accurately predicting DDIs (Mei and Zhang 2021). Methods based on feature similarity posit that drugs sharing similar attributes often exhibit comparable reaction patterns, relying largely on drug properties such as fingerprinting (Vilar et al. 2013), chemical structures (Takeda et al. 2017), pharmacological phenotypes (Li et al. 2015), and RNA profiles (Li et al. 2022). Enhancements in model efficacy are achieved by integrating various features. For instance, the DDI-IS-SL model forecasts DDIs through a blend of integrated similarity measures and semi-supervised learning techniques (Yan et al. 2020). Despite their advancements, these feature similarity-based methods often overlook the structural details of drugs, and their feature selection heavily depends on specialized knowledge and experience.

Graph neural networks (GNNs) have widely been implemented to analyze the chemical structures of drugs and forecast DDIs. Contemporary GNN methodologies are divided into two main types. The first type focuses on embedding features directly from the molecular graphs of drugs, effectively utilizing a straightforward method to encapsulate graph-based data (Gilmer et al. 2017). In this method, atoms within the molecular graph are treated as nodes, with chemical bonds serving as the connecting edges. This setup allows for the embedding of the molecular graph by learning features of individual atoms and the interactions conveyed through the chemical bonds. For instance, SSI-DDI deconstructs the DDI prediction task between two drugs to pinpoint pairwise interactions among their respective substructures (Nyamabo et al. 2021). DSN-DDI is a dual-view drug representation learning network specifically engineered to concurrently learn drug substructures from individual drugs and drug pairs (Li et al. 2023). The second type leverages existing drug interaction networks, where drugs are nodes and their interactions are edges, treating the task of DDI prediction as akin to link prediction within these networks. Clearly, the latter type exhibits a common limitation: they lack inductive capability, are unable to accommodate novel drugs not present in the interaction network, and struggle to maintain diverse types of associations between entities. KGNN applies a knowledge-based GNN approach to extract relational data from knowledge graphs to enhance DDI prediction (Lin et al. 2020). MIRACLE employs multi-view graph contrastive representation learning to simultaneously capture the structural interplay and interactions within and between molecules, enhancing the prediction of DDIs (Wang et al. 2021). Lastly, HTCL-DDI applies a hierarchical triple-view contrastive learning framework for predicting DDIs (Zhang et al. 2023). These diverse approaches illustrate the adaptive use of GNNs in addressing the complexities of predicting drug interactions, combining structural and relational data for improved predictive accuracy.

While current deep learning approaches have demonstrated promising results in predicting DDIs, there remains considerable potential for further enhancement. Firstly, methods that rely solely on a single self-attention mechanism for feature extraction may not comprehensively characterize drug information, potentially missing complex interactions between different substructures. Additionally, integrating multiple sources of feature information could introduce redundant features and noise, unnecessarily complicating the model. Secondly, overfitting during model training significantly impacts prediction results, often resulting in biased predictions. The main contributions of this work are outlined as follows:

• We designed a new Molecular Substructure-based Dual Attention Feature Learning framework for predicting DDIs (MSDAFL). This framework leverages both self-attention and interactive attention mechanisms to effectively extract and process interaction information between drug substructures, enhancing the accuracy of DDI predictions.

• To uncover the hidden features of interactions between drug substructures, we computed the cosine similarity matrix. This approach has shown that these similarity vectors significantly contribute to the accuracy of predicting DDIs.

• Additionally, to reduce overfitting during model training, we adopted a normalization strategy. This not only retains the essential interaction features but also improves the predictability and reliability of DDI outcomes.

2 Materials and methods

2.1 Dataset

To evaluate the scalability and robustness of MSDAFL, we test our model on three public datasets, which vary in scale, density and widely used in previous studies. The scale of the dataset is determined by the number of drugs included. According to previous studies, we also treat the observed DDIs as positive samples and also randomly sample the non-existing DDIs to generate the negative samples. We perform stratified splitting to divide all the drug pairs into a training set, a validation set, and a testing set in a ratio of 6:2:2 (three-fold cross-validation) and 8:1:1 (five-fold cross-validation). We run the experiments on three random folds and five random folds, respectively. As shown in Supplementary Table S1, the statistics of the preprocessed datasets are listed as follows:

• ZhangDDI dataset (Zhang et al. 2017) is of small-scale, consisting of 544 drugs and 45 720 pairwise DDIs.

• ChCh-Miner dataset (Ma et al. 2018) is of medium-scale, consisting of 997 drugs and 21 486 pairwise DDIs.

• DeepDDI dataset (Gilmer et al. 2017) is of large-scale, consisting of 1704 drugs and 191 870 pairwise DDIs.

2.2 Problem formulation

The DDI prediction task is formulated as a binary classification problem aimed at discerning the existence of interactions between pairs of drugs. A drug’s molecular structure can be abstractly represented by a graph G, where $X∈RN×d$ denotes the node feature matrix and $A∈RN×N$ denotes the adjacency matrix. Within molecular graphs, nodes correspond to atoms, and edges represent chemical bonds between them. GNNs primarily employ the Message Passing mechanism, which integrates information from neighboring nodes to update their representations (Gilmer et al. 2017). Prominent variants of GNNs include Graph Convolutional Networks (GCNs) (Kipf and Welling 2016), Graph Attention Networks (GATs) (Velickovic et al. 2017), and Graph Isomorphism Networks (GINs) (Xu et al. 2018). In this study, we adopt GIN as the foundational architecture for our model. The detailed description of how construct the node feature matrix X is illustrated in Supplementary Section S2. In the context of DDI prediction, given the adjacency matrix A representing the molecular structure graph G and the node feature matrix X, the objective is to derive a predictive function $f(d1,d2)→[0,1]$⁠. This function aims to estimate the likelihood of interaction between any pair of drugs $d1$ and $d2$⁠.

2.3 Overview of MSDAFL

The framework of our model is depicted in Fig. 1. In the GNN encoder, we use RDKit to convert drug SMILES sequences into molecular graphs, which are then encoded using GIN. Subsequently, we employ two Transformer-like encoders: a self-attention mechanism encoder and an interaction attention mechanism encoder, both equipped with learnable pattern vectors, to compress the graphs into M representative vectors. Within the GSAT encoder, cosine similarity is computed for each pair of representative vectors from two drugs, resulting in an $M×M$ similarity matrix. After flattening the similarity matrix, the resulting vectors encapsulate rich features of drug interactions. Simultaneously, in the interaction attention mechanism encoder GIAT, drug pair features are obtained after the interaction attention mechanism to derive feature matrices $O1$ and $O2$⁠. These matrices undergo average pooling and standardization, yielding vector pairs containing interaction features of substructures between drug pairs. Finally, the vectors obtained from the GSAT and GIAT encoders are concatenated and input into an MLP （Multilayer Perceptron） layer to produce the final prediction.

Figure 1.

Overview of the proposed MSADAFL framework. The overall framework includes a GNN encoder, a self-attention mechanism encoder (GSAT encoder), and an interactive attention mechanism encoder (GIAT). The features from GSAT and GIAT are concatenated in an MLP for prediction, yielding the final drug–drug interaction prediction results.

Open in new tabDownload slide

2.4 GSAT encoder

Figure 2.

The detail of the GSAT Encoder. The drug features obtained through the GNN encoder are processed by the self-attention mechanism to generate different feature matrices between the drugs. The cosine similarity matrix S is then computed, and after flattening, it results in the vector s that contains the interaction features.

Open in new tabDownload slide

The similarity module non-parametrically characterizes interactions among the substructures of the two drugs. The elements of $Sij$ denote the strength of interaction, thereby enhancing the interpretability of prediction outcomes.

2.5 GIAT encoder

To more closely examine the substructural effects in drug pairs, we use an encoder with a cross-attention mechanism to learn the interaction patterns between drug pairs, as shown in Fig. 3. The formulation can be described as follows:

Figure 3.

The detailed operational principle of the GIAT encoder employs a cross-attention mechanism to process drug matrices derived from the GNN encoder, thereby producing distinct feature matrices denoted as $O1$ and $O2$ for each drug pair. Subsequently, through average pooling and normalization, interaction vectors $inter1$ and $inter2$ are computed for these drug pairs.

Open in new tabDownload slide

2.6 Normalization module

The variables $inter1$ and $inter2$ represent the interaction vector representations of different drugs. The symbols $μO¯1$ and $μO¯2$ represent the mean values of $O¯1$ and $O¯2$⁠, respectively. The symbols $σO¯1$ and $σO¯2$ represent the standard deviations of $O¯1$ and $O¯2$⁠, respectively.

2.7 MLP layer

3 Experimental result

3.1 Evaluation metrics and experimental setup

In our experimental evaluation, we have chosen four metrics area under the receiver operating characteristic curve (AUROC), average precision (AP), F1-score (F1), and accuracy (ACC) to comprehensively assess the model’s performance in predicting DDIs. To ensure the reliability of our results and mitigate the impact of random variability, each experiment is conducted five times, and we report the mean values of these metrics. For a detailed description of these four metrics, please refer to Supplementary Section S3.

We conducted all experiments on an Ubuntu release 20.04 system utilizing the NVIDIA A40-PCIE GPU card with 48 GB of memory. To ensure equitable performance comparisons, all models were implemented in PyTorch. Our model was trained for 300 epochs, with a learning rate of 0.001 for the first 150 epochs and 0.0001 for the subsequent 150 epochs. The batch size was set to 512, and the node embedding dimension was fixed at 128. We utilized 60 representative vectors M and employed five layers in the GIN structure. Additionally, we initialized the DDI using Xavier initialization (Glorot and Bengio 2010) and optimized the model parameters using the Adam optimizer (Kingma and Ba 2014).

3.2 Comparison model description

In comparative experiments, we compare ten state-of-the-art methods: MR-GNN, GCN-BMP, EPGCN-DS, DeepDrug, MIRACLE, SSI-DDI, CSGNN, DeepDDS, DSN-DDI, and HTCL. Here are some brief introductions:

• MR-GNN (Xu et al. 2019) employs a multi-resolution architecture to capture local features of each graph and extract interaction features between pairwise graphs.

• GCN-BMP (Chen et al. 2020) leverages GNN for DDI prediction by employing an end-to-end graph representation learning framework.

• EPGCN-DS (Sun et al. 2020) detects DDIs from molecular structures using an encoder with expressive GCN layers and a decoder that outputs the probability of DDI.

• DeepDrug (Cao et al. 2020) employs residual graph convolutional networks (RGCNs) along with convolutional networks (CNNs) to enhance the accuracy of DDI prediction.

• MIRACLE (Wang et al. 2021) offers a multi-view framework that simultaneously captures the intra-view molecular structure and the inter-view DDIs between molecules.

• SSI-DDI (Nyamabo et al. 2021) deconstructs the DDI prediction task between two drugs to pinpoint pairwise interactions among their respective substructures.

• CSGNN (Zhao et al. 2021) incorporates a mix-hop neighborhood aggregator into a GNN to capture high-order dependencies in DDI networks and utilizes a contrastive self-supervised learning task as a regularizer.

• DeepDDS (Wang et al. 2022) is a deep learning model that employs GNNs and an attention mechanism to identify synergistic drug combinations.

• DSN-DDI (Li et al. 2023) is a dual-view drug representation learning network designed to learn drug substructures from individual drugs and drug pairs simultaneously.

• HTCL-DDI (Zhang et al. 2023) is a hierarchical triple-view contrastive learning framework for predicting DDIs.

3.3 Model performance comparison

In our study, we compared MSDAFL with 10 competitive DDI prediction models across three datasets of varying scales, using three widely adopted evaluation metrics (AUROC, AUPRC, and F1) to assess their predictive performance. Table 1 summarizes the experimental outcomes of MSDAFL and other baseline methods across these datasets on training, validation and testing sets in a ratio of 6:2:2. MSDAFL consistently demonstrated superior performance across multiple evaluation metrics and datasets. On the ZhangDDI dataset, HTCL-DDI achieved superior results in AP and F1 metrics by leveraging diverse view relationships and integrating multi-view features for DDI prediction. On the ChCh-Miner and DeepDDI datasets, MSDAL showed substantial improvements compared to HTCL. For instance, on DeepDDI, MSDAFL improved AUROC by approximately 5% and ACC by around 6%. SSI-DDI deconstructs the DDI prediction task between drug pairs to identify pairwise interactions among their respective substructures. In contrast, DSN-DDI is a dual-view drug representation learning network designed to simultaneously learn drug substructures from individual drugs and drug pairs. MSDAFL comprehensively outperforms SSI-DDI and DSN-DDI across all metrics on the three datasets. As a multi-attention mechanism network framework, MSDAFL effectively applies various attention mechanisms to predict DDIs, processing drugs from different interaction perspectives to achieve robust and diverse drug representations. The outstanding performance of MSDAFL across these datasets underscores the potential of interactive attention mechanisms in capturing critical features relevant to DDIs.

In addition, we also compare MSDAFL and other baseline methods across these datasets on training, validation and testing sets in a ratio of 8:1:1. The results are shown in the Table 2. The MSADFL model demonstrates improved performance on the ZhangDDI dataset. For the ChCh-Miner and DeepDDI datasets, the model’s performance remains stable across all four metrics. This stability highlights the strong capabilities of our model’s self-attention mechanism and interaction attention. In contrast, the HTCL-DDI model shows a decline in performance across all datasets, particularly on the DeepDDI dataset. This decline underscores the limitations of the approach that leverages existing drug interaction networks, where drugs are nodes and their interactions are edges, which can negatively impact drug interaction prediction. Our model outperforms others across all three datasets, indicating that it surpasses other models in predictive performance. Furthermore, to assess the generalization performance of the MSADFL model, we further conduct cross-datum study experiments on three datasets: ZhangDDI, DeepDDI, and ChCh-Miner. The experimental results of MSADFL and HTCL-DDI are presented in Supplementary Tables S2 and S3, respectively. Compared to previous experiments across three datasets with varying scales, the prediction performances of MSDAFL and HTCL-DDI are declined. Notably, when DeepDDI is used as the training set and ChCh-Miner as the test set, the AUC reaches 0.8227, highlighting the model’s robust generalization ability.

3.4 Ablation experiment

The outstanding performance of MSDAFL stems from three carefully designed strategies: the cross-attention mechanism strategy between drug pairs, normalization of the interaction matrix, and self-attention mechanism strategy with cosine similarity. To ascertain the efficacy of each drug feature type, we conducted ablation experiments on the ZhangDDI dataset across these three configurations on training, validation and testing sets in a ratio of 6:2:2.The results are shown in the Supplementary Fig. S1, demonstrating the effectiveness of the module we proposed.

3.5 Parameter sensitivity

To investigate the influence of crucial parameters on prediction performance, we systematically vary these parameters and assessed their impact on the MSDAFL model’s efficacy using the ZhangDDI dataset on training, validation and testing sets in a ratio of 6:2:2. We analyze the batch size for model training, the parameter $λ$⁠, and the number of GIN layers. By holding other parameters constant, we explore how varying key parameter settings impacted the performance of MSDAFL. As illustrated in the Supplementary Fig. S2, we investigated how these parameters affect model performance. Specifically, we found that the model performs optimally when the batch size is set to 512, $λ$ to 0.75, and the number of GIN layers to 5.

3.6 Case study

To assess the practical utility of MSDAFL in real-world scenarios, we performed an analysis of clinical studies evaluating the prediction outcomes for four drug pairs using MSDAFL on the ZhangDDI testing set, as depicted in Supplementary Fig. S3. For the analysis of these four drugs, we can confirm the powerful performance of the MSDAFL model in predicting DDIs.

4 Discussion and conclusion

In this work, we introduce a molecular substructure-based dual attention feature learning framework for predicting DDIs. This framework integrates multiple attention mechanisms, including a self-attention encoder that extracts substructures from individual drugs and computes a cosine similarity matrix between the feature matrices of drug pairs. In the interactive attention encoding segment, we employ an interactive attention mechanism to investigate the strength of interactions between substructures of drug pairs, culminating in the regularization of the interactive feature matrix. Extensive experiments are conducted across three public datasets to evaluate the efficacy of our MSDAFL model and assess the contributions of its various modules. The findings decisively establish MSDAFL as a robust and promising tool for predicting DDIs, significantly contributing to medication safety and drug side effect research. Our study can be further advanced in three key domains: (i) by integrating heterogeneous biomedical information to augment representation learning, (ii) by expanding MSDAFL to more complex and practical application scenarios, and (iii) by supplementing with wet-lab experiments to further validate certain DDI prediction outcomes.

Supplementary data

Supplementary data are available at Bioinformatics online.

Conflict of interest

None declared.

Funding

This work was supported in part by the National Natural Science Foundation of China (62473149, 61962050, and 62072473), Natural Science Foundation of Hunan Province of China (2022JJ30428) and Excellent youth funding of Hunan Provincial Education Department (22B0372).

Data availability

Our code and data are available at: https://github.com/27167199/MSDAFL.

References