MetaHMEI: meta-learning for prediction of few-shot histone modifying enzyme inhibitors

Abstract

Motivation

Histones are the chief protein components of chromatin, and the chemical modifications on histones crucially influence the transcriptional state of related genes. Histone modifying enzyme (HME), responsible for adding or removing the chemical labels, has emerged as a very important class of drug target, with a few HME inhibitors launched as anti-cancerous drugs and tens of molecules under clinical trials. To accelerate the drug discovery process of HME inhibitors, machine learning-based predictive models have been developed to enrich the active molecules from vast chemical space. However, the number of compounds with known activity distributed largely unbalanced among different HMEs, particularly with many targets of less than a hundred active samples. In this case, it is difficult to build effective virtual screening models directly based on machine learning.

Results

To this end, we propose a new Meta-learning-based Histone Modifying Enzymes Inhibitor prediction method (MetaHMEI). Our proposed MetaHMEI first uses a self-supervised pre-training approach to obtain high-quality molecular substructure embeddings from a large unlabeled chemical dataset. Then, MetaHMEI exploits a Transformer-based encoder and meta-learning framework to build a prediction model. MetaHMEI allows the effective transfer of the prior knowledge learned from HMEs with sufficient samples to HMEs with a small number of samples, so the proposed model can produce accurate predictions for HMEs with limited data. Extensive experimental results on our collected and curated HMEs datasets show that MetaHMEI is better than other methods in the case of few-shot learning. Furthermore, we applied MetaHMEI in the virtual screening process of histone JMJD3 inhibitors and successfully obtained three small molecule inhibitors, further supporting the validity of our model.

INTRODUCTION

Histone modification is an essential mechanism of epigenetic regulation, which controls gene expression without changing the gene sequence. A variety of histone modification enzymes (HMEs) are involved in adding (writers), removing (erasers) and reading (readers) the chemical groups on histones. Since aberrant histone modifications lead to the occurrence and development of human diseases, HMEs as an important class of drug targets have attracted interest from academia to industry. Despite that more than 120 HMEs have been discovered, up to now, only a few HMEs were successfully launched, including histone deacetylases (HDACs) and histone methyltransferases (HMT) EZH2, whereas the majority of HMEs are still under clinical or pre-clinical phases [1, 2]. As the pioneer of HME inhibitors, histone deacetylase (HDAC) inhibitors have been approved for the treatment of multiple myeloma and T-cell lymphoma [3]. The potential of HME inhibitors for other diseases is under clinical trials or pre-clinical investigation, such as solid tumors [4], immune system diseases [5], neurological disorders [6], diabetes [7], etc. One of the major challenges of HME inhibitors development is the poor isoform-selectivity, which limits the application of HME inhibitors due to the off-target toxicity. The substrates and catalytic centers of HMEs share similarities to a different extent, bringing the difficulty to design selective inhibitors among similar targets [8, 9]. In this case, determining active and selective scaffolds during preliminary screening would accelerate the following drug R&D process.

In contrast with traditional wet-experiment-based screening, machine learning-based approaches for predicting HME inhibitors are highly efficient and low-cost, highly desired by us. Yang et al. [10] applied long short-term memory neural network model in generating novel small molecules that target the p300/CBP protein–protein interaction, resulting in the discovery of a highly potent inhibitory. Recently, Norberto et al. [11, 12] reported a machine learning-based predictive tool for 55 epigenetic targets, including HMEs, kinases, chromatin remodelers, etc., and systematically compared the performance of different algorithms and descriptors [11]. Evaluating the activity against multiple targets is very useful in the discovery of selective inhibitors. However, there are only 29 HMEs included in the previous model, due to the difficulty of using traditional machine learning methods on the targets with insufficient compound-target activity data. There are other machine learning-based virtual screening methods [13–15], which assume that the model has sufficient samples for training, without considering the few-shot sample learning.

Generally, using machine learning-based methods to solve the virtual screening problem of HME inhibitors still faces the following challenges: First, as far as we know, there is no public and well-curated small molecule dataset dedicated to the prediction of histone-modified enzyme inhibitors. This poses an obstacle to the virtual screening of HME inhibitors using data-driven machine learning. Second, by analyzing and investigating the data of some known histone-modifying enzyme inhibitor families, such as HDACs, we found that the number of compounds with known activity distributed largely imbalanced among different HMEs, especially with many targets of less than several or dozens of active samples. The issues of the imbalance distribution of samples and few-shot samples challenge the effective training of machine learning prediction models. In particular, the few-shot samples issue will seriously hinder the ligand-based prediction for the inhibitors of some newly discovered HMEs, which often lack compound-target activity data.

To meet the above challenges, the main contributions of this study are as follows.

First, we constructed a small molecule compound data set (named HMEDB) for HMEs prediction by collecting and curating compound-target activity data of HMEs from the open ChEMBL database [16]. HMEDB includes four groups of histones modifying enzymes, a total of 56 different protein targets, and 97 363 known active and 130 114 inactive compound-protein data points. As far as we know, this is the first dataset for the activity prediction task of HMEs.

Second, meta-learning [17] is a new learning paradigm for few-shot sample scenarios, focused on deriving common prior knowledge across related learning tasks, to rapidly adapt to a new learning task with the prior and a small amount of training data. Since HMEs in the same family have similar substrates and catalytic mechanisms, inhibitors of related HMEs may share the same chemical space. Therefore, HME inhibitor prediction is suitable to be formulated as a meta-learning problem, where each task is constructed for an HME target to learn the specific prediction model. From the tasks for the HME target with sufficient training samples, the meta-learner learns a prior with strong generalization capacity during meta-training, such that it can be easily and quickly adapted to the new tasks of the target with scarce data during meta-testing. To work on the few-shot issue of many HME targets, we established a novel meta-learning-based HME inhibitor predictor MetaHMEI which consists of the following three modules: (a) Self-supervised Pretraining Module, (b) transformer-based representation learning module for compound SMILEs and (c) meta-learning-based few-shot HME prediction module to tackle the aforementioned few-shot samples prediction challenges.

Finally, we conducted extensive experiments to verify the MetaHMEI prediction performance. The experimental results show that MetaHMEI achieves superior prediction performance compared with the baselines in terms of AUC under the few-shot samples. In particular, to further evaluate the performance of our proposed model for histone inhibitor screening, we conducted a case study on a specific HME, i.e. JMJD3. MetaHMEI was used to screen 500 000 compounds on TopScience Database and get 5000 candidates, followed by structure-based virtual screening. Finally, 15 compounds were selected for the enzymatic inhibitory test, resulting in the discovery of three new JMJD3 inhibitors with the IC₅₀ < 10 μM. It demonstrated the effectiveness and feasibility of the proposed method in screening large compound libraries for HME inhibitors, especially for the targets with the few-shot issue.

MATERIALS

The dataset was collected and curated from the ChEMBL database (https://www.ebi.ac.uk/chembl/CHEMBL). We focused on the compound-target activity data of histone-modifying enzymes, including HMT, histone acetylases, histone demethylases (HDM) and HDAC. The primary dataset was processed in the following steps.

(i) Remove the records with no SMILES and no ‘Standard Value’, as well as those with the ‘Standard Type’ other than Inhibition, IC₅₀, EC₅₀, K_i and K_d.

(ii) Define the active compounds as with the IC₅₀ / EC₅₀ / K_i / K_d ≤ 10 μM, and the inactive compounds are with IC₅₀ /EC₅₀/ K_i/K_d > 10 μM. For the records with the ‘standard type’ as Inhibition, we checked the ‘Assay Description’ for the compound concentration to identify them as active or inactive. Then, the records cannot be classified into active or inactive (e.g. IC₅₀ > 5 μM or Inhibition > 70% under 30 μM) and were removed from the dataset.

(iii) For the compounds with multiple activity records, the active or inactive label was decided by the majority rule. If the activity records of the same compound contradicted seriously each other, this compound would be removed.

As a result, the final dataset consists of 56 HME targets, 97 363 active data points and 130 114 inactive data points. The details information on the dataset can be found in Table S1 (see Supplementary Data available online at https://github.com/ljatynu/MetaHMEI/).

METHODS

In this section, we mainly introduce the basic idea and architecture of MetaHMEI to solve the problem of few-shot HME inhibitor prediction.

The general framework of MetaHMEI

As shown in Figure 1, the overall architecture of MetaHMEI consists of the following three modules.

(i) Self-supervised pre-training module. MetaHMEI uses the Simplified Molecular Input Line Entry System (SMILES) of compounds as input to train the prediction model. MetaHMEI first exploited a substructure tokenized method to obtain a substructures vocabulary of SMILES. Then, Mol2vec [18] was used to obtain molecular substructure embeddings from a large unlabeled chemical dataset (e.g. ChEMBL). The learned substructure embeddings convey chemical semantics and ensure to point in similar directions for chemically related substructures.

(ii) Meta-learning-based prediction model. The substructure embeddings were then input into a Transformer-based encoder to produce comprehensive and high-quality vector representation of the compounds (SMILES). Our proposed prediction model adopted a multi-tasks meta-learning framework to learn a well-generalized model from HMEs with sufficient samples, which can fast adapt to target HMEs with limited samples. As MetaHMEI allows effective learning to transfer the prior knowledge learned from HMEs with sufficient samples to HMEs with a small number of samples, the proposed model can produce accurate predictions for HMEs with limited data.

(iii) Target task adaptation module. The target task (i.e. HME prediction task with insufficient samples) adapted the prior learned during the meta-training stage via one or a few gradient steps w.r.t. its support set and finally produces the parameters specific to the target task.

Self-supervised pretraining for substructures of compounds

Mol2vec is an NLP-inspired technique that considers compound substructures derived from the Extended Connectivity Fingerprint (ECFP) [19] as ‘words’ and compounds as ‘sentences’. By application of the Word2vec algorithm [20] on a corpus of compound sequences (e.g. ChEMBL), low-dimensional dense embeddings of substructures are obtained, where the vectors for chemically related substructures direct the same direction of vector space. Mol2vec is a self-supervised method that is initially trained on unlabeled data to obtain feature vectors of substructures, which can be summed to obtain compound vectors.

Specifically, a total of 1 938 745 canonical SMILES sequences are collected from ChEMBL [16] as a corpus of compound sequences for self-supervised learning. A vocabulary including 65 different substructures was obtained by ECFP on the corpus of compound sequences. The SMILE of each compound is divided into substructures using ECFP and obtains substructure list |$L=\left\{{s}_1,{s}_2,\dots, {s}_l\right\}$|⁠. The model parameters in Mol2vec include two transformation matrices denoted as |${\mathrm{W}}_{V\times d}$| and |${\mathrm{W}}_{d\times V}^{\prime }$| (⁠|$V$| is the size of various substructures, and |$d$| is the size of the dimension of embeddings), respectively. For each substructure |${s}_t$|⁠, we aim to predict the substructure |$\left\{{s}_{t-k},{s}_{t-k+1},\dots, {s}_{t-1}\right\}$| and the substructure |$\left\{{s}_{t+1},{s}_{t+2},\dots, {s}_{t+k}\right\}$| (i.e. Skip-gram model). The matrices |${\mathrm{W}}_{N\times d}$| and |${\mathrm{W}}_{d\times V}^{\prime }$| are learned through back-propagation of the loss function. |${\mathrm{W}}_{N\times d}$| (or |${\mathrm{W}}_{d\times V}^{\prime }$|⁠) is considered to be as the embeddings matrix for substructures.

Meta-learning-based HMEs prediction model

Based on the embedded representation of the substructure, we construct a compound representation learning model based on Transformer [21]. Specifically, let |${\mathrm{e}}_i$| be the embedding for the substructure |${s}_i$|⁠. If the substructure does not exist in the vocabulary of substructure or is a placeholder, zero vector |$0$| is used as its embedding. Self-attention mechanism [21] is applied to capture the interaction information between substructures, i.e.

where |$\mathrm{E}=\left[{\mathrm{e}}_1,{\mathrm{e}}_2,\dots, {\mathrm{e}}_l\right]$| is the embedding representations of all substructures. |${\mathrm{W}}_K$|⁠, |${\mathrm{W}}_Q$| and |${\mathrm{W}}_V$| are three learnable feature transformation matrices. Then, the attention coefficient between the compound substructure is calculated as the weight, and the substructure eigenvector after transformation is weighted summed

where |${\mathrm{E}}^{\prime }$| is the embedding matrix considering the interaction between substructures. The final embedding representation of the compound is obtained by summing the embeddings of the substructure of the compound.

First, we introduce our HME active prediction model. For each HME, a simple multi-layer MLPs are adopted as task prediction model, i.e.

where |$x$| is the final embedding representation of the compound, |$\sigma$| denotes the activation function (we used RELU here) and |${W}_i$| and |${b}_i$| are learnable parameters of MLP. Finally, we pass |$x$| into the predict function, output the prediction result |${y}^{pre}$|

where |${\mathrm{\omega}}_{pre}$| and |${b}_{pre}$| are learnable parameters of predict function and |$\sigma$| denotes the activation function (we used Sigmoid here).

Due to the lack of sufficient samples for model training, the prediction performance of the HMEs with insufficient samples will be degraded. The data sparsity thus raises a challenge for the effective prediction of HMEs. To alleviate the data scarcity problem, in this paper, we propose a novel meta-learning approach, named MetaHMEI, for the active prediction of HMEs. MetaHMEI consists of two phases: meta-training and meta-test (few-shot samples adaptation for target task). In the meta-training phase, multiple HMEs with sufficient samples are adopted as meta-training tasks to obtain a well-initialized model which could be fast adapted to a new HME with limited data. In the target task adaptation phase, a few (e.g. <5) known active and inactive samples from a new target HME are used to fine-tune the model on this HME to capture its specific model. With the transferability and fast adaptability between meta-training tasks and the new tasks with few-shot samples, MetaHMEI can be applied to mitigate the data scarcity issue. Before formally describe and define MetaHMEI, we introduce some mathematical notations. Each task |${\mathcal{T}}_k,.$| is constructed for a HME |$k$|⁠. Let |$\mathcal{T}={\mathcal{T}}_{\mathrm{tr}}\cup{\mathcal{T}}_{\mathrm{ts}}$| (⁠|${\mathcal{T}}_{\mathrm{tr}}\cap{\mathcal{T}}_{\mathrm{ts}}=\varnothing$|⁠) be the total tasks set. |${\mathcal{T}}_{\mathrm{tr}}=\left\{{\mathcal{T}}_1,{\mathcal{T}}_2,\dots, {\mathcal{T}}_{\ell}\right\}$| denotes the set of tasks with sufficient samples. |${\mathcal{T}}_{\mathrm{ts}}=\left\{{\mathcal{T}}_{\ell +1},{\mathcal{T}}_{\ell +2},\dots, {\mathcal{T}}_m\right\}$| denotes the set of tasks with few-shot samples. |${\Omega}^{+}$| (⁠|${\Omega}^{-}$|⁠) is the compounds set of inhibitory active (inactive). Each task |${\mathcal{T}}_k=\left\{{S}_{{\mathcal{T}}_k},{Q}_{{\mathcal{T}}_k}\right\}$| for a kinase k consists of a support compound set |${S}_{{\mathcal{T}}_k}$| and a query compound set |${Q}_{{\mathcal{T}}_k}$|⁠, where |${S}_{{\mathcal{T}}_k}\subset{\Omega}_{{\mathcal{T}}_k}^{+}\cup{\Omega}_{{\mathcal{T}}_k}^{-} {Q}_{{\mathcal{T}}_k}\subset{\Omega}_{{\mathcal{T}}_k}^{+}\cup{\Omega}_{{\mathcal{T}}_k}^{-}$| sampled from the set of active or inactive compounds for the HME k, such that the support and query compounds are mutually exclusive, i.e. |${S}_{{\mathcal{T}}_k}\cap{Q}_{{\mathcal{T}}_k}=\varnothing$|

For the convenience of description, we use |$\mathrm{\theta}$| to represent all the learnable parameters in MetaHMEI which includes both the parameters in compound representation learning and prediction model. MetaHMEI consists of two following phases. (i) Meta-training Phase (⁠|${\mathrm{\theta}}^{\prime}\leftarrow \mathrm{MT}\left({\mathcal{T}}_{\mathrm{tr}}|\mathrm{\theta} \right)$|⁠). Starting with random initializing parameters |$\mathrm{\theta}$|⁠, the learned meta parameters |${\mathrm{\theta}}^{\prime }$| are obtained by the meta-training algorithm |$\mathrm{MT}\left(\bullet \right)$| using head tasks |${\mathcal{T}}_{\mathrm{tr}}$| as training tasks. The parameters |${\mathrm{\theta}}^{\prime }$| learned by the |$\mathrm{MT}\left(\bullet \right)$| contain the prior knowledge of all head tasks which is desired to be generalized to all target tasks. For each head task |${\mathcal{T}}_k=\left\{{S}_{{\mathcal{T}}_k},{Q}_{{\mathcal{T}}_k}\right\}\in{\mathcal{T}}_{\mathrm{tr}}$|⁠. The meta-learner adapts the global prior |$\mathrm{\theta}$| to task-specific parameters |${\mathrm{\theta}}_{{\mathcal{T}}_k}^{\prime }$| w.r.t. the loss on the support set |${S}_{{\mathcal{T}}_k}$|

The loss function |$\mathcal{L}$| is defined as a cross entropy between label and the predicted output described by Equation (4). Equation (5) is called the inner-loop update process of meta-training. The |$\alpha$| is the inner-loop learning rate and is fixed as a hyperparameter and shared by all meta-training tasks. For simplicity of notation, one gradient update is shown in Equation (5), but using multiple gradient updates is allowed as well.

For each query set |${Q}_{{\mathcal{T}}_k}$|⁠, the loss under the task-specific parameters |${\mathrm{\theta}}_{{\mathcal{T}}_k}^{\prime }$| is calculated and the backward propagation is exploited to update the global |$\mathrm{\theta}$| using the loss sum of all meta-training tasks.

Equation (6) is called the outer-loop update process of meta-training where |$\beta$| is called the outer-loop learning rate which is fixed as a hyperparameter. Algorithm 1 describes the complete procedure of meta-training.

Algorithm 1. Meta-training of MetaHMEI

Input: the HME tasks with sufficient samples, denoted by |${\mathbf{\mathcal{T}}}_{\mathrm{tr}}=\left\{{\mathcal{T}}_1,{\mathcal{T}}_2,\dots, {\mathcal{T}}_{\ell}\right\}$|⁠.

Output: the meta parameters |${\boldsymbol{\mathrm{\theta}}}^{\prime }$|⁠.

1: Initialize the meta parameter |$\boldsymbol{\mathrm{\theta}},$|and the learn rates |$\alpha, \beta$|⁠.

2: while not converge do

3: Sample batch of tasks |$\mathcal{B}$| from |${\mathcal{T}}_{\mathrm{ts}}$|⁠;

4: for each |${\mathcal{T}}_k\in \mathcal{B}$|do

5: Obtaining the parameters specific to the task |${\mathcal{T}}_k$| by the process of gradient descent:

|${\boldsymbol{\mathrm{\theta}}}_{{\mathcal{T}}_k}^{\prime}\leftarrow \boldsymbol{\mathrm{\theta}} -\alpha{\nabla}_{\boldsymbol{\mathrm{\theta}}}\mathcal{L}\left({S}_{{\mathcal{T}}_k}|\boldsymbol{\mathrm{\theta}} \right)$|⁠;

6: end for

7: Update the meta parameters via gradient descent:

|${\boldsymbol{\mathrm{\theta}}}^{\prime}\boldsymbol{\leftarrow}\boldsymbol{\mathrm{\theta} } -\beta{\nabla}_{\boldsymbol{\mathrm{\theta}}}{\sum}_{{\mathcal{T}}_k\in{\mathcal{T}}_{\mathrm{tr}}}\mathcal{L}\left({Q}_{{\mathcal{T}}_k}|{\boldsymbol{\mathrm{\theta}}}_{{\mathcal{T}}_k}^{\prime}\right)$|⁠;

8: end while

9: output|${\boldsymbol{\mathrm{\theta}}}^{\prime }$|

(ii) Few-shot Adaptation Phase (⁠|${\mathrm{\theta}}_j^{\prime \prime}\leftarrow \mathrm{apt}\big({\mathcal{T}}_j|{S}_{{\mathcal{T}}_j},{\mathrm{\theta}}^{\prime}\big)$|⁠). For each tail task |${\mathcal{T}}_j\in{\mathcal{T}}_{\mathrm{ts}}$|⁠, the support set |${S}_{{\mathcal{T}}_j}$| still contains a small number of active and inactive compounds for HME |$j$|⁠. The MetaHMEI adapts the prior |${\mathrm{\theta}}^{\prime }$| learned during meta-training stage via one or a few gradient steps w.r.t. its support set |${S}_{{\mathcal{T}}_j}$| and finally obtains the parameters|${\mathrm{\theta}}_j^{\prime \prime }$| specific to the task |${\mathcal{T}}_j$|⁠, i.e.

Now, each few-shot HME prediction task |${\mathcal{T}}_j$| corresponds to the specific model parameters |${\mathrm{\theta}}_j^{\prime \prime }$|⁠, which includes the parameters of compound representation learning and prediction model. Active compounds specific to the HME |$j$| can be predicted using this model.

EXPERIMENTAL RESULTS

In this section, we conduct extensive experiments on collected datasets of HME to compare performances of different models and analyze and discuss the result of the experiment.

Experimental setup

We carry on experiments on three datasets of histone-modifying enzymes (HDM, HDAC and HMT, as the number of HAT tasks available for experimental is too small, we exclude them in the validation experiment) to evaluate the proposed method. In the experiment, we selected several HMEs with sufficient samples in an enzyme family as the meta training task, and several HMEs with insufficient samples as the meta test task. In each task, the relation-ship between compounds and HMEs can be divided into active instances and inactive instances. An active instance means that this compound is active on this HME, and an inactive instance means that this compound is inactive on this HME, and each compound is uniquely represented by its SMILES. The details of experimental datasets are listed in Table 1.

We conduct experiments ten times with ten random seeds based on 10-fold cross validation (10-CV) for each task and calculate the average of AUC to evaluate the performance of each task. The closer the AUC is to 1, the better the prediction performance of the task. According to experimental result, we set the gradient descent step size |$\alpha$| of support set to 0.004, and set the gradient descent step size |$\beta$| of query set to 0.0001. The support set size is set as 5 and the query set size as 256 for all meta-training tasks.

A three-layer MLPs on all tasks of the HDM dataset, and a two-layer MLPs on all tasks of all other datasets. The values of the transformation matrix of the self-attention mechanism are initialized to 1.

The experimental code is implemented based on the opensource machine learning framework Pytorch (https://pytorch.org). All experiments are carried on Windows 10 operation system with a Dell Precision T5820 workstation computer of an intel W-2245 8 cores 3.91GHz CPU and 128G memory and a NVIDIA TITAN RTX 24G GPU.

Baselines

The following two recent methods about molecule properties prediction are used as baselines to compare the performance with our proposed method.

(i) MolTrans [22]: it regards compounds and proteins as linear structures and makes full use of massive un-labeled bio-medical data to learn substructure information through frequent continuous subsequences (FCS). It constructs an augmented transformer encoder to get interaction matrices between compounds and proteins; finally, the prediction results of the relationship between compounds and proteins are obtained by extracting information from the interaction matrixes through CNN [23].

(ii) MetaMGNN [24]: the first meta-learning-based method for molecule properties prediction. It regards compounds as graph structures and learns the latent representation of compounds through the pre-trained graph neural network (preGNN) [25]. Learning the parameters of a model through gradient-based meta-learning algorithm MAML [17]. In order to enhance the meta-training process, it used a self-supervised module to make full use of the unlabeled information in the molecular graph.

In order to comprehensively evaluate the performance of MetaHMEI, in addition to two recent research works [22, 24] published by others as comparison methods, we also designed and implemented two variant methods of MetaHMEI named MetaECFP and TransferHMEI to compare their prediction performance with MetaHMEI.

(iii) MetaECFP: it is based on extended connectivity fingerprints (ECFP) of the Morgan algorithm, and encodes heavy atoms (i.e. non-hydrogens) into multiple annular layers of a given diameter. In MetaECFP, the ECFPs of compounds are obtained by invoking rdkit (www.rdkit.org) and multi-layer MLPs are used as the active predictive classifier. Then, HMEs with sufficient samples are used as meta-training tasks to train multi-layer MLPs to obtain meta parameters. The model of the target task with insufficient samples is obtained by adapting the meta parameters via small samples. It is worth noting that the major difference between MetaECFP and MetaHMEI is that the two methods use different compound representations, i.e. MetaECFP simply uses ECFP fingerprints whereas MetaHMEI uses pretraining and Transformer based compound representations.

(iv) TransferHMEI: It uses the same pre-training and Transformer based compound representation learning model as MetaHMEI. However, TransferHMEI is a model based on transfer learning [26], that is, the model is first pre-trained on the tasks with a sufficient number of samples, and then, the model is adapted to the small samples to obtain the target task model.

Factors influencing the performance of TrustworthyCPI

Three hyperparameters are used in MetaHMEI, (1) |$r$| is used to control the radius around each atom to be regarded as a substructure in the process of substructure division. When r is equal to 0, each atom itself is considered a substructure. (2) |$s$| is used to control the size of the support set in each iteration during the meta-training process. (3) |$q$| is used to control the size of the query set in each iteration during the meta-training process. Experiments are carried out on HMEDB to investigate the effects of these factors on the performance of MetaHMEI.

We fix the parameters |$s=5$|⁠, |$q=256$|⁠, and set |$r$| to 0,1,2,3, respectively, and conduct experiments on the HDM, and HDAC dataset. The experimental results are shown in Figure 2. The y-axis represents the average AUC of the model on five test tasks in the HDM dataset and seven test tasks in the HDAC dataset under different |$r$| values. From Figure 2, we can see that when the division radius of our substructure is set to 0, that is, when each atom itself is regarded as a substructure, the model has the worst performance on HDM and HDAC, and when the radius is set to 1, the best performance is achieved both on HDM and HDAC.

We fix the other parameters |$r=1$|⁠, |$q=256$|⁠, and set |$s$| to 5, 10, 30, 50, 70, respectively, and conduct experiments on the HDM, HDAC dataset. The experimental results are shown in Figure 3. The y-axis represents the average AUC of the model on five test tasks in HDM dataset and seven test tasks in HDAC dataset under different |$s$| values From Figure 3, we can see that the performance of the model on HDM and HDAC decreases gradually with the increase of the size of the support set.

We conduct experiments on HDM, HDAC dataset with the parameters |$r=1$|⁠, |$s=5$|⁠, and |$q$| as 16, 32, 64, 128, 256, 384, respectively. The experimental results are shown in Figure 4. The y-axis represents the average AUC of the model on five test tasks in HDM dataset and seven test tasks in HDAC dataset under different |$q$| values. From Figure 4, we can see that the performance of the model on HDM and HDAC gradually increases with the increase of the size of the query set. When |$q=256$|⁠, the performance reaches the maximum value, and then decreases with the continuous increase of the size of the query set.

Predictive performance of MetaHMEI

In these experiments, we compare the predictive performance of MetaHMEI with that of other baselines on three groups of HMEs, i.e. HDM, HDAC and HMT where for HDM, HDAC and HMT 5/5, 7/7 and 6/6 meta-training tasks/few-shot test tasks (as described in Table 1) are, respectively, used to train the model and evaluate the performance. We compare the prediction performance of all methods in two few-shot sample cases, i.e. 1-shot and 5-shot. In 1-shot case, the task model only uses one active sample and one inactive sample to adapt the meta model (or pre-training model). In 5-shot case, the task model only uses five active samples and five inactive samples to adapt the meta model (or pre-training model).

As shown in Tables 2 and 3, we can see that our proposed MetaHMEI achieves obviously better predictive performance than other baselines in metrics of AUC under two few-shot adaptation cases. Especially, in 1-shot case, our method is 1.82%, 5.75% and 4.89% better than the best comparison baseline MetaMGNN [23] on HDM, HDAC and HMT task groups, respectively. In 5-shot case. Our method is 4.85%, 6.85% and 3.06% better than the best comparison baseline MetaMGNN on HDM, HDAC and HMT task groups, respectively.

DISCUSSION

The satisfactory performance of MetaHMEI can be explained by the following factors. First, we can see from the comparison results that the models based on effective knowledge transfer between tasks achieve better prediction results than the model without knowledge transfer (i.e. MolTrans versus other methods). Second, in the case of limited samples, the meta-learning model using ‘learn to learn’ has better generalization performance than the transfer learning model using ‘learn then fine-tune’ as seen from the comparison results between MetaHMEI and TransferHMEI. Finally, as shown in comparison result of MetaHMEI versus MetaECFP and MetaMGNN, both the high-quality molecular substructure embedding obtained on a large number of unlabeled compounds through the self-supervised pre-training method and the compound feature representation learned by the Transformer encoder also effectively improves the prediction performance of MetaHMEI.

Case study

JMJD3 (KDM6B), as a member of the jumonji family of HDM, is responsible for removing the trimethylation on histone H3 lysine 27 (H3K27me3-specific demethylase). The function of JMJD3 is correlated with human diseases such as immune system diseases, cancer, infectious diseases, etc., and JMJD3 has emerged as a potential therapeutic target, especially for inflammation and auto-immune diseases [27, 28]. Although highly efficient inhibitors GSK-J1/J4 has been reported in 2012, achieving the half maximum inhibitory concentration (IC₅₀) against JMJD3 of 60 nM, there are less than 100 inhibitors (IC₅₀ < 10 μM) recorded in the ChEMBL database, and still no JMJD3 inhibitor has been approved for human disease therapeutics [29]. In this case, the discovery of diverse and effective JMJD3 inhibitors is essential for the development of drug leads targeting this enzyme, as well as the further understanding of the function of HDM.

To further evaluate the validity of our proposed model for histone inhibitor screening, we conduct a case study on JMJD3, and the screening process is shown in Figure 5.

We adapted the MetaHMEI on the small number of known samples targeting JMJD3 and obtained a model for JMJD3 inhibitor prediction. The model was then used to predict the JMJD3 inhibitory activity of 500 000 compounds in the TopScience database. Then, 5000 compounds with the highest prediction score were selected for molecular docking (detail was presented in the supporting information.) To ensure the structural diversity of the compounds selected for the biochemical experiment, the top 5% ranked molecules among the 5000 candidates (250 molecules) were clustered into 40 classes according to ECFP6 by Discovery Studio. Then, 0–2 molecules were selected from each class, resulting in 15 compounds bought from the TopScience Co. Ltd. for inhibitory test against JMJD3.

The enzymatic inhibitory activity of the selected compounds against JMJD3 was tested by ChemPartner Co. Ltd., Shanghai, People’s Republic of China. For preliminary screening, the inhibition rates at single compound concentrations (50 μM) were tested in duplicates. For IC₅₀ estimations, five concentrations were measured using AlphaLisa assay for each compound, with a starting point of 50 μM and gradient 5-fold dilution. GSK-J1 was used as the reference compound. The raw data of enzymatic inhibitory assay were provided as supporting information (Table S1, Figure S1, see Supplementary Data available online at https://github.com/ljatynu/MetaHMEI/).

As shown in Figure 6, compounds 2, 7 and 13 exhibited potential activity with the IC₅₀ of 7.64, 2.99 and 6.27 μM, respectively. The interaction mode between these compounds and JMJD3 was investigated via molecular docking (Figure 7). Compounds 2 and 7 make three hydrogen bonds with K1381, N1400 and R1246 of JMJD3 catalytic pocket, but only one chelating interaction with Co²⁺. Compound 13, although loses the hydrogen bond with R1246, makes the bidentate interaction with Co²⁺, which is like that for the highly efficient inhibitors GSK-J1/J4 [29]. This may explain the better inhibitory activity of compound 13, compared with compounds 2 and 7. It is the first report of the JMJD3 inhibitory activity of these three compounds, providing new scaffolds for the further optimization and development of JMJD3 inhibitors.

CONCLUSION

In this work, we proposed a meta-learning-based prediction method, MetaHMEI, to solve the few-shot problem in HME inhibitor prediction. The experimental results demonstrated that MetaHMEI outperformed existing approaches in predicting kinase-specific phosphorylation sites when training samples are limited.

MetaHMEI treats compounds as sequence structures to learn their eigenvector representation. We make full use of a large amount of unlabeled information and consider the interaction information between substructures. Through meta-learning, our model learns a well-initialized model parameters, which can quickly adapt to inhibitor prediction problem of emerging histones. We conducted sufficient experiments on the dataset we collected to evaluate our model, and compared it with other baselines. The experiments showed that our model was superior to other methods. We also applied our model in the virtual screening process of histone JMJD3 inhibitors and successfully obtained 3 inhibitors, showing the effectiveness of our model.

ACKNOWLEDGEMENTS

We thank anonymous reviewers for their valuable suggestions.

FUNDING

National Natural Science Foundation of China (62262072, 82260694, 81903541); Fundamental Research Project of Yunnan Province (202001BB050052, 202201AT070297, 202201AW070012); Open Foundation of Key Laboratory in Media Convergence of Yunnan Province (220225201); Yun Ling Scholar Project to Wei-Lie Xiao, Project of Yunnan Characteristic Plant Screening and R&D Service CXO Platform (2022YKZY001).

Qi Lu is a postgraduate student at the School of Software, Yunnan University. His research focuses on bioinformatics and machine learning.

Ruihan Zhang is currently an associate professor at the Key Laboratory of Medicinal Chemistry for Natural Resource, Ministry of Education, Yunnan University. Her research interests include computer-aided drug design, cheminformatics and bioinformatics.

Hongyuan Zhou is a postgraduate student at the School of Chemical Science and Technology. His research focuses on computer-aided drug design.

Dongyuan Ni is a postgraduate student at the School of Chemical Science and Technology. His research focuses on computer-aided drug design.

Weilie Xiao is currently a professor at the Key Laboratory of Medicinal Chemistry for Natural Resource, Ministry of Education, Yunnan University. His research focuses on medicinal chemistry and drug discovery.

Jin Li is currently a professor at the School of Software, Yunnan University. His research interests include trustworthy machine learning, meta-learning and bioinformatics.

References

Author notes