DeepRNA-Twist: language-model-guided RNA torsion angle prediction with attention-inception network

Abstract

RNA torsion and pseudo-torsion angles are critical in determining the three-dimensional conformation of RNA molecules, which in turn governs their biological functions. However, current methods are limited by RNA’s structural complexity as well as flexibility, with experimental techniques being costly and computational approaches struggling to capture the intricate sequence dependencies needed for accurate predictions. To address these challenges, we introduce DeepRNA-Twist, a novel deep learning framework designed to predict RNA torsion and pseudo-torsion angles directly from sequence. DeepRNA-Twist utilizes RNA language model embeddings, which provides rich, context-aware feature representations of RNA sequences. Additionally, it introduces 2A3IDC module (Attention Augmented Inception Inside Inception with Dilated CNN), combining inception networks with dilated convolutions and multi-head attention mechanism. The dilated convolutions capture long-range dependencies in the sequence without requiring a large number of parameters, while the multi-head attention mechanism enhances the model’s ability to focus on both local and global structural features simultaneously. DeepRNA-Twist was rigorously evaluated on benchmark datasets, including RNA-Puzzles, CASP-RNA, and SPOT-RNA-1D, and demonstrated significant improvements over existing methods, achieving state-of-the-art accuracy. Source code is available at https://github.com/abrarrahmanabir/DeepRNA-Twist

Introduction

RNA molecules are fundamental to numerous biological processes and their functionality is closely linked to their three-dimensional structures, akin to proteins. The functionality of RNA critically depends on its structural configuration, highlighting the need for precise characterization of its complex three-dimensional conformation. Traditional experimental methods for determining RNA structure, such as NMR [1], X-ray crystallography [2], and cryo-EM [3], while reliable, are often limited by high costs and time-consuming processes.

Central to RNA’s structural complexity are its torsion angles, comprising seven specific angles |$(\alpha , \beta , \gamma , \delta , \epsilon , \zeta \text{ and} \chi )$| along the ribose-phosphate backbone. Additionally, two pseudo-torsion angles |$(\eta \text{ and} \theta )$| provide a simplified represen-tation of the RNA backbone (Fig. 1). These angles are crucial as they dictate RNA’s folding patterns and ultimately its three-dimensional shape, enabling RNA to perform diverse biological roles—from catalysis as ribozymes to regulating gene expression through various noncoding RNA mechanisms. Understanding these torsion angles enhances the capability to design therapeutic agents [4], potentially altering their function to treat diseases at a molecular level. Moreover, RNA torsion angles play a critical role in computational biology beyond structure reconstruction. They are widely used in evaluating and comparing predicted RNA 3D models, where they serve as a foundation for similarity metrics like Mean Circular Quantities (MCQ) and LCS-TA (Longest Common Segment—Torsion Angles) [5]. Their significance extends to fields such as NMR studies, where torsion angles are used to interpret and refine experimental data on RNA conformations.

Predicting RNA torsion angles and structures presents substantial challenges due to RNA’s inherent properties and the limitations of current methodologies. Unlike proteins which have only three backbone torsion angles, RNA’s structure is determined by a more complex system of torsion angles that contribute to its flexibility and dynamic nature, allowing multiple conformations [6]. This complexity is compounded by RNA’s engagement in noncanonical interactions such as Hoogsteen base pairing, base triples, and various loop interactions, vital for its biological functions.

Recent advancements in protein torsion angle prediction have effectively utilized deep learning methods to enhance prediction accuracy [7, 8]. Inspired by this, researchers have employed various deep learning approaches to predict RNA torsion angles as well, albeit this area still remains significantly under-explored. SPOT-RNA-1D [9], the first RNA backbone torsion angle prediction method, utilized a dilated convolutional neural network to predict both torsion and pseudotorsion angles of RNA from single sequence inputs. Currently, the development of language models that can predict RNA structural features solely from sequence data is very limited. RNA-TorsionBERT [10] model leveraged language model approach to predict torsion and pseudo-torsion angles directly from sequence data. Additionally, this model introduced RNA Torsion-A, a scoring function that assesses the quality of predicted RNA structures using the torsion angles generated by the model, further refining the evaluation of RNA structural predictions.

In this study, we present DeepRNA-Twist, a novel deep learning approach for predicting RNA torsion and pseudo-torsion angles directly from sequence data. A key novelty of DeepRNA-Twist is the utilization of embeddings from an RNA language model (RiNALMo [12]), which has contributed to the model’s improved performance. This novel approach to feature representation has not been explored before for RNA torsion angle prediction. Additionally, we have introduced the 2A3IDC (Attention Augmented Inception Inside Inception with Dilated CNN) module that considerably enhanced the prediction capabilities of DeepRNA-Twist compared with the state-of-the-art models. This module combines inception block and dilated convolution with multi-head attention mechanism. The use of dilated CNN allows the model to capture a broader context and long-range dependencies in the sequence data while enhancing computational efficiency by reducing the number of parameters required to achieve a large receptive field. This enables the model to process sequences more efficiently without sacrificing performance. On the other hand, the multi-head attention mechanism further improves the model’s ability to focus on different parts of the sequence simultaneously, enhancing the capture of both local and global structural features. DeepRNA-Twist effectively captures both short- and long-range interactions among nucleotides. We rigorously evaluated DeepRNA-Twist against existing leading approaches on widely recognized benchmark datasets such as RNA-Puzzles [13] and CASP-RNA [14] and SPOT-RNA-1D Dataset [9]. We have employed Mean Absolute Error (MAE) and MCQ as our evaluation metrics. Our results demonstrate that DeepRNA-Twist significantly outperforms existing methods, achieving state-of-the-art accuracy in RNA torsion angle prediction.

Materials and methods

Dataset

To train our model, we have used the training dataset from SPOT-RNA-1D [9]. We then evaluated our model in two independent test datasets: the SPOT-RNA-1D Test dataset and RNA-TorsionBERT [10] Test dataset. Table 1 shows the detailed statistics for the datasets. The maximum length of RNA sequences in Table 1 is important to note because RNA sequence length can impact downstream structural prediction tasks. Longer sequences tend to exhibit increased conformational variability and structural complexity. Including this information thus helps contextualize the diversity of RNA sequences in our dataset and ensures that the model is evaluated across a broad range of sequence lengths. Furthermore, Fig. 2 demonstrates the distribution of all torsion and pseudo-torsion angles of both datasets.

SPOT-RNA-1D dataset

For SPOT-RNA-1D, RNA structures were sourced from the Protein Data Bank (PDB) [15], selecting those with X-ray resolutions finer than 3.5 Å as of 3 October 2020. As described by the authors, these structures were segmented into individual chains using Biopython, then clustered with CD-HIT-EST [16] at an 80% identity threshold to form the training set, with unclustered sequences making up a noncluster set. To refine the data, BLAST-N [17] filtered out sequences with internal or cross-set similarities using an e-value cutoff of 10. The noncluster sequences were subsequently divided into a validation set (VL) and two test sets (TS1 and TS2), ensuring minimal redundancy by constructing covariance models using the INFERNAL tool’s cmbuild and cmsearch programs, applying strict e-value cutoffs to eliminate remote homologs. An additional test set (TS3) was later formed from NMR structures, processed similarly to ensure nonredundancy. The final datasets included 286 RNA chains for training, with 30 for validation and 63, 30, and 54 for TS1, TS2, and TS3, respectively, where the maximum sequence length is 418. Native torsion angles were extracted using the DSSR program [18].

RNA-TorsionBERT test dataset

RNA-TorsionBERT [10] employed a combined test set consisting of two prominent datasets: RNA Puzzles [13] and CASP-RNA [14]. This merged dataset consists of 52 structures—40 from RNA Puzzles and 12 from CASP-RNA. The length of sequences of this dataset ranges from 27 to 720 nucleotides.

While there are limited numbers of RNA sequences in all the datasets, the torsion angle prediction is performed on a per nucleotide basis. As demonstrated in Table 1, our training set has a sizable amount of nucleotides (21 736). Also, all our test sets combined contain a total of 15 445 nucleotides. Therefore the dataset is well poised for deep learning architecture, and the multiple test sets are able to check the generalizability of our model.

Feature representation

DeepRNA-Twist takes an RNA sequence feature vector |$ X = \{x_{1}, x_{2}, \ldots , x_{i}, x_{i+1}, \ldots , x_{N}\} $| as input, where |$ x_{i} $| is the vector corresponding to the |$ i $|th nucleotide of that RNA. Inspired by the success of protein language models in capturing biological patterns and in downstream tasks, we used RiNALMo, an RNA language model developed in [12], to generate input features. We chose RiNALMo because it is the largest RNA language model developed to date, with 650 million parameters and pretrained on 36 million noncoding RNA sequences sourced from multiple databases. RiNALMo generates a sequence of embedding vectors |$ X = \{x_{1}, x_{2}, \ldots , x_{N}\} $|⁠, where each |$ x_{i} \in \mathbb{R}^{d_{\text{RiNALMo}}} $| and |$ d_{\text{RiNALMo}} = 1280 $| represent the features for |$ i $|th nucleotide of RNA sequence.

Overview of DeepRNA-Twist framework

We propose DeepRNA-Twist, a novel deep learning framework designed to predict RNA torsion angles directly from RNA sequences by effectively capturing both local and global structural patterns within the sequences. As shown in Fig. 4, the pipeline begins by encoding each nucleotide with a feature vector derived from RiNALMo [12], which serves as the foundation for all subsequent computations. These feature vectors are first refined through a Transformer Encoder layer that employs multi-head self-attention and feed-forward networks, thereby capturing essential contextual relationships among nucleotides while maintaining stability via residual connections and layer normalization. The refined representations are then processed through two consecutive 2A3IDC modules, which integrate inception blocks, dilated convolutional layers, and attention mechanism. This combination enables the framework to extract multi-scale features and model long-range dependencies effectively. In these modules, parallel convolutional pathways with different dilation rates capture varying levels of local and global information, while the integrated attention mechanisms further refine these features by focusing on the most relevant positional relationships within the RNA sequence. Following these processing stages, the resulting feature maps are passed through a 1D convolutional layer and an additional attention module before being fed into a dense layer with 18 regression nodes with tanh activation function. These nodes predict the sine and cosine values for each of the nine torsion angles per nucleotide. The training objective is to minimize the mean squared error (MSE) between the predicted and true sine and cosine values across all nucleotides. This comprehensive pipeline enables DeepRNA-Twist to robustly predict RNA torsion angles, providing a strong foundation for downstream structural reconstruction and analysis. Next, we present detailed description of different modules of DeepRNA-Twist.

Transformer encoder layer

DeepRNA-Twist incorporates a Transformer Encoder Layer [19] to process the RNA sequence feature vector |$ X = \{x_{1}, x_{2}, \ldots , x_{N}\} $|⁠, where each |$ x_{i} $| is a 1280-dimensional vector corresponding to the |$ i $|th nucleotide, provided by the RiNALMo [12]. Each Transformer Encoder Layer consists of two main components: a multi-head self-attention mechanism and a position-wise feed-forward network. The multi-head self-attention mechanism refines the representation of each nucleotide by attending to all other nucleotides in the sequence. For each nucleotide |$ x_{i} $|⁠, the self-attention mechanism produces an intermediate representation |$ \mathbf{a}_{i} $| that captures contextual dependencies. Following the self-attention mechanism, each intermediate representation |$ \mathbf{a}_{i} $| is independently passed through a position-wise feed-forward network, resulting in a transformed representation |$ \mathbf{z}_{i} $|⁠. Formally, this can be expressed as

where |$\text{FFN}$| represents the feed-forward network. Each sub-layer in the encoder, including the self-attention and feed-forward networks, incorporates residual connections followed by layer normalization. This configuration stabilizes the learning process and enhances the model’s ability to capture complex dependencies across the RNA sequence. The overall transformation of |$ x_{i} $| through the encoder can be summarized as

where |$\mathbf{a}_{i}$| is obtained from the multi-head self-attention mechanism applied to |$ x_{i} $|⁠. This process results in the refined representation of the initial embeddings, which captures both local and global structural information of the RNA sequence.

Multi-head attention module

The multi-head attention module, denoted by Multi-HeadAttention(.), is designed to dynamically weigh the importance of different elements in the input data, adjusting the focus based on the input’s context. This module integrates a positional encoding sub-module, which provides crucial positional information to enhance the model’s ability to capture sequential relationships.

Positional encoding: to encode positional information [19], the positional encoding function |$\text{PE}_{p}$| for a position |$p$| is defined as

where |$i$| is the dimension. This allows the model to learn to attend to relative positions. The input |$\mathbf{X}$| is augmented with the positional encoding, resulting in a new representation:

This new representation |$\mathbf{X}_{\text{pos}}$| retains both the information from previous layers and the positional information of each element.

Projection to Query, Key, and Value

The input sequence feature vector |$ \mathbf{X} = \{x_{1}, x_{2}, \ldots , x_{N}\} $| is first augmented with positional encoding to obtain |$ \mathbf{X}_{\text{pos}} $|⁠. This augmented input is then linearly transformed to create three matrices: Query (⁠|$\mathbf{Q}$|⁠), Key (⁠|$\mathbf{K}$|⁠), and Value (⁠|$\mathbf{V}$|⁠):

where |$\mathbf{W}^{Q}$|⁠, |$\mathbf{W}^{K}$|⁠, and |$\mathbf{W}^{V}$| are learnable parameter matrices.

Attention calculation

The multi-head self-attention mechanism allows the model to jointly attend to information from different representation subspaces. The input is split into |$ h $| heads, and the attention scores are computed for each head using the scaled dot-product of |$\mathbf{Q}$| and |$\mathbf{K}$|⁠, followed by scaling and masking. For the |$ i $|th head:

The attention scores (⁠|$\mathbf{A}_{i}$|⁠) are computed as

where |$d_{k}$| is the dimension of the keys. The output for each head (⁠|$\mathbf{O}_{i}$|⁠) is then computed by

The outputs from all heads are concatenated to form the final output of the multi-head attention mechanism:

The concatenated output is then linearly transformed:

where |$\mathbf{W}^{O}$| is a learnable parameter matrix. The resulting output is then subjected to dropout and batch normalization to enhance training stability and performance.

By integrating positional encoding and using multiple attention heads, the attention module can effectively capture both the content and positional relationships within the input, making it suitable for processing RNA sequences.

2A3IDC module

The 2A3IDC module integrates inception blocks, dilated convolutional layers, and attention mechanisms to process input data through two parallel paths, enhancing the model’s ability to capture both local and global patterns. This design builds on the 2A3I (Attention Augmented Inception Inside Inception) module that was very successful in protein secondary structure prediction [20]. Each path in the 2A3IDC module begins with an inception block (Fig. 3), consisting of four parallel convolutional pathways, to capture multi-scale features. Following the inception block, a dilated convolutional layer is added. Given an input feature map |$ X $|⁠:

where |$d$| is the dilation rate, allowing the model to capture long-range dependencies efficiently without increasing the number of parameters. The two parallel paths of the 2A3IDC module use different dialations rates: 2 and 5. The output from the dilated convolutional block is then passed through a multi-head attention module. This mechanism focuses on different parts of the sequence simultaneously, enhancing the model’s ability to capture both local and global structural features. Finally, the outputs from both parallel paths (⁠|$X_{Attention1}$|⁠, |$X_{Attention2}$|⁠) are concatenated and batch-normalized to form the final representation:

Training details

The loss function used to train DeepRNA-Twist is the MSE between the predicted and true sine and cosine values of the torsion angles across all nucleotides of all the RNAs. Let |$N$| be the number of RNAs, |$L_{n}$| the number of nucleotides in the |$n$|th RNA in the dataset. Let |$A = 9$| be the number of torsion angles per nucleotide. Each angle has both sine and cosine values, leading to |$2A = 18$| values per nucleotide. The MSE loss function is defined as

where |$ y_{n, l, a} $| and |$ \hat{y}_{n, l, a} $| are the true and predicted sine or cosine values corresponding to the torsion angles for the |$ l $|th nucleotide in the |$ n $|th RNA. Here, |$ a $| ranges from 1 to 18, indexing the sine and cosine values for each of the nine torsion angles. We trained DeepRNA-Twist for 120 epochs. We used the Adam optimizer with a learning rate of 0.0001.

Evaluation metric

Mean absolute error

The predictive performance of RNA torsion angle is quantitatively assessed using the MAE. MAE is calculated for each torsion angle across all nucleotides in the sequence, considering the periodicity of the angles. The MAE for a specific torsion angle |$ \theta $| is defined as follows:

where |$ \Delta \theta _{i} = | \theta _{\text{pred}, i} - \theta _{\text{true}, i} | $| is the absolute difference between the predicted angle |$ \theta _{\text{pred}, i} $| and the true experimentally determined angle |$ \theta _{\text{true}, i} $| and |$ N $| is the total number of nucleotides in the RNA sequence for which torsion angles are predicted. Lower MAE indicated better prediction.

Mean circular quantities

The MCQ is used to measure the angular similarity between actual and predicted RNA structures, |$S$| and |$S^{\prime}$|⁠, respectively, based on their torsion angles. When the RNA structure consists of |$n$| residues, its trigonometric representation can be expressed as a matrix containing |$9n$| values of torsion angles |$t_{ij}$|⁠, where |$i = 1, \dots , n$|⁠, |$j = 1, \dots , |T|$|⁠, and |$T$| is the set of torsion angles defined for the structure. To calculate the MCQ between two structures of equal length, the angular differences |$\Delta (t_{ij}, t^{\prime}_{ij})$| for corresponding torsion angles are computed while respecting the periodicity of angular values. The MCQ metric aggregates these differences using the sine and cosine components as follows:

where |$\Delta (t_{ij}, t^{\prime}_{ij})$| is the minimum angular difference defined as

Earlier works, such as SPOT-RNA-1D, only used the MAE metric. We on the other hand have used MCQ in addition to MAE to assess the proposed model and make a comparative analysis with the state-of-the-art models. This makes our analysis more rigorous.

Results

We evaluated our model on test datasets of SPOT-RNA-1D and RNA-TorsionBERT, the results of which are reported below.

Performance on SPOT-RNA-1D dataset

DeepRNA-Twist was trained on train dataset of SPOT-RNA-1D [9] and validated using validation dataset VL. Table 2 shows the performance comparison between DeepRNA-Twist and SPOT-RNA-1D on VL and three independent test sets TS1, TS2, and TS3. In the VL, our model demonstrated a noticeable improvement over SPOT-RNA-1D, showing percentage reductions in MAE that ranged from |$\sim $|10% to 15% across most torsion angles. Particularly, the reductions for the angles |$\alpha $|⁠, |$\eta $|⁠, and |$\gamma $| were among the most significant. This trend of enhancement is almost consistent across the test sets, indicating robustness in handling both standard and pseudo-torsion angles. TS1 displayed similarly substantial improvements with reductions particularly pronounced for the |$\alpha $| and |$\theta $| angles. In TS2, our model excelled particularly in the prediction of |$\gamma $| and |$\theta $|⁠, achieving more than a 15% improvement in MAE compared with SPOT-RNA-1D. TS3, derived from structurally diverse NMR datasets, showed our model’s adaptability with significant reductions in MAE for |$\beta $| and |$\gamma $|⁠. The performance in this set validates our model’s utility in handling RNA datasets generated from different experimental techniques.

The improvements in MAE were statistically significant, as evidenced by |$P$|-values from paired t-tests (Table 3). Moreover, the prediction performance is not sensitive to the length of RNA sequences, as shown in Fig. 5. The MAE remains relatively stable across different RNA length bins, with no significant upward or downward trends. This indicates that DeepRNA-Twist generalizes well across RNA molecules of varying lengths.

Our analysis confirms that torsion angles with narrower distributions, such as |$\delta $|⁠, |$\epsilon $|⁠, and |$\chi $|⁠, tend to be more predictable. As shown in Fig. 2, these angles exhibit sharp peaks and narrowly distributed values, indicating lower variability across RNA structures. This observation aligns with SPOT-RNA-1D [9], which suggests that angles with more constrained distributions are easier to predict. Conversely, angles like |$\alpha $|⁠, |$\zeta $|⁠, and |$\theta $| display broader distributions with widespread values, making them inherently more challenging to predict. Figure 2 illustrates how these angles exhibit greater variability, often spanning a wide range of values. However, as shown in Table 2, our model achieves lower MAE for these more variable angles compared with SPOT-RNA-1D, demonstrating its robustness in handling torsion angles with higher degrees of flexibility.

Performance for nucleotides in various pairing interactions

The performance of our model was evaluated on torsion angle prediction across different RNA interactions within the test datasets TS1, TS2, and TS3. Each dataset was analyzed based on the nucleotides involved in various structural interactions: unpaired bases, lone pairs, pseudoknots, multiplets, noncanonical pairs, and canonical nested base pairs. The predictive accuracy of our model was compared with the SPOT-RNA-1D as shown in Tables 4, 5, and 6.

Analysis revealed that nucleotides involved in tertiary interactions, such as lone pairs, pseudoknots, multiplets, and noncanonical pairs, consistently presented greater challenges in torsion angle prediction across all three test sets. This was evident from the consistently higher MAEs for these interactions compared with canonical nested base pairs, which had the lowest MAEs, indicating their relative predictability due to less structural complexity. In TS1, the unpaired bases showed the most significant challenge in prediction due to more variability and flexibility in structure, with MAEs of 59.55 for |$\alpha $|⁠, considerably higher than those for canonical nested pairs, which had an MAE of 32.02 for the same angle. This pattern was consistent across all angles and other test datasets. TS3, characterized by different experimental derivation techniques (NMR vs. X-ray crystallography for TS1 and TS2), still maintained the general trend where more complex tertiary interactions resulted in higher prediction errors.

Comparative analysis between our model and SPOT-RNA-1D [9] demonstrated that despite the inherent challenges posed by complex interactions, our model achieved consistently lower MAEs across most interactions and torsion angles, indicating an overall improvement in prediction accuracy. Our findings suggest that the complexity of nucleotide interactions significantly influences the difficulty of predicting torsion angles in RNA structures. More specifically, interactions that introduce greater structural complexity and variability tend to result in higher prediction errors. The comprehensive evaluation across multiple test sets confirms the robustness of our model, particularly in its superior performance over the baseline in handling the intricate and variable interactions within RNA structures. We conducted this experiment on RNA-TorsionBERT dataset as well and noticed the same trend (Table 7).

Performance on RNA-TorsionBERT dataset

Table 8 presents the performance of our method, DeepRNA-Twist, in predicting RNA torsion angles compared with various state-of-the-art approaches including RNA-TorsionBERT [10], SPOT-RNA-1D [9], and methods from the recent State-of-the-RNArt [21] benchmark on RNA-TorsionBERT dataset. DeepRNA-Twist demonstrates superior accuracy with the lowest mean MAE of 19.78|$^{\circ }$| across all torsion angles. This represents a significant improvement over RNA-TorsionBERT [10], which records a mean MAE of 22.6|$^{\circ }$|⁠, and a more pronounced advantage over SPOT-RNA-1D (mean MAE of 30.0|$^{\circ }$|⁠). Specifically, DeepRNA-Twist achieves the best performance in predicting |$\delta $| with an MAE of 12.1|$^{\circ }$|⁠, surpassing RNA-TorsionBERT’s MAE of 13.6|$^{\circ }$| and greatly outperforming SPOT-RNA-1D’s 28.3|$^{\circ }$|⁠. The largest improvement over RNA-TorsionBERT [10] is observed in the |$\gamma $| angle, where DeepRNA-Twist reduces the MAE by nearly 4|$^{\circ }$|⁠. Moreover, across other angles such as |$\chi $|⁠, |$\epsilon $|⁠, and |$\beta $|⁠, DeepRNA-Twist consistently maintains lower MAEs compared with all benchmarked methods. We also notice the same trend in the relationship between prediction difficulty and the distribution of angles. Angles associated with wider distribution, such as |$\alpha $|⁠, |$\gamma $|⁠, and |$\zeta $|⁠, are more difficult to predict compared with angles with narrower distribution such as |$\delta $| and |$\epsilon $|⁠. DeepRNA-Twist consistently shows statistically significant improvements in MAE across all torsion angles in this dataset over all other methods (Table 3).

Performance comparison based on MCQ

Table 9 shows the MCQ of DeepRNA-Twist, SPOT-RNA-1D, and RNA-TorsionBERT on both datasets. The MCQ metric quantifies the dissimilarity between the actual and predicted three-dimensional structures. A higher MCQ value indicates greater dissimilarity between the actual and predicted structures, while a lower MCQ value signifies greater similarity. On both datasets, DeepRNA-Twist achieves the least MCQ, which highlights the effectiveness of our approach in predicting RNA three-dimensional structures.

Ablation study

To evaluate the contributions of different components within the DeepRNA-Twist architecture, we conducted an ablation study in two parts: (1) a comparison between RiNALMo embeddings and one-hot encoded features, and (2) an assessment of individual architectural components by systematically removing them. The results are summarized in Tables 10 and 11, which include results for both RNA-TorsionBERT and SPOT-RNA-1D datasets. We first compared the performance of DeepRNA-Twist model using RiNALMo embeddings with a simpler feature encoding approach with one-hot encoded nucleotides. As shown in Table 10, the model using RiNALMo embeddings consistently outperforms the one-hot encoded model on both RNA-TorsionBERT and SPOT-RNA-1D datasets. Furthermore, we also investigated the performance of different RNA language models as shown in Fig. 6. For this experiment, we chose BiRNA-BERT [31] and RNA-FM [32] as the two competing language models. RiNALMo turns out to be the best-performing model on both datasets and thus it validates our design choice.

In the second part of our ablation study, we evaluated the contributions of key components within DeepRNA-Twist architecture by systematically removing them. Each component’s removal allowed us to assess its impact on the overall model performance, as reflected by the mean MAE of all torsion angles. Table 11 presents the results of the component removal experiments. The removal of the 2A3IDC modules led to the highest increase in mean MAE across both datasets, emphasizing their critical role in capturing both local and global dependencies within the sequence. Removing the Transformer Encoder Layer and Multi-head Attention also resulted in performance drops, highlighting their importance in the model’s architecture.

To further investigate the effectiveness of 2A3IDC module, we replaced this module with similar type of architectures (2A3I [20] and 3I [33]) while keeping all the other parts of the network intact. Table 12 records the results, clearly showing the superiority of 2A3IDC module over the other two. In particular, its superior performance over 2A3I is commendable since it has far less parameters, demonstrating its efficiency. This is attributed to its design, which combines Inception networks, dilated convolutions, and multi-head attention. The dilated CNNs capture broader context and long-range dependencies with fewer parameters, while the multi-head attention enhances the capture of local and global features. In contrast, 3I, which lacks an attention mechanism, has the fewest parameters but delivers the worst performance. These results show 2A3IDC’s ability to balance computational efficiency and accuracy. The ablation study confirms that the 2A3IDC modules and language model embeddings are vital, validating our design choices in DeepRNA-Twist.

Case study

In this case study, we started with native RNA structures (as PDB files) with PDB IDs 4R4V and 7PTK and obtained predicted backbone torsion angles using DeepRNA-Twist. These predicted angles were then applied to the corresponding nucleotides of native structures in PyMOL [34] using the set_dihedral function, generating new RNA conformations. The reconstructed structures were subsequently superimposed onto their native PDB structures (see Fig. 7), and the root mean square deviation (RMSD) values were calculated to quantify the accuracy of the predictions. All modifications, visualization, and evaluation were performed within PyMOL. In the superimposed visualizations, the native structures are displayed in cyan, while the predicted structures are shown in magenta, allowing for a direct comparison of the model’s predictions with the experimental data. For 7PTK, the RMSD between the predicted and native structures was 6.59 Å, and for 4R4V, the RMSD was 3.31 Å. We also employed two other torsion angle prediction models—SPOT-RNA-1D and RNA-TorsionBERT and two recent sequence to 3D structure prediction models—AlphaFold3 [23] and RhoFold+ [35] for this experiment as shown in Fig. 8. For the sequence to 3D structure prediction models, the RNA sequence was given as input and the predicted 3D structure was the output, which we used for comparison. These results demonstrate the capability to closely align predicted RNA structures with their experimentally determined counterparts, highlighting the effectiveness of DeepRNA-Twist in RNA torsion angle prediction over the state-of-the-art models.

Discussion

In this study, we introduced DeepRNA-Twist, a novel deep learning approach for predicting RNA torsion and pseudo-torsion angles solely from sequence data. DeepRNA-Twist integrates several key modules, each contributing unique strengths. The transformer encoder captures complex dependencies and contextual relationships, essential for RNA torsion angle prediction. The 2A3IDC modules extract multi-scale features and recognize long-range dependencies while refining focus on critical structural variations through multi-head attention mechanisms. Attention modules throughout the architecture dynamically weigh contributions, ensuring significant features are prioritized. Our evaluations on benchmark datasets such as RNA-Puzzles [13], CASP-RNA [14], and SPOT-RNA-1D [9] demonstrate that DeepRNA-Twist significantly outperforms existing methods, achieving state-of-the-art accuracy. On the SPOT-RNA-1D dataset, the model showed 10% to 15% MAE improvements, depicting its robustness and accuracy. Angles with wider distribution like |$\alpha $|⁠, |$\zeta $|⁠, and |$\theta $| showed notable improvements, whereas angles like |$\delta $| and |$\epsilon $|⁠, which have narrower distribution, were easier to predict. Despite complexities in predicting angles for nucleotides involved in tertiary interactions, DeepRNA-Twist consistently achieved lower MAEs compared with other methods. Further analysis on RNA-Puzzles and CASP-RNA datasets highlights that DeepRNA-Twist outperforms deep learning based state-of-the-art methods, including RNA-TorsionBERT [10], SPOT-RNA-1D [9], and AlphaFold 3 [23], with an overall MAE reduction of more than 10|$^{\circ }$| and a 35% improvement in accuracy compared with template-based and ab initio methods. Moreover, we show the effectiveness of DeepRNA-Twist in predicting RNA three-dimensional structures by using MCQ as evaluation metric. These results highlight the model’s superior performance and generalizability. Despite these advancements, inherent limitations remain. Reducing MAE below a certain threshold is challenging due to RNA structural complexity and variability. Angles like |$\alpha $| and |$\gamma $| with wider distribution are particularly difficult to predict accurately, necessitating continued research, the integration of additional biophysical constraints, and more comprehensive datasets. In our future work, we aim to address these issues to further improve prediction accuracy.

Funding

M Saifur Rahman is partially supported by basic research grant from BUET.

References