BRAN2SAT: Redundant satisfiability logic in Lyapunov-based discrete Hopfield neural network

Abstract

This study introduces a novel non-systematic logical structure, termed B-type Random 2-Satisfiability, which incorporates non-redundant first- and second-order clauses, as well as redundant second-order clauses. The proposed logical rule is implemented in the discrete Hopfield neural network using the Wan Abdullah method, with the corresponding cost function minimized through an exhaustive search algorithm to reduce the inconsistency of the logical rules. The inclusion of redundant literals is intended to enhance the capacity of the model to extract overlapping knowledge. Additionally, the performance of B-type Random 2-Satisfiability with varying clause proportions in the discrete Hopfield neural network is evaluated using various metrics, including learning error, retrieval error, weight error, energy analysis, and similarity analysis. Experimental results indicate that the model demonstrates superior efficiency in synaptic weight management and offers a broader solution space when the number of the three types of clauses is selected randomly.

Graphical Abstract

Open in new tabDownload slide

1 Introduction

Artificial neural networks (ANNs) are mathematical models inspired by the perceptual processing of the human brain. Composed of interconnected neurons, ANNs adjust synaptic weights in response to external inputs, enabling the generation of accurate outputs for specific tasks (Rosenblatt 1958). Nowadays, ANNs have numerous applications, such as in natural language processing (Raza et al. 2024), autonomous driving (Xu et al. 2024), medical image processing (Lee et al. 2023; Azad et al. 2024), financial forecasting (Mousapour Mamoudan et al. 2023; Lee & Kang 2023), and smart manufacturing (Sakkaravarthi et al. 2024; Farahani et al. 2025). One of the earliest ANNs was the Hopfield neural network (HNN) proposed by Hopfield and Tank (Hopfield 1982). This single-layer feedback network lacks hidden layers and features neurons that form loops, allowing outputs to feed back into the input layer. The Discrete hopfield neural network (DHNN), a specific form of HNN, utilizes discrete binary states for neurons and features bidirectional connections through synaptic weights. It is trained with the Hebbian learning rule (Hopfield & Tank 1985), where neurons are iteratively updated until convergence to a near-optimal state. The asynchronous updating process ensures DHNN minimizes an energy function, leading to a stable state suitable for real-world problems (Ma et al. 2023; Yu et al. 2024; Wang et al. 2024a; Deng et al. 2024). However, as the number of neurons increases, the DHNN faces challenges such as local minima and limited storage capacity. Therefore, the structure of DHNN must be optimized and improved. To address the storage capacity limitations of conventional DHNN (approximately $0.15N$⁠, where $N$ denotes the number of neurons) and ensure stable convergence to the global optimum while storing a large number of patterns, Abdullah (1992) first modelled neurons in DHNN using symbolic logic with satisfiability (SAT) structures. Synaptic weights are determined by comparing cost functions derived from logical rules with the final energy function, which have proven superior to the traditional Hebbian learning method (Hopfield 1999). Sathasivam (2010) later advanced this work by introducing Horn satisfiability (HORNSAT) into DHNN. The study implemented neuron relaxation mechanisms to prevent oscillations that could lead to local minima. The study demonstrated that the proposed HORNSAT logic effectively converges to global minimum energy states. This marked the first evaluation of neuron states based on logical rules within DHNN, laying the foundation for subsequent research on systematic logical rules. Building on this foundation, Kasihmuddin et al. (2019) and Mansor et al. (2017) subsequently proposed 2-Satisfiability (2SAT) and 3-Satisfiability (3SAT), respectively. The primary difference between these structures lies in the number of literals per clause: 2SAT includes two literals per clause, while 3SAT involves three. Despite the successful implementation of systematic logic in DHNNs, it consists solely of clauses of the same order, which lead to the overfitting of neuron states during the retrieval phase. Consequently, the model exhibits a lack of generalizability on real-world data sets.

Recent studies have proposed various non-systemic logics to address these limitations. These logics are composed of clauses that are randomly arranged in different orders, offering greater flexibility than system logic. This flexibility enables non-systemic logic to generate a wider diversity of neuron states, enhancing its adaptability to a wide range of real-world problems. Sathasivam et al. (2020) proposed Random k-Satisfiability (RAN2SAT), combining first- and second-order clauses. Although the quality of neuron states declines as the number of neurons increases, RAN2SAT exhibits greater variations in synaptic weights. Building on this idea, Zamri et al. (2022) proposed a new non-systematic logical rule, weighted Random 2-Satisfiability (⁠$r$2SAT). This rule adjusts the logical structure by considering the expected proportion of negated literals (⁠$r$⁠) and utilizes a Genetic Algorithm (GA) to generate suitable $r$SAT structures. In DHNN, $r$SAT with weighted ratios of negated literals improves global minimum rates and ensures the correct application of negations, leading to higher neuron permutations. Inspired by these developments, Abdeen et al. (2023) propose a novel non-systematic logic, termed S-type Random k Satisfiability, which constructs a logical framework based on two given statistical parameters and assigns negative literals within first- and second-order clauses. Reports indicate that when these parameters reach higher values, the network demonstrates significant effectiveness in synaptic weight management and global minimum solution generation. Subsequently, Roslan et al. (2024) further expanded the perspective on non-systematic logic by introducing Conditional Random k Satisfiability, which intentionally omits positive literals in second-order clauses. Studies show that the proposed logical rules, combined with the newly introduced non-monotonic Smish activation function, achieve a diversified set of global minima in DHNN. Non-systematic logical rules in DHNN result in more diverse final neuron states than systematic rules. However, the predetermined combination and proportional distribution of clauses in different orders may constrain the solution spaces. To achieve greater solution diversity, researchers have combined systematic and non-systematic logic to create flexible hybrid logical rules. These hybrid rules randomly generate clauses of different orders, incorporating both systematic and non-systematic characteristics, thereby enhancing logical diversity. Guo et al. (2022) proposed Y-type Random 2-Satisfiability, which randomly assigns first- and second-order clauses. This approach integrates both systematic and non-systematic logic by using a random clause generator, thereby enhancing storage capacity and connection diversity. While systematic logic provides higher storage capacity for DHNN through its clause structures, non-systematic logic offers richer connections for first-order logic, further improving connection diversity. Although these logical rules for DHNN have received significant attention, a common drawback is the lack of redundancy in variables, as all clauses consist of non-redundant literals. Therefore, incorporating redundant literals is necessary. However, introducing redundant literals with opposing values into a logical expression generally results in non-satisfiability logic.

Maximum satisfiability (MAXSAT) is a variant of logical rules that is inherently unsatisfiable. According to Prugel-Bennett & Tayarani-Najaran (2011), MAXSAT aims to maximize the number of satisfiable clauses, the outcome of MAXSAT logic is consistently False. Kasihmuddin et al. (2018) first introduced non-satisfiability logic into DHNN through maximum 2-satisfiability (MAX2SAT), where only satisfiable clauses are considered in the cost function, as synaptic weights for non-satisfiable clauses are set to zero. This MAX2SAT approach, utilizing exhaustive search (ES), reportedly achieves global minimum energy for smaller neuron sets. Building on this, Someetheram et al. (2022) proposed a new non-satisfiability logic, Random maximum 2-satisfiability (RM2SAT), which combines MAX2SAT and Random 2-Satisfiability. RM2SAT is embedded in DHNN, employing the Election Algorithm during the learning phase to find a consistent interpretation that minimizes the cost function and achieves optimal performance in both learning and testing. While MAXSAT has been successfully applied in DHNN, no existing literature has explored the implementation of satisfiability logic with redundant literals in DHNN. Therefore, it is necessary to consider introducing redundant literals in satisfiability logic. On the one hand, logical rules with redundant literals can represent $\frac{n}{2}$ neurons in DHNN with $n$ clauses, simplifying the network structure. In terms of synaptic weights, the synaptic weights between neurons with redundant literal representations change from a single weight value to the sum of two weight values. This adjustment allows the network to more precisely and flexibly modulate the connection strength between neurons, thereby enhancing its adaptability and generalization ability while reducing the risk of getting trapped in local minima. On the other hand, logical rules with redundant literals can update fewer neuron states during retrieval, allowing DHNN to reach stable states with less iteration, thus improving storage capacity and resistance to interference.

As a follow-up to the previously mentioned literature review, it is worthwhile to investigate the following research questions:

• What alternative representations of non-systematic logic can effectively simplify the structure of the DHNN by introducing redundant and non-redundant literals?

• What alternative representations of non-systematic logic can precisely adjust synaptic weights through redundant literals, thereby effectively modelling neuron connections in the DHNN?

• What experimental design and performance metrics are most suitable for validating the effectiveness of embedding non-systematic logical rules with redundant literals into the DHNN during the learning and retrieval phases?

To answer these questions, B-type Random 2-Satisfiability (BRAN2SAT) is proposed by incorporating redundant second-order clauses, as well as non-redundant second- and first-order clauses. The inclusion of redundant second-order clauses enables more diverse connections with fewer neurons in the DHNN and facilitates control over neuron state updates. This work represents the first application of logical rules with redundant literals in DHNN, demonstrating their potential to enhance network performance. The contributions of this study are as follows:

• To propose a novel non-systematic logical rule termed BRAN2SAT. This logic consists of redundant second-order clauses combined with non-redundant first- and second-order clauses. This combination preserves the fault tolerance of redundant clauses and reflects the simplicity and efficiency of non-redundant clauses, which achieves equilibrium between logical complexity and reasoning efficiency.

• To embed the proposed BRAN2SAT into the DHNN, each literal is mapped to the corresponding neuron in the network, forming the DHNN–BR2SAT model. The inconsistencies in BRAN2SAT are then minimized, and the correct synaptic weights of the DHNN are determined by comparing the associated cost function with the energy function. The influence of BRAN2SAT is reflected in the behaviour of the DHNN–BR2SAT through the synaptic weights.

• To conduct extensive experiments to demonstrate the effectiveness of DHNN–BR2SAT. DHNN–BR2SAT is evaluated by considering different clause orders. The performance of the learning and retrieval phases of DHNN–BR2SAT is comprehensively evaluated in terms of the performance metrics of learning error, synaptic weight error, energy distribution, retrieval error, total variation, and similarity metrics to justify the behaviour of the proposed BRAN2SAT.

The rest of the study is structured as follows. Section 2 provides the motivation. Section 3 presents the overview structure of the novel BRAN2SAT. Section 4 explains the implementation of BRAN2SAT in DHNN. Section 5 displays the experimental setup and performance evaluation metrics employed in the whole simulation experiment. Section 6 explains the results, discusses the behaviour of DHNN combined with BRAN2SAT, and addresses the limitations of the model. Finally, Section 7 presents conclusions and future work.

2 Motivation

2.1 The satisfiability logic in the DHNN lacks redundant literals

A significant limitation of DHNN lies in their tendency to become trapped in local minima during energy descent, which hinders the ability of the network to recall stored patterns accurately. Embedding logical rules within DHNN enhances interpretability by guiding neuron update paths. Moreover, the final neuron states of the improved DHNN model offer potential solutions for practical optimization problems. Various types of logical rules have been successfully incorporated into DHNN, including systematic logical rules (Kasihmuddin et al. 2019; Mansor et al. 2017), non-systematic logical rules (Sathasivam et al. 2020), and hybrid logical rules (Guo et al. 2022; Gao et al. 2022). However, these rules are based on non-redundant literals, where each literal appears only once in the logical rules. Since each literal represents a neuron that connects to others as well as to itself, when literals appear only once, the associated neurons exhibit similar synaptic weights, lacking sufficient variability. This can lead to overfitting during the DHNN retrieval phase. To address this issue, the embedding of MAXSAT with two redundant variables of opposing values in DHNN achieves optimal performance in learning and retrieval phases (Kasihmuddin et al. 2018; Someetheram et al. 2022). However, the logic constrains the values and number of redundant variables, limiting their flexibility in regulating synaptic weights. To address these limitations, we propose a satisfiability logic that incorporates redundant literals. This is achieved by randomly generating redundant second-order clauses alongside non-redundant first- and second-order clauses, providing DHNN with more flexible logical rules. This approach provides a novel method for modelling in real-world applications.

2.2 Limited storage capacity of DHNN

The storage capacity of DHNN is relatively limited. Theoretically, the maximum capacity is approximately 15% of the total number of neurons $N$⁠, allowing for up to $0.15N$ patterns to be stored (Hopfield 1982). Exceeding this limit can overload the network, leading to spurious memories or failure to converge correctly. This limitation renders DHNN unsuitable for large-scale pattern storage. To address this issue, Storkey & Valabregue (1999) proposed a weight update method that effectively reduces interference among patterns during training. This algorithm improves upon the Hebbian learning rule, making it better suited for handling more significant numbers of patterns and reducing confusion among similar patterns. However, compared to Hebbian learning, the Storkey rule has higher computational complexity and may be less effective under high-noise conditions. Additionally, Okada (1996) focused on sparse coding strategies that store information by activating only a small fraction of neurons, thereby reducing overlap among patterns and significantly enhancing storage capacity and noise resistance. Building on this, we propose embedding a logical rule with redundant variables in DHNN. By introducing redundant variables into the logical rules, the corresponding neurons possess two weights of opposite signs, which suppresses neuron activation, reduces pattern overlap, and facilitates more accurate pattern retrieval. This characteristic enhances the storage capacity and noise resistance of DHNN to a certain extent.

3 B-type Random 2-Satisfiability

BRAN2SAT is a non-systematic satisfiability logical rule with redundant literals, represented in conjunctive normal form (CNF). The logical structure comprises clauses containing two redundant literals and clauses with at most two non-redundant random literals. Literals are combined using the OR operator ( $\vee$ ), while clauses are connected using the AND operator ( $\wedge$ ). It should be noted that the second-order clauses containing redundant literals are subject to additional restrictions on clause selection (Liberatore 2008). In contrast, no such restrictions apply to clauses with non-redundant random literals. The proposed structure of BRAN2SAT is as follows:

• A set of $x$ literals. $z_1,z_2,\dots ,z_x$⁠, where each takes a value from the set $\{z_i,\mathrm{\neg }{z}_i\}$⁠, representing positive or negative states.

• A set of $y$ clauses. $C_1,C_2,\dots ,C_y$⁠.

  $$\begin{array}{r}x=4v+2m+n,\end{array}$$

  (1)

   

  $$\begin{array}{r}y=2v+m+n,\end{array}$$

  (2)

  where $2v$ represents the total number of redundant second-order clauses, $m$ represents the total number of non-redundant second-order clauses, and $n$ represents the total number of first-order clauses.
• The formulation for Redundancy 2-satisfiability is presented as follows (Liberatore 2008):

  $$\begin{array}{r}{P}_{\mathrm{R}2\mathrm{S}\mathrm{A}\mathrm{T}}={\wedge }_{i=1}^v{C}_i^{\prime }{\wedge }_{i=1}^v{C}_i^{\prime \prime },\end{array}$$

  (3)

  where $v\ge 1$⁠, the clause of $P_{\mathrm{R}2\mathrm{S}\mathrm{A}\mathrm{T}}$ is structured based on $C_i^{\prime }$ and $C_i^{\prime \prime }$ as follows:

  $$\begin{array}{r}{C}_i^{\prime }\in \{(x_i\vee y_i),(\mathrm{\neg }{x}_i\vee \mathrm{\neg }{y}_i)\},\end{array}$$

  (4)

   

  $$\begin{array}{r}{C}_i^{\prime \prime }\in \{(\mathrm{\neg }{x}_i\vee y_i),(x_i\vee \mathrm{\neg }{y}_i)\}\end{array}$$

  (5)

  The literals $x_i$ and $y_i$ are redundant in the formulation, and their states are restricted. In the $P_{\mathrm{R}2\mathrm{S}\mathrm{A}\mathrm{T}}$ formulation, each clause is constructed by selecting one clause from two sets. Specifically, one clause is randomly selected from set $C_i^{\prime }$⁠, which contains the literals $(x_i\vee y_i)$ and $(\mathrm{\neg }{x}_i\vee \mathrm{\neg }{y}_i)$⁠, while another clause is randomly chosen from set $C_i^{\prime \prime }$⁠, which contains the literals $(\mathrm{\neg }{x}_i\vee y_i)$ and $(x_i\vee \mathrm{\neg }{y}_i)$⁠.
• The formulation for Random 2-Satisfiability is given as:

  $$\begin{array}{r}{P}_{\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}={\wedge }_{i=1}^m{C}_i^{(2)}{\wedge }_{i=1}^n{C}_i^{(1)},\end{array}$$

  (6)

  the clause of $P_{\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$ is structured based on $C_i^{(2)}$ and $C_i^{(1)}$ as follows:

  $$\begin{array}{r}{C}_i^{(k)}=\{\begin{array}{cc}(a_i\vee b_i),& k=2,1\le i\le m\\ c_i,& k=1,1\le i\le n\end{array},\end{array}$$

  (7)

  where the literals $a_i$⁠, $b_i$⁠, and $c_i$ are non-redundant, and their states are randomly determined, with $a_i\in \{a_i,\mathrm{\neg }{a}_i\}$⁠, $b_i\in \{b_i,\mathrm{\neg }{b}_i\}$⁠, and $c_i\in \{c_i,\mathrm{\neg }{c}_i\}$⁠. The clauses of $P_{\text{RAN2SAT}}$ are formed by combining literals from two sets, $C_i^{(2)}$ and $C_i^{(1)}$⁠. For each $C_i^{(2)}$ clause (⁠$1\le i\le m$⁠), it is the disjunction of $a_i$ and $b_i$⁠, where each may be the literal or its negation. For each $C_i^{(1)}$ clause (⁠$1\le i\le n$⁠), it is formed by $c_i$⁠, which may also be the literal or its negation. Thus, the overall structure of $P_{\text{RAN2SAT}}$ depends on the combination of these clauses, with the form of each clause determined by the randomly assigned truth values of the literals.
• The general formulation of $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$ incorporates Redundancy 2-Satisfiability and Random 2-Satisfiability as follows:

  $$\begin{array}{r}{P}_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}=P_{\mathrm{R}2\mathrm{S}\mathrm{A}\mathrm{T}}\wedge P_{\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}},\end{array}$$

  (8)

   

  $$\begin{array}{r}{P}_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}={\wedge }_{i=1}^v{C}_i^{\prime }{\wedge }_{i=1}^v{C}_i^{\prime \prime }{\wedge }_{i=1}^m{C}_i^{(2)}{\wedge }_{i=1}^n{C}_i^{(1)},\end{array}$$

  (9)

  where $C_i^{{}^{\prime }}$ and $C_i^{\prime \prime }$ define the redundant second-order clauses, while $C_i^{(2)}$ and $C_i^{(1)}$ define non-redundant second- and first-order clauses, respectively.

The four different examples for $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$ are as follows:

According to Equations 10–13, the bipolar value for each literal of the CNF takes either 1 or −1, representing TRUE and FALSE, respectively (Sathasivam 2010). The logical formula is satisfied (i.e. $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}=1$⁠) if at least one literal in each clause is satisfied. In other words, $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}=1$ requires that all clauses in the formula be satisfied. There are only four possible combinations of redundant second-order clauses in the BRAN2SAT model, implying that these combinations will repeat as the number of redundant second-order clauses $v$ increases. Unlike the previous study RAN2SAT proposed by Sathasivam et al. (2020), where the proportion in Equation 7 is predetermined despite the randomness of the literals, this approach removes randomness from the proportional selection of the number of clauses. In this study, second-order clauses with redundant literals are incorporated, and only the number of non-redundant second- and first-order clauses is restricted to $m\ge 1$ and $n\ge 1$⁠, with no restriction on the proportion of clauses. In this context, $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$ provides a more flexible logical rule in terms of clause structure and is more applicable to real-world scenarios.

4 BRAN2SAT in the discrete Hopfield neural network

4.1 Learning phase

The DHNN is a fully interconnected feedback network consisting of a single layer of binary neurons. The neuron state is typically represented by a bipolar value of +1 or −1. The neuron update formula is as follows (Hopfield 1982):

from Equation 14, $W_{ij}$ represents the synaptic weight between neurons $i$ and $j$⁠, while $S_i$ denotes the state of unit $i$⁠. $\theta$ represents the threshold. Notably, $W_{ij}$ exhibits two key properties: (a) no self-feedback is allowed, as $W_{ii}=W_{jj}=0$⁠, indicating that each neuron cannot provide feedback to itself. (b) Symmetry weights, with $W_{ij}=W_{ji}$⁠, ensure that the interaction between two neurons is reciprocal and equal. The DHNN–BR2SAT model consists of two primary phases: the learning and the retrieval phases. During the learning phase, the Wan Abdullah (WA) method (Abdullah 1994) is applied, where each literal in $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$ is mapped to a corresponding neuron in the DHNN, thereby generating the cost function $E_{P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}}$⁠. The consistent interpretation of $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$ is then obtained to minimize $E_{P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}}$ to zero. It is worth noting that the number of unsatisfied clauses in $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$ is directly proportional to the value of $E_{P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}}$ (Kasihmuddin et al. 2017). Equation 15 below presents the cost function for DHNN–BR2SAT based on the WA method:

where $v$⁠, $m$⁠, and $n$ represent the number of clauses. The formula for $Q_{ij}$ corresponding to the inconsistent interpretation of $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$ is as follows:

where $X$ represents the set of literals $\{x_i,y_i,a_i,b_i,c_i\}$⁠, with $S_X$ assigned based on the polarity of each literal. The synaptic weights are computed by comparing the coefficients of $E_{P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}}$ with the Lyapunov energy function (Equation 21). To ensure the correct synaptic weights for DHNN–BR2SAT, at least one set of neuron states must be identified that satisfies $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$ (Abdullah 1992). The resulting synaptic weights are then stored in a content addressable memory (CAM). CAM functions by using the stored synaptic weights to guide the network toward optimal neuron states during its evolution. Thus, finding a consistent interpretation of $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$ is crucial for DHNN to store memory patterns correctly. The probability of achieving a consistent interpretation of $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$ is given as follows:

where $P$ represents the probability, and $(1-\frac{1}{2^k})$ denotes the probability that a $k$-order clause is satisfiable.

4.2 Retrieval phase

During the retrieval phase, the neuron states are updated asynchronously. This iterative updating process is governed by Equations 18–20, as provided below:

where the local field $h_i(t)$ represents the composite of the weighted input signals $W_{ij}\cdot S_j$ received by neuron $i$ from all other neurons $j$⁠, along with a bias term $W_i$⁠. $h_i(t)$ serves as the input variable in the hyperbolic tangent activation function (HTAF, Rumelhart et al. 1986). $s_i(t)$ denotes the updated state of neuron $i$ at time $t$⁠. It is important to note that when $\tanh (h_i)=0$⁠, the updated neuron state remains the same as its previous state $S_i(t-1)$⁠. This occurs only when $h_i(t)=0$⁠, representing a state of balance where the excitatory and inhibitory signals received by the neuron cancel each other out (Willems 1972).

When the DHNN reaches a stable state, the energy function converges to the nearest local minimum. The Lyapunov energy function $H_{P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}}$ and the global minimum energy $H_{P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}}^{min}$ are given as follows:

where $2v$⁠, $m$⁠, and $n$ represent the total number of redundant second-order, non-redundant second-, and first-order clauses, respectively.

To determine whether the final neuron state corresponds to the global minimum, the following formula is applied:

where $\mathrm{Tol}$ is a predefined tolerance value that determines the convergence criterion for the network.

The stable state of a neural network refers to the condition in which, after a certain point in time, the neuron states no longer change, i.e. $S_i(t+1)=S_i(t)$⁠.

 

A stable state exists in the DHNN only if Theorem 4.1 is satisfied. Under this condition, the DHNN evolves in the direction of decreasing energy until a stable state is reached. The following provides proof that the energy function decreases monotonically.

The Lyapunov energy function for neuron $i$ at time $t$ and $t+1$ is given by:

The change in energy from time $t$ to $t+1$ is given by:

The energy change from $t$ to $t+1$ can be simplified as:

From Equation 18, we obtain $\mathrm{\Delta }{H}_{P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}}\le 0$⁠. Since neuron $i$ represents an arbitrary neuron in the network, and all neurons in the network update their states according to the same rule, the total energy change of the network must be less than or equal to zero, as follows:

It follows that the DHNN evolves in the direction of Lyapunov energy function minimization.

Figure 1 presents the schematic diagram of DHNN–BR2SAT. The model incorporates non-redundant first-, second-, and redundant second-order clauses. Algorithm 1 outlines the pseudo-code of DHNN–BR2SAT with an ES algorithm in the learning phase.

5 Experimental setup

This section evaluates the impact of varying numbers of clauses at different orders on the behaviour of DHNN–BR2SAT through simulation experiments. The evaluation is conducted in three aspects: the learning phase, the retrieval phase, and both total variation and the similarity index. A comprehensive analysis is provided on how different logical orders affect BRAN2SAT performance, along with an evaluation of parameter perturbation.

The experiment was conducted in a Windows 11 Home Edition environment with an i5-1035G4 1.5GHz CPU and 8 GB RAM. MATLAB R2016b was used as the development tool. The detailed parameter settings for the algorithm are provided in Table 1.

To evaluate the efficiency of DHNN–BR2SAT, this section examines the performance of the model across five aspects: learning error analysis and weight analysis during the learning phase, energy analysis, and global solution analysis during the retrieval phase, and similarity index analysis. Tables 2–4 present the parameters associated with all evaluation metrics.

While many performance measures have been proposed and applied to evaluate the accuracy of DHNN, no single metric has been universally established as a standard benchmark. This lack of consensus complicates the comparison of different network models. Consequently, various metrics must be employed for performance evaluation, and it is important to observe whether these metrics provide a consistent performance ranking across different DHNNs. Among these, the mean absolute error (MAE, Zheng et al. 2023) is one of the most direct measures of prediction error. It represents the average of the absolute differences between predicted and actual values, with smaller MAE values indicating better model performance. As a linear metric, MAE assigns equal weight to all individual errors, thereby providing a clear representation of the overall prediction error. In contrast, the root mean squared error (RMSE, Fan et al. 2021) is a fundamental statistical indicator for assessing model performance. By squaring errors before averaging, RMSE assigns greater weight to larger errors. Consequently, RMSE is particularly effective in evaluating the performance of the model when handling extreme data. Compared to MAE, RMSE is more sensitive to larger extreme errors. The mean absolute percentage error (Wang et al. 2021), which expresses errors as percentages, provides improved interpretability. It is especially suitable for evaluating the relative accuracy of model predictions.

During the learning phase, error is measured using ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$⁠, ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$⁠, and ${\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$⁠. ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ represents the average error between the current neuron fitness $f_i$ and the maximum fitness value $f_{NC}$⁠. To assess the presence of extreme errors in the learning process, ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ is applied, which calculates the RMSE between $f_i$ and $f_{NC}$⁠. Additionally, it is essential to evaluate relative error, which is captured by ${\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$⁠. Together, these three metrics enable a comprehensive evaluation of model performance, considering overall, extreme local, and relative errors from multiple perspectives. The formulas are as follows:

The computation of these three equations primarily involves the maximum fitness and the fitness of the current iteration. In this context, fitness is defined as the number of satisfied clauses in the logical formula. Since achieving optimal synaptic weights requires maximizing the adaptation value based on the given logical rules, the ability to obtain these optimal weights reflects the performance of the learning outcomes of the DHNN–BR2SAT model. To evaluate this, ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ (Gao et al. 2022) and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ (Gao et al. 2022) are used to measure the error in the learning results, capturing both the average error and the local extreme error of the weights, respectively. The formulas are as follows:

where $W_i$ and $W_{\mathrm{W}\mathrm{A}}$ represent the current weight and the optimal weight of the neuron, respectively. The parameters used in the learning error analysis and synaptic weight analysis are presented in Table 2.

During the retrieval phase, the retrieval efficiency of DHNN–BR2SAT is evaluated by measuring the energy error of the retrieval process using ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y}}$ (Guo et al. 2022). To further assess the final neuron state upon reaching stability, ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ (Gao et al. 2022) is used to measure the error of the final solution. At the same time, the retrieval capability of DHNN is evaluated using the global minimum ratio (⁠$\mathrm{Z}\mathrm{M}$⁠, Zamri et al. 2022). The formulas are as follows:

where $|H_{\min }-H_f|$ represents the absolute difference between the final energy and the minimum energy, and $N_{tc}$ denotes the total number of iterations for all logical rules. ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ is calculated as the ratio of the number of local minimum solutions to the total number of final solutions, while $\mathrm{Z}\mathrm{M}$ represents the ratio of global minimum solutions to the total number of final solutions. The parameters involved in the retrieval phase are provided in Table 3.

Finally, the quality of each final neuron state is assessed using the similarity index based on the global minimum of the final neuron states. The similarity index reflects the generalization ability of the DHNN–BR2SAT model (Kasihmuddin et al. 2019). The ideal neuron state $S_i^{\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{a}\mathrm{l}}$ is defined as follows:

where $X$ and $\mathrm{\neg }X$ represent the positive and negative literals in each clause, Table 4 lists the variables that influence the value of the similarity index (Mansor & Sathasivam 2021). It is important to note that only the final neuron state of the global minimum solution is considered in the similarity analysis. The Jaccard index $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$ is used to evaluate the similarity between two sets of solutions and is defined as the ratio of the intersection to the union (Amer et al. 2021).

The value of $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$ ranges from 0 to 1. A value closer to 1 indicates a higher similarity between the two sets of solutions, while a lower value signifies greater differences. Total variation (⁠$\mathrm{T}\mathrm{V}$⁠) measures the change between the states of different final neurons. It reflects the degree of variation in the solutions by accumulating the differences between neighbouring states. This reveals whether the model produces diverse solutions during the retrieval process (Karim et al. 2021)

where $J_i^n$ is used to determine whether the two final neuron states are identical. $J_i^n=0$ if the two states are the same, and $J_i^n=1$ if the two states differ. A higher $\mathrm{T}\mathrm{V}$ value indicates that the retrieved global minimum solutions are more distinct from one another, demonstrating greater diversity. By combining these two metrics, the final solutions of the model can be comprehensively assessed in terms of diversity, both from the $\mathrm{T}\mathrm{V}$ and the $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$⁠.

To provide readers with a clearer understanding of the model, the flowchart in Figure 2 illustrates the overall experimental steps of DHNN–BR2SAT. During the learning phase, the ES algorithm (Schuurmans & Southey 2001) is employed to achieve a consistent interpretation of $P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}$⁠, ensuring correct synaptic weights. In the subsequent retrieval phase, random neuron states are input, and the global minimum solution is evolved through the dynamic updating of the network state.

6 Results and discussion

The purpose of this section is to analyse the effect of the number of redundant second-order clauses $v$ and non-redundant second-order clauses $m$ and first-order clauses $n$ on the performance of the proposed BRAN2SAT. Since $v$⁠, $m$⁠, and $n$ are randomly generated, the proportion of clauses will be based on the proportion of redundant second-order clauses $\alpha$ and non-redundant second-order clauses $\beta$⁠; the situation is shown in Table 5.

Table 5 presents three different cases for the BRAN2SAT. In Case I, the number of clauses for all three types is generated randomly. In Case II, the proportion of redundant second-order clauses (⁠$\alpha$⁠) is greater than or equal to 0.5, indicating that redundant clauses represent at least half of the total number of clauses. Case III requires the proportion of non-redundant second-order clauses (⁠$\beta$⁠) to be greater than or equal to 0.5, meaning that non-redundant clauses number for at least half of the total clauses. By constructing Cases I–III, the effects of different proportions of redundant and non-redundant clauses on the overall performance of the DHNN–BR2SAT model are systematically investigated. The following sections provide a detailed analysis of the performance of DHNN–BR2SAT under these three cases.

6.1 Learning phase

The goal of this phase is to evaluate the learning capability of BRAN2SAT with varying clause proportions within the DHNN framework. During the learning process, these models are embedded into DHNN, and an ES algorithm (Schuurmans & Southey 2001) is employed to achieve a consistent interpretation of BRAN2SAT, thereby establishing the correct synaptic weights for DHNN. Figures 3–5 and Tables 6–8 present the results for ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$⁠, ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$⁠, and ${\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ across different DHNN–BR2SAT models. As observed in Figures 3 and 4, ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ exhibit a linear increase with the number of neurons. This trend is attributed to the dependence of the ES algorithm on a trial-and-error method to achieve consistent interpretations of BRAN2SAT, which is only effective with smaller neurons. Thus, optimization algorithms incorporating global and local search operators can be introduced during the DHNN–BR2SAT learning phase to enhance learning efficiency, facilitating consistent interpretations of BRAN2SAT (Kilani 2010). Notably, from Tables 6 and 7, ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ values in Case I fluctuate considerably but remain relatively low. This fluctuation is due to the random allocation of the three clause types within the model, where first-order clauses are the least likely to satisfy SAT allocations compared to other-order clauses (Giráldez-Cru & Levy 2021). Therefore, when the number of randomly assigned first-order clauses in DHNN–BR2SAT is large, the ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ values are relatively high. Conversely, second-order clauses have a higher likelihood of satisfying SAT, leading to lower ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ when their proportion is higher in Case I. The bolded values represent the minimum ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ for the same number of neurons. In Case II, the proportion of redundant second-order clauses exceeds 0.5, which results in a multiplicative increase in the number of redundant second-order clauses as the number of neurons rises. This increase complicates ES retrieval, raising ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ values Massacci & Marraro (2000). In contrast, Case III exhibits a proportion of non-redundant clauses exceeding 0.5. While the total number of second-order clauses in Case III is lower than in Case II, the presence of some redundant clauses improves solution efficiency Heule et al. (2019), resulting in relatively low ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ values. Although errors increase with the number of neurons, the ratio of ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ to ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ remains close to 1:1, indicating that the error growth is uniform and controllable, with no significant outliers affecting model performance (Gerrard et al. 2020). According to Figure 5 and Table 8, ${\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ values in Case I fluctuate considerably, peaking at 31.732691 around the number of neurons of 20, indicating a higher proportion of unsatisfied clauses relative to the total number of clauses (Aribowo & Muslim 2020). When $\mathrm{NN}$ is between 25 and 40, the three models show similar stability, with trends in Cases II and III being closely aligned, primarily due to the higher proportion of second-order clauses.

Friedman rank tests are conducted on ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ for all the numbers of neurons. The Chi-square value is ${\chi }^2=8.6667$⁠, with a degree of freedom of $\mathrm{df}=2$⁠, and $p<0.05$⁠. Therefore, the null hypothesis of no significant difference for the DHNN–BR2SAT is rejected. According to Tables 6 and 7, Case I demonstrates competitive ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ values, with its average rank surpassing that of the other two models. These results statistically confirm the superiority of Case I in terms of ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$⁠.

Figures 6 and 7 and Tables 9 and 10 illustrate the ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ for all DHNN–BR2SAT models. The results indicate that Case II has the highest ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ values. This suggests that when more redundant second-order clauses are present in BRAN2SAT logical rules, DHNN generates more synaptic weights, leading to higher learning errors. However, the additional weight connections enhance information transfer between neurons without significantly increasing network size (Yu et al. 2024). The bolded values in Tables 9 and 10 represent the minimum ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ for the same number of neurons, with most values observed in Cases I and III. This suggests that Cases I and III perform better, as they contain more non-redundant literals and relatively fewer total literals, leading to fewer synaptic weights. Consequently, DHNN requires fewer learning iterations to minimize the cost function (Mézard et al. 2002). Another notable finding is that the trends of ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ are similar to those of ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$⁠. This occurs because, in the DHNN–BR2SAT model, obtaining the correct synaptic weights via the WA method depends on $E{P}_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}=0$ (Chen et al. 2023). Therefore, when ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ are minimized, DHNN retrieves the optimal final neuron states that correspond to the behaviour of the BRAN2SAT model. $E_{P_{\mathrm{B}\mathrm{R}\mathrm{A}\mathrm{N}2\mathrm{S}\mathrm{A}\mathrm{T}}}=0$⁠. Therefore, when ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ are minimized, DHNN retrieves the optimal final neuron states that correspond to the behaviour of the DHNN–BR2SAT model.

Friedman rank tests are conducted on ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ for all the numbers of neurons. The Chi-square value is ${\chi }^2=7.6000$⁠, with a degree of freedom of $\mathrm{df}=2$⁠, and $p<0.05$⁠. Therefore, the null hypothesis of no significant difference for the DHNN–BR2SAT is rejected. According to Tables 9 and 10, Case I demonstrates competitive ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ values, with its average rank surpassing that of the other two models. These results statistically confirm the superiority of Case I in terms of ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$⁠.

6.2 Retrieval phase

After the DHNN–BR2SAT completes the retrieval of logical consistency during the learning phase, optimal synaptic weights are obtained using the WA method (Abdullah 1992) and stored in the CAM. The model then enters the retrieval phase, where its task is to restore random input neuron states to their stored states. The network automatically adjusts the neuron states based on the input until it reaches a stable state. However, due to the tendency of DHNN to generate repetitive states, output overfitting can occur. Therefore, it is necessary to evaluate the performance of the retrieval phase of DHNN–BR2SAT by assessing energy error, retrieval error, and the quality of global minima (Zamri et al. 2022). Figures 8–10 and Tables 11–13 present the $\mathrm{Z}\mathrm{M}$ values, ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y}}$⁠, and ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ for different DHNN–BR2SAT models during the retrieval phase. From Table 11, when $\mathrm{NN}=10$⁠, all models achieve a $\mathrm{Z}\mathrm{M}$ value of 1. This indicates that, with a smaller number of neurons, DHNN effectively manages synaptic weights during the learning phase, enabling the storage of accurate patterns for all global minima. As shown in Figure 8, $\mathrm{Z}\mathrm{M}$ values for all DHNN–BR2SAT models decrease as the number of neurons increases. As more neurons are added, the prevalence of suboptimal synaptic weights in DHNN grows, reducing its ability to retrieve global minima and affecting its computation of optimal neuron states in the local field (Guo et al. 2022). For BRAN2SAT models with different clause proportions, the rate and extent of $\mathrm{Z}\mathrm{M}$ reduction differ. Compared to other models, Case I exhibits the largest fluctuations in $\mathrm{Z}\mathrm{M}$ values, attributed to the random allocation of its first- and second-order clauses. First-order clauses have a lower probability of achieving a consistent interpretation, and suboptimal synaptic weights may cause the final neuron state to become trapped in local minima. Meanwhile, Cases II and III show similar trends in $\mathrm{Z}\mathrm{M}$ values, though Case III maintains relatively higher $\mathrm{Z}\mathrm{M}$ values. Higher $\mathrm{Z}\mathrm{M}$ values reflect lower ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{n}}$ during the learning phase, indicating that optimal synaptic weights promote convergence of neuron states to global minima.

Further examination of ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y}}$ and ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ of DHNN–BR2SAT shows that a close relationship exists between global minima and final energy (Sathasivam 2010). When neuron states converge to global minima, the absolute difference between the final energy and the minimum energy defined in Equation 23 is minimized. As illustrated in Tables 12 and 13, ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y}}$ and ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ values of zero indicate that all final neuron states effectively converge to global minima. From Figures 9 and 10, as the number of neurons increases, energy error and the number of local minima increase across all models. This rise is primarily due to the limitations of the learning algorithm of DHNN, resulting in some suboptimal synaptic weights that trap final neuron states in local minimal energy states (Guo et al. 2022). This is further evidenced by trends in ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}$⁠. Another cause of local minimal energy is suboptimal neuron updates during retrieval, which lead to a higher energy error by generating more unsatisfied clauses. This issue is particularly evident when first-order clauses are embedded in DHNN, as illustrated by high ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y}}$ of Case I at $\mathrm{NN}=20$ in Figure 9. Most of the time, Case II displays higher ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y}}$ and ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ values than other models due to the larger number of clauses in its logical expressions, which produce more suboptimal synaptic weights during the learning phase. In other words, optimal synaptic weights obtained during the learning phase affect all performance metrics during retrieval, emphasizing the importance of maximizing fitness in the learning phase. Notably, Case III performs better than other models, suggesting that introducing an appropriate proportion of redundant second-order clauses in BRAN2SAT positively impacts the performance of DHNN.

Friedman rank tests are conducted on $\mathrm{Z}\mathrm{M}$⁠, ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$⁠, and ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y}}$ for all the numbers of neurons. The Chi-square values for $\mathrm{Z}\mathrm{M}$ and ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ are ${\chi }^2=7.5882$⁠, and for ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y}}$⁠, the Chi-square value is ${\chi }^2=7.3714$⁠, with a degree of freedom of $df=2$ and $p<0.05$⁠. Thus, the null hypothesis of no significant difference for the proposed DHNN–BR2SAT is rejected. According to Tables 11–13, Case I demonstrates competitive $\mathrm{Z}\mathrm{M}$ and ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ values, with its average rank surpassing that of the other two models. Case III shows competitive ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y}}$ values, with its average rank exceeding those of the other two models. These results statistically validate the superiority of Case I in terms of $\mathrm{Z}\mathrm{M}$ and ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ and Case III in terms of ${\mathrm{M}\mathrm{A}\mathrm{E}}_{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y}}$⁠.

6.3 Total variation and similarity analysis

Previous metrics primarily assessed the quantity of DHNN–BR2SAT neuron states, but evaluating the quality of final neuron states is equally essential. This study uses the $\mathrm{T}\mathrm{V}$ and the $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$ to assess the diversity quality of global solutions for each BRAN2SAT model. Both $\mathrm{T}\mathrm{V}$ and $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$ are based on global solution analysis. In other words, when $\mathrm{Z}\mathrm{M}$ equals zero, neither $\mathrm{T}\mathrm{V}$ nor $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$ exists. Higher $\mathrm{T}\mathrm{V}$ values indicate greater diversity among final neuron states in the global solutions (Guo et al. 2022). According to the definition of $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$⁠, it is calculated based on the similarity between final neuron states and reference neuron states. Lower $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$ values reflect more significant differences between final neuron states and reference neuron states (Real & Vargas 1996). As shown in Figure 11 and Table 14, overall, the $\mathrm{T}\mathrm{V}$ values for different DHNN–BR2SAT models decrease with increasing NN and eventually approach zero. This trend reflects a decline in the ability of each model to retrieve global minima. It suggests that the suboptimal performance of the ES algorithm during the DHNN learning phase leads to an increase in local minima, thereby reducing the number of global minima. Case II shows the lowest $\mathrm{T}\mathrm{V}$ value, as it retrieves more local minima due to the suboptimal learning phase associated with the higher number of second-order clauses compared to the other two models. Additionally, Cases I and III exhibit higher $\mathrm{T}\mathrm{V}$ values due to the balanced distribution of redundant and non-redundant clauses, which allows for a more diverse set of solutions. However, since the quantities of redundant and non-redundant clauses in Case I are randomly chosen, significant fluctuations are observed in its $\mathrm{T}\mathrm{V}$ trend. As shown in Figure 12 and Table 15, $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$ values for different DHNN–BR2SAT models remain relatively stable between 0.5 and 0.7. Overall, Cases II and III perform better than Case I. Cases II and III contain more second-order clauses, with each clause including two variables. This structure allows for a greater variety of neuron state combinations that align with global minima, resulting in larger differences between the final neuron states and the benchmark state. In contrast, Case I contains more first-order clauses, with each clause having only one variable, which limits Case I to a single neuron state corresponding to the global minimum energy. Unfortunately, the global solution for Case I matches the benchmark state, indicating a lack of solution diversity. As NN increases, the inefficiency of the ES algorithm causes DHNN to generate more local minima, reducing the diversity of final neuron states. Thus, selecting a BRAN2SAT model with redundant clauses helps enhance model generalization and interpretability, thereby lowering $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$ values.

Friedman rank tests are conducted on $\mathrm{T}\mathrm{V}$ and $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$ for all the numbers of neurons. The Chi-square value for $\mathrm{T}\mathrm{V}$ is ${\chi }^2=11.0286$⁠, with a degree of freedom of $\mathrm{df}=2$ and $p<0.05$⁠. Therefore, the null hypothesis of no significant difference for the proposed DHNN–BR2SAT model is rejected. According to Tables 14 and 15, Case I demonstrates competitive $\mathrm{T}\mathrm{V}$ values, with its average rank surpassing that of the other two models. Case III shows superior performance in the $\mathrm{T}\mathrm{V}$ aspect. In contrast, the Friedman test for $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$ reveals $p>0.05$⁠, suggesting no significant difference between the DHNN–BR2SAT model and the other models in terms of $S_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}$⁠.

6.4 Qualitative analysis of DHNN–BR2SAT

Table 16 presents a qualitative analysis of the DHNN–BR2SAT model under different clause proportions. Based on prior results, it is inferred that Case I achieves optimal performance in both the learning and retrieval stages. Notably, ‘$\times$’ indicates poor performance on a given metric, while ‘$✓$’ indicates good performance. In summary, Case I exhibits superior learning efficiency and increases the number of global solutions.

6.5 The limitation of DHNN–BR2SAT

Through the above analysis, we have gained a comprehensive understanding of the distributions of learning error, synaptic weight error, energy, final solutions, neuron variations, and similarity indices, which has allowed for a systematic evaluation of the overall behaviour of DHNN–BR2SAT. The results indicate that embedding BRAN2SAT within DHNN facilitates obtaining ideal global minima and achieving diverse neuron states. However, based on the no free lunch theorem (Wolpert & Macready 1997), we conclude that no single model is optimal for all scenarios, as each model produces unique outcomes. This study is limited by computational time. The presence of redundant variables doubles the number of clauses, making it challenging for the model to find the correct interpretation during the learning phase. With larger $\mathrm{NN}$⁠, DHNN tends to fall into a trial-and-error process, which further extends the computational time. Additionally, due to the limitations of the ES algorithm, this study restricts the neuron count to fewer than 50. Future research could consider employing metaheuristic algorithms, such as GA (Acampora et al. (2023), Particle Swarm Optimization (Felicetti & Wang 2024), and Black-winged Kite Algorithm (Wang et al. 2024b), as alternatives to ES to enhance computational efficiency.

7 Conclusion

This study introduces a novel logical rule, termed BRAN2SAT, consisting of redundant second-order clauses and non-redundant first- and second-order clauses. The logical rule is implemented into DHNN by minimizing the cost function of the network, forming DHNN–BR2SAT. During the learning phase, the optimal synaptic weights are computed by comparing the cost function with the energy function. In the retrieval phase, the network achieves a stable state by updating fewer neurons. The effectiveness of the DHNN–BR2SAT model is evaluated by varying the proportions of different clause orders, with performance evaluated using various metrics during both the learning and retrieval phases. Experimental results indicate that when the numbers of the three types of clauses are randomly selected, the DHNN–BR2SAT model achieves optimal performance. Notably, this is the first attempt to implement logic containing redundant literals into DHNN, highlighting the potential of this approach to improve model performance when dealing with overlapping data.

Future directions. Metaheuristic algorithms are employed to optimize the retrieval phase, facilitating the generation of a more diverse final solution. Furthermore, the DHNN–BR2SAT model offers a novel approach to logic mining, with potential applications in practical problems such as sentiment analysis, industrial data classification, and financial investment forecasting. The DHNN–BR2SAT model represents a significant advancement in logic mining, filling the gap in extracting overlapping knowledge, and offering substantial potential for applications in various domains.

Conflicts of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Author Contributions

Binbin Yang: Methodology, Writing – original draft, Writing – review & editing. Guoxiang Li: Project administration, Data curation. Adila Aida Azahar: Writing – review & editing, Data curation. Mohd Shareduwan Mohd Kasihmuddin: Supervision, Methodology. Yuan Gao: Methodology, Data curation. Suad Abdeen: Writing – review & editing, Methodology. Baorong Yu: Visualization, Writing - review & editing.

Funding

The authors are grateful for the financial support from the National Natural Science Foundation of China (no. 62162006). Additionally, we acknowledge the support by Guangxi Key Laboratory of Big Data in Finance and Economics Project Fund, Guangxi First-class Discipline statistics Construction Project Fund and Guangxi University Young and Middle-aged Teachers’ Research Capacity Enhancement Project (nos 2021KY0907, 2021KY0378, and 2023KY1293) and Guangxi Key Laboratory of Seaward Economic Intelligent System Analysis and Decision-making (no. 2024C013).

Data Availability

No data were used for the research described in the article.

Acknowledgments

The authors would like to thank the anonymous referees for their valuable comments and suggestions.

References

Author notes