Deep learning-based multivariate load forecasting for integrated energy systems

Abstract

With the continuous development of integrated energy utilization technology and the diversification of users’ energy demand, and the existing single load forecasting method is difficult to deal with the complex coupling relationship derived between various types of loads, resulting in the inaccuracy of multivariate load forecasting, which makes the accurate forecasting of multivariate loads of integrated energy systems more challenging. To address the aforementioned issues, we propose a short-term forecasting method for integrated energy multivariate loads based on GRU-MTL. Firstly, we conduct a correlation analysis using the hierarchical analysis method and Copula theory, and select the model input features based on the final correlation metric results. Secondly, we construct a multivariate load forecasting model for electricity, cooling, and heating based on gated cyclic unit and multi-task learning. Finally, a comparison was made with the traditional model, and the results indicate that the constructed model has better predictive accuracy and is more efficient in terms of time.

1 Introduction

In the twenty-first century, energy has become a global issue. Currently, the world’s leading energy producer in terms of global manufacture, consumption, and imports. The development of energy is affected by sundry factors, such as energy transformation and international political and economic factors. In the report on the 19th National Congress of the Communist Party of China, General Secretary Xi Jinping highlighted the importance of advancing the development of China’s modern energy system to a new level during the ‘14th Five-Year Plan’ period [1]. Major countries worldwide are striving to augment the ratio of clean and reproducible energy expend while reducing the use of primary energy sources such as oil and coal. As a major energy country, China has been tasked by General Secretary Xi Jinping to prioritize the development of clean energy, which holds strategic significance. To complete this task, the development of renewable energy is essential. The (integrated energy system, IES) includes the renewable energy (combined cooling heating and power, CCHP) system [2, 3], which integrates power generation, heating, and cooling processes. The CCHP system comprises heating equipment, cooling equipment, heat exchangers, internal combustion generator sets, and other equipment. IES is based on the CCHP system and can access various forms of renewable energy, such as natural gas, photovoltaic, and wind power. Integrated Energy Systems offer environmental, economic, and social benefits.

Compared to traditional energy systems, IES have closely linked energy subsystems that are more conducive to unified management and scheduling of energy. IES are designed to match the functional structure of various building types in the region, resulting in improved energy efficiency. This promotes the effective construction of China’s energy system [4]. However, the introduction of renewable energy has strong intermittent and fluctuating characteristics. This poses a challenge to meeting renewable energy quotas. To truly achieve a gradient of energy use and settle a matter of reproducible energy expend, energy regulation by IES is essential. Accurately predicting multiple loads is a fundamental prerequisite for performing well in IES energy management, operational scheduling, optimal design, and related tasks [5]. Multi-load forecasting for (regional integrated energy systems, RIES) is based on historical load data, supplemented by meteorological, calendar, social, and other relevant information. The analysis of historical data, including meteorological, calendar, social, and economic information, along with other relevant factors, is used to establish the correct mapping relationship for the law of change in multiple load time series. This enables the prediction of the next moment’s energy demand values for various types of energy [6]. Currently, the conventional energy system operates independently without considering the correlation between heterogeneous energy sources. Integrated energy systems exhibit a relevant between multi-energy loads within the system on account of the existence of numerous energy coupling devices. This is in contrast to other energy systems [7, 8]. Furthermore, the widespread implementation of reproducible energy sources, for instance wind power, introduces uncertainty to the prediction of multi-energy loads [9]. To ensure optimal scheduling and safety assessment of the integrated energy system’s electricity, heat, gas, and other networks, it is crucial to conduct accurate and effective multi-energy load predictions [10, 11]. This article addresses the short-term forecasting of multi-energy loads in RIES, with a time scale ranging from minutes to hours, typically set at 1 h. The forecasting results are critical to the energy system’s operation and provide the primary data support for energy regulation and control work.

2 Current status of domestic and international research on integrated energy systems

Compared to traditional electric load forecasting, forecasting for integrated energy systems is more complex due to the presence of heterogeneous energy loads and interactions between them, as well as uncertainty in load change patterns. Load prediction can be divided into two categories: deterministic prediction (point prediction) and probabilistic prediction, which includes interval prediction and probability density prediction. In recent years, research on domestic and foreign load forecasting has mainly taken the form of point prediction. Point prediction provides a single deterministic prediction of future load, but does not take into account the uncertainty that exists in future load. Therefore, this method is not suitable for loads that are random and volatile, resulting in large prediction errors and difficulty in achieving a high degree of accuracy. The method for predicting load probability can provide a level of confidence based on the probability distribution, typically in the form of a confidence interval or probability density. This allows for a comprehensive understanding of future load uncertainty prediction information. Therefore, the uncertainty problem faced by the integrated energy system becomes more prominent with the continuous access of wind power and photovoltaic. It is of great research significance and engineering application value to carry out work related to the probabilistic prediction of multi-energy loads.

2.1 Current status of load forecasting research on integrated energy systems at home and abroad

A comprehensive energy system can achieve the organic coordination and optimization of producing, transmitting, distributing, transforming, storing, and consuming energy. It also involves the integrated construction of energy production, supply, and marketing [12]. The energy input types mainly include electricity, gas, heat, and new energy output. The energy output includes electricity-heat-cold and gases loads. The internal equipment comprises natural gas internal combustion units, cooling, heating and power cascade, wind power photovoltaic units, waste heat recovery system, electric refrigeration system, and absorption refrigeration system. Europe was the first area to introduce the comprehensive energy system design and put it into practice [13]. European countries conduct extensive research into the integrated use of energy systems built on their individual needs. Governments have introduced various energy systems and invested in demonstration projects to promote the development of integrated energy systems. Currently, North American countries prioritize the development of integrated energy systems. Research and demonstration projects related to this are more advanced [14].

China’s IES is comparatively underdeveloped due to the lack of coordination between other species of energy and the absence of coordinated policy support. This, to some extent, hinders the progress of China’s integrated energy technology. There is a relationship between the multi-energy loads of the integrated energy system and these loads are affected by external environmental factors. This makes predicting multi-energy loads difficult. Literature [15] presents a multivariate collaborative forecasting model for an IES built on the mechanism of multi-energy coupling. The article analyses the coupling characteristics among electricity-heat-cold loads by combining Copula theory. It usage the SSA tweak algorithm to solve the key parameters in the least-squares support vector machine (LSSVM) and establishes the SSA-LSSVM multivariate load forecasting model. The literature [16] proposes a learn while multitasking model that uses a ResNet-LSTM network and the attention mechanism to adapt to the spatial and temporal coupling ratio between multi-energy loads. The ResNet network serves as the feature extraction unit for multi-energy load data. The temporal features of the multi-energy load data are then extracted through the residual structure of the bidirectional LSTM network. Finally, the attention mechanism is employed to assign varying degrees of attention to the shared features of multi-task learning, enabling the joint prediction of multivariate loads. In literature [17], a short-term load forecasting method is proposed that considers the spatial–temporal correlation of multi-energy. On the first place, K-means clustering and Pearson’s correlation coefficient are used to ‘reconstruct’ pixels of different types of basic load units so that they have certain correlation characteristics horizontally and vertically, and on the second place, multi-channel convolutional networks are used to extract features from the reconstructed 2D pixels in high latitude space. And finally, LSTM network is used for load prediction and better prediction results are achieved. Literature [18] suggests a short-term joint forecasting method for cold, heat, and power based on weather information. The method makes full use of regional weather forecast data and historical load data, and uses a BP neural network for joint forecasting of cold, heat, and power loads. Literature [19] proposes a multi-task load forecasting method that uses LSTM as the common layer of. The sharing mechanics is used to learn the information about the coupling between multiple loads. The experiments proved that the multi-task learning model can fully explore the coupling information between sub-tasks, resulting in significantly improved prediction accuracy. The literature [20] suggests using feature migration learning to predict small sample electric loads in integrated energy systems. This involves constructing a feature extraction model using a fusion clustering algorithm and GRU network, implementing a feature fusion strategy, and ultimately proposing a load forecast method basis on the fusion of features using migration learning and input features.

2.2 Research status of deep learning load forecasting algorithms at home and abroad

Traditional machine learning methods depend on creating input feature metrics, and feature engineering is required to enable feature construction. To implement relevant load prediction, the feature construction of load data is the key to improvement of measurement accuracy of load prediction. Hinton first proposed the restricted Boltzmann machine (RBM) deep learning model, and then the deep belief network was proposed on top of it, and these two deep models brought hope to solve multi-layer complex networks [21]. In the following years, deep learning techniques were developed, and convolutional neural networks, recurrent neural networks, self-encoders, and generative adversarial networks were proposed. These deep neural network structures have expanded the application areas of deep learning theory [22]. However, deep learning techniques present significant challenges when dealing with certain issues, such as the interpretability of deep models, data scarcity, and model complexity. To address these challenges, new methods have been put forward, such as deep transfer learning, deep multi-task learning, and graph neural networks [23]. The use of deep learning methods has become widespread in various fields of study and increasing computing power. Load forecasting has also garnered attention from researchers both domestically and internationally. Literature [24] proposes a method for short-term load forecasting by fusing multiple neural networks. The method takes advantage of different deep neural networks, using a core network framework consisting of a deep neural network with parallel architecture and an attention mechanism network. The prediction results are output through a fully connected network. The arithmetic example demonstrates that the proposed method achieves higher prediction smoothness and accuracy. Literature [25] has also proposed a method for short-term load prediction by deep recurrent neural network, which adds an association layer based on multiple implicit layers, so that the deep network has a dynamic performance and can effectively tap the dynamic characteristics of the load.

In summary, after years of research by scholars both domestically and internationally, load prediction has reached a high level of accuracy. However, RIES is a combination of multiple small systems for electricity, cooling, and heat supply. Its complexity is much greater than that of traditional single energy systems. Therefore, the complex coupling conditions between various kinds of energy carriers must be taken into account to accurately predict the multivariate loads of RIES. This report suggests the following machine learning-based method for short-term prediction of multiple loads in RIES. The method is developed has been based on the outcome of the previous research and addresses the identified issues.

3 Research on improved particle swarm optimization algorithm

RIES multiple load forecasting is different from single load forecasting, because of its multiple load types and more sensitive to various influencing factors, which makes the relationship between energy changes more complex. Therefore, the reasonable selection of the input features of the forecasting model is the key part of the RIES multivariate load short-term forecasting research and the necessary prerequisite for the model to achieve good forecasting results. In view of this, this chapter makes an in-depth study on RIES multivariate load correlation analysis in order to achieve a more reasonable input feature data set, and proposes a prediction model input feature selection method based on (analytic hierarchy process, AHP) and Copula theory. It provides a basis for the selection of input features of the model.

3.1 Quantitative analysis of multivariate load influencing factors correlation based on AHP–Copula method

In order to obtain more scientific correlation analysis results and make the selection of model input features more reasonable, this section measures the correlation of the above main influencing factors (historical data of various loads, meteorological information and calendar information) based on AHP and Copula theory, and calculates the correlation coefficient between loads and various influencing factors. In this section, the correlation analysis method based on Copula theory is used to calculate the correlation coefficient between various types of loads and various influencing factors. Although the widely used Pearson correlation analysis method can effectively describe the linear correlation characteristics between variables, its ability to describe the nonlinear correlation characteristics is weak, and it is not suitable for describing the correlation relationship between multivariate loads and various types of loads and other influencing factors in FUES. The correlation analysis method based on Copula theory has strong processing ability for nonlinear correlation characteristics and can more flexibly describe the correlation between RIES multivariate loads and between various types of loads and other influencing factors. The steps of calculating the correlation coefficients between various types of loads and between various types of loads and other influencing factors based on Copula theory are shown in Fig. 1.

The first is marginal cumulative distribution estimation: In this chapter, the marginal cumulative distribution of electricity, cold and heat load and each influence factor is determined in a non-parametric way based on kernel density estimation method, where the specific expression of kernel density estimation method is shown in Equation 1.

where, n represents the number of samples, h is the window width, K is the kernel function，and x₁, x₂,…, x_n is the sample point of the random variable X.

Then, the parameter estimation of the Copula function model: in view of the marginal cumulative distribution of various loads and influencing factors have been obtained in the previous step, this article uses the stepwise maximum likelihood estimation to solve the marginal cumulative distribution and the parameters to be estimated in each Copula function model, respectively. Specifically, we first estimate the parameters θ₁，θ₂ of the marginal cumulative distribution:

where, θ₁，θ₂ after derivation are maximum likelihood values, f(x,θ₁) and g(y,θ₂) are the probability density functions of marginal cumulative distribution functions F(x,θ₁) and G(y,θ₂) of random variables X and Y, respectively. Then, put the derivative A and B into Equation 4 to obtain the parameter α to be evaluated in the Copula function:

The next step is to determine the optimal Copula function model: Since there are many kinds of Copula function models, and different Copula function models have different correlation characteristics to describe the above coupling relationship, the correlation analysis based on Copula theory needs to choose the best Copula function model according to the fitting results of various Copula function models. In this section, the L₂-norm minimum method is used to test the goodness-of-fit of sample data of each Copula function model and then determine the optimal Copula function. Among them, the specific calculation results of the goodness-of-fit test of various Copula function models on the above important influencing factors are shown in Table 1.

In the table, N, G, t, F, and C refer to the five Copula function models of Gaussian–Copula, Gumbel–Copula, t-Copula, Frank–Copula, and Clayton–Copula, respectively. E, C, H, H ‘, T, C ‘, P, and W represent the eight influencing factors of electric load, cooling load, heat load, humidity, temperature, calendar information, air pressure, and wind speed, respectively.

Finally, the correlation coefficient index is calculated: the correlation measure index derived based on Copula theory mainly includes Kendall’s and Spearman’s rank correlation coefficients. Compared with Kendall rank correlation coefficient, Spearman rank correlation coefficient has a good performance in describing the correlation characteristics between multivariate loads and various types of loads and other influencing factors. Therefore, this section uses Spearman rank correlation coefficient to describe the correlation characteristics between multivariate loads and between various loads and other influencing factors. The Spearman rank correlation coefficient index of each influencing factor derived from the calculation result of L₂-norm is shown in Table 2, and the Spearman rank correlation coefficient ρ calculation formula is shown in Equation 5.

The first three rows of this table record Spearman’s Rank of significant correlation co-efficients of the influencing variables factors under various load forecasting subtasks, and the last row records Spearman’s Rank of significant correlation co-efficients of the influencing variables factors under electrical, cold, and thermal multivariate load forecasting tasks, which are calculated by taking the average Spearman’s Rank of significant correlation co-efficients of the influencing variables factors under various load forecasting subtasks. By observing the calculation results in Table 2, it can be found that the correlation between the two influencing factors of meteorological information, air pressure, and wind speed, and multivariate load forecasting is much greater than that of calendar information due to the difficulty of reasonably datalizing calendar information, which is not consistent with practical experience. It can be seen that if only the rank correlation coefficient of each influence factor derived by the Copula function model is used, sometimes the calculation results cannot accurately provide the basis for determining the input characteristics due to the data quality and other data problems. In view of this, this article uses AHP to weight each influence factor from the indicator dimension to deal with the above situation.

3.2 Overall framework of multivariate load short-term forecasting method

The overall RIES multivariate load short-term forecasting method based on machine learning proposed in this article, the first part is the data processing part: all the collected data including outlier processing, missing value processing, normalization processing, and other data preprocessing operations. The specific operations of outlier handling are as follows. Due to the continuity of each input characteristic time series data involved in this article, the corresponding time series curve shows a smooth and continuous trend, so if the data value corresponding to a point on the curve and the data value corresponding to its adjacent points are too large, beyond the specified threshold range, that is, the horizontal processing method can be used to correct. The corrected value of this point is shown in Equations 6.

where X(n,k) represents the original data at time k on day n of the input feature. The specific operation of normalization processing is as follows: In order to prevent the activation function from entering the oversaturated state as far as possible and to perform dimensionless processing on input features to solve the comparability problem of different input feature indicators, the expression of normalization processing on input feature data is shown in Equation 7.

The second part is multivariate load correlation analysis and model input feature selection. The processed data are drawn into load curves of different time scales, and the main influencing factors that affect the change of energy demand are qualitatively analyzed by mining the change rules in them. Then, the correlation analysis method based on AHP–Copula is used to calculate the weighted correlation index of the main influencing factors, and the coupling relationship is quantitatively analyzed. Finally, the input features of the model are determined according to the weighted correlation index. Then comes the modeling part of the model: the constructed model is RIES electricity, cold, and heat multivariate load forecasting model basis on GRU and MTL. In order to realize the effective learning of complex coupling information between multiple energy sources and establish a clear mapping relationship between multi-input and multi-output, the proposed model adopts the hard sharing mechanism of hidden layer parameters of MTL, and GRU is used to build the sharing layer. At the same time, the Adam optimization algorithm, which is suitable for solving problems with large-scale data and parameter optimization, and the Dropout technique, which can reduce the risk of over fitting, are used to ensure the overall performance of the model. Finally, the model evaluation part: after dividing the data set into a training set and a test set, the data samples of the training set are used to train the model, and then the data samples of the test set are input into the proposed model.

4 Example analysis of multivariate load forecasting in integrated energy system

Aiming at the problem that it is difficult to achieve accurate short-term prediction of RIES multivariate load caused by the complex coupling relationship between RIES multivariate loads, this article proposes a RIES multivariate load short-term prediction method based on GRU and MTL. To be able to verify the effectiveness of the proposed method, this chapter conducts a case research about the RIES multivariate short-term load forecasting model basis on GRU and MTL based on the annual hourly RIES load data of a university campus in the United States. In order to prove that the model proposed in this document can better deal with the strong coupling problem between RIES multi-load, two comparison models are designed: single task model and multi-output model analyzed.

4.1 Experimental results and analysis of joint cold-heat load prediction

Basis on the historical load data of cooling load and heat load and other input feature data, the comparison of prediction results of single-task prediction model, multi-output prediction model and multi-task prediction model is shown in Fig. 2, and the comparison results of performance evaluation indicators are shown in Table 3.

The analysis of Fig. 2 and Table 3 shows that, in comparison to the forecasting system for an individual task, the multi-output prediction model does not show better prediction effect in the prediction of cooling load as the electrical-cooling and electrical–thermal joint prediction, but makes the MAPE value increase by 2.44 percentage points. Although there is no negative growth compared with the prediction results of cooling load, its MAPE value is only 0.65 percentage points lower than that of the single task prediction model. In comparison to the forecasting system for an individual task, the multi-task prediction model still significantly improves the prediction accuracy of each load in the joint prediction of cold and heat load. In comparison to the forecasting system for an individual task, the MAPE values of the predicted cold load and heat load are reduced by 3.9 and 4.47 percentage points, respectively, and the reduction ratio is 49.1% and 55.0%, respectively. in comparison to the forecasting system for multi-output task, the prediction error of the multi-task prediction model does not show the phenomenon of increasing instead of decreasing when predicting the cooling load, and the prediction effect of the heat load is further improved on the basis of the multi-output prediction model, and the MAPE value of the multi-task prediction model is reduced by 3.82 percentage points, and the reduction ratio is 51.1%. The experimental results show that the multi-task prediction model can still mine the coupling information between cooling load and heating load in the joint prediction of cold and heat load, so that the prediction results of both kinds of loads can be significantly improved.

4.2 Experimental results and analysis of electric-cold-heat load joint prediction

Basis on the historical load data of electric-cold-heat loads and other input feature data, the comparison of prediction results of the single-task prediction model, the multi-output prediction model and the multi-task prediction model is shown in Fig. 3, and the comparison results of performance evaluation indicators are shown in Table 4.

The analysis of Fig. 3 and Table 4 shows that when the three kinds of electricity, cold and heat loads are predicted simultaneously, the prediction effect of the multi-output prediction model is general. In comparison to the forecasting system for an individual task, the prediction effect of the multi-output prediction model only has a slight advantage when the heat load is predicted, and the MAPE value is reduced by 0.24 percentage points, while the prediction task of electricity load and cooling load has negative benefits. The MAPE value increased by 1.81 percentage points and 0.76 percentage points respectively. In contrast, the multi-task prediction model performs better when the three prediction tasks of electricity-cold-heat are predicted at the same time, and the prediction precision of the three loads is improved. Specifically, in comparison to the forecasting system for an individual task, the MAPE values of the three load prediction tasks of electricity-cold-heat are reduced by 4.94, 4.26, and 4.49 percentage points, respectively, and the reduction ratio is 51.9%, 53.5%, and 55.3%m respectively. Compared with the multi-output prediction model, the multi-task prediction model’s learning ability for the coupling relationship between multiple loads is more highlighted. The MAPE values of electricity-cold-heat load prediction are decreased by 6.75, 5.02, and 4.25 percentage points, respectively, and the reduction ratio is 59.6%, 57.5%, and 53.9%, respectively. The experimental results show that when the three kinds of electricity, cold and hot loads are jointly predicted, the multi-task prediction model can realize the transfer learning of the input information of the three kinds of load prediction tasks by means of the hard sharing mechanism of multi-task learning, and effectively utilize the learned knowledge, so that the prediction effects of the three kinds of electricity, cold and hot loads are significantly improved.

5 Conclusion and outlook

The trend of energy transition is increasingly apparent. The single type of energy system is gradually being replaced by the growing development of renewable energy sources (RIESs). Short-term prediction of RIES multiple loads is an important prerequisite for effective RIES energy management, operation scheduling, and optimal design. This has become a hot topic among researchers.

5.1 Summary

To overcome the challenges posed by the complexity of the RIES itself and the influence of the coupling relationship between its multiple types of energy sources on the RIES short-term multivariate load forecasting researchers, the article designs a machine learning-based short-term multivariate load forecasting method for the RIES. The qualitative analysis of the factors that affect the change in energy demand is based on load curves of different time scales. The correlation measure, based on AHP and Copula theory, is then used to measure the correlation of the influencing factors. Based on the results of the measure, the input features of the proposed forecasting model are determined. A short-term forecasting model for RIES electricity-cooling–heating multivariate load is constructed based on GRU and MTL. In a simultaneous comparative experiment, the proposed model is compared with the single-task forecasting model and the multi-output forecasting model, and the experimental results show that the model constructed in this paper has a good forecasting effect.

5.2 Prospect

Although this article has conducted a detailed study on the short-term multivariate load forecasting problem of RES, there are still areas that require further improvement and research due to personal limitations in ability and time. These areas primarily include:

(1) This article presents a short-term prediction model for electric, cooling, and heating multiple loads based on GRU and MTL. The shared layer part of the proposed model is built using only the GRU layer. However, to enhance the model’s information processing and learning ability, the shared layer part can be constructed in a more detailed manner, which can further improve the prediction accuracy.

(2) The multivariate load forecasting system designed and developed in this article can simulate the real-time data acquisition process during the operation test. Due to the limitations of experimental conditions, it can be applied in actual engineering projects in the next step to better test the system’s operational performance.

Author contributions

Lei Sun (Conceptualization [equal], Data curation [equal], Formal analysis [equal], Funding acquisition [equal], Investigation [equal], Methodology [equal], Project administration [equal], Resources [equal], Software [equal], Supervision [equal], Validation [equal], Visualization [equal], Writing—original draft [equal], Writing—review & editing [equal]), Xinghua Liu (Conceptualization [equal], Data curation [equal], Formal analysis [equal], Funding acquisition [equal], Investigation [equal], Methodology [equal], Project administration [equal], Resources [equal], Software [equal], Visualization [equal], Writing—original draft [equal], Writing—review & editing [equal]), Gushuai Liu (Conceptualization [equal], Data curation [equal], Funding acquisition [equal], Investigation [equal], Project administration [equal], Resources [equal], Software [equal], Validation [equal], Visualization [equal], Writing—original draft [equal], Writing—review & editing [equal]), Zengjian Yang (Conceptualization [equal], Data curation [equal], Formal analysis [equal], Funding acquisition [equal], Investigation [equal], Resources [equal], Software [equal], Supervision [equal], Validation [equal], Writing—original draft [equal]), Shuai Liu (Conceptualization [equal], Data curation [equal], Formal analysis [equal], Investigation [equal], Methodology [equal], Resources [equal], Supervision [equal], Validation [equal], Visualization [equal]), Yao Li (Conceptualization [equal], Data curation [equal], Formal analysis [equal], Funding acquisition [equal], Investigation [equal], Resources [equal], Supervision [equal], Writing—original draft [equal], Writing—review & editing [equal]), and Xiaoming Wu (Data curation [equal], Formal analysis [equal], Funding acquisition [equal], Methodology [equal], Resources [equal], Visualization [equal], Writing—original draft [equal], Writing—review & editing [equal]).

Funding

None declared.

References