1. Introduction

2. Literature Survey

3. Materials and Methods

3.3. Prediction models

To perform an extensive comparison of the ascendancy of various deep learning models in day-ahead cooling load prediction, four standalone models and two hybrid models are implemented in this work. DNN, CNN, RNN and LSTM are the standalone models while CNN-LSTM and sequence-to-sequence are the hybrid models. A brief description about each of these learning algorithms is given in the sub-sections below.

3.3.1. Deep neural network

Deep neural networks have a layered architecture. It is like Artificial Neural Networks in terms of its functioning and architecture, but the difference mainly comes from the number of layers added to make the network ‘deep’. Each layer will help the network to extract more higher-level features. Unlike conventional neural networks, DNN can add more densely connected layers, each of which can use different activation functions if required. Its deftness in adding dropout layers also makes them superior to the ANN architecture [9]. This feature makes it capable of dropping a percentage of neurons randomly from any layer, and consequently avoids overfitting. The input layer will accept the input, hidden layers process the input to extract higher level features and finally output layer provides the output. This network can be used as a multi-output model. Depending on the number of neurons present in the output layer, multi-output feature can be easily tailored to this architecture.

3.3.2. Convolutional neural network

A convolutional Neural Network is a kind of Deep learning network with feed forward structure. They use kernels or filters to process the input, which slide over the input and compute the cross correlation between the kernel and each part of the input. In this way highly correlated part of the input will get more weightage. This method was originally built for image processing tasks, but nowadays found successful in time series regression problems like load prediction [18,25,26] as well. In addition to the input layer, convolution layers and pooling layers, CNN can have dense layer also at the end of the network. This architecture makes it suitable for prediction tasks. With a suitably designed dense layer at the end, multi-output prediction is also possible with this network. Upon removal of the eventual dense layer, the same architecture serves the purpose of a feature extractor. Depending on the size of the kernels used in filtering, higher-level features can be extracted from the input.

When used as a standalone model, CNN follows its conventional architecture with a dense layer at the end. Convolution layers can be one dimensional or two-dimensional, the selection commensurate with the shape of the filter. One dimensional convolution layer requires only the length of the filter to be mentioned, whereas the width is automatically set equal to the width of the input data set. Width in this case refers to the number of features in the dataset. Conversely, two-dimensional convolution layer works with a two-dimensional kernel which will be usually mentioned like 2x2, 3x3 or 5x5. Convolution layers extract higher level features from the input data, which helps it to predict better.

3.3.3. Recurrent neural network

Recurrent neural networks have a structure different from the feed forward neural networks. In feed forward networks, input flows in the forward direction only, whereas in recurrent neural networks, recurrent connections are possible, which means that output from an internal node can impact the subsequent input to the same node. These recurrent connections make this network specifically suitable for learning from sequential data like time series. The basic architecture of an RNN is depicted in Figure 2.

Recurrent neural network cell has multiple fixed numbers of hidden units one for each time step the network processes. Each unit can maintain an internal hidden state which represents the memory that unit holds regarding the sequential data. This hidden state will be updated at each timestep to reflect the changes in the sequential data. Hidden state update is done as per equation 2.

$$h_t=f\left(W_h{h}_{t-1}+W_x{X}_t+b_h\right)$$

(2)

where $h_t$ is the vector representing hidden state at time t, $W_x$ denotes the weights associated with inputs, $W_h$ are the weights associated with hidden neurons, $X_t$ is the input vector and $b_h$is the bias vector at the hidden layer. $f$ denotes the activation function used at the hidden layer. Based on the number of inputs and outputs, RNN can be configured as one-to-one, one-to-many, many-to-one, or many-to-many. In this work, the RNN takes multiple time steps as input and produces multiple time steps as output. Hence it is used as a many-to-many network.

3.3.4. Long short-term memory

Long short-term memory (LSTM) is an advanced sort of recurrent neural network, specially designed to contend with the problem of vanishing gradients. With the help of a cell and three gates as the basic building blocks, it enhances the memorizing capability of recurrent neural networks. These three gates are termed as input, output and forget gates. LSTM network can memorize thousands of time steps, and this makes it particularly suitable for handling time series problems. The gates are used to regulate the flow in and out of cell state. The input gate regulates the flow of input to the memory cell and hence decides what latest information is to be added to the current state. The output gate determines what information from the memory cell is to be passed to the output. Conversely, forget gate helps to decide what information is to be discarded or forgot from the current state. With forget gate, LSTM network efficiently forgets the old or irrelevant information. In short, LSTM can discriminatively maintain or discard information as the data flows through the network, which makes it suitable for learning long sequences. LSTM performs its work with the following four steps.

Step 1: Output of the forget gate, $f_t$ is calculated in the first step.

$$\begin{array}{c}{f}_t=\sigma \left(w_f\left[X_t,Y_{t-1}\right]+b_f\right)\end{array}$$

(3)

where $w_f$ and $b_f$ denote the weight vector and the bias vector at forget gate, respectively. $X_t$is the input at current iteration, $Y_{t-1}$ is the output from the previous iteration, and σ is the activation function.

Step 2: The cell state output $C_t^{\text{'}}$and the output of the input gate $i_t$ are calculated.

$$\begin{array}{c}{C}_t^{\text{'}}=a_i\left(w_c\left[X_t,Y_{t-1}\right]+b_c\right)\end{array}$$

(4)

(5)

Here, $a_i$ is the input activation function, $w_c$ and $b_c$ are the weight and bias vectors of the cell state, $w_i$ and $b_i$ are the weight and bias vectors at input gate.

Step 3: The cell state value $C_t$ is updated in this step.

$$\begin{array}{c}{C}_t=i_t . C_t^{\text{'}}+f_t . C_{t-1}\end{array}$$

(6)

Step 4: Output of the output gate, $O_t$ is calculated in this step. This gate combines previous iterations output $Y_{t-1}$ and the current input $X_t$, using $b_o$as the bias and $w_o$ as the weight vector. Finally, the output $Y_t$is computed.

(7)

$$\begin{array}{c}{Y}_t=O_t . a_o\left(C_t\right)\end{array}$$

(8)

3.3.5. CNN-LSTM

Convolutional neural network and long short-term memory are connected sequentially to make a hybrid structure [27]. The structure of this hybrid model is shown in Figure 3. CNN is used as a feature extractor here. During the feature selection process done in the beginning, it was identified that the historical cooling load is the most influential parameter in prediction. Hence extensive experiments are conducted to analyze what combination of inputs to CNN and LSTM will enhance the results. Through experiments, it is concluded that the hybrid model provides maximum performance when the inputs excluding cooling load are given as input to the CNN. More informative features are extracted from these input parameters using CNN. Then the extracted features are combined with the original cooling load features before providing as input to the LSTM model. This process is shown in Figure 3.

Outputs of LSTM network is processed by a dense layer to finally obtain the desired multi-step prediction. One dimensional as well as two-dimensional convolution layers are tried in CNN. The LSTM can consider the sequential behavior of historical cooling load as the cooling load is given in raw form to the LSTM network, without passing through the CNN. The configuration of the CNN-LSTM model used for processing SIT@Dover dataset in this work is shown in Table 3. The number of features and the kernel size at each convolution layer are set in an ascending order as depicted in the table. One-dimensional convolution gave better performance compared to the two-dimensional one.

3.3.6. Sequence-to-sequence

The sequence-to-sequence model, also known as encoder-decoder model is a combination of two LSTM networks, the first one being used for encoding the information contained in the input sequence and the second one being used for generating the output sequence. A dense layer will be connected at the end of LSTM network to take multi-step prediction as output. Encoder LSTM passes its hidden states and cell states to the decoder; hence this state is the encoded information. Encoder and decoder can have diverse configurations including the number of hidden units, activation functions and other parameters. In this work, both LSTM networks use 64 hidden neurons and ‘tanh’ as the activation function. Final dense layer has ‘h’ neurons and ‘linear’ activation being used. The encoder’s return state is set to true to retrieve the hidden state information from the encoder.

In addition to the state information retrieved from the encoder, decoder LSTM can take separate input from outside, mentioned as input2 in Figure 4. Generally, on decoder side, previous LSTM units’ output can be given as input to the consequent LSTM unit. Another alternative option is to provide ground truth information as input to the decoder. This method is known as teacher forcing and is usually popular in language processing models. Teacher forcing ensures fast and effective training of the model. A problem associated with this method is that it might cause errors while using a model trained this way in real scenarios for prediction. The model might confront unforeseen sequences in real prediction scenarios. In this work also, it is not found realistic to provide the ground truth as input to the decoder, and hence an alternate option is adopted here. From the historical cooling load series, the most similar day’s load sequence is given as input to the decoder. During correlation analysis it is observed that the time lags that have more resemblance to the target load sequence are the previous day’s load sequence and the previous week’s same day’s load sequence. Experiments revealed that instead of previous day’s load sequence, previous week’s, same day’s load sequence is giving better results. Hence this load sequence is given as input to the decoder model.

3.3.7. Performance metrics

The performance evaluation metrics are carefully chosen to epitomize an extensive validation and comparison of the developed models. To ensure the reliability of results, 10 trial runs are conducted for each experiment and the performance measures are averaged over these trials. Root Mean Squared Error (RMSE), Coefficient of variation of Root Mean Squared Error (CV-RMSE), Mean Absolute Percentage Error (MAPE), and Mean Absolute Error (MAE) are the performance metrics selected in this work. Computation of these metrics can be found in equations 9 to 12.

$$RMSE=\sqrt{\frac{1}{N}{\displaystyle \sum }_{i=1}^N{\left(y_i^{\text{'}}-y_i\right)}^2}$$

(9)

$$CV-RMSE=\frac{\sqrt{\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N{\left(y_i^{\text{'}}-y_i\right)}^2}}{\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N{y}_i^{\text{'}}}$$

(10)

$$\begin{array}{c}MAPE=\frac{1}{N}{\displaystyle {\displaystyle \sum }_{i=1}^N}\frac{\left|y_i^{\text{'}}-y_i\right|}{y_i}*100\% \end{array}$$

(11)

(12)

where N is the number of samples in test set, $y_i^{\text{'}}$is the predicted value and $y_i$ is the original value of the i^th sample in the test set.

4. Results and discussion

5. Conclusion

References

1. Saber, E. M., Tham, K. W., & Leibundgut, H. (2016). A review of high temperature cooling systems in tropical buildings. In Building and Environment (Vol. 96, pp. 237–249). Elsevier Ltd. [CrossRef]
2. Rui Jing, Meng Wang, Ruoxi Zhang, Ning Li, Yingru Zhao, A study on energy performance of 30 commercial office buildings in Hong Kong, Energy and Buildings, Volume 144, 2017, Pages 117-128, ISSN 0378-7788. [CrossRef]
3. Chalapathy, R., Khoa, N. L. D., & Sethuvenkatraman, S. (2021). Comparing multi-step ahead building cooling load prediction using shallow machine learning and deep learning models. Sustainable Energy, Grids and Networks, 28. [CrossRef]
4. Fan, C., Xiao, F., & Wang, S. (2014). Development of prediction models for next-day building energy consumption and peak power demand using data mining techniques. Applied Energy, 127, 1–10. [CrossRef]
5. Guo, Y., Nazarian, E., Ko, J., & Rajurkar, K. (2014). Hourly cooling load forecasting using time-indexed ARX models with two-stage weighted least squares regression. Energy Conversion and Management, 80, 46–53. [CrossRef]
6. Lehna, M., Scheller, F., & Herwartz, H. (2022). Forecasting day-ahead electricity prices: A comparison of time series and neural network models taking external regressors into account. Energy Economics, 106. [CrossRef]
7. Fan, C., & Ding, Y. (2019). Cooling load prediction and optimal operation of HVAC systems using a multiple nonlinear regression model. Energy and Buildings, 197, 7–17. [CrossRef]
8. Borchani, H., Varando, G., Bielza, C., & Larrañaga, P. (2015). A survey on multi-output regression. In Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery (Vol. 5, Issue 5, pp. 216–233). Wiley-Blackwell. [CrossRef]
9. Fan, C., Xiao, F., & Zhao, Y. (2017). A short-term building cooling load prediction method using deep learning algorithms. Applied Energy, 195, 222–233. [CrossRef]
10. Zhongfu Tan, Gejirifu De, Menglu Li, Hongyu Lin, Shenbo Yang, Liling Huang, Qinkun Tan, Combined electricity-heat-cooling-gas load forecasting model for integrated energy system based on multi-task learning and least square support vector machine, Journal of Cleaner Production, Volume 248, 2020, 119252, ISSN 0959-6526. [CrossRef]
11. Lahouar, A., & ben Hadj Slama, J. (2015). Day-ahead load forecast using random forest and expert input selection. Energy Conversion and Management, 103, 1040–1051. [CrossRef]
12. Dudek, G. (2015). Short-term load forecasting using random forests. Advances in Intelligent Systems and Computing, 323, 821–828. [CrossRef]
13. Bae, D. J., Kwon, B. S., & Song, K. bin. (2022). XGboost-based day-ahead load forecasting algorithm considering behind-the-meter solar PV generation. Energies, 15(1). [CrossRef]
14. R. Khan, S. Razzaq, T. Alquthami, M. R. Moghal, A. Amin and A. Mahmood, "Day ahead load forecasting for IESCO using Artificial Neural Network and Bagged Regression Tree," 2018 1st International Conference on Power, Energy and Smart Grid (ICPESG), Mirpur Azad Kashmir, Pakistan, 2018, pp. 1-6. [CrossRef]
15. Luo, X. J. (2020). A novel clustering-enhanced adaptive artificial neural network model for predicting day-ahead building cooling demand. Journal of Building Engineering, 32. [CrossRef]
16. M. Beccali, M. Cellura, V. Lo Brano, A. Marvuglia, Forecasting daily urban electric load profiles using artificial neural networks, Energy Conversion and Management, Volume 45, Issues 18–19, 2004, Pages 2879-2900, ISSN 0196-8904. [CrossRef]
17. Rahmat-Allah Hooshmand, Habib Amooshahi, Moein Parastegari, A hybrid intelligent algorithm based short-term load forecasting approach, International Journal of Electrical Power & Energy Systems, Volume 45, Issue 1, 2013, Pages 313-324, ISSN 0142-0615. [CrossRef]
18. Cai, M., Pipattanasomporn, M., & Rahman, S. (2019). Day-ahead building-level load forecasts using deep learning vs. traditional time-series techniques. Applied Energy, 236, 1078–1088. [CrossRef]
19. Liu, X., Zhang, Z., & Song, Z. (2020). A comparative study of the data-driven day-ahead hourly provincial load forecasting methods: From classical data mining to deep learning. Renewable and Sustainable Energy Reviews, 119. [CrossRef]
20. Li, L., Su, X., Bi, X., Lu, Y., & Sun, X. (2023). A novel Transformer-based network forecasting method for building cooling loads. Energy and Buildings, 296. [CrossRef]
21. Kim, W., Han, Y., Kim, K. J., & Song, K. W. (2020). Electricity load forecasting using advanced feature selection and optimal deep learning model for the variable refrigerant flow systems. Energy Reports, 6, 2604–2618. [CrossRef]
22. Sehovac, L., Nesen, C., & Grolinger, K. (2019). Forecasting Building Energy Consumption with Deep Learning: A Sequence to Sequence Approach. Available online: https://ir.lib.uwo.ca/electricalpubhttps://ir.lib.uwo.ca/electricalpub/166.
23. Li, A., Xiao, F., Zhang, C., & Fan, C. (2021). Attention-based interpretable neural network for building cooling load prediction. Applied Energy, 299. [CrossRef]
24. Fan, C., Wang, J., Gang, W., & Li, S. (2019). Assessment of deep recurrent neural network-based strategies for short-term building energy predictions. Applied Energy, 236, 700–710. [CrossRef]
25. K. Amarasinghe, D. L. Marino and M. Manic, "Deep neural networks for energy load forecasting," 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE), Edinburgh, UK, 2017, pp. 1483-1488. [CrossRef]
26. L. Li, K. Ota and M. Dong, "Everything is Image: CNN-based Short-Term Electrical Load Forecasting for Smart Grid," 2017 14th International Symposium on Pervasive Systems, Algorithms and Networks & 2017 11th International Conference on Frontier of Computer Science and Technology & 2017 Third International Symposium of Creative Computing (ISPAN-FCST-ISCC), Exeter, UK, 2017, pp. 344-351. [CrossRef]
27. Alhussein, M., Aurangzeb, K., & Haider, S. I. (2020). Hybrid CNN-LSTM Model for Short-Term Individual Household Load Forecasting. IEEE Access, 8, 180544–180557. [CrossRef]
28. Solcast, 2019. Global solar irradiance data and PV system power output data. URL. Available online: https://solcast.com/.
29. Zhang, L., Wen, J., Li, Y., Chen, J., Ye, Y., Fu, Y., & Livingood, W. (2021). A review of machine learning in building load prediction. Applied Energy, 285. [CrossRef]