Forecasting Power demand Using Artificial Neural Network for Sri Lankan Electricity Power System
K.P.M. Madhugeeth
Department of Computer Science
School of Computing
University of Colombo
madhugeethkpm@gmail.com
H.L. Premarathna
Department of Computer Science
School of Computing
University of Colombo
hlp@ucsc.cmb.ac.lk
Abstract
Accurate models for electric power load forecasting are essential to the operation and planning of an electricity company. Neural Networks are considered as a computational model that is capable of doing non linear curve fitting. In this research, the application of neural networks to study the design of short-term load forecasting (STLF) Systems for Sri Lanka was explored. Three layered neural network architecture with back propagation algorithm is proposed to model STLF. The results show that neural network gives the minimum forecasting error compared to the statistical forecasting models and hence it can be considered as an effective method to model the STLF systems for Sri Lankan electricity power system.
Keywords: short term Load Forecasting, Backpropagation, Non linear curve fitting, statistical forecasting models.
- INTRODUCTION
T
he ability to accurately predict the future is fundamental to many decision processes in planning, scheduling, purchasing, strategy formulation, policy making, and supply chain operations. Therefore people always try to find out accurate forecasting models. This research will examine and analyze the use of neural networks as a forecasting tool for electricity load forecasting and how can it be applied to the Sri Lankan Electrical Power System.
The total amount of electric power consumed by people must be balanced with the amount of generated power. There is no efficient way to store large amounts of electrical energy. To maintain this power balance between production and consumption it should be forecast future power needs.
Load forecasts can be divided into three categories: called, Short-term forecasts, Medium-term forecasts and Long-term forecasts.
Natures of these forecasts are different as well. Short-term load forecasting can help to estimate load flows and to make decisions that can prevent overloading. Timely implementations of such decisions lead to the improvement of network reliability and to the reduced occurrences of equipment failures and blackouts. On the other hand, Long term forecasting helps to know the electric load that want in the future and take necessary actions like building more power stations, to fulfill the needs of people in the country.
In terms of statistics, forecasting is an approximation of future data. The electric load forecasting task is usually carried out by statistical methods with some adjustments by experienced operators in central load dispatching centers. Traditional linear models, such as Autoregressive (AR) and Auto-regressive Moving Average (ARMA), have been used in time series forecast in practice. These models are straightforward for implementation, but there are limitations to improve forecasting accuracy by these methods, because these models are constructed by linear functions. On the other hand, an important contribution of Artificial Neural Networks (ANN) is that their capability to do non-linear curve fitting. ANN models are particularly powerful when applied to a very complex data set and when the structure of the model is unknown. This paper describes the use of ANN model for forecast electricity load for Sri Lankan electricity power system.
- BACKGROUND
Over the last few decades a number of forecasting methods have been developed for forecasting task. End-use and econometric approaches are two broadly used methods for medium and long term forecasting.
Similar day approach, various regression models, time series, and statistical learning algorithms, are used for short-term forecasting. Statistical approaches usually require a mathematical model that represents load as a function of different factors such as time, weather, calendar data and customer class. Additive models and multiplicative models can be introduced as such mathematical models [4],[5],[1].
Statistical models based on the end-use approach [8] have included descriptions of appliances used by customers, the sizes of the houses, the age of equipment, technology changes, customer behavior, and population dynamics. These models are based on the principle that electricity demand is derived from customer's demand for light, cooling, heating, refrigeration, etc. The problem of these models that; it is sensitive to the amount and quality of end-use data. Also end-use forecast requires less historical data but more information about customers and their equipment.
Econometric models include economic factors such as per capita incomes, employment levels, and electricity prices other than the factors included in end-use approach. This approach combines economic theory and statistical techniques for forecasting electricity demand.
Similar day approach is based on searching historical data for days within one, two, or three years with similar characteristics to the forecast day. Similar characteristics include weather, day of the week, and the date. The load of a similar day is considered as a forecast.
Regression methods are used to model the relationship of load consumption and other factors such as weather, day type, and customer class [6].
A time series is defined to be an ordered set of data values of a certain variable. Time series methods are based on the assumption that the data have an internal structure, such as autocorrelation, trend, or seasonal variation. Time series forecasting methods detect and explore such a structure. ARMA (autoregressive moving average), ARIMA (autoregressive integrated moving average), ARMAX (autoregressive moving average with exogenous variables), and ARIMAX (autoregressive integrated moving average with exogenous variables) are the most often used classical time series methods. Apart from these statistical methods, It has been done some research on time series forecasting using ANNs[13],[14],[15].
Limitations of some of above statistical methods are, they require a large amount of information relevant to appliances, customers, economics, etc. Also their application is complicated and requires human participation.
A support vector machine is a more recent powerful technique for solving classification and regression problems. Support vector machines perform a nonlinear mapping of the data into a high dimensional space. Then support vector machines use simple linear functions to create linear decision boundaries in the new space. It has described a method to forecast the electricity load using this method in [7].
With the advancement of computer power, people tried to solve load forecasting problem using efficient computing. It had able to explore the complexity in the load data using that computer power. Rule-based and fuzzy logic expert systems have been used to model the complexity in the data. Artificial neural networks have made it possible to experiment with data rich, models that can identify the complex non-linear relationships in the data and understand future behavior.
In [9], it has been described a river flow forecasting application that use neural network as a forecasting tool. It has used multi layer network which is trained using the back propagation algorithm. In [10] it has been developed a multi layer perceptron that model with Generalized Delta Learning to predict the population of India.
Paper establishes suitability of non-linear ANN as predictive tool for population in India.
Paper [11], explains another neural network forecasting application called Cash flow Forecasting. This system is trying to predict cash requirement within reasonable accuracy.
According to the forecasting methods discussed above, it cannot be found a best method for forecasting. Statistical methods have some limitations like external factors which cannot be considered in forecasting. In statistical methodology, error is also high, but it can be seen that neural networks can be used as a better forecasting tool. Above neural network applications confirm it by forecasting values with less error than in statistical methods.
- DESIGN OF THE IMPLEMENTATION MODEL
A feed-forward Artificial Neural Network model is proposed for the short term load forecasting. The network is trained using the back propagation learning algorithm [12] with a momentum factor. The performance of the network for one-day ahead load forecast is compared with the historical data available. The data is divided into two groups called training data to train the neural network and testing data to test the ANN model (Fig. 1).
Fig.1. Basic model for the electricity load forecasting system
- Input Data preparation
Variables can be categorized in to two groups, called Categorical variables [16] and Ordinal variables. Categorical variables do not have natural ordering. Sometimes these variables haven't numerical values. But it should have to transform to a numerical value for the purpose of give as inputs to the neural networks. So it can be used binary encoding method. For example when doing electricity load forecast it can be affected month of the year. Therefore the month can be converted to binary input as below.
January 001
February 010
March 011 etc...
Like above, Week days input can be present as 3 bytes binary code. Holiday can be represented by one byte (0 or 1). Ordinal Variables are variables which having natural ordering. These variables can be directly feed in to the neural networks as inputs.
In order to make the ANN model more effective, it is important to understand the relevance and relative importance of model inputs. When doing electricity load forecasting it can be expected there has a relationship between electricity load and calendar data (holidays, week days, weekends) as well as weather factors (temperature). According to the real electricity load data it can be seen clear decreasing of electricity load on holidays compared to the other days. So these parameters can be used as initial inputs to the neural network. After that it can be identified most relevant data for electricity load forecasting, using trial and error method. Sometimes it may be seen initial inputs not directly affect for the problem as it thought before.
Before input to the raw data in to the neural network, it should check the validity of data. Sometimes it can be contain unacceptable data within the gathered data. For example it cannot be contained temperature as 75ºC because, it can be clearly identified it is an abnormal value. So it should be discard this value and fill this value using another method like statistical (regression) method.
Filling the missing data in the collected data set is other important task. These missing data cannot be ignored. So it should use extrapolation or interpolation method to fill those values. Next step is to data partitioning. Collected data set should be mainly divided in two parts called training set and test set. Training dataset is use to train the neural network and testing set is to measure the performance of the neural network.
This system was used 85% data for training purposes and 15% for testing the neural network.
Before the inputs are presented to the ANN model the data should be Pre-process. Accuracy of the outputs of neural network is depending on the data preprocessing step. Following are the steps that should be done in data pre processing stage.
- Remove the noise from the data set.
- Normalization
- Extract main features of the dataset
- Network Architecture
Calendar data such as day type, month type and date, historical load, minimum temperature and maximum temperature are used as inputs to the neural network. Number of layers in the neural network, neurons in the hidden layer has been chosen by a trial and error method. Too many numbers of neurons increase the error and too less make the network inefficient to train itself to the given variable inputs.
A number of adjoining data points of the time series and other inputs are mapped to the interval [1,-1](normalizing) and used as activation levels for the units of the neural network. In a forward path, these activation levels are propagated over one hidden layer to one output unit with the transformed value of the time series at time t+1.This error is propagated back to those between hidden and output layer (fig. 2).
Fig.2. Neural network architecture
Weights update process happen during the learning phase has shown in Fig. 3.
Fig.3. Weights update process
System has implement to gather training data from spread sheet and train the neural network. Then System has provided the functionality to give predated value for next day by using the trained neural network.
- EVALUATION OF THE IMPLEMENTATION
To evaluate the forecasting results, two criteria are used: Root Mean Square Errors (RMSE) and Mean Absolute Errors (MAE). Also, correlation coefficient of predicted and actual dataset is considered when analyzing the results. The correlation coefficient a concept from statistics is a measure of how well trends in the predicted values follow trends in past actual values. It is a measure of how well the predicted values from a forecast model "fit" with the real data.
- Evaluation Process
Evaluation process is divided in to several stages.
- Varying the neural network architecture (Using different no. of hidden neurons, different no. of hidden layers, and using different activation functions) find out the e efficiency of the neural network for electricity load forecasting.
2. Analyze the efficiency of network by varying the input parameters to the neural network.
3. Compared the results of neural network forecasting model with the statistical forecasting models such as moving average and the regression.
- Deciding input nodes
When deciding the amount of input nodes, as mansion in the design, previous electricity loads, Month of the year, day of the week, maximum temperature and minimum temperature has used as initial inputs to the neural network. But in the testing phase some of these variables are seems to be not directly affect to the load forecast.
If it gives day of the week as inputs network not gives accurate forecast. Fig.4. show the forecasted load with day effect. It gives low correlation coefficient - 0.1545.
Fig.4. Day as input to the ANN
Network used previous 7 day electricity load. Day effect is implicitly included in that inputs. Therefore it has decided not to be include day as separate input.
It couldn't be found a relationship between daily maximum temperature and the daily electricity load. Fig.5. show the correlation between the maximum temperature and the electricity load.
Fig.5. Electricity load Vs. Max Temperature
In country like Sri Lanka, temperature varies mostly the range within the 25ºC to 35ºC. So it cannot be intended a big effect to electricity load by temperature. But if compared with other countries, it can be seen direct influence on temperature to the electricity load. Because temperature of those countries is varies within huge scale. Sometimes it has range of -10ºC to 40ºC. So it consumes huge electricity load for heat or cool the environment. That's why it shows huge correlation between electricity load and temperature, of those countries. The final neural network architecture has 11 inputs. They are;
- Previous 7 days electricity load
- Month of the year(3 inputs encoded with binary values)
- Holiday effect(1 input)
- Find the optimum number of hidden neurons
Next phase is to find the optimum no. of hidden neurons in the network. This can do by analyzing the error values for different number of hidden neurons. Results can be shown as Fig. 6.
Fig.6. No. of hidden neurons Vs. RMSE
It can be clearly see the lowest RMSE value give for when the neural network use 7 hidden neurons. If it consider about the Correlation Coefficient of the predicted and actual electricity load data sets, it can be seen heighest correlation coefficient when using 10 hidden neurons Fig.7.
Fig.7. No. of hidden neurons Vs. Correlation Coefficient
By considering the both the correlation coefficient and RMSE factors can be concluded use of 10 hidden neurons is better. Because, neural network gives low RMSE value and high correlation coefficient when it using 10 hidden neurons. In most case of neural network applications, they have used three layered neural network architectures. As same as, in this system it also used three layered neural network architecture. Whether it has tested with four layers, it couldn't give significant different in forecasting. Followings are the compared values with three layered and four layered neural network architectures (Table. I). Therefore this system is used three layered neural network architecture.
Table I. Network accuracy for different no. of hidden layers
No. of layers | RMSE | Correlation Coefficient |
3 | 3.499 | 0.9209 |
4 | 3.409 | 0.903 |
The final neural network architecture for electricity load forecasting is include 11 input neurons, 10 hidden neurons and one output neuron. The sample forecasting output for neural network architecture can be shown as Fig.8. At this moment it has not used momentum value and learning rate is 0.05
Fig.8. A forecasting example without the momentum factor
This configuration of the neural network gave correlation coefficient of 0.8828. Still network not give accurate forecasting. It can be increase the accuracy by increasing the momentum factor.
Change of momentum factor has given outputs as table: II
Table II. Effect of momentum factor
Momentum | Correlation coefficient | RMSE |
0 | 0.8161 | 3.6623 |
0.1 | 0.8627 | 3.6387 |
0.2 | 0.8987 | 3.4496 |
0.3 | 0.8978 | 3.8086 |
0.4 | 0.8895 | 3.5990 |
0.5 | 0.869 | 3.6066 |
0.6 | 0.8978 | 3.6526 |
0.7 | 0.9015 | 3.5093 |
0.8 | 0.9019 | 3.5381 |
0.9 | 0.9179 | 3.4468 |
When increasing the momentum, Correlation coefficient is also increasing. When increasing the momentum, RMSE value is decreasing. By considering both of those factors it can be concluded that the momentum value of 0.9 is suitable for this neural network architecture.
Learning rate is directly affected to the forecasting process.
Graph in the Fig.9.Shows relationship between learning rate and the correlation coefficient. It clearly shows forecasting gives high accuracy when using the low learning rate.
Fig.9. Learning rate Vs. Correlation Coefficient
When using the high learning rate neural network unable to find the global minima. It cannot be seen gradually decreasing the error during the training (Fig.10).
Fig.10. Error during the learning when use higher learning rate
Considering the above analyses, it decided the suitable learning rate for electricity load forecasting system is 0.05.
Use of different activation function gives different forecasting values. This system tested with several activation functions and gave results as Fig.11. It has kept the momentum on 0.9 and learning rate on 0.05 during this analysis.
Fig.11. Effect of Activation function
Highest correlation coefficient gives when using the bipolar sigmoid function. So this system chose the bipolar sigmoid function as the most suitable activation function. Improved network with bipolar sigmoid function gives better forecast than the initial network. It gives correlation coefficient of 0.9251. Sample forecasted values shown as Fig.12.
Fig.12. Improved network forecast with bipolar sigmoid function
Fig.13. show how the error decreases during the training phase.
Fig.13. Error during the training phase
The proposed neural network electricity load forecasting model has been compared with the statistical models. It is found that neural network model is far superior to those statistical models. To do that comparison it has used moving average and regression models. Results can be shown as in Fig.14, 15.
Fig.14. 4 period moving average and Neural network models.
Fig.15. Linear regression and Neural network models
It's very clear that according to the above figures, conventional statistical models can't do accurate electricity load forecast compared to the neural network model.
Fig.15. gives more comparable view of neural network and all the other forecasting models.
It shows,
4 periods moving average – (Black line)
8 periods moving average – (Purple line)
4th order polynomial regression – (light blue line)
Linear regression – (Yellow line)
Naïve – (Green line) and
Neural network
– (Dark blue line) forecasts, for the same data set.
Fig.15. Comparison of Neural network forecasting model with statistical models
- CONCLUSIONS AND FUTURE WORK
The existing forecasting models based on statistics were studied. Advantages and disadvantages of this model were reviewed. A representative neural network aided electricity load forecasting model was developed. Using data gathered from the CEB, the neural network was trained. Results were compared to the actual electricity load. This research has found out how accuracy it can be forecast the electricity load. Also it has analyzed what is the best neural network model for forecasting by testing with different neural network architectures and testing with different parameters. The most suitable neural architecture for electricity load forecasting can be shown as below.
No of input nodes : 11
No of hidden nodes : 10
No of layers : 3
Momentum factor : 0.9
Learning rate : 0.05
Activation function : Bipolar Sigmoid function
During the evaluation phase it was able to drawn following conclusions:
- With having above configurations, neural network can predict the electricity load with Correlation coefficient of 0.9179 and RMSE 3.4468.
- Neural network is found to be a viable alternative in electricity load forecasting application.
3. Neural network is found to be a more efficient and accurate mean to forecast electricity load than using conventional techniques.
Future studies on this work can incorporate information about the customer class into the network so as to obtain a more representative forecast of future load. Also this system unable to forecast correctly if there is a sudden increase or decrease of demand. For example, if there is a cricket match, usage of television, increase rapidly. Hence the demand can be increased suddenly. It should find out a way to give this information to the neural network to do better forecast. In country like Sri Lanka electricity is mostly depend on the hydraulic power. If there is a sudden drought period, it has to cut down the electricity. This can be effect on the forecasting system. This system can be improved to give solutions to such situations. For example it can give new inputs such as water level of the reservoir, amount of rainfall to the neural network.
References
[1] A. Feinberg and D. Genethliou. Load Forecasting. In: H. Chow, F. Wu, J.J.Momoh. Applied Mathematics for Restructured Electric Power Systems: Optimization, Control, and Computational Intelligence. New York. pp. 269-285. 2005.
[2] R. Weron and A. Misiorek. Modeling And Forecasting Electricity Loads: A Comparison. Proceedings of the International Conference on European Electricity Market. Poland. September 20-22, 2004.
[3] R. Eeron, B. Kozlowska and J. Nowicka. Modeling electricity loads in California: a continuous-time approach. Physica A 299, pp.344-350. 2001.
[4] A. Feinberg, D. Genethliou and T. Hauagos. Load Pocket Modeling. Proceedings of the 2nd IASTED International Conference on Power and Energy Systems. Crete pp.50-54. 2002.
[5] A. Feinberg, D. Genethliou and T. Hauagos. Statistical Load Modeling. Proceedings of the 7th IASTED International Multi Conference on Power and Energy Systems. Palm Springs, CA. pp.88-91. 2003.
[6] F. Engle, C. Mustafa and J. Rece. Modeling Peak Electricity Demand. International Journal of Forecasting. pp.241-251, 1992.
[7] M. Mohandes. Support Vector Machines for Short-Term Electrical Load Forecasting. International Journal of Energy Research pp.335-345, 2002.
[8] W. Gellings and L. Barron. Demand Forecasting for Electricity. The Fairmont Press Inc.PE, pp.538 - 566, 1992.
[9] F. Atiya, M. El-Shoura, I. Shaheen and S. El-Sherif. A Comparison between Neural Network Forecasting Techniques-Case Study: River Flow Forecasting. IEEE Transactions on neural networks. Vol. 10, No. 2. march 1999.
[10] G. Bandyopadhyay and S. Chattopadhyay. An Artificial Neural Net approach to forecast the population of India. India. 2006.
[11] P. Kumar and E. Walia. Cash Forecasting: An Application of Artificial Neural Networks in Finance. International Journal of Computer Science & Applications Vol. III, No. I, pp. 61 – 77. 2006.
[12] Schimann, M.Joost, R.Werner. Optimization of the Backpropagation Algorithm for Training Multilayer Perceptrons. University of Koblenz, Institute of Physics, September 29, 1994
[13] T. Kolarik and G.Rudorfer. Time series forecasting using neural networks. Proceedings of the international conference on APL : the language and its applications:, Antwerp, Belgium.1994.
[14] P. Cortez, J. Machado and J. Neves. An Evolutionary Artificial Neural Network Time Series Forecasting System. Proceedings of IASTED International Conference on Artificial Intelligence, Expert Systems and Neural Networks. Honolulu, Havaii, USA, pp. 278-281, August,1996.
[15] S. Agular, M. Rodriguez and M. Cabrera-Rios. Statistical characterization and optimization of Artificial Neural Network in time series forecasting:The one period forecasting case. Mexico. Computaion systems. Vol 10 no.1.pp.69-81. 2006.
[16] University of Idaho [Online]. [cited 2008 Mar 03]; Available from: URL: www.uidaho.edu/ag/statprog/sas/workshops/catmod/handout1 cat.pdf
