Residential Energy Scheduling for Variable Weather Solar Energy Based on Adaptive Dynamic Programming

Derong Liu; Yancai Xu; Qinglai Wei; Xinliang Liu

doi:10.1109/JAS.2017.7510739

Volume 5 Issue 1

Jan. 2018

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2018 > 5(1): 36-46

Derong Liu, Yancai Xu, Qinglai Wei and Xinliang Liu, "Residential Energy Scheduling for Variable Weather Solar Energy Based on Adaptive Dynamic Programming," IEEE/CAA J. Autom. Sinica, vol. 5, no. 1, pp. 36-46, Jan. 2018. doi: 10.1109/JAS.2017.7510739

Citation:

Derong Liu, Yancai Xu, Qinglai Wei and Xinliang Liu, "Residential Energy Scheduling for Variable Weather Solar Energy Based on Adaptive Dynamic Programming," IEEE/CAA J. Autom. Sinica, vol. 5, no. 1, pp. 36-46, Jan. 2018. doi: 10.1109/JAS.2017.7510739

Citation:

PDF( 2171 KB)

Residential Energy Scheduling for Variable Weather Solar Energy Based on Adaptive Dynamic Programming

doi: 10.1109/JAS.2017.7510739

Derong Liu^{1
,
,},
Yancai Xu^2
,,
Qinglai Wei^2
,,
Xinliang Liu^3
,

1.
School of Automation, Guangdong University of Technology, Guangzhou 510006, China
2.
The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
3.
The State Key Laboratory of Management and Control for Complex Systems, University of Chinese Academy of Sciences, Beijing 100049
4.
The Bureau of Informationization Development, Cyberspace Administration of China, Beijing 100010, China

Funds:

the National Natural Science Foundation of China 61533017

the National Natural Science Foundation of China U1501251

the National Natural Science Foundation of China 61374105

the National Natural Science Foundation of China 61722312

More Information

Author Bio:
Derong Liu (S'91-M'94-SM'96-F'05) received the Ph.D. degree in electrical engineering from the University of Notre Dame in 1994. He was a Staff Fellow with General Motors Research and Development Center, from 1993 to 1995. He was an Assistant Professor with the Department of Electrical and Computer Engineering, Stevens Institute of Technology, from 1995 to 1999. He joined the University of Illinois at Chicago in 1999, and became a Full Professor of Electrical and Computer Engineering and of Computer Science in 2006. He was selected for the "100 Talents Program" by the Chinese Academy of Sciences in 2008, and he served as the Associate Director of The State Key Laboratory of Management and Control for Complex Systems at the Institute of Automation. He has published 18 books. He is the Editor-in-Chief of Artificial Intelligence Review (Springer). He was the Editor-in-Chief of the IEEE Transactions on Neural Networks and Learning Systems from 2010 to 2015. He received the Faculty Early Career Development Award from the National Science Foundation in 1999, the University Scholar Award from University of Illinois from 2006 to 2009, the Overseas Outstanding Young Scholar Award from the National Natural Science Foundation of China in 2008, and the Outstanding Achievement Award from Asia Pacific Neural Network Assembly in 2014. He is a Fellow of the IEEE, a Fellow of the International Neural Network Society, and a Fellow of the International Association of Pattern Recognition. (e-mail: derongliu@foxmail.com)

Yancai Xu is a Ph.D. candidate in The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences. His research interests include adaptive dynamic programming, optimal control and energy management. (e-mail: yancai.xu@ia.ac.cn)

Qinglai Wei (M'11) received the B.S. degree in automation, the M.S. degree in control theory and control engineering, and the Ph.D. degree in control theory and control engineering from Northeastern University, Shenyang, China, in 2002, 2005, and 2008, respectively. From 2009 to 2011, he was a Post-Doctoral Fellow with the State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China, where he is currently an Professor. He has authored one book, and published over 40 international journal papers. His current research interests include neural-networks-based control, adaptive dynamic programming, optimal control, and nonlinear systems and their industrial applications. (e-mail: qinglai.wei@ia.ac.cn)

Xinliang Liu received the B.S. degree in electronic science and technology from Beijing University of Technology in 2014, the M.S. degree in electrical and computer engineering from University of Illinois at Chicago in 2015. He was a researcher in China's Center for Information Industry Development and authored several publications. Currently, he works for the Cyberspace Administration of China. His research interests include neural-networks-based control, optimal control, and integrated circuit design. (e-mail: liuxinliang@ccidtank.com)
Corresponding author: Derong Liu, e-mail:derongliu@foxmail.com
Received Date: 2017-08-01
Accepted Date: 2017-09-17

Abstract

Abstract

The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable energy resources, are combined together as a nonlinear, time-varying, indefinite and complex system, which is difficult to manage or optimize. Many nations have already applied the residential real-time pricing to balance the burden on their grid. In order to enhance electricity efficiency of the residential micro grid, this paper presents an action dependent heuristic dynamic programming (ADHDP) method to solve the residential energy scheduling problem. The highlights of this paper are listed below. First, the weather-type classification is adopted to establish three types of programming models based on the features of the solar energy. In addition, the priorities of different energy resources are set to reduce the loss of electrical energy transmissions. Second, three ADHDP-based neural networks, which can update themselves during applications, are designed to manage the flows of electricity. Third, simulation results show that the proposed scheduling method has effectively reduced the total electricity cost and improved load balancing process. The comparison with the particle swarm optimization algorithm further proves that the present method has a promising effect on energy management to save cost.
- Action dependent heuristic dynamic programming,
- adaptive dynamic programming,
- control strategy,
- residential energy management,
- smart grid

FullText(HTML)

References(55)

References

[1]	P. Palensky and D. Dietrich, "Demand side management: Demand response, intelligent energy systems, and smart loads, " IEEE Transactions on Industrial Informatics, vol. 7, no. 3, pp. 381-388, Jun. 2011. http://ieeexplore.ieee.org/document/5930335/
[2]	A. Chaouachi, R. M. Kamel, R. Andoulsi, and K. Nagasaka, "Multiobjective intelligent energy management for a microgrid, " IEEE Transactions on Industrial Electronics, vol. 60, no. 4, pp. 1688-1699, Apr. 2013. http://ieeexplore.ieee.org/document/6157610/
[3]	B. Huang, Y. Li, H. Zhang, and Q. Sun, "Distributed optimal co-multi-microgrids energy management for energy internet, " IEEE/CAA Journal of Automatica Sinica, vol. 3, no. 4, pp. 357-364, Oct. 2016. http://ieeexplore.ieee.org/document/7589482
[4]	Q. Dong, L. Yu, W. Song, J. Yang, Y. Wu, and J. Qi, "Fast Distributed Demand Response Algorithm in Smart Grid, " IEEE/CAA Journal of Automatica Sinica, vol. 4, no. 2, pp. 280-296, Apr. 2017. http://kns.cnki.net/KCMS/detail/detail.aspx?filename=zdhb201702015&dbname=CJFD&dbcode=CJFQ
[5]	M. R. Alam, M. St-Hilaire, and T. Kunz, "Computational Methods for Residential Energy Cost Optimization in Smart Grids: A Survey, " ACM Computing Surveys, vo, . 49, no. 1, pp. 1-34, Jul. 2016. doi: 10.1145/2897165
[6]	Z. Hong, R. Wang, and X. Li, "A clustering-tree topology control based on the energy forecast for heterogeneous wireless sensor networks, " IEEE/CAA Journal of Automatica Sinica, vol. 3, no. 1, pp. 68-77, Jan. 2016. http://ieeexplore.ieee.org/document/7373764/
[7]	R. Hemmati, R. A. Hooshmand, and A. Khodabakhshian, "Coordinated generation and transmission expansion planning in deregulated electricity market considering wind farms, " Renewable Energy, vol. 85, pp. 620-630, Jan. 2016. http://www.sciencedirect.com/science/article/pii/S0960148115301142
[8]	F. Zhao, C. Zhang, B. Sun, "Initiative optimization operation strategy and multi-objective energy management method for combined cooling heating and power, " IEEE/CAA Journal of Automatica Sinica, vol. 3, no. 4, pp. 385-393, Oct. 2016. http://ieeexplore.ieee.org/document/7589484
[9]	M. S. Hossain, N. A. Madlool, N. A. Rahim, J. Selvaraj, A. K. Pandey, and A. F. Khan, "Role of smart grid in renewable energy: An overview, " Renewable and Sustainable Energy Reviews, vol. 60, pp. 1168-1184, Jul. 2016. https://www.researchgate.net/publication/296211867_Role_of_smart_grid_in_renewable_energy_An_overview
[10]	J. Chen and F. Yang, "Data-driven subspace-based adaptive fault detection for solar power generation systems, " IET Control Theory & Applications, vol. 7, no. 11, pp. 1498-1508, Jul. 2013. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6587888
[11]	U. S. Department of Energy, http://www.energy.gov/science-innovation/energy-sources/renewable-energy/solar, avaibable online Dec. 2015.
[12]	P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling, " in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, D. A. White and D. A. Sofge, Editors. New York: Van Nostrand Reinhold, Chapter 13, 1992. https://www.researchgate.net/publication/243651581_Approximate_dynamic_programming_for_real-time_control_and_neural_modeling
[13]	A. Heydari and S. N. Balakrishnan, "Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics, " IEEE Transactions on Neural Networks and Learning Systems, vol. 24, no. 1, pp. 145-157, Jan. 2013. http://europepmc.org/abstract/med/24808214
[14]	Q. Wei, R. Song, and P. Yan, "Data-driven zero-sum neuro-optimal control for a class of continuous-time unknown nonlinear systems with disturbance using ADP, " IEEE transactions on Neural Networks and Learning Systems, vol. 27, no. 2, pp. 444-458, Feb. 2016. http://www.ncbi.nlm.nih.gov/pubmed/26292346
[15]	Q. Wei and D. Liu, "A novel iterative θ-Adaptive dynamic programming for discrete-time nonlinear systems, " IEEE Transactions on Automation Science and Engineering, vol. 11, no. 4, pp. 1176-1190, Oct. 2014. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6609148
[16]	Q. Wei and D. Liu, "Adaptive dynamic programming for optimal tracking control of unknown nonlinear systems with application to coal gasification, " IEEE Transactions on Automation Science and Engineering, vol. 11, no. 4, pp. 1020-1036, Oct. 2014. http://ieeexplore.ieee.org/document/6656960/
[17]	Q. Wei, D. Liu, and Y. Xu, "Neuro-optimal tracking control for a class of discrete-time nonlinear systems via generalized value iteration adaptive dynamic programming approach, " Soft Computing, vol. 20, no. 2, pp. 697-706, Feb. 2016. doi: 10.1007/s00500-014-1533-0
[18]	H. He, Z. Ni, and J. Fu, "A three-network architecture for on-line learning and optimization based on adaptive dynamic programming, " Neurocomputing, vol. 78, no. 1, pp. 3-13, Feb. 2012. http://www.sciencedirect.com/science/article/pii/S0925231211004760
[19]	H. Li, L. Wang, H. Du, and A. Boulkroune, "Adaptive fuzzy backstepping tracking control for strict-feedback systems with input delay, " IEEE Transactions on Fuzzy Systems, vol. 25, no. 3, pp. 642-652, Jun. 2017. http://ieeexplore.ieee.org/document/7469360/
[20]	Q. Wei, D. Liu, and G. Shi, "A novel dual iterative Q-learning method for optimal battery management in smart residential environments, " IEEE Transactions on Industrial Electronics, vol. 62, no. 4, pp. 2509-2518, Apr. 2015. http://ieeexplore.ieee.org/document/6915886/
[21]	D. V. Prokhorov and D. C. Wunsch, "Adaptive critic designs, " IEEE Transactions on Neural Networks, vol. 8, no. 5, pp. 997-1007, Sep. 1997. http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=623201
[22]	J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, "Adaptive dynamic programming, " IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, vol. 32, no. 2, pp. 140-153, May 2002. http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=1039198
[23]	B. Luo, H. Wu, and T. Huang, "Off-policy reinforcement learning for H∞ control design, " IEEE Transactions on Cybernetics, vol. 45, no. 1, pp. 65-76, Jan. 2015. http://www.ncbi.nlm.nih.gov/pubmed/25532162
[24]	Y. Jiang and Z. P. Jiang, "Robust adaptive dynamic programming and feedback stabilization of nonlinear systems, " IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 5, pp. 882-893, May 2014. http://europepmc.org/abstract/med/24808035
[25]	Q. Wei and D. Liu, "Numerical adaptive learning control scheme for discrete-time non-linear systems, " IET Control Theory & Applications, vol. 7, no. 11, pp. 1472-1486, Jul. 2013. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6587886
[26]	Y. Tang, H. He, J. Wen, and J. Liu, "Power system stability control for a wind farm based on adaptive dynamic programming, " IEEE Transactions on Smart Grid, vol. 6, no. 1, pp. 166-177, Jan. 2015. http://ieeexplore.ieee.org/document/6915705/
[27]	C. Watkins and P. Dayan, "Q-learning, " Machine Learning, vol. 8, no. 3/4, pp. 279-292, May 1992.
[28]	F. L. Lewis, D. Vrabie, and K. G. Vamvoudakis, "Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers, " IEEE transactions on Control Systems Technology, vol. 32, no. 6, pp. 76-105, Dec. 2012. http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=6315769
[29]	F. L. Lewis and D. Vrabie, "Reinforcement learning and adaptive dynamic programming for feedback control, " IEEE Circuits and Systems Magazine, vol. 9, no. 3, pp. 32-50, Aug. 2009. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5227780
[30]	Q. Wei and D. Liu, "A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems, " Science China Information Sciences, vol. 58, no. 12, pp. 122203: 1-122203: 15, Dec. 2015.
[31]	Y. Liang, L. He, X. Cao, and Z. J. Shen, "Stochastic control for smart grid users with flexible demand, " IEEE Transactions on Smart Grid, vol. 4, no. 4, pp. 2296-2308, Dec. 2013. http://ieeexplore.ieee.org/document/6558842/
[32]	B. Xu, C. Yang, and Z. Shi, "Reinforcement learning output feedback NN control using deterministic learning technique, " IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 3, pp. 635-641, Mar. 2014. http://ieeexplore.ieee.org/document/6681972/
[33]	S. Mohagheghi, G. K. Venayagamoorthy, and R. G. Harley, "Fully evolvable optimal neurofuzzy controller using adaptive critic designs, " IEEE Transactions on Fuzzy Systems, vol. 16, no. 6, pp. 1450-1461, Dec. 2008. http://ieeexplore.ieee.org/document/4529088/
[34]	Y. Riffonneau, S. Bacha, F. Barruel, and S. Ploix, "Optimal power flow management for grid connected PV systems with batteries, " IEEE Transactions on Sustainable Energy, vol. 2, no. 3, pp. 309-320, Jul. 2011. http://ieeexplore.ieee.org/document/5713847/
[35]	D. K. Maly and K. S. Kwan, "Optimal battery energy storage system (BESS) charge scheduling with dynamic programming, " IEEE Proceedings -Science, Measurement and Technology, vol. 142, no. 6, pp. 453-458, Nov. 1995. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=487634
[36]	C. Chen, S. Duan, T. Cai, and B. Liu, "Energy trading model for optimal microgrid scheduling based on genetic algorithm, " in Proceedings of IEEE 6th International Power Electronics and Motion Control Conference, Wuhan, China, Jul. 2009, pp. 2136-2139. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5157753
[37]	L. Wang, H. Li, Q. Zhou, and R. Lu, "Adaptive fuzzy control for nonstrict feedback systems with unmodeled dynamics and fuzzy dead zone via output feedback, " IEEE Transactions on Cybernetics, vol. 47, no. 9, pp. 2400-2412, Sep. 2017. http://europepmc.org/abstract/MED/28422700
[38]	T. Huang and D. Liu, "A self-learning scheme for residential energy system control and management, " Neural Computing and Applications, vol. 22, no. 2, pp. 259-269, Feb. 2013. doi: 10.1007/s00521-011-0711-6
[39]	M. Boaro, D. Fuselli, F. D. Angelis, D. Liu, Q. Wei, and F. Piazza, "Adaptive dynamic programming algorithm for renewable energy scheduling and battery management, " Cognitive Computation, vol. 5, no. 2, pp. 264-277, Jun. 2013. doi: 10.1007/s12559-012-9191-y
[40]	D. Fuselli, F. D. Angelis, and M. Boaro, "Action dependent heuristic dynamic programming for home energy resource scheduling, " International Journal of Electrical Power and Energy Systems, vol. 48, no. 1, pp. 148-160, Jun. 2013. http://www.sciencedirect.com/science/article/pii/S014206151200676X
[41]	Q. Wei, D. Liu, Y. Liu, and R. Song, "Optimal constrained self-learning battery sequential management in microgrid via adaptive dynamic programming, " IEEE/CAA Journal of Automatica Sinica, vol. 4, no. 2, pp. 168-176, Apr. 2017. http://kns.cnki.net/KCMS/detail/detail.aspx?filename=zdhb201702003&dbname=CJFD&dbcode=CJFQ
[42]	Y. Tang, H. He, Z. Ni, J. Wen, and X. Sui, "Reactive power control of grid-connected wind farm based on adaptive dynamic programming, " Neurocomputing, vol. 125, pp. 125-133, Feb. 2014. http://dl.acm.org/citation.cfm?id=2562346.2562654
[43]	C. Mu, Z. Ni, C. Sun, and H. He, "Air-Breathing Hypersonic Vehicle Tracking Control Based on Adaptive Dynamic Programming, " IEEE Transactions on Neural Networks & Learning Systems, vol. 28, no. 3, pp. 584-598, Mar. 2017. http://ieeexplore.ieee.org/document/7398119/
[44]	Q. Wei and D. Liu, "Data-driven neuro-optimal temperature control of water-gas shift reaction using stable iterative adaptive dynamic programming, " IEEE Transactions on Industrial Electronics, vol. 61, vol. 11, pp. 6399-6408, Nov. 2014. http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=6718005
[45]	Y. Xu, D. Liu, and Q. Wei, "Action dependent heuristic dynamic programming based residential energy scheduling with home energy inter-exchange, " Energy Conversion and Management, vol. 103, pp. 553-561, Oct. 2015. http://www.sciencedirect.com/science/article/pii/S0196890415005890
[46]	Weather Underground, http://www.wunderground.com, available online Oct. 2015.
[47]	C. Chen, S. Duan, T. Cai, and B. Liu, "Online 24-h solar power forecasting based on weather type classification using artificial neural network, " Solar Energy, vol. 85, no. 11, pp. 2856-2870, Nov. 2011. http://www.sciencedirect.com/science/article/pii/S0038092X11003008
[48]	C. Tao, S. Duan, and C. Chen, "Forecasting power output for grid-connected photovoltaic power system without using solar radiation measurement, " in Proceedings of 2nd IEEE International Symposium on Power Electronics for Distributed Generation Systems, Hefei, China, Jun. 2010, pp. 773-777. http://ieeexplore.ieee.org/document/5545754/
[49]	P. Bacher, H. Madsen, and H. A. Nielsen, "Online short-term solar power forecasting, " Solar Energy, vol. 83, no. 10, pp. 1772-1783, Oct. 2009. http://www.sciencedirect.com/science/article/pii/S0038092X09001364
[50]	ComEd, http://hourlypricing.comed.com, available online May 2017.
[51]	OpenEI, http://en.openei.org/community, available online Aug. 2014.
[52]	T. Yau, L. N. Walker, H. L. Gupta, and A. Gupta, "Effects of battery storage devices on power system dispatch, " IEEE Transactions on Power Apparatus and Systems, vol. PAS-100, no. 1, pp. 375-383, Jan. 1981. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4110487
[53]	F. Wang, H. Zhang, and D. Liu, "Adaptive dynamic programming: an introduction, " IEEE Computational Intelligence Magazine, vol. 4, no. 2, pp. 39-47, May 2009. http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=4840325
[54]	R. E. Bellman, Dynamic Programmming. Princeton, NJ: Princeton University Press, 1957.
[55]	M. R. AlRashidi and M. E. El-Hawary, "A survey of particle swarm optimization applications in electric power systems, " IEEE Transactions on Evolutionary Computation, vol. 13, no. 4, pp. 913-918, Aug. 2009. http://ieeexplore.ieee.org/document/4358752/

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(10) / Tables(5)

Get Citation

PDF

XML

Article Metrics

Article views (1573) PDF downloads(173)

Residential Energy Scheduling for Variable Weather Solar Energy Based on Adaptive Dynamic Programming

doi: 10.1109/JAS.2017.7510739

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Export File

Citation

Format

Content