Keywords
Optimal control
Economic dispatch
Adaptive dynamic programming
How to Cite
Abstract
Both adaptive dynamic programming and other intelligent algorithms can solve the economic dispatch problem in the microgrid. Adaptive dynamic programming can reduce the computational burden, which the intelligent algorithms suffer from, by using function approximation structure to approximate performance index function. In recent years, it has been also widely used in economic dispatch in the microgrid. In this article, we introduce some recent research trends within the field of adaptive dynamic programming based economic dispatch. Adaptive dynamic programming is firstly reviewed. Then, the current research works about adaptive dynamic programming based economic dispatch are summarized and compared. Furthermore, we point out some topics for future studies.
References
Werbos PJ. Computational intelligence for the smart grid-history, challenges, and opportunities. IEEE Computational Intelligence Magazine, 2011; 6(3): pp. 14-21. https://doi.org/10.1109/MCI.2011.941587
Venayagamoorthy GK. Dynamic, stochastic, computational, and scalable technologies for smart grids. IEEE Computational Intelligence Magazine, 2011; 6(3): pp. 22-35. https://doi.org/10.1109/MCI.2011.941588
Nguyen DT, Le LB. Optimal bidding strategy for microgrids considering renewable energy and building thermal dynamics. IEEE Transactions on Smart Grid, 2014; 5(4): pp. 1608-1620. https://doi.org/10.1109/TSG.2014.2313612
Talari S, Yazdaninejad M, Haghifam M. Stochastic-based scheduling of the microgrid operation including wind turbines, photovoltaic cells, energy storages and responsive loads. IET Generation, Transmission Distribution, 2015; 9(12): pp. 1498-1509. https://doi.org/10.1049/iet-gtd.2014.0040
Changjin X, et al. Further exploration on bifurcation of fractional-order six-neuron bi-directional associative memory neural networks with multi-delays. Applied Mathematics and Computation, 2021; 410: 126458. https://doi.org/10.1016/j.amc.2021.126458
Chen Z, Wang J, Ma K, et al. Fuzzy adaptive two‐bits‐triggered control for nonlinear uncertain system with input saturation and output constraint. International Journal of Adaptive Control and Signal Processing, 2020; 34(4): 543-559. https://doi.org/10.1002/acs.3098
Changjin X, et al. Bifurcation Dynamics in a Fractional-Order Oregonator Model Including Time Delay. Match-Communications in Mathematical and in Computer Chemistry, 2022; 87(2): 397-414. https://doi.org/10.46793/match.87-2.397X
Changjin X, et al. Bifurcation control strategy for a fractional-order delayed financial crises contagions model. AIMS Mathematics, 2022; 7(2): 2102-2122. https://doi.org/10.3934/math.2022120
Lewis FL. Vrabie D. Reinforcement learning and adaptive dynamic programming for feedback control. IEEE circuits and systems magazine, 2009; 9(3): pp. 32-50. https://doi.org/10.1109/MCAS.2009.933854
Lewis FL, Vrabie D, Vamvoudakis KG. Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers. IEEE Control Systems Magazine, 2012; 32(6): pp. 76-105. https://doi.org/10.1109/MCS.2012.2214134
Nguyen TA, Crow ML. Stochastic optimization of renewablebased microgrid operation incorporating battery operating cost. IEEE Transactions on Power Systems, 2016; 31(3): pp. 2289-2296. https://doi.org/10.1109/TPWRS.2015.2455491
Werbos P. Advanced forecasting methods for global crisis warning and models of intelligence. General System Yearbook, 1977; pp. 25-38.
Wei Q, Song R, Li B, Lin X. Self-Learning Optimal Control of Nonlinear Systems. Springer, 2018; 103: https://doi.org/10.1007/978-981-10-4080-1
Si J, Wang Y-T. Online learning control by association and reinforcement. IEEE Transactions on Neural networks, 2001; 12(2): pp. 264-276. https://doi.org/10.1109/72.914523
Al-Tamimi A, Lewis FL, Abu-Khalaf M. Discrete-time nonlinear hjb solution using approximate dynamic programming: Convergence proof. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008; 38(4): pp. 943-949. https://doi.org/10.1109/TSMCB.2008.926614
Wang D, Liu D, Wei Q, Zhao D, Jin N. Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica, 2012; 48(8): pp. 1825-1832. https://doi.org/10.1016/j.automatica.2012.05.049
Wei Q, Liu D, Lin H. Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems. IEEE Transactions on cybernetics, 2015; 46(3): pp. 840-853. https://doi.org/10.1109/TCYB.2015.2492242
Wei Q, Lewis FL, Liu D, Song R, Lin H. Discrete-time local value iteration adaptive dynamic programming: Convergence analysis. 2018; 48(6): pp. 875-891. https://doi.org/10.1109/TSMC.2016.2623766
Liu D, Wei Q. Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems, 2013; 25(3): pp. 621-634. https://doi.org/10.1109/TNNLS.2013.2281663
Wang J, Zhang H, Ma K, Liu Z, Chen CLP. Neural Adaptive Self-Triggered Control for Uncertain Nonlinear Systems With Input Hysteresis. IEEE transactions on neural networks and learning systems, 2021; https://doi.org/10.1109/TNNLS.2021.3072784
Wang J, Gong Q, Huang K, et al. Event-triggered Prescribed Settling Time Consensus Compensation Control for a Class of Uncertain Nonlinear Systems with Actuator Failures, IEEE transactions on neural networks and learning systems, 2021; https://doi.org/10.1109/TNNLS.2021.3129816
Xue S, Luo B, Liu D. Event-triggered adaptive dynamic programming for zero-sum game of partially unknown continuous-time nonlinear systems. IEEE Transactions on Systems, Man, and Cybernetics, 2019; pp. 1-11.
Vamvoudakis KG, Lewis FL, Hudas G. Multi-agent differential graphical games: Online adaptive learning solution for synchronization with optimality. Automatica, 2012; 48(8): pp. 1598-1611. https://doi.org/10.1016/j.automatica.2012.05.074
Abouheaf M, Lewis FL, Vamvoudakis KG, Haesaert S, Babuska R. Multi-agent discrete-time graphical games and reinforcement learning solutions. Automatica, 2014; 50(12): pp. 3038-3053. https://doi.org/10.1016/j.automatica.2014.10.047
Wei Q, Liu D. Data-driven neuro-optimal temperature control of water-gas shift reaction using stable iterative adaptive dynamic programming. IEEE Transactions on Industrial Electronics, 2014; 61(11): pp. 6399-6408. https://doi.org/10.1109/TIE.2014.2301770
Huang T, Liu D. A self-learning scheme for residential energy system control and management. Neural Computing and Applications, 2013; 22(2): pp. 259-269. https://doi.org/10.1007/s00521-011-0711-6
Boaro M, Fuselli D, De Angelis F, Liu D, Wei Q, Piazza F. Adaptive dynamic programming algorithm for renewable energy scheduling and battery management. Cognitive Computation, 2013; 5(2): pp. 264-277. https://doi.org/10.1007/s12559-012-9191-y
Fuselli D, De Angelis F, Boaro M, Squartini S, Wei Q, Liu D, Piazza F. Action dependent heuristic dynamic programming for home energy resource scheduling. International Journal of Electrical Power & Energy Systems, 2013; 48: pp. 148-160. https://doi.org/10.1016/j.ijepes.2012.11.023
Xu Y, Liu D, Wei Q. Action dependent heuristic dynamic programming based residential energy scheduling with home energy interexchange. Energy Conversion and Management, 2015; 103: pp. 553-561. https://doi.org/10.1016/j.enconman.2015.06.048
Liu D, Xu Y, Wei Q, Liu X. Residential energy scheduling for variable weather solar energy based on adaptive dynamic programming. IEEE/CAA Journal of Automatica Sinica, 2017; 5(1): pp. 36-46. https://doi.org/10.1109/JAS.2017.7510739
Shuai H, Fang J, Ai X, Wen J, He H. Optimal real-time operation strategy for microgrid: An adp-based stochastic nonlinear optimization approach. IEEE Transactions on Sustainable Energy, 2018; 10(2): pp. 931-942. https://doi.org/10.1109/TSTE.2018.2855039
Shuai H, Fang J, Ai X, Tang Y, Wen J, He H. Stochastic optimization of economic dispatch for microgrid based on approximate dynamic programming. IEEE Transactions on Smart Grid, 2018; 10(3): pp. 2440-2452. https://doi.org/10.1109/TSG.2018.2798039
Wei Q, Liao Z, Song R, Zhang P, Wang Z, Xiao J. Self-Learning Optimal Control for Ice-Storage Air Conditioning Systems via Data-Based Adaptive Dynamic Programming. in IEEE Transactions on Industrial Electronics, 2021; 68(4): pp. 3599-3608. https://doi.org/10.1109/TIE.2020.2978699
Wei Q, Liu D, Liu Y, Song R. Optimal constrained self-learning battery sequential management in microgrid via adaptive dynamic programming. IEEE/CAA Journal of Automatica Sinica, 2016; 4(2): pp. 168-176. https://doi.org/10.1109/JAS.2016.7510262
Wei Q, Shi G, Song R, Liu Y. Adaptive dynamic programmingbased optimal control scheme for energy storage systems with solar renewable energy. IEEE Transactions on Industrial Electronics, 2017; 64(7): pp. 5468-5478. https://doi.org/10.1109/TIE.2017.2674581
Shi G, Liu D, Wei Q. Echo state network-based q-learning method for optimal battery control of offices combined with renewable energy. IET Control Theory & Applications, 2017; 11(7): pp. 915-922. https://doi.org/10.1049/iet-cta.2016.0653
Zhu Y, Zhao D, Li X, Wang D. Control-limited adaptive dynamic programming for multi-battery energy storage systems. IEEE Transactions on Smart Grid, 2019; 10(4): pp. 4235-4244. https://doi.org/10.1109/TSG.2018.2854300
Lewis FL, Vrabie D, Syrmos VL. Optimal control. John Wiley & Sons, 2012. https://doi.org/10.1002/9781118122631
Lewis FL, Vamvoudakis KG. Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2010; 41(1): pp. 14-25. https://doi.org/10.1109/TSMCB.2010.2043839
Vrabie D, Lewis F. Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Networks, 2009; 22(3): pp. 237-246. https://doi.org/10.1016/j.neunet.2009.03.008
Liu D, Wei Q. Finite-approximation-error-based optimal control approach for discrete-time nonlinear systems. IEEE Transactions on Cybernetics, 2013; 43(2): pp. 779-789. https://doi.org/10.1109/TSMCB.2012.2216523
Wei Q, Liu D, Shi G. A novel dual iterative q-learning method for optimal battery management in smart residential environments. IEEE Transactions on Industrial Electronics, 2014; 62(4): pp. 2509-2518. https://doi.org/10.1109/TIE.2014.2361485
Wei Q, Liu D, Shi G, Liu Y. Multibattery optimal coordination control for home energy management systems via distributed iterative adaptive dynamic programming. IEEE Transactions on Industrial Electronics, 2015; 62(7): pp. 4203-4214. https://doi.org/10.1109/TIE.2014.2388198
Venayagamoorthy GK, Sharma RK, Gautam PK, Ahmadi A. Dynamic energy management system for a smart microgrid. IEEE transactions on neural networks and learning systems, 2016; 27(8): pp. 1643-1656. https://doi.org/10.1109/TNNLS.2016.2514358
Wei Q, Lewis FL, Shi G, Song R. Error-tolerant iterative adaptive dynamic programming for optimal renewable home energy scheduling and battery management. IEEE Transactions on Industrial Electronics, 2017; 64(12): pp. 9527-9537. https://doi.org/10.1109/TIE.2017.2711499
Wei Q, Liu D, Lewis FL, Liu Y, Zhang J. Mixed iterative adaptive dynamic programming for optimal battery energy control in smart residential microgrids. IEEE Transactions on Industrial Electronics, 2017; 64(5): pp. 4110-4120. https://doi.org/10.1109/TIE.2017.2650872
Hong SH, Yu M, Huang X. A real-time demand response algorithm for heterogeneous devices in buildings and homes. Energy, 2015; 80: pp. 123-132. https://doi.org/10.1016/j.energy.2014.11.053
Yang X, Zhang Y, He H, Ren S, Weng G. Real-time demand side management for a microgrid considering uncertainties. IEEE Transactions on Smart Grid, 2019; 10(3): pp. 3401-3414. https://doi.org/10.1109/TSG.2018.2825388
Wang F-Y, Jin N, Liu D, Wei Q. Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with -error bound. IEEE Transactions on Neural Networks, 2010; 22(1): pp. 24-36. https://doi.org/10.1109/TNN.2010.2076370
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright (c) 2022 Zitao Chen, Quanbin Deng, Kairui Chen