Keywords
Adaptive control
Multi-armed bandits
Model predictive control
State-space representation
How to Cite
Abstract
This paper presents a novel approach to introducing adaptation in Model Predictive Control (MPC). Assuming limited a priori knowledge about the process, we consider a finite set of possible models (a dictionary), and use the theory of adversarial multi-armed bandits to develop an adaptive version of MPC called adversarial adaptive MPC (AAMPC). Under weak assumptions on the dictionary components, we then establish theoretical bounds on the performance of AAMPC and show its empirical behaviour via simulation examples.
References
Aggelogiannaki E, Sarimveis H. Multiobjective constrained MPC with simultaneous closed-loop identification. Int. J. Adapt. Control and Sig. Proc., 20(4):145-173, 2006. https://doi.org/10.1002/acs.892
Arora S, Hazan E, Kale S. The multiplicative weights update method: a meta-algorithm and applications. Th. of Computing, 8:121-164, 2012.
Åström KJ, Wittenmark B. Adaptive Control, 2nd Edition. Addison-Wesley, 1995.
Auer P, Cesa-Bianchi N, Freund Y, Schapire RE. The nonstochastic multi-armed bandit problem. SIAM J. Comput., 32(1):48-77, 2002. https://doi.org/10.1137/S0097539701398375
Bubeck S, Cesa-Bianchi N. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends in Machine Learning, 5(1):1-122, 2012. https://doi.org/10.1561/2200000024
Cesa-Bianchi N, Lugosi G. Prediction, Learning, and Games. Cambridge Univ. Press, 2006. https://doi.org/10.1017/CBO9780511546921
Coulson J, Lygeros J, Dörfler F. Data-enabled predictive control: In the shallows of the DeePC. In Proc. 18th Eur. Control Conf., pages 307-312, 2019. https://doi.org/10.23919/ECC.2019.8795639
Ebadat A, Annergren M, Larsson CA, Rojas CR, Wahlberg B, Hjalmarsson H, Sjöberg J. Application set approximation in optimal input design for model predictive control. In Proc. 13th Eur. Control Conf., pages 744-749, Strasbourg, France, 2014. https://doi.org/10.1109/ECC.2014.6862496
Ebadat A, Valenzuela PE, Rojas CR, Wahlberg B. Model predictive control oriented experiment design for system identification: A graph theoretical approach. J. Proc. Control, 52:75-84, 2017. https://doi.org/10.1016/j.jprocont.2017.02.001
Goodwin GC, Seron MM, De Don A. Constrained Control and Estimation: An Optimisation Approach. Springer, 2005. https://doi.org/10.1007/b138145
Grüne L, Pannek J. Nonlinear Model Predictive Control, 2nd Edition. Springer, 2017. https://doi.org/10.1007/978-3-319-46024-6
Hale ET, Qin SJ. Subspace model predictive control and a case study. In Proc. Amer. Control Conf., pages 4758-4763, 2002. https://doi.org/10.1109/ACC.2002.1025411
Heirung TAN, Foss B, Ydstie BE. MPC-based dual control with online experiment design. J. Proc. Control, 32:64-76, 2015. https://doi.org/10.1016/j.jprocont.2015.04.012
Iannelli A, Khosravi M, Smith RS. Structured exploration in the finite horizon linear quadratic dual control problem. arXiv preprint arXiv:1910.14492, 2019. https://doi.org/10.1016/j.ifacol.2020.12.1263
Katayama T. Subspace Methods for System Identification. Springer, 2005. https://doi.org/10.1007/1-84628-158-X
Kumar PR. An adaptive controller inspired by recent results on learning from experts. In K.J. Å ström, G.C. Goodwin, and P.R. Kumar, editors, Adapt. Control, Filtering, and Sig. Proc., pages 199-204. Springer, 1995. https://doi.org/10.1007/978-1-4419-8568-2_8
Larsson CA, Annergren M, Hjalmarsson H, Rojas CR, Bombois X, Mesbah A, Modén PE. Model predictive control with integrated experiment design for output error systems. In Proc. Eur. Control Conf., pages 3790-3795, Zurich, Switzerland, 2013. https://doi.org/10.23919/ECC.2013.6669533
Larsson CA, Ebadat A, Rojas CR, Bombois X, Hjalmarsson H. An application-oriented approach to dual control with excitation for closed-loop identification. Eur. J. of Control, 29:1-16, 2016. https://doi.org/10.1016/j.ejcon.2016.03.001
Lattimore T, Szepesvári C. Bandit algorithms. Cambridge Univ. Pr., 2020. https://doi.org/10.1017/9781108571401
Ljung L. System Identification: Theory for the User, 2nd Edition. Prentice Hall, 1999.
Löfberg J. YALMIP: A toolbox for modeling and optimization in MATLAB. In Proc. CACSD Conf., pages 284-289, Taipei, Taiwan, 2004.
Lorenzen M, Cannon M, Allgöwer F. Robust mpc with recursive model update. Automatica, 103:461-471, 2019. https://doi.org/10.1016/j.automatica.2019.02.023
Marafioti G, Bitmead RR, Hovd M. Persistently exciting model predictive control. Int. J. Adapt. Cont. Sig. Proc., 28(6):536-552, 2014. https://doi.org/10.1002/acs.2414
Marafioti G. Enhanced model predictive control: dual control approach and state estimation issues. Ph.d. thesis, Department of Engineering Cybernetics, Norwegian University of Science and Technology (NTNU), Norway, 2010.
Mardi N. Data-driven subspace-based model predictive control. Ph.d. thesis, RMIT University, Australia, 2010.
Mesbah A. Stochastic model predictive control with active uncertainty learning: A survey on dual control. Ann. Rev. Control, 45:107-117, 2018. https://doi.org/10.1016/j.arcontrol.2017.11.001
Van Overschee P, De Moor BL. Subspace identification for linear systems: Theory-Implementation-Applications. Springer, 2012.
Parsi A, Iannelli A, Smith RS. An explicit dual control approach for constrained reference tracking of uncertain linear systems. IEEE Transactions on Automatic Control, 2022. https://doi.org/10.1109/TAC.2022.3176800
Patwardhan RS, Gopaluni RB. A moving horizon approach to input design for closed loop identification. J. Proc. Control, 24(3):188-202, 2014. https://doi.org/10.1016/j.jprocont.2013.10.018
Raginsky M, Rakhlin A, Yuksel S. Online convex programming and regularization in adaptive control. In Proc. 49th IEEE Conf. Decision and Control, pages 1957-1962, Atlanta, USA, 2010. https://doi.org/10.1109/CDC.2010.5717262
Rallo G, Formentin S, Rojas CR, Oomen T, Savaresi SM. Data-driven -norm estimation via expert advice. In Proc. 56th IEEE Conf. Decision and Control, pages 1560-1565, Melbourne, Australia, 2017.
Rawlings JB, Mayne DQ, Diehl MM. Model Predictive Control: Theory, Comp., and Design, 2nd Ed. Nob Hill Pub., 2017.
Shouche MS, Genceli H, Nikolaou M. Effect of on-line optimization techniques on model predictive control and identification (mpci). Computers & Chemical Engineering, 26(9):1241-1252, 2002. https://doi.org/10.1016/S0098-1354(02)00091-1
Shouche M, Genceli H, Vuthandam P, Nikolaou M. Simultaneous constrained model predictive control and identification of DARX processes. Automatica, 34(12):1521-1530, 1998. https://doi.org/10.1016/S0005-1098(98)80005-8
Sutton RS, Barto AG. Reinforcement Learning, An Introduction, 2nd Ed. MIT Press, 2018.
Wachel P, Sliwinski, P. Aggregative modeling of nonlinear systems. IEEE Signal Processing Letters, 22(9):1482-1486, 2015. https://doi.org/10.1109/LSP.2015.2405613
Wachel P. Wiener system modelling by exponentially weighted aggregation. International Journal of Control, 90(11):2480-2489, 2017. https://doi.org/10.1080/00207179.2016.1254818
Willems JC, Rapisarda P, Markovsky I, De Moor BLM. A note on persistency of excitation. Syst. & Contr. Lett., 54(4):325-329, 2005. https://doi.org/10.1016/j.sysconle.2004.09.003
Zhu YC. System identification for process control: recent experiment and outlook. In IFAC Symposium on System Identification, pages 89-103, Newcastle, Australia, 2006. Plenary presentation, available at http://taijicontrol.com/SYSID06YZhu.pdf.
Zinkevich M. Online convex programming and generalized infinitesimal gradient ascent. In Proc. Int. Conf. Mach. Learn. (ICML), pages 928-936, 2003.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright (c) 2022 Pawel Wachel, Cristian R. Rojas