Reinforcement Learning Based Controller Synthesis for Flexible Aircraft Wings

Manoj Kumar; Karthikeyan Rajagopal; Sivasubramanya Nadar Balakrishnan; Nhan T. Nguyen

Volume 1 Issue 4

Oct. 2014

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2014 > 1(4): 435-448

Manoj Kumar, Karthikeyan Rajagopal, Sivasubramanya Nadar Balakrishnan and Nhan T. Nguyen, "Reinforcement Learning Based Controller Synthesis for Flexible Aircraft Wings," IEEE/CAA J. of Autom. Sinica, vol. 1, no. 4, pp. 435-448, 2014.

Citation:

Manoj Kumar, Karthikeyan Rajagopal, Sivasubramanya Nadar Balakrishnan and Nhan T. Nguyen, "Reinforcement Learning Based Controller Synthesis for Flexible Aircraft Wings," IEEE/CAA J. of Autom. Sinica, vol. 1, no. 4, pp. 435-448, 2014.

Citation:

Manoj Kumar, Karthikeyan Rajagopal, Sivasubramanya Nadar Balakrishnan and Nhan T. Nguyen, "Reinforcement Learning Based Controller Synthesis for Flexible Aircraft Wings," IEEE/CAA J. of Autom. Sinica, vol. 1, no. 4, pp. 435-448, 2014.

PDF( 6080 KB)

Reinforcement Learning Based Controller Synthesis for Flexible Aircraft Wings

1. Missouri University of Science & Technology, Rolla, Missouri 65409, USA;
2. NASA Ames Research Center, Moffet Field, CA 94035, USA

Funds:

This work was supported by National Science Foundation of USA (1002333).

Abstract

Abstract

Aeroelastic study of flight vehicles has been a subject of great interest and research in the last several years. Aileron reversal and flutter related problems are due in part to the elasticity of a typical airplane. Structural dynamics of an aircraft wing due to its aeroelastic nature are characterized by partial differential equations. Controller design for these systems is very complex as compared to lumped parameter systems defined by ordinary differential equations. In this paper, a stabilizing statefeedback controller design approach is presented for the heave dynamics of a wing-fuselage model. In this study, a continuous actuator in the spatial domain is assumed. A control methodology is developed by combining the technique of "proper orthogonal decomposition" and approximate dynamic programming. The proper orthogonal decomposition technique is used to obtain a low-order nonlinear lumped parameter model of the infinite dimensional system. Then a near optimal controller is designed using the single-network-adaptive-critic technique. Furthermore, to add robustness to the nominal single-network-adaptive-critic controller against matched uncertainties, an identifier based adaptive controller is proposed. Simulation results demonstrate the effectiveness of the single-network-adaptive-critic controller augmented with adaptive controller for infinite dimensional systems.
- Single-network-adaptive-critic,
- proper orthogonal decomposition,
- partial differential equation,
- adaptive critic,
- uncertainty,
- adaptive control,
- flexible wing

FullText(HTML)

References(32)

References

[1]	Lasiecka I. Control of systems governed by partial differential equations:a historical perspective. In: Proceedings of the 34th Conference onDecision and Control. New Orleans, LA: IEEE, 1995. 2792-2796
[2]	Burns J A, King B B. Optimal sensor location for robust control ofdistributed parameter systems. In: Proceedings of the 1994 Conferenceon Decision and Control. Lake Buena Vista, FL, USA: IEEE, 1994.3967-3972
[3]	Curtain R F, Zwart H J. An introduction to Infinite Dimensional LinearSystems Theory. New York: Springer-Verlag, 1995.
[4]	Padhi R, Balakrishnan S N. Optimal dynamic inversion control designfor a class of nonlinear distributed parameter systems with continuousand discrete actuators. The Institute of Engineering and Technology,Control Theory, and Applications, 2007, 1(6): 1662-2671
[5]	Haykin S. Neural Networks. New York: Macmillan College Company,1994.
[6]	Gupta S K. Numerical Methods for Engineers. New Delhi: New AgeInternational Ltd., 1995.
[7]	Christofides P D. Nonlinear and Robust Control of PDE Systems:Methods and Applications to Transport-Reaction Processes. Boston:Birkhauser Boston Inc., 2001.
[8]	Holmes P, Lumley J L, Berkooz G. Turbulence Coherent Structures,Dynamical Systems and Symmetry. Cambridge: Cambridge UniversityPress, 1996. 87-154
[9]	Bryson A E, Ho Y C. Applied Optimal Control: Optimization, Estimation,and Control. Washington: Taylor and Francis, 1975
[10]	Lewis F. Applied Optimal Control and Estimation. New Jersey: Prentice-Hall, 1992.
[11]	Werbos P J. Neurocontrol and supervised learning: an overview andevaluation. Handbook of Intelligent Control: Neural, Fuzzy and AdaptiveApproaches. New York: Van Nostrand Reinhold, 1992
[12]	Balakrishnan S N, Biega V. Adaptive-critic based neural networks foraircraft optimal control. Journal of Guidance, Control and Dynamics,1996, 19(4): 893-898
[13]	Padhi R, Unnikrishnan N, Wang X, Balakrishnan S N. A single networkadaptive critic (SNAC) architecture for optimal control synthesis for aclass of nonlinear systems. Neural Networks, 2006, 19: 1648-1660
[14]	Padhi R, Balakrishnan S N. Optimal beaver population managementusing reduced-order distributed parameter model and single networkadaptive critics. In: Proceedings of the 2004 American Control Conference.Boston, MA, 2004. 1598-1603
[15]	Yadav V, Padhi R, Balakrishnan S N. Robust/optimal temperature profilecontrol of a re-entry vehicle using neural networks. In: Proceedings ofthe 2006 AIAA Atmospheric Flight Mechanics Conference and Exhibit.Keystone, Colorado: AIAA, 2006. 21-24
[16]	Prabhat P, Balakrishnan S N, Look D C Jr. Experimental implementationof adaptive-critic based infinite time optimal neurocontrol for aheat diffusion system. In: Proceedings of the 2002 American ControlConference. Alaska, USA: IEEE, 2002. 2671-2676
[17]	Chakravarthy A, Evans K A, Evers J. Sensitivity & functional gains fora flexible aircraft-inspired model. In: Proceedings of the 2011 AmericanControl Conference. Baltimore, MD, USA: IEEE, 2010. 4893-4898
[18]	Chakravarthy A, Evans K A, Evers J, Kuhn L M. Target tracking strategiesfor a nonlinear, flexible aircraft-inspired model. In: Proceedings ofthe 2011 American Control Conference. San Francisco: IEEE, 2011.1783-1788
[19]	Narendra K S, Annaswamy A M. Stable Adaptive Systems. EnglewoodCliffs, NJ: Prentice-Hall, 1989.
[20]	Ioannou P A, Sun J. Robust Adaptive Control. Englewood Cliffs, NJ:Prentice-Hall, 1995.
[21]	Böhm M, Demetriou M A, Reich S, Rosen I G. Model reference adaptivecontrol of distributed parameter systems. SIAM Journal of Control andOptimization, 1998, 36(1): 33-81
[22]	Hong K S, Bentsman J. Direct adaptive control of parabolic systems:algorithm synthesis and convergence and stability analysis. IEEE Transactionsof Automatic Control, 1994, 39(10): 2018-2033
[23]	Hong K S, Yang K J, Kang D H. Model reference adaptive control of atime-varying parabolic system.KSME International Journal, 2000, 14(2):168-176
[24]	Krstic M, Smyshlyaev A. Adaptive control of PDEs. Annual Reviews inControl, 2008, 32(2): 149-160
[25]	Demetriou A M, Rosen I G. On-line robust parameter identification forparabolic systems. International Journal of Adaptive Control and SignalProcessing, 15(6): 615-631
[26]	Sirovich L. Turbulence and the dynamics of coherent structures. Quarterlyof Applied Mathematics, 1987, 45(3): 561-590
[27]	Ravindran S S. Proper Orthogonal Decomposition in Optimal Controlof Fluids, NASA/TM-1999-209113. USA, 1999.
[28]	Padhi R, Balakrishnan S N. Proper orthogonal decomposition basedoptimal control design of heat equation with discrete actuators usingneural networks. American Control Conference, ACC02-IEEE 1545,2002.
[29]	Padhi R, Balakrishnan S N. Proper orthogonal decomposition basedfeedback optimal control synthesis of distributed parameter systemsusing neural networks. In: Proceedings of the 15th American ControlConference. Anchorage, AK: IEEE, 2002. 4389-4394
[30]	Ravindran S S. Adaptive reduced-order controllers for a thermal flowsystem using proper orthogonal decomposition. SIAM Journal on ScientificComputing, 2002, 23(6): 1924-1942
[31]	Popov V M. Hyperstability of Control Systems. Berlin: Springer, 1973.
[32]	Olver P J. Introduction to Partial Differential Equations. New York:Springer-Verlag, 2013.

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Get Citation

PDF

XML

Article Metrics

Article views (1213) PDF downloads(16)

Reinforcement Learning Based Controller Synthesis for Flexible Aircraft Wings

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Export File

Citation

Format

Content