Evolutionary Algorithm Based on Surrogate and Inverse Surrogate Models for Expensive Multiobjective Optimization

Qi Deng; Qi Kang; MengChu Zhou; Xiaoling Wang; Shibing Zhao; Siqi Wu; Mohammadhossein Ghahramani

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2025 > In Press, Accepted Manuscript

Q. Deng, Q. Kang, M. C. Zhou, X. Wang, S. Zhao, S. Wu, and M. Ghahramani, “Evolutionary algorithm based on surrogate and inverse surrogate models for expensive multiobjective optimization,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 5, pp. 1–13, May 2025.

Citation:

Q. Deng, Q. Kang, M. C. Zhou, X. Wang, S. Zhao, S. Wu, and M. Ghahramani, “Evolutionary algorithm based on surrogate and inverse surrogate models for expensive multiobjective optimization,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 5, pp. 1–13, May 2025.

Citation:

Q. Deng, Q. Kang, M. C. Zhou, X. Wang, S. Zhao, S. Wu, and M. Ghahramani, “Evolutionary algorithm based on surrogate and inverse surrogate models for expensive multiobjective optimization,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 5, pp. 1–13, May 2025.

PDF( 1689 KB)

Evolutionary Algorithm Based on Surrogate and Inverse Surrogate Models for Expensive Multiobjective Optimization

Funds: This work was supported in part by the National Natural Science Foundation of China (51775385), the Natural Science Foundation of Shanghai (23ZR1466000), the Shanghai Industrial Collaborative Science and Technology Innovation Project (2021-cyxt2-kj10), the Innovation Program of Shanghai Municipal Education Commission (202101070007E00098), and Fundo para o Desenvolvimento das Ciencias e da Tecnologia (FDCT) (0147/2024/AFJ)

More Information

Author Bio:
Qi Deng received the B.S. degree in automation from Tongji University in 2020. He is currently pursuing the Ph.D. degree in control science and engineering at Tongji University. He is presently with New Jersey Institute of Technology, USA as a visiting doctoral student. His current research interest includes evolutionary computation and multi-objective optimization

Qi Kang (Senior Member, IEEE) received the B.S degree in automatic control and the M.S. and Ph.D degrees in control theory and control engineering from Tongji University in 2002, 2005 and 2009, respectively. From 2007 to 2008, he was a Research Associate with the University of Illinois at Chicago, USA. From 2014 to 2015, he was a Visiting Scholar with the New Jersey Institute of Technology, Newark, USA. He is currently a Professor with the Department of Control Science and Engineering and the Shanghai Institute of Intelligent Science and Technology, Tongji University. His research interests include swarm intelligence, evolutionary computation, machine learning, and intelligent control and optimization.He is currently served as the General Chair of the 19th IEEE International Conference on Networking, Sensing and Control (ICNSC 2022), and an Associate Editor for IEEE Transactions on Intelligent Transportation Systems. He is the Secretary General with Shanghai Association for System Simulation

MengChu Zhou (Fellow, IEEE) received the B.S. degree from Nanjing University of Science and Technology in 1983, the M.S. degree from Beijing Institute of Technology in 1986, and the Ph.D. degree from Rensselaer Polytechnic Institute, USA in 1990. He then joined New Jersey Institute of Technology, USA and is now a Distinguished Professor in Electrical and Computer Engineering. His research interests are intelligent automation/transportation, robotics, Petri nets, Internet of Things, edge/cloud computing, and big data analytics. He has over 1300 publications including 17 books, over 900 journal papers including over 700 IEEE Transactions papers, 31 patents and 32 book-chapters. His recently co-authored books include Learning Automata and their Applications to Intelligent Systems, IEEE Press/Wiley, Hoboken, NJ, 2024 (with J. Zhang), Device-Edge-Cloud Continuum Paradigms, Architectures and Applications, Springer Nature, 2023 (with C. Savaglio, G. Fortino, and J. Ma) and Sustainable Manufacturing Systems: An Energy Perspective, IEEE Press/Wiley, Hoboken, Hoboken, NJ, 2022 (with L. Li). He is a recipient of Excellence in Research Prize and Medal from NJIT, Humboldt Research Award for US Senior Scientists from Alexander von Humboldt Foundation, and Franklin V. Taylor Memorial Award and the Norbert Wiener Award from IEEE Systems, Man, and Cybernetics Society, and Edison Patent Award from the Research & Development Council of New Jersey. He is a life member of Chinese Association for Science and Technology-USA and served as its President in 1999. He is Fellow of IEEE, International Federation of Automatic Control (IFAC), American Association for the Advancement of Science (AAAS), Chinese Association of Automation (CAA) and National Academy of Inventors (NAI)

Xiaoling Wang received the M.S. degree from Central South University in 2018. She is currently pursuing the Ph.D. degree in control science and engineering, Tongji University. Her research interests include evolutionary computation, multi-task optimization, and multi-objective optimization

Shibing Zhao received the B.S. degree in automation from Tongji University in 2022. He is currently pursuing the M.S. degree in control science and engineering, Tongji University. His research interests include evolution computation and multi-objective optimization

Siqi Wu received the B.S. degree from Tongji University in 2023. She is currently pursuing the M.S. degree in control science and engineering, Tongji University. Her research interests include evolution computation

Mohammadhossein Ghahramani (Member, IEEE) received the B.S. and M.S. degrees in information technology engineering from Amirkabir University of Technology-Tehran Polytechnic. He received the Ph.D. degree in computer technology and application from Macau University of Science and Technology, Macao, China in 2018. He was a Member of the Insight Centre for Data Analytics and a Research Fellow at University College Dublin, Ireland. Currently, he is an Assistant Professor of Data Science at Birmingham City University, UK. His research interests include smart systems, artificial intelligence, optimization, smart cities, and IoT. Dr Ghahramani has published over ten peer-reviewed journal papers as the first author in reputable journals and has received several awards, including the Best Automation Paper in Technology by the IEEE Robotics & Automation Society. He serves as a co-chair of the IEEE SMCS Technical Committee on AI-based Smart Manufacturing Systems and as an Associate Editor of IEEE Internet of Things Journal
Corresponding author: Qi Kang, e-mail: qkang@tongji.edu.cn; MengChu Zhou, e-mail: zhou@njit.edu
Received Date: 2024-09-16
Revised Date: 2024-12-05
Accepted Date: 2024-12-19

Available Online: 2025-03-28

Abstract

Abstract

When dealing with expensive multiobjective optimization problems, majority of existing surrogate-assisted evolutionary algorithms (SAEAs) generate solutions in decision space and screen candidate solutions mostly by using designed surrogate models. The generated solutions exhibit excessive randomness, which tends to reduce the likelihood of generating good-quality solutions and cause a long evolution to the optima. To improve SAEAs greatly, this work proposes an evolutionary algorithm based on surrogate and inverse surrogate models by 1) Employing a surrogate model in lieu of expensive (true) function evaluations; and 2) Proposing and using an inverse surrogate model to generate new solutions. By using the same training data but with its inputs and outputs being reversed, the latter is simple to train. It is then used to generate new vectors in objective space, which are mapped into decision space to obtain their corresponding solutions. Using a particular example, this work shows its advantages over existing SAEAs. The results of comparing it with state-of-the-art algorithms on expensive optimization problems show that it is highly competitive in both solution performance and efficiency.
- Expensives multi-objective optimization,
- reverse model,
- surrogate-assisted evolutionary algorithms (SAEAs)

FullText(HTML)

References(49)

References

[1]	F. Li, L. Gao, W. Shen, X. Cai, and S. Huang, “A surrogate-assisted offspring generation method for expensive multi-objective optimization problems,” in Proc. IEEE Congr. Evolutionary Computation, Glasgow, UK, 2020, pp. 1–8.
[2]	B. Li, Y. Lu, H. Qian, W. Hong, P. Yang, and A. Zhou, “Regularity model based offspring generation in surrogate-assisted evolutionary algorithms for expensive multi-objective optimization,” Swarm Evol. Comput., vol. 86, p. 101506, Apr. 2024. doi: 10.1016/j.swevo.2024.101506
[3]	X. Wang, Q. Kang, M. Zhou, S. Yao, and A. Abusorrah, “Domain adaptation multitask optimization,” IEEE Trans. Cybern., vol. 53, no. 7, pp. 4567–4578, Jul. 2023. doi: 10.1109/TCYB.2022.3222101
[4]	X. Wang, Q. Kang, M. Zhou, Q. Deng, Z. Fan, and H. Liu, “Knowledge classification-assisted evolutionary multitasking for two-task multiobjective optimization problems,” IEEE/CAA J. Autom. Sinica, 2024, doi: 10.1109/JAS.2024.125070.
[5]	M. C. Zhou, M. Cui, D. Xu, S. Zhu, Z. Zhao, and A. Abusorrah, “Evolutionary optimization methods for high-dimensional expensive problems: A survey,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 5, pp. 1092–1105, May 2024. doi: 10.1109/JAS.2024.124320
[6]	L. Zhang, Q. Kang, Q. Deng, L. Xu, and Q. Wu, “A line complex-based evolutionary algorithm for many-objective optimization,” IEEE/CAA J. Autom. Sinica, vol. 10, no. 5, pp. 1150–1167, May 2023. doi: 10.1109/JAS.2023.123495
[7]	A. Kenny, T. Ray, and H. K. Singh, “An iterative two-stage multi-fidelity optimization algorithm for computationally expensive problems,” IEEE Trans. Evol. Comput., vol. 27, no. 3, pp. 520–534, Jun. 2023. doi: 10.1109/TEVC.2022.3170970
[8]	J. T. Wilson, F. Hutter, and M. P. Deisenroth, “Maximizing acquisition functions for Bayesian optimization,” in Proc. 32nd Int. Conf. Neural Information Processing Systems, Montreal, Canada, 2018, pp. 9906–9917.
[9]	T. Chugh, Y. Jin, K. Miettinen, J. Hakanen, and K. Sindhya, “A surrogate-assisted reference vector guided evolutionary algorithm for computationally expensive many-objective optimization,” IEEE Trans. Evol. Comput., vol. 22, no. 1, pp. 129–142, Feb. 2018. doi: 10.1109/TEVC.2016.2622301
[10]	Z. Song, H. Wang, C. He, and Y. Jin, “A Kriging-assisted two-archive evolutionary algorithm for expensive many-objective optimization,” IEEE Trans. Evol. Comput., vol. 25, no. 6, pp. 1013–1027, Dec. 2021. doi: 10.1109/TEVC.2021.3073648
[11]	Y. Jin, H. Wang, T. Chugh, D. Guo, and K. Miettinen, “Data-driven evolutionary optimization: An overview and case studies,” IEEE Trans. Evol. Comput., vol. 23, no. 3, pp. 442–458, Jun. 2019. doi: 10.1109/TEVC.2018.2869001
[12]	J. Tian, Y. Tan, J. Zeng, C. Sun, and Y. Jin, “Multiobjective infill criterion driven Gaussian process-assisted particle swarm optimization of high-dimensional expensive problems,” IEEE Trans. Evol. Comput., vol. 23, no. 3, pp. 459–472, Jun. 2019. doi: 10.1109/TEVC.2018.2869247
[13]	D. Zhan, Y. Cheng, and J. Liu, “Expected improvement matrix-based infill criteria for expensive multiobjective optimization,” IEEE Trans. Evol. Comput., vol. 21, no. 6, pp. 956–975, Dec. 2017. doi: 10.1109/TEVC.2017.2697503
[14]	R. G. Regis, “Evolutionary programming for high-dimensional constrained expensive black-box optimization using radial basis functions,” IEEE Trans. Evol. Comput., vol. 18, no. 3, pp. 326–347, Jun. 2014. doi: 10.1109/TEVC.2013.2262111
[15]	B. Liu, Q. Zhang, and G. G. E. Gielen, “A Gaussian Process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems,” IEEE Trans. Evol. Comput., vol. 18, no. 2, pp. 180–192, Apr. 2014. doi: 10.1109/TEVC.2013.2248012
[16]	H. Wang and Y. Jin, “A random forest-assisted evolutionary algorithm for data-driven constrained multiobjective combinatorial optimization of trauma systems,” IEEE Trans. Cybern., vol. 50, no. 2, pp. 536–549, Feb. 2020. doi: 10.1109/TCYB.2018.2869674
[17]	D. Guo, X. Wang, K. Gao, Y. Jin, J. Ding, and T. Chai, “Evolutionary optimization of high-dimensional multiobjective and many-objective expensive problems assisted by a dropout neural network,” IEEE Trans. Syst., Man, Cybern.: Syst., vol. 52, no. 4, pp. 2084–2097, Apr. 2022. doi: 10.1109/TSMC.2020.3044418
[18]	Z. Wang, Q. Zhang, Y. S. Ong, S. Yao, H. Liu, and J. Luo, “Choose appropriate subproblems for collaborative modeling in expensive multiobjective optimization,” IEEE Trans. Cybern., vol. 53, no. 1, pp. 483–496, Jan. 2023. doi: 10.1109/TCYB.2021.3126341
[19]	C. Sun, Y. Jin, R. Cheng, J. Ding, and J. Zeng, “Surrogate-assisted cooperative swarm optimization of high-dimensional expensive problems,” IEEE Trans. Evol. Comput., vol. 21, no. 4, pp. 644–660, Aug. 2017. doi: 10.1109/TEVC.2017.2675628
[20]	J. Knowles, “ParEGO: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems,” IEEE Trans. Evol. Comput., vol. 10, no. 1, pp. 50–66, Feb. 2006. doi: 10.1109/TEVC.2005.851274
[21]	N. Namura, K. Shimoyama, and S. Obayashi, “Expected improvement of penalty-based boundary intersection for expensive multiobjective optimization,” IEEE Trans. Evol. Comput., vol. 21, no. 6, pp. 898–913, Dec. 2017. doi: 10.1109/TEVC.2017.2693320
[22]	Q. Zhang, W. Liu, E. Tsang, and B. Virginas, “Expensive multiobjective optimization by MOEA/D with Gaussian process model,” IEEE Trans. Evol. Comput., vol. 14, no. 3, pp. 456–474, Jun. 2010. doi: 10.1109/TEVC.2009.2033671
[23]	Y. Sun, H. Wang, B. Xue, Y. Jin, G. G. Yen, and M. Zhang, “Surrogate-assisted evolutionary deep learning using an end-to-end random forest-based performance predictor,” IEEE Trans. Evol. Comput., vol. 24, no. 2, pp. 350–364, Apr. 2020. doi: 10.1109/TEVC.2019.2924461
[24]	S. D. Handoko, C. K. Kwoh, and Y. S. Ong, “Feasibility structure modeling: An effective chaperone for constrained memetic algorithms,” IEEE Trans. Evol. Comput., vol. 14, no. 5, pp. 740–758, Oct. 2010. doi: 10.1109/TEVC.2009.2039141
[25]	L. Pan, C. He, Y. Tian, H. Wang, X. Zhang, and Y. Jin, “A classification-based surrogate-assisted evolutionary algorithm for expensive many-objective optimization,” IEEE Trans. Evol. Comput., vol. 23, no. 1, pp. 74–88, Feb. 2019. doi: 10.1109/TEVC.2018.2802784
[26]	C. He, S. Huang, R. Cheng, K. C. Tan, and Y. Jin, “Evolutionary multiobjective optimization driven by generative adversarial networks (GANs),” IEEE Trans. Cybern., vol. 51, no. 6, pp. 3129–3142, Jun. 2021. doi: 10.1109/TCYB.2020.2985081
[27]	T. Sonoda and M. Nakata, “Multiple classifiers-assisted evolutionary algorithm based on decomposition for high-dimensional multiobjective problems,” IEEE Trans. Evol. Comput., vol. 26, no. 6, pp. 1581–1595, Dec. 2022. doi: 10.1109/TEVC.2022.3159000
[28]	H. Wang, L. Jiao, and X. Yao, “Two_Arch2: An improved two-archive algorithm for many-objective optimization,” IEEE Trans. Evol. Comput., vol. 19, no. 4, pp. 524–541, Aug. 2015. doi: 10.1109/TEVC.2014.2350987
[29]	I. Giagkiozis and P. J. Fleming, “Pareto front estimation for decision making,” Evol. Comput., vol. 22, no. 4, pp. 651–678, Dec. 2014. doi: 10.1162/EVCO_a_00128
[30]	Q. Deng, Q. Kang, L. Zhang, M. Zhou, and J. An, “Objective space-based population generation to accelerate evolutionary algorithms for large-scale many-objective optimization,” IEEE Trans. Evol. Comput., vol. 27, no. 2, pp. 326–340, Apr. 2023. doi: 10.1109/TEVC.2022.3166815
[31]	Q. Deng, Q. Kang, M. Zhou, X. Wang, and A. Albeshri, “Activation function-assisted objective space mapping to enhance evolutionary algorithms for large-scale many-objective optimization,” IEEE Trans. Syst., Man, Cybern.: Syst., vol. 55, no. 1, pp. 183–195, Jan. 2025. doi: 10.1109/TSMC.2024.3454051
[32]	R. Cheng, Y. Jin, K. Narukawa, and B. Sendhoff, “A multiobjective evolutionary algorithm using Gaussian process-based inverse modeling,” IEEE Trans. Evol. Comput., vol. 19, no. 6, pp. 838–856, Dec. 2015. doi: 10.1109/TEVC.2015.2395073
[33]	H. Hao, A. Zhou, H. Qian, and H. Zhang, “Expensive multiobjective optimization by relation learning and prediction,” IEEE Trans. Evol. Comput., vol. 26, no. 5, pp. 1157–1170, Oct. 2022. doi: 10.1109/TEVC.2022.3152582
[34]	K. Deb, L. Thiele, M. Laumanns, and E. Zitzler, “Scalable test problems for evolutionary multiobjective optimization,” Evolutionary Multiobjective Optimization, A. Abraham, L. Jain, and R. Goldberg, Eds. London, U K: Springer, 2005, pp. 105–145.
[35]	D. R. Jones, “A taxonomy of global optimization methods based on response surfaces,” J. Global Optim., vol. 21, no. 4, pp. 345–383, Dec. 2001. doi: 10.1023/A:1012771025575
[36]	M. T. M. Emmerich, K. C. Giannakoglou, and B. Naujoks, “Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels,” IEEE Trans. Evol. Comput., vol. 10, no. 4, pp. 421–439, Aug. 2006. doi: 10.1109/TEVC.2005.859463
[37]	H. B. Nielsen, S. N. Lophaven, and J. Søndergaard, DACE—A MATLAB Kriging Toolbox, Technical University of Denmark, Copenhagen, Denmark, 2002.
[38]	E. Zitzler, L. Thiele, M. Laumanns, C. M. Fonseca, and V. G. Da Fonseca, “Performance assessment of multiobjective optimizers: An analysis and review,” IEEE Trans. Evol. Comput., vol. 7, no. 2, pp. 117–132, Apr. 2003. doi: 10.1109/TEVC.2003.810758
[39]	L. While, P. Hingston, L. Barone, and S. Huband, “A faster algorithm for calculating hypervolume,” IEEE Trans. Evol. Comput., vol. 10, no. 1, pp. 29–38, Feb. 2006. doi: 10.1109/TEVC.2005.851275
[40]	Y. Tian, R. Cheng, X. Zhang, and Y. Jin, “PlatEMO: A MATLAB platform for evolutionary multi-objective optimization [Educational Forum],” IEEE Comput. Intell. Mag., vol. 12, no. 4, pp. 73–87, Nov. 2017. doi: 10.1109/MCI.2017.2742868
[41]	J. Derrac, S. García, D. Molina, and F. Herrera, “A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms,” Swarm Evol. Comput., vol. 1, no. 1, pp. 3–18, Mar. 2011. doi: 10.1016/j.swevo.2011.02.002
[42]	R. Cheng, M. Li, Y. Tian, X. Zhang, S. Yang, Y. Jin, and X. Yao, “A benchmark test suite for evolutionary many-objective optimization,” Complex Intell. Syst., vol. 3, no. 1, pp. 67–81, Mar. 2017. doi: 10.1007/s40747-017-0039-7
[43]	H. Jain and K. Deb, “An evolutionary many-objective optimization algorithm using reference-point based nondominated sorting approach, part II: Handling constraints and extending to an adaptive approach,” IEEE Trans. Evol. Comput., vol. 18, no. 4, pp. 602–622, Aug. 2014. doi: 10.1109/TEVC.2013.2281534
[44]	Y. Zhao, C. Sun, J. Zeng, Y. Tan, and G. Zhang, “A surrogate-ensemble assisted expensive many-objective optimization,” Knowl.-Based Syst., vol. 211, p. 106520, Jan. 2021. doi: 10.1016/j.knosys.2020.106520
[45]	J. Li, M. Zhou, Q. Sun, X. Dai, and X. Yu, “Colored traveling salesman problem,” IEEE Trans. Cybern., vol. 45, p. 11, Nov. 2015.
[46]	Y. Feng, M. Zhou, G. Tian, Z. Li, Z. Zhang, Q. Zhang, and J. Tan, “Target disassembly sequencing and scheme evaluation for CNC machine tools using improved multiobjective ant colony algorithm and fuzzy integral,” IEEE Trans. Syst., Man, Cybern.: Syst., vol. 49, no. 12, pp. 2438–2451, Dec. 2019. doi: 10.1109/TSMC.2018.2847448
[47]	L. Huang, M. Zhou, H. Han, S. Wang, and A. Albeshri, “Learning-inspired immune algorithm for multiobjective-optimized multirobot maritime patrolling,” IEEE Internet Things J., vol. 11, no. 6, pp. 9870–9881, Mar. 2024. doi: 10.1109/JIOT.2023.3326567
[48]	H. Zhu, M. Zhou, Y. Xie, and A. Albeshri, “A self-adapting and efficient dandelion algorithm and its application to feature selection for credit card fraud detection,” IEEE/CAA J. Autom. Sinica, vol. 11, no. 2, pp. 377–390, Feb. 2024. doi: 10.1109/JAS.2023.124008
[49]	Z. Qiu, S. Wang, D. You and M. Zhou, “Bridge bidding via deep reinforcement learning and belief Monte Carlo search,” IEEE/CAA Journal of Automatica Sinica, vol. 11, no. 10, pp. 2111−2122, Oct. 2024.

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(9) / Tables(5)

Get Citation

PDF

XML

Article Metrics

Article views (115) PDF downloads(40)

Evolutionary Algorithm Based on Surrogate and Inverse Surrogate Models for Expensive Multiobjective Optimization

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Export File

Citation

Format

Content