Surrogate-Assisted Particle Swarm Optimization Algorithm With Pareto Active Learning for Expensive Multi-Objective Optimization

Zhiming Lv; Linqing Wang; Zhongyang Han; Jun Zhao; Wei Wang

doi:10.1109/JAS.2019.1911450

Volume 6 Issue 3

May 2019

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2019 > 6(3): 838-849

Zhiming Lv, Linqing Wang, Zhongyang Han, Jun Zhao and Wei Wang, "Surrogate-Assisted Particle Swarm Optimization Algorithm With Pareto Active Learning for Expensive Multi-Objective Optimization," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 838-849, May 2019. doi: 10.1109/JAS.2019.1911450

Citation:

Zhiming Lv, Linqing Wang, Zhongyang Han, Jun Zhao and Wei Wang, "Surrogate-Assisted Particle Swarm Optimization Algorithm With Pareto Active Learning for Expensive Multi-Objective Optimization," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 838-849, May 2019. doi: 10.1109/JAS.2019.1911450

Citation:

Zhiming Lv, Linqing Wang, Zhongyang Han, Jun Zhao and Wei Wang, "Surrogate-Assisted Particle Swarm Optimization Algorithm With Pareto Active Learning for Expensive Multi-Objective Optimization," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 838-849, May 2019. doi: 10.1109/JAS.2019.1911450

PDF( 1014 KB)

Surrogate-Assisted Particle Swarm Optimization Algorithm With Pareto Active Learning for Expensive Multi-Objective Optimization

doi: 10.1109/JAS.2019.1911450

Funds: This work was supported by the National Natural Sciences Foundation of China (61603069, 61533005, 61522304, U1560102) and the National Key Research and Development Program of China (2017YFA0700300)

More Information

Author Bio:
Zhiming Lv graduated from Shenyang University of Chemical Technology, China, in 2006. He received the M.S. degree from Shenyang University of Chemical Technology in 2010. He is currently working toward Ph.D. degree at the School of Control Science and Engineering, Dalian University of Technology, Dalian, China. His research interests include industrial process modeling, optimization scheduling, and intelligent optimization algorithm

Linqing Wang graduated from Northeastern University (NEU), China, in 2007. She received the M.S. degree and Ph.D. degree from NEU in 2010 and 2014, respectively. She is currently a Senior Lecturer at the School of Control Sciences and Engineering, Dalian University of Technology. Her research interests include the prediction algorithm, industrial optimization, and intelligent algorithm

Zhongyang Han graduated from City Institute, Dalian University of Technology, China, in 2010. He received the Ph.D. degree from Dalian University of Technology, China, in 2016. He is currently engaging in the postdoctoral research with the School of Control Science and Engineering, Dalian University of Technology, Dalian, China. His research interests include computer integrated manufacturing systems, artificial intelligence, data mining, and machine learning

Jun Zhao (M’10) graduated from Dalian Jiaotong University, China, in 2003. He received the Ph.D. degree in engineering from Dalian University of Technology, in 2008. He is currently a Professor with the School of Control Science and Engineering, Dalian University of Technology. His current research interests include industrial production scheduling, computer integrated manufacturing, intelligent optimization, and machine learning

Wei Wang (SM’13) received the B.S., M.S., and Ph.D. degrees from Northeastern University, Shenyang, China, in 1982, 1986, and 1988, respectively, all in industrial automation. He was a postdoctoral fellow with the Division of Engineering Cybernetics, Norwegian Science and Technology University, Trondheim, Norway, from 1990 to 1992, a Professor and Vice Director of the National Engineering Research Center of Metallurgical Automation of China, China, from 1995 to 1999, and a Research Fellow with the Department of Engineering Science, University of Oxford, Oxford, UK, from 1998 to 1999. He is currently a Professor with the School of Control Sciences and Engineering, Dalian University of Technology, China. His current research interests include adaptive controls, computer integrated manufacturing, and computer controls of industrial processes
Corresponding author: All the authors are with the Faculty of Electronics Information and Electrical Engineering, Dalian University of Technology, Dalian 116024, China (e-mail: lv_zhiming@126.com; wanglinqing@dlut.edu.cn; hzydut@mail.dlut.edu.cn; zhaoj@dlut.edu.cn; wangwei@dlut.edu.cn)
Received Date: 2018-08-24
Revised Date: 2018-12-12
Accepted Date: 2019-01-07

Available Online: 2019-04-24

Abstract

Abstract

For multi-objective optimization problems, particle swarm optimization (PSO) algorithm generally needs a large number of fitness evaluations to obtain the Pareto optimal solutions. However, it will become substantially time-consuming when handling computationally expensive fitness functions. In order to save the computational cost, a surrogate-assisted PSO with Pareto active learning is proposed. In real physical space (the objective functions are computationally expensive), PSO is used as an optimizer, and its optimization results are used to construct the surrogate models. In virtual space, objective functions are replaced by the cheaper surrogate models, PSO is viewed as a sampler to produce the candidate solutions. To enhance the quality of candidate solutions, a hybrid mutation sampling method based on the simulated evolution is proposed, which combines the advantage of fast convergence of PSO and implements mutation to increase diversity. Furthermore, $ \varepsilon $-Pareto active learning ($ \varepsilon $-PAL) method is employed to pre-select candidate solutions to guide PSO in the real physical space. However, little work has considered the method of determining parameter $ \varepsilon $. Therefore, a greedy search method is presented to determine the value of $ \varepsilon $ where the number of active sampling is employed as the evaluation criteria of classification cost. Experimental studies involving application on a number of benchmark test problems and parameter determination for multi-input multi-output least squares support vector machines (MLSSVM) are given, in which the results demonstrate promising performance of the proposed algorithm compared with other representative multi-objective particle swarm optimization (MOPSO) algorithms.
- Multiobjective optimization,
- Pareto active learning,
- particle swarm optimization (PSO),
- surrogate

FullText(HTML)

References(37)

References

[1]	G. R. Zavala, A. J. Nebro, F. Luna, and C. A. C. Coello, " A survey of multi-objective metaheuristics applied to structural optimization,” Struct. Multidiscipl. Optim., vol. 49, no. 4, pp. 537–558, Apr. 2014. doi: 10.1007/s00158-013-0996-4
[2]	M. Park, M. Nassar, and H. Vikalo, " Bayesian active learning for drug combinations,” IEEE Trans. Biomed. Eng., vol. 60, no. 11, pp. 3248–3255, Nov. 2013. doi: 10.1109/TBME.2013.2272322
[3]	D. Ronco, C. Comis, R. Ponza, and E. Benini, " Aerodynamic shape optimization of aircraft components using an advanced multi-objective evolutionary approach,” Comput. Methods Appl. Mech. Eng., vol. 285, pp. 255–290, Mar. 2015. doi: 10.1016/j.cma.2014.10.024
[4]	J. Snoek, I. Larochelle, and R. P. Adans., " Practical Bayesian optimization of machine learning algorithms,” in Proc. 25th Int. Conf. NIPS, Lake Tahoe, Nevada, 2012, pp. 2951–2959.
[5]	H. D. Wang, Y. C. Jin, and J. O. Jansen, " Data-driven surrogate-assisted multiobjective evolutionary optimization of a trauma system,” IEEE Trans. E Comput., vol. 20, no. 6, pp. 939–952, Dec. 2016. doi: 10.1109/TEVC.2016.2555315
[6]	Q. L. Zhu, Q. Z. Lin, W. N. Chen, K. C. Wong, C. A. Coello Coello, J. Li, J. Chen, and J. Zhang, " An external archive-guided multiobjective particle swarm optimization algorithm,” IEEE Trans. Cybern., vol. 47, no. 9, pp. 2794–2808, Sep. 2017. doi: 10.1109/TCYB.2017.2710133
[7]	Q. Z. Lin, S. B. Liu, Q. L. Zhu, C. Y. Tang, R. Z. Song, J. Y. Chen, C. A. Coello Coello, K. C. Wong, and J. Zhang, " Particle swarm optimization with a balanceable fitness estimation for many-objective optimization problems,” IEEE Trans. E Comput., vol. 22, no. 1, pp. 32–46, Feb. 2018. doi: 10.1109/TEVC.2016.2631279
[8]	W. J. Kong, T. Y. Chai, J. L. Ding, and Z. W. Wu, " A real-time multiobjective electric energy allocation optimization approach for the smelting process of magnesia,” Acta Autom. Sinica, vol. 40, no. 1, pp. 51–61, Jan. 2014.
[9]	W. Hu and G. G. Yen, " Adaptive multiobjective particle swarm optimization based on parallel cell coordinate system,” IEEE Trans. E Comput., vol. 19, no. 1, pp. 1–18, Feb. 2015. doi: 10.1109/TEVC.2013.2296151
[10]	A. Trivedi, D. Srinivasan, K. Sanyal, and A. Ghosh, " A survey of multiobjective evolutionary algorithms based on decomposition,” IEEE Trans. E Comput., vol. 21, no. 3, pp. 440–462, Jun. 2017.
[11]	Q. Kang, S. W. Feng, M. C. Zhou, A. C. Ammari, and K. Sedraoui, " Optimal load scheduling of plug-in hybrid electric vehicles via weight-aggregation multi-objective evolutionary algorithms,” IEEE Trans. Intell. Transp. Syst., vol. 18, no. 9, pp. 2557–2568, Sep. 2017. doi: 10.1109/TITS.2016.2638898
[12]	R. T. Haftka, D. Villanueva, and A. Chaudhuri, " Parallel surrogate-assisted global optimization with expensive functions — a survey,” Struct. Multidiscip. Optim., vol. 54, no. 1, pp. 1–11, Apr. 2016. doi: 10.1007/s00158-016-1491-5
[13]	M. Tabatabaei, J. Hakanen, M. Hartikainen, K. Miettinen, and K. Sindhya, " A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods,” Struct. Multidiscip. Optim., vol. 52, no. 1, pp. 1–25, Jul. 2015. doi: 10.1007/s00158-015-1226-z
[14]	Y. Jin, " Surrogate-assisted evolutionary computation: recent advances and future challenges,” Swarm E Comput., vol. 1, no. 2, pp. 61–70, Jun. 2011. doi: 10.1016/j.swevo.2011.05.001
[15]	C. Ma and L. Y. Qu, " Multiobjective optimization of switched reluctance motors based on design of experiments and particle swarm optimization,” IEEE Trans. Energy Convers., vol. 30, no. 3, pp. 1144–1153, Sep. 2015. doi: 10.1109/TEC.2015.2411677
[16]	X. R. Ye, H. Chen, H. M. Liang, X. J. Chen., and J. X. You, " Multi-objective optimization design for electromagnetic devices with permanent magnet based on approximation model and distributed cooperative particle swarm optimization algorithm,” IEEE Trans. Magn., vol. 54, no. 3, pp. 1–5, Mar. 2018.
[17]	H. X. Jie, Y. Z. Wu, J. J. Zhao, J. W. Ding, and L. liang, " An efficient multi-objective PSO algorithm assisted by Kriging metamodel for expensive black-box problems,” J. Glob. Optim., vol. 67, no. 1–2, pp. 399–423, Jan. 2017.
[18]	F. Bittner and I. Hahn, " Kriging-assisted multi-objective particle swarm optimization of permanent magnet synchronous machine for hybrid and electric cars,” in Proc. IEEE Int. Elect. Mach. Drives Conf., Chicago, IL, USA, 2013, pp. 15–22.
[19]	H. D. Wang, Y. C. Jin, and J. O. Jansen, " Committee-based active learning for surrogate-assisted particle swarm optimization of expensive problems,” IEEE Trans. Cybern., vol. 47, no. 9, pp. 2664–2677, Jun. 2017. doi: 10.1109/TCYB.2017.2710978
[20]	M. Parno, T. Hemker, and K. R. Fowler, " Applicability of surrogates to improve efficiency of particle swarm optimization for simulation-based problems,” Eng. Optim., vol. 44, no. 5, pp. 521–535, Sep. 2012. doi: 10.1080/0305215X.2011.598521
[21]	Z. C. Cao, C. R. Lin, M. C. Zhou, and R. Huang, " Scheduling semiconductor testing facility by using cuckoo search algorithm with reinforcement learning and surrogate modeling,” IEEE Trans. Autom. Sci. Eng.. 2018. DOI: 10.1109/TASE.2018.2862380.
[22]	X. Y. Sun, D. W. Gong, Y. C. Jin, and S. S. Chen, " A new surrogate-assisted interactive genetic algorithm with weighted semisupervised learning,” IEEE Trans. Cybern., vol. 43, no. 2, pp. 685–698, Apr. 2013. doi: 10.1109/TSMCB.2012.2214382
[23]	P. Campigotto, A. Passerini, and R. Battiti, " Active learning of Pareto fronts,” IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 3, pp. 506–519, Mar. 2014. doi: 10.1109/TNNLS.2013.2275918
[24]	D. Reker, P. Schneider, and G. Schneider, " Multi-objective active machine learning rapidly improves structure-activity models and reveals new protein-protein interaction inhibitors,” Chem. Sci., vol. 7, pp. 3919–3727, Mar. 2016. doi: 10.1039/C5SC04272K
[25]	M. Zuluaga, G. Sergent, A. Krause, and M. Püschel, " Active learning for multi-objective optimization,” in Proc. Int. Conf. Mach. Learning, Atlanta, GA, USA, 2013, pp. 462–470.
[26]	M. Zuluaga, A. Krause, and M. Puschel, " ε-PAL: an active learning approach to the multi-objective optimization problem,” J. Mach. Learn. Res., vol. 17, no. 1, pp. 3619–3650, Aug. 2016.
[27]	D. F. Wang, and L. Meng, " Performance analysis and parameter selection of PSO algorithms,” Acta Autom. Sinica, vol. 42, no. 10, pp. 1552–1561, Jan. 2016.
[28]	C. E. Rasmussen and H. Nickisch, " Gaussian processes for machine learning (GPML) toolbox,” J. Mach. Learn. Res., vol. 11, pp. 3011–3015, Nov. 2010.
[29]	S. Z. Martínez and C. C. Coello, " A multi-objective particle swarm optimizer based on decomposition,” in Proc. Genetic Evol. Comput., Dublin, Ireland, 2011, pp. 69–76.
[30]	A. J. Nebro, J. J. Durillo, J. Garcia-Nieto, C. A. coello coeelo, F. Luna, and R. Aeba, " SMPSO: a new PSO-based metaheuristic for multi-objective optimization,” in Proc. IEEE Symp. Comput. Intell. Miulticriteria Decision Making, Nashville, TN, USA, 2009, pp. 66–73.
[31]	Y. Tian, R. Cheng, X. Y. Zhang, and Y. C. Jin, " PlatEMO: A MATLAB platform for evolutionary multi-objective optimization,” IEEE Comput. Intell. Mag., vol. 12, pp. 73–87, Nov. 2017.
[32]	Q. F. Zhang, A. M. Zhou, S. Z. Zhao, P. Suganthan, W. D. Liu, and S. Tiwari,, " Multiobjective optimization test instances for the CEC-2009 special session and competition,” Nanyang Technol. Univ., Singapore, Tech. Rep., 2008. http://www.ntu.edu.sg/home/epnsugan/.
[33]	Q. F. Zhang, A. M. Zhou, and Y. C. Jin, " RM-MEDA: a regularity model-based multiobjective estimation of distribution algorithm,” IEEE Trans. E Comput., vol. 21, no. 1, pp. 41–63, Feb. 2008.
[34]	S. Xu, X. An, X. D. Qiao, L. J. Zhu, and L. Li, " Multi-output least-squares support vector regression machines,” Pattern Recognit. Lett., vol. 34, no. 9, pp. 1078–1084, Jul. 2013. doi: 10.1016/j.patrec.2013.01.015
[35]	J. Bergstra and Y. Bengio, " Random search for hyper-parameter optimization,” J. Mach. Learn. Res., vol. 13, no. 1, pp. 281–305, Feb. 2012.
[36]	S. Chen, " Multi-output regression using a locally regularised orthogonal least square algorithm,” IEEE Proc. Vision Image Signal Process., vol. 149, no. 4, pp. 185–195, Aug. 2002. doi: 10.1049/ip-vis:20020401
[37]	Z. Y. Han, Y. Liu, J. Zhao, and W. Wang, " Real time prediction for converter gas tank levels based on multi-output least square support vector regressor,” Control Eng. Pract., vol. 20, no. 12, pp. 1400–1409, Dec. 2012. doi: 10.1016/j.conengprac.2012.08.006

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(4) / Tables(7)

Get Citation

PDF

XML

Article Metrics

Article views (1492) PDF downloads(117)

Surrogate-Assisted Particle Swarm Optimization Algorithm With Pareto Active Learning for Expensive Multi-Objective Optimization

doi: 10.1109/JAS.2019.1911450

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Export File

Citation

Format

Content