FISS GAN: A Generative Adversarial Network for Foggy Image Semantic Segmentation

Kunhua Liu; Zihao Ye; Hongyan Guo; Dongpu Cao; Long Chen; Fei-Yue Wang

doi:10.1109/JAS.2021.1004057

Volume 8 Issue 8

Aug. 2021

IEEE/CAA Journal of Automatica Sinica

JCR Impact Factor: 15.3, Top 1 (SCI Q1)

CiteScore: 23.5, Top 2% (Q1)
Google Scholar h5-index: 77， TOP 5

Turn off MathJax

Article Contents

Article Navigation > IEEE/CAA Journal of Automatica Sinica > 2021 > 8(8): 1428-1439

K. H. Liu, Z. H. Ye, H. Y. Guo, D. P. Cao, L. Chen, and F.-Y. Wang, "FISS GAN: A Generative Adversarial Network for Foggy Image Semantic Segmentation," IEEE/CAA J. Autom. Sinica, vol. 8, no. 8, pp. 1428-1439, Aug. 2021. doi: 10.1109/JAS.2021.1004057

Citation:

K. H. Liu, Z. H. Ye, H. Y. Guo, D. P. Cao, L. Chen, and F.-Y. Wang, "FISS GAN: A Generative Adversarial Network for Foggy Image Semantic Segmentation," IEEE/CAA J. Autom. Sinica, vol. 8, no. 8, pp. 1428-1439, Aug. 2021. doi: 10.1109/JAS.2021.1004057

Citation:

PDF( 1308 KB)

FISS GAN: A Generative Adversarial Network for Foggy Image Semantic Segmentation

doi: 10.1109/JAS.2021.1004057

Kunhua Liu^1
,,
Zihao Ye^1
,,
Hongyan Guo^2
,,
Dongpu Cao^3
,,
Long Chen^{1
,
,},
Fei-Yue Wang^{4, 5
,}

1.
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006, China
2.
State Key Laboratory of Automotive Simulation and Control and the Department of Control Science and Engineering, Jilin University (Campus Nanling), Changchun 130025, China
3.
Mechanical and Mechatronics Engineering Department at the University of Waterloo, 200 University Avenue West Waterloo, ON N2L 3G1, Canada
4.
State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
5.
Institute of Systems Engineering, Macau University of Science and Technology, Macau 999078, China

Funds: This work was supported in part by the National Key Research and Development Program of China (2018YFB1305002), the National Natural Science Foundation of China (62006256), the Postdoctoral Science Foundation of China (2020M683050), the Key Research and Development Program of Guangzhou (202007050002), and the Fundamental Research Funds for the Central Universities (67000-31610134)

More Information

Author Bio:
Kunhua Liu received the Ph.D. degree from Shandong University of Science and Technology in 2019. She is currently a post-doctoral student with the School of Computer Science and Engineering, Sun Yat-sen University. Her research interests include computing vision and SLAM

Zihao Ye received the B.E. degree in software engineering from Sun Yat-sen University, in 2020. He is currently a master student in software engineering at the Sun Yat-Sen University. His research interests include simultaneous localization and mapping and computer vision

Hongyan Guo (M’17) received the Ph.D. degree from Jilin University, in 2010. She joined Jilin University, Changchun, China, in 2011, where she became an Associate Professor in 2014, and has been a Professor since 2018. She was a Visiting Scholar at Cranfield University in 2017. She became the Tang Aoqing Professor in Jilin University in 2020. Her research interests include stability control, path planning and decision making of intelligent vehicles

Dongpu Cao (M’08) received the Ph.D. degree from Concordia University, Canada, in 2008. He is the Canada Research Chair in Driver Cognition and Automated Driving, and currently an Associate Professor and Director of Waterloo Cognitive Autonomous Driving (CogDrive) Laboratory at University of Waterloo, Canada. His current research focuses on driver cognition, automated driving and cognitive autonomous driving. He has contributed more than 200 papers and 3 books. He received the SAE Arch T. Colwell Merit Award in 2012, IEEE VTS 2020 Best Vehicular Electronics Paper Award and three Best Paper Awards from the ASME and IEEE conferences. Prof. Cao serves as Deputy Editor-in-Chief for IET Intelligent Transport Systems Journal, and an Associate Editor for IEEE Transactions on Vehicular Technology, IEEE Transactions on Intelligent Transportation Systems, IEEE/ASME Transactions on Mechatronics, IEEE Transactions on Industrial Electronics, IEEE/CAA Journal of Automatica Sinica, IEEE Transactions on Computational Social Systems, and ASME Journal of Dynamic Systems, Measurement and Control. He was a Guest Editor for Vehicle System Dynamics, IEEE Transactions on SMC: Systems and IEEE Internet of Things Journal. He serves on the SAE Vehicle Dynamics Standards Committee and acts as the Co-Chair of IEEE ITSS Technical Committee on Cooperative Driving. Prof. Cao is an IEEE VTS Distinguished Lecturer

Long Chen (SM’18) received the Ph.D. degree in signal and information processing from Wuhan University, in 2013. From October 2010 to November 2012, he was a co-trained Ph.D. candidate at the National University of Singapore. He is currently an Associate Professor at the School of Computer Science and Engineering, Sun Yat-sen University. He received the IEEE Vehicular Technology Society 2018 Best Land Transportation Paper Award, IEEE Intelligent Vehicle Symposium 2018 Best Student Paper Award and Best Workshop Paper Award. His research interests include autonomous driving, robotics, artificial intelligence where he has contributed more than 110 publications. He serves as an Associate Editor for IEEE Transactions on Intelligent Transportation Systems, IEEE Transactions on Intelligent Vehicles, IEEE/CAA Journal of Automatica Sinica and IEEE Technical Committee on Cyber-Physical Systems Newsletter

Fei-Yue Wang (S’87–M’89–SM’94–F’03) received the Ph.D. degree in computer and systems engineering from the Rensselaer Polytechnic Institute, Troy, NY, USA, in 1990. He joined the University of Arizona in 1990 and became a Professor and the Director of the Robotics and Automation Laboratory and the Program in Advanced Research for Complex Systems. In 1999, he founded the Intelligent Control and Systems Engineering Center at the Institute of Automation, Chinese Academy of Sciences (CAS), Beijing, China, under the support of the Outstanding Chinese Talents Program from the State Planning Council, and in 2002, was appointed as the Director of the Key Laboratory of Complex Systems and Intelligence Science, CAS, and the Vice President of the Institute of Automation, CAS in 2006. In 2011, he became the State Specially Appointed Expert and the Founding Director of the State Key Laboratory for Management and Control of Complex Systems. Dr. Wang has been the Chief Judge of Intelligent Vehicles Future Challenge (IVFC) since 2009 and the Director of China Intelligent Vehicles Proving Center (IVPC) at Changshu since 2015. Currently, he is the Director of Intel's International Collaborative Research Institute on Parallel Driving with CAS and Tsinghua University. His current research focuses on methods and applications for parallel intelligence, social computing, and knowledge automation. He is a Fellow of INCOSE, IFAC, ASME, and AAAS. In 2007, he received the National Prize in Natural Sciences of China, numerous best papers awards from IEEE Transactions, and became an Outstanding Scientist of ACM for his work in intelligent control and social computing. He received the IEEE ITS Outstanding Application and Research Awards in 2009, 2011, and 2015, respectively, the IEEE SMC Norbert Wiener Award in 2014, and became the IFAC Pavel J. Nowacki Distinguished Lecturer in 2021. Since 1997, he has been serving as the General or Program Chair of over 30 IEEE, INFORMS, IFAC, ACM, and ASME conferences. He was the President of the IEEE ITS Society from 2005 to 2007, the IEEE Council of RFID from 2019 to 2021, the Chinese Association for Science and Technology, USA, in 2005, the American Zhu Kezhen Education Foundation from 2007 to 2008, the Vice President of the ACM China Council from 2010 to 2011, the Vice President and the Secretary General of the Chinese Association of Automation from 2008–2018. He was the Founding Editor-in-Chief (EiC) of the International Journal of Intelligent Control and Systems from 1995 to 2000, the IEEE ITS Magazine from 2006 to 2007, the IEEE/CAA Journal of Automatica Sinica from 2014 to 2017, China’s Journal of Command and Control from 2015 to 2021, and China’s Journal of Intelligent Science and Technology from 2019 to 2021. He was the EiC of the IEEE Intelligent Systems from 2009 to 2012, the IEEE Transactions on Intelligent Transportation Systems from 2009 to 2016, and the IEEE Transactions on Computational Social Systems from 2017 to 2020. Currently, he is the President of CAA’s Supervision Council and Vice President of IEEE Systems, Man, and Cybernetics Society
Corresponding author: Long Chen, e-mail: chenl46@mail.sysu.edu.cn
Received Date: 2021-03-11
Accepted Date: 2021-04-17

Available Online: 2021-05-14

Abstract

Abstract

Because pixel values of foggy images are irregularly higher than those of images captured in normal weather (clear images), it is difficult to extract and express their texture. No method has previously been developed to directly explore the relationship between foggy images and semantic segmentation images. We investigated this relationship and propose a generative adversarial network (GAN) for foggy image semantic segmentation (FISS GAN), which contains two parts: an edge GAN and a semantic segmentation GAN. The edge GAN is designed to generate edge information from foggy images to provide auxiliary information to the semantic segmentation GAN. The semantic segmentation GAN is designed to extract and express the texture of foggy images and generate semantic segmentation images. Experiments on foggy cityscapes datasets and foggy driving datasets indicated that FISS GAN achieved state-of-the-art performance.
- Edge GAN,
- foggy images,
- foggy image semantic segmentation GAN,
- semantic segmentation

FullText(HTML)

References(68)

References

[1]	L. Chen, W. J. Zhan, W. Tian, Y. H. He, and Q. Zou, “Deep integration: A multi-label architecture for road scene recognition,” IEEE Trans. Image Process., vol. 28, no. 10, pp. 4883–4898, Oct. 2019. doi: 10.1109/TIP.2019.2913079
[2]	K. Wada, K. Okada, and M. Inaba, “Joint learning of instance and semantic segmentation for robotic pick-and-place with heavy occlusions in clutter,” in Proc. IEEE Int. Conf. Robotics and Autom., Montreal, Canada, 2019, pp. 9558–9564.
[3]	Y. C. Ouyang, L. Dong, L. Xue and C. Y. Sun, “Adaptive control based on neural networks for an uncertain 2-DOF helicopter system with input deadzone and output constraints,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 807–815, May 2019. doi: 10.1109/JAS.2019.1911495
[4]	Y. H. Luo, S. N. Zhao, D. S. Yang, and H. W. Zhang, “A new robust adaptive neural network backstepping control for single machine infinite power system with TCSC,” IEEE/CAA J. Autom. Sinica, vol. 7, no. 1, pp. 48–56, Jan. 2020.
[5]	N. Zerari, M. Chemachema, and N. Essounbouli, “Neural network based adaptive tracking control for a class of pure feedback nonlinear systems with input saturation,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 1, pp. 278–290, Jan. 2019. doi: 10.1109/JAS.2018.7511255
[6]	D. Wu and X. Luo, “Robust latent factor analysis for precise representation of high-dimensional and sparse data,” IEEE/CAA J. Autom. Sinica, pp. 766–805, Dec. 2019.
[7]	X. Luo, Y. Yuan, S. L. Chen, N. Y. Zeng, and Z. D. Wang, “Position-transitional particle swarm optimization-incorporated latent factor analysis,” IEEE Trans. Knowl. Data. En., pp. 1–1, Oct. 2019.
[8]	A. Cantor, “Optics of the atmosphere: Scattering by molecules and particles,” IEEE J. Quantum. Elect., vol. 14, no. 9, pp. 698–699, Sept. 1978.
[9]	S. G. Narasimhan and S. K. Nayar, “Vision and the atmosphere,” Int. J. Comput. Vision, vol. 48, no. 3, pp. 233–254, Jul. 2002. doi: 10.1023/A:1016328200723
[10]	P. Luc, C. Couprie, S, Chintala, and J. Verbeek, “Semantic segmentation using adversarial networks,” arXiv preprint arXiv: 1611.08408, Dec. 2016.
[11]	A. Arnab, S. Jayasumana, S. Zheng, and P. H. S. Torr, “Higher order conditional random fields in deep neural networks,” in Proc. European Conf. Computer Vision, B. Leibe, J. Matas, N. Sebe and M. Welling Eds. Cham, Germany: Springer, 2016, pp. 524–540.
[12]	P. Isola, J. Y. Zhu, T. H. Zhou, and A. Efros, “Image-to-image translation with conditional adversarial networks,” in Proc. IEEE. Conf. Computer Vision and Pattern Recognition, Honolulu, USA, 2017, pp. 1125–1134.
[13]	J. Hoffman, E. Tzeng, T. Park, Y. J. Zhu, P. Isola, K. Saenko, A. Efros, and T. Darrell, “Cycada: Cycle-consistent adversarial domain adaptation,” in Proc. 35th Int. Conf. Machine Learning, Stockholm, Sweden, 2018, pp. 1989–1998.
[14]	K. Nazeri, E. Ng, T. Joseph, F. Z. Qureshi, and M. Ebrahimi, “Edgeconnect: Generative image inpainting with adversarial edge learning,” arXiv preprint arXiv: 1901.00212, Jan. 2019.
[15]	O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Proc. Int. Conf. Medical Image Computing and Computer-assisted Intervention. Cham, Germany: Springer, 2015, pp. 234–241.
[16]	J. Y. Kim, L. S. Kim, and S. H. Hwang, “An advanced contrast enhancement using partially overlapped sub-block histogram equalization,” in Proc. Int. Conf. IEEE Symposium on Circuits and Systems, Geneva, Switzerland, 2000, pp. 537–540.
[17]	A. Eriksson, G. Capi, and K. Doya, “Evolution of meta-parameters in reinforcement learning algorithm,” in Proc. IEEE/RSJ. Int. Conf. Intelligent Robots and System, Las Vegas, USA, 2003, pp. 412–417.
[18]	M. J. Seow and V. K. Asari, “Ratio rule and homomorphic filter for enhancement of digital colour image,” Neurocomputing, vol. 69, no. 7–9, pp. 954–958, Mar. 2006. doi: 10.1016/j.neucom.2005.07.003
[19]	S. Shwartz, E. Namer, and Y. Y. Schechner, “Blind haze separation,” in Proc. IEEE. Int. Conf. Computer Vision and Pattern Recognition, New York, USA, 2006, pp. 1984–1991.
[20]	Y. Y. Schechner and Y. Averbuch, “Regularized image recovery in scattering media,” IEEE Trans. Pattern Anal., vol. 29, no. 9, pp. 1655–1660, Sept. 2007. doi: 10.1109/TPAMI.2007.1141
[21]	R. T. Tan, “Visibility in bad weather from a single image,” in Proc. IEEE. Int. Conf. Computer Vision and Pattern Recognition, Anchorage, USA, 2008, pp. 1–8.
[22]	R. Fattal, “Single image dehazing,” ACM Trans. Graphic., vol. 27, no. 3, pp. 1–9, Aug. 2008.
[23]	K. M. He, J. Sun, and X. O. Tang, “Single image haze removal using dark channel prior,” IEEE Trans. Pattern Anal., vol. 33, no. 12, pp. 2341–2353, Dec. 2011. doi: 10.1109/TPAMI.2010.168
[24]	K. B. Gibson and T. Q. Nguyen, “On the effectiveness of the dark channel prior for single image dehazing by approximating with minimum volume ellipsoids,” in Proc. IEEE. Int. Conf. Acoustics, Speech and Signal Processing, Prague, Czech Republic, 2011, pp. 1253–1256.
[25]	D. F. Shi, B. Li, W. Ding, and Q. M. Chen, “Haze removal and enhancement using transmittance-dark channel prior based on object spectral characteristic,” Acta Autom. Sinica, vol. 39, no. 12, pp. 2064–2070, Dec. 2013.
[26]	S. G. Narasimhan and S. K. Nayar, “Interactive (de) weathering of an image using physical models,” in Proc. IEEE Workshop on color and photometric Methods in computer Vision, vol. 6, no. 4, Article No. 1, Jan. 2003.
[27]	S. G. Narasimhan and S. K. Nayar, “Chromatic framework for vision in bad weather,” in Proc. IEEE. Int. Conf. Computer Vision and Pattern Recognition, Hilton Head, USA, 2000, pp. 598–605.
[28]	J. Tarel and N. Hautière, “Fast visibility restoration from a single color or gray level image,” in Proc. 12th IEEE Int. Conf. Computer Vision, Kyoto, Japan, 2009, pp. 2201–2208.
[29]	H. Zhang, V. Sindagi, and V. M. Patel, “Joint transmission map estimation and dehazing using deep networks,” IEEE Trans. Circ. Syst. Vid., vol. 30, no. 7, Jul. 2020.
[30]	W. Q. Ren, S. Liu, H. Zhang, J. S. Pan, X. C. Cao, and M. H. Yang, “Single image dehazing via multi-scale convolutional neural networks,” in Proc. European Conf. Computer Vision, B. Leibe, J. Matas, N. Sebe and M. Welling Eds. Cham, Germany: Springer, 2016, pp. 154–169.
[31]	H. Zhang and V. M. Patel, “Densely connected pyramid defogging network,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018, pp. 3194–3203.
[32]	B. L. Cai, X. M. Xu, K. Jia, C. M. Qing, and D. C. Tao, “DehazeNet: An end-to-end system for single image haze removal,” IEEE Trans. Image Process., vol. 25, no. 11, pp. 5187–5198, Nov. 2016. doi: 10.1109/TIP.2016.2598681
[33]	D. D. Chen, M. M. He, Q. N. Fan, J. Liao, L. H. Zhang, D. D. Hou, L. Yuan, and G. Hua, “Gated context aggregation network for image dehazing and deraining,” in Proc. IEEE Winter Conf. Applications of Computer Vision, Waikoloa, USA, 2019, pp. 1375–1383.
[34]	S. Y. Huang, H. X. Li, Y. Yang, B. Wang, and N. N. Rao, “An end-to-end dehazing network with transitional convolution layer,” Multidim. Syst. Sign P., vol. 31, no. 4, pp. 1603–1623, Mar. 2020. doi: 10.1007/s11045-020-00723-2
[35]	H. Zhang, V. Sindagi, and V. M. Patel, “Multi-scale single image dehazing using perceptual pyramid deep network,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, Salt Lake City, USA, 2018, pp. 902–911.
[36]	X. Qin, Z. L. Wang, Y. C. Bai, X. D. Xie, and H. Z. Jia, “FFA-Net: Feature fusion attention network for single image defogging,” arXiv preprint arXiv: 1911.07559, Nov. 2019.
[37]	Q. S. Yi, A. W. Jiang, J. C. Li, J. Y. Wan, and M. W. Wang, “Progressive back-traced dehazing network based on multi-resolution recurrent reconstruction,” IEEE Access, vol. 8, pp. 54514–54521, Mar. 2020. doi: 10.1109/ACCESS.2020.2981491
[38]	B. Y. Li, X. L. Peng, Z. Y. Wang, J. Z. Xu, and D. Feng, “An all-in-one network for defogging and beyond,” arXiv preprint arXiv: 1707.06543, Jul. 2017.
[39]	H. Y. Zhu, X. Peng, V. Chandrasekhar, L. Y. Li, and J. H. Lim, “DehazeGAN: When image dehazing meets differential programming,” in Proc. 27th Int. Joint Conf. Artificial Intelligence, Stockholm, Sweden, 2018, pp. 1234–1240.
[40]	R. D. Li, J. S. Pan, Z. C. Li, and J. H. Tang, “Single image dehazing via conditional generative adversarial network,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018, pp. 8202–8211.
[41]	D. Engin, A. Genc, and H. K. Ekenel, “Cycle-dehaze: enhanced cycleGAN for single image dehazing,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, Salt Lake City, USA, 2018, pp. 825–833.
[42]	G. Kim, J. Park, S. Ha, and J. Kwon, “Bidirectional deep residual learning for haze removal,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, Long Beach Convention and Entertainment Center, USA, 2018, pp. 46–54.
[43]	A. Dudhane and S. Murala, “CDNet: Single image de-hazing using unpaired adversarial training,” in Proc. IEEE Winter Conf. Applications of Computer Vision, Waikoloa, USA, 2019, pp. 1147–1155.
[44]	P. Sharma, P. Jain, and A. Sur, “Scale-aware conditional generative adversarial network for image defogging,” in Proc. IEEE Winter Conf. Applications of Computer Vision, Snowmass, USA, 2020, pp. 2355–2365.
[45]	W. D. Yan, A. Sharma, and R. T. Tan, “Optical flow in dense foggy scenes using semi-supervised learning,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, USA, 2020, pp. 13259–13268.
[46]	J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, USA, 2015, pp. 3431–3440.
[47]	L. Chen, W. J. Zhan, J. J. Liu, W. Tian, and D. P. Cao, “Semantic segmentation via structured refined prediction and dual global priors,” in Proc. IEEE Int. Conf. Advanced Robotics and Mechatronics, Toyonaka, Japan, 2019, pp. 53–58.
[48]	Y. H. Yuan, X. L. Chen, and J. D. Wang, “Object-contextual representations for semantic segmentation,” arXiv preprint arXiv: 1909.11065, Sept. 2019.
[49]	S. Choi, J. T. Kim, and J. Choo, “Cars can’t fly up in the sky: Improving urban-scene segmentation via height-driven attention networks,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, USA, 2020, pp. 9373–9383.
[50]	M. H. Yin, Z. L. Yao, Y. Cao, X. Li, Z. Zhang, S. Lin, and H. Hu, “Disentangled non-local neural networks,” arXiv preprint arXiv: 2006.06668, Sept. 2020.
[51]	H. S. Zhao, J. P. Shi, X. J. Qi, X. G. Wang, and J. Y. Jia, “Pyramid scene parsing network,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, USA, 2017, pp. 2881–2890.
[52]	L. C. Chen, Y. K. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in Proc. European Conf. Computer Vision, Munich, Germany, 2018, pp. 801–818.
[53]	H. Y. Chen, L. H. Tsai, S. C. Chang, J. Y. Pan, Y. T. Chen, W. Wei, and D. C. Juan, “Learning with hierarchical complement objective,” arXiv preprint arXiv: 1911.07257, Nov, 2019.
[54]	J. Fu, J. Liu, H. J. Tian, Y. Li, Y. J. Bao, Z. W. Fang, and H. Q. Lu, “Dual attention network for scene segmentation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Long Beach, USA, 2019, pp. 3146–3154.
[55]	C. Zhang, G. S. Lin, F. Y. Liu, R. Yao, and C. H. Chen, “Canet: Class-agnostic segmentation networks with iterative refinement and attentive few-shot learning,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Long Beach, USA, 2019, pp. 5217–5226.
[56]	J. J. He, Z. Y. Deng, L. Zhou, Y. L. Wang, and Y. Qiao, “Adaptive pyramid context network for semantic segmentation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Long Beach, USA, 2019, pp. 7519–7528.
[57]	J. Hoffman, D. Q. Wang, F. Yu, and T. Darrell, “FCNs in the wild: Pixel-level adversarial and constraint-based adaptation,” arXiv preprint arXiv: 1612.02649, Dec, 2016.
[58]	Y. H. Zhang, Z. F. Qiu, T. Yao, D. Liu, and T. Mei, “Fully convolutional adaptation networks for semantic segmentation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018, pp. 6810–6818.
[59]	Z. Murez, S. Kolouri, D. Kriegman, R. Ramamoorthi, and K. Kim, “Image to image translation for domain adaptation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018, pp. 4500–4509.
[60]	W. X. Hong, Z. Z. Wang, M. Yang, and J. S. Yuan, “Conditional generative adversarial network for structured domain adaptation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018, pp. 1335–1344.
[61]	Y. W. Luo, L. Zheng, T. Guan, J. Q. Yu, and Y. Yang, “Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Long Beach, USA, 2019, pp. 2502–2511.
[62]	Y. S. Li, L. Yuan, and N. Vasconcelos, “Bidirectional learning for domain adaptation of semantic segmentation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Long Beach, USA, 2019, pp. 6936–6945.
[63]	J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal., vol. PAMI-8, no. 6, pp. 679–698, Nov. 1986. doi: 10.1109/TPAMI.1986.4767851
[64]	J. Y. Zhu, T. Park, P. Isola and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proc. IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 2223–2232.
[65]	C. Sakaridis, D. X. Dai, and L. Van Gool, “Semantic foggy scene understanding with synthetic data,” Int. J. Comput. Vision, vol. 126, no. 9, pp. 973–992, Mar. 2018. doi: 10.1007/s11263-018-1072-8
[66]	V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proc. 27th Int. Conf. on Int. Conf. on Machine Learning, F. Johannes and J. Thorsten, Eds, 2010, pp. 807–814.
[67]	A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acoustic models,” in Proc. ICML, vol. 30, no. 1, pp. 3, 2013.
[68]	D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv: 1412.6980, Dec. 2014.

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(9) / Tables(6)

Get Citation

PDF

XML

Article Metrics

Article views (1426) PDF downloads(124)

Highlights

No method has previously been developed to directly explore the relationship between foggy images and semantic segmentation images. We investigated this relationship and propose a generative adversarial network (GAN) for foggy image semantic segmentation.
We propose a novel efficient network architecture based on a combination of concepts from U_Net, called a dilated convolution U_Net. By incorporating dilated convolution layers and adjusting the feature size in the convolutional layer, dilated convolution U_Net has shown improved feature extraction and expression ability.
A direct FISS GAN that generates semantic segmentation images under edge information guidance is proposed. We show our method’s effectiveness through extensive experiments on foggy cityscapes datasets and foggy driving datasets and achieve state-of-the-art performance.

FISS GAN: A Generative Adversarial Network for Foggy Image Semantic Segmentation

doi: 10.1109/JAS.2021.1004057

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Highlights

Export File

Citation

Format

Content