A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 6 Issue 6
Nov.  2019

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2019  >  6(6): 1428-1437
Qiusheng Lian, Wenfeng Yan, Xiaohua Zhang and Shuzhen Chen, "Single Image Rain Removal Using Image Decomposition and a Dense Network," IEEE/CAA J. Autom. Sinica, vol. 6, no. 6, pp. 1428-1437, Nov. 2019. doi: 10.1109/JAS.2019.1911441
Citation: Qiusheng Lian, Wenfeng Yan, Xiaohua Zhang and Shuzhen Chen, "Single Image Rain Removal Using Image Decomposition and a Dense Network," IEEE/CAA J. Autom. Sinica, vol. 6, no. 6, pp. 1428-1437, Nov. 2019. doi: 10.1109/JAS.2019.1911441
shu

Single Image Rain Removal Using Image Decomposition and a Dense Network

doi: 10.1109/JAS.2019.1911441
  • 1.

    School of Information Science and Engineering, Yanshan University, Qinhuangdao 066004, China

  • 2.

    School of Mathematics and Information Science and Technology, Hebei Normal University of Science and Technology, Qinhuangdao 066004, China

Funds:

the National Natural Science Foundation of China 61471313

the Natural Science Foundation of Hebei Province F2019203318

More Information
    Abstract
  • Removing rain from a single image is a challenging task due to the absence of temporal information. Considering that a rainy image can be decomposed into the low-frequency (LF) and high-frequency (HF) components, where the coarse scale information is retained in the LF component and the rain streaks and texture correspond to the HF component, we propose a single image rain removal algorithm using image decomposition and a dense network. We design two task-driven sub-networks to estimate the LF and non-rain HF components of a rainy image. The high-frequency estimation sub-network employs a densely connected network structure, while the low-frequency sub-network uses a simple convolutional neural network (CNN). We add total variation (TV) regularization and LF-channel fidelity terms to the loss function to optimize the two subnetworks jointly. The method then obtains de-rained output by combining the estimated LF and non-rain HF components. Extensive experiments on synthetic and real-world rainy images demonstrate that our method removes rain streaks while preserving non-rain details, and achieves superior de-raining performance both perceptually and quantitatively.

     

    • FullText(HTML)
  • loading
    • References(40)
  • [1]
    S. Ren, K. He, R. Girshick, and S. Jian, "Faster r-CNN: towards realtime object detection with region proposal networks, " in Proc. Int. Conf. Advances in Neural Information Processing Systems, Montreal, Canada, 2015, pp. 91-99. http://cn.bing.com/academic/profile?id=4f54714904ee7ce1557ceb832f17d8b7&encoded=0&v=paper_preview&mkt=zh-cn
    [2]
    H. Nam and B. Han, "Learning multi-domain convolutional neural networks for visual tracking, " Computer Vision and Pattern Recognition (CVPR), pp. 4293-4302, 2015.
    [3]
    J. H. Kim, C. Lee, J. Y. Sim, and C. S. Kim, "Single-image deraining using an adaptive nonlocal means filter, " in Proc. IEEE Int. Conf. Image Processing, Melbourne, Australia, 2013, pp. 914-917. http://cn.bing.com/academic/profile?id=09f21b1d97446c2963d9f36005365fcf&encoded=0&v=paper_preview&mkt=zh-cn
    [4]
    Y. L. Chen and C. T. Hsu, "A generalized low-rank appearance model for spatio-temporally correlated rain streaks, " in IEEE Int. Conf. Computer Vision, 2013, Sydney, Australia, pp. 1968-1975. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CC0213993873
    [5]
    Y. Chang, L. Yan, and S. Zhong, "Transformed low-rank model for line pattern noise removal, " in Proc. IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 1726-1734. http://cn.bing.com/academic/profile?id=fa1f9f42d4b9336a0e4d56ba4aa7fb34&encoded=0&v=paper_preview&mkt=zh-cn
    [6]
    L. W. Kang, C. W. Lin, and Y. H. Fu, "Automatic single-image-based rain streaks removal via image decomposition, " IEEE Trans. Image Processing, vol. 21, no. 4, pp. 1742-1755, 2012. doi: 10.1109/TIP.2011.2179057
    [7]
    D. A. Huang, L. W. Kang, Y. C. F. Wang, and C. W. Lin, "Self-learning based image decomposition with applications to single image denoising, " IEEE Trans. Multimedia, vol. 16, no. 1, pp. 83-93, 2013. http://cn.bing.com/academic/profile?id=3881e0312572bdfec7772ee257a5e4cb&encoded=0&v=paper_preview&mkt=zh-cn
    [8]
    S. H. Sun, S. P. Fan, and Y. C. F. Wang, "Exploiting image structural similarity for single image rain removal, " in Proc. IEEE Int. Conf. Image Processing, Quebec, Canada, 2015, pp. 4482-4486. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CC0214832869
    [9]
    Y. Luo, Y. Xu, and H. Ji, "Removing rain from a single image via discriminative sparse coding, " in Proc. IEEE Int. Conf. Computer Vision, Santiago, Chile, 2015, pp. 3397-3405. http://cn.bing.com/academic/profile?id=842d028b636e0d047d7b6e7d2837fda9&encoded=0&v=paper_preview&mkt=zh-cn
    [10]
    Y. Li, R. T. Tan, X. J. Guo, J. B. Lu, and M. S. Brown, "Rain streak removal using layer priors, " in Proc. Computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 2736-2744.
    [11]
    L. Zhu, C. W. Fu, D. Lischinski, and P. A. Heng, "Joint bi-layer optimization for single-image rain streak removal, " in Proc. IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 2526-2534. http://cn.bing.com/academic/profile?id=0be776bba237ff6bad63d7a0ff7b54fa&encoded=0&v=paper_preview&mkt=zh-cn
    [12]
    D. Eigen, D. Krishnan, and R. Fergus, "Restoring an image taken through a window covered with dirt or rain, " in Proc. IEEE Int. Conf. Computer Vision, Sydney, Australia, 2013, pp. 633-640. http://cn.bing.com/academic/profile?id=63bb9b7a86ac419a429205e7f6bacc46&encoded=0&v=paper_preview&mkt=zh-cn
    [13]
    X. Fu, J. Huang, X. Ding, Y. Liao, and J. Paisley, "Clearing the skies: a deep network architecture for single-image rain removal, " IEEE Trans. Image Processing, vol. 26, no. 6, pp. 2944-2956, 2017. doi: 10.1109/TIP.2017.2691802
    [14]
    X. Fu, J. Huang, D. Zeng, Y. Huang, X. Ding, and J. Paisley, "Removing rain from single images via a deep detail network, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, 2017.
    [15]
    W. Yang, R. T. Tan, J. Feng, J. Liu, Z. Guo, and S. Yan, "Deep joint rain detection and removal from a single image, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Puerto Rico, USA, 2017, 1357-1366. http://cn.bing.com/academic/profile?id=77fbb15463a804f934a3a95cb2a7ac2d&encoded=0&v=paper_preview&mkt=zh-cn
    [16]
    H. Zhang, V. Sindagi, and V. M. Patel, "Image de-raining using a conditional generative adversarial network, " IEEE Trans. Circuits and Systems for Video Technolagy, pp. 99, Jan. 2017.
    [17]
    Z. F. Fan, H. F. Wu, X. Y. Fu, Y. Huang, and X. Y. Ding, "Residual-guide feature fusion network for single image deraining, " [Online]. available: https://arxiv.org/abs/1804.07493. Mar. 26, 2019.
    [18]
    S. Li, W. Ren, J. Zhang, J. Yu, and X. Guo, "Fast single image rain removal via a deep decomposition-composition network, " [Online]. available: https://arxiv.org/abs/1804.02688. Mar. 26, 2019.
    [19]
    G. Huang, Z. Liu, V. D. M. Laurens, and K. Q. Weinberger, "Densely connected convolutional networks, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Puerto Rico, USA, 2017.
    [20]
    J. Bossu, N. Hautire, and J. P. Tarel, "Rain or snow detection in image sequences through use of a histogram of orientation of streaks, " Int. J. Computer Vision, vol. 93, no. 3, pp. 348-367, 2011. doi: 10.1007/s11263-011-0421-7
    [21]
    K. Garg and S. K. Nayar, "Detection and removal of rain from videos, " in Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition, CVPR. Washington, USA, 2004, pp. Ⅰ-528-Ⅰ-535.
    [22]
    J. H. Kim, J. Y. Sim, and C. S. Kim, "Video deraining and desnowing using temporal correlation and low-rank matrix completion, " IEEE Trans. Image Processing, vol. 24, no. 9, pp. 2658-2670, 2015. doi: 10.1109/TIP.2015.2428933
    [23]
    V. Santhaseelan and V. K. Asari, "Utilizing local phase information to remove rain from video, " Int. J. of Computer Vision, vol. 112, no. 1, pp. 71-89, 2015. doi: 10.1007/s11263-014-0759-8
    [24]
    S. You, R. T. Tan, R. Kawakami, Y. Mukaigawa, and K. Ikeuchi, "Adherent raindrop modeling, detectionand removal in video, " IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 38, no. 9, pp. 1721-1733, 2016. doi: 10.1109/TPAMI.2015.2491937
    [25]
    C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images, " Computer Vision, pp. 839-846, 1998.
    [26]
    K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a Gaussian denoiser: residual learning of deep cnn for image denoising, " IEEE Trans. Image Processing, vol. 26, no. 7, pp. 3142-3155, Jul. 2017. https://arxiv.org/pdf/1608.03981.pdf
    [27]
    B. Cai, X. Xu, K. Jia, C. Qing, and D. Tao, "Dehazenet: an end-to-end system for single image haze removal, " IEEE Trans. Image Processing, vol. 25, no. 11, pp. 5187-5198, 2016. doi: 10.1109/TIP.2016.2598681
    [28]
    C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks, " IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295-307, 2016. doi: 10.1109/TPAMI.2015.2439281
    [29]
    K. He, J. Sun, and X. Tang, "Guided image filtering, " IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no. 6, pp. 1397- 1409, 2013. doi: 10.1109/TPAMI.2012.213
    [30]
    K. M. He, X. Y. Zhang, S. Q. Ren, and J. Sun, "Deep residual learning for image recognition, " in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 770-778.
    [31]
    A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks, " in Proc. Int. Conf. Neural Information Processing Systems, Lake Tahoe, Nevada, USA, 2012, pp. 1097-1105.
    [32]
    F. Yu and V. Koltun, "Multi-scale Context Aggregation by Dilated Convolutions, " [Online]. Available: https://arxiv.org/abs/1511.07122. Mar. 26, 2019.
    [33]
    H. Le and A. Borji, "What are the Receptive, Effective Receptive, and Projective Fields of Neurons In Convolutional Neural Networks?" [Online]. Available: https://arxiv.org/abs/1705.07049. Mar. 26, 2019.
    [34]
    L. I. Rudin, S. Osher, and E. Fatemi, "Nonlinear total variation based noise removal algorithms, " Physica D: Nonlinear Phenomena, , 1992, vol. 60, no. 1-4, pp. 259-268, 1992. doi: 10.1016/0167-2789(92)90223-A
    [35]
    W. Zhou, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity, " IEEE Trans Image Process, vol. 13, no. 4, pp. 600-612, 2004. doi: 10.1109/TIP.2003.819861
    [36]
    S. Patterson, "Photoshop Photo Effects Tutorials, " [Online]. available: http://www.photoshopessentials.com/photo-effects. Mar. 26, 2019.
    [37]
    A. Pablo, M. Michael, F. Charless, and M. Jitendra, "Contour detection and hierarchical image segmentation, " IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 898-916, 2011. doi: 10.1109/TPAMI.2010.161
    [38]
    M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, and M. Devin, "Tensorflow: a system for large-scale machine learning, " [Online]. available: https://arxiv.org/abs/1605.08695. Mar. 26, 2019.
    [39]
    D. Kingma and J. Ba, "Adam: a method for stochastic optimization, " [Online]. available: https://arxiv.org/abs/1412.6980. Mar. 26, 2019.
    [40]
    X. Glorot and Y. Bengio, "Understanding the difficulty of training deep feedforward neural networks, " in Proc. 13th Int. Conf. Artificial Intelligence and Statistics, Sardinia, Italy, 2010, pp. 249-256.
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(12)  / Tables(4)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (1571) PDF downloads(131) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return