A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 6 Issue 2
Mar.  2019

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2019  >  6(2): 540-552
Xiaohua Wang and Haibin Duan, "Hierarchical Visual Attention Model for Saliency Detection Inspired by Avian Visual Pathways," IEEE/CAA J. Autom. Sinica, vol. 6, no. 2, pp. 540-552, Mar. 2019. doi: 10.1109/JAS.2017.7510664
Citation: Xiaohua Wang and Haibin Duan, "Hierarchical Visual Attention Model for Saliency Detection Inspired by Avian Visual Pathways," IEEE/CAA J. Autom. Sinica, vol. 6, no. 2, pp. 540-552, Mar. 2019. doi: 10.1109/JAS.2017.7510664
shu

Hierarchical Visual Attention Model for Saliency Detection Inspired by Avian Visual Pathways

doi: 10.1109/JAS.2017.7510664
  • 1.

    State Key Laboratory of Virtual Reality Technology and Systems, School of Automation Science and Electrical Engineering, Beihang University, Beijing 100083, China

  • 2.

    Peng Cheng Laboratory, Shenzhen 518055, China

Funds:

Natural Science Foundation of China 61425008

Natural Science Foundation of China 61333004

Natural Science Foundation of China 61273054

More Information
    Abstract
  • Visual attention is a mechanism that enables the visual system to detect potentially important objects in complex environment. Most computational visual attention models are designed with inspirations from mammalian visual systems. However, electrophysiological and behavioral evidences indicate that avian species are animals with high visual capability that can process complex information accurately in real time. Therefore, the visual system of the avian species, especially the nuclei related to the visual attention mechanism, are investigated in this paper. Afterwards, a hierarchical visual attention model is proposed for saliency detection. The optic tectum neuron responses are computed and the self-information is used to compute primary saliency maps in the first hierarchy. The "winner-take-all" network in the tecto-isthmal projection is simulated and final saliency maps are estimated with the regularized random walks ranking in the second hierarchy. Comparison results verify that the proposed model, which can define the focus of attention accurately, outperforms several state-of-the-art models. This study provides insights into the relationship between the visual attention mechanism and the avian visual pathways. The computational visual attention model may reveal the underlying neural mechanism of the nuclei for biological visual attention.

     

    • FullText(HTML)
  • loading
    • References(55)
  • [1]
    C. Koch and S. Ullman, "Shifts in selective visual attention: towards the underlying neural circuitry, " Hum. Neurobiol., vol. 4, no. 4, pp. 219-227, Feb. 1985. https://www.ncbi.nlm.nih.gov/pubmed/3836989
    [2]
    S. Kastner and L. G. Ungerleider, "Mechanisms of visual attention in the human cortex, " Annu. Rev. Neurosci., vol. 23, no. 1, pp. 315-341, Mar. 2000. doi: 10.1146/annurev.neuro.23.1.315
    [3]
    J. P. Gottlieb, M. Kusunoki, and M. E. Goldberg, "The representation of visual salience in monkey parietal cortex, " Nature, vol. 391, no. 6666, pp. 481-484, Jan. 1998. https://www.nature.com/articles/35135?error=cookies_not_supported&code=49457ee0-5c90-424a-b2c2-c21a2e34201a
    [4]
    H. C. Mueller, "Factors influencing prey selection in the American kestrel, " Auk, vol. 91, no. 4, pp. 705-721, Oct. 1974.
    [5]
    W. M. Harmening, J. Orlowski, O. Ben-Shahar, and H. Wagner, "Overt attention toward oriented objects in free-viewing barn owls, " Proc. Natl. Acad. Sci. USA, vol. 108, no. 20, pp. 8461-8466, May 2011. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3100958/
    [6]
    R. Desimone and J. Duncan, "Neural mechanisms of selective visual attention, " Annu. Rev. Neurosci., vol. 18, no. 1, pp. 193-222, Mar. 1995.
    [7]
    H. B. Duan and X. H. Wang, "Visual attention model based on statistical properties of neuron responses, " Sci. Rep., vol. 5, Article No. 8873, Mar. 2015. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4352866/
    [8]
    W. M. Harmening and H. Wagner, "From optics to attention: visual perception in barn owls, " J. Comp. Physiol. A, vol. 197, no. 11, pp. 1031-1042, Nov. 2011. doi: 10.1007%2Fs00359-011-0664-3.pdf
    [9]
    E. T. Rolls and G. Deco, "Attention in natural scenes: neurophysiological and computational bases, " Neural Netw., vol. 19, no. 9, pp. 1383-1394, Nov. 2006. https://www.sciencedirect.com/science/article/pii/S0893608006001857
    [10]
    H. B. Duan, Y. M. Deng, X. H. Wang, and C. F. Xu, "Small and dim target detection via lateral inhibition filtering and artificial bee colony based selective visual attention, " PLoS ONE, vol. 8, no. 8, pp. e72035, Aug. 2013. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3749114/
    [11]
    L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998. https://ieeexplore.ieee.org/document/730558
    [12]
    J. Harel, C. Koch, and P. Perona, "Graph-based visual saliency, " in Proc. Advances in Neural Information Processing Systems, Vancouver, Canada, 2006, pp. 545-552.
    [13]
    C. Yang, L. H. Zhang, H. C. Lu, X. Ruan, and M. H. Yang, "Saliency detection via graph-based manifold ranking, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Portland, USA, 2013, pp. 3166-3173.
    [14]
    C. Y. Li, Y. C. Yuan, W. D. Cai, Y. Xia, and D. D. Feng, "Robust saliency detection via regularized random walks ranking, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, 2015, pp. 2710-2717.
    [15]
    L. Itti and P. Baldi, "Bayesian surprise attracts human attention, " Vis. Res., vol. 49, no. 10, pp. 1295-1306, Jun. 2009.
    [16]
    Y. L. Xie, H. C. Lu, and M. H. Yang, "Bayesian saliency via low and mid level cues, " IEEE Trans. Image Process., vol. 22, no. 5, pp. 1689-1698, May 2013. https://ieeexplore.ieee.org/document/6291786/
    [17]
    L. Y. Zhang, M. H. Tong, T. K. Marks, H. H. Shan, and G. W. Cottrell, "SUN: A Bayesian framework for saliency using natural statistics, " J. Vis., vol. 8, no. 7, pp. 1-20, Dec. 2008. https://jov.arvojournals.org/article.aspx?articleid=2297284
    [18]
    N. D. B. Bruce and J. K. Tsotsos, "Saliency, attention, and visual search: An information theoretic approach, " J. Vis., vol. 9, no. 3, pp. 1-24, Mar. 2009. https://jov.arvojournals.org/article.aspx?articleid=2193531
    [19]
    X. D. Hou and L. Q. Zhang, "Saliency detection: a spectral residual approach, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Minneapolis, USA, 2007, pp. 1-8.
    [20]
    D. K. Yang, "Object search and area resolution improved in large visual field based on biological eagle vision, " M.S. thesis, Beijing Institute of Technology, Beijing, China, 2016.
    [21]
    B. J. Frost, L. Z. Wise, B. Morgan, and D. Bird, "Retinotopic representation of the bifoveate eye of the kestrel (Falco sparverius) on the optic tectum, " Visual Neurosci., vol. 5, no. 3, pp. 231-239, Sep. 1990. https://www.cambridge.org/core/journals/visual-neuroscience/article/retinotopic-representation-of-the-bifoveate-eye-of-the-kestrel-falco-sparverius-on-the-optic-tectum/79C037E8675359DFCB246E1FB4777671
    [22]
    R. Shlaer, "An eagle's eye: quality of the retinal image, " Science, vol. 176, no. 4037, pp. 920-922, May 1972.
    [23]
    H. B. Duan, Y. M. Deng, X. H. Wang, and F. Liu, "Biological eagle-eye–based visual imaging guidance simulation platform for unmanned flying vehicles, " IEEE Aerosp. Electron. Syst. Mag., vol. 28, no. 12, pp. 36-45, Dec. 2013.
    [24]
    C. Feng, "The research of several vision bionics optical systems, " M.S. thesis, Beijing Institute of Technology, Beijing, China, 2015.
    [25]
    J. Orlowski, C. Beissel, F. Rohn, Y. Adato, H. Wagner, and O. Ben-Shaha, "Visual pop-out in barn owls: Human-like behavior in the avian brain, " J. Vis., vol. 15, no. 14, pp. 1-13, Oct. 2015. https://www.ncbi.nlm.nih.gov/pubmed/26448146
    [26]
    S. E. Allan and D. S. Blough, "Feature-based search asymmetries in pigeons and humans, " Percept. Psychophys., vol. 46, no. 5, pp. 456-464, Nov. 1989. https://www.ncbi.nlm.nih.gov/pubmed/2813030
    [27]
    M. J. Acerbo, O. F. Lazareva, J. McInnerney, E. Leiker, E. A. Wasserman, and A. Poremba, "Figure-ground discrimination in the avian brain: The nucleus rotundus and its inhibitory complex, " Vis. Res., vol. 70, pp. 18-26, Oct. 2012. https://www.ncbi.nlm.nih.gov/pubmed/22917681
    [28]
    Y. Zahar, H. Wagner, and Y. Gutfreund, "Responses of tectal neurons to contrasting stimuli: an electrophysiological study in the barn owl, " PLoS ONE, vol. 7, no. 6, pp. e39559, Jun. 2012. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3380014/
    [29]
    S. Ohayon, W. Harmening, H. Wagner, and E. Rivlin, "Through a barn owl's eyes: interactions between scene content and visual attention, " Biol. Cybern., vol. 98, no. 2, pp. 115-132, Feb. 2008. doi: 10.1007%2Fs00422-007-0199-4.pdf
    [30]
    S. P. Mysore, A. Asadollahi, and E. I. Knudsen, "Global inhibition and stimulus competition in the owl optic tectum, " J. Neurosci., vol. 30, no. 5, pp. 1727-1738, Feb. 2010.
    [31]
    Y. C. Wang and B. J. Frost, "Visual response characteristics of neurons in the nucleus isthmi magnocellularis and nucleus isthmi parvocellularis of pigeons, " Exp. Brain Res., vol. 87, no. 3, pp. 624-633, Dec. 1991. doi: 10.1007%2FBF00227087
    [32]
    W. J. Zhu, S. Liang, Y. C. Wei, and J. Sun, "Saliency optimization from robust background detection, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Columbus, USA, 2014, pp. 2814-2821.
    [33]
    F. Perazzi, P. Krähenbühl, Y. Pritch, and A. Hornung, "Saliency filters: Contrast based filtering for salient region detection, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Providence, USA, 2012, pp. 733-740.
    [34]
    Y. C. Wei, F. Wen, W. J. Zhu, and J. Sun, "Geodesic saliency using background priors, " in Proc. European Conf. Computer Vision, Florence, Italy, 2012, pp. 29-42.
    [35]
    X. D. Hou, J. Harel, and C. Koch, "Image signature: Highlighting sparse salient regions, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 1, pp. 194-201, Jan. 2012. https://ieeexplore.ieee.org/document/5963689/
    [36]
    W. Zheng, F. Zhou, and Z. F. Wang, "Robust and accurate monocular visual navigation combining IMU for a quadrotor, " IEEE/CAA J. Autom. Sinica, vol. 2, no. 1, pp. 33-44, Jan. 2015. http://www.ieee-jas.org/en/article/id/a1e6d5a0-cf9e-4d05-a35c-2494d0357ec8
    [37]
    N. F. Wen, L. L. Zhao, X. H. Su, and P. J. Ma, "UAV online path planning algorithm in a low altitude dangerous environment, " IEEE/CAA J. Autom. Sinica, vol. 2, no. 2, pp. 173-185, Apr. 2015. http://www.ieee-jas.org/en/article/id/61d41e9e-7928-41bf-a25f-dc95adbc5f43
    [38]
    Z. L. Wang, Q. Wang, and C. Y. Dong, "Asynchronous $H_\infty$ control for unmanned aerial vehicles: switched polytopic system approach, " IEEE/CAA J. Autom. Sinica, vol. 2, no. 2, pp. 207-216, Apr. 2015.
    [39]
    D. R. W. Wylie, C. Gutierrez-Ibanez, J. M. P. Pakan, and A. N. Iwaniuk, "The optic tectum of birds: mapping our way to understanding visual processing, " Can. J. Exp. Psychol., vol. 63, no. 4, pp. 328-338, Dec. 2009.
    [40]
    W. Woodson, T. Shimizu, J. M. Wild, J. Schimke, K. Cox, and H. J. Karten, "Centrifugal projections upon the retina: an anterograde tracing study in the pigeon (Columba livia), " J. Comparat. Neurol., vol. 362, no. 4, pp. 489-509, Nov. 1995. https://scholarcommons.usf.edu/psy_facpub/376/
    [41]
    J. M. Wild, M. F. Kubke, and J. L. Peña, "A pathway for predation in the brain of the barn owl (Tyto alba): projections of the gracile nucleus to the "claw area'' of the rostral wulst via the dorsal thalamus, " J. Comp. Neurol., vol. 509, no. 2, pp. 156-166, Jul. 2008. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2587498/
    [42]
    S. R. Wang, Y. C. Wang, and B. J. Frost, "Magnocellular and parvocellular divisions of pigeon nucleus isthmi differentially modulate visual responses in the tectum, " Exp. Brain Res., vol. 104, no. 3, pp. 376-384, Jun. 1995. doi: 10.1007/BF00231973
    [43]
    Y. C. Wang and B. J. Frost, "Visual response characteristics of neurons in the nucleus isthmi magnocellularis and nucleus isthmi parvocellularis of pigeons, " Exp. Brain Res., vol. 87, no. 3, pp. 624-633, Dec. 1991. doi: 10.1007%2FBF00227087
    [44]
    A. R. Koene and Z. P. Li, "Feature-specific interactions in salience from combined feature contrasts: evidence for a bottom-up saliency map in V1, " J. Vis., vol. 7, no. 7, pp. 1-14, May 2007. http://core.ac.uk/display/22930323
    [45]
    R. Sylvester, J. D. Haynes, and G. Rees, "Saccades differentially modulate human LGN and V1 responses in the presence and absence of visual stimulation, " Curr. Biol., vol. 15, no. 1, pp. 37-41, Jan. 2005. https://www.sciencedirect.com/science/article/pii/S0960982204010449
    [46]
    Y. B. Saalmann, I. N. Pigarev, and T. R. Vidyasagar, "Neural mechanisms of visual attention: how top-down feedback highlights relevant locations, " Science, vol. 316, no. 5831, pp. 1612-1615, Jun. 2007. http://science.sciencemag.org/content/316/5831/1612.full
    [47]
    H. J. Bischof and S. Watanabe, "On the structure and function of the tectofugal visual pathway in laterally eyed birds, " Eur. J. Morphol., vol. 35, no. 4, pp. 246-254, Oct. 1997. https://pub.uni-bielefeld.de/publication/1627601
    [48]
    B. A. Olshausen, and D. J. Field, " Sparse coding with an overcomplete basis set: A strategy employed by V1" Vis. Res., vol. 37, no. 23 pp. 3311-3325, Dec. 1997. https://www.sciencedirect.com/science/article/pii/S0042698997001697
    [49]
    L. Grady, "Random walks for image segmentation, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 11, pp. 1768-1783, Nov. 2006.
    [50]
    Q. Yan, L. Xu, J. P. Shi, and J. Y. Jia, "Hierarchical saliency detection, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Portland, USA, 2013, pp. 1155-1162.
    [51]
    J. P. Shi, Q. Yan, L. Xu, and J. Y. Jia, "Hierarchical image saliency detection on extended CSSD, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 4, pp. 717-729, Apr. 2016. http://ieeexplore.ieee.org/document/7182346/
    [52]
    M. M. Cheng, N. J. Mitra, X. L. Huang, P. H. S. Torr, and S. M. Hu, "Global contrast based salient region detection, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 3, pp. 569-582, Mar. 2015.
    [53]
    A. Borji and L. Itti, "State-of-the-art in visual attention modeling, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 185-207, Jan. 2013.
    [54]
    R. J. Peters, A. Iyer, L. Itti, and C. Koch, "Components of bottom-up gaze allocation in natural images, " Vis. Res., vol. 45, no. 18, pp. 2397-2416, Aug. 2005. https://www.sciencedirect.com/science/article/pii/S0042698905001975
    [55]
    U. Rajashekar, A. C. Bovik, and L. K. Cormack, "Visual search in noise: revealing the influence of structural cues by gaze-contingent classification image analysis, " J. Vis., vol. 6, no. 4, pp. 379-386, Mar. 2006. https://jov.arvojournals.org/article.aspx?articleid=2192970
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(13)  / Tables(6)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (1518) PDF downloads(50) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return