A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 7 Issue 4
Jun.  2020

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2020  >  7(4): 1038-1044
Yaojie Zhang, Bing Xu and Tiejun Zhao, "Convolutional Multi-Head Self-Attention on Memory for Aspect Sentiment Classification," IEEE/CAA J. Autom. Sinica, vol. 7, no. 4, pp. 1038-1044, July 2020. doi: 10.1109/JAS.2020.1003243
Citation: Yaojie Zhang, Bing Xu and Tiejun Zhao, "Convolutional Multi-Head Self-Attention on Memory for Aspect Sentiment Classification," IEEE/CAA J. Autom. Sinica, vol. 7, no. 4, pp. 1038-1044, July 2020. doi: 10.1109/JAS.2020.1003243
shu

Convolutional Multi-Head Self-Attention on Memory for Aspect Sentiment Classification

doi: 10.1109/JAS.2020.1003243
Funds:  This work was supported by the National Key Research and Development Program of China (2018YFC0830700)
More Information
    Abstract
  • This paper presents a method for aspect based sentiment classification tasks, named convolutional multi-head self-attention memory network (CMA-MemNet). This is an improved model based on memory networks, and makes it possible to extract more rich and complex semantic information from sequences and aspects. In order to fix the memory network’s inability to capture context-related information on a word-level, we propose utilizing convolution to capture n-gram grammatical information. We use multi-head self-attention to make up for the problem where the memory network ignores the semantic information of the sequence itself. Meanwhile, unlike most recurrent neural network (RNN) long short term memory (LSTM), gated recurrent unit (GRU) models, we retain the parallelism of the network. We experiment on the open datasets SemEval-2014 Task 4 and SemEval-2016 Task 6. Compared with some popular baseline methods, our model performs excellently.

     

    • FullText(HTML)
  • loading
    • 1 http://alt.qcri.org/semeval2014/task4/
    2 http://alt.qcri.org/semeval2016/task6/
    3 http://nlp.stanford.edu/projects/glove/
    References(25)
  • [1]
    Liu B and Zhang L, A Survey of Opinion Mining and Sentiment Analysis. Boston, MA: Springer US, 2012, pp. 415–463.
    [2]
    Pang B and Lee L, “Opinion mining and sentiment analysis,” Foundations and Trends. in Information Retrieval, vol. 2, no. 1–2, pp. 1–135, 2008.
    [3]
    M. Pontiki, D. Galanis, J. Pavlopoulos, H. Papageorgiou, I. Androutsopoulos, and S. Manandhar, “Semeval-2014 task 4: Aspect based sentiment analysis,” in Proc. 8th Int. Workshop on Semantic Evaluation, SemEval@COLING, Dublin, Ireland, Aug. 2014, pp. 27–35.
    [4]
    M. Pontiki, D. Galanis, H. Papageorgiou, S. Manandhar, and I. Androutsopoulos, “Semeval-2015 task 12: Aspect based sentiment analysis,” in Proc. 9th Int. Workshop on Semantic Evaluation, SemEval@NAACL-HLT, Denver, Colorado, USA, Jun. 2015, pp. 486–495.
    [5]
    M. Pontiki, D. Galanis, H. Papageorgiou, I. Androutsopoulos, S. Manandhar, M. Al-Smadi, M. Al-Ayyoub, Y. Y. Zhao, B. Qin, O. D. Clercq, V. Hoste, M. Apidianaki, X. Tannier, N. V. Loukachevitch, E. V. Kotelnikov, N. Bel, S. M. J. Zafra, and G. C. Eryigit, “Semeval-2016 task 5: Aspect based sentiment analysis,” in Proc. 10th Int. Workshop on Semantic Evaluation, SemEval@NAACLHLT, San Diego, CA, USA, Jun. 2016, pp. 19–30.
    [6]
    L. Shu, H. Xu, and B. Liu, “Lifelong learning CRF for supervised aspect extraction,” in Proc. 55th Annual Meeting of the Association for Computational Linguistics, ACL, Vancouver, Canada: vol. 2, 2017, pp. 148–154.
    [7]
    Y. Q. Wang, M. Huang, X. Y. Zhu, and L. Zhao, “Attention-based LSTM for aspect-level sentiment classification,” in Proc. Conf. Empirical Methods in Natural Language Processing, EMNLP, Austin, Texas, USA, Nov. 2016, pp. 606–615.
    [8]
    H. H. Do, P. W. C. Prasad, A. Maag, and A. Alsadoon, “Deep learning for aspect-based sentiment analysis: A comparative review,” Expert Syst. Appl., vol. 118, pp. 272–299, 2019. doi: 10.1016/j.eswa.2018.10.003
    [9]
    W. Xue and T. Li, “Aspect based sentiment analysis with gated convolutional networks,” in Proc. 56th Annual Meeting of the Association for Computational Linguistics, ACL, Melbourne, Australia: Association for Computational Linguistics, Jul. 2018, pp. 2514–2523.
    [10]
    X. W. Ding, B. Liu, and P. S. Yu, “A holistic lexicon-based approach to opinion mining,” in Proc. Int. Conf. Web Search and Web Data Mining, WSDM, Palo Alto, California, USA, Feb. 2008, pp. 231–240.
    [11]
    W. X. Zhao, J. Jiang, H. F. Yan, and X. M. Li, “Jointly modeling aspects and opinions with a maxent-lda hybrid,” in Proc. Conf. Empirical Methods in Natural Language Processing, EMNLP, MIT Stata Center, Massachusetts, USA, 2010, pp. 56–65.
    [12]
    S. Kiritchenko, X. D. Zhu, C. Cherry, and S. Mohammad, “NRC-Canada-2014: Detecting aspects and sentiment in customer reviews,” in Proc. 8th Int. Workshop on Semantic Evaluation, SemEval@COLING, Dublin, Ireland, Aug. 2014, pp. 437–442.
    [13]
    D. Y. Tang, B. Qin, X. C. Feng, and T. Liu, “Target-dependent sentiment classification with long short term memory,” CoRR, vol. abs/1512.01100, 2015.
    [14]
    D. Y. Tang, B. Qin, and T. Liu, “Aspect level sentiment classification with deep memory network,” in Proc. Conf. Empirical Methods in Natural Language Processing, EMNLP, Austin, Texas, USA, Nov. 2016, pp. 214–224.
    [15]
    P. Chen, Z. Q. Sun, L. D. Bing, and W. Yang, “Recurrent attention network on memory for aspect sentiment analysis,” in Proc. Conf. Empirical Methods in Natural Language Processing, EMNLP, Copenhagen, Denmark, Sep. 2017, pp. 452–461.
    [16]
    B. T. Do, “Aspect-based sentiment analysis using bitmask bidirectional long short term memory networks,” in Proc. 31st Int. Florida Artificial Intelligence Research Society Conf., FLAIRS, Melbourne, Florida, USA. May 2018, pp. 259–264.
    [17]
    N. Majumder, S. Poria, A. F. Gelbukh, M. S. Akhtar, E. Cambria, and A. Ekbal, “IARM: inter-aspect relation modeling with memory networks in aspect-based sentiment analysis,” in Proc. Conf. Empirical Methods in Natural Language Processing, Brussels, Belgium, Oct. – Nov. 2018, pp. 3402–3411.
    [18]
    P. S. Zhu and T. Y. Qian, “Enhanced aspect level sentiment classification with auxiliary memory,” in Proc. 27th Int. Conf. Computational Linguistics, COLING, Santa Fe, New Mexico, USA, Aug. 2018, pp. 1077–1087.
    [19]
    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. Advances in Neural Information Processing Systems 30: Annual Conf. Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 6000–6010.
    [20]
    S. Gao, A. Ramanathan, and G. D. Tourassi, “Hierarchical convolutional attention networks for text classification,” in Proc. 3rd Workshop on Representation Learning for NLP, Rep4NLP@ACL, Melbourne, Australia, Jul. 2018, pp. 11–23.
    [21]
    S. Mohammad, S. Kiritchenko, P. Sobhani, X. D. Zhu, and C. Cherry, “Semeval-2016 task 6: Detecting stance in tweets,” in Proc. 10th Int. Workshop on Semantic Evaluation, SemEval@NAACL-HLT, San Diego, CA, USA, Jun. 2016, pp. 31–41.
    [22]
    J. Pennington, R. Socher, and C. D. Manning, “Glove: Global vectors for word representation,” in Proc. Conf. Empirical Methods in Natural Language Processing, EMNLP, Doha, Qatar, 2014, pp. 1532–1543.
    [23]
    S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997. doi: 10.1162/neco.1997.9.8.1735
    [24]
    D. H. Ma, S. J. Li, X. D. Zhang, and H. F. Wang, “Interactive attention networks for aspect-level sentiment classification,” in Proc. 26th Int. Joint Conf. Artificial Intelligence, IJCAI, Melbourne, Australia, Aug. 2017, pp. 4068–4074.
    [25]
    X. Y. Wang, G. L. Xu, J. Y. Zhang, X. W. Sun, L. Wang, and T. L. Huang, “Syntaxdirected hybrid attention network for aspect-level sentiment analysis,” IEEE Access, vol. 7, pp. 5014–5025, 2019. doi: 10.1109/ACCESS.2018.2885032
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(4)  / Tables(5)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (1174) PDF downloads(93) Cited by()

    Highlights

    • Using convolution and self-attention to capture semantic information of n-gram and sequence itself.
    • The aspect-sequence modeling ability and network parallelism of memory network are preserved.
    • Can complete ACSA and ATSA tasks and win in baseline.

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return