A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Advanced Search
Volume 4 Issue 4
Oct.  2017

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 15.3, Top 1 (SCI Q1)
    CiteScore: 23.5, Top 2% (Q1)
    Google Scholar h5-index: 77， TOP 5
Turn off MathJax
Article Contents
Article Navigation >  IEEE/CAA Journal of Automatica Sinica  > 2017  >  4(4): 639-650
Saugat Bhattacharyya, Amit Konar and D.N. Tibarewala, "Motor Imagery and Error Related Potential Induced Position Control of a Robotic Arm," IEEE/CAA J. Autom. Sinica, vol. 4, no. 4, pp. 639-650, Oct. 2017. doi: 10.1109/JAS.2017.7510616
Citation: Saugat Bhattacharyya, Amit Konar and D.N. Tibarewala, "Motor Imagery and Error Related Potential Induced Position Control of a Robotic Arm," IEEE/CAA J. Autom. Sinica, vol. 4, no. 4, pp. 639-650, Oct. 2017. doi: 10.1109/JAS.2017.7510616
shu

Motor Imagery and Error Related Potential Induced Position Control of a Robotic Arm

doi: 10.1109/JAS.2017.7510616
  • 1.

    BCI-LIFT Project Team, Institut National de Recherche en Informatique et en Automatique(INRIA), Montpellier 34015, France

  • 2.

    Artificial Intelligence Laboratory, ETCE Department, Jadavpur University, Kolkata 700032, India

  • 3.

    Centre for Healthcare Science and Technology, Indian Institute of Engineering Science and Technology, Howrah 711102, India

Funds:  This work was supported by UGC Sponsored UPE-Ⅱ Project in Cognitive Science of Jadavpur University, Kolkata
More Information
    Abstract
  • The paper introduces an electroencephalography (EEG) driven online position control scheme for a robot arm by utilizing motor imagery to activate and error related potential (ErrP) to stop the movement of the individual links, following a fixed (pre-defined) order of link selection. The right (left) hand motor imagery is used to turn a link clockwise (counterclockwise) and foot imagery is used to move a link forward. The occurrence of ErrP here indicates that the link under motion crosses the visually fixed target position, which usually is a plane/line/point depending on the desired transition of the link across 3D planes/around 2D lines/along 2D lines respectively. The imagined task about individual link's movement is decoded by a classifier into three possible class labels:clockwise, counterclockwise and no movement in case of rotational movements and forward, backward and no movement in case of translational movements. One additional classifier is required to detect the occurrence of the ErrP signal, elicited due to visually inspired positional link error with reference to a geometrically selected target position. Wavelet coefficients and adaptive autoregressive parameters are extracted as features for motor imagery and ErrP signals respectively. Support vector machine classifiers are used to decode motor imagination and ErrP with high classification accuracy above 80%. The average time taken by the proposed scheme to decode and execute control intentions for the complete movement of three links of a robot is approximately 33 seconds. The steady-state error and peak overshoot of the proposed controller are experimentally obtained as 1.1% and 4.6% respectively.

     

    • FullText(HTML)
  • loading
    • References(56)
  • [1]
    G. Dornhege, J. D. R. Millán, T. Hinterberger, D. J. McFarland, and K. R. Müller, Toward Brain-Computer Interfacing. Cambridge, Massachusetts, USA: MIT Press, 2007.
    [2]
    H. H. Alwasiti, I. Aris, and A. Jantan, "Brain computer interface design and applications: Challenges and future, " World Appl. Sci. J. , vol. 11, no. 7, pp. 819-825, Jan. 2010. http: //www. mendeley. com/research/brain-computer-interface-design-applications-challenges-future/
    [3]
    S. Sanei and J. A. Chambers, EEG Signal Processing. Chichester, UK: Wiley, 2007.
    [4]
    A. Nijholt, D. Tan, G. Pfurtscheller, C. Brunner, J. D. R. Millán, B. Allison, B. Graimann, F. Popescu, B. Blankertz, and K. R. Müller, "Brain-computer interfacing for intelligent systems, " IEEE Intell. Syst. , vol. 23, no. 3, pp. 72-79, May-Jun. 2008. http: //dl. acm. org/citation. cfm?id=1373158
    [5]
    M. D. Serruya, N. G. Hatsopoulos, L. Paninski, M. R. Fellows, and J. P. Donoghue, "Brain-machine interface: Instant neural control of a movement signal, " Nature, vol. 416, no. 6877, pp. 141-142, Mar. 2002. http: //scitation. aip. org/getabs/servlet/GetabsServlet?prog=normal & id=VIRT02000003000006000078000001 & idtype=cvips & gifs=Yes
    [6]
    S. M. Grigorescu, T. Lüth, C. Fragkopoulos, M. Cyriacks, and A. Gärser, "A BCI-controlled robotic assistant for quadriplegic people in domestic and professional life, " Robotica, vol. 30, no. 3, pp. 419-431, May 2012. http: //journals. cambridge. org/abstract_S0263574711000737
    [7]
    J. R. Millán, F. Renkens, J. Mourino, and W. Gerstner, "Noninvasive brain-actuated control of a mobile robot by human EEG, " IEEE Trans. Biomed. Eng. , vol. 51, no. 6, pp. 1026-1033, Jun. 2004. http: //europepmc. org/abstract/med/15188874
    [8]
    S. Bhattacharyya, A. Sengupta, T. Chakraborti, D. Banerjee, A. Khasnobish, A. Konar, D. N. Tibarewala, and R. Janarthanan, "EEG controlled remote robotic system from motor imagery classification, " in Proc. 3rd International Conference on Computing Communication & Networking Technologies (ICCCNT), Coimbatore, India, 2012, pp. 1-8. http: //ieeexplore. ieee. org/document/6395890/
    [9]
    Y. Chae, J. Jeong, and S. Jo, "Toward brain-actuated humanoid robots: Asynchronous direct control using an EEG-based BCI, " IEEE Trans. Robot. , vol. 28, no. 5, pp. 1131-1144, Oct. 2012. http: //ieeexplore. ieee. org/document/6214617/
    [10]
    C. J. Bell, P. Shenoy, R. Chalodhorn, and R. P. N. Rao, "Control of a humanoid robot by a noninvasive brain-computer interface in humans, " J. Neural Eng. , vol. 5, no. 2, pp. 214-220, May 2008. http: //www. europepmc. org/abstract/MED/18483450
    [11]
    J. Y. Long, Y. Q. Li, H. T. Wang, T. Y. Yu, J. H. Pan, and F. Li, "A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair, " IEEE Trans. Neural Syst. Rehab. Eng. , vol. 20, no. 5, pp. 720-729, Sep. 2012. http: //dl. acm. org/citation. cfm?id=1983237
    [12]
    F. Gálan, M. Nuttin, E. Lew, P. W. Ferrez, G. Vanacker, J. Philips, and J. D. R. Millán, "A brain-actuated wheelchair: Asynchronous and non-invasive brain-computer interfaces for continuous control of robots, " Clin. Neurophysiol. , vol. 119, no. 9, pp. 2159-2169, Sep. 2008. http: //www. ncbi. nlm. nih. gov/pubmed/18621580/
    [13]
    R. Scherer, F. Lee, A. Schlogl, R. Leeb, H. Bischof, and G. Pfurtscheller, "Toward self-paced brain-computer communication: Navigation through virtual worlds, " IEEE Trans. Biomed. Eng. , vol. 55, no. 2, pp. 675-682, Feb. 2008. http: //www. ncbi. nlm. nih. gov/pubmed/18270004
    [14]
    E. Curran, P. Sykacek, M. Stokes, S. J. Roberts, W. Penny, I. Johnsrude, and A. M. Owen, "Cognitive tasks for driving a brain-computer interfacing system: A pilot study, " IEEE Trans. Neural Syst. Rehab. Eng. , vol. 12, no. 1, pp. 48-54, Mar. 2004. http: //www. ncbi. nlm. nih. gov/pubmed/15068187
    [15]
    S. Bermudez I Badia, A. Garcia Morgade, H. Samaha, and P. F. M. J. Verschure, "Using a hybrid brain computer interface and virtual reality system to monitor and promote cortical reorganization through motor activity and motor imagery training, " IEEE Trans. Neural Syst. Rehab. Eng. , vol. 21, no. 2, pp. 174-181, Mar. 2013. http: //www. ncbi. nlm. nih. gov/pubmed/23204287
    [16]
    S. Bordoloi, U. Sharmah, and S. M. Hazarika, "Motor imagery based BCI for a maze game, " in Proc. 4th International Conference on Intelligent Human Computer Interaction (IHCI), Kharagpur, India, 2012, pp. 1-6. http: //ieeexplore. ieee. org/document/6481848/
    [17]
    J. K. Chapin, K. A. Moxon, R. S. Markowitz, and M. A. Nicolelis, "Realtime control of a robot arm using simultaneously recorded neurons in the motor cortex, " Nat. Neurosci. , vol. 2, no. 7, pp. 664-670, Jul. 1999. http: //www. springerlink. com/content/fulltext. pdf?id=doi: 10. 1007/978-1-4471-0765-1_34
    [18]
    J. Wessberg, C. R. Stambaugh, J. D. Kralik, P. D. Beck, M. Laubach, J. K. Chapin, J. Kim, S. J. Biggs, M. A. Srinivasan, and M. A. Nicolelis, "Real-time prediction of hand trajectory by ensembles of cortical neurons in primates, " Nature, vol. 408, no. 6810, pp. 361-365, Nov. 2000. http: //www. nature. com/nature/journal/v408/n6810/abs/408361a0. html
    [19]
    D. M. Taylor, S. I. H. Tillery, and A. B. Schwartz, "Direct cortical control of 3D neuroprosthetic devices, " Science, vol. 296, no. 5574, pp. 1892-1832, Jun. 2002. http: //www. ncbi. nlm. nih. gov/pubmed/12052948
    [20]
    S. Bhattacharyya, A. Konar, and D. N. Tibarewala, "Motor imagery, P300 and error-related EEG-based robot arm movement control for rehabilitation purpose, " Med. Biol. Eng. Comp. , vol. 52, no. 12, pp. 1007-1017, Dec. 2014. http: //www. ncbi. nlm. nih. gov/pubmed/25266261
    [21]
    G. Schalk, "Sensor modalities for brain-computer interfacing, " in Human-Computer Interaction. Novel Interaction Methods and Techniques, J. A. Jacko, Ed. Berlin, Heidelberg, Germany: Springer, 2009, pp. 616-622. http: //www. springerlink. com/openurl. asp?isbn=978-3-642-02576-1
    [22]
    S. G. Mason, A. Bashashati, M. Fatourechi, K. F. Navarro, and G. E. Birch, "A comprehensive survey of brain interface technology designs, " Ann. Biomed. Eng. , vol. 35, no. 2, pp. 137-169, Feb. 2007. http: //www. ncbi. nlm. nih. gov/pubmed/17115262
    [23]
    J. R. Millán, R. Rupp, G. R. Müller-Putz, R. Murray-Smith, C. Giugliemma, M. Tangermann, C. Vidaurre, F. Cincotti, A. Kübler, R. Leeb, C. Neuper, K. R. Müller, and D. Mattia, "Combining brain-computer interfaces and assistive technologies: State-of-the-art and challenges, " Front. Neurosci. , vol. 4, pp. 161, Sep. 2010. http: //www. ncbi. nlm. nih. gov/pmc/articles/PMC2944670/
    [24]
    M. Higger, M. Akcakaya, H. Nezamfar, G. LaMountain, U. Orhan, and D. Erdogmus, "A bayesian framework for intent detection and stimulation selection in SSVEP BCIs, " IEEE Signal Process. Lett. , vol. 22, no. 6, pp. 743-747, Jun. 2015. http: //ieeexplore. ieee. org/document/6951348/
    [25]
    T. Hinterberger, S. Schmidt, N. Neumann, J. Mellinger, B. Blankertz, G. Curio, and N. Birbaumer, "Brain-computer communication and slow cortical potentials, " IEEE Trans. Biomed. Eng. , vol. 51, no. 6, pp. 1011-1018, Jun. 2004. http: //ieeexplore. ieee. org/xpls/icp. jsp?arnumber=1300796
    [26]
    L. A. Farwell and E. Donchin, "Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials, " Electroencephalogr. Clin. Neurophysiol. , vol. 70, no. 6, pp. 510-523, Dec. 1988. http: //www. ncbi. nlm. nih. gov/pubmed/2461285
    [27]
    G. Pfurtscheller and F. H. Lopes da Silva, "Event-related EEG/MEG synchronization and desynchronization: Basic principles, " Clin. Neurophysiol. , vol. 110, no. 11, pp. 1842-1857, Nov. 1999. http: //www. ncbi. nlm. nih. gov/pubmed/10576479
    [28]
    Q. Chen, H. Peng, and H. Q. Feng, "Experiment study of the relation between motion complexity and event-related desynchronization/synchronization, " in Proc. 1st International Conference on Neural Interface and Control, Wuhan, China, 2005, pp. 14-16. http: //ieeexplore. ieee. org/xpls/icp. jsp?arnumber=1499831
    [29]
    A. Combaz, N. Chumerin, N. V. Manyakov, A. Robben, J. A. K. Suykens, and M. M. Van Hulle, "Error-related potential recorded by EEG in the context of a p300 mind speller brain-computer interface, " in Proc. 2010 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), Kittila, Finland, 2010, pp. 65-70. http: //ieeexplore. ieee. org/xpls/abs_all. jsp?arnumber=5589217
    [30]
    A. Combaz, N. Chumerin, N. V. Manyakov, A. Robben, J. A. K. Suykens, and M. M. Van Hulle, "Towards the detection of error-related potentials and its integration in the context of a P300 speller braincomputer interface, " Neurocomputing, vol. 80, pp. 73-82, Mar. 2012. http: //dl. acm. org/citation. cfm?id=2770442
    [31]
    P. W. Ferrez and J. D. R. Millán, "Simultaneous real-time detection of motor imagery and error-related potentials for improved BCI accuracy, " in Proc. 4th International Brain-Computer Interface Workshop and Training Course, Graz, Austria, 2008, pp. 197-202. https: //www. mendeley. com/research-papers/simultaneous-realtime-detection-motor-imagery-errorrelated-potentials-improved-bci-accuracy/
    [32]
    B. C. Kuo, Automatic Control Systems. 3rd ed. New Jersey, USA: Prentice-Hall Inc. , 1975.
    [33]
    URL for three videos: Position Ccontrol of a planar robot arm, one 3-Link configuration and one 4-Link configuration of the Jaco robot arm[Online]. Available: http://www.computationalintelligence.net/ResearchVideos/videos31may2016.html
    [34]
    L. Bougrain, O. Rochel, O. Boussaton, and L. Havet, "From the decoding of cortical activities to the control of a JACO robotic arm: a whole processing chain, " Control Architecture of Robots, 2012. http: //arxiv. org/abs/1212. 0083
    [35]
    K. J. Miller, G. Schalk, E. E. Fetz, M. den Nijs, J. G. Ojemann, and R. P. N. Rao, "Cortical activity during motor execution, motor imagery, and imagery-based online feedback, " Proc. Natl. Acad. Sci. USA. , vol. 107, no. 9, pp. 4430-4435, Mar. 2010. http: //www. ncbi. nlm. nih. gov/pubmed/20160084
    [36]
    R. Vocat, G. Pourtois, and P. Vuilleumier, "Unavoidable errors: a spatiotemporal analysis of time-course and neural sources of evoked potentials associated with error processing in a speeded task, " Neuropsychologia, vol. 46, no. 10, pp. 2545-2555, Aug. 2008. http: //europepmc. org/abstract/med/18533202
    [37]
    P. W. Ferrez and J. D. R. Millán, "EEG-based brain-computer inter-action: improved accuracy by automatic single-trial error detection, " in Proc. Advances in Neural Information Processing Systems, Barcelona, Spain, 2007, pp. 441-448. http: //www. mendeley. com/catalog/eegbased-braincomputer-interaction-improved-accuracy-automatic-singletrial-error-detection/
    [38]
    V. Krishnaveni, S. Jayaraman, S. Aravind, V. Hariharasudhan, and K. Ramadoss, "Automatic identification and removal of ocular artifacts from EEG using wavelet transform, " Measur. Sci. Rev. , vol. 6, no. 4, Nov. 2006. http: //citeseerx. ist. psu. edu/viewdoc/summary?doi=10. 1. 1. 99. 9641
    [39]
    H. Zeng, A. S. Song, R. Q. Yan, and H. Y. Qin, "EOG artifact correction from eeg recording using stationary subspace analysis and empirical mode decomposition, " Sensors, vol. 13, no. 11, pp. 14839-14859, Nov. 2013. http: //www. ncbi. nlm. nih. gov/pmc/articles/PMC3871096/
    [40]
    R. J. Croft and R. J. Barry, "Removal of ocular artifact from the EEG: a review, " Neurophysiol. Clin. , vol. 30, no. 1, pp. 5-19, Feb. 2000. http: //www. ncbi. nlm. nih. gov/pubmed/10740792
    [41]
    R. Palaniappan, "Brain computer interface design using band powers extracted during mental tasks, " in Proc. 2nd International IEEE EMBS Conference on Neural Engineering, Arlington, VA, USA, 2005, pp. 321-324. http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=1419622
    [42]
    M. D. J. Tran, C. P. Lim, C. Abeynayake, and L. C. Jain, "Feature extraction and classification of metal detector signals using the wavelet transform and the fuzzy ARTMAP neural network, " J. Int. Fuzzy Syst. , vol. 21, no. 1-2, pp. 89-99, Apr. 2010. http: //dl. acm. org/citation. cfm?id=1734988
    [43]
    S. Darvishi and A. Al-Ani, "Brain-computer interface analysis using continuous wavelet transform and adaptive neuro-fuzzy classifier, " in Proc. 29th Annual International Conference of the Engineering in Medicine and Biology Society, Lyon, France, 2007, pp. 3220-3223. http://www.academia.edu/11558434/Brain-computer_interface_analysis_using_continuous_wavelet_transform_and_adaptive_neuro-fuzzy_classifier
    [44]
    S. Bhattacharyya, P. Rakshit, A. Konar, D. N. Tibarewala, and R. Janarthanan, "Feature selection of motor imagery EEG signals using firefly temporal difference Q-learning and support vector machine, " in in Swarm, Evolutionary, and Memetic Computing. Lecture Notes in Computer Science, B. Panigrahi, P. N. Suganthan, S. Das, and S. S. Dash, Eds. Cham, Switzerland:Springer, 2013, pp. 534-545. doi: 10.1007/978-3-319-03756-1_48
    [45]
    M. A. Hall, "Correlation-based feature selection for discrete and numeric class machine learning, " in Proc. 7th International Conference on Machine Learning, San Francisco, CA, USA, 2000, pp. 359-366.
    [46]
    R. Kamei and A. L. Ralescu, "Piecewise linear separability using support vector machines, " in Proc. 14th Midwest Artificial Intelligence and Cognitive Sciences Conference, Cincinnati, OH, USA, 2003, pp. 52-56.
    [47]
    D. Srinivasan, R. J. Howlett, I. Lovrek, L. C. Jain, and C. P. Lim, "Design and application of neural networks and intelligent learning systems, " Neurocomputing, vol. 73, no. 4-6, pp. 591-592, Jan. 2010. http: //en. cnki. com. cn/Article_en/CJFDTOTAL-LYSZ199802026. htm
    [48]
    C. S. Fang, J. Storrs, A. Ralescu, J. H. Lee, and J. Lu, "Detecting Parkinson's brain changes using local feature based regional SVM ensemble on MRI images, " in Human Brain Mapping 2011, Quebec, Canada, 2011.
    [49]
    D. J. Sebald and J. A. Bucklew, "Support vector machine techniques for nonlinear equalization, " IEEE Trans. Signal Process. , vol. 48, no. 11, pp. 3217-3226, Nov. 2000. http: //ieeexplore. ieee. org/xpls/abs_all. jsp?arnumber=875477
    [50]
    G. S. Dharwarkar and O. Basir, "Enhancing temporal classification of AAR parameters in EEG single-trial analysis for brain-computer interfacing, " in Proc. 27th Annual International Conference of Engineering in Medicine and Biology Society, Shanghai, China, 2006, pp. 5358-5361. http: //www. ncbi. nlm. nih. gov/pubmed/17281462
    [51]
    M. Pal, S. Bhattacharyya, A. Konar, D. N. Tibarewala, and R. Janarthanan, "Decoding of wrist and finger movement from electroencephalography signal, " in Proc. 2014 IEEE International Conference on Electronics, Computing and Communication Technologies (IEEE CONECCT), Bangalore, India, 2014, pp. 1-6. http: //ieeexplore. ieee. org/document/6740323/
    [52]
    E. Alpaydin, Introduction to Machine Learning. Cambridge, MA, USA: MIT Press, 2004.
    [53]
    B. Graimann, B. Allison, and G. Pfurtscheller (Eds. ), Brain-Computer Interfaces: Revolutionizing Human-Computer Interaction, Springer, 2010.
    [54]
    G. Pfurtscheller, B. Z. Allison, C. Brunner, G. Bauernfeind, T. SolisEscalante, R. Scherer, T. O. Zander, G. Mueller-Putz, C. Neuper, and N. Birbaumer, "The hybrid BCI, Front. Neurosci. , vol. 4, pp. 42, Apr. 2010.
    [55]
    B. G. Xu, S. Peng, A. G. Song, R. H. Yang, and L. Z. Pan, "Robotaided upper-limb rehabilitation based on motor imagery EEG, " Int. J. Adv. Robot. Syst. , vol. 8, no. 4, pp. 88-97, Jan. 2011. http: //www. oalib. com/paper/2576028
    [56]
    A. Khasnobish, A. Konar, D. N. Tibarewala, and A. K. Nagar, "Bypassing the natural visual-motor pathway to execute complex movement related tasks using interval type-2 fuzzy sets, " IEEE Trans. Neural Syst. Rehab. Eng. , vol. 25, no. 1, pp. 91-105, Jan. 2017. http: //www. ncbi. nlm. nih. gov/pubmed/27323367
    • Related Articles
    • Supplements(0)

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(8)  / Tables(8)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (1557) PDF downloads(93) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return