research-article

Music and natural sounds in an auditory steady-state response based braincomputer interface to increase user acceptance

Authors:

Seunghyeok Hong,

Jeong Su Lee, and

Kwang Suk ParkAuthors Info & Claims

Computers in Biology and Medicine, Volume 84, Issue C

Pages 45 - 52

https://doi.org/10.1016/j.compbiomed.2017.03.011

Published: 01 May 2017 Publication History

Abstract

Patients with total locked-in syndrome are conscious; however, they cannot express themselves because most of their voluntary muscles are paralyzed, and many of these patients have lost their eyesight. To improve the quality of life of these patients, there is an increasing need for communication-supporting technologies that leverage the remaining senses of the patient along with physiological signals. The auditory steady-state response (ASSR) is an electro-physiologic response to auditory stimulation that is amplitude-modulated by a specific frequency. By leveraging the phenomenon whereby ASSR is modulated by mind concentration, a braincomputer interface paradigm was proposed to classify the selective attention of the patient. In this paper, we propose an auditory stimulation method to minimize auditory stress by replacing the monotone carrier with familiar music and natural sounds for an ergonomic system. Piano and violin instrumentals were employed in the music sessions; the sounds of water streaming and cicadas singing were used in the natural sound sessions. Six healthy subjects participated in the experiment. Electroencephalograms were recorded using four electrodes (Cz, Oz, T7 and T8). Seven sessions were performed using different stimuli. The spectral power at 38 and 42Hz and their ratio for each electrode were extracted as features. Linear discriminant analysis was utilized to classify the selections for each subject. In offline analysis, the average classification accuracies with a modulation index of 1.0 were 89.67% and 87.67% using music and natural sounds, respectively. In online experiments, the average classification accuracies were 88.3% and 80.0% using music and natural sounds, respectively. Using the proposed method, we obtained significantly higher user-acceptance scores, while maintaining a high average classification accuracy.

References

[1]

B. Allison, T. Luth, D. Valbuena, A. Teymourian, I. Volosyak, A. Graser, BCI demographics: how many (and what kinds of) people can use an SSVEP BCI?, IEEE Trans. Neural Syst. Rehabil. Eng., 18 (2010) 107-116.

[2]

H.J. Baek, H.S. Kim, J. Heo, Y.G. Lim, K.S. Park, Braincomputer interfaces using capacitive measurement of visual or auditory steady-state responses, J. Neural Eng., 10 (2013) 024001.

[3]

G. Bauer, F. Gerstenbrand, E. Rumpl, Varieties of the locked-in syndrome, J. Neurol., 221 (1979) 77-91.

[4]

D.L. Beck, D. Speidel, M. Petrak, Auditory steady-state response (ASSR): a beginner's guide, Hear. Rev., 14 (2007) 34.

[5]

A. Calvo, A. Chi, E. Castellina, F. Corno, L. Farinetti, P. Ghiglione, V. Pasian, A. Vignola, Eye tracking impact on quality-of-life of ALS patients, in: Proceedings of the International Conference on Computers for Handicapped Persons, Springer, 2008, pp. 7077.

Digital Library

[6]

J.L. Collinger, B. Wodlinger, J.E. Downey, W. Wang, E.C. Tyler-Kabara, D.J. Weber, A.J. Mcmorland, M. Velliste, M.L. Boninger, A.B. Schwartz, High-performance neuroprosthetic control by an individual with tetraplegia, Lancet (2012).

[7]

M. De Vos, K. Gandras, S. Debener, Towards a truly mobile auditory braincomputer interface: exploring the P300 to take away, Int. J. Psychophysiol., 91 (2014) 46-53.

[8]

L.A. Farwell, E. Donchin, Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials, Electroencephalogr. Clin. Neurophysiol., 70 (1988) 510-523.

[9]

J. Hhne, K. Krenzlin, S. Dhne, M. Tangermann, Natural stimuli improve auditory BCIs with respect to ergonomics and performance, J. Neural Eng., 9 (2012) 045003.

[10]

N. Hill, B. Schlkopf, An online braincomputer interface based on shifting attention to concurrent streams of auditory stimuli, J. Neural Eng., 9 (2012) 026011.

[11]

N.J. Hill, A. Moinuddin, A.-K. Huser, S. Kienzle, G. Schalk, Communication and control by listening: toward optimal design of a two-class auditory streaming brain-computer interface, Front. Neurosci., 6 (2012).

[12]

H.-J. Hwang, J.-H. Lim, Y.-J. Jung, H. Choi, S.W. Lee, C.-H. Im, Development of an SSVEP-based BCI spelling system adopting a QWERTY-style LED keyboard, J. Neurosci. Methods, 208 (2012) 59-65.

[13]

S. Ikegami, K. Takano, N. Saeki, K. Kansaku, Operation of a P300-based braincomputer interface by individuals with cervical spinal cord injury, Clin. Neurophysiol., 122 (2011) 991-996.

[14]

A. Kbler, A. Furdea, S. Halder, E.M. Hammer, F. Nijboer, B. Kotchoubey, A braincomputer interface controlled auditory eventrelated potential (P300) spelling system for lockedin patients, Ann. N. Y. Acad. Sci., 1157 (2009) 90-100.

[15]

A. Kbler, N. Neumann, J. Kaiser, B. Kotchoubey, T. Hinterberger, N.P. Birbaumer, Brain-computer communication: self-regulation of slow cortical potentials for verbal communication, Arch. Phys. Med. Rehabil., 82 (2001) 1533-1539.

[16]

T. Kaufmann, S. Schulz, C. Grnzinger, A. Kbler, Flashing characters with famous faces improves ERP-based braincomputer interface performance, J. Neural Eng., 8 (2011) 056016.

[17]

D.-W. Kim, H.-J. Hwang, J.-H. Lim, Y.-H. Lee, K.-Y. Jung, C.-H. Im, Classification of selective attention to auditory stimuli: toward vision-free braincomputer interfacing, J. Neurosci. Methods, 197 (2011) 180-185.

[18]

E.C. Lalor, S.P. Kelly, C. Finucane, R. Burke, R. Smith, R.B. Reilly, G. Mcdarby, Steady-state VEP-based brain-computer interface control in an immersive 3D gaming environment, EURASIP J. Appl. Signal Process., 2005 (2005) 3156-3164.

Digital Library

[19]

J.S. Lee, C.M. Han, J.H. Kim, K.S. Park, Reverse-curve-arch-shaped dry EEG electrode for increased skinelectrode contact area on hairy scalps, Electron. Lett., 51 (2015) 1643-1645.

[20]

F.-C. Lin, J.K. Zao, K.-C. Tu, Y. Wang, Y.-P. Huang, C.-W. Chuang, H.-Y. Kuo, Y.-Y. Chien, C.-C. Chou, T.-P. Jung, SNR analysis of high-frequency steady-state visual evoked potentials from the foveal and extrafoveal regions of human retina, in: Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2012, pp. 18101814.

[21]

M. Lopez-Gordo, E. Fernandez, S. Romero, F. Pelayo, A. Prieto, An auditory braincomputer interface evoked by natural speech, J. Neural Eng., 9 (2012) 036013.

[22]

M.-A. Lopez, H. Pomares, F. Pelayo, J. Urquiza, J. Perez, Evidences of cognitive effects over auditory steady-state responses by means of artificial neural networks and its use in braincomputer interfaces, Neurocomputing, 72 (2009) 3617-3623.

Digital Library

[23]

Z.R. Lugo, J. Rodriguez, A. Lechner, R. Ortner, I.S. Gantner, S. Laureys, Q. Noirhomme, C. Guger, A vibrotactile p300-based braincomputer interface for consciousness detection and communication, Clin. EEGNeurosci. (2014).

[24]

D. Lul, Q. Noirhomme, S.C. Kleih, C. Chatelle, S. Halder, A. Demertzi, M.-A. Bruno, O. Gosseries, A. Vanhaudenhuyse, C. Schnakers, Probing command following in patients with disorders of consciousness using a braincomputer interface, Clin. Neurophysiol., 124 (2013) 101-106.

[25]

T. Nakamura, H. Namba, T. Matsumoto, Classification of auditory steady-state responses to speech data, in: Proceedings of the 6th International IEEE/EMBS Conference on Neural Engineering (NER),IEEE, 2013, pp. 10251028.

[26]

H. Namba, T. Nakamura, T. Matsumoto, Classification of Auditory Steady-State Responses Incorporating Alpha Waves, BCI meeting, 2013.

[27]

F. Nijboer, E. Sellers, J. Mellinger, M. Jordan, T. Matuz, A. Furdea, S. Halder, U. Mochty, D. Krusienski, T. Vaughan, A P300-based braincomputer interface for people with amyotrophic lateral sclerosis, Clin. Neurophysiol., 119 (2008) 1909-1916.

[28]

G. Pfurtscheller, F.L. Da Silva, Event-related EEG/MEG synchronization and desynchronization: basic principles, Clin. Neurophysiol., 110 (1999) 1842-1857.

[29]

T.W. Picton, M.S. John, A. Dimitrijevic, D. Purcell, Human auditory steady-state responses: respuestas auditivas de estado estable en humanos, Int. J. Audiol., 42 (2003) 177-219.

[30]

C. Pokorny, D.S. Klobassa, G. Pichler, H. Erlbeck, R.G. Real, A. Kbler, D. Lesenfants, D. Habbal, Q. Noirhomme, M. Risetti, The auditory P300-based single-switch braincomputer interface: paradigm transition from healthy subjects to minimally conscious patients, Artif. Intell. Med., 59 (2013) 81-90.

Digital Library

[31]

G. Rance, The Auditory Steady-State Response: Generation, Recording, and Clinical Application, Plural Publishing, 2008.

[32]

M. Schreuder, B. Blankertz, M. Tangermann, A new auditory multi-class brain-computer interface paradigm: spatial hearing as an informative cue, PLoS One, 5 (2010) e9813.

[33]

M. Schreuder, A. Riccio, M. Risetti, S. Dhne, A. Ramsay, J. Williamson, D. Mattia, M. Tangermann, User-centered design in braincomputer interfacesA case study, Artif. Intell. Med., 59 (2013) 71-80.

Digital Library

[34]

E.W. Sellers, E. Donchin, A P300-based braincomputer interface: initial tests by ALS patients, Clin. Neurophysiol., 117 (2006) 538-548.

[35]

N. Simon, I. Kthner, C.A. Ruf, E. Pasqualotto, A. Kbler, S. Halder, An auditory multiclass brain-computer interface with natural stimuli: usability evaluation with healthy participants and a motor impaired end user, Front. Hum. Neurosci., 8 (2015) 1039.

[36]

E. Smith, M. Delargy, Locked-in syndrome, BMJ:Br. Med. J., 330 (2005) 406.

[37]

C. Vidaurre, C. Sannelli, K.-R. Mller, B. Blankertz, Co-adaptive calibration to improve BCI efficiency, J. Neural Eng., 8 (2011) 025009.

[38]

J.R. Wolpaw, N. Birbaumer, W.J. Heetderks, D.J. Mcfarland, P.H. Peckham, G. Schalk, E. Donchin, L.A. Quatrano, C.J. Robinson, T.M. Vaughan, Brain-computer interface technology: a review of the first international meeting, IEEETrans. Rehabil. Eng., 8 (2000) 164-173.

[39]

J. Xiao, Q. Xie, Y. He, T. Yu, S. Lu, N. Huang, R. Yu, Y. Li, An auditory BCI system for assisting CRS-R behavioral assessment in patients with disorders of consciousness, Sci. Rep., 6 (2016) 32917.

Cited By

Zhang HXie JXiao YCui GZhu XXu GTao QYang YRen ZLi M(2024)The effects of synchronous and asynchronous steady-state auditory-visual motion on EEG characteristics in healthy young adultsExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122640241:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122640
Chaki J(2021)Pattern analysis based acoustic signal processing: a survey of the state-of-artInternational Journal of Speech Technology10.1007/s10772-020-09681-324:4(913-955)Online publication date: 1-Dec-2021
https://dl.acm.org/doi/10.1007/s10772-020-09681-3
Sun KHsieh KSyu S(2020)Towards an Accessible Use of a Brain-Computer Interfaces-Based Home Care System through a SmartphoneComputational Intelligence and Neuroscience10.1155/2020/18432692020Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.1155/2020/1843269
Show More Cited By

Recommendations

A system for recording of auditory evoked responses

A system for recording of evoked potentials from auditory stimulation was developed. The system consists of a PC equipped with an audio bandwidth board with analog input and output channels. The sound stimulus signal is generated in the computer, D/A ...
Read More
A hybrid steady-state visual evoked response-based brain-computer interface with MEG and EEG
Abstract
While recent developments in electroencephalogram (EEG)-based brain-computer interfaces (BCIs) have enabled a bridge between the brain and external devices with relatively high communication speed, there is still room for improvement. Notably, ...
Read More
Age-related differences in SSVEP-based BCI performance

BrainComputer Interface (BCI) systems analyze brain signals to generate control commands for computer applications or external devices. Utilized as alternative communication channel, BCIs have the potential to assist people with severe motor ...
Read More

Comments

Information & Contributors

Information

Published In

cover image Computers in Biology and Medicine

Computers in Biology and Medicine Volume 84, Issue C

May 2017

262 pages

ISSN:0010-4825

Issue’s Table of Contents

Copyright © Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 May 2017

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Zhang HXie JXiao YCui GZhu XXu GTao QYang YRen ZLi M(2024)The effects of synchronous and asynchronous steady-state auditory-visual motion on EEG characteristics in healthy young adultsExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122640241:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122640
Chaki J(2021)Pattern analysis based acoustic signal processing: a survey of the state-of-artInternational Journal of Speech Technology10.1007/s10772-020-09681-324:4(913-955)Online publication date: 1-Dec-2021
https://dl.acm.org/doi/10.1007/s10772-020-09681-3
Sun KHsieh KSyu S(2020)Towards an Accessible Use of a Brain-Computer Interfaces-Based Home Care System through a SmartphoneComputational Intelligence and Neuroscience10.1155/2020/18432692020Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.1155/2020/1843269
Baek HChang MHeo JPark K(2019)Enhancing the Usability of Brain-Computer Interface SystemsComputational Intelligence and Neuroscience10.1155/2019/54271542019Online publication date: 16-Jun-2019
https://dl.acm.org/doi/10.1155/2019/5427154
Bablani AEdla DTripathi DCheruku R(2019)Survey on Brain-Computer InterfaceACM Computing Surveys10.1145/329771352:1(1-32)Online publication date: 13-Feb-2019
https://dl.acm.org/doi/10.1145/3297713

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents