This is the html version of the file https://www.taylorfrancis.com/chapters/edit/10.4324/9781003190912-8/using-non-invasive-brain-stimulation-investigate-second-language-nick-pand%C5%BEa.
Google automatically generates html versions of documents as we crawl the web.
Page 1
72
DOI: 10.4324/9781003190912-8
6
USING NON- INVASIVE BRAIN
STIMULATION TO INVESTIGATE
SECOND LANGUAGE
Nick B. Pandža
Introduction
Learning a second language (L2) is difficult for adults, which is unsurprising due to the cognitive
demands it places on multiple memory systems, attentional mechanisms, and perceptual abilities.
These cognitive processes have commonly been targeted with behavioral training paradigms in
attempts to increase L2 learning rate and retention (e.g., Ingvalson et al., 2014). Stimulation of the
nervous system, or neurostimulation, is a category of methods that modulate neural activity and,
with the advent of non- invasive techniques, can also be used to investigate and enhance L2 learning
and processing. The two primary uses of neurostimulation to investigate L2 are through its ability
to localize behavior to a particular brain area or function and, in a more applied context, its ability
to actively modulate brain function to positively affect cognition and behavior, including language
learning outcomes.
A major distinction between neurostimulation and other methodological approaches addressed in
this volume (electroencephalography (EEG), see Dickson & Pelzl, this volume, and Mottarella & Prat
et al., this volume; magnetic resonance imaging (MRI), see Kousaie & Klein, this volume, and Rossi
et al., this volume) is that neurostimulation is inherently a neurocognitive intervention rather than a
neurocognitive measure. This paradigm shift is a fascinating new direction for neurolinguistics in
second language acquisition (SLA) as it allows us to leverage what we know about the neurocognition
of SLA, much of it foundationally built on observational neurocognitive methods, to causally affect
language learning. Compared to those other neurocognitive methods, neurostimulation research is
nascent and presently under- represented in research on L2 learning and processing.
Historically, neurostimulation has been a set of invasive and/ or sometimes painful techniques
developed for clinical applications, but painless, non- invasive ways to modulate neural activity have
been developed in recent decades using electric current and magnetic fields on the outside of the body.
This chapter reviews multiple methods of non- invasive brain stimulation (NIBS), including
transcranial magnetic stimulation (TMS), transcranial electrical stimulation (tES), and transcutaneous
peripheral nerve stimulation (PNS). In reality, these three are broad families of methods under which
differences in, for example, timing, intensity, stimulation pattern, or electrode placement can not
only affect the strength of any effects but even change the underlying mechanism of action (Polan�a
et al., 2018). Following a brief history of non- invasive neurostimulation methods to examine cogni-
tion, their use and potential in the investigation of L2 learning and processing is discussed, including

Page 2
Non-Invasive Brain Stimulation to Investigate Second Language
73
experimental paradigms, example studies, and advantages and disadvantages of specific techniques.
The chapter ends with a discussion of future directions for neurostimulation methods in the investi-
gation of L2.
Critical Definitions
Neurostimulation involves the application of stimulation (e.g., electrical, magnetic) to modulate the
activity of the nervous system. There are a variety of non- invasive techniques that target different
neurocognitive mechanisms, many of which support language learning. TMS and tES, being
transcranial, involve placing neurostimulators (e.g., electrodes) on or above the surface of the scalp
in order to affect cortical activity and have been evaluated for improving cognitive and language per-
formance (e.g., Miniussi et al., 2008). PNS, being stimulation of a peripheral cranial nerve, involves
the placement of electrodes on the surface of the skin in strategic locations such as the ear, neck, or
even forehead to electrically stimulate branches of cranial nerves to carry the stimulation back to
nuclei in the brainstem and facilitate the release of neurotransmitters with the intent to affect cogni-
tion and language (e.g., Colzato & Beste, 2020).
TMS uses a strong magnetic field produced by a coil, typically a combination of two circular coils
to optimize spatial resolution. These magnetic fields penetrate through the scalp and skull under the
coil’s position to the cortex and induce electrical fields that can stimulate neuronal activity (Sandrini
et al., 2011). A pulse of current can temporarily disrupt neural activity, and TMS has been used to
simulate lesions to localize the brain regions necessary for a given task, including several regions
necessary for language processing (Pascual- Leone et al., 2000). TMS provides a high degree of
accuracy in identifying where task- critical regions are in the brain (i.e., spatial localization), espe-
cially when combined with structural MRI (see Rossi et al., this volume). TMS pulses can also be
repeated over an extended period of time (repetitive TMS; rTMS) to facilitate or inhibit neural activity
(Miniussi et al., 2008). It has been used in people with aphasia to promote better language recovery
(e.g., Miniussi et al., 2008), and healthy individuals to facilitate picture naming and other language
tasks (e.g., Mottaghy et al., 1999). The potential mechanisms of action for TMS are still under active
investigation, and while there is some evidence it can elevate gamma- aminobutyric acid (GABA)
levels to suppress brain activity, the underlying cause of the virtual lesion effect is still unknown: it
could be the suppression of neural signals or an artifact of adding random noise to an ongoing process
(Sandrini et al., 2011).
tES uses electrical currents applied at low intensities to positively or negatively affect cortical
excitability. There are many techniques under the tES umbrella that are hypothesized to be mechan-
istically different based on differences in the stimulation pattern, including transcranial direct current
stimulation (tDCS), transcranial alternating current stimulation (tACS), and transcranial random
noise stimulation (tRNS). For example, the most commonly employed method, tDCS, uses con-
stant electrical currents applied at low intensities (~1– 2 milliamps (mA)) between electrodes on the
head such that the current passes through the cortex in between to facilitate or inhibit cortical excit-
ability by affecting resting membrane potentials (DaSilva et al., 2015; Miniussi et al., 2008). tDCS
has inferior spatial localization to TMS but is able to penetrate somewhat deeper brain structures
(DaSilva et al., 2015) and is a silent intervention, although it can produce a physical sensation on
the scalp. Stimulation has been found to facilitate long- term memory for word pairs (Marshall et al.,
2004) and vocabulary learning (e.g., Meinzer et al., 2014).
Rather than targeting specific cortical areas directly, peripheral nerve stimulation (PNS)
involves stimulating the peripheral branches of a cranial nerve to modulate cortical function more
broadly. Cranial nerves are sensory and motor neurons that project from the brainstem and supply
nerves to (i.e., innervate) the body, especially the head and neck. Stimulation of their peripheral

Page 3
Nick B. Pandža
74
branches leads to changes in the activity of neuromodulatory systems, which regulate nervous
system activity via neurotransmitters, such as changes in attention with the release of norepin-
ephrine (NE) throughout many areas of the cortex. While there are several types of cranial nerves
being targeted with neurostimulation, the most well- studied PNS to date— and thus, the type of
PNS in focus in this chapter— involves stimulation of the vagus nerve. Transcutaneous vagus
nerve stimulation (tVNS) is a type of PNS that involves electrical stimulation applied at low
levels to the skin over branches of the vagus nerve located in the ear (inner ear, tragus, or cymba
conchae) for transcutaneous auricular VNS (taVNS) or the neck for transcutaneous cervical VNS
(tcVNS) that carry nerve impulses back to the brain. The most well- studied mechanism underlying
VNS benefits for memory and cognition (Vonck et al., 2014) involves the nucleus of the solitary
tract’s innervation of the locus coeruleus (LC) brainstem nucleus, though other mechanisms are
also under investigation (e.g., George et al., 2008). The LC produces all of the neocortex’s supply
of the neurotransmitter norepinephrine (NE), and tVNS- related benefits may be due in part to the
LC- NE system’s role in optimizing behavior by controlling the trade- off between scanning and
focused states of attention (Colzato & Beste, 2020).
Relevant to the study of language learning, under certain parametrizations, TMS, tES, and PNS
have been mechanistically associated with long- term potentiation (LTP), or a facilitation of synaptic
transmission, which is arguably the major cellular mechanism underlying learning and memory for-
mation (Polan�a et al., 2018). Longer tDCS stimulation periods have also been associated with LTP
(Nitsche & Paulus, 2001), rTMS has been associated with LTP (Polan�a et al., 2018), and tVNS
has also been implicated in LTP given that tVNS is believed to indirectly release NE, which in turn
facilitates cortical LTP (Vonck et al., 2014).
Mechanistically, one of the ways by which LTP occurs is thought to be via modulation of brain-
derived neurotropic factor (BDNF), which is a protein encoded in the BDNF gene which has been
associated with rTMS- , tDCS- , and VNS- induced LTP (Cheeran, et al., 2008; Follesa et al., 2007;
Fritsch, et al., 2010) and a potential source of behavioral and physiological variability in NIBS-
facilitated effects (Polan�a et al., 2018). However, common to all NIBS methods, their growing popu-
larity continues to generate debates about their mechanisms of action, application techniques, ethics,
and applied use.
Table 6.1 features a reference list of the critical terms and acronyms related to non- invasive
neurostimulation research presented in this chapter. Note that this list is far from comprehensive, and
there is some variation in terminology in the literature.
Historical Perspectives
TMS, tES, and PNS were all first (and still are) investigated in clinical contexts. TMS currently
has FDA approval for conditions like obsessive- compulsive disorder and major depressive disorder,
tDCS is still being actively investigated for major depressive disorder, and VNS has FDA approval
for epilepsy and depression. In addition to these more or less primary uses, each of these methods has
been investigated for myriad other clinical, applied, and basic research topics.
Historically, TMS’s main use outside of clinical settings has been to disrupt neural activity to make
causal inferences about specific brain regions and cognitive functions (Sandrini et al., 2011). One of
the original and more well- known experimental paradigms for TMS is creating a virtual lesion, a safe,
temporary disruption of brain activity to imitate the effects of an actual brain lesion, such as from
stroke. For example, in the context of language research, a TMS- induced virtual lesion over Broca’s
Area would be expected to produce temporary deficits in language production that mimic Broca’s
Aphasia. TMS’s ability to disrupt brain activity and simulate lesions allow more methodologically
desirable research designs for localizing brain function than trying to find research participants with
rare brain lesions of comparable location and size.

Page 4
Non-Invasive Brain Stimulation to Investigate Second Language
75
While TMS indirectly induces electrical fields in cortex, tES applies weak electrical currents
directly to the scalp. The concept of tES had its origin as early as the eighteenth century, being
investigated more seriously in clinical settings starting in the 1960s (Zoefel & Davis, 2017). The
most common tES technique, tDCS, uses a weak direct current between the electrodes applied to the
scalp, and the current affects the cortex through which it partially passes. TMS and tES both have
an extensive research history related to the enhancement of motor learning (Polan�a et al., 2018) and
substantial literature on their utility as a potential treatment for aphasia (Zoefel & Davis, 2017).
Unique among these three methods is PNS, which was first (and is still) an invasive manipula-
tion before non- invasive techniques were developed. VNS has been investigated invasively in clin-
ical populations since the mid- 1980s for its efficacy as an antiepileptic and antidepressant (Vonck
et al., 2014). More recently, its effects on auditory processing, memory, and cognition have also been
studied (see Colzato & Beste, 2020). The vagus nerve is the tenth cranial nerve and originates from
the medulla in the brainstem. Stimulation to the vagus nerve projects along nerve fibers to the nucleus
of the solitary tract in the brainstem. Recent innovations have led to user- friendly, non- invasive tVNS
technologies that stimulate the vagus by passing electrical current on the skin of the ears or neck
allowing its use and study with neurotypical populations.
Table 6.1 Reference List of Non- invasive Brain Stimulation Acronyms and Terms Presented in This Chapter
non- invasive
brain
stimulation
(NIBS)
methods
transcranial magnetic
stimulation (TMS)
repetitive TMS (rTMS)
high frequency rTMS (HF- rTMS)
low frequency rTMS (LF- rTMS)
theta burst stimulation
(TBS)
continuous TBS (cTBS)
intermittent TBS (iTBS)
transcranial electrical
stimulation (tES)
transcranial direct current
stimulation (tDCS)
anodal tDCS (atDCS)
cathodal tDCS (ctDCS)
transcranial alternating
current stimulation
(tACS)
transcranial random noise
stimulation (tRNS)
peripheral (cranial)
nerve stimulation
(PNS)
vagus nerve stimulation
(VNS)
transcutaneous VNS
(tVNS)
auricular tVNS
(taVNS)
cervical tVNS
(tcVNS)
trigeminal nerve
stimulation (TNS)
NIBS
mechanisms
of action
long- term potentiation
(LTP)
a proposed mechanism of action for cognitive effects of TMS, tES,
and PNS
brain- derived
neurotrophic factor
(BDNF)
a protein through which LTP is thought to occur
locus coeruleus-
norepinephrine
(LC- NE) system
the LC is a region in the brainstem responsible for producing NE, a
neurotransmitter implicated in PNS research
gamma- aminobutyric
acid (GABA)
a neurotransmitter implicated in TMS research

Page 5
Nick B. Pandža
76
Methods and Paradigms
There are multiple ways in which NIBS may be implemented, common ones including priming (i.e.,
conditioning or offline), concurrent (i.e., online), and peristimulus (i.e., stimulus time- locked) stimu-
lation. All three NIBS methods reviewed here can easily be implemented via priming, which involves
applying stimulation for a specified number of seconds or minutes prior to starting a critical task,
presumably inducing tonic shifts in arousal and thus cortical excitability that prepare the individual
to be in an optimal state for learning or performance throughout the task. For example, VNS studies
have observed an increase in activity in LC and related brain structures (Frangos et al., 2015), and
concentrations of norepinephrine in the cortex and hippocampus (Follesa et al., 2007). In TMS terms,
the rTMS technique can be used to prime participants before a task (Klooster et al., 2016). For tES,
studies have found cathodal tDCS (ctDCS) to have longer- lasting priming effects than anodal tDCS
(atDCS) (Monte- Silva et al., 2013), although more robust learning effects have been observed for
atDCS (Simonsmeier et al., 2018).
Peristimulus (peristim) stimulation involves delivering a pulse train of stimulation just prior to
the presentation of critical stimuli, presumably inducing phasic changes in task- related attention to
and consolidation of specific to- be- learned information. Work exploring peristim VNS has shown
effectiveness at improving low- level auditory processing (Engineer et al., 2011), and peristim (or
paired pulse) TMS is mainly used to assess cortical excitability (Klooster et al., 2016). Both delivery
methods may engage LTP but, while priming NIBS is possible with TMS, tES, and PNS, peristim
may be the most practical for language learning protocols with PNS given the noise artifacts of TMS
and the low temporal resolution of tDCS.
For TMS, there are numerous implementations of the method, although rTMS is perhaps most
relevant for investigating longer- lasting changes to cognition. Typically, low- frequency rTMS (LF-
rTMS) is administered ≤ 1 Hz while high- frequency rTMS (HF- rTMS) is administered ≥ 1 Hz, and
this distinction has been found in the motor system to decrease and increase cortical excitability,
respectively (Sandrini et al., 2011). rTMS can be administered concurrently to a task of interest or
beforehand, priming the participant before a task.
Another pattern of pulses, theta burst stimulation (TBS) utilizes trains of pulses applied with
short gaps that can also be used to induce longer- term effects of TMS on cognition. Continuous TBS
(cTBS) tends to be inhibitory whereas more intermittent TBS (iTBS) is excitatory (see Sandrini et al.,
2011 for a discussion of additional implementations).
Similar to TMS, changing the parameters of tES can create either facilitative or inhibitory effects.
Among tES techniques, tDCS is the most frequently used. Differences in atDCS and ctDCS refer
to differences in experimental setup. Both involve an anode (positively charged electrode) and a
cathode (negatively charged electrode), and so atDCS involves placing the anode near a brain region
of interest and the cathode at a control region, and ctDCS is the reverse. While tDCS broadly is a con-
stant current applied at about 1– 2 mA, tACS is applied at lower intensity (0.2– 1 mA) and is a bidir-
ectional/ biphasic current that can be applied at different frequencies (Simonsmeier et al., 2018). Even
lower in intensity is tRNS (- 500 to 500 microamps (μA)), which uses alternating current with random
amplitudes and frequencies (Moreno- Duarte et al., 2014). These are not an exhaustive list of tES
techniques, but are the most commonly employed, and the types included in a recent meta- analysis
of tES on language learning in healthy adults (Balboa- Bandeira et al., 2021). While there is a logic
to both pairing NIBS with learning (e.g., to potentially improve consolidation of new information)
and assessment (e.g., to potentially improve recall of learned information), a meta- analysis of tES
recently found a more robust effect when paired with learning than with assessment, and the effect
was only significant for anodal and not cathodal tDCS (Simonsmeier et al., 2018). Balboa- Bandeira
et al. (2021) conducted a meta- analysis for the effect of tES on adult language learning (nonwords,
artificial grammar, and foreign languages) and found a moderate effect of tES. While they did not

Page 6
Non-Invasive Brain Stimulation to Investigate Second Language
77
find an effect on follow- up data or response times, there are a number of limitations in the review,
including the number of studies that could be included (11 in total) and disparate tES protocols,
as effects are assessed across studies encompassing tRNS, tACS, atDCS, ctDCS, and high density
tDCS. They rightfully call for further work to tease apart the potentially mechanistically different
effects of each tES technique, especially with multiple stimulation sessions. In particular they note
that a majority of the studies are tDCS and show effects with one day of training even though tRNS
may also have long- lasting effects on language learning processes. A number of studies included in
the meta- analysis involved continuous stimulation during the process of learning rather than before
as in a priming paradigm.
VNS can be implemented at the auricular or cervical branches, the former being more exten-
sively implemented to date, especially in language research. For the auricular branch of the vagus
nerve, there’s still an open question over the optimal location for stimulation. Yakunina et al. (2017)
compared three different stimulation points on the ear within individuals versus a sham control of
the earlobe: the outer ear canal, the inner tragus, and the cymba conchae. With confirmation from
functional MRI (fMRI; see Kousaie & Klein, this volume), they found stimulation of the cymba
conchae to produce the most reliable activation of the nucleus of the solitary tract and the locus
coeruleus in the brainstem compared to the other locations. However, in comparing this to other
research it’s worth noting that participants in this study were all delivered tVNS at 0.1 mA below
their pain threshold, and results here may reflect a higher tolerable threshold for the cymba conchae
than other stimulation sites on the ear, and results may have been different if stimulation was kept
below their sensory thresholds. For example, taVNS applied to the outer ear canal below sensory
threshold has been successfully implemented for both priming and peristim in the same training
study and has been found to both show positive effects but on different aspects of testing, reaction
time and accuracy, respectively (Pandža et al., 2020). These effects provide evidence for potentially
different mechanisms of action for taVNS on learning based solely on where in the learning process
NIBS is implemented.
For each of TMS, tES, and PNS, there are many ways for these methods and the techniques under
each of these umbrellas to be implemented. Care needs to be taken when deciding when, where, how,
and at what intensity NIBS are implemented so that effects can be reliably detected, the mechanism(s)
of action can be properly investigated, and the results can be replicated.
Example Studies
TMS
Of the three types of NIBS reviewed here, TMS is the most under- studied for the specific use case of
language learning. Of the few studies conducted to date, they are focused largely on the investigation
of neurocognitive mechanisms underlying language learning and/ or clinical application. Many report
interesting uses of rTMS in which it enhanced implicit learning mechanisms in adults. For example,
Ambrus et al. (2020) disrupted the left and right dorsolateral prefrontal cortex with inhibitory rTMS,
leading participants to better implicit statistical learning via consolidation of non- adjacent second-
order dependencies in an alternating serial reaction time task. In contrast, Sliwinska et al. (2017)
found positive effects of rTMS on explicit vocabulary learning and focused on implications for pro-
moting post- aphasia recovery. After first conducting a word learning study with fMRI to identify
functional networks of interest, the authors conducted a second experiment on a subset of participants
in which they applied priming rTMS for 10 minutes each day in three training sessions. They found
both improved accuracy and reaction time in a paired- associates translation judgment task with feed-
back during early stages of word learning, and they posit the potential for rTMS for use in aphasia
rehabilitation.

Page 7
Nick B. Pandža
78
tES
Perceval et al. (2020) implemented an ambitious atDCS multisession double- blind design in which
younger and older adults were tasked with learning non- word “names” for “space alien” characters
and two semantic attributes for each. Active atDCS was administered for 20 minutes during the
learning phase on each day over the left inferior frontal gyrus to target areas implicated in verbal asso-
ciative learning. Training was administered over five consecutive days with immediate testing and
day- after testing to assess retention, and follow- up testing done one day, one week, and three months
after the last day of training. In both younger and older adults, effects for atDCS were only observed
for those with lower scores of baseline learning ability measured at pretest. This study shows the
potential for individual differences to interact with neurostimulation to show more nuanced effects, in
this case a modest learning “boost” for those participants who were a priori disadvantaged. Learning
facilitated by atDCS was largely immediate and persisted up to three months for the low ability older
adults, contrasting with younger adults for which there were no immediate stimulation effects (pos-
sibly due to ceiling effects) but retention was enhanced at delayed testing up to three months. While
the authors conclude uncertainty around the mechanisms of action to explain their results, they pro-
pose that future research focus on the neural mechanisms underlying their novel finding of differen-
tial effectiveness of atDCS according to baseline ability.
In the minority are non- tDCS tES studies like Pasqualotto et al. (2015) looking at tRNS and
Antonenko et al. (2016) looking at tACS. In comparison to Perceval et al. (2020), Antonenko et al.
(2016) found increased retrieval accuracy in older adults but not younger adults with tACS during
implicit language learning, also finding an advantage for older adults with tES that is worth further
mechanistic exploration. Pasqualotto et al. (2015) was the first to investigate the potential of tRNS on
language learning and found a potential memory consolidation advantage for active tRNS delivered
over posterior parietal areas during learning at a one- week delayed posttest.
PNS
While PNS is a broad category of techniques, there is a rapidly growing L2 literature using a spe-
cific subtype of PNS and VNS: taVNS. Pandža et al. (2020) were the first to directly investigate
taVNS- facilitated language learning, specifically for Mandarin tone. In a double- blind study, they
directly compared priming and peristim taVNS protocols compared to a sham control in a two- day
Mandarin tone word training for tone- na�ve native speakers of English. Active stimulation was
applied before (priming) or during (peristim) learning and testing tasks. They found peristim but
not priming to reliably improve accuracy on lexical recognition while priming but not peristim
improved reaction time on lexical recognition. In an analysis of pupillometry data to assess cog-
nitive effort and ties to the LC- NE system, they found a reduction in sustained effort day- to- day
for the sham group in line with the idea of less effort being applied to better- learned words, and a
significantly stronger effect for peristim. For priming, results were interpreted as possibly in line
with a tonic, rather than phasic (task- evoked) effect on arousal. In total, the authors concluded
that effects are in line with the idea that peristim taVNS supported better encoding of new to- be-
learned information whereas priming taVNS may instead support better lexical access of already-
learned information.
Thakkar et al. (2020) also found positive taVNS results to extend to orthography acquisition.
Separately, Calloway et al. (2020) investigated the potential for another researched mechanism
of action for VNS to affect language learning, the mitigation of anxiety (George et al., 2008),
and found 10 minutes of priming taVNS to reduce negative affect, somatic anxiety, and cognitive
anxiety.

Page 8
Non-Invasive Brain Stimulation to Investigate Second Language
79
There is still a dearth of language learning research with TMS, tES, and PNS, so there are numerous
ways to expand the field. Much more research is needed with each of these techniques to form firm
conclusions around their efficacy.
Considerations and Limitations of NIBS Methods
There are a number of pros and cons for each of these NIBS methods. All result in at least some-
what indirect neuromodulation as stimulation is applied on the outside of the body versus directly to
brain tissue. TMS and tES can be combined with structural MRIs to pinpoint where brain areas are
specific to a person and even specific to language processing (e.g., Broca’s Area, Wernicke’s Area),
given that brain morphology varies widely. In this way, TMS and tES can accurately encapsulate
a brain region of interest; however, given the spatial resolution of both methods, they are inexact
and likely to also affect neighboring regions and their associated functions. PNS has both better
and worse spatial resolution: better in that there is a specific brainstem nucleus (e.g., nucleus of
the solitary tract for VNS) that will be stimulated in turn by the stimulated cranial nerve, but worse
in that the effects of interest for PNS interventions are more about what happens downstream from
that initial nucleus, for example stimulation of the LC and the release of neurotransmitters like NE
that are dispersed throughout the cortex. Thus, a major advantage for TMS and tES over PNS is the
inability of PNS to target specific brain structures of interest. However, while TMS and tES can only
target a variety of surface- level cortical structures on the order of centimeters, PNS on the other hand
indirectly stimulates structures in the brainstem leading to connected structures such as the LC and
to the release of myriad neurotransmitters throughout the brain to produce broader (if more domain-
general) impacts to cognition and language learning. Nevertheless, TMS, tES, and PNS can each be
combined with other behavioral, physiological, and neurocognitive methods to more conclusively
pinpoint brain areas and functions and causally associate them with a particular behavior. Again, most
critically, all three techniques have also been associated with LTP under certain conditions and thus
to the causal enhancement of learning and memory processes broadly, making each relevant to the
investigation of language learning.
In terms of practical concerns around data collection, there are a number of considerations. It’s
important to note that TMS has a noise artifact (repetitive clicking noise). This may not be a problem
for NIBS priming methodologies, but may be problematic if concurrent or peristimulus stimulation is
mechanistically desired for a particular research design. Minimally, this could be distracting to a par-
ticipant during a task, and, maximally, could actively interfere with auditory task designs. While tES
and PNS are silent interventions, tES can induce somatosensory artifacts across the scalp, which can
also be distracting to participants. PNS can also produce a physical sensation under the electrodes or
even harmless muscle spasms, depending on the location and stimulation intensity. For instance, the
cervical branch of the vagus nerve is embedded deeper under the skin of the neck than the auricular
branch is under the skin of the ear. Because of this, tcVNS is practically impossible without some
physical sensation (even harmless muscle spasms in the neck if placed incorrectly) which may be
uncomfortable or distracting to individuals. However, multiple studies have now shown that taVNS
can be effectively delivered below a participant’s individual perceptual threshold while still showing
positive language learning outcomes (e.g., Calloway et al., 2020; Pandža et al., 2020).
To show causal evidence of NIBS on behavior, double- blind studies are ideal in which participants
and proctors are blind to whether the participant is receiving active stimulation vs. sham stimula-
tion or belongs to a no- stimulation control. At the minimum, single- blind studies can also be used
in which the participants but not the proctors are blind to participant group. Properly blinding NIBS
methods means replicating the TMS clicking noise for a sham control, using tES over a control
brain region to produce similar somatosensory effects or, if using PNS at a location or intensity that

Page 9
Nick B. Pandža
80
necessarily causes physical sensation, using PNS over a control skin area that is not innervated by the
cranial nerve of interest.
While all three techniques involve potentially costly equipment upfront and require training for
their effective and safe use, relative to other neurocognitive methodologies there are some advantages
to neurostimulation. For example, the equipment for these techniques is cheaper than MRI data
collection, participant setup and takedown are often faster and easier than for EEG, and no specially
shielded room is required for data collection. However, methodologically sound TMS and to a lesser
extent tES studies involve “neuronavigation” such that researchers can be sure they are targeting
brain regions of interest for each individual as precisely as possible (Polan�a et al., 2018), and this
is done in conjunction with a structural MRI, which increases the cost of the research and adds MRI
contraindications back into the research design.
In terms of safety, TMS, tES, and PNS appear to be safe when used properly (Antal et al., 2017;
Redgrave et al., 2018; Rossi et al., 2021). However, rare side effects can occur of which researchers
should be aware. The most serious adverse effect is for TMS, which in extremely rare circumstances,
if proper use guidelines are not followed, may cause acute seizures (Rossi et al., 2021). TMS is also
contraindicated for individuals with metallic or electronic implants near the TMS coil (Rossi et al.,
2021). For tES and PNS protocols, in which electrodes come into physical contact with the skin and
scalp, the most common side effects can include uncomfortable heating or sensation directly under
the electrode (Antal et al., 2017; Redgrave et al., 2018), but mild pain is also common under the
stimulation site for TMS (Rossi et al., 2021). The potential for headaches has also been observed for
all three methods (Antal et al., 2017; Redgrave et al., 2018; Rossi et al., 2021). Because of a remote
possibility of cardiac effects (Farmer et al., 2021), tVNS is usually administered to the left ear or
left side of the neck as right branches of the vagus nerve are more closely connected to the heart.
Because of the still relative novelty of some of these methods and the study of their implementa-
tion, researchers often impose restrictive eligibility criteria out of an abundance of caution where
safety research is still currently lacking, such as administering tVNS to the left side as mentioned or
excluding participants from tVNS if they have a history of fainting spells (vasovagal syncope is a
condition in which an overactive vagus nerve can result in fainting).
TMS, tES, and PNS in the literature currently all suffer from a replication problem due to still
active uncertainty around how different parameters affect a number of factors ranging from not only
variation in sensation artifacts but also, crucially, the neurocognitive mechanisms affected. Indeed,
it has been noted in particular that variation in how TMS and tES are implemented has actively
prevented conclusive findings from meta- analyses (see Polan�a et al., 2018). All three methods can
be implemented in myriad ways, for example, at different levels of intensity, with different stimula-
tion wave forms, or with different electrode placements. The conclusive impact of any modification
of these settings is still under active investigation (e.g., see discussion in Farmer et al., 2021 for
tVNS), but researchers looking to implement these techniques should take particular care in choosing
parameters as changes in any of these could affect not only the strength of any NIBS effects but even
change the underlying mechanism of action (Polan�a et al., 2018).
Innovations and Future Directions
New NIBS parameters and techniques are still being developed for existing methods to overcome
their current limitations or even to target additional neurological mechanisms, such as to improve
both spatial resolution and the ability to target deeper brain structures for tES (Datta et al., 2009).
At the same time, new neurostimulation techniques are also being developed. For example, early
findings for transcranial focused ultrasound stimulation (tFUS) suggest it can successfully overcome
the inherent limitations of existing transcranial methods by being able to easily target deeper brain

Page 10
Non-Invasive Brain Stimulation to Investigate Second Language
81
structures and with a higher spatial resolution, inducing cortical excitation on the order of millimeters
compared to centimeters for TMS and tES (Tufail et al., 2010).
The topic of language learning is still relatively new to exploration with NIBS interventions, but
the cited research provides preliminary evidence that various NIBS methods have the potential to
meaningfully impact language learning outcomes. The effects of NIBS on attention and memory
consolidation could promote more effective language learning, and some of the cited studies suggest
potential for longer- lasting effects. Assessing the benefits of any new intervention on language learning
outcomes is non- trivial. Does the intervention serve to increase phoneme and/ or word recognition
accuracy overall? Increase the overall learning rate? Reduce the mental load associated with learning
an individual item, freeing up mental resources for other aspects of learning? Are there interactions
with NIBS efficacy and learner individual differences? Particularly challenging for NIBS is that,
due to a paucity of established research with comparable implementations and parameterizations,
expected effect sizes have not been firmly established, and thus it is possible that NIBS- induced
changes in neural function might be subtle or very focused, and thus primarily relevant for only
a subset of possible language learning outcomes. Given the range of possibilities, assessments of
NIBS- driven language- learning benefits should encompass multiple outcome measures whenever
possible, to include indices like accuracy, reaction time, pupillometry, EEG, and fMRI. Perhaps most
importantly, rigorous assessments of longer- lasting effects of NIBS in the form of delayed posttests
are critical to evaluate the potential for any truly practical implications of NIBS to support language
learning.
As currently more tDCS and tVNS studies exist in the realm of language learning enhancement, a
promising additional line of research would be to directly pit the two NIBS methods against each other
with the same training paradigm. Given the potential differences in mechanisms of action, it would
be of interest to investigate differences in effect size improvements between the two techniques.
Additionally, there is a dearth of literature on TMS to enhance language learning, and further explor-
ation of that methodology should be made to empirically evaluate its utility versus other alternatives.
Likewise, further exploration of tES techniques in addition to tDCS (including tACS and tRNS) is
warranted given preliminary positive results. While the PNS studies cited here focused on taVNS,
tcVNS also has the potential to accelerate language learning with some incipient research showing
lasting cognitive effects after brief stimulation (e.g., Lewine et al., 2019), as does transcutaneous
trigeminal nerve stimulation (TNS) across the forehead (Colzato & Vonck, 2017).
Contrasting with more correlational neurocognitive techniques, neurostimulation provides the
opportunity to make causal inferences between particular cognitive functions and associated behav-
ioral outcomes, and it can be combined with those correlational methods for a more targeted or mech-
anistically validated impact. Research on NIBS in L2 is clearly still at the initial stages, but it has the
potential to transform the field of SLA.
Further Readings
For a comprehensive overview of NIBS research, covering primarily TMS and tES techniques:
Polan�a, R., Nitsche, M.A., & Ruff, C.C. (2018). Studying and modifying brain function with non- invasive brain
stimulation. Nature Neuroscience, 21(2), 174– 187. https:// doi.org/ 10.1038/ s41 593- 017- 0054- 4
For a systematic review of high- frequency rTMS studies stimulating over prefrontal cortex, many of which found
improvements on verbal measures:
Guse, B., Falkai, P., & Wobrock, T. (2010). Cognitive effects of high- frequency repetitive transcranial magnetic
stimulation: a systematic review. Journal of Neural Transmission, 117(1), 105– 122. https:// doi.org/ 10.1007/
s00 702- 009- 0333- 7
For a meta- analysis of tES effects on second and foreign language learning, which covers multiple tES methods,
identifies gaps in the literature, and makes recommendations for future research:

Page 11
Nick B. Pandža
82
Balboa- Bandeira, Y., Zubiaurre- Elorza, L., Ibarretxe- Bilbao, N., Ojeda, N., & Pe�a, J. (2021). Effects of
transcranial electrical stimulation techniques on second and foreign language learning enhancement in
healthy adults: A systematic review and meta- analysis. Neuropsychologia, 107985. https:// doi.org/ 10.1016/
j.neuro psyc holo gia.2021.107 985
For an empirical taVNS study of Mandarin tone word learning that analyzed event- related potentials from Pandža
et al. (2020), showing evidence of stronger lexico- semantic encoding via the N400 after taVNS:
Phillips, I., Calloway, R.C., Karuzis, V.P., Pandža, N.B., O’Rourke, P., & Kuchinsky, S.E. (2021). Transcutaneous
auricular vagus nerve stimulation strengthens semantic representations of foreign language tone words during
initial stages of learning. Journal of Cognitive Neuroscience, 1– 26. https:// doi.org/ 10.1162/ jocn_ a_ 01 783
Acknowledgments
This material is based in part upon work supported by the Naval Information Warfare Center and Defense
Advanced Research Projects Agency under Cooperative Agreement No. N66001– 17- 2- 4009. The views,
opinions, and/ or findings contained in this material are those of the author and should not be interpreted as
representing the official views or policies of the Department of Defense or the U.S. Government.
References
Ambrus, G.G., V�kony, T., Janacsek, K., Trimborn, A. B., Kov�cs, G., & Nemeth, D. (2020). When less is
more: Enhanced statistical learning of non- adjacent dependencies after disruption of bilateral DLPFC.
Journal of Memory and Language, 114, 104144. https:// doi.org/ 10.1016/ j.jml.2020.104 144
Antal, A., Alekseichuk, I., Bikson, M., Brockm�ller, J., Brunoni, A.R., Chen, R., … & Paulus, W. (2017). Low
intensity transcranial electric stimulation: safety, ethical, legal regulatory and application guidelines. Clinical
Neurophysiology, 128(9), 1774– 1809. https:// doi.org/ 10.1016/ j.cli nph.2017.06.001
Antonenko, D., Faxel, M., Grittner, U., Lavidor, M., & Fl�el, A. (2016). Effects of transcranial alternating current
stimulation on cognitive functions in healthy young and older adults. Neural Plasticity, 2016. https:// doi.org/
10.1155/ 2016/ 4274 127
Balboa- Bandeira, Y., Zubiaurre- Elorza, L., Ibarretxe- Bilbao, N., Ojeda, N., & Pe�a, J. (2021). Effects of
transcranial electrical stimulation techniques on second and foreign language learning enhancement in
healthy adults: A systematic review and meta- analysis. Neuropsychologia, 107985. https:// doi.org/ 10.1016/
j.neuro psyc holo gia.2021.107 985
Calloway, R., Karuzis, V., Tseng, A., Martinez, D., & O’Rourke, P. (2020). Auricular Transcutaneous Vagus
Nerve Stimulation (tVNS) Affects Mood and Anxiety during Second Language Learning. In S. Denison., M.
Mack, Y. Xu, & B.C. Armstrong (Eds.), Proceedings of the 42nd annual conference of the Cognitive Science
Society (pp. 3496– 3502). Cognitive Science Society.
Cheeran, B., Talelli, P., Mori, F., Koch, G., Suppa, A., Edwards, M., … & Rothwell, J.C. (2008). A common
polymorphism in the brain- derived neurotrophic factor gene (BDNF) modulates human cortical plasticity
and the response to rTMS. The Journal of Physiology, 586(23), 5717– 5725. https:// doi.org/ 10.1113/ jphys
iol.2008.159 905
Colzato, L., & Beste, C. (2020). A literature review on the neurophysiological underpinnings and cognitive effects
of transcutaneous vagus nerve stimulation: challenges and future directions. Journal of Neurophysiology,
123(5), 1739– 1755. https:// doi.org/ 10.1152/ jn.00057.2020
Colzato, L.S., & Vonck, K. (2017). Transcutaneous vagus and trigeminal nerve stimulation. In L.S. Colzato (Ed.),
Theory-driven approaches to cognitive enhancement (pp. 115– 126). Springer, Cham.
DaSilva, A.F., Truong, D.Q., DosSantos, M.F., Toback, R.L., Datta, A., & Bikson, M. (2015). State- of- art neuro-
anatomical target analysis of high- definition and conventional tDCS montages used for migraine and pain
control. Frontiers in Neuroanatomy, 9, 89. https:// doi.org/ 10.3389/ fnana.2015.00089
Datta, A., Bansal, V., Diaz, J., Patel, J., Reato, D., & Bikson, M. (2009). Gyri- precise head model of transcranial
direct current stimulation: improved spatial focality using a ring electrode versus conventional rectangular
pad. Brain Stimulation, 2(4), 201– 207. https:// doi.org/ 10.1016/ j.brs.2009.03.005
Dickson, D., & Pelzl, E. (this volume). Using time- based electroencephalography to investigate second lan-
guage. In K. Morgan- Short and J.G. van Hell (Eds.), The Routledge handbook of second language acquisition
and neurolinguistics. Routledge.
Engineer, N.D., Riley, J.R., Seale, J.D., Vrana, W.A., Shetake, J.A., Sudanagunta, S.P., … & Kilgard, M.P.
(2011). Reversing pathological neural activity using targeted plasticity. Nature, 470, 101– 104. https:// doi.org/
10.1038/ natu re09 656

Page 12
Non-Invasive Brain Stimulation to Investigate Second Language
83
Farmer, A.D., Strzelczyk, A., Finisguerra, A., Gourine, A.V., Gharabaghi, A., Hasan, A., … & Koenig, J. (2021).
International consensus based review and recommendations for minimum reporting standards in research on
transcutaneous vagus nerve stimulation (version 2020). Frontiers in Human Neuroscience, 14, 409. https://
doi.org/ 10.3389/ fnhum.2020.568 051
Follesa, P., Biggio, F., Gorini, G., Caria, S., Talani, G., Dazzi, L., … & Biggio, G. (2007). Vagus nerve stimu-
lation increases norepinephrine concentration and the gene expression of BDNF and bFGF in the rat brain.
Brain Research, 1179, 28– 34. https:// doi.org/ 10.1016/ j.brain res.2007.08.045
Frangos, E., Ellrich, J., & Komisaruk, B.R. (2015). Non- invasive access to the vagus nerve central projections
via electrical stimulation of the external ear: fMRI evidence in humans. Brain Stimulation, 8(3), 624– 636.
https:// doi.org/ 10.1016/ j.brs.2014.11.018
Fritsch, B., Reis, J., Martinowich, K., Schambra, H.M., Ji, Y., Cohen, L.G., & Lu, B. (2010). Direct current
stimulation promotes BDNF- dependent synaptic plasticity: potential implications for motor learning. Neuron,
66(2), 198– 204. https:// doi.org/ 10.1016/ j.neu ron.2010.03.035
George, M.S., Ward Jr, H.E., Ninan, P.T., Pollack, M., Nahas, Z., Anderson, B., … & Ballenger, J.C. (2008). A
pilot study of vagus nerve stimulation (VNS) for treatment- resistant anxiety disorders. Brain Stimulation,
1(2), 112– 121. https:// doi.org/ 10.1016/ j.brs.2008.02.001
Ingvalson, E.M., Ettlinger, M., & Wong, P.C. (2014). Bilingual speech perception and learning: A review of recent
trends. International Journal of Bilingualism, 18(1), 35– 47. https:// doi.org/ 10.1177/ 13670 0691 2456 586
Klooster, D.C., de Louw, A.J., Aldenkamp, A.P., Besseling, R.M.H., Mestrom, R.M.C., Carrette, S., … & Boon,
P. (2016). Technical aspects of neurostimulation: Focus on equipment, electric field modeling, and stimu-
lation protocols. Neuroscience & Biobehavioral Reviews, 65, 113– 141. https:// doi.org/ 10.1016/ j.neubio
rev.2016.02.016
Kousaie, S., & Klein, D. (this volume). Using functional neuroimaging to investigate second language organiza-
tion. In K. Morgan- Short & J.G. van Hell (Eds.), The Routledge handbook of second language acquisition
and neurolinguistics. Routledge.
Lewine, J.D., Paulson, K., Bangera, N., & Simon, B.J. (2019). Exploration of the impact of brief noninvasive
vagal nerve stimulation on EEG and event- related potentials. Neuromodulation: Technology at the Neural
Interface, 22(5), 564– 572. https:// doi.org/ 10.1111/ ner.12864
Marshall, L., M�lle, M., Hallschmid, M., & Born, J. (2004). Transcranial direct current stimulation during
sleep improves declarative memory. Journal of Neuroscience, 24(44), 9985– 9992. https:// doi.org/ 10.1523/
JNEURO SCI.2725- 04.2004
Meinzer, M., J�hnigen, S., Copland, D.A., Darkow, R., Grittner, U., Avirame, K., … & Fl�el, A. (2014).
Transcranial direct current stimulation over multiple days improves learning and maintenance of a novel
vocabulary. Cortex, 50, 137– 147. https:// doi.org/ 10.1016/ j.cor tex.2013.07.013
Miniussi, C., Cappa, S.F., Cohen, L.G., Fl�el, A., Fregni, F., Nitsche, M.A., … & Walsh, V. (2008). Efficacy
of repetitive transcranial magnetic stimulation/ transcranial direct current stimulation in cognitive
neurorehabilitation. Brain Stimulation, 1(4), 326– 336. https:// doi.org/ 10.1016/ j.brs.2008.07.002
Monte- Silva, K., Kuo, M.F., Hessenthaler, S., Fresnoza, S., Liebetanz, D., Paulus, W., & Nitsche, M.A. (2013).
Induction of late LTP- like plasticity in the human motor cortex by repeated non- invasive brain stimulation.
Brain Stimulation, 6(3), 424– 432. https:// doi.org/ 10.1016/ j.brs.2012.04.011
Moreno- Duarte, I., Gebodh, N., Schestatsky, P., Guleyupoglu, B., Reato, D., Bikson, M., & Fregni, F. (2014).
Transcranial electrical stimulation: Transcranial direct current stimulation (tDCS), transcranial alternating
current stimulation (tACS), transcranial pulsed current stimulation (tPCS), and transcranial random noise
stimulation (tRNS). In The Stimulated Brain (pp. 35– 59). Academic Press.
Mottaghy, F.M., Hungs, M., Br�gmann, M., Sparing, R., Boroojerdi, B., Foltys, H., … & T�pper, R. (1999).
Facilitation of picture naming after repetitive transcranial magnetic stimulation. Neurology, 53(8), 1806–
1806. https:// doi.org/ 10.1212/ WNL.53.8.1806
Mottarella, M., & Prat, C.S. (this volume). Using quantitative electroencephalography (qEEG) to investigate
second language learning. In K. Morgan- Short and J.G. van Hell (Eds.), The Routledge handbook of second
language acquisition and neurolinguistics. Routledge.
Nitsche, M.A. & Paulus, W. (2001). Sustained excitability elevations induced by transcranial DC motor cortex
stimulation in humans. Neurology, 57, 1899– 1901. https:// doi.org/ 10.1212/ WNL.57.10.1899
Pandža, N.B., Phillips, I., Karuzis, V.P., O’Rourke, P., & Kuchinsky, S.E. (2020). Neurostimulation and
pupillometry: New directions for learning and research in applied linguistics. Annual Review of Applied
Linguistics, 40, 56– 77. https:// doi.org/ 10.1017/ S02671 9052 0000 069
Pascual- Leone, A., Walsh, V., & Rothwell, J. (2000). Transcranial magnetic stimulation in cognitive neuroscience–
virtual lesion, chronometry, and functional connectivity. Current Opinions in Neurobiology, 10, 232– 237.
https:// doi.org/ 10.1016/ S0959- 4388(00)00081- 7

Page 13
Nick B. Pandža
84
Pasqualotto, A., Kobanbay, B., & Proulx, M.J. (2015). Neural stimulation has a long- term effect on foreign
vocabulary acquisition. Neural Plasticity, 2015, 671705. https:// doi.org/ 10.1155/ 2015/ 671 705
Perceval, G., Martin, A.K., Copland, D.A., Laine, M., & Meinzer, M. (2020). Multisession transcranial direct
current stimulation facilitates verbal learning and memory consolidation in young and older adults. Brain and
Language, 205, 104788. https:// doi.org/ 10.1016/ j.bandl.2020.104 788
Polan�a, R., Nitsche, M.A., & Ruff, C.C. (2018). Studying and modifying brain function with non- invasive brain
stimulation. Nature Neuroscience, 21, 174– 187. https:// doi.org/ 10.1038/ s41 593- 017- 0054- 4
Redgrave, J., Day, D., Leung, H., Laud, P.J., Ali, A., Lindert, R., & Majid, A. (2018). Safety and tolerability
of transcutaneous vagus nerve stimulation in humans; a systematic review. Brain Stimulation, 11(6), 1225–
1238. https:// doi.org/ 10.1016/ j.brs.2018.08.010
Rossi, E., Voits, T., & DeLuca, V. (this volume). Using structural neuroimaging to investigate second language.
In K. Morgan- Short and J.G. van Hell (Eds.), The Routledge handbook of second language acquisition and
neurolinguistics. Routledge.
Rossi, S., Antal, A., Bestmann, S., Bikson, M., Brewer, C., Brockm�ller, J., … & Hallett, M. (2021). Safety and
recommendations for TMS use in healthy subjects and patient populations, with updates on training, ethical
and regulatory issues: expert guidelines. Clinical Neurophysiology, 132(1), 269– 306. https:// doi.org/ 10.1016/
j.cli nph.2020.10.003
Sandrini, M., Umilt�, C., & Rusconi, E. (2011). The use of transcranial magnetic stimulation in cognitive neuro-
science: a new synthesis of methodological issues. Neuroscience & Biobehavioral Reviews, 35(3), 516– 536.
https:// doi.org/ 10.1016/ j.neubio rev.2010.06.005
Simonsmeier, B.A., Grabner, R.H., Hein, J., Krenz, U., & Schneider, M. (2018). Electrical brain stimulation
(tES) improves learning more than performance: A meta- analysis. Neuroscience & Biobehavioral Reviews,
84, 171– 181. https:// doi.org/ 10.1016/ j.neubio rev.2017.11.001
Sliwinska, M.W., Violante, I.R., Wise, R.J., Leech, R., Devlin, J.T., Geranmayeh, F., & Hampshire, A. (2017).
Stimulating multiple- demand cortex enhances vocabulary learning. Journal of Neuroscience, 37(32), 7606–
7618. https:// doi.org/ 10.1523/ JNEURO SCI.3857- 16.2017
Thakkar, V.J., Engelhart, A.S., Khodaparast, N., Abadzi, H., & Centanni, T.M. (2020). Transcutaneous auricular
vagus nerve stimulation enhances learning of novel letter- sound relationships in adults. Brain Stimulation,
13(6), 1813– 1820. https:// doi.org/ 10.1016/ j.brs.2020.10.012
Tufail, Y., Matyushov, A., Baldwin, N., Tauchmann, M.L., Georges, J., Yoshihiro, A., … & Tyler, W. J. (2010).
Transcranial pulsed ultrasound stimulates intact brain circuits. Neuron, 66(5), 681– 694. https:// doi.org/
10.1016/ j.neu ron.2010.05.008
Vonck, K., Raedt, R., Naulaerts, J., De Vogelaere, F., Thiery, E., Van Roost, D., … & Boon, P. (2014). Vagus
nerve stimulation… 25 years later! What do we know about the effects on cognition? Neuroscience &
Biobehavioral Reviews, 45, 63– 71. https:// doi.org/ 10.1016/ j.neubio rev.2014.05.005
Yakunina, N., Kim, S.S., & Nam, E.C. (2017). Optimization of transcutaneous vagus nerve stimulation using
functional MRI. Neuromodulation: Technology at the Neural Interface, 20(3), 290– 300. https:// doi.org/
10.1111/ ner.12541
Zoefel, B. & Davis, M.H. (2017). Transcranial electric stimulation for the investigation of speech perception
and comprehension. Language, Cognition and Neuroscience, 32(7), 910– 923, https:// doi.org/ 10.1080/ 23273
798.2016.1247 970