Clustering of vocal types in early human development

To uncover mechanisms underlying language acquisition and the distant origins of language, various learning mechanisms have been proposed, including imitation and social interaction [1,2]. Infants’ endogenous vocal production and exploration of vocal types has received comparatively little attention in the literature until recently [3]. The present study provides empirical evidence of the early emergence of vocal development in infants by systematically quantifying the occurrence of infant vocal categories, suggesting that active vocal exploration is fundamental to subsequent speech development and provides a foundation for interaction with caregivers.

The work is inspired by the fact that various vocal types of early human vocalizations known to be precursors to speech (“protophones”) have been observed to occur in clusters [4,5] of particular phonatory types. There are three predominant types: 1) vocants are vowel-like sounds produced with normal phonation in the mid-pitch range of the individual infant; 2) squeals are produced at very high pitch, often in falsetto phonation; and 3) growls are low pitched or harshly phonated sounds often in fry. Clustering can be observed in that a baby may produce several squeals in a short period of time, and then for the next few minutes no squeals at all, while other vocal types continue to be produced. Also, from day to day there appear to occur wide variations in the number of squeals babies produce. Similar clustering has been observed for growls and other less frequently occurring sounds such as raspberries or whispers [6]. In the present paper we will greatly expand the quantitative exploration of this presumable non-randomness of infant vocal types by reporting on a sample of 130 infants recorded longitudinally all-day in their homes. In the present paper we define clustering as the non-random occurrence of particular protophones across sessions, that is, across specified time periods. The paper does not address immediate “repetition” of particular protophone types, one after another within sessions.

The pattern leads us to consider why a tendency for clustering exists. The activity often seems playful, and one cannot avoid wondering if it might constitute practice. But practice for what? If for language, then we are forced to ask how being capable of producing such sounds as squeals and growls could form foundations for language, because such utterances are not in and of themselves elements of language. They are not well-formed syllables, and they do not, indeed they cannot become words. We are driven to question why there could be any advantage to practicing such sounds. Our paper is based on the supposition that clustering of infant vocal types 1) helps to establish the principle of vocal category formation, the mastery of which is required for vocal language [6] and 2) provides information to caregivers about infant wellness.

Foundations for mature vocal communication in infancy

In the first year of life, human infants engage in remarkably extensive vocal activity with protophones, a term that includes both precanonical and canonical speech-like sounds, while excluding cries, laughs and vegetative sounds. This vocal activity has been estimated to involve an average of 4–5 protophones per minute every waking hour from the first month of life and continuing throughout the first year [3]. Caregivers respond to protophones in face-to-face sustained interactions called “protoconversations” [7,8] that have no precedent in the animal kingdom as far as we know.

Surprisingly, even though these salient interactive periods between human caregivers and infants have generated considerable interest and speculation about foundations for language, the great bulk of the protophone production of human infants does not actually occur during social interaction. Instead, examination of extensive laboratory recordings along with all-day recordings in infant homes has led to the conclusion that more than 90% of protophones are produced in endogenous, non-socially-directed activity [9], much of which might be termed infant “vocal exploration” or “vocal play” [5]. In an attempt to understand the mechanisms of language development, researchers have explored cross-species “babbling” in non-human primates and songbirds (for more information, see [10–14]). However, active vocal exploration seems to be uniquely observed in human infants, distinguishing them from other species. In all the cases where birds, bats, or humans engage in seemingly playful infant vocal activity, it is presumed that the babbling is a precursor to the mature form of vocal communication, and that the activity can be thought of as a kind of exploratory practice laying groundwork for the elaborate forms of song and speech.

Categories of early vocalization and the idea of infant vocal practice

Widely accepted descriptions of early human vocal development designate categories of phonation that appear to occur universally in the first months of life [15–18]. Although they can be broken down into further subcategories, the three most common phonatory categories include: 1) vocants (also called vowel-like sounds) produced in normal phonation, the kind that dominates the utterances of natural languages, and produced at the mid-pitch range of the individual, 2) squeals, which are saliently high-pitched sounds, produced with typically at least twice the fundamental frequency (f₀) of the infant’s usual voice, and 3) growls, produced typically with salient harsh, noisy phonation, often at low pitch, or produced in vocal fry (“pulse”) at very low pitch with respect to the infant’s typical f₀ [19]; generally, no sound that is deemed to occur at higher than the mid-pitch range for the individual is categorized as a growl.

These three vocal types have been reported to occur not only by ethologically oriented observers who have tracked infants with longitudinal recordings [5,16,20], but also by parents who have often responded to open-ended questions about their infants’ vocal sounds in the first half year of life with terms drawn from the common parlance to describe them, in English, vowel-like sounds (which in our technical terminology are called vocants), squeals and growls. Other languages also often have terms to designate the primary phonatory protophones [6].

Of course, additional infant sounds do occur (raspberries, ingressive sounds, whispers, and so on) though far less frequently, and many utterances combine phonatory characteristics of the three primary protophones. As the first year progresses, infant utterances begin to show more speech-like characteristics, and by the second half year, more complex syllables occur, culminating in canonical babbling, with well-formed syllables such as “ba”, “da”, or “na”, and often reduplicated sequences of these sounds (“baba”, “nanana”, and so on) [21–23]. The phonatory characteristics of vocants, squeals and growls persist throughout the first year, and even the most speech-like fully canonical sequences are produced with phonatory properties that allow them to be categorized as vocants, squeals and growls, just as in the case of precanonical protophones.

Research having tracked the occurrence of protophones has long posited that vocal play, in particular with the three primary phonatory types, involves clustered production of the individual types, a non-random occurrence that has suggested infants may be engaged in vocal practice, perhaps in an attempt to consolidate the categories themselves [16,22]. This reasoning is founded in the idea that the categories emerge from exploration, rather than being innate vocal givens or learned from parents. The three principal protophones are thought to be creations of the infant, and the fact that they seem to fall into particular categories is thought of as the result of exploration in a non-random landscape of phonatory possibilities, where Waddingtonian wells of attraction [24] tend to draw the exploratory activity into the three phonatory types. The pattern of development is reminiscent of the idea of quantal categorization in speech perception [25].

The fact that parents seem to notice the distinctions among the three has been proposed to be attributable at least in part to their non-randomness of occurrence across time [6]. An infant may be heard producing a train of squeals, for example, and although the great majority of protophones are vocants, the parent may notice the squeals as salient departures from the more typical vocants especially if they tend to be repeated or to occur in concentrated clusters. Parents engaging in face-to-face vocal interaction with their infants often attempt to elicit one of the three phonatory protophones by producing imitated versions of the sounds they believe to be in the infant repertoire. It is as if parents seek to confirm their infants’ emerging vocal competence in a game of mutual imitation, where parents most often initiate imitation by trying to draw infants into producing one of the protophone types already under their command [26].

So the non-random occurrence of the protophone types is seen as an important feature of infant vocal development in two ways. First, clustering may constitute a type of practice, where infants seek to firm up their control of vocal categories through intentional manipulation of each type. This manipulation and learning of new types appears to constitute a critical foundation for language, because one of the necessary requirements of language is the ability to adapt to using new vocal categories (intonational types as well as syllables and phonemes) that are language specific and to provide the basis for learning to use indefinitely large inventories of possible words and sentences [6]. Second, if the reasoning is correct, clustering provides a basis for caregivers to recognize infant progress toward language in the sense that they can thereby discern infant category control. This parental discernment would seem to begin with simple recognition of the primary phonatory protophones, a recognition forming a basis for elaborate vocal interchanges with infants. Later in the first year, parents recognize the appearance of repetitive canonical syllables, which results in an intuitive reaction whereby parents begin to “negotiate” with infants [27] over the possible meanings of particular syllable sequences (yes, you said ba, it’s a ball, say ba…) [28].

Why has there been so little research on vocal practice by clustering in human infancy?

Given the seeming importance of clustered production of vocal types in infants, it may be seen as odd that very little research has been devoted to quantitative determination of the extent of such practice or its course of development. As far as we know, only we and some of the collaborators of the University of Memphis Origin of Language Laboratories (OLL) have addressed the issue empirically and in each case at small scale. 1) In the Supplementary Material to one of our papers [4], we presented a demonstration using lag sequential analysis [29,30] of a tendency of 9 infants in brief laboratory recordings to produce the three primary phonatory types in non-random sequences where the identity of an infant utterance (for example, a growl) predicted the likelihood that the next utterance in sequence would be of the same type at a higher than chance level. 2) In a separate demonstration reported in the same Supplementary Material, recurrence quantification analysis [31] was used to illustrate the clustering of squeals in a sequence of utterances from an infant, that is, a tendency for few squeals to be present at the beginning of a particular recording session, but a large number of squeals to cluster at the end of the session. 3) In a separate paper [32], we reported “session effects”, whereby 3 infants showed a tendency to produce durational patterns in babbling that were notably different from one period (or session) of recording to another for the same infant, even within the same day.

Such session effects have been recognized throughout decades of longitudinal research on vocal development in infancy by the OLL, having required us to conclude that small-scale sampling by recording of infant vocalizations could not be expected to yield representative samples of any aspect of infant vocal patterns (see also [33]). From one period to another, a wakeful and alert infant can change from being completely silent to being very voluble, and during periods of volubility [34], from producing a very limited repertoire of vocal types to producing a wide array of types. With a comfortable baby, the most common protophones by far are vocants [4]. But we have long noted that periods occur where squeals or growls can abruptly take center stage, and when they do, an observant individual cannot help but take notice.

There are several reasons that vocal exploration, particularly the quantification of vocal clustering, has received limited attention in the study of infant vocal development. First, researchers have historically focused heavily on the anatomic and physiological aspects of infant vocal development, viewing the vocal products as showing gradual progression from immature and unstructured forms to mature speech [35–39]. While this perspective has merit, it tends to overlook the critical role of infants as exploratory agents in their own vocal development. This perspective has been influenced by longstanding views, from the concept of the "tabula rasa" in Western philosophy to Jakobson’s claim that prelinguistic vocalizations are random byproducts of biological inclinations [40]. These views are outdated, yet the tendency to underplay infant active involvement in vocal development persists.

Second, while some scholars have recognized and described vocal exploration in infancy, the precanonical stage of phonatory control development has received considerably less attention than the canonical stage and higher-level language development (e.g., words and sentences) in childhood (see review in [41]). Yet without the fundamental capability of phonatory control, articulatory development as manifest in canonical babbling would seem largely superfluous, since syllables overwhelmingly require phonation in their nuclei, and articulatory movements without phonation are largely soundless.

Third, in terms of the mechanisms of language development, social interaction and imitation have been the primary focus in the literature, with the seeming assumption that infants learn vocal categories by listening to parents, interacting with them vocally and imitating the existing baby sounds [42–44]. But in fact, only a small proportion of infant vocalizations are produced during vocal interaction, and imitation of parental sounds is extremely rare in early life [26,45]. In fact, it appears that the vast majority of apparent imitation by infants may actually be based on parental elicitation of the sounds parents know the infants already have in their repertoires, for example, squeals, growls, and vocants [28]. It is difficult to find evidence that infants introduce any new sounds into their exploratorily developed vocal repertoires as a result of listening to parental talk until late in the first year, when the first words begin to appear.

Lastly, most prior studies on infant vocal development have been conducted in a laboratory, for only a few minutes to about an hour [2,46,47]. Therefore, vocal exploration may often not have been salient enough to capture researchers’ attention during such constrained time periods, and especially since parents are often instructed to elicit vocalization from their infants during such recordings. However, with the widespread availability of all-day home recordings in recent time [48–50] more representative data are now being obtained from naturalistic environments.

Purposes of the present study

Our goal is to provide a large-sample quantitative assessment of the tendency of typically developing human infants to produce the three primary protophone categories in clusters, rather than simply randomly distributing the categories across segments of time. This assessment, taking advantage of a massive database of recordings described in Methods, will also offer perspective on the degree to which infants across the first year give evidence of vocal category development, which is a crucial capacity in language. The work should also provide perspective on vocal practice in infancy.

The present study targets two primary questions: 1) to what extent do infants tend to produce phonatory categories in clusters rather than randomly across segments in recordings, and 2) to what extent do clustering patterns change with age during the first year of life.

Participants

A total of 130 typically developing, English-learning infants participated in the present study (55% were male and 63% were designated as living in homes of relatively high SES as indicated by maternal education level). Participants whose data were analyzed in this study were recruited from September 2012 to October 2020. These infants were recruited in the Atlanta area as part of a much larger longitudinal study conducted by the Marcus Autism Center, Children’s Healthcare of Atlanta and Emory University School of Medicine (NIH P50 MH100029) investigating vocal development in infants who 1) had no family history of autism or other developmental disorders or 2) had an older sibling with confirmed diagnosis of autism and thus were at elevated likelihood for autism. All 130 infants included in the present study were classified as having no clinical features at 2 and 3 years after thorough evaluations by Marcus Autism Center expert clinicians. Among the 130 infants, 103 were classified at enrollment as being at low likelihood for autism diagnosis. We report data on all 130 after having verified that the 27 at elevated likelihood showed very similar proportions of recordings with significant growl and squeal clustering to the low likelihood group of 103.

The OLL contributed human coding of the recordings of the infants as part of a collaborative NIDCD longitudinal study between the University of Memphis and the Marcus Autism Center/Emory University (NIH R01 DC015108). All the recording/coding protocols and consent documents signed by all the parents were approved by the Institutional Review Boards (IRBs) of Emory University and the University of Memphis. The third author at Emory University had access to identifying information for all participants in the present study. The data for the present work are not secondary. The recordings were collected under the supervision of the third author in Atlanta, and the coding was conducted under the supervision of the last author in Memphis.

Recordings and the selection of 5-minute segments for coding

Language ENvironment Analysis (LENA) all-day recorders [51] were used to collect the audio data. The recorders are small and light enough (about the size of an iPod) for infants to carry in a pocket of a vest without any disturbance. Because LENA allows recording up to 16 hours a day with high audio quality (16 kHz sampling rate), it enables researchers to obtain representative data on infant vocalization and the auditory environment in the home. The recorder has been used routinely in a wide variety of research on early language development, starting with Zimmerman et al. [48].

Caregivers were instructed as in a variety of our prior studies on how to use a LENA device, including how to start, stop and pause recording. All recordings were carried out remotely in the home, once every month from birth to two years of age, and were scheduled as far as possible on the same calendar day of each month to ensure rotation of weekdays. Recordings were mailed out and retrieved via USPS priority mail. Each completed recording was uploaded through the LENA software to save the data. For further recording details see [52] for a description of the procedures that were used at the Marcus Autism Center, where the recordings were conducted.

A total of 1154 all-day recordings were collected, an average of 8.9 recordings per infant (range 4–12). The recordings were always available for coding in the OLL and have been accessed repeatedly from September 1^st, 2015 to the present. The human coding was conducted on 21 randomly selected 5-minute segments from each of the recordings. At the point of analysis, each available recording was assigned to one of six age groups as follows: 0–2 months, 3–4 months, 5–6 months, 7–8 months, 9–10 months, and 11–13 months.

Coders, training, and coding environment

Thirty-six English-speaking graduate students in the University of Memphis School of Communication Sciences and Disorders were trained and coded the recordings. They were trained in phonetic transcription in their program of study but were more intensively trained for infant vocalization research by the fifth and the last author regarding all the coding parameters, including the primary phonatory types (i.e., squeals, growls and vocants). The six- to eight-week training for the coding of infant vocalizations is described in greater detail in some of our prior work [3,52]. The coders met periodically across the training period for weekly one-to-two-hour lectures and presentation of audio examples drawn from the OLL archives of infant vocalizations. After each lecture session, they conducted practice coding on recording segments. At the end of the week, these practice sessions were reviewed with the trainers during the lecture sessions and as needed individual coders had individual sessions with the trainers to address discrepancies between the categorizations that had been made by the trainees and also by key codes that are available for many of the segments based on coding by the last author, who is the longest term researcher on vocal development among any of the OLL participants. Coders were required to meet standards of agreement with the key codes (to be within 10% of the counts on protophones) by the 6^th week of training or else additional training would ensue.

As a part of the training and providing a basis for coder agreement research on the three phonatory protophone categories, all the coders completed some practice coding on several of 9 full recordings where they coded all 21 randomly selected 5-minute segments independently of any other coder. These were not the research recordings on the 130 infants but were drawn instead from the same archives of LENA recordings at the Marcus Autism Center that produced the data on the 130 infants. These recordings are part of a set of recordings from the Marcus Autism Center utilized in the OLL for training of coders and about which agreement data are presented below.

Once trainees had completed training, they were admitted to the “coding team”, which typically consisted of about 12 individuals year-by-year, with a new group of trainees being recruited and trained in the fall semester of each year, replacing individuals having graduated that spring or summer.

A total of 24,234 5-minute segments were coded. The coders were assigned to groups of four infants, being blind to the age, risk status, and diagnostic outcome of the infants. They coded each recording completely (all 21 randomly selected segments in chronological order as occurring in the recordings from morning to night time) before proceeding to the next recording, which was randomly selected from among all the recordings from four infants assigned to each coder. Many of the coders were assigned to additional groups of four infants depending on their availability during the period of the research. The order of infants whose recordings were to be coded by each coder was also randomly assigned. Each individual coded all the recordings of an average of 4 infants (range 1 to 9 infants per coder).

Action Analysis, Coding, and Training (AACT [53]) was used for identification of each vocal type. AACT is a software environment enabling coders to label any kind of action in audio or video or both (see [4] Supplementary Material for details). Coders used keystroke or mouse-selected coding for each of the three protophone categories (squeals, vocants, growls), along with a few additional very infrequently occurring protophone categories (ingresses, whispers, raspberries, and non-phonated frication sounds) plus cries, whimpers, and laughs, while listening to each of the 5-minute segments. Coding was conducted in real time, so that each 5-minute segment took 5 minutes to code. Each coder responded to a brief questionnaire at the end of each coding segment, adding about another minute to coding time per segment. They answered the questions on a Likert-like scale from one to five in order to provide additional relevant information for each segment. For the present study, the key question concerned infant sleep. If the coder determined the infant to be asleep during the whole 5-minute segment, a value of 5 was assigned to that question, and the segment was then not included in the clustering evaluation for that recording.

Coding categories

The coding yielded counts of squeals, vocants, and growls, leaving aside many other possible vocalizations of infants that occur far less frequently and are not considered to be precursors to speech. All coding was conducted at the utterance (that is, the breath-group) level. Extensive additional coding category information as well as details on how squeals, vocants, and growls were defined, along with spectrographic illustrations, are provided in Supporting Information, in the initial section called Coding categories, definitions and coding criteria.

Data processing and statistical analysis of clustering

Coder agreement

To establish a foundation for assessing the reliability of coding across three protophone types, we conducted a comprehensive evaluation of coder agreement through various analyses (for details, refer to Supporting Information, Coder agreement). Our evaluation focused on a very large coding agreement set, comprising 21 segments from each of the 9 recordings, coded during the training period. The agreement set enabled us to obtain a very large number of intercoder agreement pairings because, on average, about 32 coders independently coded the same 21 segments for each recording. A second, though smaller, agreement study involved 9 coders, which was conducted toward the end of the coding by those individuals on 523 segments semi-randomly selected from the data reported in Results. It makes sense to assume coder agreement would be lower during the training period and higher after actual data collection. The higher agreement obtained during the second, though smaller evaluation, tends to confirm this assumption.

These evaluations including coefficients of variation and Spearman rank order correlations demonstrated that coder agreement was highly statistically significant (see Supporting Information, Coder agreement). Permutation tests conducted on the 9 recording agreement set further confirmed that coder agreement was more than adequate to justify the analyses of clustering reported below.

Percentage of recordings determined to show clustering

There was a considerable tendency for the infants to show significant clustering patterns. For squeals, 40% of the recordings showed significant clustering (p < .05) by the Fisher’s test, and for growls the value was 39% (p < .05). These percentages of recordings that met the significant clustering criterion for each infant at each age were computed to include the NA recordings (which of course were not analyzed by Fisher’s test) in their denominators, and consequently they supply an indication of squeal and growl clustering across all the recordings available for each infant. Since we only selected 21 segments from each all-day recording for coding (only about 15% of the available recording time), these values surely underestimate the percentage of days the infants actually engaged in some clustering of squeals and/or growls. The data thus estimate a lower bound on the amount of clustering of squeals and growls at around 40% of all-day recordings.

Furthermore, evaluating whether individual infants showed either significant squeal clustering or significant growl clustering on each recording, we found that 61% of recordings showed a significant amount of clustering for one or the other. So it can be concluded that even though we sampled randomly from a relatively small proportion of the day, infants usually showed some discernible clustering activity of protophones.

At the infant level, we evaluated all the recordings available for each infant and found that 87% of infants showed at least one age at which their recordings showed significant squeal clustering and at least one age with significant growl clustering. There was not a single infant who, on evaluation of all the available recordings for the infant, showed neither a significant case of squeal clustering nor of growl clustering. Only 8 infants (6%) had 3 or fewer recordings with significant squeal or growl clustering. In contrast, 35 infants showed 6 or more recordings with significant growl clustering and 30 showed 6 or more with significant squeal clustering. Some infants seemed clearly to cluster growls more than squeals (about 10% of infants showed at least 3 more recordings with significant growl clustering than squeal clustering) while some (about 8%) clustered squeals more than growls.

Percentage of recordings with significant clustering across ages

When the data were analyzed by age for each vocal type comparison, we found significant clustering patterns of vocal types across all age groups. Fig 1A–1C supply a summary of the data, indicating that clustering occurred at all the ages. The error bars represent 95% bootstrapped confidence intervals computed around the means for all infants at each age level. Infant data were averaged for all the recordings of each infant within the age interval prior to computing means and confidence intervals.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Clustering across ages: The figure shows means of statistically significant clustering for recordings from infants at each of 6 age intervals.

Values were computed at the infant level, such that all recordings for any infant within the age intervals indicated were averaged first, and 95% bootstrapped confidence intervals displayed as error bars were computed based on the averages for the number of infants who had recordings within those age intervals. a. Growls showed significant clustering for more than 30% of the infants at all ages. b. Squeals showed significant clustering for more than 30% of infants at all ages except the 3–4-month interval, where 27% of infants had significant clustering of squeals. c. In the final panel we display the proportion of infants who showed either significant growl or squeal clustering at each of the age intervals. From 48% to 69% of infants showed significant clustering of either growls or squeals at the various age intervals, and all the age intervals beyond 5 months revealed more than 60% of infants had significant clustering.

https://doi.org/10.1371/journal.pone.0299140.g001

In panels 1a and 1b the data show considerable occurrence of squeal and growl clustering in the very first months of life. Interestingly, the highest amount of clustering did not fall within the 3–4 month range, traditionally thought of as the period for vocal play in stage models of vocal development [16]. In fact, the 3–4-month interval showed the lowest mean values for both squeal and growl clustering. Squeal clustering showed a tendency to increase toward the middle of the year, peaking at 7–8 months. The squeal clustering pattern appeared to vary more across age than the growl clustering pattern, but in both cases, the final interval of the year, like the first interval, showed substantial clustering. Thus, a pattern of clustering occurred at all ages.

In Fig 1C, the data are presented in such a way that either squeal clustering or growl clustering was counted. Thus, each data point represents the proportion of infants whose recordings at the age in question showed either significant growl or squeal clustering. Consequently, the means are higher at each age than for either Fig 1A or 1B, illustrating the fact that infants often tended to produce recordings with clustering of squeals or growls but not both.

The confidence intervals supply information about possible significant differences across the ages. The apparent differences suggest, for example, a possible tendency for clustering overall (panel 1c) or clustering of squeals (panel 1b) to be more common beyond 5 months of age than before 5 months. Yet there are important provisos to offer about these apparent age differences, as we explain below.