On the evolution of human infant vocal patterns

Why is the human infant so inclined to explore vocalization? We and (independently) John L. Locke have proposed an “ultimate” answer, in the terminology of Tinbergen [51], an answer involving reflections on ancient hominin history. The work is intended to posit evolutionary conditions that fostered the emergence of vocal foundations for language, long before language existed. The proposal suggests that during the evolution of our species, humans came to be endowed with a tendency to take pleasure in playful, exploratory vocalization [52–55]. The reasoning relies in part on the altriciality (that is, the relative helplessness) of the hominin infant, a condition believed to have been caused by the narrowing of the hominin pelvis owing to our obligate bipedalism originating in ancient times and the consequent requirement that the schedule of development be revised to afford a smaller head at birth [56–58]. The whole term of development was thus apparently slowed in ancient hominins, yielding infants with greater need for long-term caregiver investment than in related species. In modern times the development of human infants to the point of self-provisioning is proportionally about twice as long as in other apes [59–61].

As the reasoning goes, the altricial hominin infant thus came under selective pressure to provide new kinds of fitness signals to caregivers, who then offered selective care to those infants whose signals were most effective. The vocal system, being in a sense largely free of other utilitarian requirements (unlike the hands, which are regularly used for manipulation and grasping), became a particular target for selection in response to altriciality. And thus, if the reasoning is correct, hominins became vocal fitness signalers, at first in infancy, but as evolution proceeded, the vocal system was modified to have far greater complexity and power and came to be used at all stages of life, routinely as a fitness signal, but also (and simultaneously during speech acts) as a tool for social engagement, planning, group coordination, and so on. According to the reasoning, language became possible eventually after additional capabilities evolved, but it has always been dependent upon the vocal flexibility that was the product of copious vocal fitness signaling evolved first in hominin infants.

This hypothesis is the only proposal on the table, as far as we know, to account for the initial split of hominins from their ape relatives in terms of vocal activity and vocal communication. The need for language cannot have supplied selection pressure for that split because language did not exist at that point. All naturally-selected changes must be accounted for in terms of pressures available at the time of the change and cannot be referred to possibilities that might occur in the future [62–65]. We have argued extensively that without the inclination and ability to produce vocalization flexibly, vocal language could not have begun to be evolved in the hominin line, just as we have argued that vocal language could not be developed in modern human infants without the vocal exploratory tendency—all acts of human language require vocal flexibility [55, 66–68].

It can, of course, be reasoned that the existence of language, as it began to emerge across hominin evolution could also have resulted, presumably gradually, in selective pressure on infant vocalization as an early language-like exercise, in addition to the pressures that yielded vocal fitness signaling. Thus there is reason to suspect that nowadays infant vocal activity and practice even in the first months of life does indeed support the development of language [69, 70]. This does not contradict the existence of an even deeper inclination for exploratory vocalization in the modern infant that was selected in ancient times. Our reasoning is thus compatible with at least two factors, one supported by adaptation for vocal fitness-signaling and one supplying preparation for language, in the emergence and maintenance human infant vocal tendencies.

The fitness-signaling hypothesis offers a reason, we have proposed, that human infants seem to enjoy exploratory vocal play. They appear to have been endowed with a tendency to treat the voice in much the way all primates treat playful activity with the hands. No other ape shows the tendency to produce purely playful vocalization to our knowledge [41], and yet the human infant tends to produce exploratory vocalization every day at prodigious rates.

But how often does this playful activity coincide with or occur independently from vocal turn-taking? The availability of all-day recording technology opens the door to more representative sampling to assess this question quantitatively by evaluating the relative amounts of endogenous vocal production and vocal turn taking in human infants.

Canonical babbling in vocal play and social interaction

The reasoning about the fitness-signaling hypothesis responds to Tinbergen’s ultimate questions about why capabilities and inclinations exist (the two questions regarding evolution and survival value), but it does not respond to Tinbergen’s proximate questions (the two questions regarding development and causation mechanism) [51]. There is a well-developed and growing literature on the proximate questions that takes particular note of the emergence of increasingly advanced protophones across the first two years of life [27, 69–71].

A key indicator of especially advanced protophone production, and conceivably a particularly salient fitness signal, is canonical babbling, which consists of syllables that are often so well-formed that they can constitute speech if used in the right circumstances [30, 72]. Canonical babbling has been long established as a robust stage of prelinguistic vocal development, occurring prior to the emergence of early words, and constituting a necessary foundation for extensive vocabulary development [73–75]. To our knowledge, there is no published research evaluating the role of exploratory motivation in infants’ production of canonical babbling and no direct evaluation of the extent to which social engagement in vocal turn taking affects it. In the present research, we observed canonical babbling in naturalistic settings recorded longitudinally for infants in the first year. Segments extracted from all-day home audio recordings were rated for levels of infant independent exploratory vocal play and vocal turn taking, allowing measurement of the degree of social and non-social vocal activity and thus, exploratory and social motivations, respectively. The research offers the opportunity to assess not only the quantity, but also the quality of infant vocalizations as reflected in canonical babbling syllables as a proportion of all protophone syllables.

In line with the fitness-signaling hypothesis, it seems reasonable to expect for infants to produce their most advanced vocal forms during periods of maximal caregiver attention, i.e., during social interaction. Empirical evidence has been presented that caregivers are keenly aware of their infants’ developmental capabilities, including their vocal sophistication [76–78]. Higher rates of canonical syllables (as opposed to less advanced protophones) during social interaction suggests a motivation to display advanced protophones when caregivers are attentive. Furthermore, it seems reasonable to expect high levels of exploratory vocal play to be accompanied by particularly high levels of canonical babbling because exploratory activity reveals infant attention to development of a sound system, and because the attention of caregivers is surely drawn to the quality of protophones during periods where the infant is particularly active vocally.

Testable propositions

Our study then addresses infant vocalization in both social and nonsocial circumstances, observed naturalistically in all-day recordings, which have made it possible during more than a decade to assess vocal patterns with greater ecological validity than was possible in the past [79–83]. The data were coded in 5-min segments drawn from the recordings. In one kind of analysis, we assessed the extent to which production of protophones appeared to be driven endogenously, as opposed to being elicited by caregivers. The dependent variables for this kind of analysis were 1) human coder estimates of the amount of protophone production during each 5-minute segment that constituted independent, presumably endogenous vocalization, which we shall refer to as vocal play, or “VP” [48], and 2) the amount of protophone production in the same 5-minute segments that constituted vocal interaction with caregivers, which we shall refer to as turn taking, or “TT”.

To be clear, the VP and TT categories were not exclusive within coded segments; some VP and some TT could occur in the same segments. Only vocalizations deemed to be protophones for the present work (squeals, vocants, or growls) were judged to constitute either VP or TT; other less frequently occurring protophones such as non-phonated sounds or ingressive sounds were not included. Importantly, some of the vocalizations deemed to be protophones included canonical syllables and some did not. Also, VP and TT were judged by coders to occur only in cases where the recognized protophones were judged to be prominent (loud and/or long) enough that a nearby caregiver might have noticed them. Further, protophones judged to consist of whining (interpreted as “complaints”) were not allowed to be treated as VP. These and other coding constraints are elaborated below under Coding Categories.

In additional analyses, the dependent variable was the frequency of occurrence of advanced infant vocal forms, as reflected in the canonical babbling ratio (CBR), the number of canonical syllables divided by the number of all protophone syllables, during VP and TT. Evidence has been presented suggesting that social motivation drives the production of canonical babbling and that infant-directed speech promotes it [84–86], but there has been no research to our knowledge quantifying the relative extent of canonical babbling in VP and TT for infants in the natural environment of the home.

The research focused on data at three ages during the second half-year of life (7.5, 9.5, and 12 mo). Earlier ages were not addressed since earlier protophones are almost entirely non-canonical and cannot yield useful information about CBR nor its relation to VP or TT.

Comparison of vocal play and turn taking

We know of no previous research that has addressed relative rates of VP and TT based on all-day recordings. We suspect that the tendency for infants to engage in VP as opposed to TT may be especially high in the naturalistic environment of the home as opposed to its occurrence in laboratory recordings [31]. Consequently, we propose:

Proposition 1: At all Ages, coded ratings will indicate infants produce VP more often than TT.

To our knowledge, there has been no attempt to directly evaluate a possible role for endogenous VP on the extent of infant canonical babbling. We reason that there may be a tendency of infants to explore their most speech-like utterance capabilities when they are most actively involved in VP, times during which they may be most attentive to their own vocal product, stimulating awareness of categorical growth and calibrating their vocal capacities. Further, if, as we suppose, infant protophones have been naturally selected as fitness signals, periods of VP may tend to attract attention from caregivers, the persons who apply the selection pressure on the nature and growth of infant vocal patterns according to the fitness-signaling hypothesis. Thus, we predict:

Proposition 2: Higher CBRs will occur during recorded segments with Some VP compared to those with No VP.

Prior research has also suggested that caregiver interaction influences infants to produce canonical babbling, suggesting an important effect of caregiver interaction on the growth of the speech capacity [84, 87]. There is also evidence that the amount of speech-related vocalization is stimulated by vocal interaction, evidence relying on the LENA algorithms applied to all-day recordings [85]; but the LENA analyses, based on automated categorizations, have not addressed canonical babbling specifically, only differentiating speech-related vocalization from other sounds such as cries or vegetative sounds. So far, to our knowledge, there has been no attempt to directly test a possible influence of TT on canonical babbling based on samples from all-day recordings. The naturalistic circumstance in the home is the most representative one to allow testing the expectation that interactions with parents actually stimulate infants to display and perhaps develop their most advanced vocal forms. It makes sense that if protophones are fitness signals [52, 55], human infants may have been naturally selected to produce more advanced vocal forms when caregivers are presumably paying attention to them. Thus, we predict:

Proposition 3: Higher infant CBRs will occur during recording segments with Some TT compared to those with No TT.

Finally, it seems likely that if the fitness-signaling hypothesis is on target, infants should be motivated to generate their most speech-like vocalizations when caregivers are most attentive, that is, during the circumstance of vocal interaction. Independent of the fitness-signaling hypothesis, the human infant shows many signs of being socially motivated, and consequently during vocal interaction, it makes sense that they would display their most advanced vocal forms. Thus, we make a prediction that to our knowledge has never been considered in prior research:

Proposition 4: Higher infant CBRs will occur during recording segments with Some TT compared to Some VP.

Participants

As part of an NIH-funded Autism Center of Excellence conducted at the Marcus Autism Center in Atlanta, Georgia (in affiliation with Emory University), more than 300 families of newborn infants were recruited via flyers, advertisements, social media and community referrals to participate in a longitudinal sibling study of development across the first three years of life. Infants were recruited as being either at high risk (HR) or low risk (LR) for autism. They were deemed HR if they had at least one older biological sibling with a confirmed autism diagnosis, and LR if they had no familial history of autism in 1^st, 2^nd, or 3^rd degree relatives.

The present study focuses on LR infants only, selected among those for whom all the data relevant to the present study have been coded completely and evaluated for coder agreement and for whom no clinical features (e.g., autism, language delay, subthreshold ASD) were detected at 24- or 36-month evaluations. The sample includes 40 LR infants, with 9 LR infants having been excluded because there was suspicion of some developmental anomaly at the time of the final evaluation. Thus, although the broader study focused on autism risk, the present analysis focuses entirely on a sample of verified typically developing infants.

In such observational research, there are many potentially influential variables that cannot practically be controlled or evaluated quantitatively. But two demographic variables for which we have relevant data are of potential interest: socio-economic status (SES) and sex. We conducted precursor analyses on these to provide additional perspective and to provide a rationale for simplifying our analyses by excluding them. Sex and self-reported maternal education (as a proxy for SES) measures were balanced in subject recruitment to the extent possible. Low/high SES groups were based on a median split of maternal education in the entire cohort (range 13 to 25 years, median 18 years). The sample for the present study was somewhat biased toward males (23 of 40) and toward higher SES (27 of 40). The latter pattern is common in longitudinal research because families of higher SES tend to volunteer for and maintain enrollment in longitudinal research more frequently.

Both SES groups and both sex groups exhibited very similar patterns as the whole group with regard to the relations between VP and TT (all far higher VP than TT) and with regard to the CBRs as a function of VP and TT (see S1 Text for details and additional comparisons). Therefore, we do not include maternal education/SES or sex as independent variables in the analyses below, thus simplifying the analyses.

Audio recordings

Families were asked to complete audio recordings once a month from 1–36 months of age. The present study used data collected between 6.5 and 13 months of age to represent the typical range of expected onset for and high infant activity in canonical babbling. These data were grouped into three age ranges for analysis and are labeled in the tables and figures below with reference hereafter to the approximate mean age within each group: 6.5–8.49 months (“~7.5 months”), 8.5–10.49 months (“~9.5 months”), and 10.5–13 months (“~12 months”).

Audio recordings were completed using battery powered LENA recording devices [80] secured inside the pocket of a special vest or clothing item with button clasps. The devices can record up to 16 hours of audio per charge. They have a 16 kHz sampling rate and given the low mouth-to-microphone distance (~10 cm), usually offer excellent audio quality for human coding of recorded material.

Once a month, parents were provided with a LENA recording device and were supplied regularly with appropriately sized clothing for their infant to wear throughout the day, as well as full instructions on how to carry out recordings. The device was returned to the research project staff at the Marcus Autism Center each month following recording days for data processing. Each family completed ~5 total recordings (range: 3–7) across the ages studied, with an average recording time of approximately 11 hours per day.

We look forward to a time, hopefully not too far into the future, when all-day recordings with both audio and video can be made available for such research. Since we assume both audition and vision form the basis for caregiver judgments about infant wellness based on vocalization, we also assume video plus audio will offer a preferable basis for coding all the relevant features of infant vocalization.

Coders and segment selection

The coders were 15 graduate students in the School of Communication Sciences and Disorders at the University of Memphis. They were all trained for coding as specified below. Each of the students was semi-randomly assigned to a set of infants (including all their longitudinal recordings) from among the first 100 recorded in the broader study of infants at risk and not at risk for autism, but the coders were never given diagnostic or clinical information about the infants. The number of recordings coded by the individual coders varied widely (from 5 to 22).

In accord with the standard protocol for all-day recordings in the Origin of Language Laboratories (OLL) of the University of Memphis, where all the human coding was conducted, twenty-one 5-minute segments were randomly extracted from each of the 196 all-day recordings, for coding in “Phase 1” of infant utterances. Following coding of each segment, coders completed a questionnaire to obtain information about the infant’s environment (see S3 Text). After Phase 1 coding was complete, the software (Action Analysis, Coding and Training, AACT) used in the OLL [88] automatically selected eight of the 21 segments from each recording on the basis of the coded information and questionnaire responses. The 8 were selected based on several criteria: 1) relatively high infant volubility of infant utterance coding in every case, 2) in some cases based on relatively high infant-directed speech (IDS, as judged during the Phase 1 questionnaire), 3) in other cases based on infants being alone or not engaged in vocal interaction (again according to the Phase 1 questionnaire), and 4) in the remainder of the cases were selected based on maximum volubility alone. The segment selection algorithm for “Phase 2” coding is described in detail in S6 Text. These 8 segments were then coded in Phase 2 for infant syllable and IDS counts (further described in Coding Categories below), with another questionnaire following coding of each segment.

The final coded dataset for the present study on the 40 typically developing infants at three ages thus consisted of 1,576 five-min segments (mean = 39.4 segments/infant across the three ages; mean = 13.4 segments/infant at 7.5 mo, mean = 12.4 segments/infant at 9.5 mo, mean = 14.8 segments/infant at 12 mo) extracted from the all-day recordings. For analyses below, the number of segments was actually 1,572, eliminating 4 segments where the infants produced no protophones at all.

The coding outcomes for these segments combined from Phase 1 and Phase 2 yielded the following numbers of the present study’s most important coded categories, which are defined in the next section: >57,000 infant protophones, including >108,000 infant syllables, of which >13,000 were deemed to be canonical syllables; other speakers produced >18,000 utterances directed to the infants according to the coders and >50,000 utterances not directed to the infants.

One might ask if the relatively high volubility (7.4 protophones per min) of the 8 selected segments per recording might bias the sample to show higher VP than the larger set of 21 randomly selected segments per recording, which had lower volubility (4.9 protophones per minute). In fact, the correlation between volubility and VP (some VP vs no VP) was negative in the selected sample (ρ = -.022) while the correlation between volubility and TT (some TT vs no TT) was positive (ρ = .132). The pattern seems to contradict the suggestion that amount of VP could be overestimated with respect to amount of TT because of the selection of relatively high volubility segments. One might also suggest that a difference in volubility might explain any difference found between CBR in segments with some VP as opposed to some TT. Indeed, volubility was higher in segments with some TT (8.4 protophones per min) than in segments with some VP (7.3 protophones per min). Volubility could then be thought to drive a tendency for high CBR in TT if canonical babbling is indeed associated with volubility.

Coding categories

One might imagine the best approach would be automated analysis of the whole recording set, obviating the need for random selection of a subset of recording segments and bypassing human coding. But there are no existing automated analyses, to our knowledge, that differentiate reliably among the categories of interest. In all cases, anyway, the gold standard for evaluation of automated methods is human coding. The very goal of automated analysis of infant vocalizations is to mimic human perceptions of speech-like quality.

The categories for human coding of infant vocalizations utilized in the present work are described in detail in the supplementary material to prior work from our laboratory [33], where spectrographic displays are presented to help illustrate the definitions, which are provided in S2 Text. These categories are utilized widely for research in the OLL at the University of Memphis, where all the present coding was conducted. Here, we summarize the definitions for the categories of interest in the present work.

Infant “utterances” are defined as breath-groups of phonation [89]. Thus, inhalation, or a pause perceived to leave room for inhalation, is treated as a boundary between utterances. “Fixed signals,” that is, cries, whimpers, and laughs are coded, but are distinguished from the protophones, the presumed precursors to speech, which include most prominently the phonatory categories of squeals, growls, and vowel-like sounds (vocants), and far less frequently occurring sounds (whispers, raspberries, ingressive sounds, clicks, etc.). It is important to acknowledge that ~15% of protophones are perceived auditorily as manifesting negative affect [90]. We refer to these protophones as “whining”, which is distinct in our scheme from crying or whimpering. A questionnaire item (see below) provides an estimate from coders on how much whining (protophones manifesting complaint) occurs in each segment.

Protophones occurring at very low intensity—so low that coders judge them to be below the normal threshold of caregiver attention—are also not coded. The distinction between cries and whimpers (both of which are treated as naturally-selected signals of negativity or distress) is formal in this research and is defined in detail, along with spectrographic exemplars, in prior studies from our laboratory [33, 91].

Vegetative utterances (burps, coughs, sneezes, effort grunts, etc.) are not coded. In our system “grunts” are always required to have a sharp glottal onset accompanying some kind of movement or strain that involves a glottal hold and diaphragmatic tension to impose rigidity on the internal torso—the effort grunt occurs at the release of the glottal hold, and is therefore considered an artifact of this movement. We do not use the term “grunt” in our coding system to refer to utterances called “attention grunts” or “communicative grunts” in a widely recognized body of work on early vocal communication by McCune et al. [92]. Instead, such sounds were coded as “vocants” for the current work. It should be noted however that only a subcategory of vocants, termed “quasivowels,” see [32], would satisfy the definitions of “attention grunts” or “communicative grunts” in McCune et al.’s work.

During Phase 1, protophones, cries, whimpers and laughs are coded in real-time for each segment, as they are in the vast majority of OLL projects. During Phase 2 coding, each infant syllable within any protophone is coded as canonical or non-canonical, again in real-time. The notion “syllable” is based on perceived rhythmic beats in infant utterances, counted in a way that is analogous to the auditorily-based counting of syllables in adult speech. The notion of the canonical syllable has been utilized widely in research on human infancy since its introduction more than four decades ago [72, 74, 93, 94]. Roughly, a canonical syllable consists of at least one well-formed, supraglottally articulated consonant-like element and at least one well-formed nucleus or vowel-like element (e.g., ma, da, etc.). Reduplicated examples of canonical babbling sequences, among the most salient indicators of infants’ having reached the “canonical stage,” may be heard, for example, as “baba” or “nana.” Non-canonical syllables are typically composed of vowel-like nuclei only. Utterances can, of course, consist of multiple syllables.

Caregiver or other speaker “utterances” are bounded differently from infant utterances during OLL coding, since grown-ups often produce long phonated sequences without inhalation, much longer than the protophones of infants. Hence, the breath-group rule used for infant utterances is replaced for caregiver utterances with a different rule. A perceived pause or perceived sentential intonational boundary is treated as a boundary between caregiver utterances. Caregiver utterances and other-speaker utterances are categorized in real-time during Phase 2 as “infant-directed” (that is, directed to the infant wearing the recorder) or “other-directed.” The distinction is usually easy to make because of the common use of “baby register” [95] or “motherese” [96] in speaking to infants and because the semantics of caregiver utterances usually makes it abundantly clear whether an infant is being addressed.

In both Phase 1 and Phase 2, coders respond to questionnaires immediately after coding each segment. There are 34 questions in all, 17 for each phase (see S3 Text). The questions of most relevance for the present work are: Phase 1, question 1 on infant-directed speech (IDS): “Did any other person talk to the baby?” Phase 2, question 1 on whining: “Were any of the infant’s protophones used to complain?” Phase 2, question 3 on VP: “Were any of the infant’s protophones purely vocal play or vocal exploration (not social, not trying to get something, etc.)?” And Phase 2, question 4 on TT: “Were any of the infant’s protophones used in vocal turn taking with another speaker?” Detailed instructions accompanying the questions are provided in a formal instructions protocol (see S3 Text).

The questionnaire items of interest in both Phase 1 and Phase 2 invoke a 5-point Likert-scale for response to the relevant questions, e.g., for IDS, “Did any other person talk to the baby?” The scale aligns to the following designations: 1 = Never, 2 = Less than half the time, 3 = About half the time, 4 = More than half the time, 5 = Close to the whole time. For example, a TT rating of 5 is to be applied to segments where a caregiver is judged to be speaking to the infant and the infant to be responding with protophones in back-and-forth vocal interaction throughout the whole 5-minute segment (although no rating of 5 was ever assigned to a segment with regard to TT in the data for the present paper). Segments with a VP rating of 5, likewise is to be applied to segments where the coder perceived the vast majority of infant protophones across the segment to be playful and/or exploratory and not directed to another person (a rating of 5 was not uncommon for VP). The subjective Likert Scale response system is intended to simulate the kinds of judgments caregivers make (we presume often unconsciously) regarding the relative rate of occurrence of various functions of vocalizations produced by infants throughout the day.

Vocal Play (VP) is defined, as inspired by Stark [48], to refer to infant protophones produced independent of social interaction or any attempt on the part of the infant to initiate an interaction, as in calling to the caregiver. Often in VP, infants appear to be manipulating or practicing parameters of which speech sounds are composed (pitch, duration, loudness, and spectral characteristics) without any social intention.

Coders are instructed not to treat protophones that are also interpreted to be “complaints” (whining) as instances of VP, since whining can be thought of as motivated by discomfort or frustration rather than exploratory interest, and it seems clear that some whining is socially directed. The first questionnaire item in Phase 2 specifically asks coders to designate how many of the protophones in the segment were complaints, the illocutionary category indicating whining. This question is intended to supply an additional method of focusing coder attention on designating VP (the third question) only in cases where infants are not expressing distress, and indeed the coders of the present data seemed to abide by the instruction since they rarely rated segments (<0.5%) in such a way that the sum of the VP and complaint ratings was greater than 6—such a sum could be taken to mean the coder had assigned too much time in the segment to either VP or complaint.

Turn-Taking (TT) (Phase 2, question 4) is defined to refer to infant protophones produced during audible back and forth of vocalizations and speech between the infant and another person, respectively (i.e., caregiver, sibling, etc.). TT in this definition requires that the infant be perceived as responding to infant-directed speech, not merely vocalizing during caregiver talk, which is often directed to other speakers. Coders are encouraged to use an intuitive definition of the notion “conversation”, that is, a period during which parties are perceived to produce and sometimes maintain a mutual exchange of vocalizations. The perceived end of a conversation often appears to correspond more or less to a five-second rule; that is, any period of five seconds or more during which no vocalizations occur is typically deemed to be a boundary between conversations, and the next occurring vocalization is treated as potentially starting a new conversation.

Additional coding criteria for VP and TT

Cries and whimpers, which are also coded in Phase 1, are never treated as either VP or TT in Phase 2, since such sounds are not protophones at all. Similarly, laughter is not categorized as protophone material and so is not treated as either VP or TT.

The coders are instructed to ignore (i.e., not code at all and not include in judgments of VP or TT) any vocalization of either the infant or any other speaker unless the vocalization is salient in terms of amplitude and/or duration. In essence, utterances are not coded if their saliency is so low that the coder deems them to be not likely to have been noticed by the infant (as a listener to other-speaker utterances) or by other speakers (as listeners to the infant). Also, vegetative utterances (coughs, burps, etc.) are not coded, nor are effort grunts.

The goal of these procedures is to limit the judgments of VP to utterances deemed to be voluntary precursors to speech, not inspired by discomfort or desire to attract attention, and of sufficient saliency to be noticed by a nearby caregiver. Also, the saliency criterion limits counting of speaker utterances other than the infant to those that seem most likely to have potential effect on the infant.

VP and TT, in accord with our definitions, are in principle independent of each other—no infant utterance can properly be treated as both. Thus, VP is not socially directed and excludes socially interactive vocalization even if the interaction is deemed to be playful. Put another way, any protophone produced in vocal interaction is deemed TT, but not VP, even if the interaction is thought to be playful. In addition, some protophones produced in the absence of vocal interaction, especially those produced with low intensity, may be judged as pertaining to neither TT nor VP, at the coder’s discretion.

Although TT and VP are in principle mutually exclusive, a particular segment can of course include both TT and VP at different points in time. We have previously discussed the potential for VP to function as a catalyst for parent-initiated social interaction [31]. If, for example, the infant is playing with a new sound, parents hearing the playful activity may repeat this sound to the infant, initiating a back-and-forth vocal sequence. Thus, in segments with high infant vocal activity containing both interactive and non-interactive protophones, a designation by the coder of Some TT and Some VP can occur. Thus, a rating of 3 for both would suggest that about half the protophones in the 5-minute segment were produced in TT and that about half of the others were VP. The coders assigned ratings for VP+TT of > 6 on a particular segment in only 6.5% of the coded segments. 62% of the ratings for individual segments were < 6 for VP +TT, suggesting that coders judged many protophones to be neither VP nor TT. About 20% of segments were judged to include at least some complaint protophones, which were not supposed to be assigned to either VP or TT, and these segments seem to have contributed to the fact that far fewer than half the segments were given VP+TT ratings as high as 6. But there appear to have been additional cases where coders chose not to treat some protophones as either VP or TT, perhaps simply because they were not sufficiently confident in making the relevant judgment.

A recurring difficulty for coding occurs in cases where multiple speakers (sometimes as many as five or six in these recordings) are involved in a segment. Sometimes infants are silent during such periods, but on occasions where they do vocalize, it is often ambiguous whether they are trying to break into a conversation (bidding for attention, presumably engaging in neither VP nor TT), whether they are trying to respond to utterances that are not directed to them (attempting to engage in TT even without IDS), or whether they may simply be stimulated to vocalize in a playful way in the context of group vocal activity (which the coders could conceivably interpret as VP). Coders sometimes choose TT and in other cases VP to characterize such infant utterances. On yet other occasions they appear to treat the utterances in such segments as neither TT nor VP. The choice is left up to the coders as intuitive judges, in the same way that we presume caregivers make implicit judgments about infant motives.

Syllable coding

In Phase 2, the 8 selected segments for each recording are coded in real-time for infant canonical and non-canonical syllables. A keystroke for each syllable provides the data. Listeners identified a total of 13,775 canonical syllables and 94,269 noncanonical syllables across the segments of the present study. To measure the emergence of advanced vocal forms, a canonical babbling ratio (CBR) was calculated as the total number of canonical syllables divided by the total number of syllables in each segment. Means and standard deviations of CBRs were calculated for each infant at each age. Three of the families did not complete a monthly recording at 7.5 months, and one did not complete a recording at 12 months. In cases where there were multiple recordings within an age for an infant, the means and SDs of these recordings were averaged prior to analysis.

The intuitive coding principle

Trainees for coding in the OLL are encouraged to make judgments intuitively. We have long argued that normal human listeners must have been evolved not only to recognize speech and speech-like sounds, but also to be able to make reliable judgments about infant purposes in vocalization [32], for example, whether an infant is turn-taking with them, complaining, or instead comfortably exploring vocalization independent of interacting with anyone. As parents, human adults would be, we argue, at a substantial disadvantage in competition with other parents if they could not recognize such sounds and the functions infants serve with them, because they would not be able to adjust to infant needs and communicative capabilities. Our training procedures, then, attempt to help coders bring to conscious awareness their inherent capabilities to perceive infant sounds and to judge their purposes, providing a categorical terminology for describing the observed events that presumably fit well with naturally available human intuitions; see [31, 32] for additional theoretical perspectives on making intuitive judgments of infant vocal functions.

Coding training and procedures

The six-to-eight-week coding-training procedure of the OLL is detailed in a recent publication from our laboratory [33]. For the present work, coders were blinded to all diagnostic and demographic information associated with each infant recording throughout the coding process. The training involved example presentation and supervised coding of segments from all-day recordings of infants from various demographic conditions, but none of the training segments were a part of the analyzed dataset for the present study.

Coders in the OLL are trained in phonetic transcription in their regular program of study in speech-language pathology. Usually, 6–10 trainees are initiated at the beginning of the first semester of a school year, with a two-hour introduction to infant vocal development using audio and audio-video examples presented primarily by the last author, a long-term investigator of human infancy. All other training personnel are required to have passed the same training and to have had considerable experience in coding infant vocalizations. Trainees are also introduced at the beginning to AACT [88], our coding software environment, facilitating a wide variety of coding and analysis tasks, and described in detail in the Supplementary Information to a prior publication [97]. Coders then engage in several weeks of practice coding in AACT with recorded 5-min segments of the same sort they will ultimately code for the research. They meet with the last author at least weekly in a group for supervised review of their coding. In addition, each coder meets individually with the last author for coding review. In order for trainees to be admitted to coder status, they are required to meet preset agreement standards for all the coding categories that are the focus of the present study. For details see Supporting Information to a prior publication [33].

Coder agreement

Inter-rater agreement was examined for VP ratings, TT ratings, and CBRs in two ways: In Method 1 we drew coder agreement data from two rounds of coding on 105 five-minute segments extracted from LENA recordings of infants at the same ages as in the present study, coded by 7 of the same 15 graduate student coders who provided the analyzed data for the present study. The work followed the same coding protocol used in the present study and in prior efforts [33, 34, 97, 98]. Each of the 7 individuals had coded a subset of the 105 segments, and for the agreement effort, each was assigned to code a different subset assigned to a different coder originally. All these 15 have now graduated and are no longer available as coders.

In Method 2 we conducted a new, special re-coding of 60 five-minute segments from the present dataset of recordings, where all 15 original coders were represented, each having coded 4 segments of the 60, and where 4 new coders, still available in the project, re-coded all 60 segments, being blind to the original coding. The 60 selected segments were balanced to represent all three ages (20 from each). In addition, under Method 2, the four new individuals coded 60 segments that were randomly selected from the 21 Phase 1 segments for each of the recordings. Among those 60, fourteen had also been selected by the AACT algorithm for Phase 2 coding, and consequently the total number of segments coded by the four new individuals for the agreement effort was 106 rather than 120. Below we summarize the agreement outcomes, and for details see S4 Text.

Method 1 outcomes: Correlations between segment-level outcomes for the original coding by the 7 individuals and the blind second coding by the 7 individuals reassigned to work on different subsets of the 105 were weak to moderate in magnitude though highly statistically significant for VP, r = .371, p < .001 and TT, r = .373, p < .001. The correlation for CBR, was large r = .872, p < .001. The lower correlations for VP and TT are expected given the difficulty of recalling the events of any 5-min segment at the point of the questionnaires along with the subtlety of making decisions about how to rate segments on the 5-point Likert scale. In contrast, the registration of canonical and non-canonical syllables is done in real-time while listening to the 5-minute segments with keystroke coding of each syllable and has proven to yield high agreement in past research as well [99]. Another way to assess agreement on VP and TT for the present purposes is to consider the level of agreement in terms of mean ratings. The VP ratings were only 8% higher in the re-coding than in the original, and the TT ratings were only 2% higher.

Moreover, because this agreement factor corresponds to the outcome data, we can conclude that the mean VP rating was more than 2.5 times higher than the mean TT rating in both the original and the agreement coding. Thus, the agreement data unambiguously suggest that if a different set of coders had supplied data on rates of VP and TT, the outcome conclusion regarding the rates of VP and TT would have been the same: VP occurred at a much higher rate than TT.

Method 2: Correlations for the 60 segments that were coded in their entirety by the four new coders with respect to the coding done by the original 15 coders (each of whom coded a subset of the 60) were similar to those found with Method 1. As with Method 1, the correlations (averaged across the four new coders with respect to the original 15 coders) were weak to moderate in magnitude though highly statistically significant for VP and TT (for VP, r = .477, p < .001 and TT, r = .359, p < .005). The correlation for CBR, was again large r = .714, p < .001. And again, the average VP and TT ratings were similar to the original ratings (VP 2% lower, TT 18% higher).

As in the case of Method 1, the agreement data conformed to the data from the original coders on the outcome. Both the new coding (averaged across the four individuals) and the original coding yielded much higher VP than TT values (new coders 2.3 times higher, original coders 2.9 times higher). Again, the agreement data unambiguously suggest that if a different set of coders had supplied data on rates of VP and TT, the outcome conclusion would have been the same: VP occurred at a much higher rate than TT.

For the 60 randomly selected segments, original coding was not available for Phase 2 (Phase 2 segments had been selected non-randomly by the AACT algorithm, see above), but the four new coders could be evaluated with regard to their agreement with respect to each other. Their correlations with each other across the six possible pairings of four coders were moderate and highly statistically significant for VP, r = .57, p < .001 and TT, r = .45, p < .001. The correlation for CBR, was again large r = .77, p < .001. The coders showed mean ratings that differed from each other by an average of 3% for VP and by an average of 20% for TT. And again, the difference between the mean VP ratings was substantially higher than the mean TT ratings for all four coders (VP on average 2.4 times higher). Thus, the agreement data on the randomly selected segments suggest that the higher rate of VP than TT was not a product of the non-random selection process for Phase 2 segments.

In all cases the correlations between agreement and original coders were highly statistically significant, and consequently all the statistical comparisons associated with the propositions were justified. The danger when coder agreement may be thought to be low is not Type I error, because coder disagreement introduces noise into the data, and consequently the danger is Type II error, failure to recognize real differences [100].

Statistical approach

The data we present here are complex especially because they show an imbalance of amounts of VP and TT (VP far more common), and because they show a complex arrangement of segments that were categorized as having 1) both VP and TT, 2) neither VP nor TT, 3) some VP but no TT, or 4) some TT but no VP. Consequently, interpretation of statistical analyses requires careful thought, and we have chosen to approach the matter eclectically. In each case, we have provided raw data with computed mean comparisons, effect sizes, and in some cases t-tests.

We used Generalized Estimating Equations (GEE) [101] for some of the more formal analyses involving Age as a factor. GEE is an advanced form of modeling that is well-suited to longitudinal data, because infants are not independent from themselves across ages. ANOVA as a method to evaluate longitudinal data violates the independence assumption. GEE is non-parametric and requires no independence nor normality assumption, accounts for correlations across variables and participants, and projects data in such a way as to correct conservatively for missing data. One of the advantages of using GEE analysis in the present case is that the imbalance in amounts of VP and TT as well as overlap of cells as seen in crosstabulation of VP and TT (see Table 1) do not adversely affect the analysis. The GEE method tends to yield fewer significant results than traditional approaches to analysis, so we have provided analyses by the conservative GEE approach where possible.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Distributional statistics for VP and TT.

The data revealed extremely different distributions of VP and TT for the 40 infants. All the data were calculated based on questionnaire responses of coders for the 1572 five-min segments. A. More than 90% of segments were judged to have some VP, while only 20% of segments were judged to have some TT. B. Crosstabulation showed that the vast majority of segments (74%) were judged to have Some VP and No TT. Only 1% of segments showed Some TT and No VP. C. The data breakdown in terms of the individual 1–5 ratings provided by coders for each segment showed that there were no segments judged to have TT occurring throughout the segment (a rating of 5), while VP was rated to have occurred throughout 31% of segments. Among the segments judged to have some TT, the vast majority were deemed to have occurred less than half the time (a rating of 2), while ratings suggested more than ¾ of the segments had VP occurring during at least half the time.

https://doi.org/10.1371/journal.pone.0279395.t001

Measurement limitations

The coding approach we used is founded in the notion that human judgments must be the gold standard for any measure of vocal development, because human caregivers impose the ultimate naturally selecting constraints on the infant vocal system. In any case, there is no automated method to date to replace human coding for the measures at stake here. Of course, one might imagine a more reliable approach to human coding for VP and TT as well as for CBR, an approach that might require repeat-observation coding of each infant utterance in context. But time constraints on the coders, who were students in a rigorous program, prohibited us from using repeat-observation because it requires at least tenfold more time to code the same materials. See our prior work [31] for an analysis using the more time-consuming method on a much smaller dataset.

Thus, to measure social and exploratory vocal functions, coders were required to use an intuitive and practical method, estimating on a Likert scale how often infants engaged in VP and TT for each segment. This measure, obtained immediately after real-time coding for CBR on each segment, is clearly a blunt instrument, subject to only fair point-to-point inter-observer agreement. Yet the agreement was considerably and highly significantly better than chance. More important than the absolute level, however, agreement on the key issue of Proposition 1 (the much higher estimated VP than TT) was very high indeed, with every coder in every test showing considerably higher VP than TT, always at p < .001. Coder agreement can be problematical when differences between targeted comparisons of variables are small with respect to coder differences—one runs the risk of Type II error, because low coder agreement can impose a power limitation, acting as noise in the evaluation and making it less likely to discern real differences. But the data suggest there was no such problem for the VP vs TT comparison in the present data because the differences were so large that coder disagreements could not have obscured them.

A limitation of this study is that canonical babbling was the only measure we used to assess advanced vocal forms. Future studies could, with sufficient time and money for much more elaborate coding or more advanced automated analysis, compare variation in infant vocal types based on phonatory characteristics across contexts, infants’ recombination of syllables at the utterance level, and more specific contextual social or environmental information such as response contingency and infant- and adult-register in infant-directed speech.

Interpretive issues for CBR comparisons

The present research did not acquire data from an experiment where one might insist on two exclusive conditions where all utterances by infants would be either TT in one case or VP in the other. These were instead observational results from natural home activities where both TT and VP occurred frequently within the same 5-minute periods. Our CBR comparisons are complicated by the differing distributions of VP and TT, along with the unavoidable overlap of segments having some of both VP and TT. A major advantage of using GEE is that this statistical method is not adversely affected by imbalances in cells nor by overlaps as seen in crosstabulation—GEE analysis offers assurance that statistical differences produced by the method are real.

Yet it remains worthwhile to take note of the fact that the difference between CBRs at No VP and No TT were clearly affected by the distributional imbalance; only about 8% of segments had No VP, while 80% had No TT, and notably, among those with No TT, more than 90% had Some VP. Consequently, the higher CBR for No TT than No VP may have been due largely to the presumable enhancing effect of VP on CBR in the No TT segments. Indeed, the similarity of the CBRs for No TT (mean = .105) and Some VP (mean = .119) appears to reflect the fact that the No TT segments were largely the very same ones that showed Some VP. It was rare indeed for TT to occur in the absence of VP (only 1% of segments), and segments with neither VP nor TT were also very uncommon (6%). The latter circumstance yielded extremely low CBR (.03 computed over the 99 segments with neither), an outcome that we interpret (cautiously, given the low number of segments) as reflecting the fact that there was no enhancing effect of either TT or VP on those segments. By the same token, the 293 segments with both TT and VP showed much higher CBRs (> .16) than segments showing Some TT but No VP or Some VP but No TT, circumstances that yielded intermediate CBRs (.08 and .10 respectively).

Another possible contributor to low CBR for segments with No VP concerns vocal negativity. Segments with No VP showed rates of crying and whimpering (~ 3 such utterances per min) that were more than 5 times higher than segments with Some VP. Of course, crying and whimpering were not included directly in our analyses, but the state of infants during periods of such negativity may have affected CBRs and may have influenced coders to make the judgment that VP was low or absent. Similarly segments with No VP showed high rates of whining, as indicated by the questionnaire item on complaint, where the rating for segments with No VP was ~ 4.4 (complaining the great majority of the time) while the rating for segments with Some VP was ~ 1.8 (complaining less than half the time). In accord with the instructions to coders, no utterance should have been judged to be a complaint and at the same time an exemplar of VP. So, just as the negativity indicated by crying/whimpering may have influenced low CBR and ratings of No VP, it seems likely that segments with high complaint ratings would have caused both low CBR and ratings of No VP. Segments with No VP routinely showed high complaint ratings; in fact, 71% of segments with No VP were rated as having complaint throughout the segment (a rating of 5). Thus, it should be no surprise that when babies were crying or whining, they were not only not engaging in VP, but they were not producing very speech-like protophones. In contrast, segments with No TT showed low rates of both crying and complaint, and this pattern may have contributed to the higher CBR at No TT than at No VP. More data on these issues can be found in S5 Text.