Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Nov:2022:185-195.
doi: 10.1145/3536220.3563366. Epub 2022 Nov 7.

Head Movement Patterns during Face-to-Face Conversations Vary with Age

Denisa Qori McDonald  1 , Casey J Zampella  1 , Evangelos Sariyanidi  1 , Aashvi Manakiwala  2   1 , Ellis DeJardin  1 , John D Herrington  1   2 , Robert T Schultz  1   2 , Birkan Tunç  1   2
Affiliations

Head Movement Patterns during Face-to-Face Conversations Vary with Age

Denisa Qori McDonald et al. ICMI22 Companion (2022). 2022 Nov.

Abstract

Advances in computational behavior analysis have the potential to increase our understanding of behavioral patterns and developmental trajectories in neurotypical individuals, as well as in individuals with mental health conditions marked by motor, social, and emotional difficulties. This study focuses on investigating how head movement patterns during face-to-face conversations vary with age from childhood through adulthood. We rely on computer vision techniques due to their suitability for analysis of social behaviors in naturalistic settings, since video data capture can be unobtrusively embedded within conversations between two social partners. The methods in this work include unsupervised learning for movement pattern clustering, and supervised classification and regression as a function of age. The results demonstrate that 3-minute video recordings of head movements during conversations show patterns that distinguish between participants that are younger vs. older than 12 years with 78% accuracy. Additionally, we extract relevant patterns of head movement upon which the age distinction was determined by our models.

Keywords: bag-of-words approach; behavioral analysis; conversation analysis; dyadic features; head movement patterns; monadic features; non-verbal communication; video analysis.

PubMed Disclaimer

Figures

Figure 1:
Figure 1:
Experimental setup and data collection hardware. In (a), the participant and confederate are having a conversation with the “BioSensor” camera (a slightly older model shown here) placed between them. In (b), the current model of “BioSensor” used for data collection is shown.
Figure 2:
Figure 2:
Illustration of the three angles used in the study to quantify head movement.
Figure 3:
Figure 3:
Overview of the data transformation processes and experiments. Each column of the raw data contains information about changes in each of the three basic head movement angles: yaw, pitch, and roll. Windowed data are produced, normalized to a standard scale (0 mean and 1 standard deviation), and analyzed for each angle independently. For each angle, windowed samples from all participants are clustered to identify common movement patterns. This step is repeated for multiple values of K in K–Means clustering. After the frequency–based feature construction (detailed in Figure 4), features from all angles are concatenated to produce the feature set for classification. Through 10–fold cross–validation, training data is also used to determine the best K for clustering and classifier parameters within a nested cross-validation framework. The trained model is then used for testing. Cross-validation is repeated 10 times with different random seeds for data shuffling (i.e., 10 times 10–fold nested cross-validation).
Figure 4:
Figure 4:
Illustration of the feature construction process. The monadic features are created by counting the number of windowed data that belong to each cluster. For each participant, windowed data is assigned to the closest cluster center, and the number of members in each cluster is counted. In the case of dyadic features, the monadic features are calculated both for the participant and the confederate, and concatenated to produce the final feature vector.
Figure 5:
Figure 5:
The top 10 most relevant features extracted during classification have been plotted for each feature construction scenario: monadic in (a) and dyadic in (b). For each feature, the y-axis represents the weight coefficients assigned by the SVM classifier. In the monadic case, each feature contains information about the angle (a) and cluster center within that angle (c). The dyadic feature plot contains the extra information of whether that feature belonged to the confederate (C) or the participant (P).
Figure 6:
Figure 6:
Time-dependent plots of features considered most important during SVM classification (top 10 features shown in Figure 5). In (a), head angle patterns were extracted using monadic features, and in (b), dyadic features. In each case, the cluster centers of head angles have been visualized. Each subplot shows the cluster center belonging to one angle. The x-axis shows the time dimension, and the y-axis, the change in the angle (normalized). With the dyadic features, the y-axis also contains information about who that feature belonged to: the participant (P) or confederate (C).
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Ashenfelter Kathleen T, Boker Steven M, Waddell Jennifer R, and Vitanov Nikolay. 2009. Spatiotemporal symmetry and multifractal structure of head movements during dyadic conversation. Journal of Experimental Psychology: Human Perception and Performance 35, 4 (2009), 1072. - PubMed
    1. American Psychiatric Association et al. 2013. Diagnostic and Statistical Manual of Mental Disorders (DSM-V)(5th ed.).
    1. Burgoon Judee K, Manusov Valerie, and Guerrero Laura K. 2021. Nonverbal communication. Routledge.
    1. Cappella Joseph N. 1981. Mutual influence in expressive behavior: Adult–adult and infant–adult dyadic interaction. Psychological bulletin 89, 1 (1981), 101. - PubMed
    1. Curto David, Clapés Albert, Selva Javier, Smeureanu Sorina, Junior Julio, Jacques CS, Gallardo-Pujol David, Guilera Georgina, Leiva David, Moeslund Thomas B, et al. 2021. Dyadformer: A multi-modal transformer for long-range modeling of dyadic interactions. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2177–2188.

LinkOut - more resources