Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Oct 18;19(20):13491.
doi: 10.3390/ijerph192013491.

Depth-Camera-Based Under-Blanket Sleep Posture Classification Using Anatomical Landmark-Guided Deep Learning Model

Andy Yiu-Chau Tam  1 , Li-Wen Zha  2 , Bryan Pak-Hei So  1 , Derek Ka-Hei Lai  1 , Ye-Jiao Mao  1 , Hyo-Jung Lim  1 , Duo Wai-Chi Wong  1 , James Chung-Wai Cheung  1   3
Affiliations

Depth-Camera-Based Under-Blanket Sleep Posture Classification Using Anatomical Landmark-Guided Deep Learning Model

Andy Yiu-Chau Tam et al. Int J Environ Res Public Health. .

Abstract

Emerging sleep health technologies will have an impact on monitoring patients with sleep disorders. This study proposes a new deep learning model architecture that improves the under-blanket sleep posture classification accuracy by leveraging the anatomical landmark feature through an attention strategy. The system used an integrated visible light and depth camera. Deep learning models (ResNet-34, EfficientNet B4, and ECA-Net50) were trained using depth images. We compared the models with and without an anatomical landmark coordinate input generated with an open-source pose estimation model using visible image data. We recruited 120 participants to perform seven major sleep postures, namely, the supine posture, prone postures with the head turned left and right, left- and right-sided log postures, and left- and right-sided fetal postures under four blanket conditions, including no blanket, thin, medium, and thick. A data augmentation technique was applied to the blanket conditions. The data were sliced at an 8:2 training-to-testing ratio. The results showed that ECA-Net50 produced the best classification results. Incorporating the anatomical landmark features increased the F1 score of ECA-Net50 from 87.4% to 92.2%. Our findings also suggested that the classification performances of deep learning models guided with features of anatomical landmarks were less affected by the interference of blanket conditions.

Keywords: digital health; sleep behavior; sleep monitoring; sleep posture recognition; sleep surveillance; ubiquitous health.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Figures

Figure 1
Figure 1
Seven sleep postures classified in the deep learning models: supine, left-sided log, left-sided fetal, right-sided log, right-sided fetal, prone with head turned left, and prone with head turned right.
Figure 2
Figure 2
The anatomical landmarks generated by the pose estimator (OpenPose).
Figure 3
Figure 3
Model training architectures (system units) with the ResNet-34, EfficientNet B4, and ECA-Net50 backbones: (a) Channel A: with pose estimator; (b) Channel B: without pose estimator.
Figure 4
Figure 4
System application architecture to predict sleep posture from testing data after the models were trained.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Zielinski M.R., McKenna J.T., McCarley R.W. Functions and mechanisms of sleep. AIMS Neurosci. 2016;3:67. doi: 10.3934/Neuroscience.2016.1.67. - DOI - PMC - PubMed
    1. Devaraj D., Devaraj U., Venkatnarayan K., Veluthat C., Ramachandran P., D’Souza G., Maheswari K.U. Prevalence of sleep practices, circadian types and their effect on sleep beliefs in general population: Knowledge and Beliefs About Sleep and Sleep Practices (KNOBS Survey) Sleep Vigil. 2021;5:61–69. doi: 10.1007/s41782-021-00128-6. - DOI
    1. Liu Y., Wheaton A.G., Chapman D.P., Cunningham T.J., Lu H., Croft J.B. Prevalence of healthy sleep duration among adults—United States, 2014. Morb. Mortal. Wkly. Rep. 2016;65:137–141. doi: 10.15585/mmwr.mm6506a1. - DOI - PubMed
    1. Wickwire E.M. Value-based sleep and breathing: Health economic aspects of obstructive sleep apnea. Fac. Rev. 2021;10:40. doi: 10.12703/r/10-40. - DOI - PMC - PubMed
    1. Cary D., Jacques A., Briffa K. Examining relationships between sleep posture, waking spinal symptoms and quality of sleep: A cross sectional study. PLoS ONE. 2021;16:e0260582. doi: 10.1371/journal.pone.0260582. - DOI - PMC - PubMed

Publication types