Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Aug 18;21(16):5553.
doi: 10.3390/s21165553.

A Blanket Accommodative Sleep Posture Classification System Using an Infrared Depth Camera: A Deep Learning Approach with Synthetic Augmentation of Blanket Conditions

Andy Yiu-Chau Tam  1 , Bryan Pak-Hei So  1 , Tim Tin-Chun Chan  1 , Alyssa Ka-Yan Cheung  1 , Duo Wai-Chi Wong  1 , James Chung-Wai Cheung  1   2
Affiliations

A Blanket Accommodative Sleep Posture Classification System Using an Infrared Depth Camera: A Deep Learning Approach with Synthetic Augmentation of Blanket Conditions

Andy Yiu-Chau Tam et al. Sensors (Basel). .

Abstract

Surveillance of sleeping posture is essential for bed-ridden patients or individuals at-risk of falling out of bed. Existing sleep posture monitoring and classification systems may not be able to accommodate the covering of a blanket, which represents a barrier to conducting pragmatic studies. The objective of this study was to develop an unobtrusive sleep posture classification that could accommodate the use of a blanket. The system uses an infrared depth camera for data acquisition and a convolutional neural network to classify sleeping postures. We recruited 66 participants (40 men and 26 women) to perform seven major sleeping postures (supine, prone (head left and right), log (left and right) and fetal (left and right)) under four blanket conditions (thick, medium, thin, and no blanket). Data augmentation was conducted by affine transformation and data fusion, generating additional blanket conditions with the original dataset. Coarse-grained (four-posture) and fine-grained (seven-posture) classifiers were trained using two fully connected network layers. For the coarse classification, the log and fetal postures were merged into a side-lying class and the prone class (head left and right) was pooled. The results show a drop of overall F1-score by 8.2% when switching to the fine-grained classifier. In addition, compared to no blanket, a thick blanket reduced the overall F1-scores by 3.5% and 8.9% for the coarse- and fine-grained classifiers, respectively; meanwhile, the lowest performance was seen in classifying the log (right) posture under a thick blanket, with an F1-score of 72.0%. In conclusion, we developed a system that can classify seven types of common sleeping postures under blankets and achieved an F1-score of 88.9%.

Keywords: convolutional neural network; sleep behavior; sleep disorder; sleep monitoring; sleep posture recognition; sleep surveillance.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
RGB (left column) and infrared (right column) images of typical participant under thick (first row), medium (second row), thin (third row), and control (fourth row) blanket conditions in (a) supine, (b) prone (head left), (c) log (right), and (d) fetal (right) postures.
Figure 2
Figure 2
Illustration of data fusion technique on six combinations of blanket conditions.
Figure 3
Figure 3
Flowchart of functional component linkage of model network.
Figure 4
Figure 4
Architecture of model network illustrating fully connected network (FCN) layers toward coarse-grained (4-posture) and fine-grained (7-posture) classification.
Figure 5
Figure 5
Confusion matrix across true and predicted labels for (a) 4-posture coarse-grained classification and (b) 7-posture fine-grained classification.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Lin F., Zhuang Y., Song C., Wang A., Li Y., Gu C., Li C., Xu W. SleepSense: A noncontact and cost-effective sleep monitoring system. IEEE Trans. Biomed. Circuits Syst. 2016;11:189–202. doi: 10.1109/TBCAS.2016.2541680. - DOI - PubMed
    1. Khalil M., Power N., Graham E., Deschênes S.S., Schmitz N. The association between sleep and diabetes outcomes—A systematic review. Diabetes Res. Clin. Pract. 2020;161:108035. doi: 10.1016/j.diabres.2020.108035. - DOI - PubMed
    1. Vorona R.D., Winn M.P., Babineau T.W., Eng B.P., Feldman H.R., Ware J.C. Overweight and obese patients in a primary care population report less sleep than patients with a normal body mass index. Arch. Intern. Med. 2005;165:25–30. doi: 10.1001/archinte.165.1.25. - DOI - PubMed
    1. Spiegel K., Knutson K., Leproult R., Tasali E., Cauter E.V. Sleep loss: A novel risk factor for insulin resistance and Type 2 diabetes. J. Appl. Physiol. 2005;99:2008–2019. doi: 10.1152/japplphysiol.00660.2005. - DOI - PubMed
    1. Short M.A., Booth S.A., Omar O., Ostlundh L., Arora T. The relationship between sleep duration and mood in adolescents: A systematic review and meta-analysis. Sleep Med. Rev. 2020;52:101311. doi: 10.1016/j.smrv.2020.101311. - DOI - PubMed

LinkOut - more resources