Abstract
Intensive repeated momentary diary techniques, such as the experience sampling method (ESM), place considerable burden on study participants by repeatedly prompting them to fill in a questionnaire several times throughout the day. The objective of this study was to investigate the level of disturbance reported by participants and predictors thereof using a pooled dataset of seven ESM studies including 1319 healthy participants and individuals with different mental health conditions. All studies used a design of 10 prompts per day over the course of 4 to 6 days. Participants reported no disturbance at 37% of the measurement occasions and the overall level of disturbance was relatively low. Also, clinical samples did not report significantly different levels of disturbance compared to the healthy participants. However, higher levels of disturbance were found for later study days, in the morning, and on weekends. Also, younger participants reported feeling more disturbed by the prompts. In terms of momentary states, higher levels of activity, activity stress, and tiredness were associated with higher disturbance. Also, participants experienced more disturbance when in company compared to when alone, and the level of disturbance increased when in company with less familiar people. Our findings suggest that studies using high-frequency ESM protocols can be used in mental health research, but researchers should be aware of the possible disturbance caused by their research design.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Notes
These participants missed different items at each responded beep for unknown reasons. For example, one subject filled in affect-related items on some occasions, but did not fill in stress- or context-related items. In other words, these participants consistently lacked either one or several beep-level variables of interest in the dataset.
References
Barge-Schaapveld, D. Q. C. M., Nicolson, N. A., van der Hoop, R. G., & DeVries, M. W. (1995). Changes in daily life experience associated with clinical improvement in depression. Journal of Affective Disorders, 34(2), 139–154. https://doi.org/10.1016/0165-0327(95)00012-C.
Bastiaansen, J. A., Kunkels, Y. K., Blaauw, F. J., Boker, S. M., Ceulemans, E., Chen, M., Chow, S. M., de Jonge, P., Emerencia, A. C., Epskamp, S., Fisher, A. J., Hamaker, E. L., Kuppens, P., Lutz, W., Meyer, M. J., Moulder, R., Oravecz, Z., Riese, H., Rubel, J., Ryan, O., Servaas, M. N., Sjobeck, G., Snippe, E., Trull, T. J., Tschacher, W., van der Veen, D. C., Wichers, M., Wood, P. K., Woods, W. C., Wright, A. G. C., Albers, C. J., & Bringmann, L. F. (2020). Time to get personal? The impact of researchers choices on the selection of treatment targets using the experience sampling methodology. Journal of Psychosomatic Research, 137, 110211. https://doi.org/10.1016/j.jpsychores.2020.110211.
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 51. https://doi.org/10.18637/jss.v067.i01.
Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: Capturing life as it is lived. Annual Review of Psychology, 54(1), 579–616. https://doi.org/10.1146/annurev.psych.54.101601.145030.
Brandstätter, H. (1983). Emotional responses to other persons in everyday life situations. Journal of Personality and Social Psychology, 45(4), 871–883. https://doi.org/10.1037/0022-3514.45.4.871.
Burton, C., Weller, D., & Sharpe, M. (2007). Are electronic diaries useful for symptoms research? A systematic review. Journal of Psychosomatic Research, 62(5), 553–561. https://doi.org/10.1016/j.jpsychores.2006.12.022.
Collip, D., Nicolson, N. A., Lardinois, M., Lataster, T., van Os, J., & Myin-Germeys, I. (2011). Daily cortisol, stress reactivity and psychotic experiences in individuals at above average genetic risk for psychosis. Psychological Medicine, 41(11), 2305–2315. https://doi.org/10.1017/S0033291711000602.
Collip, D., Wigman, J. T. W., Myin-Germeys, I., Jacobs, N., Derom, C., Thiery, E., Wichers, M., & van Os, J. (2013). From epidemiology to daily life: Linking daily life stress reactivity to persistence of psychotic experiences in a longitudinal general population study. PLoS One, 8(4), e62688. https://doi.org/10.1371/journal.pone.0062688.
Courvoisier, D. S., Eid, M., & Lischetzke, T. (2012). Compliance to a cell phone-based ecological momentary assessment study: The effect of time and personality characteristics. Psychological Assessment, 24(3), 713–720. https://doi.org/10.1037/a0026733.
Csikszentmihalyi, M., Larson, R., & Prescott, S. (1977). The ecology of adolescent activity and experience. Journal of Youth and Adolescence, 6, 281–294. https://doi.org/10.1007/BF02138940.
Delespaul, P. (1995). Assessing schizophrenia in daily life: The experience sampling method. Maastricht, the Netherlands: Universitaire Pers Maastricht.
Edwards, C. J., Cella, M., Tarrier, N., & Wykes, T. (2016). The optimisation of experience sampling protocols in people with schizophrenia. Psychiatry Research, 244, 289–293. https://doi.org/10.1016/j.psychres.2016.07.048.
Eisele, G., Vachon, H., Lafit, G., Kuppens, P., Houben, M., Myin-Germeys, I., & Viechtbauer, W. (2020). The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population. Assessment, 1073191120957102. https://doi.org/10.1177/1073191120957102.
Fox, J., & Weisberg, S. (2011). An R companion to applied regression. Sage Publications Retrieved from http://socserv.socsci.mcmaster.ca/jfox/Books/Companion.
Hufford, M. R. (2007). Special methdological challenges and opportunities in ecological momentary assessment. In A. A. Stone, S. Shiffman, A. A. Atienza, & L. Nebeling (Eds.), The science of real-time data capture: Self-reports in health research (pp. 54–75). Oxford University Press.
Jones, K. K., Zenk, S. N., McDonald, A., & Corte, C. (2016). Experiences of African-American women with smartphone-based ecological momentary assessment. Public Health Nursing, 33(4), 371–380. https://doi.org/10.1111/phn.12239.
Larson, R., & Csikszentmihalyi, M. (1978). Experiential correlates of time alone in adolescence. Journal of Personality, 46(4), 677–693. https://doi.org/10.1111/j.1467-6494.1978.tb00191.x.
Lataster, T., Collip, D., Lardinois, M., Van Os, J., & Myin-Germeys, I. (2010). Evidence for a familial correlation between increased reactivity to stress and positive psychotic symptoms. Acta Psychiatrica Scandinavica, 122(5), 395–404. https://doi.org/10.1111/j.1600-0447.2010.01566.x.
Lataster, T., Valmaggia, L., Lardinois, M., Van Os, J., & Myin-Germeys, I. (2013). Increased stress reactivity: a mechanism specifically associated with the positive symptoms of psychotic disorder. Psychological Medicine, 43(7), 1389–1400. https://doi.org/10.1017/S0033291712002279.
Litt, M. D., Cooney, N. L., & Morse, P. (1998). Ecological momentary assessment (EMA) with treated alcoholics: Methodological problems and potential solutions. Health Psychology, 17(1), 48–52. https://doi.org/10.1037/0278-6133.17.1.48.
Messiah, A., Grondin, O., & Encrenaz, G. (2011). Factors associated with missing data in an experience sampling investigation of substance use determinants. Drug and Alcohol Dependence, 114(2–3), 153–158. https://doi.org/10.1016/j.drugalcdep.2010.09.016.
Myin-Germeys, I., & Van Os, J. (2007). Stress-reactivity in psychosis: Evidence for an affective pathway to psychosis. Clinical Psychology Review, 27(4), 409–424. https://doi.org/10.1016/j.cpr.2006.09.005.
Myin-Germeys, I., Van Os, J., Schwartz, J. E., Stone, A. A., & Delespaul, P. A. (2001). Emotional reactivity to daily life stress in psychosis. Archives of General Psychiatry, 58(12), 1137–1144. https://doi.org/10.1001/archpsyc.58.12.1137.
Myin-Germeys, I., Peeters, F., Havermans, R., & Nicolson, N. a, DeVries, M. W., Delespaul, P., & Van Os, J. (2003). Emotional reactivity to daily life stress in psychosis and affective disorder: An experience sampling study. Acta Psychiatrica Scandinavica, 107(2), 124–131. https://doi.org/10.1034/j.1600-0447.2003.02025.x.
Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespaul, P., & Van Os, J. (2009). Experience sampling research in psychopathology: Opening the black box of daily life. Psychological Medicine, 39(9), 1533–1547. https://doi.org/10.1017/S0033291708004947.
Myin-Germeys, I., Klippel, A., Steinhart, H., & Reininghaus, U. (2016). Ecological momentary interventions in psychiatry. Current Opinion in Psychiatry, 29(4), 258–263. https://doi.org/10.1097/YCO.0000000000000255.
Myin-Germeys, I., Kasanova, Z., Vaessen, T., Vachon, H., Kirtley, O., Viechtbauer, W., & Reininghaus, U. (2018). Experience sampling methodology in mental health research: New insights and technical developments. World Psychiatry, 17(2), 123–132. https://doi.org/10.1002/wps.20513.
Pawlik, K., & Buse, L. (1982). Computer-based behavior registration in the field: Description and first psychometric evaluation of a new recording method. Zeitschrift fur Differentielle und Diagnostische Psychologie, 3(2), 101–118.
R Development Core Team. (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing Vienna Austria., 95, 389–393. https://doi.org/10.1038/sj.hdy.6800737.
Rintala, A., Wampers, M., Myin-Germeys, I., & Viechtbauer, W. (2019). Response compliance and predictors thereof in studies using the experience sampling method. Psychological Assessment, 31(2), 226–235. https://doi.org/10.1037/pas0000662.
Rintala, A., Wampers, M., Myin-Germeys, I., & Viechtbauer, W. (2020). Momentary predictors of compliance in studies using the experience sampling method. Psychiatry Research, 286, 112896. https://doi.org/10.1016/j.psychres.2020.112896.
Sokolovsky, A. W., Mermelstein, R. J., & Hedeker, D. (2014). Factors predicting compliance to ecological momentary assessment among adolescent smokers. Nicotine and Tobacco Research, 16(3), 351–358. https://doi.org/10.1093/ntr/ntt154.
Solhan, M. B., Trull, T. J., Jahng, S., & Wood, P. K. (2009). Clinical assessment of affective instability: Compating EMA indices, questionnaire reports, and retrospective recall. Psychological Assessment, 21(3), 425–436. https://doi.org/10.1037/a0016869.
Stone, A. A., Broderick, J. E., Schwartz, J. E., Shiffman, S., Litcher-Kelly, L., & Calvanese, P. (2003). Intensive momentary reporting of pain with an electronic diary: Reactivity, compliance, and patient satisfaction. Pain, 104(1–2), 343–351. https://doi.org/10.1016/S0304-3959(03)00040-X.
Trull, T. J., & Ebner-Priemer, U. W. (2009). Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: Introduction to the special section. Psychological Assessment, 21(4), 457–462. https://doi.org/10.1037/a0017653.
Vachon, H., Viechtbauer, W., Rintala, A., & Myin-Germeys, I. (2019). Compliance and retention with the experience sampling method over the continuum of severe mental disorders: Meta-analysis and recommendations. Journal of Medical Internet Research, 21(2), e11398. https://doi.org/10.2196/14475.
Van Eck, M., Berkhof, H., Nicolson, N., & Sulon, J. (1996). The effects of perceived stress, traits, mood states, and stressful daily events on salivary cortisol. Psychosomatic Medicine, 58(5), 447–458. https://doi.org/10.1097/00006842-199609000-00007.
Van Genugten, C. R., Schuurmans, J., Lamers, F., & Riese, H. (2020). Experienced burden of and adherence to smartphone-based ecological momentary assessment in persons with affective disorders. Journal of Clinical Medicine, 9(2), 322. https://doi.org/10.3390/jcm9020322.
Van Roekel, E., Keijsers, L., & Chung, J. M. (2019). A review of current ambulatory assessment studies in adolescent samples and practical recommendations. Journal of Research on Adolescence, 29(3), 560–577. https://doi.org/10.1111/jora.12471.
Verhagen, S. J. W., Hasmi, L., Drukker, M., van Os, J., & Delespaul, P. A. E. G. (2016). Use of the experience sampling method in the context of clinical trials. Evidence-Based Mental Health, 19(3), 86–89. https://doi.org/10.1136/ebmental-2016-102418.
Wen, C. K. F., Schneider, S., Stone, A. A., & Spruijt-Metz, D. (2017). Compliance with mobile ecological momentary assessment protocols in children and adolescents: A systematic review and meta-analysis. Journal of Medical Internet Research, 19(4), e132. https://doi.org/10.2196/jmir.6641.
Whalen, C. K., Jamner, L. D., Henker, B., Delfino, R. J., & Lozano, J. M. (2002). The ADHD spectrum and everyday life: Experience sampling of adolescent moods, activities, smoking, and drinking. Child Development, 73(1), 209–227. https://doi.org/10.1111/1467-8624.00401.
Acknowledgements
The authors wish to thank the research group of ESM MERGE for providing the data for this study.
Materials and/or Code Availability
The materials and/or codes of the current study are available from the corresponding author on reasonable request.
Funding
Aki Rintala and Inez Myin-Germeys was supported by the Remote Assessment of Disease and Relapse – Central Nervous System (RADAR-CNS) research programme from the Innovative Medicines Initiative 2 Joint Undertaking under a grant agreement number 115902. This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and the European Federation of Pharmaceutical Industries and Associations (EFPIA). Inez Myin-Germeys was funded by the Fonds voor Wetenschappelijk Onderzoek (FWO) Odysseus grant (GOF8416N).
Author information
Authors and Affiliations
Contributions
A.R. conceived of the presented idea. A.R., M.W., & W.V. developed the conceptualization and methodology. A.R. and G.L. worked with the analytical methods, which W.V verified. A.R. was responsible for conducting the analyses and writing the manuscript with the supervision help from W.V. All promoters (M.W., W.V., and I.M-G.) supervised the findings of this work. All authors discussed the results and contributed to the final manuscript.
Corresponding author
Ethics declarations
Ethics Approval
The local ethics committees approved all studies included in the pooled dataset.
Consent to Participate
Written informed consent was obtained from all participants.
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Rintala, A., Wampers, M., Lafit, G. et al. Perceived disturbance and predictors thereof in studies using the experience sampling method. Curr Psychol 42, 6287–6301 (2023). https://doi.org/10.1007/s12144-021-01974-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12144-021-01974-3