Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022;57(6):857-866.
doi: 10.1080/10826084.2022.2046096. Epub 2022 Mar 8.

Using Crowdsourcing for Alcohol and Nicotine Use Research: Prevalence, Data Quality, and Attrition on Amazon Mechanical Turk

Jillian M Rung  1   2   3 , Shahar Almog  1 , Andrea Vásquez Ferreiro  1 , Meredith S Berry  1   2
Affiliations

Using Crowdsourcing for Alcohol and Nicotine Use Research: Prevalence, Data Quality, and Attrition on Amazon Mechanical Turk

Jillian M Rung et al. Subst Use Misuse. 2022.

Abstract

BackgroundGauging the feasibility of using Amazon Mechanical Turk ® (MTurk) for various types of substance use research is precluded by a lack of information pertaining to the recruitment process in published studies utilizing it and concurrent information on data quality. ObjectiveThe present report addressed this gap by documenting the prevalence of alcohol and nicotine use, self-reported major health conditions, and information on data quality and retention on MTurk. Individuals 21 to 90 years old (N = 1101, Mdn age = 30) with United States-based MTurk accounts completed a stand-alone screening survey. The screening consisted of basic demographic, substance use, and physical/mental health questions, as well as items to gauge language proficiency/attention (i.e., data quality). ResultsPoor quality data was infrequent (6.5% of participants) and associated with self-reported non-United States residence, affirmative responding (e.g., currently pregnant, using both alcohol and nicotine), and other response characteristics (e.g., not disclosing health conditions). Among those passing quality checks, alcohol and nicotine use were relatively common (71.5% and 24.8%). Major physical (6.3%) and mental health conditions (14.8%) were less common. Despite not sending direct invitations, most eligible participants returned to and completed the main study (81.7%). Conclusions/Importance: Alcohol and nicotine use were relatively common among MTurk workers and retention rates were high. Together with the low prevalence of poor quality data, MTurk appears to remain a fruitful platform for substance use research; although researchers must be diligent in using appropriate screening tools, as substance use was sometimes associated with poor data quality and MTurk account information may not be reliable.

Keywords: Crowdsourcing; alcohol; nicotine; retention; substance use.

PubMed Disclaimer

Conflict of interest statement

Disclosure Statement

The authors have no conflicts of interest to declare.

Figures

Figure 1.
Figure 1.
Image shown to participants for the object identification question in the screening survey (Kolde, 2017).
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). 10.1176/appi.books.9780890425596 - DOI
    1. Arditte KA, Çek D, Shaw AM, & Timpano KR (2016). The importance of assessing clinical phenomena in Mechanical Turk research. Psychological Assessment, 28(6), 684. - PubMed
    1. Athamneh LN, Stein MD, Lin EH, Stein JS, Mellis AM, Gatchalian KM, Epstein LH, & Bickel WK (2020). Setting a goal could help you control: Comparing the effect of health goal versus general episodic future thinking on health behaviors among cigarette smokers and obese individuals. Experimental and Clinical Psychopharmacology, 29(1), 59. 10.1037/pha0000351 - DOI - PMC - PubMed
    1. Barrett TS, & Brignone E (2017). Furniture for Quantitative Scientists. The R Journal, 9(2). 10.32614/RJ-2017-037 - DOI
    1. Berinsky AJ, Huber GA, & Lenz GS (2012). Evaluating online labor markets for experimental research: Amazon.com’s mechanical turk. Political Analysis, 20(3), 351–368. 10.1093/pan/mpr057 - DOI

Publication types