JMIR Publications

ABSTRACT

Background:

The Internet is becoming an increasingly common tool for survey research, particularly among “hidden” and/or vulnerable populations such as men who have sex with men (MSM). Online research has many advantages to participants and researchers, but fraud can present a significant threat to data integrity.

Objective:

The purpose of this analysis was to evaluate fraud detection strategies in an online survey of young MSM and to describe new protocols to improve fraud detection in online survey research.

Methods:

This study involved a cross-sectional online survey that examined individual- and network-level risk factors for HIV transmission and substance use among young MSM residing in 15 counties in Central Kentucky. Each survey entry that was at least 50% complete was evaluated by study staff for fraud using an algorithm involving eight criteria based on a combination of geolocation data, survey data, and personal information. Entries were classified fraudulent, potential fraudulent, or valid. Descriptive analyses were performed to describe each fraud detection criterion among entries.

Results:

Of the 414 survey entries, the final categorization resulted in 119 (28.7%) entries identified as fraud, 42 (10.1%) as potential fraud, and 253 (61.1%) as valid. Geolocation outside of the study area (39.6%) was the most frequently violated criterion. However, 33.3% (n=82) of the entries that had ineligible geolocations belonged to participants who were in eligible locations (as verified by their request to mail payment to an address within the study area or participation at a local event). The second most frequently violated criteria was invalid phone number (22.7%), followed by mismatching names within an entry (10.4%) and unusual email addresses (8.9%). Less than 5% of entries had some combination of personal information items matching that of a previous entry (n=18).

Conclusions:

This study suggests that researchers conducting online surveys of MSM should be vigilant about the potential for fraud. Researchers should have a fraud detection algorithm in place in advance of data collection and should not rely only on IP address or geolocation, but instead use a combination of indicators.