Online Research and Bots

What are Bots?

Bots, short for "robots," are software applications that perform automated tasks for various purposes, operating on the internet or within computer systems. While bots can be beneficial and perform helpful functions, they can also be misused for unethical or harmful activities.

  • Designed to simulate conversations with human users, commonly used for customer service and providing information.

  • Used by search engines to systematically browse and index web pages, playing a crucial role in collecting information from websites.

  • Automated scripts that interact with social media platforms, performing tasks like liking posts, following users, or sharing content.

  • Flood comment sections with irrelevant or harmful content, and are used in distributed denial-of-service attacks to overwhelm websites.

Why should researchers be concerned about Bots?

Bots present challenges and ethical issues, especially for researchers collecting data online. They can interact with humans and engage in research activities without proper informed consent. Bots programmed with biases or manipulative techniques can impact research outcomes, compromising unbiased and accurate data collection.

  • Bots in online research studies can undermine the validity and reliability of data, affecting study objectives and outcomes.
  • Bots can submit multiple fake responses, skewing data and misrepresenting real-world perspectives.
  • Bots can promote specific agendas by consistently selecting certain options or providing scripted responses, distorting the true distribution of opinions.
  • Bots can overload survey platforms with high volumes of submissions, causing technical issues like crashes or slowdowns, disrupting the study and leading to data loss.
  • In social media research, bots can create fake accounts and generate large volumes of posts or interactions, distorting the perceived popularity of certain topics or opinions.

What can researchers do to prevent Bots?

  • Researchers can use several strategies to prevent bots from interfering with online recruitment. Require participants to verify their identity using methods like social media logins or email verification. This adds a layer of defense against automated systems. Additionally, include screening questions from the start that require specific details or explanations to detect automated responses.

    Targeted Recruitment/Advertising:

    •    Utilize targeted advertising on platforms less prone to bot activity.
    •    Personalize recruitment messages to make them more human-like.
    •    Avoid generic language or templates that attract bots.

    Social Media:

    •    Avoid using social media (Instagram, Facebook, X) for recruitment due to high bot attraction and lack of effective bot prevention measures.

    Audience:

    •    Target niche communities or forums relevant to the research topic, where bots are less likely to be present.

    Recruitment Sites with Verified Participants:

    •    Use real recruitment websites that limit bot access and provide verified participants.
    •    Examples: [ResearchMatch ](https://www.researchmatch.org/) (health-related online research) and [Prolific] (https://www.prolific.com/).

  • Paying participants can attract bots, so be strategic about compensation amounts. Offering a raffle prize instead of high-value gift cards can reduce bot attraction.

    Payment Methods

    • Gift Cards
    • PayPal or Venmo

    Timing of Payment

    • Payments should be issued after the completion of the study, once responses are verified.
    • Require participants to authenticate themselves using secure methods like social network logins, phone numbers, or email addresses to verify they are actual people.
    • Offer various payment options, such as PayPal transfers, gift cards, or charitable donations.
    • Place a cap on how often each participant can be paid, limiting payments to once per participant to deter bots from repeatedly participating just to receive compensation.
    • Ensure participants are fully informed about the research, including any use of automated tools or bots.
    • Include compensation specifics; Inform participants when they will be paid and the conditions under which they will not be paid.
      • Failure to meet eligibility criteria, such as human verification
      • Incomplete or insufficient responses
      • Suspicious survey completion times
      • Fraudulent activity, such as providing false information or manipulating questions
    • Encryption: Encrypt sensitive information such as personal details, participant responses, and research findings to prevent unauthorized access and maintain confidentiality.
    • Access Control: Restrict data access to authorized users only.
    • Secure Data Storage: Store data in secure, compliant environments that adhere to industry standards and regulations, such as cloud-based storage solutions with built-in security features.
    • Secure Communication: Use encrypted email or secure messaging platforms to transmit sensitive information between researchers, participants, and the IRB.
    • Strong Passwords and Authentication: Require strong passwords for accounts and consider using multi-factor authentication.
    • Incident Response Plan: Develop and maintain a plan to address security breaches or data incidents promptly and effectively.

Tools/Features for Bots Prevention

  • Attention Check Questions: Simple questions to ensure respondents are paying attention, e.g., "Please select 'AGREE' for this question."
  • Repeat Questions: Assess reliability by including similar questions at different points in the survey.
  • Conditional Logic Questions: Tailor survey flow based on respondents' previous answers.
  • Word Scrambles: Use puzzles to engage respondents and add gamification.
  • Track Study Time Stamps: Record start and end times to analyze response patterns.
  • Unique Survey Links: Prevent duplicate submissions and personalize the survey experience.
  • Email Verification: Confirm respondent identities.
  • Randomized Questions: Make it difficult for bots to predict and respond to surveys.

In Summary

Design your study with Bot prevention in mind.

Be sure to create a detailed recruitment plan with clear inclusion and exclusion criteria. 

Use tools to prevent bots and ensure instructions emphasize authentic responses.

Review survey answers for rapid or similar responses, and stay updated on bot detection and cybersecurity.