Skip to content

Create a Guide/Template: A/B testing for your project #11

New issue
New issue
Open
Open
Create a Guide/Template: A/B testing for your project#11
Labels
CoP label: checkedCoP labels have been checkedCoP: DesignCoP: EngineeringCoP: ProductCoP: UI/UX ResearchComplexity: MissingPhase: 1 - Gather Examples SoloReady for Bonnie Triagefeature : guide creationsize: 2ptCan be done in 7-12 hoursstatus: 1.1 needs wiki examplesGathering examples by checking each team's wiki
Milestone
06.06 Could Do in Future

Description

@erigilg
erigilg
opened on Mar 5, 2021

Overview

We need to create a guide to making A/B testing so that teams know how to experiment and determine the best option of the product for their audience.

Details

This is an Epic/Overview that will discuss the various ways that A/B testing can be performed on Hack for LA projects. Each of these ways should link to another guide which goes into detail.

Action Items

The phases in the guide-making process are listed below. Each phase displayed in blue is linked to a wiki page with instructions on how to complete that phase. Open the wiki page in a new tab, copy the instructions for each part into the section labelled 'Tasks' at the bottom of this issue, and complete each task listed.

Resources/Instructions

Update issue #128 with the name of item you are working

Projects to Check

Tasks

  • This is where you will copy instructions from the wiki page for the step you are currently on.

Activity

  1. Eloks7
    closed this as completedon Feb 26, 2022
  2. Eloks7
    reopened this on Feb 26, 2022

  3. aymarmsba commented on Apr 3, 2024

    @aymarmsba
    aymarmsba
    on Apr 3, 2024 · Hidden as resolved

  4. aymarmsba commented on Apr 3, 2024

    @aymarmsba
    aymarmsba
    on Apr 3, 2024 · Hidden as resolved
  5. aymarmsba
    transferred this issue fromhackforla/product-managementon Apr 3, 2024
  6. aymarmsba
    added
    CoP: Product
    milestone: missing
    size: 2ptCan be done in 7-12 hours
    Phase: 1 - Gather Examples Solo
    status: 1.1 needs wiki examplesGathering examples by checking each team's wiki
    on Apr 3, 2024
  7. aymarmsba
    added this to P: Guides: Knowledgebase Content Trackeron Apr 3, 2024
  8. aymarmsba
    moved this to Prioritized Backlog in P: Guides: Knowledgebase Content Trackeron Apr 3, 2024
  9. aymarmsba
    added this to the 06 Could Do in Future milestone on Apr 3, 2024
  10. aymarmsba
    removed
    milestone: missing
    on Apr 3, 2024
  11. ExperimentsInHonesty
    added this to P: Guides: Tracker of guide issues (in knowlegebase-content repo)on May 31, 2024

  12. 1 remaining item

  13. fchan218
    added
    Complexity: Missing
    on Aug 30, 2024
  14. ExperimentsInHonesty
    added
    feature : guide creation
    on Oct 11, 2024
  15. Aditya23soni
    added
    Ready for Bonnie Triage
    on Mar 7, 2025
  16. fchan218
    added
    CoP label: check neededneed to check if the CoP labels are correct
    on Mar 10, 2025
  17. yashhR
    added
    CoP label: shortcuts added
    on Apr 25, 2025

  18. ExperimentsInHonesty commented on May 26, 2025

    @ExperimentsInHonesty
    ExperimentsInHonesty
    on May 26, 2025
    Member

    Concise list of A/B testing methods:

    • Classic A/B Test – Split users into two groups, each sees a different variant (A vs B).
    • Multivariate Testing (MVT) – Test combinations of multiple variables simultaneously.
    • Split URL Testing – Different variants hosted on separate URLs.
    • Multi-armed Bandit – Dynamically allocates more traffic to better-performing variants.
    • Sequential Testing – Continuously analyzes results to decide when to stop early.
    • Bayesian A/B Testing – Uses Bayesian inference for probability-based conclusions.
    • Server-side Testing – Variant logic handled on the backend (good for performance-heavy tests).
    • Client-side Testing – Frontend script modifies the UI for testing (easier to implement).
    • Redirect Tests – Similar to split URL, often used for landing page comparisons.
    • Personalization/Segmentation Testing – A/B tests tailored to user segments.

    Without a website

    A/B testing approaches can be done without a live website, using mockups or prototypes. Here are the methods that work in that context:

    • Preference Testing (a variation of A/B):
      • Show mockups (images or clickable prototypes) to users and ask which they prefer.
      • Tool examples: UsabilityHub, Maze, Figma prototypes.
    • Click Tests / Heatmaps on Mockups:
      • Present static designs and track where users click or focus attention.
      • Tools: Hotjar (on prototypes), Maze.
    • Surveys & First-Click Tests:
      • Ask users where they'd click first on a static image or ask comprehension questions based on the mockup.
    • User Interviews or Think-Aloud Tests:
      • Present each version (A and B) and observe reactions or ask for task-based feedback.
    • Multivariate-like in Concept:
      • Test different combinations of copy, layout, and visuals via remote testing or in-person sessions.

    With a website

    The following A/B testing can only be done with a live website and server control

    • Server-side Testing
    • Client-side Testing (in production)
    • Multi-armed Bandit
    • Split URL Testing
    • Real-time Sequential or Bayesian Testing
      • (These require live traffic.)
  19. ExperimentsInHonesty
    added
    CoP: Engineering
    CoP: Design
    CoP: UI/UX Research
    CoP label: checkedCoP labels have been checked
    CoP label: folder shortcut neededneed to add google drive folder shortcut to CoP
    and removed
    CoP label: check neededneed to check if the CoP labels are correct
    CoP label: shortcuts added
    on May 26, 2025

  20. ExperimentsInHonesty commented on May 27, 2025

    @ExperimentsInHonesty
    ExperimentsInHonesty
    on May 27, 2025
    Member

    Guide: Understanding A/B Testing

  21. yashhR
    removed
    CoP label: folder shortcut neededneed to add google drive folder shortcut to CoP
    on May 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CoP label: checkedCoP labels have been checkedCoP: DesignCoP: EngineeringCoP: ProductCoP: UI/UX ResearchComplexity: MissingPhase: 1 - Gather Examples SoloReady for Bonnie Triagefeature : guide creationsize: 2ptCan be done in 7-12 hoursstatus: 1.1 needs wiki examplesGathering examples by checking each team's wiki

    Type

    No type

    Projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions