When designing an A/B Test, using this One-Pager template will help you plan and prepare more thoroughly and extract valuable lessons learned.
Items marked with (*) are required — try to prepare your experiment following the examples as closely as possible.
Problem identified *
Product list views are high, but the product click-through rate is low.
Analysis and definition for problem-solving *
Add qualitative and quantitative data
Hypothesis definition for the solution *
Displaying the discount rate on the product list will increase the click-through rate.
Experiment duration
Approximately 2 weeks
User distribution timing *
Distribute at the point when users view the product list
Changes applied *
Group A: Control (no change) Group B: Treatment (change applied)
Target description *
All users
Success metric *
Which metric needs to improve or decrease for this experiment to be a success
Product click-through rate (click_product / view_product_list)
Supporting metrics
Add any additional metrics worth monitoring together.
Add-to-cart rate (click_add_cart / view_product_detail)
Purchase conversion rate (complete_purchase / view_product_detail)
Guardrail metrics
Consider metrics that must be maintained during the experiment.
Order cancellation rate, Retention, etc.
URL where the experiment can be verified
Experiment results *
Winner group: Group B
What can we learn if Group A wins?
What can we learn if Group B wins?
If the click-through rate increased after showing the discount rate, users are highly price-sensitive.
Based on these learnings, consider a follow-up test topic.
If users are price-sensitive, should we try sorting by discount?
Last updated