Optimizing Popups with A/B Tests

You can follow along step by step to manage and optimize the main popup.

This document explains the process of running an A/B Test with Remote Config, using popup optimization as an example. Anyone can easily set it up by following each step in order.

Optimizing Popup Components

Regardless of industry or service type, most services display a popup when users enter a screen. This document uses a popup displayed on the home screen as an example to show how to apply Remote Config and how it can be used in A/B Tests.

1

Managing the Popup with Remote Config

Assume you are running a Fri/Sat/Sun discount promotion offering up to 75% off, as shown in the image.

Service screen example

Go to Hackle's Remote Config tab > Create New Parameter > and create a parameter key named Home_banner. Let's configure the components in JSON as desired.

Creating a Remote Config to manage the popup

As shown in the image below, include all popup components (image, title, description) in JSON and save. Changes are reflected immediately upon saving.

Managing popup components with Remote Config JSON

2

Running an A/B Test to Validate the Popup's Effectiveness

Previously, all users were shown a promotion copy emphasizing a weekend-only, time-limited offer.

Run an A/B Test against a version that emphasizes the maximum discount rate to measure its effectiveness.

• Original copy (Group A): Weekend-only, time-emphasized copy

  • "title": "Weekend Sale, Makeup Special" "description": "3 days only every Fri/Sat/Sun! Maximum benefits! Spring exclusive special"

• New copy (Group B): Maximum discount rate emphasized copy

  • "title": "Up to 75% Off, Makeup Exclusive Special" "description": "That product you've been eyeing — up to 75% off, the biggest discount ever"

Designing an A/B Test with different emphasis copy for the popup

3

Updating and Managing Condition-based Copy with Remote Config

Update the JSON in Remote Config. Set exposure rules and detailed changes to apply them immediately. You can target A/B Test groups directly from Remote Config without having to reconfigure users. Select the experiment key and update the copy for Group B as shown in the image below — changes are reflected immediately.

Managing A/B Test copy using Remote Config

4

Confirming Results and Optimizing with the A/B Test

With the Remote Config changes applied, check the A/B Test screen to view results like the ones shown below.

Review the success metric (click-through rate on the main popup) for each variation, select the better-performing test group as the Winner, and apply it to your service.

This way, you can run experiments without any separate code work using Remote Config, and quickly optimize your service by reviewing detailed A/B Test results.