# In-App Message A/B Test

You can create multiple message variants for an in-app message campaign, test each variant, and select the best-performing message for display.

{% hint style="success" %}
Curious about real-world use cases for In-App Message A/B Tests?\
[Read the In-App Message A/B Test Guide](https://blog.hackle.io/post/inapp-ab-test)
{% endhint %}

## Step 1. Writing A/B Test Messages

Click \[+ A/B Test] in the message composition area to prepare the experiment and create multiple message variants.

![](/files/Mu9smXxrM2aBISqG5iMZ)

* You can add message variants and delete them. Write a new message or copy the current variant to create multiple messages and run an A/B Test.

![](/files/6A9ulQleexgB5Hjd6KS3)

* You can add a Control Group. Click \[Add Control Group] to run an A/B Test between users who have not seen the in-app message and those who have.

![](/files/RImbG2s2IBUQTm1LbA4v)

## Step 2. A/B Test Settings

![](/files/fdTJUHXzr10YGGFyYIDk)

* The experiment measures the performance of each message based on a 'success metric', and maintains traffic distribution every 1/24/48 hours after the message is displayed, adjusting traffic distribution based on the performance of each message.
  * Click-through rate counts only the click-through rate of links entered in the in-app message settings. (The 'Close' button is not included in click-through rate aggregation.)
  * Traffic distribution uses a method called MAB. You can learn more from the [MAB Test introduction document](/en/ab-test/mab-test/create-mab-test.md).

![](/files/hJNS5knu84tfF2ooQ00o)

* Success Metric
  * Message click-through rate: The ratio of users who clicked a button or image in the in-app message relative to total exposures.
  * When selecting directly, only metrics with a calculation type of count and a denominator of test exposure can be registered.

    ![](/files/JkmM54tF8Io67l1kkDUy)

## Step 3. Running the A/B Test

* Start: Even when an experiment is configured, click the \[Start] button to start the experiment.

![](/files/ucjcD8kezYaLELVjiFFx)

* Running the experiment
  * Once the experiment is running, you can view the exposure count, conversion count, conversion rate, and probability of being the best performer for each group. ('Conversion' means a click.)
  * In the chart below, you can view detailed information such as the conversion rate trend, traffic distribution, and cumulative distribution for each group.

![](/files/NoxDSjHqUG0qF76iSCNO)

* Pausing and ending the experiment
  * After reviewing the results, if you want to display only the best-performing message, click \[End Test] to select the best-performing message (Winner).
    * Clicking \[End Test] only allows the best-performing message (e.g., Variant B) to continue being displayed.
  * Clicking \[Pause] will pause the display of that message, and the experiment will also be paused simultaneously.

![](/files/lFBFteNVnuzfGvTxJfYt)

* End
  * After ending the experiment, click the \[End] button to end the in-app message campaign.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.hackle.io/en/crm-marketing/in-app-message-guide/in-app-message-ab-test.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.