Set Goals (Add Types)

You can register multiple goals to measure in each A/B Test, and previously registered goals can be easily re-added.

About goals

A goal is the means to measure the performance of an A/B Test — it is the measurement criterion (metric) used to determine whether the Treatment (Group B, C, D, ...) achieves better results than the Control (Group A).

For example, suppose a company is considering changing the color of the purchase button. The reason for the change is to get users to click the purchase button more. In this case, to verify whether the color change is effective and which color is more effective, they can run an A/B Test on the purchase button's color. So, how should the goal be configured?

First, since the change is only meaningful if the purchase button is clicked more, you can set 'purchase button click count' as a goal. Also, getting users to click the purchase button more is ultimately about increasing the purchase conversion rate, so 'purchase conversion rate' can also be set as a goal.

🚧 You can now set goals even after starting the A/B Test.

You can add goals that you missed before starting the test.

Select goals

First, click the Set Goals button in the upper right of the Data Analytics tab on the A/B Test detail page. When you click the Set Goals button, a dialog like the image below appears.

There are two types of goals you can select:

1. Recommended: Goals recommended by Hackle.

2. Existing: Goals configured in other A/B Tests appear in the list.

If none of the recommended or existing goals match what you need, click the Create goal manually button to create one yourself. (The method for creating manually is explained right below in this document.)

At least 1 goal must be selected, and you can select multiple at once.

When 2 or more goals are selected

Once you have selected all goals, click the Save button to complete the configuration. After the goal configuration is complete, you can view the configured goals in the Goals tab on the A/B Test detail page.

Create a goal manually

Clicking the Create goal manually button displays a dialog like the one below. You can see the goal name, the goal settings to define the measurement method, and the additional settings at the bottom. In particular, you can see that the goal settings area lets you configure the numerator and denominator of the goal separately.

Create goal manually popup

Step 1. Enter goal name

Enter the name of the goal. There is a maximum length limit of 50 characters.

Step 2. Configure data settings

Configure which users to target and what data to analyze. Setting 'which users to target' is called the denominator configuration, and setting 'what data to analyze' is called the numerator configuration.

1. Numerator configuration An event is required for configuration. You can either select an existing event or create a new one. If you need help creating an event, refer to the Create Event document.

2. Denominator configuration You can choose from 2 options:

• All exposures basis: Refers to all users exposed to the A/B Test.

• Specific event basis: Refers to users who, among those exposed to the A/B Test, additionally triggered the selected event. For example, if you set the event to 'login', only users who triggered the 'login' event among all test participants will be included in the denominator. Note: a paid plan (Pro plan or higher) is required to use the specific event basis.

📘 Why are events necessary?

For example, a company is considering changing the color of the purchase button. To verify if the change is effective, they create an A/B Test and set two goals: 'purchase button click count' and 'purchase conversion rate'.

• To calculate 'purchase button click count', they need to collect the number of times users click the purchase button. So they create an event called 'purchase button click' and count the number of times that event occurred.

• To calculate 'purchase conversion rate', they need to know how many times user purchases were completed. So they create a 'purchase completed' event and collect the number of occurrences.

In this way, to determine whether a goal has been achieved, it is important to know how many times the event related to the goal occurred.

📘 How does entering an event connect to the actual code?

1. When you select an event auto-collected via SDK integration In this case, since you are selecting an event that already exists in the code, data calculation is possible automatically.

2. When you select a manually entered event In this case, since you are selecting an event that does not exist in the code, you need to instrument that event at the desired location in the code before starting the test for data calculation to work smoothly.

Step 2-1. All exposures basis

All users exposed to the A/B Test are set as the denominator of the goal. Configure the denominator's calculation type and numerator. For the numerator configuration, you can select a measurement event, calculation type, and filter. Results are provided considering the calculation type and filter conditions of the selected event relative to users exposed to the A/B Test.

All exposures basis (users exposed to A/B Test) - select event and calculation type

Filters help you configure more precise metrics by using the attribute information of users or events.

For example, in an A/B Test running on a specific category page such as fashion, you may want to measure 'fashion product purchase conversion rate'. In this situation, you can use filters to set the desired metric.

In the first box of the filter, select the desired user or event property, and in the second box, select the desired values from the selected property.

If you need to understand properties, refer to the [Properties] document.

🚧 If you create a metric with a filter selected, you will not be able to use the Data Segment Analytics feature.

This feature is coming soon.

1414

When selecting All exposures basis as the denominator, you can configure 5 types of goals:

Numerator	Denominator	Description	Example
Users	Users	Calculates the ratio of users who triggered the selected event out of all users exposed to the A/B Test.	Purchase conversion rate, banner click rate, search conversion rate
Count	Users	Calculates the average number of times the selected event was triggered by users exposed to the A/B Test.	Average button clicks per user, average purchases per user
Value	Users	Calculates the average value generated from the selected event by users exposed to the A/B Test.	Average purchase amount per user
Value	Count	Calculates the average value generated from the selected event relative to the number of times users were exposed to the A/B Test.	Average page loading speed (Latency)
Count	Count	Calculates the average number of times the selected event occurred relative to the number of A/B Test exposures.	Average clicks per A/B Test exposure

Step 2-2. Specific event basis

🚧 Please check your plan.

A paid plan (Pro plan or higher) is required to use specific event basis.

When selecting specific event basis, you must configure both the denominator and numerator of the goal. Filter configuration is also available if needed. In this case, you can select the event corresponding to the denominator of the metric, and the denominator will be set to the users who triggered the specific event or the number of event occurrences among those exposed to the A/B Test.

Manual configuration - event occurrence and basis setting

For the numerator configuration, select both the event and calculation type, the same as when selecting All exposures basis for the denominator. However, for the denominator configuration, the availability of event and calculation type selection varies depending on which calculation type was selected in the numerator configuration.

Also, only cases where the user triggered the event selected in the denominator after being exposed to the experiment, and then triggered the numerator event, are included in the metric calculation.

📘 Only cases where the user triggered the numerator event after triggering the denominator event are included in the calculation.

When selecting 'Specific event basis', only cases where the user was exposed to the A/B Test, then triggered the denominator event, and then triggered the numerator event are reflected in the metric calculation.

When specific event basis is selected as the denominator - calculated considering the before/after relationship of denominator/numerator event occurrences

The types of metrics available when selecting Specific event basis are as follows:

Numerator	Denominator	Description	Example
Users	Users	Calculates the ratio of users who triggered the numerator event after triggering the denominator event.	Purchase conversion rate from add-to-cart, button click rate from landing page entry
Value	Users	Calculates the average value generated from the numerator event after users triggered the denominator event.	Average purchase amount per buyer
Count	Count	Calculates the number of times the numerator event occurred after the denominator event occurred.	Search result click conversion rate from search attempts
Count	Users	Calculates the number of times the numerator event occurred after users triggered the denominator event.	Average purchases per buyer
Value	Count	Calculates the average value generated from the numerator event after the denominator event occurred, relative to the number of times the denominator event was triggered.	Average order value (AOV)

Step 3. Goal description

Description is optional and can be left blank, but entering a description helps other team members understand what the goal is.

Step 4. Success criteria

Select success criteria to distinguish the characteristics of the goal. For example, purchase amount is better when it increases, while the number of returns is better when it decreases.

Enter goal description and select success criteria

• Increase compared to Group A: When a higher value is better ** Example goals: purchase conversion rate, number of searches, number of subscribers, etc.

• Decrease compared to Group A: When a lower value is better ** Example goals: number of payment cancellations, page loading time, etc.

Step 5. Create

Click the Create button in the lower right to complete creating the goal manually. The created goal appears in the list as the Existing type, and you can see that it is automatically selected.

Delete a goal

If you no longer need to collect data for a specific goal or created a goal by mistake and want to delete it, you can delete that goal. However, just like goal configuration, deletion is also only possible before the A/B Test starts.

Click the Set Goals button again. You can see the currently selected goals on the right side of the dialog.

Click the X icon on the right side of each goal to delete it. After deletion, click the Save button to apply the changes.