Configure Data Segment Analytics

You can configure Data Segment Analytics in the Data Analysis tab of the A/B Test detail screen in Hackle. When Data Segment Analytics is configured, you can view granular data collected during the A/B Test (e.g., user OS information), enabling more detailed analysis.

Check the calculation criteria for Data Segment Analytics

• Both the numerator and denominator of the goals registered in the A/B Test must have property values.

• Calculations are based on user properties.

Terminology

1. Segment Criteria The subject of the detailed analysis. It represents a grouping of entities with similar characteristics or within a defined range. For example, if "analyzing by OS used by the user" is the goal, the Segment Criteria would be OS.

2. Segment Items The items selected for analysis from among the values that the Segment Criteria can have. For example, if OS is selected as the Segment Criteria, the Segment Items can include Android, iOS, Windows, etc.

3. Property For the concept of property and more detailed information, please refer to the Property document.

Relationship between Segment Criteria/Segment Items and Properties

• Segment Criteria can be thought of as the Property Key, and Segment Items can be thought of as itemized Property Values.

• Note that property data collected via SDK must exist for the Segment Criteria list to appear and for Segment Items to be selectable.

Policy

1. Configuration

   • You can set Segment Criteria per goal.

   • By default, 2 Segment Criteria are provided, and up to 3 additional Segment Criteria can be added.

2. Differences by A/B Test Status

   • When the A/B Test is in Draft, Running, or Paused status, you can add or delete Segment Criteria.

   • When the A/B Test is in Stopped or Archived status, you cannot add or delete Segment Criteria.

3. Data Viewing

   • Data for selected Segment Criteria can be viewed within 1 hour of starting the A/B Test.

   • If Segment Criteria are added while the A/B Test is running, it may take up to 1 hour for data to update.

Default Segment Criteria

The default Segment Criteria provided by Hackle are Platform and Browser. You can activate or deactivate them when the A/B Test status is Draft, Running, or Paused. When the A/B Test is in Stopped or Archived status, default Segment Criteria that have not been activated will not be visible.

Activating

You can activate neither, one, or both. For goals where no default Segment Criteria have been activated, the following default screen is displayed:

The content shown in the grayed-out area is hypothetical data for illustration purposes.

After selecting the desired Segment Criteria, click Analyze in Detail in the center of the gray area to activate that Segment Criteria. After the data updates, you can check values per Segment Item.

Data updates may take up to 1 hour.

Deactivating

Once a default Segment Criteria is activated, it cannot be changed back to an inactive state.

Custom Configuration

The steps for custom configuration are as follows:

1. Click the + Set Manually button at the far right of Data Segment Analytics to open the configuration dialog.

2. Select the desired Segment Criteria in the dialog.

3. Add Segment Items for that Segment Criteria in the dialog and save.

Assuming the configuration dialog is open, the above steps are explained in more detail below.

Selecting Segment Criteria

Select the Segment Criteria to add on the left side of the custom configuration dialog. The types of Segment Criteria available for selection are as follows:

1. Hackle-provided Properties Properties that Hackle collects and provides by default

2. Custom Properties User-defined properties collected after SDK integration

For example, assume that the Hackle-provided properties Platform, Device, and OS have been collected via SDK, and custom properties Membership, Location, User id, and Platform have also been collected.

Properties provided by Hackle have a Hackle logo to the left of the property name. Properties collected directly through the SDK only show the property name and appear below the Hackle-provided properties.

If no property data collected via SDK exists, you will see a message like the above indicating no Segment Criteria are available for selection. For how to resolve this, refer to the Troubleshooting When There Is No Data document.

Editing Segment Items

When a Segment Criteria is added, the most frequently aggregated Segment Items over the last 24 hours are automatically selected, up to 4 items. Since up to 4 Segment Items can be added, if all 4 are already selected and you want to change a Segment Item, you need to delete or add Segment Items to adjust.

Deleting a Segment Item

Click the trash icon to the right of the item you want to delete to remove that Segment Item.

Click the trash icon on the right of the item to delete it.

Adding a Segment Item

If there are fewer than 4 Segment Items, you can add more. Click the + Add Item button in the lower right corner.

You can locate the 'Add Item' button here.

Selecting Values and Saving

Click the triangle (▼) icon to the right of the item to open the dropdown list. This list shows the values collected for that Segment Criteria. Select a value from this list. You can select a single value or multiple values.

To apply the selected values, you must click Save. (If nothing is selected, the Save button will not be enabled!)

The Save button is enabled only when at least one item is selected.

About the "Others" Item

If there are selectable values that have not been chosen, they are grouped as the Others item.

If the selectable values are RED, GREEN, and YELLOW, and each has been added as a separate item, then all values are already individual items, so the "Others" item will not appear.

On the other hand, if there are six selectable values — PC, Mobile, Tablet, TV, Watch, and Etc — and PC, Mobile, Tablet, and TV have each been added as separate items, then Watch and Etc, which have not been selected for any item, are grouped as "Others."

Unselected remaining items are grouped as "Others."

Completing Custom Configuration

When there is nothing more to change in the selected items, click Save to finalize the configuration.

After data updates, you can check values per Segment Item.

Data updates may take up to 1 hour.

Deleting a Segment Criteria

Custom Segment Criteria added through the custom configuration can be deleted as needed. However, deletion is not possible when in Stopped or Archived status.

After selecting the Segment Criteria to delete, click the Delete button in the lower right corner. A confirmation popup will appear, and clicking Delete again will remove that Segment Criteria.

Custom Configuration Display Criteria

If Membership was selected as the Segment Criteria, it will appear to the right of the default Segment Criteria as shown below, and you can see that additional custom configuration is available.

Since up to 3 custom configurations are allowed, the Set Manually button does not appear when all 3 have been configured as shown below.