# Outlier

{% hint style="info" %}
The outlier exclusion option in A/B Test results is only available in the **`Data Analysis`** tab on the A/B Test detail page.
{% endhint %}

### About Outliers

In statistics, an outlier is a value that is abnormally high or low compared to other values in an observed data set.\
Outliers can severely distort the accuracy of analysis results performed using the observed data set and can potentially lead to incorrect conclusions.\
This is because the values we use in experiment goal analysis are mostly averages. As is well known, among the representative values in statistics (mean, median, mode), the mean is highly influenced by outliers.

For example, imagine an experiment at an online shopping mall to improve the total order amount per visitor.\
Typical visitors spend an average of 100,000 KRW per week at that shopping mall. If a small number of visitors place orders 100 times higher than the average (10,000,000 KRW)?\
If such extreme users are included in a specific group in the experiment goal analysis, it creates distortions in the comparison between A/B groups and can lead to incorrect conclusions from the experiment.

For this reason, Hackle provides an option to view experiment results with outliers removed in the Data Analysis tab when the denominator of the experiment goal is a count or value.

### Outlier Detection and Handling Method

When the outlier exclusion option is activated, Hackle calculates the mean and standard deviation per goal registered in the experiment, converts the aggregated result (X) per individual user used in goal analysis to a z-score (mean = 0, standard deviation = 1), and identifies values where the result (z-score) is greater than 3 or less than -3 as outliers. These are then excluded and the goals are re-aggregated.

z-score = ( X - mean ) / standard deviation

![440](/files/gaAJfEtOu8DKsHYhFrmN)

When goals that can have outliers excluded (see table below) are registered, the outlier exclusion option can be activated as shown in the figure below.\
However, even for goals where outliers can be excluded, if there are no outliers in the source data, the results with the option activated may be the same as without it.

![2076](/files/Lb06GMxV8snLuFg5SyX9)

When **All Exposed** is selected as the denominator, the outlier exclusion option is applied to the following 2 goal types:

<table><thead><tr><th width="69.80859375">Numerator</th><th width="94.84375">Denominator</th><th>Description</th><th>Example</th></tr></thead><tbody><tr><td>Value</td><td>User count</td><td>Calculates the average value generated from the selected event by users exposed to the A/B Test.</td><td>Average purchase amount per buyer</td></tr><tr><td>Count</td><td>User count</td><td>Calculates the average number of times the selected event was triggered by users exposed to the A/B Test.</td><td>Average button clicks per user, average purchases per user</td></tr></tbody></table>

When **Specific Event** is selected as the denominator, the outlier exclusion option is applied to the following 2 goal types:

<table><thead><tr><th width="70.35546875">Numerator</th><th width="95.2109375">Denominator</th><th>Description</th><th>Example</th></tr></thead><tbody><tr><td>Value</td><td>User count</td><td>Calculates the average value generated from the numerator event by users after triggering the denominator event.</td><td>Average purchase amount per buyer</td></tr><tr><td>Count</td><td>User count</td><td>Calculates the average number of times the numerator event was triggered by users after triggering the denominator event.</td><td>Average number of product clicks in search results by users who triggered a search event</td></tr></tbody></table>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.hackle.io/en/ab-test/data-analysis/outlier.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.