Hackle vs Optimizely
Purpose
This document was written to compare the key technical differences between Hackle A/B Testing and Optimizely, and to explain the additional advantages Hackle provides.
Hackle offers faster metric analysis at a more reasonable price compared to Optimizely.
Key Differences Summary
Metric Configuration
Custom metric setup (numerator/denominator), segment filtering, advanced metrics such as AOV/ARPU
Primarily preset metrics
Analysis Features
Segment-based deep analysis, real-time Funnel Analysis integrated
Basic A/B Test result analysis
Pricing
Free plan and usage-based billing (MAU), monthly contract supported
Fixed annual contract based on MAU
Technical Support
Korean-language support from a local support team (real-time communication available)
Support through overseas offices (time zone and language barriers)
Test Stability
Running experiments continue even when MAU quota is exceeded
All experiments automatically stop when MAU quota is exceeded
Result Updates
Results updated on an hourly basis regardless of data volume
Result updates may be delayed as data volume grows
Detailed Description
1. Flexible Metric Configuration
Hackle allows you to customize the numerator and denominator of metrics using any event you want, and provides a feature to filter metrics based on specific user segments. This enables precise measurement of business-critical metrics like AOV (Average Order Value) and ARPU (Average Revenue Per User) — in addition to basic conversion rates.
2. Segment-Based Deep Analysis
You can segment and analyze metrics across various criteria such as platform (iOS, Android, Web), app version, and user properties (e.g., membership tier, first purchase status). Based on the property information passed via the SDK, you can check the impact of specific segments on experiment results without any additional configuration.
3. Integrated Data Analytics
In addition to A/B Testing, advanced data analysis features such as data insights and User Funnel Analysis are provided together. This enables data-driven decision-making from hypothesis formation through to result interpretation.
4. Reasonable Pricing
Hackle offers a flexible usage-based billing policy with a monthly contract option, lowering the barrier to adoption and enabling cost-effective operations. Optimizely generally has a high fixed-cost policy based on annual contracts.
5. Rapid Technical Support
Hackle's locally based technical support team resolves issues quickly in Korean, ensuring smooth communication and stable service operation. Optimizely provides technical support through overseas offices, which can lead to time zone and language barriers.
6. Uninterrupted Testing Environment
Hackle does not stop running experiments even when the quota is exceeded, allowing tests to be completed stably without data loss. Optimizely stops all running A/B Tests when the monthly active user (MAU) quota set for the plan is exceeded.
7. Fast Result Update Cycle
Hackle updates A/B Test results quickly on a consistent hourly basis regardless of data volume. This enables fast result review and decision-making. With Optimizely, result updates may be delayed as the accumulated data volume grows.
Last updated