Hackle vs VWO

Purpose

This document introduces the key differentiators you can experience with Hackle compared to VWO (Visual Web Optimizer) Web. Through this document, you will understand the following two points:

1. Features that Hackle provides in addition to VWO

2. The advantages Hackle users can gain from those features

Key Differences

Performance & Data

Category	Hackle	VWO
Service Speed Impact	SDK-based, with no impact on service speed	Loading may slow down as visual-editor-based tests increase
Configuration Reflection Speed	Changes are reflected in near real-time	Can take several hours
Results Update	At least once per hour	Up to 24-hour delay
SPA Support	Fully supported	May not work correctly in some cases
Data Accuracy	Based on the full dataset without sampling	Not specifically stated

Features & Flexibility

Category	Hackle	VWO
Metric Configuration	Freely configure numerator/denominator — AOV/ARPU/ARPPU, etc.	Limited
Segment Analysis	Based on platform, browser, app version, and custom properties	Limited
Targeting	Custom Targeting with multi-condition combinations	Basic Targeting conditions
SDK Support	Client + Server SDK (mobile/web/server)	Web browser-based only
User Identification	Identify by any criteria you want, such as Device ID	Session-based

Operations & Support

Category	Hackle	VWO
Technical Support	Always-on Slack Hotline	Not specifically stated
Experiment/Results Limits	Unlimited concurrent experiments and results	Limits apply
Environment Separation	Both Development and Production Environments provided	No environment separation

Detailed Description

1. Hackle A/B Testing does not slow down users' service experience.

Adopting Hackle A/B Testing will not slow down your platform's service speed. In contrast, as the number of A/B tests conducted using a Visual Editor or WYSIWYG editor increases, the loading time required to render the screen shown to users can become progressively slower. This occurs because the editor determines what screen the user should see.

2. Hackle A/B Testing reflects experiment setting changes in near real-time.

After integrating the Hackle SDK, updated configuration information from the Dashboard is periodically received and applied to the code.

Compared to VWO, where starting an A/B Test can take several hours, the Hackle SDK has a short cycle for updating configuration changes, allowing near real-time control over the experiment's progress.

3. Hackle A/B Testing updates result data frequently (at least once per hour).

Since VWO provides experiment results based on data aggregated up to the day before, there can be a time lag of up to 24 hours in some cases. In this situation, users find it difficult to make immediate judgments and responses about current conditions.

With Hackle, experiment results are updated at least once per hour, keeping the time lag within 1 hour.

4. Hackle A/B Testing supports SPA (Single Page Application).

VWO may fail to implement A/B tests correctly or fail to collect user data in services built with SPA (Single Page Application) — where only the parts of the current web page that need to change are updated, rather than loading a new page in the traditional browser sense. In contrast, Hackle has no issues running A/B tests on SPA-based services.

5. Hackle A/B Testing allows you to freely configure any metrics you want.

In Hackle, you can select any event as the numerator/denominator for a metric, and through various calculation types, you can measure metrics beyond just conversion rate — including Average Order Value (AOV), Average Revenue Per User (ARPU), and Average Revenue Per Paying User (ARPPU).

Additionally, a filter configuration feature is provided to measure metrics for specific user segments.

6. Hackle A/B Testing allows metrics to be analyzed at the segment level.

The results for the metrics you want to measure in an experiment can be analyzed by segment — based on platform (iOS, Android, Web, etc.), browser, app version, or internally managed property information (e.g., membership status, first purchase status, gender, age group, etc.).

This allows you to check whether an A/B Test metric is only affecting a specific segment.

7. Hackle A/B Testing provides more advanced Targeting features.

Hackle supports setting more diverse and customizable Targeting conditions than those provided by VWO, and allows multiple conditions to be set during Targeting, enabling you to run A/B Tests for the specific users you want.

8. Hackle provides both Client SDK and Server SDK.

VWO Web only supports web browser-based A/B Testing. In contrast, Hackle provides both client and server SDKs, supporting A/B Testing and Variation creation/implementation across mobile, desktop, server environments, and all programming languages used to build platforms.

9. Hackle A/B Testing can identify users by any criteria, not just sessions — such as Device ID.

As described in Hackle's User Identifier (User Identifier) document, accurately defining user identifiers is critically important for A/B Testing. In Hackle, customers can define user identifiers using any measurement criteria they want, overcoming the limitations of session-based user identification.

10. Hackle A/B Testing provides results based on the full dataset without sampling.

Hackle A/B Testing uses the complete dataset when calculating results, providing accurate calculation outputs.

11. Hackle provides constant technical support for customers via a Hotline.

Hackle supports a private channel per customer on Slack Messenger so you can comfortably ask questions about anything you've been curious about — including implementing A/B Test code, the SDK integration process, or running experiments.

12. Hackle A/B Testing is easy to run because it provides both Development and Production Environments for each experiment.

VWO does not separate Production and Development environments, which creates the possibility that developers might confuse them. When implementing A/B Tests in code, it is necessary to test in the Development Environment first, but without environment separation there is a high risk of accidentally modifying Production settings — which are exposed to actual users.

With Hackle, both Production and Development Environments are provided for each experiment, eliminating this problem.