Hackle vs Firebase

Purpose

This document introduces the key differentiators that Hackle's A/B Testing (Experiment) and Remote Config features offer compared to Firebase, and the benefits users can gain from them.

Hackle provides more sophisticated analysis capabilities and operational efficiency compared to Firebase, supporting faster data-driven product improvement.

Key Differences

Performance & Data

Category	Hackle	Firebase
Data Reflection	Near real-time setting change updates	Up to 12-hour delay (Fetch Interval)
Data Freshness	Results updated within 1 hour	Up to 24-hour delay (based on GA data)
Data Accuracy	Analysis based on full dataset	May produce errors due to sampled data
Conversion Rate Calculation	Accurate calculation based on exposure point	Possible errors due to mismatch between distribution and exposure point
SDK Support	Both Client SDK and Server SDK provided	Only Client SDK provided

Features & Flexibility

Category	Hackle	Firebase
Metric Configuration	Custom business metrics (AOV, ARPU, etc.)	Primarily limited to default metrics
Deep Analysis	Detailed segment analysis based on properties	Limited segmentation capabilities
Targeting	Advanced targeting with multiple combined conditions	Only basic targeting conditions provided
Platform Support	Single experiment supports Android and iOS simultaneously	Separate experiments required per platform

Operations & Support

Category	Hackle	Firebase
Experiment Limits	No limit on number of goal metrics	Limited to 6 goals (1 primary + 5 secondary)
Developer Convenience	Safe deployment with complete separation of Development and Production Environments	Risk of operational errors due to no environment separation
Technical Support	1:1 real-time expert support via Slack	Limited community-based support

Detailed Description: 12 Advantages of Hackle

1. Freely Configure Metrics That Fit Your Business

You can freely set and measure any metric essential to business growth, including AOV, ARPU, and more. This is a major differentiator from Firebase, which only provides limited default goals.

2. Deep Analysis That Uncovers Hidden Insights

Experiment results can be segmented and analyzed across multiple criteria such as platform and membership tier. This provides deep insights that are difficult to discover with Firebase's limited segmentation features.

3. Precise Targeting for the Right Customers

You can precisely target the customer groups you want by combining multiple attributes. This is far more powerful than Firebase, which only provides basic targeting conditions.

4. Support Android and iOS Simultaneously with a Single Experiment

A single experiment can test both Android and iOS simultaneously, saving development resources. With Firebase, you have to create separate experiments for each platform, which adds unnecessary overhead.

5. Real-time Setting Changes Without Waiting

Changes to experiments and Remote Config settings are reflected almost instantly. Firebase's configuration update cycle is 12 hours, limiting its ability for immediate control.

6. Up-to-date Data for Fast Decision-Making

Experiment data is updated at least once every hour. Compared to Firebase, which can have a lag of up to 24 hours based on Google Analytics data, you can respond quickly to changing circumstances.

7. Both Client and Server SDK — Efficiently Test Server Logic Too

You can efficiently run server-side A/B Tests — such as search and recommendation logic — via the Server SDK. To test server logic with Firebase, which only provides a Client SDK, you would need to perform complex work on both the client and server sides.

8. Error-Free Results with 100% Data Analysis

Statistical results are calculated based on all traffic without sampling. This enables more accurate and reliable decision-making compared to Firebase, which uses sampled data.

9. Accurate Conversion Rate Calculation Logic

Conversion rates are calculated based on the moment the user is actually exposed to the experiment screen, preventing contamination of results. With Firebase, conversion rate calculation can produce errors because the distribution point and exposure point differ.

10. Dedicated Expert Support Always Available

You can receive 1:1 technical support from Hackle experts via a dedicated Slack channel for each customer. This is an incomparable advantage over Firebase's community-based support.

11. Unlimited Goal Configuration for Multi-Dimensional Performance Measurement

There is no limit to the number of goal metrics you can measure in a single experiment. This enables far more multi-dimensional performance analysis than Firebase, which limits goals to 6 per experiment.

12. A Safe Experiment Environment That Prevents Development Mistakes at the Source

Development and Production Environments are provided completely separated. This fundamentally prevents the operational mistakes that can occur with Firebase, which does not provide environment separation.