> For the complete documentation index, see [llms.txt](https://docs.hackle.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.hackle.io/en/ab-test/data-analysis/global-impact.md). # Global Impact {% hint style="info" %} Global Impact metrics are not available for MAB Tests. {% endhint %} ### What is Global Impact? Global Impact is a metric that measures the expected performance when A/B Test results are applied across the entire site.\ It predicts the actual business impact from a whole-service perspective, including users who did not participate in the experiment. ### When should you use it? #### When the experiment was conducted on a limited audience * Specific OS target: When applying an experiment targeting only iOS users to the entire user base including Android * Specific cohort: When expanding an experiment targeting only new users to all users * Regional test: When expanding an experiment targeting users in a specific region to the national/global level * Partial traffic: Predicting the effect of applying an experiment run on 10% traffic to 100% #### When the experiment has network effects * Social features: Friend recommendations, feed algorithm changes * Marketplace: Seller-buyer interaction changes * Community: Post and comment policy changes #### When the experiment is affected by or may affect internal resource constraints * Inventory management: Stock shortage due to discount policies * Server resources: System load from new features * Staffing: Increased customer service requests ### What problem does it solve? An A/B Test generally compares only the group of users who participated in the experiment. However, in actual business decision-making, you want to know the impact when a feature is deployed to the entire site. The Global Impact metric calculates the expected performance when a specific variant is applied site-wide, predicting the overall impact including not only experiment participants but also non-experiment users. ### How is it measured? 1. Basic settings * Control: Users receiving the existing experience * Treatment: Users receiving the new experience * Site-wide users: All users not participating in the test 2. Metric calculation * Direct effect: Comparison of Control vs. Treatment metrics * Site-wide effect: Change in overall service metrics #### Simple example **Experiment targeting iOS users** * Total users: 10,000 (iOS 3,000, Android 7,000) * Experiment participants: 1,500 (50% of iOS users) * Group A (Control): 500 users, 45 converting users * Group B (Treatment): 500 users, 40 converting users * Group C (Treatment): 500 users, 60 converting users * Users not participating in experiment: 8,500 users, 850 converting users **Conversion rate calculation** * Conversion rate for experiment participants * Group A: 9.0% (45 conversions / 500 users) * Group B: 8.0% (40 conversions / 500 users) * Group C: 12.0% (60 conversions / 500 users) * Conversion rate for non-experiment users: 10.0% (850 conversions / 8,500 users) * Site-wide conversion rate * Site-wide expected numerator = Non-experiment user count × Non-experiment conversion rate + Total experiment user count × Specific treatment conversion rate * Site-wide expected denominator = Total user count * Group A * Site-wide expected numerator = (8,500 × 0.10) + (1500 × 0.09) = 850 + 135 = 985 * Site-wide denominator = 10,000 * **Site-wide conversion rate = 985 ÷ 10,000 = 9.85%** * Group B * Site-wide expected numerator = (8,500 × 0.10) + (1500 × 0.08) = 850 + 120 = 970 * Site-wide denominator = 10,000 * **Site-wide conversion rate = 970 ÷ 10,000 = 9.7%** * Group C * Site-wide expected numerator = (8,500 × 0.10) + (1500 × 0.12) = 850 + 180 = 1,030 * Site-wide denominator = 10,000 * **Site-wide conversion rate = 1,030 ÷ 10,000 = 10.3%** Interpretation: If the iOS experiment results are applied to the entire service (including Android), the overall conversion rate is expected to be 9.85% for Group A, 9.7% for Group B, and 10.3% for Group C. ### How to use 1. In the Metric Management menu of the Dashboard, find the metric for which you want to understand Global Impact among the existing registered metrics, and turn on the Global Impact toggle. ![](/files/I8u2jHKHNnKd9VBd3LAz) 2. Once the A/B Test is running, the Global Impact metric is calculated automatically. ![Global Impact Metric](/files/JiDo9xf7cbAr0wyXHNfb) ![Data Summary](/files/IfxyASQ0ATfnchAF5Tx1) ### Tips * Establish prior hypotheses: Define expected site-wide impacts in advance * Phased rollout: Small-scale test → Check Global Impact → Gradual expansion * Multi-angle analysis: Comprehensive review of revenue, usability, and operating costs * Segment-by-segment validation: When running experiments on specific OS/cohort, review applicability to other segments By using Global Impact metrics, you can accurately measure the true business impact of A/B Tests and make data-driven decisions. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.hackle.io/en/ab-test/data-analysis/global-impact.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.