Datadog Integration

Datadog is a SaaS-based integrated monitoring and analytics platform for large-scale applications and infrastructure.

You can track real-time metrics for servers, containers, databases, and applications, and by integrating and analyzing these with Hackle data, you can monitor your service during the gradual rollout of Feature Flags.

Datadog Integration

To integrate with Datadog, you need to enter the Datadog API Key information in the Hackle Dashboard.

Copying the Datadog API Key

Go to Datadog and find the API Key under Organization Settings > API Keys. The Organization Settings menu can be accessed by hovering over your email account in the lower left corner.

• API Keys menu URL: https://app.datadoghq.com/organization-settings/api-keys

Organization Settings > API Keys

Entering the Datadog API Key in the Hackle Dashboard

Enter the copied Datadog API Key in the Hackle Dashboard. Click the Settings icon in the upper right corner, then configure the Datadog integration under the External Integration menu.

On the Datadog integration screen shown in the image below, enter the API Key and click the 'Integrate' button to complete the configuration.

Settings (located in the upper right GNB) > External Integration

Items Sent to Datadog

Once the Datadog integration is complete, Feature Flag exposure events that occur in the Production Environment will be sent. Other events can also be sent; if configuration is needed, please contact the Hackle team.

Category	Default	Selectable Options
Events	Feature Flag events	Feature Flag, A/B Test, Remote Config, user behavior events
Configuration changes	Feature Flag rollout percentage (Rollout %), Feature Flag status (On/Off)	-
Environment	Production Environment	Production Environment, Development Environment

What You Can Monitor in Datadog

You can create a Dashboard using the Hackle Feature Flag events and other data sent to Datadog. You can build a comprehensive monitoring Dashboard by combining monitoring charts generated from Datadog-collected data with Feature Flag charts generated from Hackle data.

Widget Name in Screenshot Above	Description
Feature Flag Rollout configuration change count	The number of times a Feature Flag status (On/Off), rollout percentage (Rollout %), or targeting condition (Targeting Rule) change has occurred.
Feature Flag distribution event count	The number of times a Feature Flag distribution event has occurred.
Feature Flag Rollout configuration changes	Shows the history of Feature Flag status, rollout percentage, and targeting condition changes made in the Hackle Dashboard.

Monitoring Feature Flag Distribution Events in Datadog

Go to Add Widgets > Graphs > Timeseries.

You can check where and how many Feature Flag calls occurred at a specific point in time to determine how many customers were impacted. By selecting Metrics > hackle.feature_flag.count, you can create a widget to view the number of Feature Flag distribution events. You can also use the following variables to monitor Feature Flag distribution events under specific conditions.

• key: Feature Flag key (e.g., 15)

• variation: Distribution result (true/false)

• decision_reason: Distribution reason Feature Flag Decision

• environment: Environment (production/development)

Example Datadog Widget configuration for monitoring Feature Flag distribution event count

Monitoring Feature Flag Rollout Configuration Changes in Datadog

Go to Add Widgets > Graphs > Timeseries or List and Streams > List.

You can identify which Feature Flag had its Rollout conditions changed at a specific point in time, which helps you estimate causal relationships with elements you are monitoring in Datadog. By entering Events > source:hackle, an event is sent to Datadog every time an engineer changes a Feature Flag's status, rollout percentage, or targeting condition in the Hackle Dashboard. (e.g., changing Feature Flag rollout from 0% to 30%)

Example Datadog Widget (Graph > Timeseries) configuration for monitoring Feature Flag Rollout configuration change count

To see who changed a specific Feature Flag's Rollout conditions, when, and how, you can create a List-type Widget.

Example Datadog Widget (List and Streams > List) configuration for viewing Feature Flag Rollout configuration change history

By selecting Reporting format size as Large (full event), you can view detailed information about Feature Flag Rollout configuration changes in the Widget as shown in the image below.