Datadog

Connect the Datadog API with read-only keys for monitors, metrics, events and logs, and send monitor notifications to the workspace webhook with the paste-in template.

How-to · Updated 2026-10-17 · Describes release 0.1.164

Overview

Datadog is two integrations in one. The connection on this page is the read side: the Incident agent reads monitors, metric queries, events and logs through it (alerting, metrics and logs are three of the categories that agent needs), the SecOps agent reads its logs, the FinOps agent reads its metrics, and the estate x-ray and the insights chat use it too. The other half is alerting: each monitor notifies a Datadog webhook that posts to this workspace, see Alerts in. Either half works on its own.

Setup

Before you start

Installation

In Datadog, under Organization Settings → API Keys, create a named API key for this connection. Under Organization Settings → Application Keys, create a scoped application key owned by a dedicated reader or service account with monitors_read and metrics_read, adding only the extra read scopes the APIs you use need; an application key with no scopes inherits its owner's full permissions. The platform refuses writes through this connection whatever the keys allow, so there is no reason to grant any.

Configuration

  1. Open Settings · Data in → Data sources and select Connect datasource.
  2. Under Type, choose Datadog. Enter a Connection name; the connected list shows it, and per-connection agent tools are named after it.
  3. Fill the fields; the TLS-skip field sits under Advanced connection settings:
Field (portal label) API key Value
Datadog API URL (e.g. https://api.datadoghq.com) url https://api.datadoghq.com for US1; the regional API URL for every other site
API key api_key Sent as DD-API-KEY; stored write-only
Application key app_key Sent as DD-APPLICATION-KEY; stored write-only
Skip TLS verify for a self-signed cert (optional — type: true) insecure Leave blank
  1. Optionally, open Optional · describe this environment and say in What should AIOps know about it? what this system is and what matters.
  2. Under Use this connection for, choose Investigate only and select Connect and verify.

The same connection through the API (POST /api/aiops/{tenant_id}/datasources, requires datasources.manage; both keys go in secret_values):

{
  "type_key": "datadog",
  "name": "datadog",
  "description": "Monitors and APM for the web tier; services are tagged service:<name>, env:prod.",
  "config": {
    "url": "https://api.datadoghq.com"
  },
  "secret_values": {
    "api_key": "<API key>",
    "app_key": "<application key>"
  },
  "role": "read",
  "enabled": true
}

Validation

Connect and verify, and Test on the row later, sends a GET to /api/v1/validate with both headers. You should see a dialog titled <connection name> is connected with Connected — verified from the control plane, and later in the row's Last test ok · HTTP 200 /api/v1/validate — {"valid":true}. That endpoint validates the API key only: a wrong or under-scoped application key still passes Test and fails on the agent's first monitor or metric read. Confirm the application key can list your monitors and read a known metric before relying on the coverage.

If something fails

The dialog shows Needs a fix, what to fix and the check's own detail; the row is saved either way. Select Edit, correct the field the detail names and save again, or select Retry once a cause outside the form (network, vendor-side permission) is fixed. Every verdict this connector produces, and what to do about it, is in Troubleshooting below.

Data collected

Tools the agent gets

There is no native Datadog tool. Responses are capped at 20,000 characters per call.

Read surface

GET and HEAD run freely, and so do POST requests to api/v1/query, api/v2/query/timeseries, api/v2/query/scalar, events/search and analytics/aggregate, matched on whole trailing path segments. Every other POST, PUT, PATCH and DELETE — creating or muting a monitor, scheduling a downtime, posting an event — is refused in an investigation, never offered for approval: Investigation connections are read-only. This operation was not executed; prepare an incident change using a write connection and obtain approval. The full rule table is under The read/write gate in detail in Connections and remediation access.

Alerts in

Datadog sends nothing useful until a payload template is pasted in. In Datadog open Integrations → Webhooks → New, put https://<your-host>/webhook/datadog/<workspace-slug> in URL, the token in Custom Headers as {"X-Webhook-Token": "<webhook-token>"}, and paste this as the payload:

{
  "alert_id": "$ALERT_ID",
  "id": "$ID",
  "alert_scope": "$ALERT_SCOPE",
  "title": "$ALERT_TITLE",
  "body": "$EVENT_MSG",
  "alert_type": "$ALERT_TYPE",
  "alert_transition": "$ALERT_TRANSITION",
  "tags": "$TAGS",
  "date": "$DATE"
}

Then mention @webhook-<name> in each monitor's message, for the alert and for the recovery. How the adapter reads it:

Alert intake & tokens has a Send a test alert card for the datadog source that produces a sample payload and the curl command to post it. Alerts from your observability platform walks through the setup; the webhook reference has the endpoint and the token forms. Field by field, with the response counters and error strings: webhook reference.

Troubleshooting

What you see Cause What to do
failed · HTTP 403 /api/v1/validate The API key is wrong, or belongs to another Datadog site than the URL. Check the key and use the API URL of the site that owns it.
Test is green, but monitor or metric reads answer HTTP 403 /api/v1/validate checks the API key only; the application key is wrong, or lacks monitors_read / metrics_read. Replace the application key on Edit with one that has the read scopes.
failed · ConnectError: … or ConnectTimeout The platform has no route to the Datadog API. Check egress from the platform.
A POST /api/v1/monitor, PATCH /api/v1/monitor/<id> or POST /api/v1/downtime answers Investigation connections are read-only. The write gate. Expected.
Datadog's webhook test answers 401 invalid webhook credentials Wrong token, wrong workspace slug or a disabled workspace; the endpoint never says which. Copy the exact URL and token from Alert intake & tokens; the header wins over a token in the path.
Monitors fire but the incident has no service The monitor is not tagged service:<name>, or $TAGS is missing from the template. Tag the monitor service:<name>; keep $TAGS in the payload.
An incident never resolves although the monitor recovered alert_transition is missing from the template, or the recovery notification does not mention the webhook. Keep alert_transition in the template and @webhook-<name> in the recovery message.
A recovery closes the incident while another group of the same monitor is still alerting $ALERT_SCOPE is missing, so every group shares one identity. Add alert_scope to the template.

The generic verdict strings are explained in Test a connection and read the failure; alert-intake symptoms are in Alert sources: troubleshooting. Every connection status word is in the status and error reference.

Next steps