Datadog
Connect the Datadog API with read-only keys for monitors, metrics, events and logs, and send monitor notifications to the workspace webhook with the paste-in template.
How-to · Updated 2026-10-17 · Describes release 0.1.164
Overview
Datadog is two integrations in one. The connection on this page is the read side: the Incident agent reads monitors, metric queries, events and logs through it (alerting, metrics and logs are three of the categories that agent needs), the SecOps agent reads its logs, the FinOps agent reads its metrics, and the estate x-ray and the insights chat use it too. The other half is alerting: each monitor notifies a Datadog webhook that posts to this workspace, see Alerts in. Either half works on its own.
Setup
Before you start
- The
datasources.managepermission; the built-in owner and operator roles carry it. - A Datadog API key and a scoped application key owned by a reader account (Installation).
Installation
In Datadog, under Organization Settings → API Keys, create a named API key for this connection. Under Organization Settings → Application Keys, create a scoped application key owned by a dedicated reader or service account with monitors_read and metrics_read, adding only the extra read scopes the APIs you use need; an application key with no scopes inherits its owner's full permissions. The platform refuses writes through this connection whatever the keys allow, so there is no reason to grant any.
Configuration
- Open Settings · Data in → Data sources and select Connect datasource.
- Under Type, choose Datadog. Enter a Connection name; the connected list shows it, and per-connection agent tools are named after it.
- Fill the fields; the TLS-skip field sits under Advanced connection settings:
| Field (portal label) | API key | Value |
|---|---|---|
| Datadog API URL (e.g. https://api.datadoghq.com) | url |
https://api.datadoghq.com for US1; the regional API URL for every other site |
| API key | api_key |
Sent as DD-API-KEY; stored write-only |
| Application key | app_key |
Sent as DD-APPLICATION-KEY; stored write-only |
| Skip TLS verify for a self-signed cert (optional — type: true) | insecure |
Leave blank |
- Optionally, open Optional · describe this environment and say in What should AIOps know about it? what this system is and what matters.
- Under Use this connection for, choose Investigate only and select Connect and verify.
The same connection through the API (POST /api/aiops/{tenant_id}/datasources, requires datasources.manage; both keys go in secret_values):
{
"type_key": "datadog",
"name": "datadog",
"description": "Monitors and APM for the web tier; services are tagged service:<name>, env:prod.",
"config": {
"url": "https://api.datadoghq.com"
},
"secret_values": {
"api_key": "<API key>",
"app_key": "<application key>"
},
"role": "read",
"enabled": true
}
Validation
Connect and verify, and Test on the row later, sends a GET to /api/v1/validate with both headers. You should see a dialog titled <connection name> is connected with Connected — verified from the control plane, and later in the row's Last test ok · HTTP 200 /api/v1/validate — {"valid":true}. That endpoint validates the API key only: a wrong or under-scoped application key still passes Test and fails on the agent's first monitor or metric read. Confirm the application key can list your monitors and read a known metric before relying on the coverage.
If something fails
The dialog shows Needs a fix, what to fix and the check's own detail; the row is saved either way. Select Edit, correct the field the detail names and save again, or select Retry once a cause outside the form (network, vendor-side permission) is fixed. Every verdict this connector produces, and what to do about it, is in Troubleshooting below.
Data collected
Tools the agent gets
<name>_request— the HTTP tool against the API URL with the two key headers:method,path,query,body,content_type. Monitors come fromGET /api/v1/monitor, metric points fromGET /api/v1/query(also accepted overPOST) and the v2POST /api/v2/query/timeseriesand/api/v2/query/scalar, events fromPOST /api/v2/events/search, log aggregates fromPOST /api/v2/logs/analytics/aggregate.
There is no native Datadog tool. Responses are capped at 20,000 characters per call.
Read surface
GET and HEAD run freely, and so do POST requests to api/v1/query, api/v2/query/timeseries, api/v2/query/scalar, events/search and analytics/aggregate, matched on whole trailing path segments. Every other POST, PUT, PATCH and DELETE — creating or muting a monitor, scheduling a downtime, posting an event — is refused in an investigation, never offered for approval: Investigation connections are read-only. This operation was not executed; prepare an incident change using a write connection and obtain approval. The full rule table is under The read/write gate in detail in Connections and remediation access.
Alerts in
Datadog sends nothing useful until a payload template is pasted in. In Datadog open Integrations → Webhooks → New, put https://<your-host>/webhook/datadog/<workspace-slug> in URL, the token in Custom Headers as {"X-Webhook-Token": "<webhook-token>"}, and paste this as the payload:
{
"alert_id": "$ALERT_ID",
"id": "$ID",
"alert_scope": "$ALERT_SCOPE",
"title": "$ALERT_TITLE",
"body": "$EVENT_MSG",
"alert_type": "$ALERT_TYPE",
"alert_transition": "$ALERT_TRANSITION",
"tags": "$TAGS",
"date": "$DATE"
}
Then mention @webhook-<name> in each monitor's message, for the alert and for the recovery. How the adapter reads it:
tag:valuepairs in$TAGSbecome labels, so a monitor taggedservice:checkoutandenv:prodcarries the identity the platform groups on for free. Severity comes only fromalert_type:erroriscritical,warningiswarning,infoandsuccessareinfo, anything elsewarning; aseverity:tag is ignored.- The alert's identity is
$ALERT_IDplus$ALERT_SCOPE, so one group of a multi-alert monitor recovering does not close the incident for the other groups. Keep$ALERT_SCOPEin the template. - A transition containing
recoveredresolves the alert;$DATEis when the monitor fired and never part of the identity. - A payload with no title and no id is ignored.
Alert intake & tokens has a Send a test alert card for the datadog source that produces a sample payload and the curl command to post it. Alerts from your observability platform walks through the setup; the webhook reference has the endpoint and the token forms. Field by field, with the response counters and error strings: webhook reference.
Troubleshooting
| What you see | Cause | What to do |
|---|---|---|
failed · HTTP 403 /api/v1/validate |
The API key is wrong, or belongs to another Datadog site than the URL. | Check the key and use the API URL of the site that owns it. |
Test is green, but monitor or metric reads answer HTTP 403 |
/api/v1/validate checks the API key only; the application key is wrong, or lacks monitors_read / metrics_read. |
Replace the application key on Edit with one that has the read scopes. |
failed · ConnectError: … or ConnectTimeout |
The platform has no route to the Datadog API. | Check egress from the platform. |
A POST /api/v1/monitor, PATCH /api/v1/monitor/<id> or POST /api/v1/downtime answers Investigation connections are read-only. |
The write gate. | Expected. |
Datadog's webhook test answers 401 invalid webhook credentials |
Wrong token, wrong workspace slug or a disabled workspace; the endpoint never says which. | Copy the exact URL and token from Alert intake & tokens; the header wins over a token in the path. |
| Monitors fire but the incident has no service | The monitor is not tagged service:<name>, or $TAGS is missing from the template. |
Tag the monitor service:<name>; keep $TAGS in the payload. |
| An incident never resolves although the monitor recovered | alert_transition is missing from the template, or the recovery notification does not mention the webhook. |
Keep alert_transition in the template and @webhook-<name> in the recovery message. |
| A recovery closes the incident while another group of the same monitor is still alerting | $ALERT_SCOPE is missing, so every group shares one identity. |
Add alert_scope to the template. |
The generic verdict strings are explained in Test a connection and read the failure; alert-intake symptoms are in Alert sources: troubleshooting. Every connection status word is in the status and error reference.
Next steps
- Select Discover this connection in the success dialog to find the services and dependencies behind it: Launch a scan.
- Send Datadog monitor alerts in through the workspace webhook: Alerts in.
- See what else each agent needs in Choose your connection.