Grafana

Connect Grafana with a Viewer service-account token so the agent reads dashboards and queries the backends behind them, and point Grafana alerting at the workspace webhook.

How-to · Updated 2026-10-23 · Describes release 0.1.164

Overview

Grafana is an umbrella: one connection reaches the dashboards and, through Grafana's datasource proxy, the metrics, logs and traces backends behind them. The catalog therefore lists it under Alerting, Metrics, Logs, Tracing and Dashboards, and a connected Grafana counts as metrics, logs and traces coverage for the Incident agent; the SecOps and FinOps agents, the estate x-ray and the insights chat read it too.

The connection is the read side. Grafana alerting reaches the platform through a webhook contact point, see Alerts in.

Setup

Before you start

• The datasources.manage permission; the built-in owner and operator roles carry it.
• A Grafana service-account token with the Viewer role (Installation).
• The Grafana URL reachable from the platform or from a private-network agent.

Installation

In Grafana open Administration → Users and access → Service accounts and add an account named aiops-reader with the Viewer role, then Add service account token and copy the token. Restrict the account's folders and datasource query permissions to the dashboards and backends the agent needs. A fix never runs through this connection, so there is no reason to raise it above Viewer.

Configuration

1. Open Settings · Data in → Data sources and select Connect datasource.
2. Under Type, choose Grafana. Enter a Connection name; the connected list shows it, and per-connection agent tools are named after it.
3. Fill the fields; the TLS-skip field sits under Advanced connection settings:

4. Optionally, open Optional · describe this environment and fill What should AIOps know about it? with the dashboards and datasource names that matter.
5. Under Use this connection for, choose Investigate only and select Connect and verify.

The same connection through the API (POST /api/aiops/{tenant_id}/datasources, requires datasources.manage; the token goes in secret_values):

{
  "type_key": "grafana",
  "name": "prod-grafana",
  "description": "Prod dashboards; the Checkout and Payments folders are the ones on call reads.",
  "config": {
    "url": "https://grafana.example.com"
  },
  "secret_values": {
    "api_key": "<service-account token>"
  },
  "role": "read",
  "enabled": true
}

Validation

Connect and verify, and Test on the row later, sends a GET to /api/health with Authorization: Bearer <token>. You should see a dialog titled <connection name> is connected with Connected — verified from the control plane, and later in the row's Last test ok · HTTP 200 /api/health — {"commit": …. Grafana answers that endpoint without authentication, so a green result proves the URL and TLS, not the token: a wrong token still passes Test and fails on the agent's first real request. Select Audit perms on the row to see what the token can read, or let the agent open one dashboard.

If something fails

The dialog shows Needs a fix, what to fix and the check's own detail; the row is saved either way. Select Edit, correct the field the detail names and save again, or select Retry once a cause outside the form (network, vendor-side permission) is fixed. Every verdict this connector produces, and what to do about it, is in Troubleshooting below.

Data collected

Tools the agent gets

• <name>_request — the HTTP tool against the base URL with Authorization: Bearer <token>: method, path, query, body, content_type. Dashboards come from GET /api/search and GET /api/dashboards/uid/<uid>, annotations from GET /api/annotations, the backends behind Grafana from POST /api/ds/query with the datasource uid in the body.

There is no native Grafana tool, and a Grafana connection does not register prometheus_query; a direct Prometheus connection does. Responses are capped at 20,000 characters per call.

Read surface

GET and HEAD run freely, and so do POST requests to api/ds/query and api/tsdb/query, the two query proxies. Every other POST, PUT, PATCH and DELETE — creating annotations or dashboards, silencing alerts, editing datasources — is refused in an investigation, never offered for approval: Investigation connections are read-only. This operation was not executed; prepare an incident change using a write connection and obtain approval. The full rule table is under The read/write gate in detail in Connections and remediation access.

Alerts in

Grafana unified alerting posts its native, Alertmanager-shaped payload. Add a contact point of type Webhook with the URL https://<your-host>/webhook/grafana/<workspace-slug> and the token as Authorization: Bearer <webhook-token>; keep resolved notifications on, because status per alert is what closes the incident. Add a service label to each alert rule's Labels section — Grafana forwards custom labels verbatim, and service is the identity the platform groups on. The endpoint and the token are in the webhook reference. Field by field, with the response counters and error strings: webhook reference.

Troubleshooting

The generic verdict strings and the SSRF guard are explained in Test a connection and read the failure. Every connection status word is in the status and error reference.

Next steps

• Select Discover this connection in the success dialog to find the services and dependencies behind it: Launch a scan.
• Send Grafana alerts in through the workspace webhook: Alerts in.
• See what else each agent needs in Choose your connection.