How the platform finds the last-known change

Which recorded change caused an incident, if any — the step in the flow, the suspect and proven-good criteria, the change sources per onboarding option and the exact prompt section.

Concept · Updated 2026-10-26 · Describes release 0.1.164

Most incidents follow a change. Deciding which change caused one, if any, is split between fixed code and the analysis. Code records the changes, sets the window, ranks every change in it and decides what is proven good. The analysis tests the top few against evidence and names one, or none. Code then checks that pick against its own ranking before anything is shown or acted on. The result is the incident's Probable cause card (Read an incident: Probable cause).

The step in the incident flow

The step is part of the analysis (L4), before its summary:

1. Error start (code) — from the alert itself: CloudWatch's first breaching datapoint, the Azure Monitor evaluation window, the sender's start time, else its arrival.
2. Healthy since (code) — the latest of: where the previous error on the same correlation key ended, where the alarm's own history last recovered, and when the fix in a verified last-known-good record of the key or resource was verified. With none of these, the look-back cap (aiops_lkg_lookback_h, 72 h).
3. Candidates (code) — every applied change the ledger recorded between those two points: the platform's own changes and every change a source below reported. Denied attempts, changes that never ran and changes made after the error started (the platform's own fixes included) are never candidates.
4. Ranking (code) — each candidate scored and given a verdict (below); the top 5 (aiops_lkg_top_n) reach the analysis, the rest are counted.
5. The prompt section (code → analysis) — the ranked list goes into the analysis's first prompt as one section, fenced as estate data and PII-scrubbed (the exact text).
6. The test (analysis) — the analysis cannot finish until it has loaded the last-change-check skill. It reads the failing resource's logs, tests each candidate from the top, and names one change, or none, in its wrap-up.
7. The check (code) — a pick outside the ranked list is flagged and never acted on; the card shows the analysis's pick only when it is in the top 5, else the platform's #1 as a ranking.

Switch: aiops_lkg_cause_block (on by default) puts the section in the prompt; off, the analysis works from change_lookup alone. Every setting named here is in the Settings reference.

Good or bad: the criteria

Code gives each candidate one verdict:

The score is 100 × relation × time band × kind, every factor shown to the analysis:

• Relation — named by the incident 1.0 · the same resource 1.0 · the same service 1.0 · attached to the resource 0.8 · one hop upstream 0.7 · two hops upstream 0.5 · the same resource group 0.4 · downstream 0.2 · time only 0.1.
• Time band (before the error start) — within 15 minutes 1.0 · within 1 hour 0.8 · within the suspect window 0.6 · within 24 hours 0.3 · older 0.1.
• Kind — a change that only edits tags × 0.3.

What makes the analysis name a change, from the last-change-check skill:

• a mechanism: how this change could cause this error (a revoked port gives timeouts on that port; a changed setting is named by the first error);
• the timing: the error started minutes after it;
• the state now against the before-state: the change is still in place;
• only a change inside the window counts; an older change needs a named delayed mechanism (an expiry, a TTL, a scheduled job, a cache flush);
• revert is set only when it is confident and the change's before-state is recorded; every candidate ruled out is listed with why.

A good change is one the evidence clears: proven_good, or tested and ruled out with a reason. An empty list is not proof that nothing changed: what the ledger cannot see is listed in the section itself (What is never captured).

Sources per onboarding option

A change can be ranked only when a source recorded it. What each onboarding option adds:

Set-up for each: Report changes to the change webhook (Kubernetes, Argo CD, repositories, and what the Probable cause card needs), Set up AWS, Set up Azure and Ticket systems. The harvest's caps and dedupe are in the Changes reference.

The prompt section the analysis reads

The section sits in the analysis's first prompt after the alert data and the service relationships, at most 2,900 characters; the ranked lines are budgeted first, the optional lines are cut to what is left. Its fixed lines, as the analysis reads them (the suspect hours are the workspace's own aiops_lkg_suspect_h; the proven-good hours are the platform's AIOPS_LKG_PROVEN_GOOD_H):

CHANGE CANDIDATES — ranked by the platform (deterministic: relation x time x kind). Name at most ONE as suspect_change, or none.
Verdicts: suspect = related x0.4 or more and at most 2 h before the error; proven_good = its alarm evaluated data and stayed OK from before it to 24 h after — never the cause, ranked last; possible = the rest. Name one only when a mechanism explains THIS error, the timing fits and its effect is still in place (the resource now vs its recorded before-state); otherwise none.
Error start: <time> (<basis>); alert fired <time>.
Window: <from> to <to> — <how the window was bounded>.
Audit log read: <connection> through <time> (<n> min after the error start) — cloud audit logs land 5-20 min after a write, so a newer change may be missing.

Then, fenced as estate data, one line per candidate, factors first:

1. CHG-1a2b3c4d · score 100 suspect · 8 min before the error · same resource as the alert x1.0 · within 15 min x1.0 · before-state recorded · RevokeSecurityGroupIngress on sg-0abc by ops-admin (aws:prod)
+2 more; 1 denied attempts ignored

After the list, each only when it applies: the changes made during the incident ("never the cause"), earlier alerts on related resources, the alarm's log groups, the restore-first advice, and what the ledger cannot see ("absence here is no proof").

The finish gate, when the analysis tries to finish without the skill:

You have not correlated this incident with recent changes. Load the 'last-change-check' skill and check deploys/commits/change tickets against the alert start time, then name the one change that caused it — or none — in suspect_change. If no change datasource is connected, state that explicitly in last_change_checked.

The answer is two fields of the wrap-up: last_change_checked (what was checked and the conclusion, or why it was not checkable) and suspect_change (change_id exactly as listed or empty, confidence, reasoning, alternatives, revert). To drill in, the analysis calls change_lookup with a resource or a service and a window; its default window is the section's.

Next steps

• Next: Storms, unmapped and flapping signals
• All Work with the agents pages
• Documentation home