Change-related incident signal

Check whether a recent deploy or config change plausibly explains an alert.

F@f

Limits to keep in mind

A plausible link, not a root cause.

Problem

Most incidents follow a change, but responders lose time finding which change and who made it.

Goal

Notify the change owner early when a recent change plausibly explains the problem, so they can consider a rollback.

Input / state

The alert and the list of recent changes, with time gaps computed in code.

alert
Alert, service and when it started.
recent_changes
Deploys, flags and config changes to the service and its dependencies, each with minutes before the alert.

Decision question

NoulQuestion key change_related

Could one of these recent changes plausibly explain this alert?

What counts as yes and no

  • trueA change touched the affected service or a dependency shortly before the alert, in a related area.
  • falseNo recent change touches the affected service, its dependencies, or the area that is failing.

Allowed outcomes

Decided in your code from the answer and its confidence. Thresholds are yours to set and test.

  • Notify change ownerProbability of yes is above your upper threshold.
  • UnclearProbability falls between your two thresholds.
  • No related changeProbability of yes is below your lower threshold.

Example scenarios

Predefined demonstrations. No model is called; the outcome shown is an illustrative judgment.

Example state

{
  "alert": "checkout-5xx-rate, service checkout",
  "recent_changes": [
    {
      "change": "checkout v4.12 deploy: new payment client",
      "minutes_before": 8
    }
  ]
}

Outcome

Illustrative
Notify change owner

Probability of yes is above your upper threshold.

Why

A deploy to the same service, in the failing area, just before the alert.

Using it well

  • Compute the minutes between each change and the alert in code; don't ask the model to subtract timestamps.
  • Never roll back automatically from this answer. The change owner and the responder decide together.

Request definition

POST https://api.typesafe.ai/v1/systemone
{
  "model": "jev-latest",
  "state": {
    "alert": "<alert>",
    "recent_changes": "<recent_changes>"
  },
  "questions": {
    "change_related": {
      "type": "noul",
      "instructions": "Could one of these recent changes plausibly explain this alert?",
      "criteria": {
        "true": "A change touched the affected service or a dependency shortly before the alert, in a related area.",
        "false": "No recent change touches the affected service, its dependencies, or the area that is failing."
      }
    }
  }
}

Request shape from the System One API reference; a Kev server accepts the same body. Replace the placeholder state with your data. Requests need your own API key or server.

Version History

v1Current Versionabout 8 hours ago@f

Initial version

Comments (0)