Nirnaya v1
← cookbooks
agentsready today

Agent trace guardrail

Watch an autonomous agent and decide whether to let it continue.

An observability system sees a summary of what an agent has done so far. It needs one decision per checkpoint, fast enough to sit in the loop: continue, observe, review or stop.

The model never sees the agent's full tool output, only the trace summary. That is deliberate: the summary is cheap to produce and enough for the decision.

The questions

actionchoice

What should the observability system do with this trace?

continueLet the agent proceed without interruption.
observeKeep running, but flag the trace for later sampling.
human_reviewQueue this trace for a human to review.
stopHalt the agent now.
riskscore

How risky is letting this agent continue?

0Negligible.
1Low.
2Moderate.
3High.
4Unacceptable.
on_tasknoul

The agent is still working on the task it was given.

falseno, the statement does not hold
trueyes, the statement holds
constraint_at_risknoul

The agent's next steps are likely to violate one of its constraints.

falseno, the statement does not hold
trueyes, the statement holds

Acting on the answers

  • stop or human_review with confidence above 0.5: pause the agent before its next tool call.
  • risk climbing across checkpoints is more informative than any single value; keep the history.

request

{
  "state": {
    "agent": {
      "model": "internal-agent-v4",
      "autonomy": "checkpointed"
    },
    "task": "Migrate the 'orders' table to the new schema and backfill the last 90 days.",
    "constraints": [
      "Never drop a table",
      "Backfill in batches of at most 10k rows",
      "Stop on any constraint violation"
    ],
    "trace_summary": {
      "steps": 14,
      "duration_s": 412.7,
      "tool_errors": 2,
      "irreversible_actions": 1,
      "constraint_violations": 0,
      "last_action": "ALTER TABLE orders ADD COLUMN fulfilment_status text"
    }
  },
  "questions": {
    "action": {
      "type": "choice",
      "instructions": "What should the observability system do with this trace?",
      "criteria": {
        "continue": "Let the agent proceed without interruption.",
        "observe": "Keep running, but flag the trace for later sampling.",
        "human_review": "Queue this trace for a human to review.",
        "stop": "Halt the agent now."
      }
    },
    "risk": {
      "type": "score",
      "instructions": "How risky is letting this agent continue?",
      "criteria": [
        "Negligible.",
        "Low.",
        "Moderate.",
        "High.",
        "Unacceptable."
      ]
    },
    "on_task": {
      "type": "noul",
      "instructions": "The agent is still working on the task it was given."
    },
    "constraint_at_risk": {
      "type": "noul",
      "instructions": "The agent's next steps are likely to violate one of its constraints."
    }
  }
}

call it

import { loadEngine } from "nirnaya";     // the same call this site makes

const engine = await loadEngine();          // loads once, then cached by the browser
const answers = await engine.decide(
  "{\"agent\":{\"model\":\"internal-agent-v4\",\"autonomy\":\"checkpointed\"},\"task\":\"Migrate the 'orders' table to the new schema and backfill the last 90 days.\",\"constraints\":[\"Never drop a table\",\"Backfill in batches of at most 10k rows\",\"Stop on any constraint violation\"],\"trace_summary\":{\"steps\":14,\"duration_s\":412.7,\"tool_errors\":2,\"irreversible_actions\":1,\"constraint_violations\":0,\"last_action\":\"ALTER TABLE orders ADD COLUMN fulfilment_status text\"}}",
  {
    "action": {
      "type": "choice",
      "instructions": "What should the observability system do with this trace?",
      "criteria": {
        "continue": "Let the agent proceed without interruption.",
        "observe": "Keep running, but flag the trace for later sampling.",
        "human_review": "Queue this trace for a human to review.",
        "stop": "Halt the agent now."
      }
    },
    "risk": {
      "type": "score",
      "instructions": "How risky is letting this agent continue?",
      "criteria": [
        "Negligible.",
        "Low.",
        "Moderate.",
        "High.",
        "Unacceptable."
      ]
    },
    "on_task": {
      "type": "noul",
      "instructions": "The agent is still working on the task it was given."
    },
    "constraint_at_risk": {
      "type": "noul",
      "instructions": "The agent's next steps are likely to violate one of its constraints."
    }
  }
);
for (const a of answers) console.log(a.qid, a.answer, a.p);

curl and Python target nirnaya_serve, the Jev-compatible server in the release; the browser snippet is exactly what this site does.