Nirnaya v1
← cookbooks
agentsbring your own workflow

Escalation check for an LLM agent

The cheapest use: a yes/no gate in front of an expensive model call.

Run in playground

You do not have to replace the LLM. Put a 200 ms decision in front of it: does this message need the big model at all, and if so, which tool budget? Most traffic does not, and the saving is the whole point.

Fit this gate to a few hundred messages from your own traffic and it learns where your expensive model is genuinely needed.

The questions

needs_llmnoul

Answering this well requires an LLM rather than a canned answer or a lookup.

falseno, the statement does not hold
trueyes, the statement holds
needs_toolsnoul

Answering this requires calling tools or fetching live data.

falseno, the statement does not hold
trueyes, the statement holds
complexityscore

How complex is this request?

0Lookup.
1Simple answer.
2Multi-step.
3Open-ended reasoning.
handoffchoice

Where should this go?

faqAnswer from the FAQ.
small_modelA small model can handle it.
large_modelSend to the frontier model.
humanA person should answer.

Acting on the answers

  • needs_llm below 0.3: answer from the FAQ, skip the model call entirely.
  • Measure the gate on your own traffic before trusting it; the threshold is yours to set.

request

{
  "state": "User message to the assistant: 'what's the difference between the pro and team plans again? just the seat count?'",
  "questions": {
    "needs_llm": {
      "type": "noul",
      "instructions": "Answering this well requires an LLM rather than a canned answer or a lookup."
    },
    "needs_tools": {
      "type": "noul",
      "instructions": "Answering this requires calling tools or fetching live data."
    },
    "complexity": {
      "type": "score",
      "instructions": "How complex is this request?",
      "criteria": [
        "Lookup.",
        "Simple answer.",
        "Multi-step.",
        "Open-ended reasoning."
      ]
    },
    "handoff": {
      "type": "choice",
      "instructions": "Where should this go?",
      "criteria": {
        "faq": "Answer from the FAQ.",
        "small_model": "A small model can handle it.",
        "large_model": "Send to the frontier model.",
        "human": "A person should answer."
      }
    }
  }
}

call it

import { loadEngine } from "nirnaya";     // the same call this site makes

const engine = await loadEngine();          // loads once, then cached by the browser
const answers = await engine.decide(
  "User message to the assistant: 'what's the difference between the pro and team plans again? just the seat count?'",
  {
    "needs_llm": {
      "type": "noul",
      "instructions": "Answering this well requires an LLM rather than a canned answer or a lookup."
    },
    "needs_tools": {
      "type": "noul",
      "instructions": "Answering this requires calling tools or fetching live data."
    },
    "complexity": {
      "type": "score",
      "instructions": "How complex is this request?",
      "criteria": [
        "Lookup.",
        "Simple answer.",
        "Multi-step.",
        "Open-ended reasoning."
      ]
    },
    "handoff": {
      "type": "choice",
      "instructions": "Where should this go?",
      "criteria": {
        "faq": "Answer from the FAQ.",
        "small_model": "A small model can handle it.",
        "large_model": "Send to the frontier model.",
        "human": "A person should answer."
      }
    }
  }
);
for (const a of answers) console.log(a.qid, a.answer, a.p);

curl and Python target nirnaya_serve, the Jev-compatible server in the release; the browser snippet is exactly what this site does.