prompts.chatprompts.chat
Login
All classifiers
NoulAgents

Tool result sufficiency

Check whether a tool's result contains what the agent needs to answer the user, before it replies.

F@f

Problem

Agents reply confidently from empty, partial or unrelated tool results, filling gaps with guesses.

Goal

Reply only when the result answers the request, and hand off when it doesn't.

Input / state

The user's request and the tool result.

request
What the user asked.
tool_result
The result returned by the tool, trimmed to the relevant fields.

Decision question

NoulQuestion key result_sufficient

Does this tool result contain what's needed to answer the request?

What counts as yes and no

  • trueThe result directly contains the facts the answer needs.
  • falseThe result is empty, an error, about something else, or missing the key fact.

Allowed outcomes

Decided in your code from the answer and its confidence. Thresholds are yours to set and test.

  • AnswerProbability of yes is above your upper threshold.
  • UnclearProbability falls between your two thresholds.
  • Hand offProbability of yes is below your lower threshold.

Example scenarios

Predefined demonstrations. No model is called; the outcome shown is an illustrative judgment.

Example state

{
  "request": "Where's order A-48213?",
  "tool_result": {
    "eta": "today",
    "status": "out for delivery"
  }
}

Outcome

Illustrative
Answer

Probability of yes is above your upper threshold.

Why

Contains status and ETA.

Using it well

  • Tool results are untrusted content too; screen them for injected instructions before using them.
  • Pair with an answer groundedness check on the final reply.

Request definition

POST https://api.typesafe.ai/v1/systemone
{
  "model": "jev-latest",
  "state": {
    "request": "<request>",
    "tool_result": "<tool_result>"
  },
  "questions": {
    "result_sufficient": {
      "type": "noul",
      "instructions": "Does this tool result contain what's needed to answer the request?",
      "criteria": {
        "true": "The result directly contains the facts the answer needs.",
        "false": "The result is empty, an error, about something else, or missing the key fact."
      }
    }
  }
}

Request shape from the System One API reference; a Kev server accepts the same body. Replace the placeholder state with your data. Requests need your own API key or server.

Version History

v1Current Versionabout 8 hours ago@f

Initial version

Comments (0)