Retrieved passage relevance

Score how useful each retrieved passage is for answering the question.

Problem

Search returns passages that share words with the question but don't answer it. Passing all of them to the answering model adds noise and invites confident wrong answers.

Goal

Keep the passages that help, drop the ones that don't, and know when nothing relevant was found.

Input / state

The user's question and one retrieved passage per question in the request.

question: The user's question.
passage: One retrieved passage. Score several in one request, one question per passage.

Decision question

ScoreQuestion key relevance

How useful is `passage` for answering `question`?

Rubric levels

0Unrelated: a different topic.
1Related topic, but doesn't help answer the question.
2Partly answers the question or gives needed context.
3Directly answers the question.

Allowed outcomes

Decided in your code from the answer and its confidence. Thresholds are yours to set and test.

IncludeScore of 1.5 or more with confidence above your threshold.
BorderlineScore of 1.5 or more with low confidence: include only if few passages qualify.
ExcludeScore below 1.5.

Example scenarios

Predefined demonstrations. No model is called; the outcome shown is an illustrative judgment.

Example state

{
  "question": "How long do refunds take to reach my card?",
  "passage": "Refunds are issued to the original payment method within 5 business days. Your bank may take up to 10 more business days to show it."
}

Outcome

Illustrative

Include

Score of 1.5 or more with confidence above your threshold.

Why

It states the timing the user asked about.

Better together with

Complementary classifiers, and what each one adds here.

Answer groundednessChoice
Relevance filters what goes into the answer; groundedness checks what comes out.
Combined inGrounded support assistant turn
Agent tool routingChoice
Runs when the router decides a search is needed.
Combined inGrounded support assistant turn
Prompt injection signalNoul
Retrieved passages are untrusted input; screen them as well as scoring them.
Combined inGrounded support assistant turn

Used in chains

Grounded support assistant turnStep 3 of 4

Sources

Primary documentation this classifier builds on. Scenarios on this page are illustrative, not taken from production use.

SourcedTypeSafe docs · Cookbook
Classifying RAG passages
Scores each retrieved passage in one request, then decides in code which passages reach the answering model.
SourcedTypeSafe docs · Pattern
Speculative fan-out
Sends questions that share a state in one request and lets code decide which answers matter.

Using it well

Score every passage in one request with one question per passage, keyed by passage ID, then sort and cut in code.
If no passage qualifies, say the documentation doesn't cover the question instead of answering anyway.

Request definition

POST https://api.typesafe.ai/v1/systemone

{
  "model": "jev-latest",
  "state": {
    "question": "<question>",
    "passage": "<passage>"
  },
  "questions": {
    "relevance": {
      "type": "score",
      "instructions": "How useful is `passage` for answering `question`?",
      "criteria": [
        "Unrelated: a different topic.",
        "Related topic, but doesn't help answer the question.",
        "Partly answers the question or gives needed context.",
        "Directly answers the question."
      ]
    }
  }
}

Request shape from the System One API reference; a Kev server accepts the same body. Replace the placeholder state with your data. Requests need your own API key or server.