prompts.chatprompts.chat
Login
All classifiers
ChoiceEducation

Course material answerability

Check whether the retrieved course material covers a student's question, partly covers it, doesn't, or whether the question needs clarifying.

F@f

Problem

Course assistants answer from general knowledge when the course material doesn't cover a question, which can contradict how the course teaches it.

Goal

Answer from the course when it covers the question, say what's missing when it partly does, and pass the rest to a teaching assistant.

Input / state

The question and the passages retrieved from the course.

question
The student's question.
passages
Retrieved passages from lecture notes, slides and the syllabus.

Decision question

ChoiceQuestion key answerability

Can this question be answered from these course passages?

Options

  • coveredThe passages contain what's needed to answer the question.
  • partially_coveredThe passages answer part of the question.
  • not_coveredThe passages don't address the question.
  • needs_clarificationThe question is too ambiguous to tell what's being asked.

Allowed outcomes

Decided in your code from the answer and its confidence. Thresholds are yours to set and test.

  • Answer from courseChoice is covered and confidence meets your threshold.
  • Answer the covered partChoice is partially_covered and confidence meets your threshold.
  • Teaching assistantChoice is not_covered and confidence meets your threshold.
  • Ask the studentChoice is needs_clarification and confidence meets your threshold.
  • Teaching assistantConfidence is below your threshold: probability is spread across options.

Example scenarios

Predefined demonstrations. No model is called; the outcome shown is an illustrative judgment.

Example state

{
  "passages": [
    "Lecture 11: if p < α we reject H0, because data this extreme would be unlikely if H0 were true."
  ],
  "question": "Why does a small p-value mean we reject the null?"
}

Outcome

Illustrative
Answer from course

Choice is covered and confidence meets your threshold.

Why

The lecture explains it.

Using it well

  • Retrieval quality matters more than this question. Score passages first and pass only relevant ones.
  • When partially covered, say which part the course material doesn't cover.

Request definition

POST https://api.typesafe.ai/v1/systemone
{
  "model": "jev-latest",
  "state": {
    "question": "<question>",
    "passages": "<passages>"
  },
  "questions": {
    "answerability": {
      "type": "choice",
      "instructions": "Can this question be answered from these course passages?",
      "criteria": {
        "covered": "The passages contain what's needed to answer the question.",
        "partially_covered": "The passages answer part of the question.",
        "not_covered": "The passages don't address the question.",
        "needs_clarification": "The question is too ambiguous to tell what's being asked."
      }
    }
  }
}

Request shape from the System One API reference; a Kev server accepts the same body. Replace the placeholder state with your data. Requests need your own API key or server.

Version History

v1Current Versionabout 8 hours ago@f

Initial version

Comments (0)