Community policy category

Name the part of your community policy a post most likely breaks, if any.

Limits to keep in mind

Moderation decisions affect people. Keep a person in the loop for anything beyond reversible actions, and give authors a way to appeal.

Problem

Moderators see every report in one queue, and a single "is this bad?" flag mixes spam, harassment and graphic content that need different responses.

Goal

Sort posts by policy area so spam can be handled by reversible rules and person-to-person harm gets human attention.

Input / state

The post, where it was posted, and a summary of the community policy.

post: The text of the post or comment.
section: Where it was posted, since some sections allow more.
community_policy: A short summary of the rules.

Decision question

ChoiceQuestion key policy_area

Which part of `community_policy`, if any, does this post most likely break?

Options

noneThe post follows the policy, including heated but civil disagreement.
spamUnsolicited advertising, repeated links or engagement bait.
harassmentInsults, threats or demeaning language aimed at a person or group.
explicit_or_graphicSexual content or graphic violence outside the sections that allow it.
other_concernA possible policy problem that none of the other options describes.

Allowed outcomes

Decided in your code from the answer and its confidence. Thresholds are yours to set and test.

No violationChoice is none and confidence meets your threshold.
Likely spamChoice is spam and confidence meets your threshold.
Possible harassmentChoice is harassment.
Explicit or graphicChoice is explicit_or_graphic.
Other concernChoice is other_concern.
Needs a personConfidence is below your threshold.

Example scenarios

Predefined demonstrations. No model is called; the outcome shown is an illustrative judgment.

Example state

{
  "post": "This prompt is honestly worse than the original. You removed the only constraint that made it useful. Please revert it.",
  "section": "Prompt discussion",
  "community_policy": "Be respectful. No spam or self-promotion. No harassment. No explicit or graphic content."
}

Outcome

Illustrative

No violation

Choice is none and confidence meets your threshold.

Why

Blunt criticism of the work, not of the person.

Better together with

Complementary classifiers, and what each one adds here.

Harm severityScore
The category says which rule; severity says how fast and how visibly to act.
Combined inCommunity post moderation

Used in chains

Community post moderationStep 1 of 2

Sources

Primary documentation this classifier builds on. Scenarios on this page are illustrative, not taken from production use.

SourcedTypeSafe docs · Cookbook
Guardrails for LLMs
Screens messages into and out of an LLM app with Noul questions per hazard and a harm Score, then passes, reviews or blocks by thresholds set in code.
SourcedTypeSafe docs · Cookbook
Twitch chat filter
Hides insults and spam in a live chat with one request per message and plain rules in code.
SourcedTypeSafe docs · Documentation
Model jaggedness
Lists known failure modes, including literal reading, arithmetic, date comparison, option order and adversarial content.

Using it well

Write the boundary cases into the option descriptions, for example that heated but civil disagreement is allowed.
Automate only reversible actions, such as hiding likely spam until someone checks it.
Ask the same question twice in testing on borderline posts to see how stable the answer is.

Request definition

POST https://api.typesafe.ai/v1/systemone

{
  "model": "jev-latest",
  "state": {
    "post": "<post>",
    "section": "<section>",
    "community_policy": "<community_policy>"
  },
  "questions": {
    "policy_area": {
      "type": "choice",
      "instructions": "Which part of `community_policy`, if any, does this post most likely break?",
      "criteria": {
        "none": "The post follows the policy, including heated but civil disagreement.",
        "spam": "Unsolicited advertising, repeated links or engagement bait.",
        "harassment": "Insults, threats or demeaning language aimed at a person or group.",
        "explicit_or_graphic": "Sexual content or graphic violence outside the sections that allow it.",
        "other_concern": "A possible policy problem that none of the other options describes."
      }
    }
  }
}

Request shape from the System One API reference; a Kev server accepts the same body. Replace the placeholder state with your data. Requests need your own API key or server.