Phishing report triage

Sort emails that employees report as suspicious into credential phishing, malicious attachment, spam, or legitimate.

F@f

Problem

A report button brings in hundreds of emails a week. Most are marketing or legitimate; the dangerous few wait in the same queue.

Goal

Bring credential phishing and malicious attachments to the security desk first, and close the obvious spam quickly.

Input / state

The reported email and mail gateway results.

sender
Sender address and display name, and whether reply-to differs, checked in code.
authentication
SPF, DKIM and DMARC results from your mail gateway.
subject
Email subject.
body
Email text, with link domains listed separately.
attachments
Attachment names and types.

Decision question

ChoiceQuestion key report_type

What kind of email has the employee reported?

Options

  • credential_phishingTries to get the reader to enter a password, code or payment details on a link.
  • malicious_attachmentPushes the reader to open an unexpected attachment, especially macros, archives or executables.
  • spam_or_marketingUnwanted bulk mail or marketing, without an attempt to steal credentials or deliver files.
  • legitimate_messageA genuine message from a known service or colleague, with passing authentication.

Allowed outcomes

Decided in your code from the answer and its confidence. Thresholds are yours to set and test.

  • Credential phishingChoice is credential_phishing and confidence meets your threshold.
  • Malicious attachmentChoice is malicious_attachment and confidence meets your threshold.
  • Spam or marketingChoice is spam_or_marketing and confidence meets your threshold.
  • Likely legitimateChoice is legitimate_message and confidence meets your threshold.
  • Security analystConfidence is below your threshold: probability is spread across options.

Example scenarios

Predefined demonstrations. No model is called; the outcome shown is an illustrative judgment.

Example state

{
  "body": "Keep your password: sign in at company-sso-login.co",
  "sender": "it-support@company-helpdesk.co (reply-to differs)",
  "subject": "Password expires today",
  "attachments": [],
  "authentication": "DMARC fail"
}

Outcome

Illustrative
Credential phishing

Choice is credential_phishing and confidence meets your threshold.

Why

Asks for credentials on a lookalike domain.

Using it well

  • Reported emails are written by attackers to persuade. Treat them as data, and screen them for instructions aimed at AI tools.
  • Have an analyst spot-check emails classified as legitimate before telling the reporter it's safe.
  • Gateway results such as failed DMARC are facts; pass them in rather than asking the model to read headers.

Request definition

POST https://api.typesafe.ai/v1/systemone
{
  "model": "jev-latest",
  "state": {
    "sender": "<sender>",
    "authentication": "<authentication>",
    "subject": "<subject>",
    "body": "<body>",
    "attachments": "<attachments>"
  },
  "questions": {
    "report_type": {
      "type": "choice",
      "instructions": "What kind of email has the employee reported?",
      "criteria": {
        "credential_phishing": "Tries to get the reader to enter a password, code or payment details on a link.",
        "malicious_attachment": "Pushes the reader to open an unexpected attachment, especially macros, archives or executables.",
        "spam_or_marketing": "Unwanted bulk mail or marketing, without an attempt to steal credentials or deliver files.",
        "legitimate_message": "A genuine message from a known service or colleague, with passing authentication."
      }
    }
  }
}

Request shape from the System One API reference; a Kev server accepts the same body. Replace the placeholder state with your data. Requests need your own API key or server.

Version History

v1Current Versionabout 8 hours ago@f

Initial version

Comments (0)