Decision Policies

Decision Policies (DECISIONS.md)

A decision policy is a short document a workspace admin writes for one agent to say how it should treat an inbound message: answer it (continue), stay out of it (ignore), or ask for a missing detail first (request_clarification). The gateway asks Jev, the reflex decision plane, that typed question about each admitted person message.

The document

The document is Markdown with exactly one fenced yaml block. The prose around it is for people and is never sent anywhere.

# Support bot decisions

```yaml
version: 1
questions:
  - id: handle_message
    instructions: Should the assistant answer this message now?
    options:
      continue: the message asks the assistant for something it can do
      request_clarification: an order question that does not include the order number
      ignore: group chatter between people that is not addressed to the assistant
    confidence_min: 0.8
overlays:
  - id: billing_channel
    match:
      channel: telegram-billing
    confidence_min: 0.9
```

Overlays narrow the question for one channel, group (a group chat id) or sender. Each overlay replaces the instructions, options or confidence_min it sets. When several match, they apply from the least to the most specific (channel, then group, then sender), so the most specific one wins each field. Every evaluation records which scope and overlays shaped it.

Modes

| Mode | What happens | | --- | --- | | off | Default. No evaluation and no Jev call. | | shadow | Each admitted person message is evaluated off the message path and recorded as a decision.evaluate trace span. Nothing changes. | | active | The decision is acted on just before the agent runs, in the conversation’s own queue. continue runs as usual. request_clarification instructs the agent to reply with one clarifying question instead of acting (an instruction, not a tool restriction: tools stay under the agent’s tool policy). ignore applies to group messages only: the agent does not run and the message is not added to the conversation. Every failure (timeout, low confidence, no credential, and so on) continues. |

The policy sees only the message that triggered the run, without the earlier group chatter some channels attach to it. A direct message is never ignored. Only a person can write a policy or switch its mode. Agent credentials, customer sessions and the system identity are refused.

Measure before you switch it on

Switching to active requires a labelled replay of the exact document. You run the policy over 20 to 100 messages that a person has labelled with the expected action. The gateway reports precision and recall for each option, the confusion matrix and correctness per scope and overlay. It stores the measurement but never the messages.

The active gate opens for 30 days when the replay meets all of these:

• at least 20 cases;
• Jev answered at least 90 % of them;
• every non-continue option the policy can decide, including in any overlay, has at least 3 labelled cases;
• precision is at least 0.95 for ignore and at least 0.90 for request_clarification;
• every overlay is reached by at least 3 cases, and what it decides meets the same precision there.

Changing the document while it is active needs a new passing replay.

# Labelled cases: JSON Lines, one per line (lines starting with # are skipped)
# {"id":"g07","message":"haha same","peer_kind":"group","expected":"ignore"}

dewee decisions replay support-bot --fixtures cases.jsonl --file DECISIONS.md --require-pass
dewee decisions set support-bot --file DECISIONS.md --mode active
dewee decisions replays support-bot
dewee decisions evaluations support-bot --json   # enforced counts, breakdown by scope and overlay

Surfaces

| Surface | Read | Write and mode | Test | Replay | Evidence | | --- | --- | --- | --- | --- | --- | | Gateway API (/v1/agents/{id}/decision-policy/...) | yes | person only | yes | yes | yes | | CLI dewee decisions | yes | set (human only) | test | replay, replays | evaluations | | Runtime dashboard, agent Decisions tab | yes | yes | yes | yes | yes | | Customer control plane, agent Decisions tab | yes | no | yes | yes | yes |

Decision policies send the policy question and the message (truncated and redacted) to Jev using the instance reflex credential. Tests and replays send their sample messages the same way. The commands are in the CLI reference and the endpoints in the API reference.