Slack channels accumulate three kinds of messages: the ones that need a fast reply, the ones that need an answer eventually, and the ones that need nothing. A triage agent's job is to make the first two visible and silence the third. Done well, it gives a small team back the cognitive cost of context-switching to a busy support channel. Done badly, it auto-replies with confident wrong answers in front of customers.

This walkthrough covers the setup most teams end up with after a fortnight of iteration. The agent runs in shadow mode first, classifies into a fixed taxonomy, escalates above-threshold matches, and never replies in a customer-facing channel until the classifier has been calibrated.

What this agent does

The agent listens on a defined channel set, applies a classifier to each new message, and routes the result. Routing for the first two weeks is purely informational: a private review channel where you can see how the agent labelled each message and disagree.

It does not auto-respond in user-facing channels. It does not @mention humans except for above-threshold urgent matches. It does not delete or archive messages.

For the broader pattern, see AI agent for inbox triage. For the cluster context, see what an AI agent can actually do.

Slack scopes and permissions

Permissions follow the principle of least privilege.

Read scopes the agent needs day one:

Write scopes the agent should not get on day one:

Use a bot user, not a user token. Rotate quarterly. For more on credential hygiene, see how to give agent access to email safely.

Taxonomy and classifier

A fixed five-label taxonomy is the right starting point.

The classifier prompt should include real examples from your channel history (10 to 20 per label is enough). The output is a label plus a confidence score. Free-text categorisation breaks the routing logic; stick to the five.

Shadow mode for two weeks

The agent classifies in real time but acts only via a private review channel. Each message gets a card with the original message, the agent's label, the confidence score, and the routing decision. You react with ✅ or ❌.

By the end of two weeks you should have at least 100 calibration data points. If disagreement rate is under 10%, you can graduate one path (typically: post a "got this, will follow up" reaction on Question and Request) to live. If above 10%, the prompt and examples need work.

For more on graduating actions, see how to limit agent actions.

Escalation routing

Routing is per label, with a confidence threshold per route.

Tune the urgent threshold over the first week. Aim for 1 to 5 pages per day. More than that, you raise the threshold; fewer than that, you lower it.

Guardrails

For broader operational guardrails, see AI agent safety and guardrails and how to monitor agent activity.

Common mistakes

Frequently asked questions

What does a Slack triage agent do?

It watches a defined set of channels, classifies each new message into a fixed taxonomy (question, request, FYI, urgent, off-topic), and routes the urgent ones somewhere a human will see them. It can also draft replies for review, post a daily digest, and resolve duplicates. It does not auto-reply in user-facing channels until the classifier has been calibrated against a real backlog.

Should the agent reply directly in the channel?

Not at first. The agent should run in shadow mode for the first two weeks: classify, route, and post drafts to a private review channel. Once you have spot-checked at least 50 messages and disagreement rate is under 10%, you can graduate one or two narrow paths (acknowledgements, FAQ answers from a known docs set) to direct reply. Open-ended replies stay shadow longer.

How does the agent know what to escalate?

Through a small classifier prompt with explicit examples of urgent, important, and routine messages from your team's actual history. The classifier outputs a label and a confidence score; only above-threshold urgent messages page or DM. Below threshold goes to a daily digest. Tune the threshold over the first week so a human sees roughly the right number of escalations per day.

What channels should the agent watch?

Start narrow. One inbound channel (support, bug-reports, internal-questions) is enough to learn the patterns. Add channels only after the first one is calibrated. Never give the agent access to channels with sensitive content (HR, legal, executive) unless the team has explicitly opted in and the agent is configured with redaction rules.

What permissions should the Slack app have?

Read scopes only at first: channels:history, groups:history, im:history (for the channels the agent watches), and users:read. Write scopes (chat:write, reactions:write) are only added when you graduate a path to direct reply. Use a granular bot user, not a user token, and rotate the token through the Slack admin every quarter.

Three takeaways before you close this tab

Sources