Somebody typed /deploy production in a chat channel and it worked. That is a production deployment triggered over an interface where the authorization check is "was this person in the channel".
ChatOps bots accumulate capability quickly because each addition is small and obviously useful. Deploy. Restart. Show me the error rate. Look up a customer. Rotate that key. Within a year the bot can do more than most individual engineers, and nothing about it went through the review that a new admin API would have.
This post is the specific weaknesses of the pattern and how to keep the convenience without the exposure. For whoever maintains the bot, which is usually whoever wrote the first command.
What makes chat different from an API
Channel membership is not authorization. Most bots authorize by checking that the request came from an allowed channel, sometimes that the user is in a group. Channel membership is managed by whoever administers chat, changes constantly, and is not connected to your production access model. Someone with no production access can be added to a channel in seconds, by someone who does not realise what that channel can do.
Identity is assertable in more ways than you think. Webhooks, integrations, other bots, and workflow automations can all post messages. If your bot parses any message matching a pattern, anything that can write to the channel can invoke it.
There is no session, so there is no step-up. A web console can demand re-authentication for a destructive action. A chat message cannot, unless you build it.
The audit trail lives in someone else's product. It exists, and it is subject to your chat retention policy, which is usually shorter than your audit requirement and not under your control.
The blast radius includes the chat platform. A compromised chat account, or a malicious app installed by a user, inherits every capability the bot exposes. Your production access control now includes the security of a SaaS product administered by someone in operations.
The failure that gets exploited
Not usually a sophisticated attack. It is a normal person doing a normal thing.
Someone is added to an engineering channel to help with an incident. They stay. Months later they run a command they saw someone else run, in a context they do not understand, against the wrong environment. There is no confirmation step because the command was designed for experts, and no guardrail because guardrails were not the point of the original commit.
Or: a bot with a broad token is connected to a new integration, and now a third party can invoke commands it was never intended to reach.
Controls that fit the pattern
Authorize against your real access model. Map the chat user to an identity in your directory, and check that identity's permissions, not their channel membership. If the person could not perform the action in your console, the bot must refuse.
# not this
if channel in ALLOWED_CHANNELS: run(command)
# this
identity = directory.resolve(chat_user_id) # may fail: refuse if so
if not authz.can(identity, "deploy", environment): refuse()
The mapping step matters. An unmapped chat user is a refusal, not a fallback.
Split read from write. Most chat commands are queries, and those are fine with light controls. The small number that change something deserve the full treatment. Put them behind a separate command namespace so the distinction is visible in the code and in the help text.
Require confirmation with context for destructive actions. Not a yes/no button, which people click. Make them restate the target:
> /deploy production
Bot: This will deploy 4a91c2 to production, affecting all customers.
Reply with: confirm deploy production 4a91c2
Typing the target is the check. It also produces an unambiguous record of intent.
Log to your own systems, not only to chat. Every invocation: who, what, when, which environment, what the result was. Ship it to the same place as the rest of your audit logging, because chat retention is not an audit trail you control.
Scope the bot's own credentials. The bot usually holds a token that can do everything any command can do. That token is the real asset. Scope it to the minimum, rotate it, and make sure it cannot be used to grant itself more.
Refuse messages that are not from a human. Check the message metadata, ignore bot-authored and webhook-authored messages unless you have deliberately allowlisted one. This closes the confused-deputy path.
Prefer approval over execution
There is a version of this pattern that keeps most of the value and drops most of the risk: the bot proposes and a system executes.
Instead of the bot deploying, it opens the deployment and posts a link. The action happens in a system with proper authentication, authorization and audit. Chat becomes the notification and coordination surface, which is what it is genuinely good at, rather than the control plane.
This is slightly slower and it removes the entire class of problem, because the authorization decision moves back to a place that was designed to make it.
Audit what yours can do
Worth an afternoon:
- List every command, including the ones nobody has used in a year.
- For each, what does it do, and what would happen if it ran against the wrong environment.
- Who can currently invoke it, computed from the actual channel membership rather than the intent.
- What token does the bot hold, and what can that token do beyond the commands.
- Where does the invocation log go, and how long is it kept.
The usual findings: several commands nobody remembers adding, a membership list considerably wider than expected, and a bot token with administrative scope because narrowing it was fiddly during setup.
The concession
The reason ChatOps exists is that it is genuinely excellent during an incident. Everyone is already in the channel, the context is shared, and the actions and the discussion live in one place. Moving everything behind a console loses real value at exactly the moment it matters most.
So the line worth drawing is not chat versus console. It is read versus write. Keep queries in chat, keep them fast, and make them the majority of what the bot does. Move the small number of destructive actions to a propose-and-approve flow. That preserves the incident-response benefit, which is mostly about seeing state quickly, and removes the part where a chat message is a production change.
The implication
Your bot is an API. It has endpoints, it takes input, it performs privileged actions, and it was never reviewed as one because it arrived a command at a time.
Read its command list as though it were a routing table someone had just handed you. That is what it is, and the question is the same: who can reach each of these, and what happens if the wrong person does.
