Your API validates every request, authenticates every caller, and rate limits by client. Behind it, a worker reads messages off a queue and acts on them with none of those checks, because the messages are internal and internal is trusted.
That worker is an API endpoint. It takes input from outside its process, parses it, and performs privileged actions based on the content. The only difference is that nobody reviewed it as one.
This post is what to check about your queue consumers. For whoever owns an event-driven backend.
The trust assumption, stated plainly
The reasoning is that only your own services publish to the queue, so the payload is trustworthy. Two things break it.
"Only our services" is a network claim, not a verified one. It holds if publishing requires a credential scoped to that topic, and that credential is not shared across everything. In most systems there is one broker credential in every service's environment, so any compromised service can publish anything to any topic.
The data in the message often came from outside. A worker processing a user.profile.updated event receives fields a user typed. Sanitising at the API boundary and then trusting the message downstream is the same mistake as trusting data read back from a database: the queue is a delay, not a sanitiser.
What to check on your consumers
Validate the schema, and reject what does not match. A consumer that parses loosely and proceeds on partial data is where unexpected behaviour lives. Fail the message explicitly rather than continuing with nulls.
Treat every field as untrusted. Escape it in queries, in shell commands, in file paths, in templates. The same discipline you apply to an HTTP handler.
Authorise the action, not just the message. A message saying "delete account 4812" needs a check that whoever originated it was allowed to. Put the originating identity in the message, signed or at least carried from the authenticated request, and check it at the consumer. Otherwise the ability to publish is the ability to perform any action any consumer offers.
Make handlers idempotent. At-least-once delivery is the norm, so duplicates happen without any attacker involved. A handler that charges a card or sends an email must be safe to run twice. Key on an identifier the producer supplies, record it, and skip if seen.
Bound the work. A message field that becomes a loop count, an allocation size, or a fetch target is a denial-of-service primitive and sometimes a request forgery one. Cap it.
The dead letter queue is a data store nobody governs
Messages that fail repeatedly land in a dead letter queue. Then:
They sit indefinitely, because nobody set a retention policy on a queue that was supposed to be empty. They contain full payloads, which for a user-related event means personal data, sitting outside your normal data inventory. Nobody is alerted when it grows, so a consumer broken for a week is discovered by accident. And when someone finally processes the backlog, a thousand old messages are replayed at once into a system whose state has moved on.
Concretely: set a retention period, alert on depth greater than zero, include the DLQ in your data map and your deletion processes, and have a documented procedure for draining it that includes deciding whether replay is still correct.
That last point matters. Replaying a three-week-old order.placed event may re-trigger a charge, an email and a shipment for an order that was cancelled.
Replay and ordering
Replay. Anything that can read a topic and write to it can replay a captured message. If your consumers are idempotent this is contained; if not, it is a way to repeat privileged actions. Message identifiers plus a timestamp with a freshness window handle most of it.
Ordering. Most brokers guarantee ordering only within a partition, and many teams assume global ordering because it usually appears to hold under light load. Under retry and rebalance it does not. A consumer that assumes created arrives before updated will eventually process them in the wrong order, and the failure will look like data corruption rather than a design assumption.
What to actually go and check
# How many distinct credentials can publish? One shared credential is the finding.
# (broker-specific; for Kafka, inspect ACLs)
kafka-acls --bootstrap-server "$BROKER" --list | head -40
# Any DLQ with depth > 0 and no alert on it?
# Any queue with no retention policy set?
Then read one consumer end to end and ask four questions: does it validate the schema, does it check authorisation, is it idempotent, and what happens if a field is a thousand times larger than expected.
In most codebases the answers are no, no, sometimes, and nobody has tried.
The concession
Full message signing, per-topic credentials and an authorisation check in every consumer is a lot of machinery, and for an internal queue moving low-stakes events between two services it is not worth it. Plenty of systems run happily with a shared credential and trusting consumers.
The line worth drawing is by consequence. A consumer that writes to a cache or updates a counter can trust its input. A consumer that charges money, sends mail to customers, changes permissions or deletes data is a privileged endpoint and deserves the same review as an HTTP route doing the same thing. Most systems have a small number of the second kind, and they are usually the ones nobody has looked at.
The implication
Queues feel like plumbing, so they get reviewed like plumbing. But a consumer takes untrusted input and performs privileged actions, which is the definition of the thing you review most carefully everywhere else.
List your consumers, mark the ones that do something irreversible, and read those as though someone had just added a new unauthenticated endpoint. That is what they are.
