The Brown M&M Rule for AI Agents
Oscar Gallo
Published on July 3, 2026
AI agents need harnesses, not blind trust. The Brown M&M rule is a useful way to think about safety checks before agents act.
There is an old story about Van Halen's concert rider: the band required a bowl of M&Ms backstage with the brown ones removed.
The point was not candy.
The point was verification. Their live shows involved heavy equipment, power, staging, and effects. If the band walked into the room and saw brown M&Ms, it meant the venue had not read the instructions carefully. If they missed the candy detail, they might have missed something that could hurt people.
AI agents need the same idea.
Not candy. Verification.
Agents need harnesses
The next era of AI products is not only about better models. It is about better harnesses.
A harness is the system around the model: the permissions, tool boundaries, approval steps, logs, tests, prompts, data filters, and workflow rules that decide what the agent can do.
The model may be powerful, but the harness determines whether that power is safe to use.
Without a harness, an agent is just a language model with tools and optimism.
The Brown M&M test for AI
In an AI product, the Brown M&M is a simple check that tells you whether the system is still inside its safety boundary.
For a coding agent, it might be:
- It must show a plan before editing files
- It must never touch production credentials
- It must ask before running destructive commands
- It must never commit generated changes without a diff review
- It must flag instructions found inside logs, tickets, or webpages as untrusted
For a customer-support agent, it might be:
- It can draft refunds but not issue them
- It can summarize invoices but not send them externally
- It can recommend an account change but not apply it
- It must escalate when a user asks for sensitive data
For a sales agent, it might be:
- It can draft outreach but not send to a new domain without review
- It must ignore instructions inside a prospect's website profile
- It cannot export CRM records without approval
The exact check depends on the product. The principle is the same: define the failure signal before the failure becomes expensive.
Why this is the product owner's job
AI labs provide models. They do not know your business process, customer obligations, regulatory risk, or internal definition of "destructive."
That responsibility belongs to the product owner and engineering team.
If your agent can delete a record, your system has to decide when deletion is allowed. If your agent can send an email, your system has to decide who approves external communication. If your agent can run code, your system has to decide which commands are safe and which require a human.
You cannot outsource that judgment to the base model.
Prompt injection makes this urgent
Prompt injection is the obvious reason harnesses matter.
An attacker can hide instructions inside data the agent reads. If the agent has broad permissions and no checkpoints, the hidden instruction can turn into action.
The model may not know the instruction is hostile. The harness should.
That means treating external content as untrusted, limiting tool permissions, separating read and write actions, and requiring approval for anything that cannot be undone.
The future is not one giant agent
The safest agent systems will not be one all-powerful assistant with access to everything.
They will be narrow agents inside controlled workflows. Each agent will have a job, a set of tools, a permission boundary, and a review process. The system will know what the agent is allowed to do and what it must never do alone.
That is less magical. It is also more useful.
Bottom line
The Brown M&M rule is a reminder that safety is not a vibe. It is a check.
AI agents need clear signals that tell the system and the human operator when something has gone wrong. They need harnesses that make unsafe behavior hard, visible, and reversible.
The model is only one part of the product. The harness is what makes it safe enough to trust.