The red team bot: one job, find what's wrong

Every team has someone who says, "Has anyone thought about…?" Good teams listen to that person. Most teams don't have one in every meeting.

What it is

A red team is a group assigned to attack a plan, a design or a defense, so the weak points show up before a real opponent finds them. The idea comes from military war-gaming and is now common in security testing.

A milder version is the devil's advocate: one person whose job, for this meeting, is to argue the other side. The name comes from an old Catholic Church role whose job was to argue against canonizing someone.

The common thread: someone's whole job is to find what's wrong, so the rest of the team doesn't have to fight its own optimism.

Why people skipped it

Nobody likes being the critic, and nobody likes being criticized. Assigning a person to it takes time and tact. And the critic is often the same busy expert who's needed elsewhere.

How a bot does it

A bot doesn't mind being the critic and doesn't take it personally when it's ignored.

  1. Give it one job. "Find what's wrong with this." Nothing else. A bot asked to "review" will often say nice things first.
  2. Give it the plan and what it's for. The goal matters. A flaw is only a flaw relative to the goal.
  3. Ask for the strongest objections, not all of them. Ranked.
  4. Ask it to argue from specific viewpoints. The customer, the auditor, the operator, the competitor, the accountant.
  5. A person decides which objections need an answer.

The bot brief

You are a red team. Your only job is to find weaknesses in the attached [plan/design/decision].
Goal of the plan: [goal]. Inputs: [files].
Do not summarize or praise. Do not soften.
Give the 10 strongest objections, ranked by how badly each would hurt the goal.
For each: the objection, the evidence in the inputs (cite it), what would have to be
true for the plan to survive it, and one question the team should answer.
Then argue briefly as each of: the customer, an auditor, the operator who runs it,
a competitor, the person paying for it.
Mark any objection that relies on facts not in the inputs as GUESS.

Starting point only. Test it on your own data and model, and adjust as needed before you rely on it.

An example

Plan: move all customer quotes to a bot by March.

#ObjectionSurvives if…Question for the team
1Pricing rules live in one estimator's headRules are written down firstWho writes them, and by when?
2No test set of past quotes to check against50 past quotes are collected with known-good pricesWho pulls them?
3Customers may object to bot-written quotesA person still reviews and signs each oneIs that in the plan?
4Plan assumes ERP export worksExport tested in week 1Has anyone tried it?

A bot red-teaming another bot

This also works on bots' own output. Before a report goes to the owner, a red team bot reads it with one job: find the weakest claim. That's a different check from the tester bot, which checks against a script. See Your tester bot is your internal auditor.

Where bots differ

A bot will always find objections, because that's what you asked for. Some will be weak or padded. Ranking helps. So does asking "what would have to be true for the plan to survive this?", which turns criticism into something you can check.

A red team bot running on the same model as the bot that wrote the plan may share its blind spots. For important decisions, use a different model, or add a person's red-team pass.

What this means for you

  • Put a red team bot on every decision that's hard to undo.
  • Give it one job: find what's wrong.
  • Ask for ranked objections and specific viewpoints.
  • Answer the top few. Note why you're ignoring the rest.

Related