← Back to the Brail website

How do you test brand skills and playbooks?

Part of: ControlLast updated 6 November 2026

Brand evals are tests that rerun known requests against good and bad examples whenever the context, a playbook or skill, or the model behind them changes, to check that output didn't get worse.

Also called: playbook evals, skill testing.

Why it matters.

The things behind AI output keep changing. A new messaging line, a small edit to the LinkedIn playbook, or a new model version in ChatGPT or Claude can quietly change what comes out. Without a test, the team finds out from a weak post that's already live.

Evals borrow a habit from software teams: keep a set of examples you know are good and a few you know are bad, rerun them after every change, and compare. It turns 'it seems worse' into something you can see, and it makes teams confident enough to keep improving their context, examples and playbooks.

Example.

BeforeThe team tweaks the sales deck playbook to shorten the opening. Next week reps notice decks have lost the proof slide, and nobody knows which change caused it.
AfterAfter the tweak, the playbook is rerun against ten reference requests. Two decks come back without a proof slide, the change is fixed before anyone uses it, and the reference set gets one more example.

How Brail handles it.

Brail keeps context, examples and playbooks as versioned files, so after any change, or a new model, the team can rerun known requests against approved examples and roll back the version that made output worse. Agents also report where they lacked context, an example or a skill, so gaps show up before a weak piece does.

Questions people ask.

How do I know a new AI model hasn't made our content worse?
Rerun a fixed set of requests through the new model with the same context and playbook, and compare the results with known good examples. If they fall short, hold the change until the instructions are adjusted.
How many examples do brand evals need?
A handful per type of output is enough to start: a few strong examples, a couple of known failures and the edge cases that have caused problems before. Add to the set whenever something slips through.
Do marketers need evals or is this an engineering thing?
The idea comes from engineering, but the examples and judgement come from marketing. Whoever owns a piece of context or a playbook should own the examples that test it.
  • Control AI content at scale: You control AI output across a team by covering every situation, tool and person: give each job the context and examples that define good, check every piece against your rules and thresholds before it ships, approve what clears them automatically, flag or redo what doesn't, and keep a record of who or which agent made what.
  • Marketing playbook: A marketing playbook is a set of instructions for one situation, such as a sales deck, a LinkedIn post or a customer report.
  • Brand skills: Brand skills are instructions packaged for AI agents, usually as SKILL.md files, that teach an agent how to apply a brand.
  • Quality floor: A quality floor is the minimum standard every piece of output has to meet before it ships, whoever or whatever made it.
  • Brand learning loop: A brand learning loop feeds approved output back into the brand system as examples, inspiration and rules, so each new piece starts closer to ready.
  • Brand versioning: Brand versioning is giving every brand file, from messaging and pricing to logos and skills, a history, an owner, a draft or approved status and sign-off for changes, so agents only use approved versions.