Test and promote ¶

EAP onboarding has three validation tiers, in order: local Codex, EAP staging, EAP production. Each tier proves a different thing. Do not skip tiers.

flowchart LR
  Local[Local Codex
CLI, app, or IDE] --> Stage[EAP staging
env_overrides.stage.repo.branch]
  Stage --> Prod[EAP production
repo.branch]
  Local -. proves .-> A[agent logic + skill triggering]
  Stage -. proves .-> B[runtime governance + approved MCP set]
  Prod -. proves .-> C[real users, real data, real audit]

Tier 1: Local Codex ¶

Run Codex from the project repository root to iterate on AGENTS.md and SKILL.md against your own test inputs.

What local validation proves:

• skill triggering works on your phrasing
• Codex discovers the intended AGENTS.md instruction chain and skills
• project routing instructions match expectations
• the workflow steps run in the right order
• output shape includes evidence, decision, and next action
• the agent stops before any state-changing action that should be gated

What local validation does not prove:

• that EAP has the same MCP set approved for your project
• that EAP uses the same model or provider as your local Codex account
• that authorization (MyID group / CI client_id) is wired correctly
• that the runtime guardrails will accept your skill's behavior
• anything about production data shape, latency, or load

Treat local validation purely as an author's smoke test. Approval still needs to happen on EAP.

Local Validation Checklist ¶

Keep one smoke-test record per skill containing:

• the input you used
• the output the agent returned
• warnings or failures observed
• date, Codex version, local model, environment, MCP dependencies

Link the record(s) in your PR description.

Tier 2: EAP Staging ¶

Staging is EAP itself, but pointed at the staging branch of your agent repo via env_overrides.stage.repo.branch in your Registry YAML.

The staging-branch convention is straightforward:

• your agent repo has both a staging branch (commonly dev) and a production branch (commonly main)
• the Registry YAML pins repo.branch: main and env_overrides.stage.repo.branch: dev
• changes flow through staging before they flow to production

version: 1
agent_runtime: codex

repo:
  url: https://sqbu-github.cisco.com/stap/<your-project-repo>.git
  branch: main          # production loads this branch

env_overrides:
  stage:
    repo:
      branch: dev       # staging loads this branch

Onboarding Validation Flow ¶

1. push your changes to the staging branch (dev)
2. open the paired PRs (agent repo + Registry) and request staging activation
3. notify the EAP platform team to deploy the staging branch to the staging environment — EAP does not auto-pull from your repo on every push; deployment is a platform-team action
4. once deployed, run your onboard test suite against the real staging runtime
5. record the staging smoke-test results in the PR
6. iterate on dev, asking EAP to redeploy after each change, until staging is green

What staging proves that local cannot:

• approved MCP set is correct end-to-end (auth, scopes, network reachability)
• the Registry selected the Codex runtime and EAP generated the expected model/MCP configuration
• MyID group / CI client_id allow-list works
• platform guardrails accept the skill's behavior
• approval gates fire where expected
• audit trail captures what you need

Common Staging Gotchas ¶

• new MCP server in your YAML — the platform team must approve and the credential must be in Keeper before staging will succeed
• staging branch missing in the agent repo — EAP will fail to load
• staging-only secrets vs prod-only secrets — confirm credential mapping per environment
• staging passing only because authorized_myid_groups happens to include your own group — verify the intended caller, not the author

Tier 3: EAP Production ¶

Production is reached by Git merge plus an explicit deploy request, not by editing prompt text in place and not by silent auto-sync.

Promotion path:

1. merge the staging branch (dev) into the production branch (main) on your agent repo
2. merge the Registry PR
3. notify the EAP platform team to deploy the production branch to the production environment — merging to main does not automatically update the live runtime; the platform team performs the deployment
4. observe the first production executions; tighten controls (MyID group scope, MCP access level, approval gates) if anything looks too broad

Rolling Back ¶

If a production execution exposes a problem:

• fast path — move repo.branch back to a known-good ref via a Registry PR (revert), then ask EAP to redeploy
• content path — revert the offending commit on the production branch in your agent repo, then ask EAP to redeploy
• kill switch — for unsafe runaway behavior, the EAP platform team can hit the runtime kill switch immediately; you still need a follow-up Git revert and a redeploy request

Do not patch around an issue by editing the live prompt outside the review path. That's exactly what the two-plane Git model exists to prevent.

Promotion Checklist ¶

Before merging to production:

• staging smoke tests recorded and green
• registered MCPs all behaved as expected in staging
• approval-gated actions actually paused (not silently passed)
• audit entries are present and useful for the cases you care about
• both PRs (agent repo + Registry) reviewed and approved
• rollback plan named in the Registry PR description

Common Mistakes ¶

• merging to main without ever running on staging
• skipping the smoke-test record because "it worked locally"
• treating staging as a place to test against fake data only — use realistic inputs
• promoting an MCP from ask to allow for state-changing calls without re-running staging
• editing the live agent prompt outside Git when something breaks in prod

Next ¶

• Change lifecycle — how the two control planes interact across the life of the project.
• Runtime and governance — guardrails, sandbox, approval, audit in depth.