bw-copilot-resources/docs/audit-workflow.md

Audit Workflow

Purpose

The audit workflow reviews the last 30 days of persisted local Copilot artifacts and produces a shortlist of reusable patterns worth promoting into the shared Copilot resource repository.

Current Scope

• macOS first
• read-only audit execution
• file-based review interface
• local per-machine audit history stored inside the repo checkout but excluded from git

Command

resources/scripts/audit-copilot-usage.sh --days 30

The runner excludes the current copilot-resources repo root from workspace indexing by default so you do not review patterns that already live in the shared repository.

Optional filters:

resources/scripts/audit-copilot-usage.sh --days 14 --workspace copilot-resources

resources/scripts/audit-copilot-usage.sh --days 30 --include-sources publish-log,repo-memories

resources/scripts/audit-copilot-usage.sh --days 30 --exclude-workspace /path/to/another/repo

Storage Model

Audit runs are written under:

.local/audits/<machine-id>/<timestamp>/

The runner creates the machine directory automatically on first use. The entire .local/ tree is git-ignored so each machine can keep its own audit history without creating commit noise.

Inputs

The first implementation reads from these local sources when they exist:

• VS Code workspace transcripts under ~/Library/Application Support/Code/User/workspaceStorage/*/GitHub.copilot-chat/transcripts/
• Repo memories under .../GitHub.copilot-chat/memory-tool/memories/repo/
• Shared resource publish history under ~/.copilot-resources-state/publish-log.tsv

Outputs

• audit-summary.md — high-level inventory and top candidates
• workspace-index.tsv — workspaceStorage hash to workspace path mapping for the run
• candidates-report.tsv — scored candidate list with source references plus transcript prompt-cost proxy columns such as token_cost_signal, avg_prompt_chars, and repeated_prompt_chars
• selection-manifest.tsv — editable approval surface with decision, review_note, detail_file, and the same transcript prompt-cost proxy columns used during triage
• pattern-details/*.md — per-candidate evidence bundles used during review, including purpose, expected benefit, audit context, score rationale, and caveats

Transcript prompt-cost fields are character-count proxies derived from persisted user message text. They help surface repeated long prompts that are likely to be more expensive, but they are not exact billing-token measurements.

Selection Interface

Use selection-manifest.tsv as the review surface and pattern-details/*.md as the one-by-one evidence bundle for each candidate.

Inside Copilot, use the shared review-audit-candidates prompt to work through pending rows one at a time. That prompt should ask for a decision with buttons, capture optional freeform review notes, explain why the candidate exists and why it scored as it did, and update selection-manifest.tsv after each answer.

Recommended decision states:

• promote-skill
• promote-instruction
• promote-agent
• promote-hook
• promote-script
• promote-prompt
• template-only
• docs-only
• discard
• needs-sanitization

Promotion Rule

If a candidate cannot be explained as a portable skill, instruction, prompt adapter, agent behavior, hook, script, or template, it should stay in audit notes and not be promoted into the shared repo.

Follow-on Draft Generation

After the manifest has approved rows, generate next-step artifacts with:

resources/scripts/prepare-audit-promotions.sh

Or target a specific run:

resources/scripts/prepare-audit-promotions.sh --audit-dir .local/audits/<machine-id>/<timestamp>

The command reads approved rows from selection-manifest.tsv and writes:

• draft-resources/ for straightforward prompt, instruction, agent, and skill drafts
• staging-notes/ for approved rows that still need manual design work
• promotion-summary.md to summarize what was generated

After promotion preparation completes, aggressive cleanup runs automatically. The cleanup step quarantines historical Copilot chat/session artifacts and older audit runs up to the completion boundary, then writes cleanup-summary.md in the same audit directory.

Manual cleanup command:

resources/scripts/cleanup-copilot-history.sh --audit-dir .local/audits/<machine-id>/<timestamp>

Execute mode (applies changes instead of dry-run):

resources/scripts/cleanup-copilot-history.sh --audit-dir .local/audits/<machine-id>/<timestamp> --execute

Cleanup quarantines data under:

~/.copilot-resources-state/quarantine/copilot-history-cleanup/

Restore the latest quarantine run in dry-run mode:

resources/scripts/restore-copilot-history.sh

Restore a specific quarantine run:

resources/scripts/restore-copilot-history.sh --quarantine-run ~/.copilot-resources-state/quarantine/copilot-history-cleanup/<timestamp> --execute

Restore only matching paths:

resources/scripts/restore-copilot-history.sh --filter workspaceStorage/483a18cf047eff8026d789521f5879e3 --execute

Restore one exact original path:

resources/scripts/restore-copilot-history.sh --path "/Users/<you>/Library/Application Support/Code/User/workspaceStorage/<id>/chatSessions" --execute

The restore command skips destinations that already exist unless --overwrite is passed.

Hard-purge old quarantine runs in dry-run mode:

resources/scripts/purge-copilot-history-quarantine.sh --before 2026-05-24T00:00:00Z

Execute purge while keeping the newest quarantine run:

resources/scripts/purge-copilot-history-quarantine.sh --before 2026-05-24T00:00:00Z --keep-latest 1 --execute

Protected paths are never cleanup targets, including:

• ~/.copilot-resources
• ~/.copilot/* symlink targets
• ~/.copilot-resources-state/publish-log.tsv
• ~/.copilot-resources-state/project-ports-registry.json
• repository source files under resources/, docs/, and templates/

Limits

• This audits persisted local artifacts only.
• It does not capture every inline completion or transient UI suggestion.
• Windows parity is deferred until the macOS workflow and report format are proven.