Audit Workflow

Purpose

The audit workflow reviews the last 30 days of persisted local Copilot artifacts and produces a shortlist of reusable patterns worth promoting into the shared Copilot resource repository.

Current Scope

macOS first
read-only audit execution
file-based review interface
local per-machine audit history stored inside the repo checkout but excluded from git

Command

resources/scripts/audit-copilot-usage.sh --days 30

The runner excludes the current copilot-resources repo root from workspace indexing by default so you do not review patterns that already live in the shared repository.

Optional filters:

resources/scripts/audit-copilot-usage.sh --days 14 --workspace copilot-resources

resources/scripts/audit-copilot-usage.sh --days 30 --include-sources publish-log,repo-memories

resources/scripts/audit-copilot-usage.sh --days 30 --exclude-workspace /path/to/another/repo

Storage Model

Audit runs are written under:

.local/audits/<machine-id>/<timestamp>/

The runner creates the machine directory automatically on first use. The entire .local/ tree is git-ignored so each machine can keep its own audit history without creating commit noise.

Inputs

The first implementation reads from these local sources when they exist:

VS Code workspace transcripts under ~/Library/Application Support/Code/User/workspaceStorage/*/GitHub.copilot-chat/transcripts/
Repo memories under .../GitHub.copilot-chat/memory-tool/memories/repo/
Shared resource publish history under ~/.copilot-resources-state/publish-log.tsv

Outputs

audit-summary.md — high-level inventory and top candidates
workspace-index.tsv — workspaceStorage hash to workspace path mapping for the run
candidates-report.tsv — scored candidate list with source references plus transcript prompt-cost proxy columns such as token_cost_signal, avg_prompt_chars, and repeated_prompt_chars
selection-manifest.tsv — editable approval surface with decision, review_note, detail_file, and the same transcript prompt-cost proxy columns used during triage
pattern-details/*.md — per-candidate evidence bundles used during review, including purpose, expected benefit, audit context, score rationale, and caveats

Transcript prompt-cost fields are character-count proxies derived from persisted user message text. They help surface repeated long prompts that are likely to be more expensive, but they are not exact billing-token measurements.

Selection Interface

Use selection-manifest.tsv as the review surface and pattern-details/*.md as the one-by-one evidence bundle for each candidate.

Inside Copilot, use the shared review-audit-candidates prompt to work through pending rows one at a time. That prompt should ask for a decision with buttons, capture optional freeform review notes, explain why the candidate exists and why it scored as it did, and update selection-manifest.tsv after each answer.

Recommended decision states:

promote-skill
promote-instruction
promote-agent
promote-hook
promote-script
promote-prompt
template-only
docs-only
discard
needs-sanitization

Promotion Rule

If a candidate cannot be explained as a portable skill, instruction, prompt adapter, agent behavior, hook, script, or template, it should stay in audit notes and not be promoted into the shared repo.

Follow-on Draft Generation

After the manifest has approved rows, generate next-step artifacts with:

resources/scripts/prepare-audit-promotions.sh

Or target a specific run:

resources/scripts/prepare-audit-promotions.sh --audit-dir .local/audits/<machine-id>/<timestamp>

The command reads approved rows from selection-manifest.tsv and writes:

draft-resources/ for straightforward prompt, instruction, agent, and skill drafts
staging-notes/ for approved rows that still need manual design work
promotion-summary.md to summarize what was generated

Limits

This audits persisted local artifacts only.
It does not capture every inline completion or transient UI suggestion.
Windows parity is deferred until the macOS workflow and report format are proven.

4.2 KiB Raw Blame History