🛠️ Update various documentation, scripts, and configuration templates to enhance clarity, functionality, and maintainability across the project

This commit is contained in:
2026-05-04 10:56:41 +00:00
parent 1a2f1510bf
commit 31975e3088
41 changed files with 4184 additions and 133 deletions

127
docs/audit-workflow.md Normal file
View File

@@ -0,0 +1,127 @@
# Audit Workflow
## Purpose
The audit workflow reviews the last 30 days of persisted local Copilot
artifacts and produces a shortlist of reusable patterns worth promoting into the
shared Copilot resource repository.
## Current Scope
- macOS first
- read-only audit execution
- file-based review interface
- local per-machine audit history stored inside the repo checkout but excluded
from git
## Command
```bash
resources/scripts/audit-copilot-usage.sh --days 30
```
The runner excludes the current `copilot-resources` repo root from workspace
indexing by default so you do not review patterns that already live in the
shared repository.
Optional filters:
```bash
resources/scripts/audit-copilot-usage.sh --days 14 --workspace copilot-resources
```
```bash
resources/scripts/audit-copilot-usage.sh --days 30 --include-sources publish-log,repo-memories
```
```bash
resources/scripts/audit-copilot-usage.sh --days 30 --exclude-workspace /path/to/another/repo
```
## Storage Model
Audit runs are written under:
```text
.local/audits/<machine-id>/<timestamp>/
```
The runner creates the machine directory automatically on first use. The entire
`.local/` tree is git-ignored so each machine can keep its own audit history
without creating commit noise.
## Inputs
The first implementation reads from these local sources when they exist:
- VS Code workspace transcripts under `~/Library/Application Support/Code/User/workspaceStorage/*/GitHub.copilot-chat/transcripts/`
- Repo memories under `.../GitHub.copilot-chat/memory-tool/memories/repo/`
- Shared resource publish history under `~/.copilot-resources-state/publish-log.tsv`
## Outputs
- `audit-summary.md` — high-level inventory and top candidates
- `workspace-index.tsv` — workspaceStorage hash to workspace path mapping for the run
- `candidates-report.tsv` — scored candidate list with source references plus transcript prompt-cost proxy columns such as `token_cost_signal`, `avg_prompt_chars`, and `repeated_prompt_chars`
- `selection-manifest.tsv` — editable approval surface with `decision`, `review_note`, `detail_file`, and the same transcript prompt-cost proxy columns used during triage
- `pattern-details/*.md` — per-candidate evidence bundles used during review, including purpose, expected benefit, audit context, score rationale, and caveats
Transcript prompt-cost fields are character-count proxies derived from persisted
user message text. They help surface repeated long prompts that are likely to be
more expensive, but they are not exact billing-token measurements.
## Selection Interface
Use `selection-manifest.tsv` as the review surface and `pattern-details/*.md` as
the one-by-one evidence bundle for each candidate.
Inside Copilot, use the shared `review-audit-candidates` prompt to work through
pending rows one at a time. That prompt should ask for a decision with buttons,
capture optional freeform review notes, explain why the candidate exists and
why it scored as it did, and update `selection-manifest.tsv` after each answer.
Recommended decision states:
- `promote-skill`
- `promote-instruction`
- `promote-agent`
- `promote-hook`
- `promote-script`
- `promote-prompt`
- `template-only`
- `docs-only`
- `discard`
- `needs-sanitization`
## Promotion Rule
If a candidate cannot be explained as a portable skill, instruction, prompt
adapter, agent behavior, hook, script, or template, it should stay in audit
notes and not be promoted into the shared repo.
## Follow-on Draft Generation
After the manifest has approved rows, generate next-step artifacts with:
```bash
resources/scripts/prepare-audit-promotions.sh
```
Or target a specific run:
```bash
resources/scripts/prepare-audit-promotions.sh --audit-dir .local/audits/<machine-id>/<timestamp>
```
The command reads approved rows from `selection-manifest.tsv` and writes:
- `draft-resources/` for straightforward prompt, instruction, agent, and skill drafts
- `staging-notes/` for approved rows that still need manual design work
- `promotion-summary.md` to summarize what was generated
## Limits
- This audits persisted local artifacts only.
- It does not capture every inline completion or transient UI suggestion.
- Windows parity is deferred until the macOS workflow and report format are
proven.