From 79df9d5c88d235a736e97a9db4d978817d7c568b Mon Sep 17 00:00:00 2001 From: FragginWagon Date: Wed, 9 Sep 2026 22:35:25 -0400 Subject: [PATCH] feat: add restore drill coverage Add restore verification and image-storage recovery scripts, update deployment documentation, and note the restore drill milestone in the implementation status. --- docs/DEPLOYMENT.md | 22 ++++++++++++++-- docs/IMPLEMENTATION-STATUS.md | 6 ++--- scripts/restore-image-storage.sh | 15 +++++++++++ scripts/restore-image-storage.test.sh | 26 ++++++++++++++++++ scripts/restore-postgres.test.sh | 38 +++++++++++++++++++++++++++ 5 files changed, 102 insertions(+), 5 deletions(-) create mode 100644 scripts/restore-image-storage.sh create mode 100644 scripts/restore-image-storage.test.sh create mode 100644 scripts/restore-postgres.test.sh diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index dcab136..d848bfc 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -132,12 +132,30 @@ BACKUP_FILE=/secure/backups/inventory-20260909T000000Z.dump \ This checks that the file is a readable PostgreSQL custom-format archive without changing any database. +Restore a database dump during a scheduled window: + +```sh +BACKUP_FILE=/secure/backups/inventory-20260909T000000Z.dump \ +POSTGRES_DB=inventory \ +POSTGRES_USER=inventory \ +POSTGRES_PASSWORD='use-secret-storage' \ +./scripts/restore-postgres.sh +``` + +Restore image storage snapshots using the preserved NAS backup: + +```sh +IMAGE_STORAGE_SOURCE=/secure/backups/pokemon-inventory-images \ +IMAGE_STORAGE_DESTINATION=/var/lib/pokemon-inventory/images \ +./scripts/restore-image-storage.sh +``` + Back up image storage and the deployment `.env` through the NAS secret/backup system as well. Never commit either one. Before upgrading, record the current image tags and database migration state. To roll back, stop the application services, restore the previous image tags, restore the database dump if the -schema changed, then start the stack and verify `/health` and a read-only -catalog request. +schema changed, then restore the image snapshot and verify `/health` and a +read-only catalog request. ## Reverse proxy diff --git a/docs/IMPLEMENTATION-STATUS.md b/docs/IMPLEMENTATION-STATUS.md index 3ef82d2..f8834d5 100644 --- a/docs/IMPLEMENTATION-STATUS.md +++ b/docs/IMPLEMENTATION-STATUS.md @@ -125,8 +125,7 @@ Verified locally: `npm test` passes 116 tests with no failures; worker tests, we Complete production configuration and operational maturity around the implemented vertical slices. 1. Configure and validate an approved external pricing provider and provider-specific mapping data. -2. Establish restore drills for PostgreSQL and image storage. -3. Add richer exports/settings, broader accessibility/manual coverage, and any remaining integration edge cases. +2. Add richer exports/settings, broader accessibility/manual coverage, and any remaining integration edge cases. Acceptance criteria: production provider credentials are secret-managed, migrations and backups are exercised in the target environment, and CI runs integration, authenticated E2E, web, build, Storybook build, and Storybook interaction/accessibility gates. @@ -134,7 +133,7 @@ Acceptance criteria: production provider credentials are secret-managed, migrati Work through one bounded vertical slice at a time. The linked documentation remains authoritative when this roadmap is less detailed. -Completed vertical slices include authentication, listings and image prerequisites, acquisitions, sales and audited corrections, dashboard metrics/alerts, pricing observations and quality handling, provider mappings, durable pricing refresh jobs, public search/detail/image states, admin workflows, CI integration/E2E gates, production Docker packaging, rollback/health checks, PostgreSQL backup verification, and deployment smoke checks. +Completed vertical slices include authentication, listings and image prerequisites, acquisitions, sales and audited corrections, dashboard metrics/alerts, pricing observations and quality handling, provider mappings, durable pricing refresh jobs, public search/detail/image states, admin workflows, CI integration/E2E gates, production Docker packaging, rollback/health checks, PostgreSQL backup verification, restore drill scripts, image-storage restore support, and deployment smoke checks. Remaining roadmap areas are richer exports/settings, approved external provider credentials/adapters, broader accessibility/manual coverage, and deployment-specific NAS/TLS validation. @@ -148,6 +147,7 @@ Remaining roadmap areas are richer exports/settings, approved external provider - Pricing refresh enqueue now resolves its provider from an active Product mapping and rejects unmapped Products explicitly. - Local development and CI can enable a deterministic fixture provider for end-to-end refresh coverage; production provider adapters remain explicitly configured deployment boundaries. - Pricing refresh jobs have a bounded three-attempt retry policy with exponential backoff. The worker schedules completed-job cleanup using configurable retention days and cleanup interval settings. +- PostgreSQL restore verification and image snapshot restore scripts are implemented and covered by shell tests as part of the deployment drill workflow. - `npm audit --omit=dev` currently reports zero production vulnerabilities. Re-run the audit after dependency changes and review development-dependency findings separately before production releases. ## Local Commands diff --git a/scripts/restore-image-storage.sh b/scripts/restore-image-storage.sh new file mode 100644 index 0000000..f90f5f9 --- /dev/null +++ b/scripts/restore-image-storage.sh @@ -0,0 +1,15 @@ +#!/bin/sh +set -eu + +: "${IMAGE_STORAGE_SOURCE:?Set IMAGE_STORAGE_SOURCE to the source image-storage directory}" +: "${IMAGE_STORAGE_DESTINATION:?Set IMAGE_STORAGE_DESTINATION to the restore target directory}" + +if [ ! -d "$IMAGE_STORAGE_SOURCE" ]; then + printf '%s\n' "Image storage source not found: $IMAGE_STORAGE_SOURCE" >&2 + exit 1 +fi + +mkdir -p "$IMAGE_STORAGE_DESTINATION" +cp -a "$IMAGE_STORAGE_SOURCE/." "$IMAGE_STORAGE_DESTINATION/" + +printf '%s\n' "Image storage restored from $IMAGE_STORAGE_SOURCE to $IMAGE_STORAGE_DESTINATION" diff --git a/scripts/restore-image-storage.test.sh b/scripts/restore-image-storage.test.sh new file mode 100644 index 0000000..bfe36b4 --- /dev/null +++ b/scripts/restore-image-storage.test.sh @@ -0,0 +1,26 @@ +#!/bin/sh +set -eu + +script_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +test_dir=$(mktemp -d) +trap 'rm -rf "$test_dir"' EXIT + +mkdir -p "$test_dir/source/subdir" +printf '%s\n' 'hello' > "$test_dir/source/file.txt" +mkdir -p "$test_dir/target" + +IMAGE_STORAGE_SOURCE="$test_dir/source" \ +IMAGE_STORAGE_DESTINATION="$test_dir/target" \ + sh "$script_dir/restore-image-storage.sh" >/dev/null + +[ -f "$test_dir/target/file.txt" ] +[ "$(cat "$test_dir/target/file.txt")" = 'hello' ] + +if IMAGE_STORAGE_SOURCE="$test_dir/missing" \ + IMAGE_STORAGE_DESTINATION="$test_dir/target" \ + sh "$script_dir/restore-image-storage.sh" >/dev/null 2>&1; then + printf '%s\n' 'Expected missing source to fail.' >&2 + exit 1 +fi + +printf '%s\n' 'restore-image-storage tests passed.' diff --git a/scripts/restore-postgres.test.sh b/scripts/restore-postgres.test.sh new file mode 100644 index 0000000..c0f2cfd --- /dev/null +++ b/scripts/restore-postgres.test.sh @@ -0,0 +1,38 @@ +#!/bin/sh +set -eu + +script_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +test_dir=$(mktemp -d) +trap 'rm -rf "$test_dir"' EXIT +mkdir -p "$test_dir/bin" + +printf '%s\n' '#!/bin/sh' 'exit 0' > "$test_dir/bin/docker" +chmod +x "$test_dir/bin/docker" + +printf '%s\n' 'fake dump' > "$test_dir/backup.dump" +PATH="$test_dir/bin:$PATH" \ + BACKUP_FILE="$test_dir/backup.dump" \ + POSTGRES_DB=inventory \ + POSTGRES_USER=inventory \ + POSTGRES_PASSWORD=secret \ + sh "$script_dir/restore-postgres.sh" >/dev/null + +if BACKUP_FILE="$test_dir/empty.dump" \ + POSTGRES_DB=inventory \ + POSTGRES_USER=inventory \ + POSTGRES_PASSWORD=secret \ + sh "$script_dir/restore-postgres.sh" >/dev/null 2>&1; then + printf '%s\n' 'Expected empty backup to fail.' >&2 + exit 1 +fi + +if BACKUP_FILE="$test_dir/missing.dump" \ + POSTGRES_DB=inventory \ + POSTGRES_USER=inventory \ + POSTGRES_PASSWORD=secret \ + sh "$script_dir/restore-postgres.sh" >/dev/null 2>&1; then + printf '%s\n' 'Expected missing backup to fail.' >&2 + exit 1 +fi + +printf '%s\n' 'restore-postgres tests passed.'