storybid/ops/unraid-install.md

# Storybid — Unraid Install Guide (CLI Only)

End-to-end installation of Storybid on a freshly provisioned Unraid server, in
the order you should perform each step. Every operation in this document runs
from the Unraid terminal (SSH or **Tools → Web Terminal**). No Community
Applications GUI workflows are required for the application stack itself.

The result is a four-container Compose stack (Postgres + Redis + API server +
Nginx client) reachable on your LAN, fronted by Nginx Proxy Manager on the
public internet, with Stripe payments, Twilio Verify SMS OTP, transactional
email, and event-night LAN failover all wired up.

---

## Contents

1. [Bill of materials](#1-bill-of-materials)
2. [External account prerequisites](#2-external-account-prerequisites)
3. [DNS and domain prep](#3-dns-and-domain-prep)
4. [Stripe account setup (web)](#4-stripe-account-setup-web)
5. [Twilio Verify setup (web)](#5-twilio-verify-setup-web)
6. [SMTP relay setup (web)](#6-smtp-relay-setup-web)
7. [Unraid server first boot](#7-unraid-server-first-boot)
8. [Unraid system configuration](#8-unraid-system-configuration)
9. [Required plugins (one-time, via terminal)](#9-required-plugins-one-time-via-terminal)
10. [Reserve the server's LAN IP](#10-reserve-the-servers-lan-ip)
11. [Open SSH and connect from your workstation](#11-open-ssh-and-connect-from-your-workstation)
12. [Lay out the appdata directory](#12-lay-out-the-appdata-directory)
13. [Get the source code on the server](#13-get-the-source-code-on-the-server)
14. [Configure `.env`](#14-configure-env)
15. [Pin Docker volumes to host paths](#15-pin-docker-volumes-to-host-paths)
16. [Build and start the stack](#16-build-and-start-the-stack)
17. [Initialize the database schema](#17-initialize-the-database-schema)
18. [Create the first organization and admin user](#18-create-the-first-organization-and-admin-user)
19. [Install Nginx Proxy Manager and issue a certificate](#19-install-nginx-proxy-manager-and-issue-a-certificate)
20. [Register the Stripe webhook](#20-register-the-stripe-webhook)
21. [UniFi event-night network configuration](#21-unifi-event-night-network-configuration)
22. [End-to-end smoke test](#22-end-to-end-smoke-test)
23. [Backups and disaster recovery](#23-backups-and-disaster-recovery)
24. [Updates and rollback](#24-updates-and-rollback)
25. [Troubleshooting](#25-troubleshooting)

---

## 1. Bill of materials

| Item | Recommended |
|---|---|
| Server hardware | x86_64 box with 4+ cores, 8 GB RAM, 256 GB SSD for appdata |
| OS | Unraid 6.12 or later, written to a 16 GB+ USB stick |
| Network | Gigabit LAN, UniFi gateway + AP (any model with Local DNS support) |
| UPS | Any battery backup with USB; 10+ minute runtime under load |
| Public domain | One A-record under your control (e.g. `bid.example.org`) |
| Workstation | A laptop with `ssh` and a browser to drive setup |

---

## 2. External account prerequisites

You will need accounts at:

- **Stripe** — payments. Sign up at <https://dashboard.stripe.com/register>.
- **Twilio** — SMS OTP. Sign up at <https://www.twilio.com/try-twilio>.
- **SMTP provider** — Postmark, Mailgun, SendGrid, AWS SES, or your own relay.
- **Domain registrar** — Cloudflare, Namecheap, Route 53, etc.

Sections 4 – 6 walk through these end to end. Do them **before** touching the
server so you can paste the credentials directly into `.env` later.

---

## 3. DNS and domain prep

You need one public hostname for bidders (e.g. `bid.example.org`). Stripe
webhooks and PWA installation both require a publicly resolvable HTTPS URL.

1. Log in to your DNS provider.
2. Create an **A record**:
   - Name: `bid` (or any subdomain you prefer)
   - Type: `A`
   - Value: your home/event router's public WAN IP
   - TTL: 300 (low, so failover edits propagate quickly)
3. If your WAN IP is dynamic, also enable Dynamic DNS or use a provider like
   Cloudflare with the API-based updater on Unraid.
4. At your home router, forward TCP **80** and **443** to the Unraid server's
   LAN IP (you'll reserve that IP in section 10).

Verify from your workstation:

```bash
dig +short bid.example.org    # should return your WAN IP
```

---

## 4. Stripe account setup (web)

The goal of this section is to walk away with three secrets: a publishable
key, a secret key, and a webhook signing secret.

### 4.1. Create the account

1. Go to <https://dashboard.stripe.com/register>.
2. Provide email, full name, and a password. Confirm via the email link.
3. Stripe starts you in **Test mode** — keep it there until you finish smoke
   testing. Toggle **View test data** in the top-right at any time.

### 4.2. Activate your account (only required for live payments)

1. In the dashboard, click **Activate account** in the left rail.
2. Provide:
   - Business type (most charities: `Non-profit organization`)
   - EIN / tax ID
   - Business address
   - Bank account for payouts (routing + account number)
   - A representative's date of birth and SSN (last 4 in the US, full SSN
     for higher payout volumes)
3. Submit. Activation usually clears within an hour for clean filings.

You can keep test mode active for now and only flip the dashboard toggle to
**live** after the smoke test in section 22 passes.

### 4.3. Grab the API keys

1. **Developers → API keys**.
2. **Publishable key** — visible by default. Copy the value that starts with
   `pk_test_…` (test mode) or `pk_live_…` (live mode).
3. **Secret key** — click **Reveal test key** (or **Create restricted key**
   for live). Copy the value starting with `sk_test_…` / `sk_live_…`.
4. Stash both somewhere safe (password manager). You'll paste them into
   `.env` in section 14.

> **Warning:** the secret key is shown **once**. If you lose it, click
> **Roll key** to generate a new one — the old one is then immediately void.

### 4.4. Note the webhook URL — you will register it after deployment

The webhook **endpoint** can only be created once your public URL serves real
TLS, so we register the webhook in section 20 (after Nginx Proxy Manager is
up). For now just note the URL pattern:

```
https://bid.example.org/api/webhooks/stripe
```

You will receive the `whsec_…` signing secret at that point.

---

## 5. Twilio Verify setup (web)

Goal: walk away with three secrets — Account SID, Auth Token, and a Verify
Service SID.

### 5.1. Create the account

1. Go to <https://www.twilio.com/try-twilio>.
2. Provide email, password, and a phone number for verification. Twilio
   sends an SMS OTP to confirm the number.
3. After login, Twilio asks a few onboarding questions. Answer:
   - **What do you want to do first?** → *Verify users*
   - **Which Twilio product?** → *Verify*
   - **What language?** → *Node.js* (label only; doesn't matter)
4. Land on the Twilio Console at <https://console.twilio.com>.

### 5.2. Get the Account SID and Auth Token

1. On the Console home, look at the **Account Info** card.
2. Copy:
   - **Account SID** — starts with `AC…` (34 chars)
   - **Auth Token** — click **Show** to reveal; starts with hex chars
3. Stash both. They go in `.env` as `TWILIO_ACCOUNT_SID` and
   `TWILIO_AUTH_TOKEN`.

### 5.3. Create a Verify service

1. In the left rail, expand **Explore Products → Verify**.
2. Click **Services → Create new** (or the **+** button).
3. Settings:
   - **Friendly name**: `Storybid OTP`
   - **Code length**: `6`
   - **Default channel**: `SMS`
   - Leave the rest at defaults; turn off Email OTP unless you've configured
     a SendGrid integration.
4. Click **Create**.
5. On the resulting service page, copy the **Service SID** — starts with
   `VA…`. This is `TWILIO_VERIFY_SERVICE_SID` in `.env`.

### 5.4. Enable production countries (only for live use)

Free trial accounts can only send to verified phone numbers. To send to any
US phone:

1. Add a payment method: **Account → Billing → Payment method**. Recharge
   $20 to start.
2. **Verify → Service → Geography** — confirm the countries you'll send to
   are enabled (US is on by default).
3. **Phone Numbers → Manage → Buy a number** is **not** required for Verify
   — Twilio uses a shared sender pool by default.

### 5.5. Optional — leave SMS off

If you don't want SMS at all, leave all three Twilio variables blank in
`.env`. Email magic-link login will still work for both bidders and staff.

---

## 6. SMTP relay setup (web)

You need any provider that gives you SMTP host + port + username + password.
Postmark is the simplest for transactional mail. Setup with any provider
follows the same pattern; Postmark example below.

1. Sign up at <https://postmarkapp.com> with the email you'll send *from*.
2. Postmark requires sender verification:
   - **Sender Signatures → Add Signature** with your `noreply@example.org`
     address, **or**
   - **Domains → Add Domain** to authorize the whole domain via DKIM
     (recommended; one-time DNS records).
3. **Servers → My First Server → API Tokens** — create a server token and
   copy it. With Postmark this token is **both** the SMTP username and
   password.
4. Note the connection details (Postmark example):
   - `SMTP_HOST=smtp.postmarkapp.com`
   - `SMTP_PORT=587`
   - `SMTP_USER=<server-token>`
   - `SMTP_PASS=<same server-token>`
   - `EMAIL_FROM=Storybid <noreply@example.org>`

Send a test message from the provider's dashboard before continuing — broken
DKIM or unverified senders will silently bounce magic-link emails.

---

## 7. Unraid server first boot

1. Download Unraid USB Creator: <https://unraid.net/download>.
2. Plug in a 16 GB+ USB 2.0 stick (USB 2.0 is more compatible than 3.0 for
   booting), pick the latest stable Unraid release, write it to the stick.
3. Plug the USB into the server, set BIOS to **boot from USB** first.
4. Power on. Pick **Unraid OS** at the GRUB prompt; the first boot takes
   2–3 minutes.
5. The console shows the LAN IP it received from DHCP, e.g.:
   ```
   Welcome to Unraid Server OS!
   Tower login:
   IP address(es) of this server: 192.168.1.50
   ```
6. From your workstation browser, go to `http://192.168.1.50` (replace with
   your IP). The Unraid web UI loads.
7. Set the root password when prompted. **Use a strong one** — this is the
   only protection on the box.
8. Register a free **Trial** key (or paste a purchased key) — **Tools →
   Registration**. The trial is 30 days, plenty for the install + first event.

---

## 8. Unraid system configuration

Do all of these **before** building the array, since some require a stopped
array.

### 8.1. Build the array

1. **Main → Array Devices**:
   - Assign one or more disks to the **Parity** slot (recommended for
     production).
   - Assign disk(s) to **Disk 1**, **Disk 2**, … as data drives.
   - If you have an SSD, assign it to the **Cache** pool — Storybid's
     `appdata` should live on cache for performance.
2. Click **Start** to format and bring the array online. First parity sync
   can take hours; it does not block the rest of the install.

### 8.2. Time, timezone, and identification

- **Settings → Date and Time** — set timezone to where the events run; check
  **Use NTP** with a public pool.
- **Settings → Identification** — name the server (`storybid-prod` is
  conventional) and set a static workgroup if you care about SMB.

### 8.3. Network

- **Settings → Network Settings → eth0** — switch from DHCP to a static IP
  matching the reservation you'll create on the router in section 10. Use
  the same value, e.g. `192.168.1.50`, mask `255.255.255.0`, gateway your
  router's LAN IP, DNS `192.168.1.1` (router) and `1.1.1.1` (fallback).
- Click **Apply**. The web UI may briefly disconnect — reconnect at the new
  static IP.

### 8.4. Docker

- **Settings → Docker → Enable Docker** = `Yes`.
- Leave **Docker data-root** at the default (`/var/lib/docker` on the cache
  pool). The Storybid stack uses ~3 GB built.
- Click **Apply**.

### 8.5. SSH

- **Settings → Management Access → Use SSH** = `Yes`.
- **Use SFTP** = `Yes` (handy for transferring files later).
- Click **Apply**.

### 8.6. Notifications (optional but recommended)

- **Settings → Notification Settings** — enable email or push (Discord,
  Pushover, etc.) for **Warning**, **Alert**, **System**. You want to know
  immediately if the array degrades during an event.

---

## 9. Required plugins (one-time, via terminal)

Open the Unraid web terminal: **top-right → Terminal icon (>_)**. The same
session is reachable from your workstation once SSH is enabled in the next
step — for now use the web terminal.

Install Community Applications, which is the bootstrap plugin for everything
else:

```bash
plugin install https://raw.githubusercontent.com/Squidly271/community.applications/master/plugins/community.applications.plgs
```

Then install **NerdTools** (gives you `git`, `nano`, `htop`, `unzip`, …):

```bash
plugin install https://raw.githubusercontent.com/dmacias72/unRAID-NerdTools/master/plugins/nerdtools.plg
```

Activate `git` (NerdTools is a package selector — use its CLI):

```bash
nerdctl install git nano unzip openssl
```

> If `nerdctl` is unavailable on your Unraid version, open **Apps →
> Installed Apps → NerdTools → Settings**, tick **git, nano, unzip,
> openssl**, click **Apply**. Plugin selection is the only GUI step
> in this guide — there is no terminal-equivalent because plugin
> selection writes to a config the plugin daemon watches.

Verify:

```bash
git --version
docker --version
docker compose version
openssl version
```

All four must print a version string.

---

## 10. Reserve the server's LAN IP

On your UniFi controller (or whatever router you use):

1. **Client Devices → Tower (the Unraid box) → Settings → Network**.
2. **Fixed IP Address** = the IP you set in section 8.3 (e.g.
   `192.168.1.50`).
3. Click **Save**. Reboot the Unraid box once to confirm it picks up the
   reserved address (`reboot` from the terminal).

---

## 11. Open SSH and connect from your workstation

From your workstation:

```bash
ssh root@192.168.1.50      # use your Unraid LAN IP
```

You'll be prompted for the root password you set in section 7. **Stay in
this SSH session for every remaining step.**

(Optional) Copy your public key to skip the password prompt next time:

```bash
ssh-copy-id root@192.168.1.50
```

---

## 12. Lay out the appdata directory

```bash
mkdir -p /mnt/user/appdata/storybid/{repo,postgres,redis,uploads,backups}
ls -la /mnt/user/appdata/storybid
```

You should see five subdirectories. They're owned by `nobody:users` (Unraid
default) — do not chown them.

---

## 13. Get the source code on the server

### 13.1. From a Git remote (preferred)

```bash
cd /mnt/user/appdata/storybid
git clone https://github.com/YOUR_ORG/storybid.git repo
cd repo
git status     # confirm "nothing to commit, working tree clean"
```

### 13.2. From a local zip (if your repo is private and unreachable)

On your workstation, build a zip of the repository, then from Unraid:

```bash
# From your workstation:
scp storybid.zip root@192.168.1.50:/mnt/user/appdata/storybid/

# Back on Unraid:
cd /mnt/user/appdata/storybid
unzip storybid.zip -d repo
ls repo/      # should contain package.json, docker-compose.yml, etc.
```

---

## 14. Configure `.env`

```bash
cd /mnt/user/appdata/storybid/repo
cp .env.example .env
```

Generate a JWT secret and capture it:

```bash
openssl rand -hex 32
```

Copy the 64-character output. Open `.env` for editing:

```bash
nano .env
```

Fill in every line. Reference values:

```bash
# ── Database ──────────────────────────────────────────────────────────────────
# Hostname is the docker-compose service name 'db'.
# Replace CHANGE_ME with a strong password — use the same value when you edit
# docker-compose.yml in section 15.
DATABASE_URL="postgresql://storybid:CHANGE_ME@db:5432/storybid"

# ── Redis ─────────────────────────────────────────────────────────────────────
REDIS_URL="redis://redis:6379"

# ── App ───────────────────────────────────────────────────────────────────────
NODE_ENV=production
PORT=3001
PUBLIC_URL="https://bid.example.org"
LOCAL_HOSTNAME="auction.event.lan"
JWT_SECRET="<paste the openssl rand -hex 32 output here>"

# ── Stripe (from section 4) ───────────────────────────────────────────────────
STRIPE_SECRET_KEY="sk_test_..."
STRIPE_PUBLISHABLE_KEY="pk_test_..."
# Leave blank for now — fill in after section 20:
STRIPE_WEBHOOK_SECRET=""

# ── Twilio Verify (from section 5; leave blank to disable SMS) ────────────────
TWILIO_ACCOUNT_SID="ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
TWILIO_AUTH_TOKEN="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
TWILIO_VERIFY_SERVICE_SID="VAxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

# ── Media storage ─────────────────────────────────────────────────────────────
UPLOAD_DIR=/app/uploads
MEDIA_BASE_URL=/media

# ── Email (from section 6) ────────────────────────────────────────────────────
SMTP_HOST="smtp.postmarkapp.com"
SMTP_PORT=587
SMTP_USER="<postmark server token>"
SMTP_PASS="<same postmark server token>"
EMAIL_FROM="Storybid <noreply@example.org>"
```

Save and exit (Ctrl+O, Enter, Ctrl+X in nano).

Strip any Windows line endings if the file was edited on Windows:

```bash
sed -i 's/\r$//' .env
```

---

## 15. Pin Docker volumes to host paths

The shipped `docker-compose.yml` uses anonymous volumes by default. Replace
the `volumes:` block at the bottom so Postgres, Redis, and uploads land in
the appdata directories you created in section 12. Also align the Postgres
password with your `.env`.

```bash
nano docker-compose.yml
```

Change the `db` service password and append/replace the bottom volumes block:

```yaml
services:
  db:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: storybid
      POSTGRES_PASSWORD: CHANGE_ME      # ← match the password in .env
      POSTGRES_DB: storybid
    volumes:
      - postgres_data:/var/lib/postgresql/data
    # Remove the "ports: 5432" block for production — DB stays on Docker net
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U storybid"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    volumes:
      - redis_data:/data
    # Remove the "ports: 6379" block for production
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5

  server:
    build:
      context: .
      dockerfile: packages/server/Dockerfile
    restart: unless-stopped
    env_file: .env
    environment:
      DATABASE_URL: postgresql://storybid:CHANGE_ME@db:5432/storybid   # ← match
      REDIS_URL: redis://redis:6379
      NODE_ENV: production
      UPLOAD_DIR: /app/uploads
      MEDIA_BASE_URL: /media
    volumes:
      - media_data:/app/uploads
    ports:
      - "3001:3001"
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy

  client:
    build:
      context: .
      dockerfile: packages/client/Dockerfile
    restart: unless-stopped
    ports:
      - "8080:80"
    depends_on:
      - server

volumes:
  postgres_data:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /mnt/user/appdata/storybid/postgres

  redis_data:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /mnt/user/appdata/storybid/redis

  media_data:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /mnt/user/appdata/storybid/uploads
```

Save and exit.

---

## 16. Build and start the stack

```bash
cd /mnt/user/appdata/storybid/repo
docker compose up -d --build
```

First build pulls base images and compiles TypeScript — allow 3–8 minutes
depending on internet speed. Watch progress in another terminal pane:

```bash
docker compose logs -f
```

When the build finishes, confirm all four services are running:

```bash
docker compose ps
```

You want all four lines reading `Up` (and `db`/`redis` reading `healthy`):

```
NAME                  STATUS
storybid-db-1         Up (healthy)
storybid-redis-1      Up (healthy)
storybid-server-1     Up
storybid-client-1     Up
```

---

## 17. Initialize the database schema

This repository does not ship Prisma migration files — production deploys
the schema with `prisma db push`, which syncs the database to
`schema.prisma` directly.

```bash
docker compose exec server npx prisma db push --skip-generate
```

Expected output:

```
🚀  Your database is now in sync with your Prisma schema.
```

Verify the tables exist:

```bash
docker compose exec db psql -U storybid -d storybid -c '\dt'
```

You should see ~15 tables (`Organization`, `AuctionEvent`, `Auction`,
`AuctionItem`, `Bidder`, `Bid`, `Invoice`, etc.).

---

## 18. Create the first organization and admin user

The server image bundles the seed script. Run it once to create a default
organization and demo event:

```bash
docker compose exec server npx tsx prisma/seed.ts
```

Then create your real organization and admin staff user. Replace the values
inline:

```bash
docker compose exec server node --input-type=module -e "
import { PrismaClient } from '@prisma/client';
const prisma = new PrismaClient();

const org = await prisma.organization.create({
  data: {
    name: 'Your Charity Name',
    slug: 'your-charity',
    primaryColor: '#2563eb',
    publicUrl: 'https://bid.example.org',
    localHostname: 'auction.event.lan',
    staffUsers: {
      create: {
        name: 'Site Admin',
        email: 'admin@example.org',
        role: 'admin',
      },
    },
  },
  include: { staffUsers: true },
});

console.log('Organization:', org.id, org.name);
console.log('Admin:', org.staffUsers[0].email);
await prisma.\$disconnect();
"
```

The command prints the generated IDs. Sign in flow:

1. Open `http://192.168.1.50:8080` in a browser on the same LAN.
2. Click **Sign in**, enter `admin@example.org`.
3. The magic-link email lands in your SMTP inbox (check spam).
4. Click the link — you're now signed in as admin.

If the email doesn't arrive, check `docker compose logs server | grep -i mail`
for SMTP errors.

---

## 19. Install Nginx Proxy Manager and issue a certificate

The application stack runs on plain HTTP internally. NPM terminates TLS in
front of it, handles Let's Encrypt renewals, and proxies WebSockets.

### 19.1. Pull and start NPM as a separate Compose stack

```bash
mkdir -p /mnt/user/appdata/npm/{data,letsencrypt}
cat > /mnt/user/appdata/npm/docker-compose.yml <<'EOF'
version: "3.9"
services:
  npm:
    image: jc21/nginx-proxy-manager:latest
    restart: unless-stopped
    ports:
      - "80:80"
      - "81:81"     # Admin UI — close after first login
      - "443:443"
    volumes:
      - /mnt/user/appdata/npm/data:/data
      - /mnt/user/appdata/npm/letsencrypt:/etc/letsencrypt
EOF

cd /mnt/user/appdata/npm
docker compose up -d
```

### 19.2. First-login admin password

```bash
# Default credentials — change these on first login.
echo "Email:    admin@example.com"
echo "Password: changeme"
```

Open `http://192.168.1.50:81` in a browser. Sign in with the defaults above
— NPM forces a password change. Set a strong admin password and a real
email.

### 19.3. Add the proxy host (CLI-driven config — done once via the web UI)

NPM does not currently expose a stable CLI for proxy hosts. Use the admin
UI on port 81 once:

| Field | Value |
|---|---|
| Domain Names | `bid.example.org` |
| Scheme | `http` |
| Forward Hostname / IP | `192.168.1.50` (the Unraid LAN IP) |
| Forward Port | `8080` |
| Block Common Exploits | ✅ |
| Websockets Support | ✅ |
| **SSL tab → SSL Certificate** | Request a new SSL Certificate (Let's Encrypt) |
| **SSL tab → Force SSL** | ✅ |
| **SSL tab → HTTP/2 Support** | ✅ |
| Email | your contact address |
| Agree to Let's Encrypt TOS | ✅ |

Save. NPM provisions the cert in 30–60 seconds. The bidder app is now
reachable at `https://bid.example.org`.

### 19.4. Close the NPM admin port

Once the proxy host is verified working, close port 81 to the internet.
Edit `/mnt/user/appdata/npm/docker-compose.yml` and remove the `81:81`
line, then:

```bash
cd /mnt/user/appdata/npm && docker compose up -d
```

The admin UI is then only reachable from the LAN via SSH tunnel:

```bash
ssh -L 8181:192.168.1.50:81 root@192.168.1.50
# Then browse http://localhost:8181 from your workstation.
```

---

## 20. Register the Stripe webhook

Now that `https://bid.example.org` serves real TLS:

1. Go to <https://dashboard.stripe.com/webhooks> (test or live mode — match
   what your `.env` keys are for).
2. Click **+ Add endpoint**.
3. **Endpoint URL**: `https://bid.example.org/api/webhooks/stripe`
4. **Events to send**: select **Select events**, then check:
   - `payment_intent.succeeded`
   - `payment_intent.payment_failed`
5. Click **Add endpoint**.
6. On the resulting endpoint page, click **Reveal** under **Signing secret**.
   Copy the value (starts with `whsec_…`).
7. Back on Unraid, paste it into `.env`:

   ```bash
   cd /mnt/user/appdata/storybid/repo
   nano .env
   # Set: STRIPE_WEBHOOK_SECRET="whsec_..."
   ```

8. Restart the server container so it picks up the new value:

   ```bash
   docker compose restart server
   ```

9. Back in the Stripe dashboard, click **Send test webhook** on the endpoint
   page. Pick `payment_intent.succeeded`. The endpoint should respond
   `200 OK` within a second. Confirm in:

   ```bash
   docker compose logs server | grep -i webhook
   ```

If you see `signature verification failed`, the secret doesn't match —
re-copy from the dashboard and restart.

---

## 21. UniFi event-night network configuration

This wiring is what makes Storybid survive a WAN outage during a live
auction. See also `ops/unifi-dns.md`.

### 21.1. Local DNS record

UniFi Network → **Settings → Networks → DNS Records** (older firmware:
**Settings → Profiles → DNS**) → **Create Entry**:

| Field | Value |
|---|---|
| Type | `A` |
| Hostname | `auction.event.lan` |
| Value | `192.168.1.50` (Unraid LAN IP) |
| TTL | 60 |

Save. `LOCAL_HOSTNAME` in `.env` already matches.

Verify from a device on the LAN:

```bash
nslookup auction.event.lan 192.168.1.1     # router IP
# → 192.168.1.50
```

### 21.2. Dedicated event SSID

UniFi Network → **Settings → WiFi → Create New WiFi Network**:

- Name: `GalaAuction`
- Password: shared on event signage / check-in
- Network: same VLAN as the server
- Band steering: ✅ (cleaner 5 GHz preference)
- Apply.

### 21.3. Failover smoke test

1. Join `GalaAuction` on a phone.
2. Open `https://bid.example.org` — it loads via WAN.
3. From the UniFi dashboard, **disable the WAN port** (or unplug the modem).
4. Reload the bidder page. The connectivity banner should turn yellow
   (*"Local network — offline-capable"*) and the catalog still works.
5. Re-enable WAN. Banner returns to green within 5 seconds.

### 21.4. UPS

Plug **server, gateway, and APs** into a single UPS. The whole local
network must stay up if shore power blips, otherwise failover is moot.

---

## 22. End-to-end smoke test

Drive these from a real phone on the LAN. Tracking against the runbook in
`event-runbook/preflight.md`:

1. **Admin** — sign in at `https://bid.example.org/admin`, create one
   `live` auction with one item, and one `silent` auction with one item
   that closes 5 minutes from now.
2. **Bidder** — sign in via SMS OTP using a real phone number (Twilio
   dashboard → **Verify → Logs** should show the sent OTP).
3. **Silent bid** — place a bid; another bidder outbids; outbid notification
   arrives.
4. **Silent close** — wait for the timer; the high bid wins.
5. **Live bid** — open `/staff/auctioneer` on a tablet, activate the live
   item, accept a paddle bid, sell it.
6. **Checkout** — go to `/checkout`, pay with Stripe test card
   `4242 4242 4242 4242`, exp `12/34`, CVC `123`. Stripe webhook fires;
   invoice flips to `paid`.
7. **Failover** — repeat step 21.3 mid-bid; bid is queued and syncs when
   WAN returns.

If every step works, flip Stripe from test to live (section 4.2), update
`STRIPE_*` keys in `.env`, restart the server, and re-issue the webhook
secret in live mode.

---

## 23. Backups and disaster recovery

### 23.1. Database snapshot script

```bash
cat > /mnt/user/appdata/storybid/backups/snapshot.sh <<'EOF'
#!/bin/bash
set -euo pipefail
BACKUP_DIR=/mnt/user/appdata/storybid/backups
STAMP=$(date +%F-%H%M)
docker exec storybid-db-1 pg_dump -U storybid storybid \
  | gzip > "$BACKUP_DIR/storybid-$STAMP.sql.gz"
# Keep 14 days
find "$BACKUP_DIR" -name 'storybid-*.sql.gz' -mtime +14 -delete
EOF
chmod +x /mnt/user/appdata/storybid/backups/snapshot.sh
```

Schedule it via Unraid's `User Scripts` plugin **or** plain cron. CLI cron
on Unraid persists across reboots only if you write it to `go`:

```bash
cat >> /boot/config/go <<'EOF'
# Storybid: nightly DB snapshot at 03:00
echo "0 3 * * * /mnt/user/appdata/storybid/backups/snapshot.sh" | crontab -
EOF
```

Reboot the server once so the cron entry takes effect, or run the line
inside `cat <<EOF` interactively now.

### 23.2. Off-site copy

Push snapshots somewhere that isn't this server:

```bash
# Example: rclone to any cloud drive after `rclone config` once.
rclone copy /mnt/user/appdata/storybid/backups/ remote:storybid-backups
```

### 23.3. Restore drill (do this **before** your first event)

```bash
# Take down the server but leave DB up
docker compose stop server
gunzip < /mnt/user/appdata/storybid/backups/storybid-YYYY-MM-DD-HHMM.sql.gz \
  | docker exec -i storybid-db-1 psql -U storybid -d storybid
docker compose start server
```

Confirm the admin login still works.

---

## 24. Updates and rollback

### 24.1. Update

```bash
cd /mnt/user/appdata/storybid/repo
git fetch --tags
git checkout <release-tag>     # e.g. v0.1.2 — never deploy off main blindly
docker compose build
docker compose up -d
docker compose exec server npx prisma db push --skip-generate
```

### 24.2. Rollback

```bash
cd /mnt/user/appdata/storybid/repo
git checkout <previous-tag>
docker compose build
docker compose up -d

# If schema changed and the new release added required columns,
# restore from the most recent pre-update backup:
docker compose stop server
gunzip < /mnt/user/appdata/storybid/backups/storybid-PRE-UPGRADE.sql.gz \
  | docker exec -i storybid-db-1 psql -U storybid -d storybid
docker compose start server
```

> Always take a snapshot **immediately before** running `git pull` on the
> day of an event:
>
> ```bash
> /mnt/user/appdata/storybid/backups/snapshot.sh
> cp /mnt/user/appdata/storybid/backups/storybid-$(date +%F)*.sql.gz \
>    /mnt/user/appdata/storybid/backups/storybid-PRE-UPGRADE.sql.gz
> ```

---

## 25. Troubleshooting

### Containers crash-loop on startup

```bash
docker compose logs server | tail -50
```

Common causes:

- `DATABASE_URL` password mismatch — re-check `.env` and the `db` service
  password in `docker-compose.yml`.
- `JWT_SECRET` blank — magic links can't sign; regenerate with
  `openssl rand -hex 32`.
- `.env` has CRLF line endings — fix with `sed -i 's/\r$//' .env`.

### Magic-link emails don't arrive

```bash
docker compose logs server | grep -iE 'mail|smtp'
```

- 535 auth — wrong SMTP user/password.
- DNS lookup failure — Unraid host DNS broken; set `1.1.1.1` in
  Settings → Network.
- Mail accepted but never delivered — DKIM/SPF not set. Verify the sending
  domain in your provider's dashboard.

### Stripe webhook returns 400 / 401 / 500

```bash
docker compose logs server | grep -i stripe
```

- `signature verification failed` — `STRIPE_WEBHOOK_SECRET` mismatch.
  Re-copy from the dashboard and `docker compose restart server`.
- `endpoint timeout` — NPM not forwarding `/api/webhooks/stripe`. Test
  manually: `curl -i https://bid.example.org/api/webhooks/stripe -X POST`
  should return `400 missing-stripe-signature`, not `502`.

### Twilio Verify "60200 — Invalid parameter"

The phone number isn't in E.164 (`+15555551234`). Make sure the bidder UI
passes the country code; the dialer hint should default to `+1`.

### Bidders can't reach `auction.event.lan`

```bash
nslookup auction.event.lan 192.168.1.1
```

If this fails:

- The UniFi DNS record didn't save — re-add via section 21.1.
- The bidder device is using a public DNS (e.g. iCloud Private Relay)
  instead of the gateway. UniFi Network → **Settings → Networks → LAN →
  DNS Server** = `Auto`, and disable Private Relay on test devices.

### Database disk fills up

```bash
docker compose exec db psql -U storybid -d storybid -c "
  SELECT pg_size_pretty(pg_database_size('storybid'));"
```

Trim audit logs older than the most recent event:

```bash
docker compose exec db psql -U storybid -d storybid -c "
  DELETE FROM \"AuditLog\" WHERE \"createdAt\" < NOW() - INTERVAL '180 days';
  VACUUM FULL;"
```

### Need to wipe and start over

```bash
cd /mnt/user/appdata/storybid/repo
docker compose down -v          # destroys named volumes
rm -rf /mnt/user/appdata/storybid/{postgres,redis,uploads}/*
mkdir -p /mnt/user/appdata/storybid/{postgres,redis,uploads}
docker compose up -d --build
docker compose exec server npx prisma db push --skip-generate
docker compose exec server npx tsx prisma/seed.ts
```

---

## Done

The server is now a clean, repeatable Storybid install. Pin this document
inside your operational runbook (`event-runbook/preflight.md` references
the same paths and commands). Re-run the smoke test before every event,
take a fresh DB snapshot the morning of, and you're set.