yttrx-welcomebot/CLAUDE.md

# CLAUDE.md

Guidance for Claude Code when working on **yttrx-welcomebot** (the welcome bot
+ abuse auto-silence bot for yttrx.com).

## What this is / where it runs

A FastAPI webhook server. One Mastodon admin webhook delivers `account.created`,
`account.approved`, and `report.created` to `POST /webhook`; the app dispatches:

- **account.created** → classify the signup IP via ipapi.is; flagged
  (datacenter/vpn/proxy/tor/abuser) signups get more scrutiny (held welcome,
  ip_blocks registration, DM to moderator) before falling through to a normal
  welcome.
- **account.approved** → always DM the welcome (dedup-safe; this is what
  releases a held welcome once a human clears the approval queue). Also where
  a held, flagged signup's suspicious-watch baseline gets captured.
- **report.created** → classify the reported account, count distinct reporters,
  and auto-**silence** young/dormant accounts past the threshold — lowered for
  accounts with a flagged signup IP (see README).

Alongside the webhook it serves a **reputation query API** — `POST
/api/reputation` — which is the pre-emptive counterpart to all of the above.
Mastodon calls it during signup, immediately before its own IpBlock /
EmailDomainBlock lookups; it classifies the IP/email domain, records it,
registers the block, and only then returns, so the lookup that follows
rejects the signup outright. It **writes**, so it is authenticated
(`X-Reputation-Key`) and 404s unless `REPUTATION_API_ENABLED` *and*
`REPUTATION_API_KEY` are both set. The caller is the
`call-welcomebot-reputation-api` patch on the Mastodon fork — see
`~/secondbrain/yttrx-documentation/mastodon-fork-patches.md`. README.md's
"Reputation query API" section has the full contract.

Alongside the webhook it serves an optional read-only **blocklist API** —
`GET /api/blocks`, `/api/blocks/ip-ranges`, `/api/blocks/email-domains` —
publishing the IP ranges and email domains the bot has auto-blocked. It is
unauthenticated, so it 404s unless `BLOCKS_API_ENABLED=true`; yttrx.com sets
that in the box's `.env`, the shipped default is off. Responses assert only
*that* a block exists — no account identifiers, and deliberately none of the
ipapi.is / check-mail.org classification behind the decision, so we're not
republishing a paid reputation feed's verdicts. See README.md's "Blocklist
API" section.

A separate scheduled module, `app/suspicious_sweep.py` (run via
`python -m app.suspicious_sweep`, not part of the webhook server), suspends
any IP- or email-flagged signup that shows zero new posts and zero new
follows within `SUSPICIOUS_GRACE_HOURS` of going live. See README.md's
"Suspicious-signup sweep" section for the full policy.

| Where | What |
|---|---|
| Workstation | Source of truth: `~/yttrx-welcomebot/` (this repo) |
| `git.blairhaus.net/yttrx-welcomebot.git` | `origin` remote (HTTP Basic Auth, `GIT_AUTH_USER`/`GIT_AUTH_PASS` in `~/docker/.env` on admin — same shared cred as every other git.blairhaus.net repo, embedded in the remote URL). `.gitea/workflows/docker-build-push.yml` runs on every push to `main` (added 2026-07-06): builds the image and pushes `gitea.blairhaus.net/pmb/yttrx-welcomebot:latest` + `:<sha>` to the Gitea container registry. |
| `admin.yttrx.com` | Deploy target: `/root/yttrx-welcomebot/` is now a **plain directory** (converted from a git checkout 2026-07-06, which itself replaced an rsync-deployed tree 2026-07-02) containing *only* `docker-compose.yml`, `.env`, and dated `.env.bak-*` snapshots — no source tree, no `.git`. Docker container `yttrx-welcomebot` on `127.0.0.1:8087`, nginx site `hooks` → `hooks.yttrx.com`. `ssh admin.yttrx.com` logs in as **root**. Compose is **`docker-compose` v1.29.2** (hyphenated, not `docker compose`). `docker-compose.yml` pulls `image: gitea.blairhaus.net/pmb/yttrx-welcomebot:latest` (switched from `build: .` 2026-07-06) — `root`'s `~/.docker/config.json` on this box holds a read-only (`read:package`) Gitea PAT for the pull. Deploy is now purely `docker-compose pull && docker-compose up -d`, no `git pull` involved at all — there's nothing left there to pull. A full tar of the old git checkout is kept at `/root/yttrx-welcomebot.pre-cleanup-20260706.tar.gz` on the box as a one-time rollback net. |
| `mammut` (`ssh mammut`) | The Mastodon instance. `tootctl`/Rails run inside the `live-web-1` container. Bot accounts + tokens live here. |

Full ops history is in secondbrain `yttrx-documentation/changelog.md`; the
user-memory note `project_welcomebot` has the current state.

## HARD RULES

- **Never overwrite the `.env` on `admin.yttrx.com`.** It holds the live
  `WEBHOOK_SECRET`, `BOT_ACCESS_TOKEN`, and `ABUSE_BOT_TOKEN`. Edit it in place
  for new vars.
- **Never commit secrets.** `.env` is git- and docker-ignored on the workstation
  repo. Tokens/passwords must not land in this repo, the changelog, or
  CLAUDE.md.
- The bot **silences** (reversible), it does not suspend by default
  (`ABUSE_ACTION=silence`). It applies the action **without** `report_id` so the
  report stays open for human review.
- **The blocklist API must stay classification-free.** `/api/blocks` is
  public and unauthenticated; entries may say a range/domain is blocked and
  when, never *why*. Don't add the ipapi.is verdict, the org name, or
  check-mail.org's disposable/risk fields back into the response — that would
  be redistributing a paid reputation feed's data. Account identifiers
  (handles, ids, emails, raw signup IPs) are out for the usual reasons.
- **The reputation API must never serve unauthenticated.** It installs IP
  blocks; an open endpoint that does that is a denial-of-service lever. An
  empty `REPUTATION_API_KEY` disables the route rather than opening it, and
  that behaviour is asserted in `test_local.py` — don't "fix" it into a
  permissive default.
- **Don't remove the block-width cap or the block expiry.** `IP_SCRUTINY_MAX_
  BLOCK_PREFIX_V4`/`_V6` (24/48) and `IP_SCRUTINY_IPBLOCK_EXPIRES_IN` (24h)
  exist because ipapi.is reports whole ASN routes: without the cap a single
  `abuser` flag on a consumer address blocks its entire ISP route (routes of
  /8–/11 are routine, and `197.211.32.0/19` was already registered against a
  Benin mobile carrier). Raising them is a policy decision, not a tuning knob.
- **`admin.yttrx.com`'s `/root/yttrx-welcomebot/` is not a git repo any more**
  (cleaned up 2026-07-06). Don't `git init`/clone it back into place — the
  registry image + `docker-compose.yml`/`.env` is the whole deploy surface now.
  All source lives in the workstation repo / Gitea only.

## Deploy procedure (workstation → admin.yttrx.com)

```bash
# 1. Test locally (offline, no network)
cd ~/yttrx-welcomebot
python3 -m venv .venv && . .venv/bin/activate && pip install -q -r requirements.txt
WEBHOOK_SECRET=testsecret BOT_ACCESS_TOKEN=x python test_local.py   # -> ALL TESTS PASSED

# 2. Commit + push to origin (git.blairhaus.net) — this fires
#    .gitea/workflows/docker-build-push.yml, which builds and pushes
#    gitea.blairhaus.net/pmb/yttrx-welcomebot:latest + :<sha>
git add -A && git commit -m "..."
git push origin main

# 3. Wait for the CI run to succeed (Gitea UI, or the Actions API), then pull
#    the freshly-built image on admin.yttrx.com and restart. No git anywhere
#    on this box any more — it's just docker-compose.yml + .env. Named volume
#    welcomebot-data persists the dedup db across this.
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && docker-compose pull && docker-compose up -d'

# 4. If new env vars were added this release, set them IN PLACE on the box, e.g.
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && \
  grep -q "^NEW_VAR=" .env || echo "NEW_VAR=value" >> .env'
```

Rollback: revert/checkout the previous commit on the workstation repo and push
(or manually retag/push an older image to `:latest`), wait for CI, then re-run
step 3. There's no local `.bak` source tree on admin.yttrx.com to fall back to
any more — the workstation repo / Gitea history is the only source of truth.
`.env.bak-*` snapshots from past deploys remain on the box for env-var
rollback specifically. A full tar of the pre-cleanup git checkout is at
`/root/yttrx-welcomebot.pre-cleanup-20260706.tar.gz` if source-level history
from that tree is ever needed (it shouldn't be — everything in it is also in
git history).

### Verify

```bash
ssh admin.yttrx.com 'curl -s localhost:8087/healthz'          # {"ok":true}
ssh admin.yttrx.com 'docker logs --tail 20 yttrx-welcomebot'  # clean startup
# confirm the running image + config:
ssh admin.yttrx.com 'docker exec yttrx-welcomebot python -c \
  "import app.main as m; print(m.ABUSE_DRY_RUN, m.ABUSE_ACTION, sorted(m.ABUSE_ALLOWLIST))"'
```

### Viewing logs

A `welcomebot-logs` CLI lives at `/root/bin/welcomebot-logs` on admin.yttrx.com
(source: `bin/welcomebot-logs` in this repo) — `/root/bin` is the homelab's
standard location for root-owned admin scripts (same convention as `bsd`'s
`peteftw-analytics` and the NAS backup scripts). A symlink at
`/usr/local/bin/welcomebot-logs` makes the bare command work over
non-interactive `ssh admin.yttrx.com welcomebot-logs`, since that doesn't
source `~/.bashrc` and so wouldn't otherwise see `/root/bin` on `PATH`. It
wraps `docker logs` for the container:

```bash
ssh admin.yttrx.com welcomebot-logs                 # last 200 lines
ssh admin.yttrx.com welcomebot-logs -f --abuse      # follow abuse activity only
ssh admin.yttrx.com welcomebot-logs -n 1000 --since 24h
ssh admin.yttrx.com welcomebot-logs --ip --since 24h  # signup-IP scrutiny decisions
ssh admin.yttrx.com 'welcomebot-logs --help'
```

### Inspecting signup-IP classification history

`welcomebot-signups` (`/root/bin/welcomebot-signups`, symlinked from
`/usr/local/bin` same as above; source `bin/welcomebot-signups`) reads the
persistent `signup_ip` sqlite table directly — a permanent record of every
classification decision, independent of log retention:

```bash
ssh admin.yttrx.com welcomebot-signups              # last 20 decisions
ssh admin.yttrx.com welcomebot-signups --flagged    # only datacenter-flagged signups
ssh admin.yttrx.com welcomebot-signups -a someuser
ssh admin.yttrx.com welcomebot-signups -n all
```

Both CLIs are tracked in this repo (`bin/welcomebot-logs`, `bin/welcomebot-signups`)
but **not** deployed via git any more — `admin.yttrx.com` has no checkout of this
repo since the 2026-07-06 cleanup. After editing either one, reinstall by copying
directly from the workstation:

```bash
scp bin/welcomebot-logs bin/welcomebot-signups admin.yttrx.com:/root/bin/
ssh admin.yttrx.com 'chmod 755 /root/bin/welcomebot-logs /root/bin/welcomebot-signups'
# /usr/local/bin/welcomebot-{logs,signups} are symlinks to /root/bin — no
# need to touch them again once created.
```

### Rollback

There's no `app/main.py.bak-*` on `admin.yttrx.com` any more — the deploy
directory holds only `docker-compose.yml` + `.env`. To roll back the app code,
revert/retag on the workstation repo, get a rolled-back image into the registry
(push an older commit and let CI rebuild, or manually `docker build`/`push` an
old checkout), then:

```bash
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && docker-compose pull && docker-compose up -d'
```

For `.env`-only rollback, `.env.bak-*` snapshots remain on the box:

```bash
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && \
  cp .env.bak-YYYYMMDD .env && docker-compose up -d'
```

## Rollout safety: dry-run

`ABUSE_DRY_RUN=true` makes the abuse handler log + DM what it *would* do without
silencing anyone. The shipped `.env.example` default is `true`; **production is
currently `false` (live and acting)**. To re-enter dry-run:

```bash
ssh admin.yttrx.com 'cd /root/yttrx-welcomebot && \
  sed -i "s/^ABUSE_DRY_RUN=.*/ABUSE_DRY_RUN=true/" .env && docker-compose up -d'
```

## The moderator bot token (gotcha)

The abuse handler posts to `POST /api/v1/admin/accounts/:id/action`, which needs
the **`admin:write:accounts`** OAuth scope — *separate* from `admin:write:reports`.
The bot account (`bot`) is an **Admin** (so it has the role permissions), but a
token created in the UI may omit `admin:write:accounts` → silence returns `403`.

**IP-scrutiny's `register_ip_block` needs an additional scope,
`admin:write:ip_blocks`**, to `POST /api/v1/admin/ip_blocks` — mint (or re-mint)
the token with it included, or `IP_SCRUTINY_AUTO_IPBLOCK` writes will 403 (logged
as an error; nothing else in the bot is affected).

**Email-domain scrutiny's `register_email_domain_block` needs another scope,
`admin:write:email_domain_blocks`**, to `POST /api/v1/admin/email_domain_blocks`
— same deal: mint (or re-mint) the token with it included, or
`CHECK_MAIL_AUTO_DOMAIN_BLOCK` writes (both the signup-time path and the
report-triggered suspend override) will 403 (logged as an error; the suspend
itself still goes through since that only needs `admin:write:accounts`).

**The blocklist reconcile (`app/reconcile_blocks.py`) needs two more scopes,
`admin:read:ip_blocks` and `admin:read:email_domain_blocks`**, to `GET` the
live `/api/v1/admin/ip_blocks` and `/api/v1/admin/email_domain_blocks` lists
and check whether the blocks we registered are still there. Added to the live
token **in place** on 2026-09-13 (`Doorkeeper::AccessToken#update!(scopes:)`
on id 11334, matched by SHA-256 digest against admin's `.env`) rather than by
minting and rotating — the token value is unchanged, so no `.env` edit was
needed. Without these scopes the reconcile 403s and exits having changed
nothing, which is the deliberate failure mode: never read an outage as "every
block was removed".

The reputation API needs no scopes beyond those above — it registers the same
`Admin::IpBlock` / `EmailDomainBlock` records as the webhook path, so
`admin:write:ip_blocks` + `admin:write:email_domain_blocks` cover it.

**The suspicious-signup sweep (`app/suspicious_sweep.py`) needs another scope,
`admin:read:accounts`**, to `GET /api/v1/admin/accounts/:id` (fetches current
post/follow counts + role/suspended state to decide whether to suspend) —
without it every sweep run 403s per-account (logged as an error, that account
is simply retried next run; nothing else in the bot is affected).

Mint a correctly-scoped token from the Rails console on mammut:

```bash
ssh mammut 'docker exec $(docker ps -f name=live-web-1 -q) bin/rails runner "
acct = Account.find_local(%q{bot})
scopes = %q{read:statuses write:statuses admin:read:reports admin:write:accounts admin:write:ip_blocks admin:write:email_domain_blocks admin:read:accounts admin:read:ip_blocks admin:read:email_domain_blocks}
app = Doorkeeper::Application.create!(name: %q{Abuse handler vN}, scopes: scopes, redirect_uri: %q{urn:ietf:wg:oauth:2.0:oob})
tok = Doorkeeper::AccessToken.create!(application_id: app.id, resource_owner_id: acct.user.id, scopes: scopes)
puts tok.token
"'
```

Put the result in `ABUSE_BOT_TOKEN` on the box's `.env`, restart, and revoke the
old token (`Doorkeeper::AccessToken.find(<id>).revoke`).

**Harmless write-permission probe** (no real account touched — `403` = missing
scope, `404` = authorized):

```bash
curl -s -o /dev/null -w "%{http_code}\n" -X POST \
  -H "Authorization: Bearer $ABUSE_BOT_TOKEN" -d "type=none" \
  https://yttrx.com/api/v1/admin/accounts/0/action
```

Same idea for the `admin:write:ip_blocks` scope IP-scrutiny needs (`403` =
missing scope; `422` = authorized, just missing/invalid params — no block is
created either way):

```bash
curl -s -o /dev/null -w "%{http_code}\n" -X POST \
  -H "Authorization: Bearer $ABUSE_BOT_TOKEN" \
  https://yttrx.com/api/v1/admin/ip_blocks
```

And for `admin:write:email_domain_blocks` (`403` = missing scope; `422` =
authorized, just missing/invalid params):

```bash
curl -s -o /dev/null -w "%{http_code}\n" -X POST \
  -H "Authorization: Bearer $ABUSE_BOT_TOKEN" \
  https://yttrx.com/api/v1/admin/email_domain_blocks
```

And for `admin:read:accounts` (`403` = missing scope; `200`/`404` = authorized):

```bash
curl -s -o /dev/null -w "%{http_code}\n" \
  -H "Authorization: Bearer $ABUSE_BOT_TOKEN" \
  https://yttrx.com/api/v1/admin/accounts/0
```

## Mastodon side (mammut)

- The webhook (Administration → Webhooks) is already subscribed to
  `account.created`, `account.approved`, `report.created` — no change needed for
  code redeploys.
- Staff are never auto-silenced: `ABUSE_SKIP_PRIVILEGED=true` reads the payload
  `target_account.role` (skips any assigned staff role, id > 0), plus the
  explicit `ABUSE_ALLOWLIST`. Current staff: `waffles`/`tommertron` (Owner),
  `davis`/`bot` (Admin).

## Appeals page

Silenced users are DMed `ABUSE_HELP_URL` =
`https://welcome.yttrx.com/posts/account-limited/`. That page is a Hugo
(Compost) content file at `admin.yttrx.com:/var/www/html/welcome/content/posts/`,
rebuilt with `./build.sh` in that dir (see secondbrain `misc-sites.md`).