Add welcomebot-reputation CLI and a --reputation log filter

The pre-signup path had no inspection tool. Its activity landed in docker logs and two sqlite tables with nothing to read them, while the reactive path has had welcomebot-signups and welcomebot-suspicious all along. welcomebot-reputation reads reputation_query and reputation_block directly so it survives log rotation: recent answers, --stats for volume and flag/block rate by source, --blocks for what is actually in force, and --deferred to see whether the hourly provider-lookup ceiling is biting. --blocks marks rows GONE once the reconcile pass finds the block no longer on the Mastodon side, so "we installed this" and "this is in force" stay distinguishable. welcomebot-logs --reputation covers the raw lines (pre-blocks, budget refusals, width-cap narrowings).
2026-09-13 16:35:11 -07:00
1 parent 47e92ca10f
commit 1bfbcfdac3
@@ -154,6 +154,23 @@ ssh admin.yttrx.com welcomebot-logs --ip --since 24h  # signup-IP scrutiny decis
 ssh admin.yttrx.com 'welcomebot-logs --help'
 ```

+### Inspecting the pre-signup reputation API
+
+`welcomebot-reputation` (`/root/bin/welcomebot-reputation`, symlinked from
+`/usr/local/bin`; source `bin/welcomebot-reputation`) reads the
+`reputation_query` + `reputation_block` tables — the pre-emptive path, as
+opposed to `welcomebot-signups`' reactive one:
+
+```bash
+ssh admin.yttrx.com welcomebot-reputation --stats    # is it earning its keep?
+ssh admin.yttrx.com welcomebot-reputation --blocks   # what is in force (GONE = reconciled away)
+ssh admin.yttrx.com welcomebot-reputation --deferred # budget ceiling refusals
+ssh admin.yttrx.com 'welcomebot-reputation --help'
+```
+
+`welcomebot-logs --reputation` gives the raw log lines instead. On mammut, the
+caller patch logs as `WelcomebotReputation` in `docker logs live-web-1`.
+
 ### Inspecting signup-IP classification history

 `welcomebot-signups` (`/root/bin/welcomebot-signups`, symlinked from
@@ -174,8 +191,10 @@ repo since the 2026-07-06 cleanup. After editing either one, reinstall by copyin
 directly from the workstation:

 ```bash
-scp bin/welcomebot-logs bin/welcomebot-signups admin.yttrx.com:/root/bin/
-ssh admin.yttrx.com 'chmod 755 /root/bin/welcomebot-logs /root/bin/welcomebot-signups'
+scp bin/welcomebot-logs bin/welcomebot-signups bin/welcomebot-reputation \
+  admin.yttrx.com:/root/bin/
+ssh admin.yttrx.com 'chmod 755 /root/bin/welcomebot-logs /root/bin/welcomebot-signups \
+  /root/bin/welcomebot-reputation'
 # /usr/local/bin/welcomebot-{logs,signups} are symlinks to /root/bin — no
 # need to touch them again once created.
 ```
@@ -439,6 +439,37 @@ rows, so neither under-reports as blocking shifts to the pre-signup path. The
 published entry shape is unchanged; `signups` still counts signups, so a range
 blocked before anyone signed up from it reads `"signups": 0`.

+### Watching it
+
+`welcomebot-reputation` (installed at `/root/bin` on admin.yttrx.com) reads
+both tables directly, so it survives log rotation:
+
+```bash
+welcomebot-reputation                    # last 20 answers
+welcomebot-reputation --stats            # volume, flag/block rate, by source
+welcomebot-reputation --blocks           # what it currently has blocked
+welcomebot-reputation --blocked --since '-24 hours'
+welcomebot-reputation --deferred         # did the hourly budget ceiling bite?
+welcomebot-reputation --ip | --email     # one side only
+```
+
+`--blocks` marks a row `GONE` once the reconcile pass finds the block no
+longer on the Mastodon side (moderator cleared it, or it expired), so the
+difference between "we installed this" and "this is in force" stays visible.
+
+For the raw log lines rather than the tables, `welcomebot-logs --reputation`
+filters to pre-blocks, budget refusals and width-cap narrowings:
+
+```bash
+welcomebot-logs --reputation -f          # follow pre-signup blocking live
+welcomebot-logs --reputation --since 24h
+```
+
+On the Mastodon side the patch logs one line per pre-block at `info`
+(`WelcomebotReputation: pre-blocked signup attempt`), plus a warning for every
+failed or timed-out lookup — `docker logs live-web-1 | grep WelcomebotReputation`
+on mammut.
+
 ## Layout

 | Path | Purpose |
@@ -449,7 +480,7 @@ blocked before anyone signed up from it reads `"signups": 0`.
 | `Dockerfile`, `docker-compose.yml` | Container build + run |
 | `.env.example` | Config template (copy to `.env`, never commit) |
 | `nginx/hooks.yttrx.com.conf` | nginx site for admin.yttrx.com (webhook, healthz, blocklist API, reputation API) |
-| `bin/welcomebot-logs`, `bin/welcomebot-signups`, `bin/welcomebot-suspicious` | Ops CLIs installed to `/root/bin` on admin.yttrx.com |
+| `bin/welcomebot-logs`, `bin/welcomebot-signups`, `bin/welcomebot-suspicious`, `bin/welcomebot-reputation` | Ops CLIs installed to `/root/bin` on admin.yttrx.com |
 | `bin/yttrx-blocklist-import` | Importer other admins can run to pull this list into their own instance |
 | `bin/backfill-block-cidr.py` | One-shot: reconstruct `block_cidr` for rows predating it (run 2026-09-13) |
 | `test_local.py` | Offline smoke test (no network) |
@@ -25,6 +25,7 @@ Options:
      --abuse          Only abuse-handler lines (reports, silences, dry-run, tiers).
      --welcome        Only welcome lines.
      --ip             Only IP-scrutiny lines (classifications, ip_block writes, ipapi.is errors).
+      --reputation     Only pre-signup reputation-API lines (pre-blocks, budget refusals).
  -g, --grep PATTERN   Only lines matching PATTERN (extended regex).
  -h, --help           Show this help.

@@ -34,6 +35,7 @@ Examples:
  welcomebot-logs -n 1000 --since 24h
  welcomebot-logs -g 'acct=spammer'
  welcomebot-logs --ip --since 24h # recent signup-IP classification decisions
+  welcomebot-logs --reputation -f  # follow pre-signup blocking as it happens
 EOF
 }

@@ -45,6 +47,7 @@ while [ $# -gt 0 ]; do
    --abuse)     pattern='welcomebot (report |auto-|\[DRY-RUN\]|tier=|silenc|holds a staff|allowlisted)' ;;
    --welcome)   pattern='welcomebot (welcomed|already welcomed|skipping non-local|queued)' ;;
    --ip)        pattern='welcomebot (flagged signup|ipapi\.is lookup failed|ip_block|unparseable ip)' ;;
+    --reputation) pattern='welcomebot (reputation query|block for ip=.*narrowed)' ;;
    -g|--grep)   shift; pattern="${1:?--grep needs a value}" ;;
    -h|--help)   usage; exit 0 ;;
    *) echo "welcomebot-logs: unknown option: $1" >&2; usage >&2; exit 2 ;;
@@ -0,0 +1,200 @@
+#!/usr/bin/env bash
+# welcomebot-reputation — inspect the pre-signup reputation query API's activity.
+#
+# Installed at /usr/local/bin/welcomebot-reputation on admin.yttrx.com.
+# Source of truth: ~/yttrx-welcomebot/bin/welcomebot-reputation (workstation).
+#
+# Reads the persistent `reputation_query` (every answer the API gave) and
+# `reputation_block` (every block it installed) sqlite tables, so this survives
+# log rotation — see app/main.py's reputation_check / README's "Reputation
+# query API". For the raw log lines instead, use `welcomebot-logs --reputation`.
+set -euo pipefail
+
+CONTAINER="${WELCOMEBOT_CONTAINER:-yttrx-welcomebot}"
+limit=20
+mode="queries"
+filter="all"
+kind_filter=""
+since=""
+
+usage() {
+  cat <<EOF
+Usage: welcomebot-reputation [options]
+
+Show what the pre-signup reputation API has been asked and what it blocked,
+newest first. This is the pre-emptive path (Mastodon asking before it creates
+an account); for the reactive signup path use welcomebot-signups.
+
+Options:
+  -n, --limit N     Show the last N rows (default: $limit; 'all' for everything).
+      --blocks      List installed blocks instead of queries (what is in force).
+      --stats       Summary: volume, flag/block rate, cache hit rate, deferrals.
+      --flagged     Queries only: just the ones that came back flagged.
+      --blocked     Queries only: just the ones that installed or hit a block.
+      --deferred    Queries only: just the ones refused by the hourly budget.
+      --ip          Only the IP side.
+      --email       Only the email-domain side.
+      --since WHEN  Only rows newer than WHEN (SQL datetime, e.g. '2026-09-13'
+                    or '-24 hours').
+  -h, --help        Show this help.
+
+Examples:
+  welcomebot-reputation                    # last $limit answers
+  welcomebot-reputation --stats            # is it earning its keep?
+  welcomebot-reputation --blocks           # what it currently has blocked
+  welcomebot-reputation --blocked --since '-24 hours'
+  welcomebot-reputation --deferred         # did the budget ceiling bite?
+EOF
+}
+
+while [ $# -gt 0 ]; do
+  case "$1" in
+    -n|--limit) shift; limit="${1:?--limit needs a value}" ;;
+    --blocks)   mode="blocks" ;;
+    --stats)    mode="stats" ;;
+    --flagged)  filter="flagged" ;;
+    --blocked)  filter="blocked" ;;
+    --deferred) filter="deferred" ;;
+    --ip)       kind_filter="ip" ;;
+    --email)    kind_filter="email" ;;
+    --since)    shift; since="${1:?--since needs a value}" ;;
+    -h|--help)  usage; exit 0 ;;
+    *) echo "welcomebot-reputation: unknown option: $1" >&2; usage >&2; exit 2 ;;
+  esac
+  shift
+done
+
+if ! command -v docker >/dev/null 2>&1; then
+  echo "welcomebot-reputation: docker not found in PATH" >&2; exit 1
+fi
+if ! docker inspect "$CONTAINER" >/dev/null 2>&1; then
+  echo "welcomebot-reputation: container '$CONTAINER' not found (is it deployed/running?)" >&2; exit 1
+fi
+
+docker exec -i \
+  -e REP_LIMIT="$limit" \
+  -e REP_MODE="$mode" \
+  -e REP_FILTER="$filter" \
+  -e REP_KIND="$kind_filter" \
+  -e REP_SINCE="$since" \
+  "$CONTAINER" python3 <<'PYEOF'
+import os
+import sqlite3
+
+conn = sqlite3.connect("/data/welcomed.db")
+mode = os.environ.get("REP_MODE", "queries")
+kind = os.environ.get("REP_KIND", "")
+since = os.environ.get("REP_SINCE", "")
+limit = os.environ.get("REP_LIMIT", "20")
+
+
+def limited(sql):
+    return sql if limit == "all" else f"{sql} LIMIT {int(limit)}"
+
+
+def tables_exist():
+    names = {r[0] for r in conn.execute(
+        "SELECT name FROM sqlite_master WHERE type = 'table'")}
+    return {"reputation_query", "reputation_block"} <= names
+
+
+if not tables_exist():
+    print("(no reputation tables yet — the query API has never been called on "
+          "this deployment)")
+    raise SystemExit(0)
+
+if mode == "stats":
+    where, params = [], []
+    if since:
+        where.append("queried_at >= datetime(?)")
+        params.append(since)
+    clause = (" WHERE " + " AND ".join(where)) if where else ""
+    row = conn.execute(
+        "SELECT COUNT(*),"
+        "       SUM(flagged), SUM(blocked),"
+        "       SUM(CASE WHEN classification = 'unknown' THEN 1 ELSE 0 END),"
+        "       MIN(queried_at), MAX(queried_at)"
+        f" FROM reputation_query{clause}", params).fetchone()
+    total, flagged, blocked, unknown, first, last = row
+    total = total or 0
+    if not total:
+        print("(no queries in range)")
+        raise SystemExit(0)
+
+    def pct(n):
+        return f"{(n or 0) / total * 100:5.1f}%"
+
+    print(f"queries          {total:>7}   {first} .. {last}")
+    print(f"  flagged        {flagged or 0:>7}   {pct(flagged)}")
+    print(f"  led to a block {blocked or 0:>7}   {pct(blocked)}")
+    print(f"  unknown        {unknown or 0:>7}   {pct(unknown)}  "
+          "(lookup failed, or refused by the hourly budget)")
+    print()
+    print("by source:")
+    for source, n, nb in conn.execute(
+            "SELECT COALESCE(source, '-'), COUNT(*), SUM(blocked) "
+            f"FROM reputation_query{clause} GROUP BY 1 ORDER BY 2 DESC", params):
+        print(f"  {source:24s} {n:>7} queries, {nb or 0:>5} blocked")
+    print()
+    print("blocks installed (still in force):")
+    for kind_, n in conn.execute(
+            "SELECT kind, COUNT(*) FROM reputation_block "
+            "WHERE block_active = 1 AND (expires_at IS NULL OR expires_at > "
+            "  strftime('%Y-%m-%dT%H:%M:%S+00:00', 'now')) GROUP BY 1"):
+        print(f"  {kind_:24s} {n:>7}")
+    raise SystemExit(0)
+
+if mode == "blocks":
+    where, params = [], []
+    if kind:
+        where.append("kind = ?")
+        params.append(kind)
+    if since:
+        where.append("first_blocked_at >= datetime(?)")
+        params.append(since)
+    clause = (" WHERE " + " AND ".join(where)) if where else ""
+    rows = conn.execute(limited(
+        "SELECT kind, target, classification, org, queries, first_blocked_at,"
+        "       last_seen_at, block_active, expires_at"
+        f" FROM reputation_block{clause} ORDER BY last_seen_at DESC"), params).fetchall()
+    if not rows:
+        print("(no blocks installed by the query API)")
+    for k, target, cls, org, queries, first, last, active, expires in rows:
+        state = "active" if active else "GONE"
+        exp = f"expires {expires}" if expires else "no expiry"
+        print(f"{last}  {k:6s} {target:22s} {state:7s} {exp:34s} "
+              f"hits={queries:<5} {cls or ''} {org or ''}".rstrip())
+    raise SystemExit(0)
+
+# mode == "queries"
+where, params = [], []
+filt = os.environ.get("REP_FILTER", "all")
+if filt == "flagged":
+    where.append("flagged = 1")
+elif filt == "blocked":
+    where.append("blocked = 1")
+elif filt == "deferred":
+    where.append("classification = 'unknown'")
+if kind:
+    where.append("kind = ?")
+    params.append(kind)
+if since:
+    where.append("queried_at >= datetime(?)")
+    params.append(since)
+clause = (" WHERE " + " AND ".join(where)) if where else ""
+
+rows = conn.execute(limited(
+    "SELECT queried_at, kind, value, classification, org, risk, flagged,"
+    "       target, blocked, source"
+    f" FROM reputation_query{clause} ORDER BY id DESC"), params).fetchall()
+if not rows:
+    print("(no matching reputation queries)")
+for ts, k, value, cls, org, risk, flagged, target, blocked, source in rows:
+    flag = "FLAGGED" if flagged else "-"
+    blk = "BLOCKED" if blocked else "-"
+    extra = org or ""
+    if k == "email" and risk:
+        extra = f"risk={risk}"
+    print(f"{ts}  {k:6s} {value or '':24s} {cls or '':22s} {flag:8s} {blk:8s} "
+          f"{target or '':22s} {source or '-':16s} {extra}".rstrip())
+PYEOF