Add a reputation query API for pre-signup blocking
docker-build-push / build-push (push) Successful in 6s

Everything the bot does today is reactive: Mastodon creates the account,
the account.created webhook arrives, we classify the signup IP and email
domain, and then we block the range and suspend what already exists.

POST /api/reputation is the pre-emptive half. A caller asks about an IP
and/or email domain; we classify it, record the answer, register the
matching Admin::IpBlock / EmailDomainBlock, and only then return. A
caller that runs its own block lookup immediately afterwards therefore
finds a block that did not exist when its request started. The intended
caller is Mastodon's signup path, which does exactly that.

It writes, so unlike /api/blocks it is authenticated and refuses to serve
without a key (REPUTATION_API_KEY): an unauthenticated endpoint that
installs IP blocks is a denial-of-service lever, so there is no open mode.

Notes on the pieces that aren't obvious:

- REPUTATION_API_MAX_LOOKUPS_PER_HOUR caps *fresh* provider lookups. The
  caller hits this on every view of the signup form, not just on
  submission, so without a ceiling a crawler walking /auth/sign_up from a
  new range each time would set our ipapi.is bill. Over budget we answer
  "unknown" and install nothing -- never "flagged". Cache hits are free
  and uncounted, and the webhook path never spends from this budget.

- Blocks installed here are recorded in their own table, keyed by the
  CIDR/domain rather than by account (there is no account yet), and are
  folded into /api/blocks and the reconcile pass so neither one
  under-reports as blocking shifts to the pre-signup path. The published
  entry shape is unchanged; a range blocked before anyone signed up from
  it simply reads "signups": 0.

- A repeat query for an already-installed block skips the Mastodon round
  trip. This runs on a live signup request, so that is saved latency.
This commit is contained in:
pmb committed 2026-09-13 16:02:26 -07:00
1 parent 8490048d2d
commit 57234ca548
5 files changed
+758 -4

No files matched your search

+45
View File
@@ -12,6 +12,10 @@ This pass closes that gap: it fetches Mastodon's live `Admin::IpBlock` and
rows whose block is gone (or has expired), back on for any that reappeared.
The API then serves only entries that are actually in force.
It covers both sources of blocks: the signup-derived ones (`signup_ip` /
`signup_email`, keyed by account) and the pre-signup ones the reputation query
API installs (`reputation_block`, keyed by the blocked CIDR/domain itself).
Run on a schedule (cron on admin.yttrx.com, see CLAUDE.md):
docker exec yttrx-welcomebot python -m app.reconcile_blocks
@@ -150,6 +154,8 @@ def reconcile(dry_run: bool = False) -> int:
"UPDATE signup_ip SET block_active = ?, block_checked_at = ? "
"WHERE account_id = ?", updates)
log.info("ip_blocks: %d live, %d bot rows checked", len(live_ips), len(rows))
changed += _reconcile_reputation(
"ip", live_ips, now, dry_run, normalise=True)
if live_domains is not None:
with _db() as conn:
@@ -175,11 +181,50 @@ def reconcile(dry_run: bool = False) -> int:
"WHERE account_id = ?", updates)
log.info("email_domain_blocks: %d live, %d bot rows checked",
len(live_domains), len(rows))
changed += _reconcile_reputation(
"email", live_domains, now, dry_run, normalise=False)
log.info("%s%d row(s) changed state", "[DRY-RUN] " if dry_run else "", changed)
return changed
def _reconcile_reputation(kind: str, live: set[str], now: str, dry_run: bool,
normalise: bool) -> int:
"""Same reconcile, for the blocks the query API installed pre-signup.
Those live in `reputation_block` keyed by what is actually blocked (a CIDR
or a domain) rather than by account, since there is no account yet when
they're written — but they are the same Admin::IpBlock /
EmailDomainBlock records, so they go stale the same ways and /api/blocks
publishes them alongside the signup-derived ones.
"""
with _db() as conn:
rows = conn.execute(
"SELECT target, block_active FROM reputation_block WHERE kind = ?",
(kind,),
).fetchall()
changed = 0
updates = []
for target, active in rows:
key = _normalise(target) if normalise else (target or "").strip().lower()
present = bool(target) and key in live
if bool(active) != present:
changed += 1
log.info("reputation %s block %s: %s -> %s", kind, target,
"active" if active else "inactive",
"active" if present else "inactive")
updates.append((int(present), now, kind, target))
if not dry_run and updates:
with _db() as conn:
conn.executemany(
"UPDATE reputation_block SET block_active = ?, block_checked_at = ? "
"WHERE kind = ? AND target = ?", updates)
log.info("reputation %s blocks: %d rows checked", kind, len(rows))
return changed
def _single_cidr(ip: str) -> str:
try:
addr = ipaddress.ip_address(ip)