Add disposable/high-risk email signup scrutiny via check-mail.org
Classifies each signup's email domain (domain-only, GDPR-friendly) alongside the existing IP scrutiny signal, with matching held-welcome and auto email_domain_block behavior. A report against an account with a flagged domain suspends immediately (no reporter-count threshold) and blocks the domain, classified live from the report payload rather than any signup-time record so it also covers pre-existing accounts. Ships CHECK_MAIL_DRY_RUN=true by default, independent of ABUSE_DRY_RUN, so the new report-triggered suspend path stays inert until watched.
This commit is contained in:
@@ -117,3 +117,31 @@ IP_SCRUTINY_IPBLOCK_SEVERITY=sign_up_requires_approval
|
||||
# informational only (mobile carriers are never scrutinized).
|
||||
IP_SCRUTINY_HOSTING_RE=amazon|aws|google|microsoft|azure|digitalocean|ovh|hetzner|linode|vultr|contabo|choopa|m247|scaleway|leaseweb|hostinger|namecheap|godaddy|cloudflare|oracle|alibaba|tencent|akamai|fastly|packet|vpn|proxy|hosting|datacenter|data center|colo(?:cation)?|server
|
||||
IP_SCRUTINY_MOBILE_RE=mobile|wireless|cellular|verizon|t-mobile|tmobile|at&t|att mobility|vodafone|telstra|sprint|three\b|orange mobile|deutsche telekom|o2\b
|
||||
|
||||
# check-mail.org disposable/high-risk email domain scrutiny (roadmap item B,
|
||||
# anti-abuse.md). Domain-only query (never the full email) against
|
||||
# POST https://api.check-mail.org/v2/, Authorization: Bearer <key>.
|
||||
# Free tier: 1,000 req/month.
|
||||
CHECK_MAIL_ENABLED=true
|
||||
CHECK_MAIL_API_KEY=
|
||||
|
||||
# Log/DM only — no held welcome, no email_domain_block write, no
|
||||
# report-triggered suspend. Keep true for rollout until you've watched the
|
||||
# false-positive rate for a while (same role as IP_SCRUTINY_DRY_RUN /
|
||||
# ABUSE_DRY_RUN).
|
||||
CHECK_MAIL_DRY_RUN=true
|
||||
|
||||
# A domain is flagged if check-mail.org marks it is_disposable OR its risk
|
||||
# score (0-100) is at or above this threshold.
|
||||
CHECK_MAIL_RISK_THRESHOLD=80
|
||||
|
||||
# Hold the welcome DM for a flagged signup until account.approved fires, same
|
||||
# semantics as IP_SCRUTINY_HOLD_WELCOME.
|
||||
CHECK_MAIL_HOLD_WELCOME=true
|
||||
|
||||
# Auto-register a flagged signup's email domain into Mastodon's native
|
||||
# Admin::EmailDomainBlock. Requires ABUSE_BOT_TOKEN to carry the
|
||||
# admin:write:email_domain_blocks scope (see CLAUDE.md) — without it this
|
||||
# 403s and is logged as an error, but nothing else in the bot is affected.
|
||||
# Also gates the report-triggered suspend+block override in handle_report().
|
||||
CHECK_MAIL_AUTO_DOMAIN_BLOCK=true
|
||||
|
||||
@@ -163,12 +163,19 @@ token created in the UI may omit `admin:write:accounts` → silence returns `403
|
||||
the token with it included, or `IP_SCRUTINY_AUTO_IPBLOCK` writes will 403 (logged
|
||||
as an error; nothing else in the bot is affected).
|
||||
|
||||
**Email-domain scrutiny's `register_email_domain_block` needs another scope,
|
||||
`admin:write:email_domain_blocks`**, to `POST /api/v1/admin/email_domain_blocks`
|
||||
— same deal: mint (or re-mint) the token with it included, or
|
||||
`CHECK_MAIL_AUTO_DOMAIN_BLOCK` writes (both the signup-time path and the
|
||||
report-triggered suspend override) will 403 (logged as an error; the suspend
|
||||
itself still goes through since that only needs `admin:write:accounts`).
|
||||
|
||||
Mint a correctly-scoped token from the Rails console on mammut:
|
||||
|
||||
```bash
|
||||
ssh mammut 'docker exec $(docker ps -f name=live-web-1 -q) bin/rails runner "
|
||||
acct = Account.find_local(%q{bot})
|
||||
scopes = %q{read:statuses write:statuses admin:read:reports admin:write:accounts admin:write:ip_blocks}
|
||||
scopes = %q{read:statuses write:statuses admin:read:reports admin:write:accounts admin:write:ip_blocks admin:write:email_domain_blocks}
|
||||
app = Doorkeeper::Application.create!(name: %q{Abuse handler vN}, scopes: scopes, redirect_uri: %q{urn:ietf:wg:oauth:2.0:oob})
|
||||
tok = Doorkeeper::AccessToken.create!(application_id: app.id, resource_owner_id: acct.user.id, scopes: scopes)
|
||||
puts tok.token
|
||||
@@ -197,6 +204,15 @@ curl -s -o /dev/null -w "%{http_code}\n" -X POST \
|
||||
https://yttrx.com/api/v1/admin/ip_blocks
|
||||
```
|
||||
|
||||
And for `admin:write:email_domain_blocks` (`403` = missing scope; `422` =
|
||||
authorized, just missing/invalid params):
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w "%{http_code}\n" -X POST \
|
||||
-H "Authorization: Bearer $ABUSE_BOT_TOKEN" \
|
||||
https://yttrx.com/api/v1/admin/email_domain_blocks
|
||||
```
|
||||
|
||||
## Mastodon side (mammut)
|
||||
|
||||
- The webhook (Administration → Webhooks) is already subscribed to
|
||||
|
||||
@@ -9,7 +9,9 @@ A small FastAPI webhook server that receives Mastodon **admin webhooks** for
|
||||
- **`report.created`** — evaluates the reported account and, when enough
|
||||
**distinct** reporters have open reports against a *young* or *dormant*
|
||||
account, auto-**silences** it (reversible) and DMs a moderator for review
|
||||
(the abuse bot).
|
||||
(the abuse bot). Special case: a report against an account whose email
|
||||
domain is flagged high-risk (see below) auto-**suspends** it immediately
|
||||
(no reporter-count threshold) and blocks the domain.
|
||||
|
||||
- **Runs on:** `admin.yttrx.com` (Docker container, nginx-proxied at
|
||||
`https://hooks.yttrx.com`)
|
||||
@@ -85,6 +87,43 @@ regexes for a while. Registering ip_blocks requires the `ABUSE_BOT_TOKEN` to
|
||||
carry the `admin:write:ip_blocks` scope in addition to its existing scopes
|
||||
(see `CLAUDE.md`).
|
||||
|
||||
## Disposable/high-risk email signup scrutiny
|
||||
|
||||
Every `account.created` delivery already carries the signup email for free
|
||||
(`Admin::Account.email`). On each new local signup, the bot:
|
||||
|
||||
1. **Classifies the domain only** (never the full email address — GDPR-
|
||||
friendly) via [check-mail.org](https://check-mail.org/), cached in sqlite.
|
||||
Flagged if the API marks it `is_disposable` or its `risk` score (0-100) is
|
||||
at or above `CHECK_MAIL_RISK_THRESHOLD` (default 80).
|
||||
2. If flagged, same scrutiny shape as the IP signal:
|
||||
- **Moderator DM** with the domain and risk score.
|
||||
- **Held welcome** (`CHECK_MAIL_HOLD_WELCOME`) — same semantics as
|
||||
`IP_SCRUTINY_HOLD_WELCOME`.
|
||||
- **Auto-registered email domain block** (`CHECK_MAIL_AUTO_DOMAIN_BLOCK`)
|
||||
— the domain is added to Mastodon's native `Admin::EmailDomainBlock`,
|
||||
blocking future signups from it.
|
||||
|
||||
A **report** against an account whose email domain is flagged is a special,
|
||||
stronger case (not merely a lowered threshold like the IP signal): it
|
||||
auto-**suspends** the account on the very first report — no distinct-reporter
|
||||
threshold — and (re-)registers the email domain block. This is classified
|
||||
**live** from the report payload's `target_account.email` (a full
|
||||
`Admin::Account`, always present), not from any record made at signup time —
|
||||
so it also catches accounts that signed up before this feature existed or
|
||||
while `CHECK_MAIL_ENABLED`/`CHECK_MAIL_API_KEY` was off. Gated by
|
||||
`ABUSE_DRY_RUN` **or** `CHECK_MAIL_DRY_RUN` (either one holds it) —
|
||||
deliberately independent of `ABUSE_DRY_RUN` alone, so this brand-new action
|
||||
ships inert by `CHECK_MAIL_DRY_RUN`'s own default even once the general
|
||||
abuse-bot is live.
|
||||
|
||||
`CHECK_MAIL_DRY_RUN=true` (the shipped default) classifies and DMs a
|
||||
moderator without holding any welcome or writing any email_domain_block —
|
||||
keep it on until you've watched the false-positive rate for a while.
|
||||
Registering email_domain_blocks requires the `ABUSE_BOT_TOKEN` to carry the
|
||||
`admin:write:email_domain_blocks` scope in addition to its existing scopes
|
||||
(see `CLAUDE.md`).
|
||||
|
||||
## Layout
|
||||
|
||||
| Path | Purpose |
|
||||
@@ -127,6 +166,12 @@ Copy `.env.example` to `.env` and fill in:
|
||||
| `IP_SCRUTINY_AUTO_IPBLOCK` | Auto-register a flagged IP into Mastodon's `Admin::IpBlock` |
|
||||
| `IP_SCRUTINY_IPBLOCK_SEVERITY` | `sign_up_requires_approval` (default), `sign_up_block`, or `no_access` |
|
||||
| `IP_SCRUTINY_HOSTING_RE` / `IP_SCRUTINY_MOBILE_RE` | Keyword regexes matched against the RDAP org/ASN description |
|
||||
| `CHECK_MAIL_ENABLED` | Master switch for disposable/high-risk email signup scrutiny |
|
||||
| `CHECK_MAIL_API_KEY` | check-mail.org API key; blank disables the check |
|
||||
| `CHECK_MAIL_DRY_RUN` | `true` — classify + DM only, no held welcome, no email_domain_block write, no report-triggered suspend |
|
||||
| `CHECK_MAIL_RISK_THRESHOLD` | Flag if `is_disposable` or `risk` ≥ this (default 80) |
|
||||
| `CHECK_MAIL_HOLD_WELCOME` | `true` — hold the welcome for a flagged signup until `account.approved` |
|
||||
| `CHECK_MAIL_AUTO_DOMAIN_BLOCK` | Auto-register a flagged domain into Mastodon's `Admin::EmailDomainBlock`; also gates the report-triggered override |
|
||||
|
||||
## Local test
|
||||
|
||||
|
||||
+256
-28
@@ -7,10 +7,16 @@ and dispatches by event:
|
||||
account a welcome toot via the welcome bot's token (scope ``write:statuses``).
|
||||
Both events are handled so the welcome works whether or not registration
|
||||
approval is enabled; the dedup store welcomes each account exactly once.
|
||||
``account.created`` also classifies the signup's IP (datacenter/hosting via
|
||||
RDAP) and email domain (disposable/high-risk via check-mail.org); a flagged
|
||||
signup gets more scrutiny (held welcome, auto ip_block/email_domain_block,
|
||||
moderator DM) before falling through to a normal welcome.
|
||||
* ``report.created`` — evaluates the reported account and, when enough
|
||||
*distinct* reporters have open reports against a young or dormant account,
|
||||
auto-**silences** it (reversible) and DMs a moderator for review. Uses a
|
||||
separate moderator bot token with admin scopes.
|
||||
separate moderator bot token with admin scopes. Special case: a report
|
||||
against an account whose email domain is high-risk auto-**suspends** it
|
||||
immediately (no reporter-count threshold) and blocks the domain.
|
||||
|
||||
Mastodon signs every delivery with:
|
||||
|
||||
@@ -145,12 +151,33 @@ IP_SCRUTINY_MOBILE_RE = re.compile(os.environ.get(
|
||||
r"vodafone|telstra|sprint|three\b|orange mobile|deutsche telekom|o2\b",
|
||||
), re.I)
|
||||
|
||||
# --- Disposable/high-risk email signup scrutiny (check-mail.org) -----------
|
||||
# Every account.created delivery already carries the signup email for free
|
||||
# (Admin::Account.email). On each new local signup, the bot classifies the
|
||||
# *domain only* (never the full address) via check-mail.org.
|
||||
CHECK_MAIL_ENABLED = os.environ.get("CHECK_MAIL_ENABLED", "true").lower() in ("1", "true", "yes")
|
||||
CHECK_MAIL_API_KEY = os.environ.get("CHECK_MAIL_API_KEY", "")
|
||||
# Log/DM only — no held welcome, no email_domain_block write. Same rollout-
|
||||
# safety role as IP_SCRUTINY_DRY_RUN.
|
||||
CHECK_MAIL_DRY_RUN = os.environ.get("CHECK_MAIL_DRY_RUN", "true").lower() in ("1", "true", "yes")
|
||||
# A domain is flagged if check-mail.org marks it is_disposable OR its risk
|
||||
# score (0-100) is at or above this threshold.
|
||||
CHECK_MAIL_RISK_THRESHOLD = int(os.environ.get("CHECK_MAIL_RISK_THRESHOLD", "80"))
|
||||
# Hold the welcome DM for a flagged signup until account.approved fires, same
|
||||
# semantics as IP_SCRUTINY_HOLD_WELCOME.
|
||||
CHECK_MAIL_HOLD_WELCOME = os.environ.get("CHECK_MAIL_HOLD_WELCOME", "true").lower() in ("1", "true", "yes")
|
||||
# Auto-register a flagged signup's email domain into Mastodon's native
|
||||
# Admin::EmailDomainBlock (blocks future signups from that domain).
|
||||
CHECK_MAIL_AUTO_DOMAIN_BLOCK = os.environ.get("CHECK_MAIL_AUTO_DOMAIN_BLOCK", "true").lower() in ("1", "true", "yes")
|
||||
|
||||
if not WEBHOOK_SECRET:
|
||||
log.warning("WEBHOOK_SECRET is empty — signature verification will reject all requests.")
|
||||
if not BOT_ACCESS_TOKEN:
|
||||
log.warning("BOT_ACCESS_TOKEN is empty — the bot cannot post welcome messages.")
|
||||
if ABUSE_ENABLED and not ABUSE_BOT_TOKEN:
|
||||
log.warning("ABUSE_BOT_TOKEN is empty — report.created handling is disabled.")
|
||||
if CHECK_MAIL_ENABLED and not CHECK_MAIL_API_KEY:
|
||||
log.warning("CHECK_MAIL_API_KEY is empty — disposable-email signup scrutiny is disabled.")
|
||||
|
||||
app = FastAPI(title="yttrx welcome-bot + abuse-bot")
|
||||
|
||||
@@ -201,6 +228,26 @@ def _init_db() -> None:
|
||||
" cached_at TEXT DEFAULT CURRENT_TIMESTAMP"
|
||||
")"
|
||||
)
|
||||
conn.execute(
|
||||
"CREATE TABLE IF NOT EXISTS signup_email ("
|
||||
" account_id TEXT PRIMARY KEY,"
|
||||
" acct TEXT,"
|
||||
" domain TEXT,"
|
||||
" is_disposable INTEGER,"
|
||||
" risk INTEGER,"
|
||||
" flagged INTEGER,"
|
||||
" domain_blocked INTEGER DEFAULT 0,"
|
||||
" classified_at TEXT DEFAULT CURRENT_TIMESTAMP"
|
||||
")"
|
||||
)
|
||||
conn.execute(
|
||||
"CREATE TABLE IF NOT EXISTS check_mail_cache ("
|
||||
" domain TEXT PRIMARY KEY,"
|
||||
" is_disposable INTEGER,"
|
||||
" risk INTEGER,"
|
||||
" cached_at TEXT DEFAULT CURRENT_TIMESTAMP"
|
||||
")"
|
||||
)
|
||||
|
||||
|
||||
@contextmanager
|
||||
@@ -294,6 +341,52 @@ def cache_whois_org(ip: str, org: str) -> None:
|
||||
)
|
||||
|
||||
|
||||
def record_signup_email(account_id: str, acct: str, domain: str, is_disposable: bool,
|
||||
risk: int, flagged: bool) -> None:
|
||||
with _db() as conn:
|
||||
conn.execute(
|
||||
"INSERT OR REPLACE INTO signup_email "
|
||||
"(account_id, acct, domain, is_disposable, risk, flagged, domain_blocked) "
|
||||
"VALUES (?, ?, ?, ?, ?, ?, "
|
||||
" COALESCE((SELECT domain_blocked FROM signup_email WHERE account_id = ?), 0))",
|
||||
(account_id, acct, domain, int(is_disposable), risk, int(flagged), account_id),
|
||||
)
|
||||
|
||||
|
||||
def get_signup_email(account_id: str) -> tuple[str, bool]:
|
||||
"""Return (domain, flagged) for a signup's classified email domain, or
|
||||
("", False) if it was never classified (check-mail disabled/unavailable)."""
|
||||
with _db() as conn:
|
||||
row = conn.execute(
|
||||
"SELECT domain, flagged FROM signup_email WHERE account_id = ?", (account_id,)
|
||||
).fetchone()
|
||||
return (row[0], bool(row[1])) if row else ("", False)
|
||||
|
||||
|
||||
def mark_email_domain_blocked(account_id: str) -> None:
|
||||
with _db() as conn:
|
||||
conn.execute(
|
||||
"UPDATE signup_email SET domain_blocked = 1 WHERE account_id = ?",
|
||||
(account_id,),
|
||||
)
|
||||
|
||||
|
||||
def cached_check_mail(domain: str) -> tuple[bool, int] | None:
|
||||
with _db() as conn:
|
||||
row = conn.execute(
|
||||
"SELECT is_disposable, risk FROM check_mail_cache WHERE domain = ?", (domain,)
|
||||
).fetchone()
|
||||
return (bool(row[0]), row[1]) if row else None
|
||||
|
||||
|
||||
def cache_check_mail(domain: str, is_disposable: bool, risk: int) -> None:
|
||||
with _db() as conn:
|
||||
conn.execute(
|
||||
"INSERT OR IGNORE INTO check_mail_cache (domain, is_disposable, risk) VALUES (?, ?, ?)",
|
||||
(domain, int(is_disposable), risk),
|
||||
)
|
||||
|
||||
|
||||
_init_db()
|
||||
|
||||
# --- Signature verification ------------------------------------------------
|
||||
@@ -340,39 +433,110 @@ def send_welcome(account_id: str, acct: str) -> None:
|
||||
acct, account_id, resp.json().get("id"))
|
||||
|
||||
|
||||
def process_signup(account_id: str, acct: str, ip: str) -> None:
|
||||
"""Classify a new signup's IP and act on account.created.
|
||||
def classify_email_domain(domain: str) -> tuple[bool, int]:
|
||||
"""Classify a signup's email domain via check-mail.org.
|
||||
|
||||
Non-flagged (or IP-scrutiny disabled / no ip in payload) signups are
|
||||
welcomed immediately, same as before this feature existed. Flagged
|
||||
(datacenter) signups get a moderator DM, an optional ip_blocks
|
||||
registration, and — unless dry-run — have their welcome held until
|
||||
account.approved fires (see _handle_account_created).
|
||||
Domain-only query (never the full email) — see CHECK_MAIL_API_KEY comment.
|
||||
Cached indefinitely in sqlite (a domain's disposable/risk classification
|
||||
doesn't meaningfully change hour to hour, and the free tier is capped at
|
||||
1,000 req/month). API failure yields (False, 0) — never flag on our own
|
||||
lookup errors, same philosophy as classify_signup_ip's RDAP failure path.
|
||||
"""
|
||||
if not (IP_SCRUTINY_ENABLED and ip):
|
||||
send_welcome(account_id, acct)
|
||||
return
|
||||
cached = cached_check_mail(domain)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
try:
|
||||
resp = httpx.post(
|
||||
"https://api.check-mail.org/v2/",
|
||||
headers={"Authorization": f"Bearer {CHECK_MAIL_API_KEY}"},
|
||||
data={"domain": domain},
|
||||
timeout=10.0,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
is_disposable = bool(data.get("is_disposable") or data.get("block"))
|
||||
risk = int(data.get("risk") or 0)
|
||||
except (httpx.HTTPError, ValueError) as exc:
|
||||
log.warning("check-mail.org lookup failed for domain=%s: %s", domain, exc)
|
||||
return False, 0
|
||||
|
||||
cache_check_mail(domain, is_disposable, risk)
|
||||
return is_disposable, risk
|
||||
|
||||
|
||||
def register_email_domain_block(domain: str, acct: str) -> None:
|
||||
"""Register a flagged signup's email domain in Mastodon's native
|
||||
Admin::EmailDomainBlock, blocking future signups from that domain.
|
||||
|
||||
Requires the admin:write:email_domain_blocks OAuth scope on
|
||||
ABUSE_BOT_TOKEN — see CLAUDE.md's moderator-token-gotcha section.
|
||||
"""
|
||||
try:
|
||||
resp = httpx.post(
|
||||
f"{MASTODON_BASE_URL}/api/v1/admin/email_domain_blocks",
|
||||
headers=_admin_headers(),
|
||||
data={"domain": domain},
|
||||
timeout=15.0,
|
||||
)
|
||||
if resp.status_code == 422:
|
||||
log.info("email_domain_block for %s already exists (or invalid), treating as handled", domain)
|
||||
else:
|
||||
resp.raise_for_status()
|
||||
except httpx.HTTPError as exc:
|
||||
log.error("failed to register email_domain_block for %s (acct=%s): %s", domain, acct, exc)
|
||||
|
||||
|
||||
def process_signup(account_id: str, acct: str, ip: str, email: str = "") -> None:
|
||||
"""Classify a new signup's IP and email domain, and act on account.created.
|
||||
|
||||
Non-flagged (or both signals disabled/unavailable) signups are welcomed
|
||||
immediately, same as before these features existed. A signup flagged by
|
||||
either signal (datacenter IP and/or high-risk email domain) gets a
|
||||
moderator DM, the matching auto-block(s) (ip_block / email_domain_block),
|
||||
and — unless dry-run — has its welcome held until account.approved fires
|
||||
(see _handle_account_created).
|
||||
"""
|
||||
reasons: list[str] = []
|
||||
hold = False
|
||||
|
||||
if IP_SCRUTINY_ENABLED and ip:
|
||||
classification, org = classify_signup_ip(ip)
|
||||
flagged = classification == "datacenter"
|
||||
record_signup_ip(account_id, acct, ip, classification, org, flagged)
|
||||
|
||||
if not flagged:
|
||||
send_welcome(account_id, acct)
|
||||
return
|
||||
ip_flagged = classification == "datacenter"
|
||||
record_signup_ip(account_id, acct, ip, classification, org, ip_flagged)
|
||||
|
||||
if ip_flagged:
|
||||
log.warning("flagged signup acct=%s ip=%s classification=%s org=%s",
|
||||
acct, ip, classification, org)
|
||||
|
||||
if IP_SCRUTINY_AUTO_IPBLOCK and not IP_SCRUTINY_DRY_RUN:
|
||||
register_ip_block(ip, acct, org)
|
||||
mark_ipblock_registered(account_id)
|
||||
|
||||
hold = IP_SCRUTINY_HOLD_WELCOME and not IP_SCRUTINY_DRY_RUN
|
||||
prefix = "[DRY-RUN] " if IP_SCRUTINY_DRY_RUN else ""
|
||||
reasons.append(f"{prefix}IP {ip} ({org or 'unknown org'}, datacenter/hosting)")
|
||||
hold = hold or (IP_SCRUTINY_HOLD_WELCOME and not IP_SCRUTINY_DRY_RUN)
|
||||
|
||||
if CHECK_MAIL_ENABLED and CHECK_MAIL_API_KEY and email and "@" in email:
|
||||
domain = email.rsplit("@", 1)[-1].strip().lower()
|
||||
is_disposable, risk = classify_email_domain(domain)
|
||||
email_flagged = is_disposable or risk >= CHECK_MAIL_RISK_THRESHOLD
|
||||
record_signup_email(account_id, acct, domain, is_disposable, risk, email_flagged)
|
||||
|
||||
if email_flagged:
|
||||
log.warning("flagged signup acct=%s email_domain=%s is_disposable=%s risk=%d",
|
||||
acct, domain, is_disposable, risk)
|
||||
if CHECK_MAIL_AUTO_DOMAIN_BLOCK and not CHECK_MAIL_DRY_RUN:
|
||||
register_email_domain_block(domain, acct)
|
||||
mark_email_domain_blocked(account_id)
|
||||
prefix = "[DRY-RUN] " if CHECK_MAIL_DRY_RUN else ""
|
||||
reasons.append(f"{prefix}email domain {domain} (disposable={is_disposable}, risk={risk})")
|
||||
hold = hold or (CHECK_MAIL_HOLD_WELCOME and not CHECK_MAIL_DRY_RUN)
|
||||
|
||||
if not reasons:
|
||||
send_welcome(account_id, acct)
|
||||
return
|
||||
|
||||
dm_moderator(
|
||||
f"{prefix}⚠️ Flagged signup @{acct} from {ip} ({org or 'unknown org'}, "
|
||||
f"datacenter/hosting)."
|
||||
f"⚠️ Flagged signup @{acct}: {'; '.join(reasons)}."
|
||||
+ (" Welcome DM held pending approval." if hold else "")
|
||||
)
|
||||
|
||||
@@ -581,8 +745,25 @@ def dm_silenced_user(acct: str) -> None:
|
||||
|
||||
|
||||
def handle_report(report_id: str, target_id: str, acct: str, is_local: bool,
|
||||
suspended: bool, silenced: bool, privileged: bool = False) -> None:
|
||||
"""Evaluate a reported account and silence it if the policy is met."""
|
||||
suspended: bool, silenced: bool, privileged: bool = False,
|
||||
email: str = "") -> None:
|
||||
"""Evaluate a reported account and silence it if the policy is met.
|
||||
|
||||
Special case: if the reported account's email domain is classified
|
||||
high-risk by check-mail.org, it's auto-**suspended** (not merely silenced)
|
||||
on this first report alone — no distinct-reporter threshold — and the
|
||||
email domain is registered in Mastodon's native email_domain_blocks.
|
||||
Classified LIVE from this report delivery's target_account.email (a full
|
||||
Admin::Account, always present on the payload) rather than from any
|
||||
account.created-time record, so this also covers accounts that signed up
|
||||
before this feature existed or while CHECK_MAIL was disabled/down.
|
||||
|
||||
Gated by ABUSE_DRY_RUN **or** CHECK_MAIL_DRY_RUN (either one holds it) —
|
||||
deliberately independent of ABUSE_DRY_RUN alone, so this new/unverified
|
||||
action path ships inert by CHECK_MAIL_DRY_RUN's own default even once the
|
||||
general abuse-bot is live, same rollout-safety pattern as every other
|
||||
signal here.
|
||||
"""
|
||||
if not (ABUSE_ENABLED and ABUSE_BOT_TOKEN):
|
||||
return
|
||||
if ABUSE_LOCAL_ONLY and not is_local:
|
||||
@@ -602,6 +783,46 @@ def handle_report(report_id: str, target_id: str, acct: str, is_local: bool,
|
||||
report_id, acct)
|
||||
return
|
||||
|
||||
report_url = f"{MASTODON_BASE_URL}/admin/reports/{report_id}"
|
||||
|
||||
if CHECK_MAIL_ENABLED and CHECK_MAIL_API_KEY and email and "@" in email:
|
||||
domain = email.rsplit("@", 1)[-1].strip().lower()
|
||||
is_disposable, risk = classify_email_domain(domain)
|
||||
if is_disposable or risk >= CHECK_MAIL_RISK_THRESHOLD:
|
||||
note = (f"Auto-suspend by abuse-bot: reported account's email domain "
|
||||
f"'{domain}' is flagged high-risk by check-mail.org "
|
||||
f"(disposable={is_disposable}, risk={risk}). Pending human "
|
||||
f"review (report left open).")
|
||||
|
||||
if ABUSE_DRY_RUN or CHECK_MAIL_DRY_RUN:
|
||||
log.warning("[DRY-RUN] would suspend acct=%s (flagged email domain=%s, "
|
||||
"risk=%d). %s", acct, domain, risk, report_url)
|
||||
record_action(target_id, acct, "dry-run:suspend", "email-risk", 1, report_id)
|
||||
dm_moderator(f"[DRY-RUN] would suspend @{acct} — email domain "
|
||||
f"'{domain}' is flagged high-risk (risk={risk}). "
|
||||
f"Review: {report_url}")
|
||||
return
|
||||
|
||||
try:
|
||||
apply_action(target_id, "suspend", note)
|
||||
except httpx.HTTPError as exc:
|
||||
log.error("report %s: failed to suspend acct=%s (email-risk): %s",
|
||||
report_id, acct, exc)
|
||||
return
|
||||
|
||||
record_action(target_id, acct, "suspend", "email-risk", 1, report_id)
|
||||
log.warning("auto-suspended acct=%s (flagged email domain=%s, risk=%d); "
|
||||
"report left open", acct, domain, risk)
|
||||
dm_moderator(f"🚨 Auto-suspended @{acct} — email domain '{domain}' flagged "
|
||||
f"high-risk by check-mail.org (risk={risk}). Report left open "
|
||||
f"for review: {report_url}")
|
||||
|
||||
if CHECK_MAIL_AUTO_DOMAIN_BLOCK:
|
||||
record_signup_email(target_id, acct, domain, is_disposable, risk, True)
|
||||
register_email_domain_block(domain, acct)
|
||||
mark_email_domain_blocked(target_id)
|
||||
return
|
||||
|
||||
try:
|
||||
profile = classify_account(target_id)
|
||||
sources = count_distinct_reporters(target_id)
|
||||
@@ -631,7 +852,6 @@ def handle_report(report_id: str, target_id: str, acct: str, is_local: bool,
|
||||
report_id, acct, sources, required)
|
||||
return
|
||||
|
||||
report_url = f"{MASTODON_BASE_URL}/admin/reports/{report_id}"
|
||||
ip_note = " Signup IP was flagged (datacenter/hosting) — threshold lowered." if flagged_ip else ""
|
||||
note = (f"Auto-{ABUSE_ACTION} by abuse-bot: {sources} distinct reporters, "
|
||||
f"account tier={tier}. Pending human review (report left open).{ip_note}")
|
||||
@@ -716,10 +936,12 @@ def _handle_account_created(obj: dict, background: BackgroundTasks, event: str):
|
||||
# Respond 200 immediately; do the API call out-of-band so Mastodon's
|
||||
# delivery doesn't block on our outbound request.
|
||||
if event == "account.created":
|
||||
# First sighting of this account — classify its signup IP (Admin::Account.ip)
|
||||
# and decide whether to welcome now or hold for account.approved.
|
||||
# First sighting of this account — classify its signup IP
|
||||
# (Admin::Account.ip) and email domain (Admin::Account.email) and
|
||||
# decide whether to welcome now or hold for account.approved.
|
||||
ip = obj.get("ip") or ""
|
||||
background.add_task(process_signup, account_id, acct, ip)
|
||||
email = obj.get("email") or ""
|
||||
background.add_task(process_signup, account_id, acct, ip, email)
|
||||
else:
|
||||
# account.approved: a human has cleared the approval-required
|
||||
# registration gate. Always welcome (dedup makes this safe) — this is
|
||||
@@ -750,8 +972,14 @@ def _handle_report_created(obj: dict, background: BackgroundTasks):
|
||||
log.warning("could not extract report/target from payload: %s", obj)
|
||||
raise HTTPException(status_code=422, detail="no report/target in payload")
|
||||
|
||||
# target_account is a full Admin::Account (per Mastodon's entity docs), so
|
||||
# its email is present here even if we never saw/recorded this account's
|
||||
# original signup (e.g. it predates this feature or CHECK_MAIL was down).
|
||||
email = target.get("email") or ""
|
||||
|
||||
background.add_task(
|
||||
handle_report, report_id, target_id, acct, is_local,
|
||||
bool(target.get("suspended")), bool(target.get("silenced")), privileged,
|
||||
email,
|
||||
)
|
||||
return {"ok": True, "report": report_id, "target": acct}
|
||||
|
||||
+99
-4
@@ -96,8 +96,8 @@ def routing_tests():
|
||||
})
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json().get("report") == "55", r.json()
|
||||
# (report_id, target_id, acct, is_local, suspended, silenced, privileged)
|
||||
assert reports == [("55", "777", "spammer", True, False, False, False)], reports
|
||||
# (report_id, target_id, acct, is_local, suspended, silenced, privileged, email)
|
||||
assert reports == [("55", "777", "spammer", True, False, False, False, "")], reports
|
||||
|
||||
# 5b. report against a staff account -> privileged=True extracted from role
|
||||
r = post({
|
||||
@@ -112,7 +112,7 @@ def routing_tests():
|
||||
},
|
||||
},
|
||||
})
|
||||
assert reports[-1] == ("56", "778", "waffles", True, False, False, True), reports[-1]
|
||||
assert reports[-1] == ("56", "778", "waffles", True, False, False, True, ""), reports[-1]
|
||||
|
||||
# 6. account.approved also welcomes — but dedup means alice (id 1001,
|
||||
# already welcomed via account.created above) is not welcomed twice.
|
||||
@@ -267,7 +267,7 @@ def ip_scrutiny_tests():
|
||||
main.process_signup("103", "dc2", "198.51.100.21")
|
||||
assert ("103", "dc2") in sent, "dry-run must not hold the welcome"
|
||||
assert ipblocks == [], "dry-run must not write an ip_block"
|
||||
assert any(d.startswith("[DRY-RUN]") for d in dms), dms
|
||||
assert any("[DRY-RUN]" in d for d in dms), dms
|
||||
|
||||
# D. abuse threshold override: a flagged account needs only
|
||||
# IP_SCRUTINY_ABUSE_THRESHOLD (1) distinct reporter to silence, vs. the
|
||||
@@ -293,8 +293,103 @@ def ip_scrutiny_tests():
|
||||
assert ("101", "silence") not in actions, actions
|
||||
|
||||
|
||||
def email_scrutiny_tests():
|
||||
"""Drive process_signup's email-domain path + the report-triggered
|
||||
suspend+domain-block override, network stubbed out."""
|
||||
sent = []
|
||||
dms = []
|
||||
domain_blocks = []
|
||||
main.send_welcome = lambda account_id, acct: sent.append((account_id, acct))
|
||||
main.dm_moderator = lambda message: dms.append(message)
|
||||
main.register_email_domain_block = lambda domain, acct: domain_blocks.append((domain, acct))
|
||||
|
||||
classifications = {
|
||||
"gmail.com": (False, 5),
|
||||
"temp-mail.org": (True, 99),
|
||||
"sketchy-disposable.example": (True, 95),
|
||||
}
|
||||
main.classify_email_domain = lambda domain: classifications[domain]
|
||||
|
||||
main.CHECK_MAIL_ENABLED = True
|
||||
main.CHECK_MAIL_API_KEY = "test-key" # required for the signup-path gate
|
||||
main.IP_SCRUTINY_ENABLED = False # isolate the email-only signal
|
||||
|
||||
# A. clean domain -> welcomed immediately, no domain-block, not flagged.
|
||||
main.CHECK_MAIL_DRY_RUN = False
|
||||
main.CHECK_MAIL_HOLD_WELCOME = True
|
||||
main.CHECK_MAIL_AUTO_DOMAIN_BLOCK = True
|
||||
main.process_signup("201", "clean1", "", "clean1@gmail.com")
|
||||
assert ("201", "clean1") in sent, sent
|
||||
assert domain_blocks == [], domain_blocks
|
||||
assert main.get_signup_email("201") == ("gmail.com", False)
|
||||
|
||||
# B. disposable domain, HOLD_WELCOME + AUTO_DOMAIN_BLOCK, live -> welcome
|
||||
# held, domain blocked, moderator DM'd; account.approved later releases
|
||||
# the welcome.
|
||||
sent.clear(); dms.clear(); domain_blocks.clear()
|
||||
main.process_signup("202", "spam1", "", "spam1@temp-mail.org")
|
||||
assert ("202", "spam1") not in sent, "welcome should be held for a flagged signup"
|
||||
assert domain_blocks == [("temp-mail.org", "spam1")], domain_blocks
|
||||
assert any("spam1" in d and "held" in d for d in dms), dms
|
||||
assert main.get_signup_email("202") == ("temp-mail.org", True)
|
||||
|
||||
main.send_welcome("202", "spam1") # simulate account.approved's unconditional welcome
|
||||
assert ("202", "spam1") in sent, sent
|
||||
|
||||
# C. same idea, but CHECK_MAIL_DRY_RUN -> welcomed immediately (no hold),
|
||||
# no domain-block write, DM carries the dry-run marker.
|
||||
sent.clear(); dms.clear(); domain_blocks.clear()
|
||||
main.CHECK_MAIL_DRY_RUN = True
|
||||
main.process_signup("203", "spam2", "", "spam2@sketchy-disposable.example")
|
||||
assert ("203", "spam2") in sent, "dry-run must not hold the welcome"
|
||||
assert domain_blocks == [], "dry-run must not write a domain block"
|
||||
assert any("[DRY-RUN]" in d for d in dms), dms
|
||||
|
||||
# D0. CHECK_MAIL_DRY_RUN alone holds the report-triggered suspend even
|
||||
# though ABUSE_DRY_RUN=False in this test env (set in the module
|
||||
# preamble) — the two dry-run switches are independent; either one
|
||||
# holding is enough to keep this brand-new action path inert.
|
||||
actions = []
|
||||
main.apply_action = lambda target_id, action, text: actions.append((target_id, action))
|
||||
main.dm_silenced_user = lambda acct: None
|
||||
domain_blocks.clear()
|
||||
|
||||
main.handle_report("R9", "997", "heldbydryrun", True, False, False, False,
|
||||
"heldbydryrun@temp-mail.org")
|
||||
assert ("997", "suspend") not in actions, actions
|
||||
assert domain_blocks == [], "CHECK_MAIL_DRY_RUN alone must prevent the domain block too"
|
||||
|
||||
# D. report-triggered override: ANY report against an account with a
|
||||
# high-risk email domain suspends immediately (no distinct-reporter
|
||||
# threshold) and blocks the domain — classified LIVE from the report
|
||||
# payload's target_account.email, independent of whatever (if
|
||||
# anything) process_signup recorded at signup time. This account was
|
||||
# never seen by process_signup at all.
|
||||
main.CHECK_MAIL_DRY_RUN = False
|
||||
actions.clear()
|
||||
domain_blocks.clear()
|
||||
|
||||
main.handle_report("R10", "999", "neversignedup", True, False, False, False,
|
||||
"neversignedup@temp-mail.org")
|
||||
assert ("999", "suspend") in actions, actions
|
||||
assert domain_blocks == [("temp-mail.org", "neversignedup")], domain_blocks
|
||||
|
||||
# E. report against an account with a clean email domain -> untouched by
|
||||
# this path, falls through to the normal reporter-count/tier policy.
|
||||
actions.clear(); domain_blocks.clear()
|
||||
noposts = {"has_posts": False, "young": False, "dormant": False,
|
||||
"newest_at": None, "oldest_seen": None, "truncated": False}
|
||||
main.classify_account = lambda tid: noposts
|
||||
main.count_distinct_reporters = lambda tid: 1
|
||||
main.handle_report("R11", "998", "clean2", True, False, False, False,
|
||||
"clean2@gmail.com")
|
||||
assert ("998", "suspend") not in actions, actions
|
||||
assert domain_blocks == [], domain_blocks
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
routing_tests()
|
||||
policy_tests()
|
||||
ip_scrutiny_tests()
|
||||
email_scrutiny_tests()
|
||||
print("ALL TESTS PASSED")
|
||||
|
||||
Reference in New Issue
Block a user