A /32 block is close to useless against what IP scrutiny fires on: a datacenter/VPN/proxy customer has the rest of the range sitting right there, so blocking the single address we saw stops nothing the same actor does a minute later from the address next door. The width ceiling added 2026-09-13 made this the common case rather than the exception: its fallback for an over-wide ASN route was the bare address, so a flagged signup out of a /18 hosting allocation got a /32. Add IP_SCRUTINY_MIN_BLOCK_PREFIX_V4/_V6 (24/48) as a floor under that ceiling. Anything narrower than the floor -- a missing route, the /32 recorded when a route didn't contain the IP, a small hosting allocation, or the fallback from a route wider than the ceiling -- is widened to the network of that width containing the address. At the defaults the floor equals the ceiling, so every v4 block is now exactly the /24 containing the flagged address. State the collateral cost rather than waving at it: at the live settings (sign_up_block severity, ~30d expiry) one flagged signup now hard-refuses registration from 256 addresses for a month. Moving the severity to sign_up_requires_approval or shortening the expiry buys that back, and setting the floor to 32/128 restores the old behaviour entirely.
This commit is contained in:
1 parent
9e4456b7df
commit
233458d6b8
4 files changed
+189
-67
No files matched your search
+94
-33
@@ -161,8 +161,9 @@ IP_SCRUTINY_ABUSE_THRESHOLD = int(os.environ.get("IP_SCRUTINY_ABUSE_THRESHOLD",
|
||||
# belonging to (see classify_signup_ip's block_cidr), not just the single
|
||||
# address — a flagged signup is almost always datacenter/VPN/proxy space, and
|
||||
# a repeat bad actor is far more likely to come back from a different address
|
||||
# in the same block than the exact same one. Falls back to a single-address
|
||||
# (/32 or /128) block if no usable route was returned/cached.
|
||||
# in the same block than the exact same one. Falls back to the
|
||||
# IP_SCRUTINY_MIN_BLOCK_PREFIX network containing the address (a /24 by
|
||||
# default) if no usable route was returned/cached.
|
||||
IP_SCRUTINY_AUTO_IPBLOCK = os.environ.get("IP_SCRUTINY_AUTO_IPBLOCK", "true").lower() in ("1", "true", "yes")
|
||||
# severity: sign_up_requires_approval | sign_up_block | no_access
|
||||
IP_SCRUTINY_IPBLOCK_SEVERITY = os.environ.get("IP_SCRUTINY_IPBLOCK_SEVERITY", "sign_up_requires_approval")
|
||||
@@ -180,13 +181,14 @@ IP_SCRUTINY_IPBLOCK_SEVERITY = os.environ.get("IP_SCRUTINY_IPBLOCK_SEVERITY", "s
|
||||
# as wide as /8, /9 and /11 show up in the cache, and one `abuser` flag on a
|
||||
# residential or mobile address inside one of those would otherwise wall off
|
||||
# millions of consumer addresses. A route wider than this is treated as "not a
|
||||
# meaningful unit to block" and falls back to the single address.
|
||||
# meaningful unit to block" and falls back to the floor width below — the
|
||||
# /24 containing the address, not the address alone.
|
||||
#
|
||||
# This deliberately costs blocking strength against large hosting allocations
|
||||
# — anything wider than a /24 (which is most of the VPS ranges registered so
|
||||
# far) now blocks one address rather than the range, so a repeat offender out
|
||||
# of the same block needs a fresh block each time — in exchange for making the
|
||||
# mobile/residential collateral-damage case impossible.
|
||||
# far) now blocks a /24 rather than the whole range, so a repeat offender out
|
||||
# of another part of the same allocation needs a fresh block — in exchange for
|
||||
# making the mobile/residential collateral-damage case impossible.
|
||||
# Applied at registration time, not retroactively — blocks already in force
|
||||
# keep whatever width they were registered with.
|
||||
IP_SCRUTINY_MAX_BLOCK_PREFIX_V4 = int(os.environ.get("IP_SCRUTINY_MAX_BLOCK_PREFIX_V4", "24"))
|
||||
@@ -194,6 +196,32 @@ IP_SCRUTINY_MAX_BLOCK_PREFIX_V4 = int(os.environ.get("IP_SCRUTINY_MAX_BLOCK_PREF
|
||||
# equivalent of "one customer's block" that a /24 represents in v4.
|
||||
IP_SCRUTINY_MAX_BLOCK_PREFIX_V6 = int(os.environ.get("IP_SCRUTINY_MAX_BLOCK_PREFIX_V6", "48"))
|
||||
|
||||
# Narrowest network a single auto-registered block may cover, as a prefix
|
||||
# length — the floor under the ceiling above. Blocking one address is close to
|
||||
# useless against what this fires on: a datacenter/VPN/proxy customer has the
|
||||
# rest of the range sitting right there, so a /32 blocks the address we saw
|
||||
# and nothing the same actor moves to a minute later. Anything narrower than
|
||||
# this — a missing route, the /32 recorded when a route didn't contain the ip,
|
||||
# a small hosting allocation, or the fallback from a route wider than the
|
||||
# ceiling — is widened to the network of this width containing the address.
|
||||
#
|
||||
# The collateral cost is real and worth stating plainly rather than waving
|
||||
# at: how much it hurts an innocent neighbour is set by the two policies
|
||||
# around this one. At what yttrx actually runs
|
||||
# (IP_SCRUTINY_IPBLOCK_SEVERITY=sign_up_block,
|
||||
# IP_SCRUTINY_IPBLOCK_EXPIRES_IN=2629800, ~30 days) one flagged signup
|
||||
# hard-refuses registration from 256 addresses for a month. Moving the
|
||||
# severity to sign_up_requires_approval (neighbour is queued for a human
|
||||
# rather than refused) or shortening the expiry buys that back; widening the
|
||||
# block without touching either is a deliberate choice to spend it.
|
||||
#
|
||||
# At the default this equals the ceiling, so every v4 block is exactly the /24
|
||||
# containing the flagged address. Set it to 32 (128 for v6) to restore the old
|
||||
# behaviour of blocking only what the route says, down to the single address.
|
||||
IP_SCRUTINY_MIN_BLOCK_PREFIX_V4 = int(os.environ.get("IP_SCRUTINY_MIN_BLOCK_PREFIX_V4", "24"))
|
||||
# /48 is the v6 analogue, matching the v6 ceiling for the same reason.
|
||||
IP_SCRUTINY_MIN_BLOCK_PREFIX_V6 = int(os.environ.get("IP_SCRUTINY_MIN_BLOCK_PREFIX_V6", "48"))
|
||||
|
||||
# How long a registered Admin::IpBlock lasts, in seconds. 0 = never expires.
|
||||
#
|
||||
# An IP block is a statement about who is *currently* behind an address, and
|
||||
@@ -740,38 +768,69 @@ def start_suspicious_watch(account_id: str, acct: str, reasons: str,
|
||||
)
|
||||
|
||||
|
||||
def _single_address_cidr(ip: str) -> str:
|
||||
"""The /32 or /128 for ip, or "" if it doesn't parse."""
|
||||
def _block_width_bounds(version: int) -> tuple[int, int]:
|
||||
"""(ceiling, floor) prefix lengths allowed for a block on this IP version.
|
||||
|
||||
Both are prefix lengths, so the ceiling — the widest network allowed — is
|
||||
the SMALLER number. A floor configured wider than the ceiling is nonsense,
|
||||
and the ceiling wins: it is the one that exists to bound collateral damage.
|
||||
"""
|
||||
host_bits = 32 if version == 4 else 128
|
||||
cap = (IP_SCRUTINY_MAX_BLOCK_PREFIX_V4 if version == 4
|
||||
else IP_SCRUTINY_MAX_BLOCK_PREFIX_V6)
|
||||
floor = (IP_SCRUTINY_MIN_BLOCK_PREFIX_V4 if version == 4
|
||||
else IP_SCRUTINY_MIN_BLOCK_PREFIX_V6)
|
||||
cap = max(0, min(cap, host_bits))
|
||||
floor = max(cap, min(floor, host_bits))
|
||||
return cap, floor
|
||||
|
||||
|
||||
def _block_floor_cidr(ip: str) -> str:
|
||||
"""The narrowest network the bot will block for ip: the
|
||||
IP_SCRUTINY_MIN_BLOCK_PREFIX_V4/V6 network containing it (a /24 or /48 by
|
||||
default), or "" if ip doesn't parse.
|
||||
|
||||
This is what a missing, unusable or too-wide route degrades to. It used to
|
||||
be the single /32 or /128, which is the wrong unit for the space these
|
||||
flags fire on — the next signup comes from the address next door.
|
||||
"""
|
||||
try:
|
||||
addr = ipaddress.ip_address(ip)
|
||||
except ValueError:
|
||||
return ""
|
||||
return f"{addr}/{32 if addr.version == 4 else 128}"
|
||||
floor = _block_width_bounds(addr.version)[1]
|
||||
return str(ipaddress.ip_network(f"{addr}/{floor}", strict=False))
|
||||
|
||||
|
||||
def capped_block_cidr(ip: str, cidr: str) -> str:
|
||||
"""The network to actually register a block for: cidr, unless it's wider
|
||||
than IP_SCRUTINY_MAX_BLOCK_PREFIX_V4/V6, in which case the single address.
|
||||
"""The network to actually register a block for: cidr, clamped to the
|
||||
width window between the IP_SCRUTINY_MAX_BLOCK_PREFIX_V4/V6 ceiling
|
||||
(anything wider is not a meaningful unit to block, so it degrades to the
|
||||
floor network containing ip) and the IP_SCRUTINY_MIN_BLOCK_PREFIX_V4/V6
|
||||
floor (anything narrower, a single address included, is widened to it).
|
||||
|
||||
Applied when deciding what to BLOCK, not when caching the classification —
|
||||
the wide ASN route is still the right cache key (the whole route really
|
||||
does belong to one operator, so one lookup answers for all of it), it is
|
||||
just the wrong thing to hand to Admin::IpBlock.
|
||||
"""
|
||||
fallback = _single_address_cidr(ip)
|
||||
floor_net = _block_floor_cidr(ip)
|
||||
if not cidr:
|
||||
return fallback
|
||||
return floor_net
|
||||
try:
|
||||
net = ipaddress.ip_network(cidr, strict=False)
|
||||
except ValueError:
|
||||
return fallback
|
||||
cap = (IP_SCRUTINY_MAX_BLOCK_PREFIX_V4 if net.version == 4
|
||||
else IP_SCRUTINY_MAX_BLOCK_PREFIX_V6)
|
||||
return floor_net
|
||||
cap, floor = _block_width_bounds(net.version)
|
||||
if net.prefixlen < cap:
|
||||
log.info("block for ip=%s narrowed from route %s (%d addresses) to %s — "
|
||||
"wider than the /%d ceiling", ip, cidr, net.num_addresses,
|
||||
fallback or "the single address", cap)
|
||||
return fallback or str(net)
|
||||
floor_net or "the route itself", cap)
|
||||
return floor_net or str(net)
|
||||
if net.prefixlen > floor and floor_net:
|
||||
log.info("block for ip=%s widened from %s to %s — narrower than the "
|
||||
"/%d floor", ip, cidr, floor_net, floor)
|
||||
return floor_net
|
||||
return str(net)
|
||||
|
||||
|
||||
@@ -1389,14 +1448,17 @@ def classify_signup_ip(ip: str) -> tuple[str, str, bool, str]:
|
||||
"+"-joined list of every matched signal (datacenter/vpn/proxy/tor/abuser),
|
||||
or "clean" if none matched. flagged is True if any signal matched.
|
||||
block_cidr is the network register_ip_block should block when flagged —
|
||||
ipapi.is's asn.route when it's a valid network that actually contains ip,
|
||||
else ip's own /32 or /128 — since a flagged signup is almost always
|
||||
datacenter/VPN/proxy space, where a repeat bad actor is far more likely to
|
||||
come back from a different address in the same block than the exact same
|
||||
one. A route wider than the IP_SCRUTINY_MAX_BLOCK_PREFIX_V4/V6 ceiling is
|
||||
narrowed back to the single address (see capped_block_cidr) — that width
|
||||
means an ISP aggregate, not a hosting allocation. API failure yields ("unknown", "", False, "<ip>/32|128") — scrutiny
|
||||
should never trigger on our own lookup errors.
|
||||
ipapi.is's asn.route when it's a valid network that actually contains ip
|
||||
and falls inside the allowed width window, else the
|
||||
IP_SCRUTINY_MIN_BLOCK_PREFIX_V4/V6 network containing ip (a /24 or /48) —
|
||||
since a flagged signup is almost always datacenter/VPN/proxy space, where
|
||||
a repeat bad actor is far more likely to come back from a different
|
||||
address in the same block than the exact same one. A route wider than the
|
||||
IP_SCRUTINY_MAX_BLOCK_PREFIX_V4/V6 ceiling is narrowed to that same floor
|
||||
network rather than blocked as-is (see capped_block_cidr) — that width
|
||||
means an ISP aggregate, not a hosting allocation. API failure yields
|
||||
("unknown", "", False, "<ip>/24|48") — scrutiny should never trigger on
|
||||
our own lookup errors.
|
||||
"""
|
||||
intel = cached_ip_intel(ip)
|
||||
if intel is None:
|
||||
@@ -1409,7 +1471,7 @@ def classify_signup_ip(ip: str) -> tuple[str, str, bool, str]:
|
||||
data = resp.json()
|
||||
except (httpx.HTTPError, ValueError) as exc:
|
||||
log.warning("ipapi.is lookup failed for ip=%s: %s", ip, exc)
|
||||
return "unknown", "", False, _single_address_cidr(ip) or f"{ip}/32"
|
||||
return "unknown", "", False, _block_floor_cidr(ip) or f"{ip}/32"
|
||||
# ipapi.is returns company/asn as flat strings, not nested dicts
|
||||
company = data.get("company")
|
||||
org_name = company if isinstance(company, str) else (company or {}).get("name", "")
|
||||
@@ -1466,8 +1528,9 @@ def register_ip_block(ip: str, acct: str, org: str, cidr: str = "",
|
||||
cidr should be classify_signup_ip's block_cidr — the ASN/route-level
|
||||
network the signup IP belongs to, so a repeat bad actor from the same
|
||||
datacenter/VPN/proxy block is blocked too, not just this one address.
|
||||
Falls back to a single-address block if no cidr is given or ip itself
|
||||
doesn't parse (defensive only; classify_signup_ip always supplies one).
|
||||
Falls back to the floor-width network containing ip (a /24 or /48, see
|
||||
_block_floor_cidr) if no cidr is given; refuses only if ip itself doesn't
|
||||
parse (defensive only; classify_signup_ip always supplies a cidr).
|
||||
|
||||
comment overrides the moderator-facing note stored with the block; the
|
||||
reputation query API passes its own, since a pre-signup block has no
|
||||
@@ -1478,12 +1541,10 @@ def register_ip_block(ip: str, acct: str, org: str, cidr: str = "",
|
||||
as blocked when it really is — /api/blocks publishes those rows.
|
||||
"""
|
||||
if not cidr:
|
||||
try:
|
||||
prefix_len = 32 if ipaddress.ip_address(ip).version == 4 else 128
|
||||
except ValueError:
|
||||
cidr = _block_floor_cidr(ip)
|
||||
if not cidr:
|
||||
log.warning("skipping ip_block registration for unparseable ip=%r (acct=%s)", ip, acct)
|
||||
return False
|
||||
cidr = f"{ip}/{prefix_len}"
|
||||
comment = (comment or
|
||||
f"welcomebot: flagged signup @{acct} ({org or 'unknown org'})")[:200]
|
||||
try:
|
||||
|
||||
Reference in new issue
Block a user