258 lines
12 KiB
Markdown
258 lines
12 KiB
Markdown
# yttrx welcome-bot + abuse-bot
|
||
|
||
A small FastAPI webhook server that receives Mastodon **admin webhooks** for
|
||
[yttrx.com](https://yttrx.com) and reacts to two events:
|
||
|
||
- **`account.created` / `account.approved`** — direct-messages a welcome toot
|
||
to every new local signup (the welcome bot). Both events are handled so it
|
||
works whether registration approval is on or off; dedup welcomes once.
|
||
- **`report.created`** — evaluates the reported account and, when enough
|
||
**distinct** reporters have open reports against a *young* or *dormant*
|
||
account, auto-**silences** it (reversible) and DMs a moderator for review
|
||
(the abuse bot).
|
||
|
||
- **Runs on:** `admin.yttrx.com` (Docker container, nginx-proxied at
|
||
`https://hooks.yttrx.com`)
|
||
- **Mastodon side:** `mammut` — one admin webhook subscribing to both events,
|
||
a welcome bot account (posts the DMs), and a separate **moderator** bot
|
||
account whose token carries the admin scopes used to silence accounts.
|
||
- Signatures are HMAC-verified; both flows dedupe in a sqlite store so nobody
|
||
is welcomed twice and no account is auto-actioned twice.
|
||
|
||
```
|
||
new signup / new report on yttrx
|
||
└─> Mastodon (mammut) fires admin webhook account.created|approved | report.created
|
||
└─> POST https://hooks.yttrx.com/webhook (X-Hub-Signature: sha256=…)
|
||
└─> nginx (admin) -> 127.0.0.1:8087 -> bot container
|
||
├─ account.created/approved -> POST /api/v1/statuses (welcome DM)
|
||
└─ report.created -> classify target, count distinct
|
||
reporters, maybe silence + DM mod
|
||
```
|
||
|
||
## Abuse-bot policy
|
||
|
||
On `report.created` against a **local**, not-already-limited account that is
|
||
**not staff** (no Admin/Owner/Moderator role — `ABUSE_SKIP_PRIVILEGED`) and not
|
||
on `ABUSE_ALLOWLIST`:
|
||
|
||
1. **Classify the account** by its authored posts (reblogs excluded):
|
||
- `young` — oldest post is newer than `ABUSE_YOUNG_MAX_DAYS` (30d)
|
||
- `dormant` — newest post is older than `ABUSE_DORMANT_MIN_DAYS` (30d)
|
||
- `no-posts` — reported account with no authored statuses
|
||
- `active` — everything else (posting for >1mo and posted recently)
|
||
2. **Count distinct reporters** with *open* reports against the target
|
||
(self-reports excluded — one person filing repeatedly counts once).
|
||
3. **Silence** the account when distinct reporters ≥ the tier threshold:
|
||
- young / dormant / no-posts → `ABUSE_SOURCES_NEWDORMANT` (2)
|
||
- active → `ABUSE_SOURCES_ACTIVE` (3)
|
||
4. The action is `silence` (reversible) and is applied **without** a
|
||
`report_id`, so the report **stays open** in the moderation queue. The bot
|
||
then DMs `MOD_ALERT_ACCT` a summary with a link to the report, and DMs the
|
||
**silenced user** a link to the appeals/help page (`ABUSE_HELP_URL`,
|
||
`https://welcome.yttrx.com/posts/account-limited/`).
|
||
|
||
`ABUSE_DRY_RUN=true` (the shipped default) logs + DMs what *would* happen
|
||
without touching any account — keep it on until you've watched it for a while.
|
||
|
||
## IP-based signup scrutiny
|
||
|
||
Every `account.created` delivery already carries the signup IP for free
|
||
(`Admin::Account.ip`). On each new local signup, the bot:
|
||
|
||
1. **Classifies the IP** via RDAP org lookup (`ipwhois`, cached in sqlite) as
|
||
`datacenter` (hosting/VPN-keyword match), `mobile` (carrier-keyword match,
|
||
informational only), or `residential` (everything else — never flagged).
|
||
RDAP failures classify as `unknown` and are never flagged.
|
||
2. If **`datacenter`**, the signup is treated with more scrutiny:
|
||
- **Moderator DM** with the IP, org, and classification.
|
||
- **Held welcome** (`IP_SCRUTINY_HOLD_WELCOME`) — the welcome DM is skipped
|
||
on `account.created` and only sent when `account.approved` fires, i.e.
|
||
once a human clears yttrx's existing approval-required registration
|
||
gate. If the signup is rejected instead, no welcome is ever sent.
|
||
- **Auto-registered IP block** (`IP_SCRUTINY_AUTO_IPBLOCK`) — the IP is
|
||
added to Mastodon's native `Admin::IpBlock` at
|
||
`IP_SCRUTINY_IPBLOCK_SEVERITY` (default `sign_up_requires_approval`,
|
||
reversible from the admin UI).
|
||
- **Lowered abuse-bot threshold** — if this account is later reported, the
|
||
usual `ABUSE_SOURCES_*` distinct-reporter threshold is replaced by
|
||
`IP_SCRUTINY_ABUSE_THRESHOLD` (whichever is lower), since a flagged
|
||
signup IP plus a report is a stronger combined signal than either alone.
|
||
|
||
`IP_SCRUTINY_DRY_RUN=true` (the shipped default) classifies and DMs a
|
||
moderator without holding any welcome or writing any ip_block — keep it on
|
||
until you've watched the false-positive rate of the hosting/mobile keyword
|
||
regexes for a while. Registering ip_blocks requires the `ABUSE_BOT_TOKEN` to
|
||
carry the `admin:write:ip_blocks` scope in addition to its existing scopes
|
||
(see `CLAUDE.md`).
|
||
|
||
## Layout
|
||
|
||
| Path | Purpose |
|
||
|---|---|
|
||
| `app/main.py` | The FastAPI app |
|
||
| `Dockerfile`, `docker-compose.yml` | Container build + run |
|
||
| `.env.example` | Config template (copy to `.env`, never commit) |
|
||
| `nginx/hooks.yttrx.com.conf` | nginx site for admin.yttrx.com |
|
||
| `test_local.py` | Offline smoke test (no network) |
|
||
|
||
## Configuration
|
||
|
||
Copy `.env.example` to `.env` and fill in:
|
||
|
||
| Var | What |
|
||
|---|---|
|
||
| `WEBHOOK_SECRET` | The secret Mastodon shows when you create the webhook |
|
||
| `BOT_ACCESS_TOKEN` | Access token of the bot account (scope `write:statuses`) |
|
||
| `MASTODON_BASE_URL` | `https://yttrx.com` |
|
||
| `WELCOME_MESSAGE` | Template; `{acct}` → new user's handle |
|
||
| `LOCAL_ONLY` | `true` — only welcome accounts local to yttrx |
|
||
| `DB_PATH` | `/data/welcomed.db` (matches the compose volume) |
|
||
| `ABUSE_BOT_TOKEN` | Moderator bot token (admin scopes); blank disables abuse handling |
|
||
| `ABUSE_ENABLED` | Master switch for report handling |
|
||
| `ABUSE_DRY_RUN` | `true` — log/DM only, take no action (rollout safety) |
|
||
| `ABUSE_ACTION` | `silence` (default, reversible) or `suspend` |
|
||
| `ABUSE_LOCAL_ONLY` | `true` — only auto-act on local accounts |
|
||
| `ABUSE_YOUNG_MAX_DAYS` / `ABUSE_DORMANT_MIN_DAYS` | Tier windows (30 / 30) |
|
||
| `ABUSE_SOURCES_NEWDORMANT` / `ABUSE_SOURCES_ACTIVE` | Distinct-reporter thresholds (2 / 3) |
|
||
| `ABUSE_MAX_STATUS_PAGES` | Backward status-scan cap (5 × 40 statuses) |
|
||
| `ABUSE_SKIP_PRIVILEGED` | `true` — never auto-act on staff (Admin/Owner/Moderator), via the payload role |
|
||
| `ABUSE_ALLOWLIST` | Extra handles never to auto-act on (comma-separated backstop) |
|
||
| `MOD_ALERT_ACCT` | Handle to DM after an auto-action; blank disables the DM |
|
||
| `ABUSE_HELP_URL` | Appeals/help page the silenced user is linked to |
|
||
| `ABUSE_USER_DM` | Template DMed to the silenced user (`{acct}`, `{help_url}`); blank disables |
|
||
| `IP_SCRUTINY_ENABLED` | Master switch for IP-based signup scrutiny |
|
||
| `IP_SCRUTINY_DRY_RUN` | `true` — classify + DM only, no held welcome, no ip_block write |
|
||
| `IP_SCRUTINY_HOLD_WELCOME` | `true` — hold the welcome for a flagged signup until `account.approved` |
|
||
| `IP_SCRUTINY_ABUSE_THRESHOLD` | Distinct-reporter threshold used (if lower) for accounts with a flagged signup IP |
|
||
| `IP_SCRUTINY_AUTO_IPBLOCK` | Auto-register a flagged IP into Mastodon's `Admin::IpBlock` |
|
||
| `IP_SCRUTINY_IPBLOCK_SEVERITY` | `sign_up_requires_approval` (default), `sign_up_block`, or `no_access` |
|
||
| `IP_SCRUTINY_HOSTING_RE` / `IP_SCRUTINY_MOBILE_RE` | Keyword regexes matched against the RDAP org/ASN description |
|
||
|
||
## Local test
|
||
|
||
```bash
|
||
python3 -m venv .venv && . .venv/bin/activate
|
||
pip install -r requirements.txt
|
||
WEBHOOK_SECRET=testsecret BOT_ACCESS_TOKEN=x python test_local.py # -> ALL TESTS PASSED
|
||
```
|
||
|
||
---
|
||
|
||
## Deploy runbook
|
||
|
||
> Every step that changes the running yttrx system must be logged in
|
||
> `yttrx-documentation/changelog.md` (auto-publishes on `git push origin main`).
|
||
|
||
### 1. Create the bot account + token (on yttrx, via the web UI)
|
||
|
||
1. Register/choose a bot account, e.g. `@welcome` (set "This is a bot account"
|
||
in Preferences → Profile).
|
||
2. Preferences → **Development** → **New application**.
|
||
- Scopes: **`write:statuses`** (untick the rest).
|
||
- Save, open it, copy **"Your access token"** → `BOT_ACCESS_TOKEN` in `.env`.
|
||
|
||
### 1b. Create the moderator bot account + token (for the abuse bot)
|
||
|
||
The abuse bot silences accounts, which needs **admin** privileges — keep this
|
||
off the public welcome bot.
|
||
|
||
1. Register a dedicated account, e.g. `@modbot` (mark it a bot account).
|
||
2. As an admin, give it a **role** that includes the **Manage Users** and
|
||
**Manage Reports** permissions (Administration → Roles, then assign it on
|
||
the account). Without these the API calls 403.
|
||
3. As `@modbot`: Preferences → **Development** → **New application** with
|
||
scopes **`admin:write:accounts`**, **`admin:read:reports`**,
|
||
**`read:statuses`**, **`write:statuses`**. Copy its access token →
|
||
`ABUSE_BOT_TOKEN` in `.env`.
|
||
4. Set `MOD_ALERT_ACCT` to the handle that should receive review DMs (e.g.
|
||
`pete`). Keep `ABUSE_DRY_RUN=true` for the initial rollout.
|
||
|
||
### 2. Build + run the container (on admin.yttrx.com)
|
||
|
||
```bash
|
||
# copy this project to admin (rsync/scp/git clone), then:
|
||
cd ~/yttrx-welcomebot # wherever it lands on admin
|
||
cp .env.example .env # fill in BOT_ACCESS_TOKEN now; WEBHOOK_SECRET in step 4
|
||
docker compose up -d --build
|
||
curl -s localhost:8087/healthz # -> {"ok":true}
|
||
```
|
||
|
||
The container listens on `127.0.0.1:8087` only.
|
||
|
||
### 3. nginx + TLS for hooks.yttrx.com (on admin.yttrx.com)
|
||
|
||
DNS: point `hooks.yttrx.com` (A/AAAA, or proxied via Cloudflare) at admin first.
|
||
|
||
```bash
|
||
sudo cp nginx/hooks.yttrx.com.conf /etc/nginx/sites-available/hooks
|
||
|
||
# issue the cert (standalone — same pattern as the other admin sites)
|
||
sudo systemctl stop nginx
|
||
sudo certbot certonly --standalone -d hooks.yttrx.com
|
||
sudo systemctl start nginx
|
||
|
||
sudo ln -s ../sites-available/hooks /etc/nginx/sites-enabled/hooks
|
||
sudo nginx -t && sudo systemctl reload nginx
|
||
curl -s https://hooks.yttrx.com/healthz # -> {"ok":true}
|
||
```
|
||
|
||
Renewal piggybacks on the existing `0 2 * * * certbot renew --nginx` cron.
|
||
|
||
### 4. Create the webhook in Mastodon (on yttrx, admin UI)
|
||
|
||
Administration → **Webhooks** → **New webhook**:
|
||
|
||
- **URL:** `https://hooks.yttrx.com/webhook`
|
||
- **Events:** check **`account.created`**, **`account.approved`** *and*
|
||
**`report.created`** (one webhook, all events → one shared secret, one
|
||
endpoint). Both account events are handled so the welcome fires whether or
|
||
not registration approval is enabled; the dedup store welcomes once.
|
||
- Save, then copy the generated **secret** → `WEBHOOK_SECRET` in `.env` on
|
||
admin, and `docker compose up -d` to restart with it.
|
||
- Use the webhook's **"Send test"** / re-enable it; confirm a `200` in
|
||
`docker compose logs -f welcomebot`.
|
||
|
||
> If you'd rather roll the abuse bot out separately, create a *second* webhook
|
||
> for `report.created` only — but then it has its **own** secret, so you'd need
|
||
> a second endpoint/secret. Subscribing one webhook to both events is simpler.
|
||
|
||
> ⚠️ Verify the exact event name and that the signing header is
|
||
> `X-Hub-Signature` against this instance (v4.5.11) when you wire it up —
|
||
> the admin UI lists the available events, and the server logs the header it
|
||
> receives. Adjust `app/main.py` if your version differs.
|
||
|
||
### 5. Smoke test end to end
|
||
|
||
Register a throwaway test account on yttrx and confirm the `@welcome` bot DMs
|
||
it. Then delete the test account (`tootctl accounts delete` on mammut) and the
|
||
test toot.
|
||
|
||
## Operations
|
||
|
||
```bash
|
||
welcomebot-logs -f --abuse # convenience CLI on admin (see bin/welcomebot-logs)
|
||
welcomebot-signups --flagged # signup-IP classification history (see bin/welcomebot-signups)
|
||
docker compose logs -f welcomebot # watch deliveries
|
||
docker compose restart welcomebot # after editing .env
|
||
docker compose down && docker compose up -d --build # redeploy after code change
|
||
```
|
||
|
||
Dedup store: `welcomebot-data` volume → `/data/welcomed.db`. To re-welcome
|
||
someone (e.g. after a failed send that got marked), delete their row:
|
||
|
||
```bash
|
||
docker compose exec welcomebot \
|
||
python -c "import sqlite3; sqlite3.connect('/data/welcomed.db').execute(\
|
||
'DELETE FROM welcomed WHERE acct=?', ('alice',)).connection.commit()"
|
||
```
|
||
|
||
## Rollback
|
||
|
||
```bash
|
||
# stop the bot
|
||
cd ~/yttrx-welcomebot && docker compose down
|
||
# disable nginx site
|
||
sudo rm /etc/nginx/sites-enabled/hooks && sudo nginx -t && sudo systemctl reload nginx
|
||
# in Mastodon admin UI: disable or delete the account.created webhook
|
||
```
|