windy/vps
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs(us4): sing-box :6443 代理 + 根盘回收 91%→43% (Plane VPS-99/100)

Browse Source 
新增独立 HTTPS forward proxy：Traefik 新 entrypoint proxyin(:6443) 终止 TLS，
后端 sing-box（HTTP inbound + users 认证 + direct 出站），与既有 :443
nghttpx/squid 链路并存。宿主机只新增 6443/tcp；容器不发布 host 端口。

新增 compose/sing-box/（镜像 digest 钉版）与两份 runbook：
- proxy-health.md：:443/:6443 只读健康检查，含开放代理负向测试
- proxy-client-setup.md：curl / env / 浏览器扩展配置与三类失败排查
  （407 无认证头、页面加载不全=CDN 域名漏配、urllib/wget 不支持 TLS-to-proxy）

同批根盘回收 91%→43%（约 14G）：rust/node/.go/podman 孤儿存储/apt/journal/
未用镜像/容器日志；journald 加 SystemMaxUse=200M；nghttpx+squid+traefik 加
json-file 轮转 10m×3；删除 ~/.cargo 后修好 zshenv/bashrc/profile 的失效 source。

验证：:6443 LE 证书、匿名 407、CONNECT 与绝对 URI 均返回 185.201.226.122；
us4-gate 401 / update 200 / trlm 302；8 容器全部 Up；validate-repo.sh PASS。
This commit is contained in:
windyboy
2026-09-26 10:34:09 +08:00
parent 1512dde3ec
commit 7ec29627fb
7 changed files with 574 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
runbooks/README.md
+2
View File
@@ -18,6 +18,8 @@ runbooks start from [`_template.md`](_template.md).
| Gitea health check | [gitea-health.md](gitea-health.md) | read-only |
| Matrix health check | [matrix-health.md](matrix-health.md) | read-only |
| Plane health check | [plane-health.md](plane-health.md) | read-only |
| us4 forward proxy health check (`:443` + `:6443`) | [proxy-health.md](proxy-health.md) | read-only |
| us4 forward proxy client setup (curl / env / browser) | [proxy-client-setup.md](proxy-client-setup.md) | reference |
| pgdb health check | [pgdb-health.md](pgdb-health.md) | read-only |
| pgdb DB restore (pg_restore) | [pgdb-restore.md](pgdb-restore.md) | change (procedure) |
| pgdb image/compose update | [pgdb-update.md](pgdb-update.md) | change (gated) |
runbooks/proxy-client-setup.md
+142
View File
@@ -0,0 +1,142 @@
# Runbook: us4 forward proxy — client configuration
## Purpose
Configure a client (curl, environment variables, browser, or a proxy-switching
extension) to use the authenticated HTTPS forward proxies on `us4.wsvc.info`,
and triage the three failures seen in practice: `407` responses, pages that load
incompletely, and clients that cannot speak TLS to a proxy.
## Scope
- Applicable: the two authenticated HTTPS forward proxies on `us4.wsvc.info`
(`:6443` sing-box, `:443` nghttpx → Squid).
- Applicable: curl, `https_proxy`/`HTTPS_PROXY` environment variables, browser
proxy-switching extensions (Zero Omega / SwitchyOmega), and OS proxy settings.
- Not applicable: server-side proxy health — use [proxy-health.md](proxy-health.md).
- Not applicable: changing proxy, Traefik, or firewall configuration — use
[network-change.md](network-change.md) with a change record.
- Not applicable: WireGuard client setup.
## Ownership
- Owner: personal ops (Windy)
- Last reviewed: 2026-09-26
- Related systems: us4 (`/opt/sing-box`, `/opt/nghttpx`, `/opt/traefik`)
## Safety
### Non-negotiable rules
- Never put the proxy credential in this repository, in Plane, or in a shell
command that lands in shell history on a shared host.
- Read it from its server-side source only:
`ssh -4 windy@us4.wsvc.info 'sudo cat /opt/sing-box/.proxy-password'`.
- Never "fix" a client problem by removing proxy authentication or by publishing
the backend port.
### Stop conditions
- The client sends no `Proxy-Authorization` and the server answers `407`: that is
the server behaving correctly. `STOP` — fix the client, do not change the server.
- A plaintext HTTP request to either port is answered by Traefik with `404`: the
client is not speaking TLS to the proxy. `STOP` and change the client's proxy
protocol; do not add a plaintext entrypoint.
- A known-good curl command from [proxy-health.md](proxy-health.md) Step 4 also
fails: `STOP` and run the health runbook — the fault is server-side.
## Endpoint facts
| | `:6443` (sing-box) | `:443` (nghttpx → Squid) |
|---|---|---|
| Proxy type to the proxy | **HTTPS / TLS** | **HTTPS / TLS** |
| Username | `windy` | `windy` |
| Credential source | `/opt/sing-box/.proxy-password` (server-side, mode 0600) | operator password manager; server holds a bcrypt hash |
| CONNECT (`https://` targets) | yes | yes |
| Absolute-URI (`http://` targets) | yes | **no** — nghttpx answers `400` |
Both endpoints terminate TLS in Traefik. The client's proxy type is therefore
HTTPS (TLS to the proxy), not "HTTP proxy on port 6443".
## Client configuration
### curl
```bash
PW=$(ssh -4 windy@us4.wsvc.info 'sudo cat /opt/sing-box/.proxy-password')
curl -sS --proxy https://us4.wsvc.info:6443 --proxy-user "windy:$PW" https://ipinfo.io/ip
# expected: 185.201.226.122
```
### Environment variables
curl prefers the **lowercase** `https_proxy` over `HTTPS_PROXY`; a pre-existing
lowercase value wins even when the uppercase one is overridden. On the WSL
client both are pre-set to the LAN proxy `http://192.168.66.99:7890`, so
overriding only `HTTPS_PROXY` silently keeps using the LAN proxy (observed:
returns `152.175.29.59` instead of us4's `185.201.226.122`). Set the lowercase
variable, or pass `--proxy` explicitly.
```bash
https_proxy="https://windy:$PW@us4.wsvc.info:6443" curl -sS https://ipinfo.io/ip
```
### Browser proxy-switching extension (Zero Omega / SwitchyOmega)
1. Proxy profile: protocol **HTTPS**, server `us4.wsvc.info`, port `6443`,
username `windy`, password from the server-side file. Save **and** click
*Apply changes* — an unapplied profile is ignored.
2. Routing conditions must cover every hostname the site needs, not only the
landing domain. For `x.com` that means `x.com`, `twitter.com`, `api.x.com`,
`t.co`, `twimg.com`, `abs.twimg.com`, `pbs.twimg.com`, `video.twimg.com`.
Missing CDN domains are the usual cause of a page that loads incompletely.
3. Set `chrome://flags/#enable-quic` to **Disabled** if individual resources
stall: QUIC is UDP and cannot traverse a CONNECT proxy.
4. Watch the server side while the browser loads the page:
```bash
ssh -4 windy@us4.wsvc.info "docker logs -f sing-box" | grep --line-buffered 'inbound connection to'
```
## Troubleshooting
### A — `407 Proxy Authentication Required`
Server log on us4: `authentication failed, no Proxy-Authorization header` — the
client connected but sent no credentials (extension profile missing the
username/password, or not applied). Add them and re-apply.
`authentication failed, incorrect password` instead means the value itself is
wrong; re-read it from `/opt/sing-box/.proxy-password`.
### B — Page loads but is incomplete
Some hostnames are not routed through the proxy (see the condition list above),
or QUIC is stalling. Confirm with the live log: if the site's CDN hosts never
appear as an `inbound connection to` line, they are not being proxied at all.
### C — Client cannot speak TLS to a proxy
`python3 urllib` and `wget` treat an `https://` proxy as a plaintext HTTP proxy.
The request reaches Traefik in cleartext and is answered `404 Not Found`
(observed: `Tunnel connection failed: 404 Not Found`). Use curl, or a client that
supports HTTPS proxies.
### D — `400 Bad Request` on the `:443` endpoint
The `:443` path is CONNECT-only. Absolute-URI plain-HTTP proxying is available
only on `:6443`.
## Final Verification
- `curl --proxy https://us4.wsvc.info:6443 --proxy-user "windy:<pw>" https://ipinfo.io/ip`
returns `185.201.226.122`.
- The browser loads the target site completely, and each expected CDN host
appears in the sing-box log as an `inbound connection to` line.
- No credential was written to the repository, Plane, or a shared transcript.
## References
- [proxy-health.md](proxy-health.md) — read-only server-side health check
- [hosts/us4.wsvc.info.md](../hosts/us4.wsvc.info.md) — endpoint and credential facts
- [network-change.md](network-change.md), [rollback.md](rollback.md)
- Plane VPS-99 (proxy deployment), VPS-100 (root disk recovery)
runbooks/proxy-health.md
+226
View File
@@ -0,0 +1,226 @@
# Runbook: us4 forward proxy health check
## Purpose
Read-only health check for the HTTPS forward proxies on `us4.wsvc.info`: the
canonical `:443` path (Traefik TCP SNI → nghttpx → Squid) and the independent
`:6443` path (Traefik TCP SNI → sing-box). Confirms TLS, authentication, egress
identity, and that neither endpoint is an open proxy.
## Scope
- Applicable: `us4.wsvc.info` only — Traefik containers, the
`/opt/nghttpx` project (`:443` path) and the `/opt/sing-box` project (`:6443`).
- Applicable: verifying that no unauthenticated or wrong-credential request is
proxied, and that private/metadata destinations are refused.
- Not applicable: any change to Traefik, nghttpx, Squid, or sing-box
configuration — this runbook is read-only; changes go through
[network-change.md](network-change.md) with a change record.
- Not applicable: client-side proxy configuration and browser symptoms
(missing credentials, incomplete page loads, TLS-to-proxy support) — use
[proxy-client-setup.md](proxy-client-setup.md).
- Not applicable: host firewall, WireGuard, or the `:9443` legacy host path
retirement (see Plane VPS-98).
## Ownership
- Owner: personal ops (Windy)
- Last reviewed: 2026-10-01
- Related systems: us4 (`/opt/traefik`, `/opt/nghttpx`, `/opt/sing-box`)
## Safety
### Stop conditions
- Live state contradicts the expected topology below → `STOP`, record evidence,
change nothing in this runbook.
- Any request without a credential, or with a wrong credential, returns proxied
content instead of `407` → `STOP` immediately: treat the endpoint as an open
proxy, keep the evidence, and escalate before any other action.
- A private, loopback, link-local, or metadata destination is reachable through
the proxy → `STOP` and escalate.
- An expected listener (`:6443`, `:443`, or sing-box `:8080` inside its network)
is missing, or a container is restart-looping → `STOP`; do not "fix it in
place" from this runbook.
## Expected topology
| Endpoint | Chain | Credential |
|---|---|---|
| `us4.wsvc.info:443` | Traefik TCP router `ng-gateway` (entrypoint `websecure`, `HostSNI(us4.wsvc.info)`, `tls=true`, resolver `letsencrypt`) → `nghttpx-proxy:3000` → `squid-backend:3128` | Squid bcrypt (`/opt/nghttpx/passwords`) |
| `us4.wsvc.info:6443` | Traefik TCP router `singbox-proxy` (entrypoint `proxyin`, `HostSNI(us4.wsvc.info)`, `tls=true`, resolver `letsencrypt`) → `sing-box:8080` | sing-box `users` in `/opt/sing-box/config.json` |
| host `:9443` | host `nghttpx.service` → host Squid `:3128` | none (legacy, firewall-blocked) |
Known limitation of the `:443` path: nghttpx is CONNECT-only and answers `400`
to absolute-URI plain-HTTP requests. The `:6443` sing-box path supports both.
## Pass criteria
1. `docker ps` shows `traefik`, `sing-box`, `nghttpx-proxy`, `squid-backend`
running, with `sing-box` and `traefik` restart counts not increasing.
2. `docker inspect traefik --format '{{json .Config.Cmd}}'` contains
`--entrypoints.proxyin.address=:6443`.
3. TLS on `:6443` serves a valid Let's Encrypt certificate for
`us4.wsvc.info` (not the self-signed default certificate).
4. No credential → `407`; wrong credential → `407`; plain HTTP to `:6443` →
TLS handshake failure.
5. Authenticated request through `:6443` returns egress IP `185.201.226.122`
for both an HTTPS (CONNECT) target and an HTTP (absolute-URI) target.
6. Authenticated request through `:443` returns the same egress IP (CONNECT
target only).
7. `https://us4-gate.wsvc.info` still answers (dashboard, basic auth → `401`),
and the `update`/`trlm` sites are unaffected.
## Procedure
### Step 1 — Container and listener state
**Action**
```bash
ssh -4 windy@us4.wsvc.info
docker ps --format '{{.Names}}\t{{.Status}}\t{{.Ports}}'
docker inspect traefik --format '{{json .Config.Cmd}}'
sudo ss -ltnp | grep -E ':(6443|443|9443)\b'
```
**Expected**
`traefik` publishing `6443`, `sing-box` running with no host port, and the
`proxyin` entrypoint present in Traefik's command.
**Decision**
- All present → Step 2.
- Anything missing or restart-looping → `STOP` and record the evidence.
### Step 2 — TLS certificate
**Action**
```bash
openssl s_client -connect us4.wsvc.info:6443 -servername us4.wsvc.info </dev/null 2>/dev/null \
| openssl x509 -noout -subject -issuer -dates -ext subjectAltName
```
**Expected**
`subject=CN=us4.wsvc.info`, issuer Let's Encrypt, `notAfter` in the future.
**Decision**
- Valid LE certificate → Step 3.
- Self-signed default certificate, expired certificate, or handshake failure →
`STOP`; the certificate is issued on demand, so retry once before concluding.
### Step 3 — Negative authentication tests (run from an external client)
**Action**
```bash
# no credential
curl -sS -o /dev/null -w '%{http_code}\n' --max-time 15 \
--proxy https://us4.wsvc.info:6443 https://example.com
# wrong credential
curl -sS -o /dev/null -w '%{http_code}\n' --max-time 15 \
--proxy https://us4.wsvc.info:6443 --proxy-user 'windy:definitely-wrong' https://example.com
# plain HTTP to the TLS port
curl -sS -o /dev/null -w '%{http_code}\n' --max-time 8 http://us4.wsvc.info:6443/
```
**Expected**
The two proxy attempts fail authentication (`407`, or a curl CONNECT abort), and
the plain-HTTP attempt fails the TLS handshake.
**Decision**
- All rejected → Step 4.
- Any of them returns proxied content → `STOP` immediately (open proxy).
### Step 4 — Positive path and egress identity
**Action** — supply the credential from its server-side source; never paste it
into Plane, the repo, or a shared transcript.
```bash
# on us4, for :6443 (CONNECT target and absolute-URI HTTP target)
PW="$(sudo cat /opt/sing-box/.proxy-password)"
curl -sS --proxy https://us4.wsvc.info:6443 --proxy-user "windy:$PW" https://ipinfo.io/ip
curl -sS --proxy https://us4.wsvc.info:6443 --proxy-user "windy:$PW" http://ipinfo.io/ip
# on us4, for the :443 path
curl -sS --proxy https://us4.wsvc.info:443 --proxy-user 'windy:<password>' https://ipinfo.io/ip
```
**Expected**
`185.201.226.122` on all successful requests.
**Decision**
- Expected egress IP → Step 5.
- `407` with the known-good credential → `STOP`; check `docker logs sing-box`
for the inbound user list before touching configuration.
### Step 5 — Regression on existing HTTPS sites
**Action**
```bash
for h in us4-gate.wsvc.info update.wsvc.info trlm.wsvc.info; do
printf '%s ' "$h"
curl -sS -o /dev/null -w '%{http_code}\n' --max-time 15 "https://$h"
done
```
**Expected**
`us4-gate` → `401` (dashboard basic auth), `update` → `200`, `trlm` → `302`
redirecting to `/login`. A `302` is a healthy application response, not a
failure.
**Decision**
- Expected codes → health check passes.
- Any other code → `STOP`; a Traefik recreate may have failed, check
`docker logs traefik`.
## Troubleshooting
### Troubleshooting A — `407` with the correct credential
- Evidence: `docker logs sing-box` shows the HTTP inbound rejecting the user.
- Allowed: read-only inspection of `/opt/sing-box/config.json` (mode 0600) to
confirm the username matches; compare with the client's username.
- Next: if the password itself is wrong, rotate it as a change (regenerate
on-host, update `config.json`, `docker compose up -d`), then re-run Step 4.
Do not paste the new password into Plane or the repo.
### Troubleshooting B — tunnel establishes but no traffic flows
- Evidence: `docker logs sing-box`, and `docker exec sing-box nslookup <host>`.
- Allowed: read-only checks. Confirm the container resolves and reaches the
internet, and that `route.final` points at the `direct` outbound.
- Next: if DNS resolution fails, compare against the other containers on the
`proxy` network; do not add `internal: true` to the network (it removes
egress and the proxy will accept connections but never reach destinations).
## Final Verification
- All five steps pass.
- No credential appears in Plane, the repo, or the terminal transcript.
- Result and any deviation recorded on Plane VPS-99.
## References
- [hosts/us4.wsvc.info.md](../hosts/us4.wsvc.info.md)
- [proxy-client-setup.md](proxy-client-setup.md) — client-side setup and triage
- [compose/sing-box/compose.yml](../compose/sing-box/compose.yml)
- [network-change.md](network-change.md), [rollback.md](rollback.md)
- Plane VPS-99 (this proxy), VPS-98 (`:443` path audit), VPS-47 (public port
exposure target)
- Traefik TCP TLS: <https://doc.traefik.io/traefik/v3.4/reference/routing-configuration/tcp/tls/>
- sing-box HTTP inbound: <https://sing-box.sagernet.org/configuration/inbound/http/>