vault-para/100-project/Personal/Software/Matrix Ess Server Install.md

Matrix ESS (Community) — Single‑Node Install on Debian 13 (K3s + Traefik + cert‑manager)

Last updated: 2025-09-25 08:45 UTC

This guide installs Element Server Suite (ESS) Community (Synapse + MAS + Element Web + Matrix RTC) on a single Debian 13 node using K3s, Traefik (default in K3s), and cert‑manager with Let’s Encrypt. It is tailored to your domain choices:

• serverName: chans.xyz
• Hosts: synapse.chans.xyz, account.chans.xyz, chat.chans.xyz, mrtc.chans.xyz

Tip: if you already have K3s and cert‑manager installed and working, you can jump to 5. Values files and 6. Install ESS.

---

0) Requirements & Ports

• Debian 13 (root/sudo), public IPv4 (and optional IPv6).
• DNS control for chans.xyz.
• Open/forward these ports to this node:
  • 80/tcp, 443/tcp (ACME + HTTPS + federation)
  • 30881/tcp, 30882/udp (Matrix RTC SFU)
• Time in sync (systemd-timesyncd or equivalent).

---

1) DNS Setup

Create A/AAAA records that point to your node’s public IP(s):

chans.xyz             A / AAAA -> <YOUR_IP>
synapse.chans.xyz     A / AAAA -> <YOUR_IP>
account.chans.xyz     A / AAAA -> <YOUR_IP>
chat.chans.xyz        A / AAAA -> <YOUR_IP>
mrtc.chans.xyz        A / AAAA -> <YOUR_IP>

Notes:

• Do not use a CNAME at the apex (chans.xyz)—use A/AAAA. Subdomains can be CNAMEs if you prefer.
• Federation relies on https://chans.xyz/.well-known/matrix/server which the chart serves for you.

---

2) (Optional) Cloud‑Init (without firewalld)

If you build the node via cloud‑init, this minimal config installs K3s & Helm and disables swap:

#cloud-config
package_update: true
package_upgrade: true
packages: [curl, ca-certificates, gnupg, lsb-release]

runcmd:
  - swapoff -a
  - sed -ri 's/^[^#].*\sswap\s/## &/g' /etc/fstab
  - curl -sfL https://get.k3s.io | sh -s - server
  - mkdir -p /home/windy/.kube
  - cp /etc/rancher/k3s/k3s.yaml /home/windy/.kube/config
  - chown windy:windy /home/windy/.kube/config && chmod 600 /home/windy/.kube/config
  - bash -lc 'echo export KUBECONFIG=$HOME/.kube/config >> /home/windy/.bashrc'
  - su - windy -c "curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash"

You can manage ports at the cloud firewall or your router (no firewalld required).

---

3) Manual K3s + Helm (if not using cloud‑init)

# Install latest K3s
curl -sfL https://get.k3s.io | sh -s - server

# kubeconfig for your user (replace 'windy' if needed)
mkdir -p ~windy/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~windy/.kube/config
sudo chown windy:windy ~windy/.kube/config
chmod 600 ~windy/.kube/config
echo 'export KUBECONFIG=$HOME/.kube/config' | sudo tee -a ~windy/.bashrc

# Helm
sudo -iu windy bash -lc 'curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash'

Verify:

kubectl get nodes -o wide
kubectl get pods -A

You should see the node Ready and traefik running in kube-system.

---

4) cert‑manager + Let’s Encrypt (ClusterIssuer)

If you haven’t installed cert‑manager yet:

helm repo add jetstack https://charts.jetstack.io --force-update
kubectl create namespace cert-manager 2>/dev/null || true
helm install cert-manager jetstack/cert-manager -n cert-manager --set crds.enabled=true

Create a production ClusterIssuer (letsencrypt-prod):

# clusterissuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-private-key
    solvers:
    - http01:
        ingress:
          class: traefik

Apply:

kubectl apply -f clusterissuer.yaml
kubectl get clusterissuer

You should see letsencrypt-prod READY=True.

---

5) Values files (hosts + TLS)

Create the directory and values files:

mkdir -p ~/ess-config-values

~/ess-config-values/hostnames.yaml

serverName: chans.xyz

elementWeb:
  ingress:
    host: chat.chans.xyz

synapse:
  ingress:
    host: synapse.chans.xyz

matrixAuthenticationService:
  ingress:
    host: account.chans.xyz

matrixRTC:
  ingress:
    host: mrtc.chans.xyz

~/ess-config-values/tls.yaml

global:
  ingress:
    className: traefik
    annotations:
      cert-manager.io/cluster-issuer: letsencrypt-prod
    tls:
      enabled: true
      issuer: letsencrypt-prod

The TLS values ensure your Ingresses are annotated for cert‑manager and include TLS host entries so Certificates are created automatically.

---

6) Install ESS (matrix‑stack chart)

kubectl create namespace ess 2>/dev/null || true

helm upgrade --install ess   oci://ghcr.io/element-hq/ess-helm/matrix-stack   -n ess   -f ~/ess-config-values/hostnames.yaml   -f ~/ess-config-values/tls.yaml   --wait

Check status:

kubectl get pods -n ess
kubectl get ingress -n ess

You should see ingresses for synapse, account, chat, mrtc, and well-known with CLASS=traefik.

---

7) Certificates issuance

Confirm the ingresses have TLS + issuer:

kubectl -n ess get ingress -o jsonpath='{range .items[*]}{.metadata.name}{"  issuer="}{.metadata.annotations.cert-manager\.io/cluster-issuer}{"  tlsHosts="}{range .spec.tls[*].hosts}{.}{" "}{end}{"\n"}{end}'

Then watch certs:

kubectl get certificate -n ess
kubectl get order,challenge -n ess

When ready, confirm live certs:

for h in synapse.chans.xyz account.chans.xyz chat.chans.xyz mrtc.chans.xyz chans.xyz; do
  echo "=== $h ==="
  openssl s_client -connect "$h:443" -servername "$h" </dev/null 2>/dev/null |     openssl x509 -noout -issuer -subject -dates
done

---

8) Well‑Known verification (federation & clients)

curl -s https://chans.xyz/.well-known/matrix/server | jq .
curl -s https://chans.xyz/.well-known/matrix/client | jq .

Expected:

• server → { "m.server": "synapse.chans.xyz:443" }
• client → { "m.homeserver": { "base_url": "https://synapse.chans.xyz" }, ... }

Optional federation tester: https://federationtester.matrix.org/#chans.xyz

---

9) Create the first admin account

Interactive:

kubectl exec -n ess -it deploy/ess-matrix-authentication-service --   mas-cli manage register-user --admin

Non‑interactive example:

kubectl exec -n ess deploy/ess-matrix-authentication-service --   mas-cli manage register-user --yes --admin   --username admin --password 'CHANGE_ME_strong_password'

Login at https://chat.chans.xyz.

---

10) Enable self‑registration (optional)

# ~/ess-config-values/mas-registration.yaml
matrixAuthenticationService:
  additional:
    registration.yaml:
      config: |
        account:
          password_registration_enabled: true
          password_recovery_enabled: true
          login_with_email_allowed: true

Apply (include this file):

helm upgrade --install ess   oci://ghcr.io/element-hq/ess-helm/matrix-stack   -n ess   -f ~/ess-config-values/hostnames.yaml   -f ~/ess-config-values/tls.yaml   -f ~/ess-config-values/mas-registration.yaml   --wait

---

11) Outbound email (MAS required, Synapse optional)

11.1 MAS SMTP (required for signup/reset)

Option A — inline values (simple):

# ~/ess-config-values/mas-email.yaml
matrixAuthenticationService:
  additional:
    user-config.yaml:
      config: |
        email:
          from: '"Matrix @ chans.xyz" <noreply@chans.xyz>'
          reply_to: '"Support" <support@chans.xyz>'
          transport: smtp
          mode: starttls
          hostname: smtp.windy.me
          port: 587
          username: noreply@chans.xyz        # authenticate as the sender
          password: "MAILBOX_PASSWORD"
        account:
          password_registration_enabled: true
          password_recovery_enabled: true
          login_with_email_allowed: true

Option B — secret ref (keeps password out of Git):

cat > /tmp/mas-user-config.yaml <<'YAML'
email:
  from: '"Matrix @ chans.xyz" <noreply@chans.xyz>'
  reply_to: '"Support" <support@chans.xyz>'
  transport: smtp
  mode: starttls
  hostname: smtp.windy.me
  port: 587
  username: noreply@chans.xyz
  password: "MAILBOX_PASSWORD"
account:
  password_registration_enabled: true
  password_recovery_enabled: true
  login_with_email_allowed: true
YAML

kubectl -n ess create secret generic mas-extra-config   --from-file=user-config.yaml=/tmp/mas-user-config.yaml

Then reference it:

# ~/ess-config-values/mas-email-secretref.yaml
matrixAuthenticationService:
  additional:
    user-config.yaml:
      configSecret: mas-extra-config
      configSecretKey: user-config.yaml

Apply (include one of the two files above):

helm upgrade --install ess oci://ghcr.io/element-hq/ess-helm/matrix-stack   -n ess   -f ~/ess-config-values/hostnames.yaml   -f ~/ess-config-values/tls.yaml   -f ~/ess-config-values/mas-email.yaml   --wait
# or replace mas-email.yaml with mas-email-secretref.yaml if you used a Secret

Mailcow 553 fix: If authenticating as zhiqiang@windy.me and sending as noreply@chans.xyz, Mailcow rejects with 553 5.7.1 Sender address rejected. Either (a) authenticate as noreply@chans.xyz by creating that mailbox in Mailcow and publishing SPF/DKIM/DMARC for chans.xyz; or (b) allow “send as” in Mailcow’s Sender ACL for zhiqiang@windy.me. Hosting the chans.xyz mailbox gives best deliverability (DKIM/DMARC alignment).

Monitor while testing:

kubectl -n ess logs deploy/ess-matrix-authentication-service -f | grep -iE 'smtp|email|send'

11.2 Synapse email notifications (optional)

# ~/ess-config-values/synapse-email.yaml
synapse:
  additional:
    email.yaml:
      config: |
        email:
          smtp_host: "smtp.windy.me"
          smtp_port: 587
          smtp_user: "noreply@chans.xyz"
          smtp_pass: "MAILBOX_PASSWORD"
          require_transport_security: true
          notif_from: "Matrix on chans.xyz <noreply@chans.xyz>"
          enable_notifs: true

Include this file in your next Helm upgrade.

---

12) Health checks & troubleshooting

Basic:

kubectl get pods,svc,ingress,certificate -n ess -o wide

Certs flow:

kubectl get certificate,order,challenge -n ess
kubectl describe challenge -n ess <name>
kubectl logs -n kube-system deploy/traefik --tail=200

Well‑known + federation:

curl -s https://chans.xyz/.well-known/matrix/server | jq .
curl -s https://chans.xyz/.well-known/matrix/client | jq .

Common pitfalls:

• Ingresses lack TLS + cert-manager.io/cluster-issuer → fix tls.yaml.
• 553 Sender address rejected from Mailcow → align SMTP auth user with sender or allow “send as”, and set SPF/DKIM/DMARC for chans.xyz.
• Port 80 blocked → Let’s Encrypt HTTP‑01 fails (check challenges).
• Apex chans.xyz not pointing at the node → .well-known fails → federation fails.

---

13) Upgrades / Uninstall

Upgrade to latest chart:

helm repo update  # if using repos
helm upgrade --install ess oci://ghcr.io/element-hq/ess-helm/matrix-stack -n ess   -f ~/ess-config-values/hostnames.yaml -f ~/ess-config-values/tls.yaml --wait

Uninstall ESS (keeps PVCs unless you delete them):

helm uninstall ess -n ess
kubectl delete namespace ess

Reset K3s (if ever needed):

sudo /usr/local/bin/k3s-uninstall.sh

---

14) Quick copy‑paste checklist

1. DNS A/AAAA for: chans.xyz, synapse., account., chat., mrtc. → your IP.
2. K3s running with Traefik; cert‑manager installed; ClusterIssuer letsencrypt-prod Ready.
3. hostnames.yaml with *.ingress.host set to your subdomains.
4. tls.yaml with global.ingress.annotations.cert-manager.io/cluster-issuer=letsencrypt-prod and TLS enabled.
5. helm upgrade --install ess … with both files.
6. kubectl get certificate -n ess → READY=True.
7. /.well-known returns correct JSON; federation tester OK.
8. Create admin via MAS CLI; log in at https://chat.chans.xyz.
9. Configure SMTP for MAS (and optionally Synapse), fix Mailcow sender policy if needed.