windy/vps
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
fea9a6560f17b546e7ed740bfaae1247e32efc75
vps/runbooks/matter-packet-capture.md
T

13 KiB
Raw Blame History

Matter packet capture (read-only)

Purpose

Capture Matter-related traffic on LAN55 (mDNS discovery + PASE/CASE commissioning + operational traffic) to determine whether a device is on the network, is in commissioning mode, and whether the commissioning handshake completes. Capture is read-only and changes no device or network state.

Scope

  • Environment: LAN55 (hass.windy.lan, Aqara M3, ESP32-C2 Matter bulbs, phone / HA matter-server all on the 55 subnet).
  • Subject: Matter over Wi-Fi and Thread relay nodes. The Thread 802.15.4 air side itself is not capturable — only IPv6 forwarding by a Thread relay such as the M3 is visible.
  • Not applicable: BLE commissioning, Thread 802.15.4 frames, cross-subnet multicast (66-subnet hosts cannot see the 55 subnet's mDNS — link-local multicast does not cross the routed 55/66 boundary, there is no reflector).
  • Read-only: no AP/device/network config is modified; state returns to normal when tcpdump exits.

Capture-point selection

Matter commissioning is a two-party conversation and the commissioner participates in every message of it, so capturing on the commissioner host equals capturing the whole flow.

Capture point Sees Blind spot Notes
hass end0 — commissioner side (recommended) The full HA-driven commissioning conversation: all mDNS queries/announcements (segment multicast) + the complete TCP 5540 PASE/CASE session Phone-as-commissioner flows (the phone's session to the device does not pass through hass) core_matter_server uses host networking, so tcpdump on end0 sees the add-on's traffic directly; / is overlay with ~42 GB free — no 60 MB tmpfs rotation needed
UAP-AC-Lite br0 (192.168.55.5) All mDNS multicast (flooded; igmp snooping off) + all wireless-client unicast + unicast to/from the AP Wired↔wired unicast — e.g. HA↔M3 TCP 5540 while a Thread device commissions via the M3 (wired, observed) — is switched locally and never traverses the AP AP /tmp is a ~60 MB tmpfs → rotating capture is mandatory

For the common "add device" case with HA matter-server as the commissioner, capture on hass end0. Use the AP br0 point for wireless-device or phone-driven flows (a wireless client's unicast to/from its AP is only visible there).

A third point, gw switch0, is verified as a limited capture point (cross-subnet/gateway/mDNS flows only — not a full mirror of LAN55) — see Capture point: gw switch0.

Ownership

  • Owner: personal ops (Windy)
  • Last reviewed: 2026-08-22
  • Related systems: UAP-AC-Lite AP 192.168.55.5 (br0), core_matter_server on hass.windy.lan (end0), Aqara M3, ESP32-C2 Matter bulbs

Preconditions

  • SSH to the capture point:
    • AP: ssh zhiqiangf@192.168.55.5 (key-only, BatchMode=yes verified).
    • hass: ssh hassio@hass.windy.lan. Non-interactive SSH does not source .zprofile, so run tcpdump as sudo -n -i tcpdump … (verified 2026-08-22).
  • tcpdump available:
    • AP: full 4.9.2 / libpcap 1.8.1 (verified 2026-08-22).
    • hass: /usr/bin/tcpdump via sudo -n -i (verified 2026-08-22).
  • Trigger source ready: put the Matter device into commissioning mode, or have HA/phone perform discovery/commissioning — otherwise no relevant packets.
  • AP /tmp is a ~60 MB tmpfs (61.3 M total, 60.4 M free): rotating capture (-C/-W) is mandatory on the AP. hass / is overlay — rotation optional but keep the habit for long captures.

Safety

Non-negotiable rules

  • Read-only diagnosis: no installs, config changes, or service restarts on the AP, hass, devices, or network.
  • pcap files are limited to /tmp; pull them off and delete them afterwards (mandatory on the AP; same hygiene on hass).
  • Never write captured content (including any plaintext key material) into this repository or Linear.

Stop conditions

  • Capture point unreachable (ssh fails) → STOP, fix the network first.
  • tcpdump reports "Permission denied" or cannot listen → STOP (admin needed; on hass verify sudo -n -i works).
  • Filter expression syntax error → STOP, use only expressions verified in this document.
  • AP /tmp nearly full (rotation file count × single-file size ≈ 60 MB) → STOP and clean old pcaps.

Procedure

Step 1 — Choose the capture point

Action

  • HA matter-server is the commissioner (the "add device" case) → hass end0.
  • Wireless device or phone-driven flow → AP br0.

Expected

  • The chosen point is reachable and tcpdump starts listening.

Decision

  • Capture point chosen and reachable → Step 2.
  • Neither applies or the choice is unclear → STOP and record why.

Step 2 — Realtime observation (quick confirm traffic appears)

Action

AP:

ssh zhiqiangf@192.168.55.5 "tcpdump -ni br0 -s 0 -tt 'udp port 5353 or tcp port 5540 or tcp port 5552'"

hass (commissioner side):

ssh hassio@hass.windy.lan "sudo -n -i tcpdump -ni end0 -s 0 -tt 'udp port 5353 or tcp port 5540 or tcp port 5552'"

Keep the window open, trigger the device behavior (enter commissioning mode / start commissioning / send a command), Ctrl+C to stop.

Expected

  • _matterc._udp / _matter._tcp mDNS announcements (UDP 5353, multicast 224.0.0.251 / ff02::fb).
  • During commissioning: TCP 5540 (PASE/CASE) SYN/SYN-ACK between the device IP and HA/M3.
  • If the target device's MAC is known, add and ether host <mac> to keep only that device (see variants).
  • 5552 is not a standard Matter port; it is an observed port for the Aqara M3 Thread-relay node (see docs/matter-pairing-troubleshoot.md).

Decision

  • Expected packets present → Step 3 to save evidence, or judge directly against the stage table (docs/matter-pairing-troubleshoot.md §4).
  • No packets at all → STOP: fix device online / commissioning-mode first; the network side is repeatedly verified healthy (see troubleshooting doc).
  • mDNS present but no 5540 → see troubleshooting doc decision tree, item 4 (§5).

Step 3 — Rotating capture + pull to WSL

Action (-C 5 = rotate every 5 MB, -W 12 = max 12 files, ≈ 60 MB ≤ AP tmpfs)

AP:

ssh zhiqiangf@192.168.55.5 "tcpdump -ni br0 -s 0 -C 5 -W 12 -w /tmp/matter.pcap 'udp port 5353 or tcp port 5540 or tcp port 5552'"

hass (rotation optional — overlay disk):

ssh hassio@hass.windy.lan "sudo -n -i tcpdump -ni end0 -s 0 -C 5 -W 12 -w /tmp/matter.pcap 'udp port 5353 or tcp port 5540 or tcp port 5552'"

Trigger the traffic, then Ctrl+C. Files are /tmp/matter.pcap, /tmp/matter.pcap1, …

Expected

  • tcpdump prints capture statistics (N packets captured).
  • ls -la /tmp/matter.pcap* shows the files; total stays < 60 MB on the AP.

Verification

ssh zhiqiangf@192.168.55.5 "ls -la /tmp/matter.pcap*"
# or
ssh hassio@hass.windy.lan "ls -la /tmp/matter.pcap*"

Pull to WSL for analysis and clean up afterwards

scp zhiqiangf@192.168.55.5:/tmp/matter.pcap* .
# or
scp hassio@hass.windy.lan:/tmp/matter.pcap* .

# clean up on the capture point
ssh zhiqiangf@192.168.55.5 "rm -f /tmp/matter.pcap*"
ssh hassio@hass.windy.lan "sudo -n -i rm -f /tmp/matter.pcap*"

Step 4 — Wireshark analysis (optional)

Action

Open the pcap in Wireshark. mDNS (UDP 5353) is plaintext and directly readable; Matter payloads on TCP/UDP 5540 show only the handshake by default — plaintext needs the dissector plus session keys (Step 5).

Expected

  • mDNS filter shows all discovery records; tcp.port==5540 shows the commissioning handshake.

Step 5 — Decrypt Matter plaintext (optional, needs session keys)

Matter payloads are encrypted (AES-CCM); mDNS plaintext contains no keys. To decrypt, one of:

  1. Capture-side key leak with a chip tool (most common): the commissioner (HA matter-server / chip-tool) prints or exports session keys during commissioning; enter them in Wireshark → Preferences → Protocols → Matter. See matter-dissector README.
  2. well-known CASE keys: both sides compiled with MATTER_CONFIG_SECURITY_TEST_MODE / CASEUseKnownECDHKey; not enabled in this environment (ESP32-C2 + HA official matter-server).

Expected

  • Matter dissector expands protocol headers, IM commands, and cluster content.

Stop condition (decryption): with no session keys or test keys obtainable, do not fabricate keys to force a decrypt — plaintext mDNS + TCP handshake still resolves most troubleshooting; for plaintext payloads, upgrade to exporting keys on the commissioner side, then return to this runbook.

Targeted capture variants

One device only (known MAC)

ssh zhiqiangf@192.168.55.5 "tcpdump -ni br0 -s 0 -tt 'ether host 34:98:7a:27:7f:08 and (udp port 5353 or tcp port 5540 or tcp port 5552)'"

MACs from docs/matter-pairing-troubleshoot.md §3 (working bulb 34:98:7a:25:a1:f0, broken bulb 34:98:7a:27:7f:08).

mDNS announcements only (no 5540 noise)

ssh zhiqiangf@192.168.55.5 "tcpdump -ni br0 -s 0 -tt 'udp port 5353'"

Rotating capture with timestamped filename (multiple runs)

ssh zhiqiangf@192.168.55.5 "tcpdump -ni br0 -s 0 -C 5 -W 12 -w /tmp/matter-\$(date +%H%M%S).pcap 'udp port 5353 or tcp port 5540 or tcp port 5552'"

The nested $(date) must be escaped as \$(...) inside the double-quoted ssh command so the remote shell expands it.

For the hass point, prefix the same commands with ssh hassio@hass.windy.lan "sudo -n -i tcpdump -ni end0 …".

Capture point: gw switch0

Status: verified 2026-08-22 — limited capture point; NOT a full mirror of LAN55.

  • gw switch0 (eth1–eth3, 192.168.55.254/24) is LAN55's L2 aggregation only while devices plug directly into the ER-X. EdgeOS ships tcpdump; tcpdump -ni switch0 follows Linux bridge semantics.
  • Live topology (verified 2026-08-22): the SE5420 core switch is deployed (management 192.168.66.253 up — TP-Link OUI f8:c9:03, web UI on :80/:443) and the ER-X uplink is a single switch0 member port: eth1 link up, eth2/eth3 down. All LAN55 wired devices (hass .11, Aqara M3 .248, SmartThings .48, UAP-AC-Lite .5) are reached via switch0 behind that one uplink. Same-segment wired↔wired unicast switches locally on the SE5420 and never reaches switch0.
  • What switch0 still sees: cross-subnet (66↔55) unicast, traffic to/from the gateway itself (DHCP, DNS forwarding, port-forwards), and LAN55 mDNS multicast (flooded up the uplink). Use it only for those flows; for a full commissioning conversation use the hass end0 or AP br0 point instead.
  • Full mirror: only via SE5420 port mirroring (the switch cannot run tcpdump). Not configured; out of scope here.
  • Verification commands (EdgeOS v3.0.1 build 5862409):
    • Interactive: ssh ubnt@192.168.66.254 (or zhiqiang), then show interfaces ethernet — port link states are the decisive check (eth1 up + eth2/eth3 down = single uplink). configure (config mode) also accepts show ....
    • Non-interactive (agent/script): show/configure are interactive-only aliases on this build; use the op wrapper: 
      ssh ubnt@192.168.66.254 '/opt/vyatta/bin/vyatta-op-cmd-wrapper show interfaces ethernet'
    • show ethernet-switch port all and show mac-address-table are NOT available on this build; the switch FDB is hardware-offloaded (brctl showmacs switch0 → "Operation not supported"). Port link state
      • ARP (show arp) are the reliable checks.
    • SE5420 liveness: ping 192.168.66.253 and :80/:443.
  • Sample capture at this point (cross-segment/gateway/mDNS flows only; tcpdump needs root — zhiqiang has passwordless sudo): 
    ssh zhiqiang@192.168.66.254 "sudo -n tcpdump -ni switch0 -s 0 'udp port 5353 or tcp port 5540 or tcp port 5552'"

Pass criteria

  • Realtime capture consistently shows the target device's mDNS announcements (_matterc / _matter._tcp) on the chosen point.
  • Commissioning shows the TCP 5540 handshake (SYN/SYN-ACK/ACK); on the hass end0 point this includes wired Thread-relay commissioning (HA↔M3), which the AP point cannot see.
  • Saved pcap opens in Wireshark and filters by mDNS / tcp.port==5540.

References

Reference in New Issue View Git Blame Copy Permalink