/cmd/main/main.go              # Application entry point
/internal
    /port                      # Port layer (interfaces/contracts)
        repository.go          # Repository interface
        publisher.go           # Publisher interface
    /domain                    # Domain models (pure Go types, no external dependencies)
        aviation.go            # Aviation domain types (FPL, DEP, ARR, etc.)
    /app                       # Application layer (business logic orchestration)
        processor.go           # Message processor
    /adapter                   # Adapter layer (implementations)
        /parser                # Message parsing adapters
        /mapper                # Data mapping (domain ↔ infrastructure)
        /dto                   # Data Transfer Objects
            telegram.go        # ParsedTelegram and MessageStatus
    /infra                     # Infrastructure layer
        /config                # Configuration management (Koanf)
        /nats                  # NATS JetStream client
        /postgres              # PostgreSQL repository (pgx)
        /log                   # Logging (Zap)
        /metrics               # Prometheus metrics
        /telemetry             # OpenTelemetry tracing
/pkg/di                        # Dependency injection (Wire)

Features

Clean Architecture: Clear separation between domain, application, and infrastructure layers
NATS JetStream: Reliable message streaming with automatic retries and dead-letter queues
TimescaleDB (PostgreSQL): High-performance persistence using pgx with batch operations and hypertables
Dependency Injection: Google Wire for compile-time dependency injection
Configuration Management: Koanf for flexible configuration loading (file + environment variables)
Structured Logging: Zap logger with configurable levels and formats
Batch Processing: Efficient batch message processing and database inserts
AFTN Protocol Validation: Optional AFTN/ICAO protocol compliance validation with detailed error tracking
Serial Reader Health Monitoring: Automatic detection of message flow interruptions and sequence gaps

AFTN Protocol Validation

The system includes comprehensive AFTN (Aeronautical Fixed Telecommunication Network) protocol validation to ensure incoming telegrams comply with ICAO standards.

Features

Protocol Field Validation
- Priority indicators: FF (Flash), GG (Immediate), QU (Distress), DD (Delay), SS (Service), KK (Correction)
- ICAO addresses: 4-character alphanumeric validation
- DateTime formats: DDHHMM with range validation (DD:01-31, HH:00-23, MM:00-59)
Serial Reader Health Monitoring
- Automatic detection of message flow interruptions
- Configurable gap threshold (default: 2 minutes)
- Message sequence gap detection (missing sequence numbers)
- Real-time health status metrics
Dead-Letter Queue (DLQ) Routing
- Invalid telegrams automatically routed to DLQ for offline review
- Detailed error categorization (priority_indicator, icao_address, datetime, multiple_errors)
- Preserves original message content for debugging
Observability
- Prometheus metrics for validation errors, message gaps, and health status
- Structured logging with error context
- OpenTelemetry tracing integration

Configuration

AFTN validation is disabled by default for safe rollout. Enable it in your configuration file:

[aftn]
# Enable AFTN protocol validation
validation_enabled = true

# Time threshold after which serial reader is considered stalled
message_gap_threshold = "2m"

# Enable detection of missing message sequence numbers
enable_sequence_gap_detection = true

Metrics

The system exposes the following Prometheus metrics:

Metric	Type	Description
`caatsm_aftn_validation_errors_total`	Counter	Total AFTN validation errors by error_type label
`caatsm_message_gap_seconds`	Gauge	Time in seconds since last message received
`caatsm_message_sequence_gap_total`	Counter	Number of detected sequence gaps (missing messages)
`caatsm_serial_reader_healthy`	Gauge	Health status: 1=healthy, 0=stalled

Example Prometheus queries:

# AFTN validation error rate
rate(caatsm_aftn_validation_errors_total[5m])

# Current message gap
caatsm_message_gap_seconds{stream="TELEGRAM", consumer="telegram-consumer"}

# Serial reader health
caatsm_serial_reader_healthy{stream="TELEGRAM", consumer="telegram-consumer"}

# Sequence gap rate
rate(caatsm_message_sequence_gap_total[5m])

Alerts

Pre-configured Prometheus alert rules are available in configs/prometheus-alerts.yml:

SerialReaderStalled (critical): No messages for > 2 minutes
HighMessageGap (warning): Gap > 60 seconds
MessageSequenceGaps (warning): Missing sequence numbers detected
HighAFTNValidationErrorRate (warning): > 5% of messages failing validation
ConsumerLagGrowing (warning): Pending messages increasing
ConsumerCriticallyBehind (critical): > 5000 pending messages

To install the alerts:

# Copy alerts to Prometheus server
cp configs/prometheus-alerts.yml /path/to/prometheus/rules/

# Add to prometheus.yml
rule_files:
  - "rules/prometheus-alerts.yml"

# Reload Prometheus
curl -X POST http://localhost:9090/-/reload

Grafana Dashboard

A pre-built Grafana dashboard is available in configs/grafana-dashboard-aftn.json with panels for:

Serial reader health status (stat panel with color coding)
Message gap time series
AFTN validation errors by type
AFTN error rate percentage
Sequence gap rate
Consumer pending messages
Message processing throughput by status

To import the dashboard:

Open Grafana UI
Navigate to Dashboards → Import
Upload configs/grafana-dashboard-aftn.json
Select your Prometheus datasource
Click "Import"

Error Handling

When AFTN validation fails:

Message Status: Set to aftn_error
Error Recording: Error details stored in error_reason field
DLQ Routing: Message published to DLQ subject (if configured)
Metrics: Validation error counter incremented with error type label
Logging: Warning logged with error context and message preview
Tracing: Error recorded in OpenTelemetry span

Example log entry:

{
  "level": "warn",
  "msg": "AFTN validation failed",
  "status": "aftn_error",
  "error_type": "priority_indicator",
  "content_preview": "ZCZC TMQ2526 141605\nXX ZBTJZPZX\n...",
  "error": "AFTN validation error [priority_indicator]: must be one of FF, GG, QU, DD, SS, KK (value: \"XX\")"
}

Testing AFTN Validation

To test AFTN validation in development:

# Enable validation in config.dev.toml
[aftn]
validation_enabled = true

# Start the application
make run

# Send a telegram with invalid priority indicator
# The message will be rejected and routed to DLQ

# Check DLQ for rejected messages
nats sub caatsm.dlq

# Check metrics
curl http://localhost:2112/metrics | grep aftn

Disabling AFTN Validation

AFTN validation can be disabled at runtime without code changes:

[aftn]
validation_enabled = false  # Disable validation

This allows for gradual rollout and quick rollback if issues arise.

Contributor Guide

For coding standards, test expectations, and release hygiene, read AGENTS.md.

Prerequisites

Go 1.22+
PostgreSQL 12+
NATS Server with JetStream enabled

Installation

Clone the repository:

git clone <repository-url>
cd go-caatsm

Install dependencies:

go mod download

Set up PostgreSQL database:

psql -U postgres -f internal/infra/postgres/telegrams.ddl

Configure the application:
- Copy configs/config.dev.toml and modify as needed
- Or set environment variables with CAATSM_ prefix

Configuration

Configuration is loaded from TOML files and environment variables. The configuration file should be located at configs/config.{env}.toml where {env} is determined by the GO_ENV environment variable (defaults to dev).

nats.url and nats.stream are required (the latter defaults to TELEGRAM when omitted).
subscription.topic is optional; when not provided the application subscribes to telegram.>.

Configuration Structure

[nats]
url = "nats://localhost:4222"
# mode: "jetstream" (required - only JetStream mode is supported)
mode = "jetstream"
stream = "TELEGRAM"
consumer = "telegram-consumer"
client = "serial-client"
cluster = "tele-cluster"

[nats.stream_limits]
# Stream retention and storage limits
max_msgs = 100000
max_bytes = 67108864
max_age = "24h"
discard = "old"
storage = "file"
replicas = 1

[nats.consumer_rules]
# Consumer delivery rules
# max_deliver: Maximum number of delivery attempts before giving up
max_deliver = 3
# ack_wait: Time to wait for ACK before redelivering message
ack_wait = "30s"
# max_ack_pending: Maximum number of unacknowledged messages before pausing delivery
max_ack_pending = 1000

[subscription]
# Optional. Defaults to "telegram.>" when omitted.
topic = "telegram.serial"
# queue_group: Queue group for load balancing (JetStream uses durable consumers)
queue_group = "tele-queue"

[publisher]
topic = "telegram.json"

[postgres]
url = "postgres://user:password@localhost:5432/aviation?sslmode=disable"
max_conns = 10
min_conns = 2

[app]
batch_size = 50
batch_timeout = "2s"
monitor_interval = "30s"

[log]
level = "info"
format = "json"

[telemetry]
enabled = false
endpoint = "http://otel-collector:4318"
insecure = true

# Production configuration example:
# enabled = true
# endpoint = "otel-collector.company.com:4318"
# insecure = false  # Use TLS in production

### Timeouts and Ack Wait

`[timeouts]` is optional, but if you plan to tune JetStream redelivery you should set `timeouts.ack_wait` and/or `[nats.consumer].ack_wait`. When neither is specified the application defaults both values to `30s`, ensuring predictable redelivery timing.

Environment Variables

You can override any configuration value using environment variables with the CAATSM_ prefix:

export CAATSM_NATS_URL="nats://nats-server:4222"
export CAATSM_POSTGRES_URL="postgres://user:pass@db:5432/aviation"
export CAATSM_LOG_LEVEL="debug"

Environment variable names are converted from CAATSM_NATS_URL to nats.url in the configuration.

NATS Mode Selection

The application supports two NATS consumption modes, controlled by nats.mode:

JetStream Mode (Recommended for Production)

Configuration: nats.mode = "jetstream" (default)

Features:

✅ Message Persistence: Messages are stored in a Stream, allowing replay and recovery
✅ ACK/NAK Mechanism: Explicit message acknowledgment ensures guaranteed delivery
✅ Error Handling: Failed messages are handled with simple backoff
✅ Dead-Letter Queue: Poison messages can be routed to a DLQ for inspection
✅ Batch Processing: Efficient batch fetching and processing
✅ Consumer Monitoring: Real-time metrics for consumer lag and pending messages
✅ At-Least-Once Delivery: Messages are guaranteed to be delivered at least once

Use Cases:

Production environments requiring message reliability
Scenarios where message loss is unacceptable
Systems needing message replay capabilities
Applications requiring retry logic for transient failures

Configuration Requirements:

Requires JetStream to be enabled on the NATS server
Stream must be created (auto-created in dev/test environments)
Consumer configuration via [nats.consumer_rules] section

How to Use JetStream Mode:

Prerequisites:
- Ensure NATS server has JetStream enabled (default in docker-compose.dev.yml)
- Set nats.mode = "jetstream" in your config file (or use CAATSM_NATS_MODE=jetstream)

Start NATS with JetStream:

# Using Docker Compose (recommended for development)
docker compose -f docker-compose.dev.yml up -d nats

# Or start NATS server manually with JetStream enabled:
# nats-server -js

Configure Stream and Consumer: The application automatically creates the Stream and Consumer on startup in dev/test environments (GO_ENV=dev or GO_ENV=test). In production, you may need to create them manually or ensure they exist.

Stream Configuration ([nats.stream_limits]):
- max_msgs: Maximum number of messages in the stream (default: 100000)
- max_bytes: Maximum total size of messages (default: 64MB)
- max_age: Maximum age of messages before deletion (default: 24h)
- storage: "file" (persistent) or "memory" (ephemeral)
- replicas: Number of stream replicas for HA (default: 1, use 3+ for production)
Consumer Configuration ([nats.consumer_rules]):
- max_deliver: Maximum delivery attempts before giving up (default: 3)
- ack_wait: Time to wait for ACK before redelivery (default: 30s)

Start the Application:

# Development mode (auto-creates stream/consumer)
GO_ENV=dev \
CAATSM_NATS_MODE=jetstream \
CAATSM_POSTGRES_URL=postgres://user:pass@localhost:5432/aviation \
  go run ./cmd/main listen

# Production mode (requires stream/consumer to exist)
GO_ENV=prod \
CAATSM_NATS_MODE=jetstream \
  ./bin/receiver listen

Publish Messages to JetStream:

# Using nats-box (included in docker-compose.dev.yml)
docker compose exec nats-box nats pub telegram.serial "ZCZC TEST123 150631..."

# Or use the seed-telegrams tool with JetStream
go run ./cmd/seed-telegrams \
  --nats-url nats://localhost:4222 \
  --jetstream \
  --stream TELEGRAM \
  --js-subject telegram.serial \
  --count 10

Monitor JetStream:

# View stream info
docker compose exec nats-box nats stream info TELEGRAM

# View consumer info
docker compose exec nats-box nats consumer info TELEGRAM telegram-consumer

# View pending messages
docker compose exec nats-box nats consumer next TELEGRAM telegram-consumer

# Or use NATS monitoring UI at http://localhost:8222

Message Processing Flow:
- Messages are published to the configured subject (e.g., telegram.serial)
- Stream stores messages according to retention policy
- Consumer pulls messages in batches (configurable via app.batch_size)
- Each message is processed and ACKed on success
- Failed messages are NAKed and redelivered with simple backoff
- After max_deliver attempts, permanent failures are routed to DLQ (if enabled)

Replay Messages:

# Replay from a specific sequence
./bin/receiver listen --replay-from seq:12345

# Replay from a specific time
./bin/receiver listen --replay-from time:2024-11-15T08:00:00Z

Dead-Letter Queue (DLQ): Enable DLQ in config to route poison messages:
```
[dlq]
enabled = true
subject = "caatsm.dlq"
```
Messages that fail after max_deliver attempts are published to the DLQ subject for manual inspection.
Troubleshooting:
- Stream not found: Ensure GO_ENV=dev for auto-creation, or create manually in production
- Consumer not found: Application auto-creates consumer on startup
- Messages not being consumed: Check consumer info for pending messages and delivery status
- High pending count: Increase batch_size or add more consumer instances
- Messages being redelivered: Check processing logs for errors; adjust ack_wait if processing takes longer

Usage

Build

The build process automatically injects build information (version, commit, build time) into the binary. This information is available via the /livez and /readyz health endpoints.

Using Make (writes bin/receiver):

make build
# Or with custom version:
VERSION=v1.0.0 make build

Using Task:

task build
# Or with custom version:
VERSION=v1.0.0 task build

Or directly with Go:

# With build info injection:
go build -ldflags "-X 'caatsm/internal/infra/buildinfo.Version=dev' -X 'caatsm/internal/infra/buildinfo.Commit=$(git rev-parse --short HEAD)' -X 'caatsm/internal/infra/buildinfo.BuiltAt=$$(go run - <<'EOF'
package main
import (
  \"fmt\"
  \"time\"
)
func main() {
  fmt.Print(time.Now().UTC().Format(time.RFC3339))
}
EOF)'" -o bin/receiver ./cmd/main

Build information is automatically populated from:

Version: VERSION environment variable (defaults to "dev")
Commit: Git commit hash (short format)
BuiltAt: UTC timestamp of build time

Run

Development Mode

Use Make targets (binary mode):

make run-dev    # GO_ENV=dev (uses JetStream mode)
make run-local  # go run ./cmd/main listen (honors GO_ENV)

Task equivalents:

task run-dev
task run-local         # go run ./cmd/main listen
task dev-run           # boots docker-compose dev stack + go run (JetStream mode)

Production Mode

For production deployment, see the comprehensive guide: Production Deployment Guide

Quick start:

# Build the binary with version information
VERSION=v1.0.0 make build

# Run in production mode
make run-prod   # GO_ENV=prod (requires config.prod.toml)

Key requirements:

JetStream mode (mandatory)
Stream and Consumer must be created manually
Production configuration file: configs/config.prod.toml
SSL/TLS for secure connections
NATS authentication configured (see docs/prod-guide.md#nats-authentication)

See docs/prod-guide.md for complete production deployment instructions, including:

NATS authentication setup
Database migrations (see docs/migrations.md)
Secret management (see docs/secret-management.md)
Performance tuning (see docs/performance.md)

Command Line Options

./bin/receiver listen --help

Flags:
  -n, --nats-url string          NATS server address
  -t, --subject string           NATS subject to listen to
      --stream string            JetStream stream name
      --consumer string          JetStream durable consumer
      --publisher-topic string   Subject used by the publisher
      --postgres-url string      PostgreSQL connection URL
      --log-level string         Logger level (debug|info|warn|error)
      --replay-from string       Deliver policy override (all|new|last|seq:<n>|time:<RFC3339>)
      --ack-wait duration        Ack wait override (e.g. 45s)
      --telemetry-enabled        Enable OpenTelemetry exporters
      --telemetry-endpoint string
                                 OTLP collector endpoint
      --telemetry-insecure       Send OTLP traffic without TLS

Critical overrides stay available through CLI flags; configuration is managed via the TOML file or CAATSM_ environment variables.

CLI flag	Config key	Purpose
`--nats-url`	`nats.url`	Point to a different NATS cluster
`--subject`	`subscription.topic`	Change the subscribed subject filter
`--stream`	`nats.stream`	Bind to another JetStream stream
`--consumer`	`nats.consumer`	Override the durable consumer name
`--publisher-topic`	`publisher.topic`	Publish parsed output to a new subject
`--postgres-url`	`postgres.url`	Redirect persistence to another DB
`--log-level`	`log.level`	Adjust runtime logging verbosity
`--telemetry-*`	`telemetry.*`	Toggle tracing/metrics exporters

Replay & Backoff

--replay-from seq:12345 replays from a specific JetStream sequence, while --replay-from time:2024-11-15T08:00:00Z starts at a timestamp.
Configure retry behavior with [nats.consumer_rules] settings.

Observability

The processor exposes three complementary observability surfaces with production-ready OpenTelemetry implementation:

OpenTelemetry (traces + metrics)
- Production-ready setup with environment-based sampling, comprehensive resource attributes, and optimized batching.
- Enable via [telemetry] enabled = true and set endpoint to your OTLP/HTTP collector (e.g., http://otel-collector:4318).
- Sampling strategy:
  - Production: 1% sampling (cost-effective)
  - Staging: 10% sampling (balanced observability)
  - Development/Test: 100% sampling (full debugging)
- CLI overrides:
  - --telemetry-enabled toggles exporters on/off.
  - --telemetry-endpoint and --telemetry-insecure adjust the OTLP HTTP endpoint and TLS behavior.
- Comprehensive traces with semantic attributes:
  - caatsm/app: Message processing spans with messaging.system, messaging.operation, caatsm.message.category
  - caatsm/postgres: Database operations with db.system, db.operation, db.table
  - caatsm/nats: NATS operations with messaging.destination, messaging.consumer.id
- Business metrics (focused set for OTEL):
  - caatsm_messages_processed_total (counter, by message.status / message.category)
  - caatsm_publish_failures_total (counter)
  - caatsm_parse_duration_seconds (histogram)
- Resource attributes include service metadata, environment, build info, and infrastructure details.
- Application code records telemetry via a thin telemetry.Recorder abstraction, which fans out to OTEL and Prometheus backends as configured.
Prometheus metrics (/metrics)
- Implemented in internal/infra/metrics and considered the primary source for SRE PromQL/SLOs.
- Key metric families:
  - caatsm_messages_total{stream,consumer,result} – per-stream/consumer throughput and results.
  - caatsm_handle_latency_seconds_bucket{stream,consumer} – end-to-end handling latency from NATS receive to handler completion.
  - caatsm_retries_total{stream,consumer,reason} – JetStream retry/NAK counts.
  - caatsm_db_queries_total{operation,result} and caatsm_db_query_latency_seconds_bucket{operation} – DB activity and latency.
  - caatsm_dlq_messages_total{stream,consumer} and caatsm_dlq_publish_failures_total{stream,consumer} – DLQ routing success/failures.
  - caatsm_nats_consumer_pending_messages{stream,consumer} – JetStream consumer backlog/lag.
- Prometheus scrapes GET /metrics on the monitoring server; Grafana dashboards under configs/grafana-dashboards.dev are wired to these series.
Health and readiness endpoints
- A lightweight monitoring server exposes:
  - GET /livez – liveness endpoint: reports process and build information, does not call external dependencies.
  - GET /readyz – readiness endpoint: pings PostgreSQL and checks NATS connection status within monitoring.health_timeout, returning 503 on failure.
  - GET /healthz – backward-compatible alias currently sharing logic with /readyz.
- Responses include build metadata and dependency status/latency (see docs/observability.md for examples).
- Configure the server via the [monitoring] block (defaults shown):

[monitoring]
addr = ":2112"
enable_metrics = true
enable_health = true
read_timeout = "5s"
write_timeout = "5s"
health_timeout = "2s"

Set monitoring.disabled = true (or addr = "") if you need to turn the HTTP server off, e.g., during certain integration tests.

Development

See docs/dev-guide.md for the full development workflow, including Docker Compose instructions, observability tooling, and troubleshooting tips.

Quick start:

# Start database + messaging
docker compose -f docker-compose.dev.yml up -d postgres nats nats-box

# Start observability stack (optional)
docker compose -f docker-compose.dev.yml up -d otel-collector jaeger prometheus grafana

Run the processor locally while the infra runs in Docker:

# Development mode (JetStream enabled)
GO_ENV=dev \
CAATSM_POSTGRES_URL=postgres://caatsm:caatsm@localhost:5432/aviation?sslmode=disable \
  go run ./cmd/main listen

# JetStream mode (for integration testing or production-like behavior)
GO_ENV=dev \
CAATSM_NATS_MODE=jetstream \
CAATSM_POSTGRES_URL=postgres://caatsm:caatsm@localhost:5432/aviation?sslmode=disable \
  go run ./cmd/main listen

Tear everything down with docker compose -f docker-compose.dev.yml down -v.

Deployment Examples

For production deployment, see the comprehensive guide: Production Deployment Guide

Additional deployment-specific guides:

docs/prod-guide.md - Complete production deployment guide with configuration, setup, and operations
docs/deploy-systemd.md - Systemd service deployment with environment file configuration
docs/deploy-k8s.md - Kubernetes deployment with ConfigMap/Secret and health probes

Additional Documentation

docs/migrations.md - Database migration strategy and best practices
docs/secret-management.md - Secret management best practices and integration guides
docs/performance.md - Performance tuning guidelines and optimization strategies
docs/observability.md - Observability setup and metrics documentation
docs/nats.md - NATS/JetStream configuration and usage guide
docs/reliability.md - Reliability patterns and error handling

Project Structure

Domain Layer (internal/domain): Pure business logic and domain models
Application Layer (internal/app): Orchestrates business flows
Adapter Layer (internal/adapter): Interfaces and adapters between layers
Infrastructure Layer (internal/infra): External concerns (NATS, PostgreSQL, config, logging)

Adding New Features

Domain Changes: Add to internal/domain (no external dependencies)
Business Logic: Add to internal/app
External Integrations: Add to internal/infra
Adapters: Add to internal/adapter to bridge between layers

Dependency Injection

Dependencies are managed using Google Wire. To add a new dependency:

Create a provider function in the appropriate package
Add it to pkg/di/wire.go
Run wire ./pkg/di to regenerate wire_gen.go

If the wire binary is missing, install it with go install github.com/google/wire/cmd/wire@v0.7.0 and ensure $GOPATH/bin is on your PATH (or run it directly via the absolute path).

Testing

The project keeps tests close to the code that they exercise:

Domain/adapter/app unit tests live under internal/** and cover parsing, validation, orchestration, and adapters. Run them all with task test (or make test), which now uses the Ginkgo CLI to run unit test suites in verbose mode (ginkgo -r -v ./cmd ./internal).
Integration tests under test/integration spin up disposable TimescaleDB and NATS JetStream instances (via testcontainers-go) and execute a full ingestion flow. Use task test-int after ensuring Docker is running.
Coverage goals are tracked via task coverage, which produces both a coverage profile and an HTML report under coverage/coverage.html.

Benchmark tests are available for performance-critical components:

# Run parser benchmarks
go test -bench=BenchmarkParse -benchmem ./internal/adapter/parser

# Run repository benchmarks (requires DB connection)
go test -bench=BenchmarkMapper -benchmem ./internal/infra/postgres

# Run processor benchmarks
go test -bench=BenchmarkHandle -benchmem ./internal/app

Purpose	Make command	Task command
Run unit tests (Ginkgo)	`make test`	`task test`
Run integration tests	`make test-int`	`task test-int`
Run unit+integration tests	`make test-all`	`task test-all`
Generate coverage html	`make coverage`	`task coverage`
Lint (golangci-lint)	`make lint`	`task lint`

Integration tests need Docker available on the host. Ginkgo-based unit tests or lint targets require the respective binaries (go install github.com/onsi/ginkgo/v2/ginkgo@latest, golangci-lint install guide). Use task install-test to bootstrap Ginkgo tooling before running task test, task test-all, or their make equivalents. make test / task test run Ginkgo in verbose mode (-v) over ./cmd and ./internal, showing each spec for easier debugging.

Message Flow

NATS Consumer receives raw telegram messages from NATS JetStream:
- Uses durable pull consumer with batch processing, ACK/NAK, and retry logic
MessageProcessor orchestrates the processing:
- Parses the message using the Parser adapter
- Stores the parsed message in PostgreSQL via Repository
- Publishes the parsed message to the output topic via Publisher
ACK/NAK is sent based on processing success/failure
Retry Logic handles transient failures automatically

Error Handling & Retries

Parser failures (invalid headers/body) are treated as permanent: the raw payload is stored in aviation.telegrams_raw, the message is ACKed, and no JetStream retries are attempted.
Repository failures are transient: the consumer returns an error, the message is NAKed, and JetStream redelivers it with simple backoff.
Publisher failures are logged and persisted as raw records, but they are marked permanent to avoid hammering downstream topics; the deduplicated output can be replayed from the raw table later.
Tune JetStream retry behavior via [nats.consumer_rules.max_deliver], [nats.consumer_rules.backoff], and CLI overrides like --ack-wait. The monitoring server plus Prometheus counters provide visibility into each failure bucket.

Failure Buckets

Messages that cannot be parsed or fail to publish are written to aviation.telegrams_raw with a status:

Status	Description
`parsed`	Successfully parsed and stored
`header_error`	Header invalid (missing start indicator, etc.)
`body_error`	Body pattern did not match any known format
`publish_error`	Downstream publisher returned an error

Each entry stores the raw payload, received timestamp, and metadata to aid replay or manual inspection.

Message Parsing

The system supports parsing of aviation telegram messages in the standard ICAO format. All messages follow a common header structure, followed by a message body that varies by message type.

Message Format

All telegrams follow this general structure:

ZCZC <MessageID> <DateTime>
<PriorityIndicator> <PrimaryAddress>
<SecondaryAddresses>
<Originator>
<Body>
NNNN

Header Fields:

ZCZC: Start indicator
MessageID: Unique message identifier (e.g., "TMQ1324")
DateTime: Message date and time (e.g., "150631")
PriorityIndicator: Message priority (e.g., "FF", "DD")
PrimaryAddress: Primary recipient address (ICAO code)
SecondaryAddresses: Additional recipient addresses
Originator: Message originator (optional)

Supported Message Types

The parser supports the following message categories:

1. ARR - Arrival Message

Arrival messages report aircraft arrival information.

Format:

(ARR-<FlightNumber>[/<SSR>]-<DepartureAirport>-<ArrivalAirport><ArrivalTime>)

Parsed Fields:

category: "ARR"
aircraft_id: Aircraft identification/flight number
ssr_mode_and_code: SSR mode and code (optional)
departure_airport: Departure airport ICAO code
departure_time: Departure time
arrival_airport: Arrival airport ICAO code
arrival_time: Arrival time
estimated_elapsed_time: Estimated flight duration (optional)
alternate_airport: Alternate airport (optional)
other_info: Additional information (optional)

Example:

ZCZC ARR1234 150631
FF ZBTJZPZX
150630 ZBACZQZX
(ARR-CCA1234-A1234-ZBTJ1500-ZGGG0135)
NNNN

2. DEP - Departure Message

Departure messages report aircraft departure information.

Format:

(DEP-<FlightNumber>[/<SSR>]-<DepartureAirport><DepartureTime>-<Destination>)

Parsed Fields:

category: "DEP"
aircraft_id: Aircraft identification/flight number
ssr_mode_and_code: SSR mode and code (optional)
departure_airport: Departure airport ICAO code
departure_time: Departure time
destination: Destination airport ICAO code
estimated_elapsed_time: Estimated flight duration
alternate_airport: Alternate airport (optional)
other_info: Additional information (optional)

Example:

ZCZC DEP5678 120915
DD KLAXZPZX
120914 KSFOZQZX
(DEP-ABC5678-A1234-ZBTJ1440-ZGGG)
NNNN

3. CNL - Cancellation Message

Cancellation messages indicate flight cancellations.

Format:

(CNL-<FlightNumber>-<DepartureAirport>-<DestinationAirport>)

Parsed Fields:

category: "CNL"
aircraft_id: Aircraft identification/flight number
departure_airport: Departure airport ICAO code
destination_airport: Destination airport ICAO code
other_info: Additional information (optional)

Example:

ZCZC CNL9012 150631
FF ZBTJZPZX
(CNL-CCA9012-ZBTJ-ZGGG)
NNNN

4. DLA - Delay Message

Delay messages report flight delays with new departure times.

Format:

(DLA-<FlightNumber>[/<SSR>]-<DepartureAirport>[<NewDepartureTime>]-<ArrivalAirport>[<ArrivalTime>])

Parsed Fields:

category: "DLA"
aircraft_id: Aircraft identification/flight number
ssr_mode_and_code: SSR mode and code (optional)
departure_airport: Departure airport ICAO code
new_departure_time: New departure time (optional)
arrival_airport: Arrival airport ICAO code
arrival_time: Estimated arrival time (optional)
other_info: Additional information (optional)

Example:

ZCZC DLA3456 150631
FF ZBTJZPZX
(DLA-CCA3456-A1234-ZBTJ1600-ZGGG0200)
NNNN

5. FPL - Flight Plan Message

Flight plan messages contain detailed flight planning information.

Format:

(FPL-<FlightNumber>-<Indicator>
-<AircraftID>/<SSR>
-<DepartureAirport><DepartureTime>
-<Speed><Level> <Route>
-<Destination><EstimatedTime> <AlternateAirport>
-<OtherInfo>)

Seed Tool: `cmd/seed-telegrams`

seed-telegrams 是一个开发/测试用的报文发生器，用来向 NATS/JetStream 持续或突发地发送合成电报（ARR/DEP/CNL/DLA/FPL），用于驱动解析与下游流水线。

支持的类别与内容

生成的电报遵循与解析器相同的格式约定：

ARR / DEP：支持无 SSR、合法简单 SSR，以及刻意构造为“当前正则无法解析”的复杂 SSR。
CNL / DLA：与 internal/adapter/parser/aviation_parser_test.go 中的测试样例同一类结构。
FPL：生成包含多行 route 与 OtherInfo 字段的完整 FPL，OtherInfo 中会随机组合 PBN/, NAV/, REG/, EET/, SEL/, PER/, RIF/, RMK/ 等片段，以覆盖解析逻辑。

命令行参数

常用参数：

--nats-url：NATS 地址（默认 nats://127.0.0.1:4222，为空字符串则完全不连接 NATS）。
--subject：普通 NATS 发布 subject（默认 telegram.raw）。
--jetstream：是否使用 JetStream 发布。
--stream / --js-subject：JetStream 相关选项。
--count：要发送的电报数量（mode=burst 或有上限的 interval/mixed 时生效）。
--category：ARR|DEP|CNL|DLA|FPL|mixed，mixed 表示在五类中随机选择。
--status：parsed|header_error|body_error|publish_error|repository_error|random。
- random 模式下，合法报文偏向标记为 parsed，刻意非法报文偏向标记为 body_error。
--error-reason：错误原因说明，将写入元数据 header（默认 synthetic test payload）。
--dry-run：只打印电报内容，不真正发布到 NATS。
--header-format：json|none，控制是否以 JSON 形式附加元数据 header。
--mode：seed 模式：
- burst：一次性快速发送完 count 条。
- interval：按照给定时间间隔持续发送。
- mixed：先按 interval 发送一部分，再以 burst 方式发送剩余。
--interval-min / --interval-max：interval/mixed 模式下两条电报之间的最小/最大间隔（默认 1s / 2s）。
--duration：interval/mixed 模式下的总持续时间，0 表示仅按 count 控制停止条件。

使用示例

1. 一次性快速打 100 条（突发流量）

go run ./cmd/seed-telegrams \
  --count=100 \
  --mode=burst \
  --category=mixed \
  --status=random

2. 模拟真实流量：每 1–2 秒发一条，持续 5 分钟

go run ./cmd/seed-telegrams \
  --mode=interval \
  --interval-min=1s \
  --interval-max=2s \
  --duration=5m \
  --status=random \
  --category=mixed

3. 慢热 + 突发：前半段慢慢发，后半段瞬间打完

go run ./cmd/seed-telegrams \
  --count=200 \
  --mode=mixed \
  --interval-min=500ms \
  --interval-max=1500ms \
  --status=random

4. 只打印合成电报，不发送（本地调试报文格式）

go run ./cmd/seed-telegrams \
  --count=5 \
  --mode=burst \
  --dry-run \
  --category=FPL

5. 持续缓慢发送（直到 Ctrl-C）

# 使用 Makefile/Taskfile 任务（推荐）
make seed-slow
# 或
task seed-slow

# 自定义间隔时间
make seed-slow INTERVAL_MIN=3s INTERVAL_MAX=8s

# 直接使用命令行
go run ./cmd/seed-telegrams \
  --nats-url nats://localhost:4222 \
  --jetstream \
  --stream TELEGRAM \
  --js-subject telegram.serial \
  --mode interval \
  --interval-min 2s \
  --interval-max 5s \
  --count 0

注意：设置 --count 0 会持续发送直到手动停止（Ctrl-C）。这对于长时间测试和监控系统行为非常有用。

该工具专门为开发与测试设计，不影响生产服务逻辑，推荐在本地或测试环境配合解析与存储流水线一起使用，用于回归测试、吞吐量观察和错误场景演练。

Parsed Fields:

category: "FPL"
flight_number: Flight number
reference_data: Reference data (optional)
aircraft_id: Aircraft identification
ssr_mode_and_code: SSR mode and code
flight_rules_and_type: Flight rules and type
cruising_speed_and_level: Cruising speed and flight level
departure_airport: Departure airport ICAO code
departure_time: Departure time
route: Flight route
destination_and_total_time: Destination and estimated total time
alternate_airport: Alternate airport (optional)
estimated_arrival_time: Estimated arrival time
pbn: Performance-based navigation equipment
navigation_equipment: Navigation equipment
estimated_elapsed_time: Estimated elapsed time
selcal_code: SELCAL code
register: Aircraft registration (optional)
performance_category: Performance category
reroute_information: Reroute information (optional)
remarks: Remarks (optional)

Example:

ZCZC FPL7890 150631
FF ZBTJZPZX
(FPL-JAE7433-IS
-B744/H-SXIRPZJWY/S
-ZBTJ1755
-K0926S0920 CG A326 VYK W80 HUR B339 GM A575 MANSA/K0919S0980
-EDDF0948 EDDK
-EET/ZMUB0100 UNKL0236
REG/B2422 SEL/JLAD
NAV/RNAV1 RNAV5 RNP4
RMK/AGCS EQUIPPED)
NNNN

Parsing Process

Header Parsing: The parser extracts header information including message ID, date/time, priority, and addresses
Category Detection: The parser identifies the message category from the body content
Body Parsing: Based on the category, the parser applies the appropriate regex pattern to extract structured data
Data Mapping: Extracted data is mapped to domain model structures (ARR, DEP, CNL, DLA, or FPL)
Storage: The parsed message is stored in PostgreSQL with:
- Raw content in content field
- Parsed structured data in body_data field (JSONB)
- Metadata in dedicated columns

Parsed Message Structure

All parsed messages are stored in the ParsedMessage domain model:

type ParsedMessage struct {
    Uuid               string      // Unique identifier
    MessageID          string      // Telegram message ID
    DateTime           string      // Message date/time
    PriorityIndicator  string      // Priority level
    PrimaryAddress     string      // Primary recipient
    SecondaryAddresses string      // Secondary recipients
    Originator         string      // Message originator
    OriginatorDateTime string      // Originator date/time
    Category           string      // Message category (ARR, DEP, CNL, DLA, FPL)
    Content            string      // Raw message content
    BodyData           interface{} // Parsed body data (ARR, DEP, CNL, DLA, or FPL struct)
    ReceivedAt         time.Time   // Reception timestamp
    ParsedAt           time.Time   // Parsing timestamp
    DispatchedAt       time.Time   // Dispatch timestamp
    NeedDispatch       bool        // Dispatch flag
    Parsed             bool        // Parsing success flag
    Comments           string      // Parsing comments/errors
}

Error Handling

Invalid Format: Messages that don't match expected formats are stored with Parsed = false and error details in Comments
Partial Parsing: Header parsing failures result in storing raw content only
Category Mismatch: Unsupported categories are logged and stored with parsing errors

Database Schema

The application uses the aviation.telegrams table. See internal/infra/postgres/telegrams.ddl for the schema definition.

Key fields:

uuid: Primary key (UUID)
message_id: Telegram message ID
content: Raw message content
body_data: Parsed body data (JSONB)
received_at, parsed_at, dispatched_at: Timestamps

Logging

Logging uses Zap with structured logging. Log levels and format can be configured:

Levels: debug, info, warn, error
Formats: json (production) or console (development)

Logs include contextual information:

Message IDs
Subject names
Stream names
Processing attempts

Performance

Batch Processing: Messages are processed in configurable batches (default: 50)
Database Inserts: Uses PostgreSQL COPY FROM for efficient batch inserts via Repository.InsertBatch. The default processor issues single inserts, but you can switch to buffered batches in high-throughput deployments.
Connection Pooling: Configurable PostgreSQL connection pool
JetStream: Reliable message delivery with automatic retries

Troubleshooting

Connection Issues

NATS: Check that NATS server is running and JetStream is enabled
PostgreSQL: Verify database connection string and that the schema exists

Message Processing Issues

Check logs for parsing errors
Verify message format matches expected telegram format
Check database constraints and indexes

Configuration Issues

Ensure GO_ENV is set correctly
Verify configuration file exists at configs/config.{env}.toml
Check environment variable names use CAATSM_ prefix

Migration from Legacy System

This project was refactored from:

Watermill → nats.go JetStream
Hasura GraphQL → PostgreSQL pgx
Viper → Koanf
Manual DI → Google Wire

The legacy code has been removed. See the project history for migration details.

License

This repository has not declared a public license yet.

Contributing

Contribution guidelines are not published; please coordinate changes via pull requests or direct maintainers.

README.md Unescape Escape

go-caatsm

Architecture

Features

AFTN Protocol Validation

Features

Configuration

Metrics

Alerts

Grafana Dashboard

Error Handling

Testing AFTN Validation

Disabling AFTN Validation

Contributor Guide

Prerequisites

Installation

Configuration

Configuration Structure

Environment Variables

NATS Mode Selection

JetStream Mode (Recommended for Production)

Usage

Build

Run

Development Mode

Production Mode

Command Line Options

Replay & Backoff

Observability

Development

Deployment Examples

Additional Documentation

Project Structure

Adding New Features

Dependency Injection

Testing

Message Flow

Error Handling & Retries

Failure Buckets

Message Parsing

Message Format

Supported Message Types

1. ARR - Arrival Message

2. DEP - Departure Message

3. CNL - Cancellation Message

4. DLA - Delay Message

5. FPL - Flight Plan Message

Seed Tool: cmd/seed-telegrams

支持的类别与内容

命令行参数

使用示例

1. 一次性快速打 100 条（突发流量）

2. 模拟真实流量：每 1–2 秒发一条，持续 5 分钟

3. 慢热 + 突发：前半段慢慢发，后半段瞬间打完

4. 只打印合成电报，不发送（本地调试报文格式）

5. 持续缓慢发送（直到 Ctrl-C）

Parsing Process

Parsed Message Structure

Error Handling

Database Schema

Logging

Performance

Troubleshooting

Connection Issues

Message Processing Issues

Configuration Issues

Migration from Legacy System

License

Contributing

README.md

Seed Tool: `cmd/seed-telegrams`