Commit Graph
13 Commits
Author SHA1 Message Date
windyboyandClaude Sonnet 4.5 7b6f6383ad Add AFTN protocol validation and serial reader health monitoring to enhance aviation telegram processing reliability and observability
Implement comprehensive AFTN/ICAO protocol compliance validation with configurable enforcement, enabling early detection of malformed telegrams and reducing downstream processing errors. Add real-time serial reader health monitoring to automatically detect message flow interruptions and sequence gaps, ensuring operational visibility into the telegram ingestion pipeline.

Key enhancements:
- AFTN validator validates priority indicators (FF/GG/QU/DD/SS/KK), ICAO addresses (4-char alphanumeric), and datetime formats (DDHHMM) with detailed error categorization
- Invalid telegrams automatically routed to DLQ with full context for offline review and correction
- Serial reader health monitoring tracks message gaps and sequence numbers to detect stalled readers or missing messages within configurable threshold (default: 2 minutes)
- Four new Prometheus metrics expose validation errors by type, message gaps, sequence gaps, and health status for operational alerting
- Pre-configured Prometheus alert rules for critical conditions (stalled reader, high error rates, consumer lag)
- Grafana dashboard provides real-time visibility into AFTN compliance and serial reader health
- Validation disabled by default for safe rollout with zero breaking changes to existing functionality

Implementation maintains clean architecture with validator in adapter layer, extends processor and consumer with health tracking, and ensures thread-safe concurrent access to tracking state. All changes fully tested with 48 validator tests, 10 processor tests, and 21 consumer tests passing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-24 14:21:59 +08:00
windyboy 6eff35b56f 🔧 Update Go module dependencies and modify NATS configuration to exclusively support JetStream mode. Upgrade go.uber.org/zap to version 1.27.1 and golang.org/x/crypto to version 0.45.0. Revise Makefile and documentation to reflect changes in NATS mode, ensuring consistent messaging behavior across development and production environments. Enhance README and configuration files to clarify JetStream usage and remove references to Core NATS mode. 2025-11-21 09:59:47 +08:00
windyboy 06fc9cb9e0 🔧 Update Go version in go.mod and enhance build process with versioning information. Modify Makefile and Taskfile to inject build metadata (version, commit, build time) into the binary. Improve README with instructions for custom version builds and document new build info features. Add benchmarks for message parsing and processing to improve performance testing capabilities. 2025-11-18 14:15:58 +08:00
windyboy 67abd66fa3 Enhance logging configuration by introducing file output options and rotation settings in config.dev.toml. Update logger implementation to support multiple output streams, including file logging with rotation using lumberjack. Improve error handling and logging across various components, ensuring consistent logging practices. Update .gitignore to include log files and compressed logs. Add new Go module dependency for lumberjack. 2025-11-17 12:58:40 +08:00
windyboy a96cb8a40b Implement dead-letter queue (DLQ) functionality for handling permanent failures in message processing. Update configuration to enable DLQ and specify the subject for routing failed messages. Enhance observability by adding metrics for message handling, retries, and database operations. Introduce structured logging for better traceability of message processing events. 2025-11-16 09:15:37 +08:00
windyboy 53208997e8 Integrate monitoring server for observability, adding health and metrics endpoints. Update configuration to enable monitoring features and enhance README with deployment examples for Kubernetes and systemd. Refactor application initialization to include monitoring server setup and improve error handling in message processing metrics. 2025-11-15 16:30:29 +08:00
windyboy 184c5453fb Expand Docker Compose development stack to include observability tools: OpenTelemetry Collector, Jaeger, Prometheus, and Grafana. Update README with new usage instructions for the observability stack and enhance Taskfile for streamlined development commands. Introduce sample telegram generation utility and improve NATS configuration for core and JetStream modes. 2025-11-15 12:08:26 +08:00
windyboy 4f16bd6d90 Introduce Docker Compose development stack with TimescaleDB and NATS services. Update README with usage instructions for local development. Upgrade Go version to 1.25.0 and update dependencies. Refactor configuration for NATS consumer rules and enhance Telegram database schema with TimescaleDB support. 2025-11-15 09:46:29 +08:00
windyboy 359e7694ab Update dependencies, enhance NATS consumer configuration, and improve error handling in message processing. Introduce telemetry support with OpenTelemetry for tracing and metrics. Refactor README to include new configuration options and update tests for improved coverage of error scenarios. 2025-11-15 09:01:24 +08:00
windyboy 88d5922f90 Update README with configuration details, add default test configuration, and implement TelegramMapper tests. Enhance message processing to set timestamps and improve error handling in the processor. Refactor NATS consumer configuration and ensure proper handling of message attributes. 2025-11-14 23:18:10 +08:00
windyboy 02e54a2670 Refactor application initialization to load configuration and enhance message processing. Introduce effective subscription topic handling and improve error management in NATS consumer. Update tests for publisher error handling and add new dependency injection method for app initialization. 2025-11-14 22:43:04 +08:00
windyboy 9cce4610b6 Enhance NATS configuration and error handling. Introduce stream limits and consumer rules in configuration files. Refactor message processing to handle permanent errors. Update README and development configuration to reflect changes. Add tests for new error handling mechanisms. 2025-11-14 21:42:04 +08:00
windyboy a574cfcf27 Upgrade Go version to 1.23.0 and update dependencies. Introduce new application structure with Clean Architecture principles, including message processing, NATS integration, and PostgreSQL repository. Add configuration management using Koanf and structured logging with Zap. Remove legacy GraphQL integration and related files. Implement dependency injection with Google Wire. 2025-11-14 08:42:26 +08:00