Commit Graph
29 Commits
Author SHA1 Message Date
windyboyandClaude Sonnet 4.5 7b6f6383ad Add AFTN protocol validation and serial reader health monitoring to enhance aviation telegram processing reliability and observability
Implement comprehensive AFTN/ICAO protocol compliance validation with configurable enforcement, enabling early detection of malformed telegrams and reducing downstream processing errors. Add real-time serial reader health monitoring to automatically detect message flow interruptions and sequence gaps, ensuring operational visibility into the telegram ingestion pipeline.

Key enhancements:
- AFTN validator validates priority indicators (FF/GG/QU/DD/SS/KK), ICAO addresses (4-char alphanumeric), and datetime formats (DDHHMM) with detailed error categorization
- Invalid telegrams automatically routed to DLQ with full context for offline review and correction
- Serial reader health monitoring tracks message gaps and sequence numbers to detect stalled readers or missing messages within configurable threshold (default: 2 minutes)
- Four new Prometheus metrics expose validation errors by type, message gaps, sequence gaps, and health status for operational alerting
- Pre-configured Prometheus alert rules for critical conditions (stalled reader, high error rates, consumer lag)
- Grafana dashboard provides real-time visibility into AFTN compliance and serial reader health
- Validation disabled by default for safe rollout with zero breaking changes to existing functionality

Implementation maintains clean architecture with validator in adapter layer, extends processor and consumer with health tracking, and ensures thread-safe concurrent access to tracking state. All changes fully tested with 48 validator tests, 10 processor tests, and 21 consumer tests passing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-24 14:21:59 +08:00
windyboy 6eff35b56f 🔧 Update Go module dependencies and modify NATS configuration to exclusively support JetStream mode. Upgrade go.uber.org/zap to version 1.27.1 and golang.org/x/crypto to version 0.45.0. Revise Makefile and documentation to reflect changes in NATS mode, ensuring consistent messaging behavior across development and production environments. Enhance README and configuration files to clarify JetStream usage and remove references to Core NATS mode. 2025-11-21 09:59:47 +08:00
windyboy 8914156a58 Update Go module dependencies and enhance NATS consumer metrics. Add github.com/kylelemons/godebug as an indirect dependency. Upgrade github.com/prometheus/common to version 0.67.3. Change NATS consumer mode to "jetstream" in the development configuration. Improve observability by ensuring metrics for pending messages are initialized and recorded correctly in JetStream mode. Update documentation to reflect changes in metrics handling and consumer behavior. 2025-11-19 16:49:53 +08:00
windyboy 06fc9cb9e0 🔧 Update Go version in go.mod and enhance build process with versioning information. Modify Makefile and Taskfile to inject build metadata (version, commit, build time) into the binary. Improve README with instructions for custom version builds and document new build info features. Add benchmarks for message parsing and processing to improve performance testing capabilities. 2025-11-18 14:15:58 +08:00
windyboy 5d05237283 Refactor NATS consumer management by introducing a dedicated ConsumerManager and StreamManager for lifecycle management. Enhance error handling with a new ErrorHandler to streamline message processing and recovery logic. Update consumer initialization to utilize the new managers, improving code organization and maintainability. Add comprehensive tests for the new components to ensure reliability and robustness in handling NATS operations. 2025-11-18 08:38:03 +08:00
windyboy 704c7b80f6 Add repository guidelines and enhance documentation for project structure, build commands, coding standards, and testing practices. Introduce AGENTS.md for contributor guidance, update README.md to reference new guidelines, and improve configuration documentation for NATS modes. Update Makefile and Taskfile with clearer run commands and requirements for development and production modes. Add production deployment guide and improve logging configuration for better observability. 2025-11-17 16:25:23 +08:00
windyboy 67abd66fa3 Enhance logging configuration by introducing file output options and rotation settings in config.dev.toml. Update logger implementation to support multiple output streams, including file logging with rotation using lumberjack. Improve error handling and logging across various components, ensuring consistent logging practices. Update .gitignore to include log files and compressed logs. Add new Go module dependency for lumberjack. 2025-11-17 12:58:40 +08:00
windyboy 204217015d Add configuration for Code Review Automation and enhance .gitignore. Introduce .coderabbit.yml for automated reviews with profiles for correctness, maintainability, security, and performance. Update paths to include relevant directories and exclude generated files. Modify .gitignore to include coverage reports and generated files. Refactor Docker Compose to use updated paths for database initialization scripts. Update Go module dependencies and enhance Makefile with new code generation tasks. Transition domain models to a new DTO structure for better separation of concerns. 2025-11-17 11:47:23 +08:00
windyboy 06b2495d77 Refactor logging implementation by removing the custom logging utility and transitioning to direct usage of the Zap logger. Update logger configuration in the development environment and clean up related files, including the removal of unused logger configuration JSON. Ensure consistent logging practices across the application. 2025-11-16 13:41:51 +08:00
windyboy 53208997e8 Integrate monitoring server for observability, adding health and metrics endpoints. Update configuration to enable monitoring features and enhance README with deployment examples for Kubernetes and systemd. Refactor application initialization to include monitoring server setup and improve error handling in message processing metrics. 2025-11-15 16:30:29 +08:00
windyboy 4f16bd6d90 Introduce Docker Compose development stack with TimescaleDB and NATS services. Update README with usage instructions for local development. Upgrade Go version to 1.25.0 and update dependencies. Refactor configuration for NATS consumer rules and enhance Telegram database schema with TimescaleDB support. 2025-11-15 09:46:29 +08:00
windyboy 359e7694ab Update dependencies, enhance NATS consumer configuration, and improve error handling in message processing. Introduce telemetry support with OpenTelemetry for tracing and metrics. Refactor README to include new configuration options and update tests for improved coverage of error scenarios. 2025-11-15 09:01:24 +08:00
windyboy 31839cf21f Upgrade Go version to 1.24.0 and update various dependencies, including NATS, Viper, and Go modules. Refactor error handling in configuration loading and validation processes for improved clarity and consistency. 2025-11-14 23:42:39 +08:00
windyboy a574cfcf27 Upgrade Go version to 1.23.0 and update dependencies. Introduce new application structure with Clean Architecture principles, including message processing, NATS integration, and PostgreSQL repository. Add configuration management using Koanf and structured logging with Zap. Remove legacy GraphQL integration and related files. Implement dependency injection with Google Wire. 2025-11-14 08:42:26 +08:00
windyboy 5470118461 Update github.com/urfave/cli/v2 to v2.27.4. (#3) 2024-08-12 17:48:05 +08:00
windyboy 091a054eab refactor: Rename 'text' field to 'content' in telegrams table 2024-08-09 10:29:17 +08:00
windyboy e4fa138089 refactor: Update constants and remove unused code in aviation_parser.go and constants.go 2024-08-07 10:47:15 +08:00
windyboy 1e37e21331 chore: Update go.mod dependencies to latest versions 2024-08-06 12:44:43 +08:00
windyboy c54bcbc0a8 refactor: Update Taskfile.yml to use lowercase variable names for consistency 2024-08-05 10:11:33 +08:00
windyboy 6a689b073f deps 2024-08-02 16:43:26 +08:00
windyboy 70e33742fa mod tidy 2024-08-02 16:39:34 +08:00
zhiqiang feng d91e61ab5d up 2024-08-02 10:01:49 +08:00
windyboy 0ea7ce22c9 refactor: Update parsing of flight schedule data for improved accuracy and functionality 2024-07-31 12:25:21 +08:00
windyboy 9982b1408d refactor: Update parsing of flight schedule data for improved accuracy and functionality 2024-07-31 11:05:30 +08:00
windyboy 91d6957eba refactor: Update CNL pattern in config.go
The code changes in `config.go` update the CNL pattern to correctly match CNL messages. This update ensures accurate extraction of data from CNL messages and improves the overall functionality of the code.
2024-07-29 23:40:51 +08:00
windyboy faed181940 refactor: Update go.mod dependencies
Update go.mod dependencies to the latest stable versions. This includes updating github.com/nats-io/nats.go to v1.36.0, github.com/onsi/ginkgo/v2 to v2.19.1, github.com/onsi/gomega to v1.34.0, go.uber.org/zap to v1.27.0, and golang.org/x/oauth2 to v0.21.0. Also update github.com/go-logr/logr to v1.4.2 and github.com/klauspost/compress to v1.17.9. These updates ensure that the project is using the most recent dependencies, improving stability and compatibility.
2024-07-29 14:20:20 +08:00
windyboy 6a45244694 refactor: Update BodyParser constructor to accept body parameter 2024-07-24 16:38:22 +08:00
windyboy 496dc2b9d0 refactor: Update go.mod dependencies
This commit updates the go.mod file to include the latest versions of the following dependencies:
- github.com/Khan/genqlient v0.7.0
- github.com/google/uuid v1.6.0

These updates ensure that the project is using the most recent and stable versions of the dependencies, improving the overall reliability and functionality of the code.
2024-07-24 10:33:12 +08:00
windyboy 77a7c7eae0 refactor: Update ARR body parsing logic 2024-07-23 11:29:12 +08:00