windy/go-caatsm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
170 Commits 1 Branch 0 Tags
dbde6524b3ac25f7ad92017d0df06af078b61710
Commit Graph
141 Commits
Author SHA1 Message Date
windyboy dbde6524b3 refactor weather parser 2025-12-24 17:05:51 +08:00
windyboyandClaude Sonnet 4.5 d0fd461e38 ✨ Add weather report parsing for METAR, SPECI, and TAF 
Implement comprehensive weather parsing capabilities following Clean
Architecture principles with composite parser pattern for routing between
aviation and weather messages.

## Features Added

- Weather report parsing (METAR, SPECI, TAF)
- Composite parser pattern for message routing
- Lenient parsing with warnings for unrecognized tokens
- Support for PROB and RMK sections in TAF
- Rich domain modeling with typed weather elements

## Architecture

**Domain Layer** (internal/domain/weather/):
- WeatherMessage interface with Metar and Taf implementations
- Weather elements: Wind, Visibility, Cloud, Temperature, Altimeter, Phenomenon
- Domain errors: ErrInvalidFormat, ErrMissingStation, ErrMissingTime

**Port Layer** (internal/port/weather_parser.go):
- WeatherParser interface with CanParse and Parse methods

**Adapter Layer** (internal/adapter/parser/weather/):
- WeatherParserImpl with classification and parsing logic
- Comprehensive regex patterns for weather elements
- METAR/SPECI parser with element extraction
- TAF parser with period handling (FM, TEMPO, BECMG, PROB)
- Helper functions for time parsing and unit conversions

**Composite Parser** (internal/adapter/parser/composite.go):
- Routes weather reports to weather parser
- Falls back to aviation parser for telegrams
- Converts WeatherMessage to ParsedTelegram format

## Integration

- Updated ProvideParser to create composite parser with weather parser
- Added weather parser to Wire DI configuration
- Updated processor_bench_test.go for weather parser integration
- Documentation added in docs/weather-parser.md

## Testing

- 29 comprehensive tests for weather parsing (all passing)
- Tests for classification, METAR, SPECI, TAF, and composite routing
- Benchmark compatibility maintained

## Fixes Applied

- TAF PROB parsing: Include PROB/RMK in special section detection
- Composite test: Updated to use properly formatted AFTN telegram
- Linter issues: Switch statement refactor, removed unused patterns
- Ineffective break statement fixed in TAF parser

## Coverage

~1,743 lines of new code with:
- Complete METAR/SPECI parsing
- TAF parsing with period support
- Lenient error handling with warnings
- Unit conversions and time utilities

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-24 16:38:03 +08:00
windyboy cf8e196248 🔧 Add nil logger handling in NewConsumerManager and NewStreamManager; update integration test to pass config 2025-12-24 14:53:14 +08:00
windyboyandClaude Sonnet 4.5 7b6f6383ad ✨ Add AFTN protocol validation and serial reader health monitoring to enhance aviation telegram processing reliability and observability 
Implement comprehensive AFTN/ICAO protocol compliance validation with configurable enforcement, enabling early detection of malformed telegrams and reducing downstream processing errors. Add real-time serial reader health monitoring to automatically detect message flow interruptions and sequence gaps, ensuring operational visibility into the telegram ingestion pipeline.

Key enhancements:
- AFTN validator validates priority indicators (FF/GG/QU/DD/SS/KK), ICAO addresses (4-char alphanumeric), and datetime formats (DDHHMM) with detailed error categorization
- Invalid telegrams automatically routed to DLQ with full context for offline review and correction
- Serial reader health monitoring tracks message gaps and sequence numbers to detect stalled readers or missing messages within configurable threshold (default: 2 minutes)
- Four new Prometheus metrics expose validation errors by type, message gaps, sequence gaps, and health status for operational alerting
- Pre-configured Prometheus alert rules for critical conditions (stalled reader, high error rates, consumer lag)
- Grafana dashboard provides real-time visibility into AFTN compliance and serial reader health
- Validation disabled by default for safe rollout with zero breaking changes to existing functionality

Implementation maintains clean architecture with validator in adapter layer, extends processor and consumer with health tracking, and ensures thread-safe concurrent access to tracking state. All changes fully tested with 48 validator tests, 10 processor tests, and 21 consumer tests passing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-24 14:21:59 +08:00
windyboy 6eff35b56f 🔧 Update Go module dependencies and modify NATS configuration to exclusively support JetStream mode. Upgrade go.uber.org/zap to version 1.27.1 and golang.org/x/crypto to version 0.45.0. Revise Makefile and documentation to reflect changes in NATS mode, ensuring consistent messaging behavior across development and production environments. Enhance README and configuration files to clarify JetStream usage and remove references to Core NATS mode. 2025-11-21 09:59:47 +08:00
windyboy 5ad86d6296 ✨ Enable Dead-Letter Queue (DLQ) in development configuration and enhance NATS consumer error handling. Update config.dev.toml to enable DLQ routing for failed messages. Modify NATS consumer to append DLQ subject to stream subjects if enabled. Improve error logging in message processing to include message IDs and DLQ routing status, ensuring better observability and error management. 2025-11-19 17:09:52 +08:00
windyboy 8914156a58 ✨ Update Go module dependencies and enhance NATS consumer metrics. Add github.com/kylelemons/godebug as an indirect dependency. Upgrade github.com/prometheus/common to version 0.67.3. Change NATS consumer mode to "jetstream" in the development configuration. Improve observability by ensuring metrics for pending messages are initialized and recorded correctly in JetStream mode. Update documentation to reflect changes in metrics handling and consumer behavior. 2025-11-19 16:49:53 +08:00
windyboy f01dd3bf1f ✨ Refactor telemetry and metrics handling to enhance observability. Remove TLS configuration from telemetry initialization, replacing it with an insecure option for HTTP. Update OpenTelemetry endpoint handling to normalize URLs. Introduce new metrics for publish failures categorized by message type. Enhance Grafana dashboards with detailed descriptions and additional metrics for better monitoring insights. Update documentation to reflect changes in metrics and observability features. 2025-11-19 14:53:58 +08:00
windyboy c687fdcde8 ✨ Update agent guidelines and improve documentation structure. Refactor AGENTS.md to streamline commands and code style guidelines, enhancing clarity and usability. Update README.md with refined NATS consumer configuration details and observability metrics. Modify .gitignore to exclude dynamically generated Prometheus target files. Enhance configuration files for development and production environments, ensuring consistency and clarity in settings. 2025-11-19 13:07:09 +08:00
windyboy 06fc9cb9e0 🔧 Update Go version in go.mod and enhance build process with versioning information. Modify Makefile and Taskfile to inject build metadata (version, commit, build time) into the binary. Improve README with instructions for custom version builds and document new build info features. Add benchmarks for message parsing and processing to improve performance testing capabilities. 2025-11-18 14:15:58 +08:00
windyboy 7f44b5389d ✨ Enhance observability and error handling in NATS integration. Introduce comprehensive OpenTelemetry support with environment-based sampling and semantic attributes for tracing and metrics. Implement an advisory dead-letter queue (DLQ) handler for managing message delivery failures. Update NATS consumer to utilize structured logging and improve error handling strategies. Refactor configuration files for OpenTelemetry collector in both development and production environments, ensuring robust telemetry integration. Enhance documentation to reflect new features and best practices for observability. 2025-11-18 11:47:35 +08:00
windyboy 5d05237283 ✨ Refactor NATS consumer management by introducing a dedicated ConsumerManager and StreamManager for lifecycle management. Enhance error handling with a new ErrorHandler to streamline message processing and recovery logic. Update consumer initialization to utilize the new managers, improving code organization and maintainability. Add comprehensive tests for the new components to ensure reliability and robustness in handling NATS operations. 2025-11-18 08:38:03 +08:00
windyboy 61647cf849 ✨ Update Prometheus configuration and enhance monitoring capabilities. Refactor docker-compose.dev.yml to dynamically generate target configurations for the CAATSM receiver. Modify Makefile and Taskfile to allow setting the monitoring address via environment variables. Update observability settings in config.dev.toml and Prometheus scrape configurations to improve integration with the monitoring stack. Enhance CLI options for monitoring address and disable monitoring features as needed. 2025-11-17 17:53:24 +08:00
windyboy 704c7b80f6 ✨ Add repository guidelines and enhance documentation for project structure, build commands, coding standards, and testing practices. Introduce AGENTS.md for contributor guidance, update README.md to reference new guidelines, and improve configuration documentation for NATS modes. Update Makefile and Taskfile with clearer run commands and requirements for development and production modes. Add production deployment guide and improve logging configuration for better observability. 2025-11-17 16:25:23 +08:00
windyboy 67abd66fa3 ✨ Enhance logging configuration by introducing file output options and rotation settings in config.dev.toml. Update logger implementation to support multiple output streams, including file logging with rotation using lumberjack. Improve error handling and logging across various components, ensuring consistent logging practices. Update .gitignore to include log files and compressed logs. Add new Go module dependency for lumberjack. 2025-11-17 12:58:40 +08:00
windyboy 204217015d ✨ Add configuration for Code Review Automation and enhance .gitignore. Introduce .coderabbit.yml for automated reviews with profiles for correctness, maintainability, security, and performance. Update paths to include relevant directories and exclude generated files. Modify .gitignore to include coverage reports and generated files. Refactor Docker Compose to use updated paths for database initialization scripts. Update Go module dependencies and enhance Makefile with new code generation tasks. Transition domain models to a new DTO structure for better separation of concerns. 2025-11-17 11:47:23 +08:00
windyboy 12fda00df9 ✨ Introduce seed-telegrams tool for generating synthetic telegrams for testing and development. Enhance README with detailed usage instructions, command-line parameters, and examples. Add unit tests for telegram generation and status selection logic to ensure robustness. Update documentation to reflect new features and usage scenarios. 2025-11-16 17:24:16 +08:00
windyboy 06b2495d77 ✨ Refactor logging implementation by removing the custom logging utility and transitioning to direct usage of the Zap logger. Update logger configuration in the development environment and clean up related files, including the removal of unused logger configuration JSON. Ensure consistent logging practices across the application. 2025-11-16 13:41:51 +08:00
windyboy e27328378a ✨ Revise observability features by introducing new health and liveness endpoints (/livez and /readyz), updating Prometheus metrics to track NATS consumer pending messages, and enhancing telemetry integration for better monitoring. Update documentation to reflect these changes and ensure consistency in metric naming conventions. 2025-11-16 13:14:46 +08:00
windyboy 3d8572a69f ✨ Implement dead-letter queue (DLQ) enhancements by adding configuration options and validation logic. Update observability metrics to track DLQ message counts and publish failures. Enhance documentation to reflect new metrics and DLQ behavior in JetStream mode. 2025-11-16 11:55:13 +08:00
windyboy 6616c7e10d ✨ Add new dev-run-compose task to Taskfile for running the CAATSM receiver in a Docker Compose development stack. Update monitoring configuration in config.dev.toml to use console format. Modify Prometheus scrape configuration to target a specific IP address. Enhance NATS consumer error handling with retry logic for JetStream availability and adjust logging level for consumer metrics. 2025-11-16 11:38:49 +08:00
windyboy 7e7b8ca412 ✨ Update observability configuration by enabling monitoring in the development environment, adding a new Prometheus scrape job for the CAATSM receiver, and enhancing documentation with recommended scrape configurations and validation steps for metrics. Refactor NATS consumer initialization to ensure idempotent consumer creation. 2025-11-16 10:34:53 +08:00
windyboy 487a0acb48 ✨ Refactor testing framework to utilize Ginkgo for unit tests and introduce a new test-all target for running both unit and integration tests. Update README and Taskfile to reflect changes in test execution commands and clarify dependencies. Migrate domain models to a new model package for better separation of concerns. 2025-11-16 09:58:49 +08:00
windyboy a96cb8a40b ✨ Implement dead-letter queue (DLQ) functionality for handling permanent failures in message processing. Update configuration to enable DLQ and specify the subject for routing failed messages. Enhance observability by adding metrics for message handling, retries, and database operations. Introduce structured logging for better traceability of message processing events. 2025-11-16 09:15:37 +08:00
windyboy c98baa0ff4 Refactor NATS consumer defaults (#7) 2025-11-16 08:24:29 +08:00
windyboy 53208997e8 ✨ Integrate monitoring server for observability, adding health and metrics endpoints. Update configuration to enable monitoring features and enhance README with deployment examples for Kubernetes and systemd. Refactor application initialization to include monitoring server setup and improve error handling in message processing metrics. 2025-11-15 16:30:29 +08:00
windyboy c3f98e9c7c ✨ Add NATS exporter to Docker Compose for enhanced observability, update Prometheus and Grafana configurations to integrate new metrics, and enhance documentation to reflect changes in the development stack. 2025-11-15 15:27:53 +08:00
windyboy cdb0cea8f7 ✨ Enhance Docker Compose setup by adding initialization services for PostgreSQL and NATS, improving JetStream configuration, and updating the Taskfile to use Docker Compose instead of Podman. Modify database schema to enforce unique constraints on (uuid, received_at) for telegrams and telegrams_raw tables. 2025-11-15 14:24:11 +08:00
windyboy 184c5453fb ✨ Expand Docker Compose development stack to include observability tools: OpenTelemetry Collector, Jaeger, Prometheus, and Grafana. Update README with new usage instructions for the observability stack and enhance Taskfile for streamlined development commands. Introduce sample telegram generation utility and improve NATS configuration for core and JetStream modes. 2025-11-15 12:08:26 +08:00
windyboy 4f16bd6d90 ✨ Introduce Docker Compose development stack with TimescaleDB and NATS services. Update README with usage instructions for local development. Upgrade Go version to 1.25.0 and update dependencies. Refactor configuration for NATS consumer rules and enhance Telegram database schema with TimescaleDB support. 2025-11-15 09:46:29 +08:00
windyboy 359e7694ab ✨ Update dependencies, enhance NATS consumer configuration, and improve error handling in message processing. Introduce telemetry support with OpenTelemetry for tracing and metrics. Refactor README to include new configuration options and update tests for improved coverage of error scenarios. 2025-11-15 09:01:24 +08:00
windyboy 31839cf21f ✨ Upgrade Go version to 1.24.0 and update various dependencies, including NATS, Viper, and Go modules. Refactor error handling in configuration loading and validation processes for improved clarity and consistency. 2025-11-14 23:42:39 +08:00
windyboy 88d5922f90 ✨ Update README with configuration details, add default test configuration, and implement TelegramMapper tests. Enhance message processing to set timestamps and improve error handling in the processor. Refactor NATS consumer configuration and ensure proper handling of message attributes. 2025-11-14 23:18:10 +08:00
windyboy 02e54a2670 ✨ Refactor application initialization to load configuration and enhance message processing. Introduce effective subscription topic handling and improve error management in NATS consumer. Update tests for publisher error handling and add new dependency injection method for app initialization. 2025-11-14 22:43:04 +08:00
windyboy 9cce4610b6 ✨ Enhance NATS configuration and error handling. Introduce stream limits and consumer rules in configuration files. Refactor message processing to handle permanent errors. Update README and development configuration to reflect changes. Add tests for new error handling mechanisms. 2025-11-14 21:42:04 +08:00
windyboy a574cfcf27 ✨ Upgrade Go version to 1.23.0 and update dependencies. Introduce new application structure with Clean Architecture principles, including message processing, NATS integration, and PostgreSQL repository. Add configuration management using Koanf and structured logging with Zap. Remove legacy GraphQL integration and related files. Implement dependency injection with Google Wire. 2025-11-14 08:42:26 +08:00
windyboy bafbbf470c ✨ Refactor uuid to id in message handling & processing. (#6) 2024-08-14 14:29:24 +08:00
windyboy a5fa4955f4 ✨ Add configuration to interfaces. 2024-08-14 11:48:18 +08:00
windyboy 6b44c2d7a0 refactor subscribe (#4) 
* ✨ Refactor nats subscribe method and remove unused argument.

* ✨ Add logging for message insertion 📝
 2024-08-14 09:40:47 +08:00
windyboy 8a4c46aa34 update handler (#2) 
* 🔧 Remove commented-out regex patterns and initialization in config.

* ✨ Add NATS message publisher with configuration 📩🚀

* ✨ Add new PlainTextMarshaler and refactor handler in nats package.
 2024-08-12 17:47:34 +08:00
windyboy fd58395db3 🔧 Remove commented-out regex patterns and initialization in config. (#1) 2024-08-12 14:23:31 +08:00
windyboy e468e1f89c refactor: Add ToString method to ParsedMessage struct 2024-08-09 11:13:25 +08:00
windyboy 85ce23cb16 refactor: Update 'Content' field in HasuraRepository.CreateNew method 2024-08-09 11:07:07 +08:00
windyboy 091a054eab refactor: Rename 'text' field to 'content' in telegrams table 2024-08-09 10:29:17 +08:00
windyboy 622bd8e11b refactor: Remove commented-out code in schedule_parser.go 2024-08-07 20:01:14 +08:00
windyboy 3a1f533d63 hasura 2024-08-07 17:16:07 +08:00
windyboy 15ce2c033a refactor: Update aviation_parser.go to remove unused code and update constants 2024-08-07 17:05:42 +08:00
windyboy e1cc851cc3 refactor: Rename 'body_and_footer' column to 'text' in telegrams table 2024-08-07 14:10:15 +08:00
windyboy f340c66b19 refactor: Update aviation_parser.go to remove unused code and update constants 2024-08-07 11:10:18 +08:00
windyboy 55a286f521 refactor: Improve error handling in NATS message handling 2024-08-07 11:10:08 +08:00