diff --git a/docs/architecture.md b/docs/architecture.md index ea56828..3ab0eba 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -10,7 +10,7 @@ msgexchange-v2 是机场 OMMS 的上游报文处理中间件，用于替换旧 - **主要入口**：轮询共享 MySQL 的 `CMINMSGS`。 - **兼容入口**：`POST /cminmsgs/send`，供现役兼容、手工工具和对拍使用；写入信箱后返回记录 ID，不是生产收报主路径。 - **输出**：Kafka 的 `msg` / `schd` 消息、共享 MySQL 的 `COUTMSGS` 出站信箱，以及查询 HTTP 接口；不直接推送前端。 -- **当前范围（阶段 A）**：运营航班权威状态落自有 PostgreSQL（表 `FLIGHT_SCHD` 与 `SCHD_GEN`，ACM2-28 定案采纳选项 C），Redis 彻底退出动态权威与全部写路径。ES 历史投影及相关清场流程属于暂缓的阶段 B。 +- **当前范围（阶段 A）**：航班当前态落自有 PostgreSQL（`FLIGHT_SCHD` + 资源明细表 + `FLIGHT_ROUTE_POINT`，权威口径见 [flight-state.md](flight-state.md)）。无 Redis 依赖；ES 历史投影属暂缓的阶段 B。 本文描述架构约束，不代表所有能力已实现；实现缺口见第 9 节。模块交互、状态机和参数详见 [design.md](design.md)，需求见 [user-stories.md](user-stories.md)。历史报文契约仍以 [SIS 接口规范](legacy/SIS_AODB_RMS-V0.1.md) 和 [XSD](legacy/unisysaodbsis.xsd) 为兼容依据，其他 legacy 资料仅作参考。 @@ -32,7 +32,7 @@ CIIMS / AODB 等上游 │ processing：取 FIFO 队头 → 解析 / 去重 → Handler 决策 │ │ └─ PG 单事务：航班变更 + 终态 + 待发事件│ │ │ -│ jobs：在主泵空闲或消息退避窗口内执行维护作业 │ +│ jobs：独立维护线程（回填补偿 / 历史归档 / 留痕清理） │ │ delivery：读取 PG 待发事件 → 投递 / 重试 │ └─────────────────────────┬──────────────────────────────┘ ├─ Kafka：msg / schd @@ -42,7 +42,7 @@ CIIMS / AODB 等上游 查询接口读取航班动态，不参与状态写入。 ``` -收报、处理和投递各使用一条专用线程，不占用 HTTP 事件循环。**只有主泵可以写航班动态及快照版本**，维护作业也必须遵守这一规则。 +收报、处理、投递与维护作业各使用独立线程，不占用 HTTP 事件循环。**航班当前态的写入只发生在持有 `PIPELINE_LOCK` 的事务内**，由主泵串行驱动。 采用 Kotlin + JDK 25、Micronaut 编译期依赖注入和 JDBC 持久化。数据库变更由 Flyway 管理，但只作用于自有 PostgreSQL。具体依赖版本以 `build.gradle.kts` 为准，不在架构文档重复维护。 @@ -52,20 +52,19 @@ CIIMS / AODB 等上游 |---|---| | `ingress` | 轮询信箱、持久化入队、补偿重扫及兼容 HTTP 写入；不解析业务报文。 | | `codec` | XML 解码，区分非法报文与可修复的解码失败。 | -| `processing` | FIFO 调度、业务身份绑定与去重、Handler 决策、快照处理及状态提交。Handler 只返回决策，不直接访问数据库或 Kafka。 | +| `processing` | FIFO 调度、业务身份绑定与去重、处理器决策（SCHD/FLOP/FDEL/ADFT）、事务提交。处理器只产出决策与落库计划，不直接触碰 Kafka。 | | `delivery` | 消费待发事件，负责按目标保序、`schd` 聚合、投递和失败重试。 | -| `jobs` | 持久化维护作业，由主泵在允许的窗口执行；阶段 B 作业暂不启用。 | -| `reference` | 静态参考数据和上游请求跟踪。 | +| `jobs` | 回填补偿扫描、历史归档与物理清除（§8.2）、留痕保留期清理；独立线程执行，不参与 FIFO。 | | `domain` / `config` | 领域状态、事件和决策模型，以及运行参数。 | -| `infra` | 仓储、外部适配器、重试、健康检查与日志；通过接口隔离基础设施（Redis 已退出核心写路径）。 | +| `infra` | 仓储（JDBC/stub）、外部适配器、重试、健康检查与日志；通过接口隔离基础设施。 | ## 4. 主流程 ### 收报与处理 1. `InboxPoller` 默认每秒扫描 `DATE_PROCESSED IS NULL` 的信箱记录，在自有 PG 中建立 `PROC_STATE(PENDING)`。重复扫描不能重复入队；入队失败留待重扫。 -2. 主泵只处理最小未完成 `CMINMSGS_ID`。解析报文、绑定业务身份并去重后，调用对应 Handler 生成决策。 -3. 在自有 PG 本地事务中同时保存航班状态变更（`FLIGHT_SCHD` / `SCHD_GEN`）、处理结果和 `MSG_EVENT` 待发事件。 +2. 主泵只处理最小未完成 `MSG_ID`。解析报文、绑定业务身份并去重后，分派给 SCHD/FLOP/FDEL/ADFT 处理器。 +3. 在自有 PG 同一事务内（先取 `PIPELINE_LOCK`）保存航班状态变更（`FLIGHT_SCHD` 与明细表）、处理结果、`MSG_EVENT` 待发事件与回填待办预登记。 4. 事务提交后，回填共享信箱的处理标记（外部副作用，补偿保障）。 ### 投递 @@ -77,10 +76,10 @@ CIIMS / AODB 等上游 ## 5. 必须保持的约束 - **消息严格 FIFO**：队头失败并退避时，后续消息仍不能越过它。只有队头完成或按失败策略进入终态后，队列才继续推进。收报重扫和水位设计必须防止较小 ID 漏入队而被后续消息越过。 -- **动态状态单写者**：`FLIGHT_SCHD` 运营航班表和快照 `SCHD_GEN` 只由主泵单线程写入。事务通过 `PIPELINE_LOCK` 的 `FLIGHT_SCHD_WRITER` 行执行 `SELECT ... FOR UPDATE` 互斥；该方案在 PG/Oracle 11g 均无需数据库扩展或额外 DBA 特权。不能通过增加实例或处理线程直接扩容。 +- **动态状态单写者**：`FLIGHT_SCHD` 及明细表只由主泵单线程写入。事务内第一步对 `PIPELINE_LOCK` 单行 `SELECT ... FOR UPDATE` 互斥；该方案在 PG/Oracle 11g 均无需数据库扩展或额外 DBA 特权。不能通过增加实例或处理线程直接扩容。 - **身份去重**：同一业务身份只能绑定一条有效处理记录，重复报文不应再次产生业务副作用。具体身份组成和重放规则见设计文档。 -- **快照可恢复**：快照覆盖、旧数据清理和版本推进需要原子性与重放保护；不能在恢复时把旧代数据重新写回。 -- **作业不与消息混排**：`PUMP_JOB` 是独立队列，只在没有消息队头或队头处于退避窗口时执行。执行窗口与饥饿边界需要明确验证。 +- **快照可恢复**：快照以 `PROC_STATE` 成功终态判定重放（§5.1）；每次成功写入推进 `STATE_VERSION`；`OPERATION_DAY` 一经确定不可变（§5.3）。 +- **物理清除只发生在历史归档**：删除一律先标记（FDEL）或由生命周期清除；历史存储未接通时必须删 0 条（§8.2）。 这些约束优先于吞吐量优化。单写者降低了并发复杂度，代价是队头阻塞和吞吐上限；如需并行化，必须先重新定义顺序与状态归属，不能只调整线程数。 @@ -88,7 +87,7 @@ CIIMS / AODB 等上游 | 存储 | 承载内容 | 职责说明 | |---|---|---| -| 自有 PostgreSQL | 处理状态 `PROC_STATE`、待发事件 `MSG_EVENT`、维护作业 `PUMP_JOB`、请求跟踪 `REQ_TRACK`、静态主数据 `REF_MASTER`、运营航班表 `FLIGHT_SCHD` 与日代 `SCHD_GEN` | 本系统唯一业务数据库。消息处理、快照推进与待发事件在单事务内原子提交；本地事务只在此库。 | +| 自有 PostgreSQL | 单行锁 `PIPELINE_LOCK`、处理状态 `PROC_STATE`、待发事件 `MSG_EVENT`、请求跟踪 `REQ_TRACK`、回填待办 `BACKFILL_TODO`、航班当前态 `FLIGHT_SCHD` + 9 张明细表、留痕 `SCHD_SNAP_LOG` | 本系统唯一业务数据库。消息处理、状态推进与待发事件在单事务内原子提交；本地事务只在此库。 | | 共享 MySQL | `CMINMSGS` 入站信箱、`COUTMSGS` 出站信箱 | 外部系统所有。仅执行约定的信箱读写和处理标记回填，不建表、不迁移 schema、不写历史表。兼容 HTTP 入口可按既有契约写入入站信箱。 | **不使用跨库事务。** PG 事务只能保证“处理结果与待发事件一起提交”，不能覆盖 Redis 更新、MySQL 回填或 Kafka 发送。跨存储依靠幂等、重试和持久化补偿恢复： @@ -107,7 +106,7 @@ CIIMS / AODB 等上游 | 编号 | 决策及理由 | |---|---| -| D1 | 业务报文严格 FIFO，维护作业窗口执行，优先保护航班状态的时序正确性。 | +| D1 | 业务报文严格 FIFO，优先保护航班状态的时序正确性；维护作业独立线程执行，不参与消息序。 | | D2 | 阶段 B 暂缓。历史写入成功后才可生成删除事件；顺序调用本身不保证原子性，恢复与去重方案需在启用前补齐。 | | D3 | `schd` 只从 `flushSchd` 聚合发送，减少已被覆盖的中间状态通知。 | | D4 | 未实现的报文类型按 `UNSUPPORTED` 可恢复失败处理，不当作非法报文直接丢弃；补齐能力后按重放规则恢复。 | @@ -130,7 +129,7 @@ CIIMS / AODB 等上游 **替换旧系统** -采用“影子对拍 → 切流 → 旧系统冻结”。共享信箱不能让新旧系统同时认领和回填；影子输入使用只读水位或回放。影子环境须隔离 PG schema/实例、Kafka topic 和服务注册身份，并禁止误写生产信箱。切流时保证只有一个权威写者。影子期采用 `FlightStoreDiffTool` 跨存储对拍。 +采用“影子对拍 → 切流 → 旧系统冻结”。共享信箱不能让新旧系统同时认领和回填；影子输入使用只读水位或回放。影子环境须隔离 PG schema/实例、Kafka topic 和服务注册身份，并禁止误写生产信箱。切流时保证只有一个权威写者。 **可观测性要求** @@ -143,8 +142,8 @@ CIIMS / AODB 等上游 上线前至少需要完成并验证： - 真实 PG 事务、信箱水位与补扫、回填补偿、出站信箱，以及所需业务 Handler。 -- `FLIGHT_SCHD` 事务原子性、快照 SQL CAS 版本校验、故障中断回滚恢复与影子对拍 cross-store diff 工具。 -- FIFO、身份去重、作业窗口和投递故障下的回归测试。 +- `FLIGHT_SCHD` 事务原子性、`STATE_VERSION` 推进与 `OPERATION_DAY` 不可变校验、故障中断回滚恢复。 +- FIFO、身份去重、FDEL/ADFT 生命周期、历史归档顺序和投递故障下的回归测试。 - 生产启动校验、单实例排他保护、影子隔离和 Kafka 配置约束；当前配置仍允许 Kafka 参数覆盖，且默认 in-flight 值与 D11 要求不同。 - 死信告警、人工重放、端到端追踪、积压指标及安全边界。 diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/PipelineLifecycle.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/PipelineLifecycle.kt index 396ef50..389862c 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/PipelineLifecycle.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/PipelineLifecycle.kt @@ -3,6 +3,7 @@ package com.gzzn.omms.msgexchange import com.gzzn.omms.msgexchange.delivery.Dispatcher import com.gzzn.omms.msgexchange.ingress.InboxPoller import com.gzzn.omms.msgexchange.processing.Pump +import com.gzzn.omms.msgexchange.jobs.JobRunner import io.micronaut.context.annotation.Requires import io.micronaut.runtime.event.annotation.EventListener import io.micronaut.runtime.server.event.ServerStartupEvent @@ -20,6 +21,7 @@ class PipelineLifecycle( private val poller: InboxPoller, private val pump: Pump, private val dispatcher: Dispatcher, + private val jobRunner: JobRunner, ) { private val log = org.slf4j.LoggerFactory.getLogger(PipelineLifecycle::class.java) private val threads = mutableListOf() @@ -38,7 +40,8 @@ class PipelineLifecycle( threads += spawn("msgx-inbox-poller", poller::loop) threads += spawn("msgx-pump", pump::loop) threads += spawn("msgx-dispatcher", dispatcher::loop) - log.info("pipeline loops started (inbox-poller, pump, dispatcher)") + jobRunner.start() + log.info("pipeline loops started (inbox-poller, pump, dispatcher, jobs)") } private fun spawn(name: String, body: () -> Unit): Thread = @@ -49,6 +52,7 @@ class PipelineLifecycle( poller.stop() pump.stop() dispatcher.stop() + jobRunner.stop() threads.forEach { it.interrupt() } // 解除 Thread.sleep 阻塞，加速退出 threads.forEach { runCatching { it.join(3000) } } log.info("pipeline loops stopped") diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/codec/JacksonXmlCodec.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/codec/JacksonXmlCodec.kt index f682731..1305479 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/codec/JacksonXmlCodec.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/codec/JacksonXmlCodec.kt @@ -1,37 +1,37 @@ package com.gzzn.omms.msgexchange.codec +import com.fasterxml.jackson.databind.DeserializationFeature +import com.fasterxml.jackson.module.kotlin.registerKotlinModule +import com.fasterxml.jackson.dataformat.xml.JacksonXmlModule +import com.fasterxml.jackson.dataformat.xml.XmlFactory +import com.fasterxml.jackson.dataformat.xml.XmlMapper import com.gzzn.omms.msgexchange.domain.DecodedMessage import com.gzzn.omms.msgexchange.domain.ErrorClass import com.gzzn.omms.msgexchange.domain.MetaFields import com.gzzn.omms.msgexchange.domain.MsgKind +import com.gzzn.omms.msgexchange.domain.flight.ScheduleRecord import jakarta.inject.Singleton -import org.w3c.dom.Element -import org.w3c.dom.Node -import org.xml.sax.InputSource -import java.io.StringReader -import javax.xml.XMLConstants -import javax.xml.parsers.DocumentBuilderFactory +import javax.xml.stream.XMLInputFactory /** - * SIS XML 解码（XXE 防护；META + TYPE 分派 + FLOP 集合解析）。 - * 阶段 2 先覆盖 FLOP-GTDT 纵向链路所需字段；其余 STYP 仅完成 META/FLID 提取。 + * SIS XML 解码：jackson-dataformat-xml 直接映射到 [SisMessage] 数据类（不再手写 DOM）。 + * XXE 防护：XMLInputFactory 禁 DTD/外部实体。 + * 覆盖：SCHD DNLD/RESP（FLTR 记录集，§5 入口）、SCHD ADFT（单记录 §2.1）、 + * FLOP（单航班增量 §6.1）、FDEL（终止实例 §6.2）。 */ @Singleton class JacksonXmlCodec : XmlCodec { - private val builderFactory: DocumentBuilderFactory = DocumentBuilderFactory.newInstance().apply { - isNamespaceAware = false - isExpandEntityReferences = false - setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true) - setFeature("http://apache.org/xml/features/disallow-doctype-decl", true) - setFeature("http://xml.org/sax/features/external-general-entities", false) - setFeature("http://xml.org/sax/features/external-parameter-entities", false) - try { - setAttribute(XMLConstants.ACCESS_EXTERNAL_DTD, "") - setAttribute(XMLConstants.ACCESS_EXTERNAL_SCHEMA, "") - } catch (_: IllegalArgumentException) { - // JDK 实现差异：忽略不支持的属性 - } + private val mapper: XmlMapper = XmlMapper( + XmlFactory( + XMLInputFactory.newInstance().apply { + setProperty(XMLInputFactory.SUPPORT_DTD, false) + setProperty(XMLInputFactory.IS_SUPPORTING_EXTERNAL_ENTITIES, false) + }, + ), + ).apply { + registerKotlinModule() + configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false) } override fun decode(rawXml: String): DecodeResult { @@ -39,47 +39,42 @@ class JacksonXmlCodec : XmlCodec { if (trimmed.isEmpty() || !trimmed.startsWith("<")) { return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "empty-or-non-xml")) } - return try { - val doc = builderFactory.newDocumentBuilder().parse(InputSource(StringReader(trimmed))) - val root = doc.documentElement ?: return DecodeResult.Err( - DecodeFailure(ErrorClass.MALFORMED, "missing-root"), - ) - if (!root.tagName.equals("MSG", ignoreCase = true)) { - return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "root-not-msg:${root.tagName}")) - } - - val metaEl = firstChildElement(root, "META") - ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-meta")) - val sndr = childText(metaEl, "SNDR") - ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-sndr")) - val seqn = childText(metaEl, "SEQN")?.toLongOrNull() - ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-or-invalid-seqn")) - val dttm = childText(metaEl, "DTTM")?.toLongOrNull() - ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-or-invalid-dttm")) - val type = childText(metaEl, "TYPE")?.uppercase() - ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-type")) - val styp = childText(metaEl, "STYP")?.uppercase() - ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-styp")) - - val kind = kindOf(type, styp) - val body = when (kind) { - is MsgKind.Flop -> parseFlopBody(root, type) - is MsgKind.Schd -> null - } - - DecodeResult.Ok( - DecodedMessage( - meta = MetaFields(sndr = sndr, type = type, styp = styp, seqn = seqn, dttm = dttm), - kind = kind, - rawXml = trimmed, - body = body, - ), - ) - } catch (e: NumberFormatException) { - DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "invalid-number:${e.message}")) + val msg = try { + mapper.readValue(trimmed, SisMessage::class.java) } catch (e: Exception) { - DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "xml-parse:${e.message ?: e.javaClass.simpleName}")) + return DecodeResult.Err( + DecodeFailure(ErrorClass.MALFORMED, "xml-parse:${e.message?.take(200) ?: e.javaClass.simpleName}"), + ) } + val meta = msg.meta + ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-meta")) + val sndr = meta.sndr?.trim()?.takeIf { it.isNotEmpty() } + ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-sndr")) + val seqn = meta.seqn + ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-or-invalid-seqn")) + val dttm = meta.dttm + ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-or-invalid-dttm")) + val type = meta.type?.uppercase() + ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-type")) + val styp = meta.styp?.uppercase() + ?: return DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "missing-styp")) + + val kind = kindOf(type, styp) + val body: Any? = when (kind) { + is MsgKind.Schd -> parseScheduleBody(msg) + is MsgKind.Flop -> parseFlightSection(msg.flop) + MsgKind.Fdel -> parseFlightSection(msg.fdel) + is MsgKind.Unsupported -> null + } + + return DecodeResult.Ok( + DecodedMessage( + meta = MetaFields(sndr = sndr, type = type, styp = styp, seqn = seqn, dttm = dttm), + kind = kind, + rawXml = trimmed, + body = body, + ), + ) } override fun encodeRqrd(kind: String, rangeJson: String): String = @@ -89,82 +84,92 @@ class JacksonXmlCodec : XmlCodec { "SCHD" -> MsgKind.Schd( when (styp) { "RESP" -> MsgKind.SchdSubtype.RESP - "DNLD" -> MsgKind.SchdSubtype.DNLD "ADFT" -> MsgKind.SchdSubtype.ADFT else -> MsgKind.SchdSubtype.DNLD }, ) + "FDEL" -> MsgKind.Fdel "FLOP" -> MsgKind.Flop(styp) - else -> MsgKind.Flop(styp) + else -> MsgKind.Unsupported("$type-$styp") // §9 未支持类型，不猜测分派 } - private fun parseFlopBody(root: Element, type: String): FlopPayload? { - val section = firstChildElement(root, type) ?: return null + /** SCHD DNLD/RESP/ADFT：RECS 缺失/非法 → -1（处理层 §5.2 整包拒绝）。 */ + private fun parseScheduleBody(msg: SisMessage): ScheduleBody? { + val section = msg.schd ?: return null + val records = section.fltr.mapNotNull { recordOf(it) } + return ScheduleBody(recsDeclared = section.recs ?: -1, records = records) + } + + /** FLOP/FDEL：单航班段 Map → 载荷；无 FLID 返回 null（调用方按缺载荷处理）。 */ + private fun parseFlightSection(section: Map?): FlopPayload? { + section ?: return null val scalars = linkedMapOf() val collections = linkedMapOf>>() - - section.childNodes.let { nodes -> - for (i in 0 until nodes.length) { - val node = nodes.item(i) - if (node.nodeType != Node.ELEMENT_NODE) continue - val el = node as Element - val tag = el.tagName.uppercase() - when { - tag in COLLECTION_TAGS -> { - collections.getOrPut(tag) { mutableListOf() }.add(parseCollectionItem(el)) - } - tag == "FLID" -> scalars["FLID"] = el.textContent.trim() - hasElementChildren(el) -> Unit - else -> scalars[tag] = el.textContent.trim() + var flid: String? = null + section.forEach { (key, value) -> + val tag = key.uppercase() + when { + tag == "FLID" -> flid = value.toString().trim() + tag in COLLECTION_TAGS -> value.toCollectionItems()?.let { items -> + collections.getOrPut(tag) { mutableListOf() }.addAll(items) } + value is String -> scalars[tag] = value.trim() } } - - val flid = scalars.remove("FLID") ?: return null - return FlopPayload(flid = flid, scalars = scalars, collections = collections.mapValues { it.value.toList() }) + return flid?.takeIf { it.isNotEmpty() }?.let { + FlopPayload(flid = it, scalars = scalars, collections = collections.mapValues { m -> m.value.toList() }) + } } - private fun parseCollectionItem(el: Element): Map { - val item = linkedMapOf() - if (el.hasAttributes()) { - for (i in 0 until el.attributes.length) { - val attr = el.attributes.item(i) - item[attr.nodeName.uppercase()] = attr.nodeValue.trim() - } - } - el.childNodes.let { nodes -> - for (i in 0 until nodes.length) { - val node = nodes.item(i) - if (node.nodeType != Node.ELEMENT_NODE) continue - val child = node as Element - item[child.tagName.uppercase()] = child.textContent.trim() - } - } - return item - } - - private fun firstChildElement(parent: Element, tag: String): Element? { - parent.childNodes.let { nodes -> - for (i in 0 until nodes.length) { - val node = nodes.item(i) - if (node.nodeType == Node.ELEMENT_NODE && (node as Element).tagName.equals(tag, ignoreCase = true)) { - return node + /** FLTR Map → ScheduleRecord：标量/集合分离，集合项含源序号属性。 */ + private fun recordOf(fltr: Map): ScheduleRecord? { + val scalars = linkedMapOf() + val collections = linkedMapOf>>() + var flid: String? = null + var seqn = 0L + fltr.forEach { (key, value) -> + val tag = key.uppercase() + when { + tag == "FLID" -> flid = value.toString().trim() + tag == "SEQN" -> seqn = (value as? Number)?.toLong() ?: value.toString().trim().toLongOrNull() ?: 0L + tag in COLLECTION_TAGS -> value.toCollectionItems()?.let { items -> + collections.getOrPut(tag) { mutableListOf() }.addAll(items) } + value is String -> scalars[tag] = value.trim() } } - return null + return flid?.takeIf { it.isNotEmpty() }?.let { + ScheduleRecord(flid = it, scalars = scalars, collections = collections.mapValues { m -> m.value.toList() }, seqn = seqn) + } } - private fun childText(parent: Element, tag: String): String? = - firstChildElement(parent, tag)?.textContent?.trim()?.takeIf { it.isNotEmpty() } + /** + * 集合值归一：重复元素 → List（属性与子元素展平）；单元素 → 单项列表； + * 混合文本（如 DELY 备注文本）落到 REMC 键（XmlMapper 绑定的已知弱项，尽力保留）。 + */ + private fun Any?.toCollectionItems(): List>? = when (this) { + is List<*> -> mapNotNull { it?.toCollectionItem() } + is Map<*, *> -> listOfNotNull(toCollectionItem(this)) + is String -> listOf(mapOf("REMC" to trim())) + else -> null + } - private fun hasElementChildren(el: Element): Boolean { - el.childNodes.let { nodes -> - for (i in 0 until nodes.length) { - if (nodes.item(i).nodeType == Node.ELEMENT_NODE) return true + private fun toCollectionItem(map: Map<*, *>): Map { + val out = linkedMapOf() + map.forEach { (k, v) -> + val key = k.toString().uppercase() + when (v) { + is String -> out[key] = v.trim() + is Number -> out[key] = v.toString() } } - return false + return out + } + + private fun Any?.toCollectionItem(): Map? = when (this) { + is Map<*, *> -> toCollectionItem(this) + is String -> mapOf("REMC" to trim()) + else -> null } companion object { diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/codec/SisMessageBody.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/codec/SisMessageBody.kt index 8e84068..a776068 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/codec/SisMessageBody.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/codec/SisMessageBody.kt @@ -1,11 +1,73 @@ package com.gzzn.omms.msgexchange.codec +import com.fasterxml.jackson.annotation.JsonIgnoreProperties +import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlElementWrapper +import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlProperty +import com.gzzn.omms.msgexchange.domain.flight.ScheduleRecord +import java.time.LocalDate + /** - * FLOP 报文解析体（v2 §4 命令化输入）。 - * scalars：FLID/FFID 等标量；collections：GTDT/CKDT 等重复集合（含源序号属性）。 + * FLOP 报文解析体（单航班增量 §6.1）。 + * scalars：FLID 以外的标量；collections：GTDT/CKDT 等重复集合（含源序号属性）。 */ data class FlopPayload( val flid: String, val scalars: Map = emptyMap(), val collections: Map>> = emptyMap(), ) + +/** + * SCHD（DNLD/RESP/ADFT）解析体（docs/flight-state.md §5）。 + * RECS 声明数与实收 FLTR 记录分离承载，完整性校验（§5.2）在处理层执行； + * 协议无报文级覆盖日字段——scope 由处理层按记录 SODT 推导。 + */ +data class ScheduleBody( + val recsDeclared: Int, + val records: List, + val scopeStart: LocalDate? = null, + val scopeEnd: LocalDate? = null, +) + +// ===================================================================== +// XML 直接映射（jackson-dataformat-xml 数据绑定，不再手写 DOM 遍历）。 +// 信封强类型（META/段结构），FLTR/航班段内标签为开放集 → Map +// 泛型承载（45+ 标签逐一建模不成比例）；集合项 = 属性 + 子元素展平的 Map。 +// 已知弱项：DELY 的混合文本（元素文本）在 XmlMapper 绑定下不保证保留， +// 属性 CODE/STRT/DURA 保留；备注文本待真实报文验收（§10）。 +// ===================================================================== + +@JsonIgnoreProperties(ignoreUnknown = true) +data class SisMessage( + @JacksonXmlProperty(localName = "META") + val meta: SisMeta? = null, + @JacksonXmlProperty(localName = "SCHD") + val schd: SchdSection? = null, + @JacksonXmlProperty(localName = "FLOP") + val flop: Map? = null, + @JacksonXmlProperty(localName = "FDEL") + val fdel: Map? = null, +) + +@JsonIgnoreProperties(ignoreUnknown = true) +data class SisMeta( + @JacksonXmlProperty(localName = "SNDR") + val sndr: String? = null, + @JacksonXmlProperty(localName = "SEQN") + val seqn: Long? = null, + @JacksonXmlProperty(localName = "DTTM") + val dttm: Long? = null, + @JacksonXmlProperty(localName = "TYPE") + val type: String? = null, + @JacksonXmlProperty(localName = "STYP") + val styp: String? = null, +) + +/** SCHD 段：RECS 声明数 + FLTR 记录集（SIS §3.16 样例结构）。 */ +@JsonIgnoreProperties(ignoreUnknown = true) +data class SchdSection( + @JacksonXmlProperty(localName = "RECS") + val recs: Int? = null, + @JacksonXmlElementWrapper(useWrapping = false) + @JacksonXmlProperty(localName = "FLTR") + val fltr: List> = emptyList(), +) diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/config/HistoryProps.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/config/HistoryProps.kt new file mode 100644 index 0000000..8e34d70 --- /dev/null +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/config/HistoryProps.kt @@ -0,0 +1,25 @@ +package com.gzzn.omms.msgexchange.config + +import io.micronaut.context.annotation.ConfigurationProperties + +/** 生命周期与保留期（docs/flight-state.md §8）。窗口按机场时区计算。 */ +@ConfigurationProperties("msgx.history") +class HistoryProps { + /** 已取消（CNCL 非空）超过 N 小时。 */ + var cancelledHours: Long = 48 + + /** 到港/离港终态（NAAT/NEAT，含义待术语表确认 §10）超过 N 小时。 */ + var terminalHours: Long = 48 + + /** STATE = DELETED 超过 N 小时。 */ + var deletedHours: Long = 48 + + /** 无终态字段：最后有效更新超过兜底期限（默认 7 天）。 */ + var idleHours: Long = 24 * 7 + + /** 留痕 SCHD_SNAP_LOG 保留天数（§8.3，按 (SCOPE_END, RECV_AT) 清理）。 */ + var snapLogRetentionDays: Long = 90 + + /** 是否接通历史存储——未接通时 HISTORY_SWEEP 必须删 0 条（§8.2）。 */ + var historyStoreEnabled: Boolean = false +} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/config/OperationDayProps.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/config/OperationDayProps.kt new file mode 100644 index 0000000..69e887a --- /dev/null +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/config/OperationDayProps.kt @@ -0,0 +1,16 @@ +package com.gzzn.omms.msgexchange.config + +import io.micronaut.context.annotation.ConfigurationProperties + +/** + * 运营日计算（docs/flight-state.md §3.5）：由 SODT（ddMMMyyHHmm）与机场时区计算； + * 切日边界业务配置——不得假设等于接收日期或自然日零点（默认 0 点为占位，待业务确认 §10）。 + */ +@ConfigurationProperties("msgx.operation-day") +class OperationDayProps { + /** 机场时区（IANA）。 */ + var zone: String = "Asia/Shanghai" + + /** 切日边界：SODT 本地时刻早于该小时的归属前一运营日（0–23）。 */ + var cutoffHour: Int = 0 +} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/config/PipelineProps.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/config/PipelineProps.kt index 5815a88..f16015d 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/config/PipelineProps.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/config/PipelineProps.kt @@ -5,20 +5,16 @@ import java.time.Duration /** * ACMA-8 参数表（v4）初值；阶段 0 现网基线校准。 - * U03（N02）：Micronaut 要求嵌套配置类同样标注 @ConfigurationProperties，否则 msgx.pipeline/schd/ - * identity/consistency-check.* 全部静默回落 Kotlin 默认值（当前 yml 初值与默认值一致，现象被掩盖）。 + * U03（N02）：Micronaut 要求嵌套配置类同样标注 @ConfigurationProperties，否则 + * msgx.pipeline/schd/identity.* 全部静默回落 Kotlin 默认值。 */ @ConfigurationProperties("msgx") class PipelineProps { - var phase: Phase = Phase.A var serviceName: String = "msgexchangeapi" var registerEureka: Boolean = true var pipeline: Pipeline = Pipeline() var schd: Schd = Schd() var identity: Identity = Identity() - var consistencyCheck: ConsistencyCheck = ConsistencyCheck() - - enum class Phase { A, B } @ConfigurationProperties("pipeline") class Pipeline { @@ -50,10 +46,4 @@ class PipelineProps { /** CONFIRM（矩阵 #11）：SEQN 重置作用域确认前保持 false，计算集中此处（I3）。 */ var includeDayBoundary: Boolean = false } - - @ConfigurationProperties("consistency-check") - class ConsistencyCheck { - var onStartup: Boolean = true // 阶段 A 必选哨兵 - var dailySampleRatio: Double = 0.01 - } } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/delivery/Dispatcher.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/delivery/Dispatcher.kt index bfc1a52..0787088 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/delivery/Dispatcher.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/delivery/Dispatcher.kt @@ -1,9 +1,9 @@ package com.gzzn.omms.msgexchange.delivery -import com.fasterxml.jackson.databind.ObjectMapper import com.gzzn.omms.msgexchange.config.PipelineProps import com.gzzn.omms.msgexchange.domain.ErrorClass import com.gzzn.omms.msgexchange.domain.EventStatus +import com.gzzn.omms.msgexchange.domain.EventType import com.gzzn.omms.msgexchange.domain.MsgEvent import com.gzzn.omms.msgexchange.domain.Targets import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository @@ -12,26 +12,29 @@ import jakarta.inject.Singleton import java.time.Duration import java.time.Instant -/** 对外投递端口（Kafka 同步确认；阶段 B 追加 ES/Redis 投影写入）。 */ +/** 对外投递端口（Kafka 同步确认，at-least-once；阶段 B 追加 ES 投影写入）。 */ interface DeliveryPort { - /** Kafka 发送（同步确认，at-least-once）。topic 由 target 映射：KAFKA:msg→"msg"，KAFKA:schd→"schd"。 */ - fun sendKafka(topic: String, payloadJson: String) + /** KAFKA_MSG 变化通知（key=FLID）。 */ + fun sendKafka(topic: String, key: String, payloadJson: String) - /** 阶段 B：ES flight_hts 写入。 */ - fun indexFlightHts(payloadJson: String) + /** + * KAFKA_SCHD 整态（key=FLID）。KAFKA_MSG 与 KAFKA_SCHD 映射同一 topic 语义由适配层定； + * target→topic：KAFKA:msg→"msg"，KAFKA:schd→"schd"。 + */ + fun sendKafkaSchd(topic: String, key: String, payloadJson: String) + /** TOMBSTONE：key=FLID、value=null——整态键缺失表示删除旧值（§7.3）。 */ + fun sendKafkaNull(topic: String, key: String) /** 连通性探测（健康检查用）；默认 true，真实 Kafka 实装时覆写为 producer metadata 校验。 */ fun ping(): Boolean = true } /** - * ACMA-8 流程 3：投递调度——每 target 严格 FIFO（I1 双层同策略）。 - * N03（U06）：KAFKA_SCHD 不走逐条循环（唯一出口是 flushSchd 批量聚合），逐条循环显式排除。 - * U08 闭环：逐条与批量失败都在持有具体事件/批次的边界完成状态迁移—— - * · 逐条：scheduleRetry(attempts+1, backoff) / 达上限 markDead(EXHAUSTED)（DLQ 保留行）； - * · 批量（flushSchd）：整批退避，队首 nextAttemptAt 未到不 claim；达上限整批 DEAD/DLQ； - * · loop 只作最后防线，不吞 InterruptedException（致命/中断错误不被普通恢复吞掉）。 + * 投递调度（docs/flight-state.md §7.3）：逐条 KAFKA_MSG 严格 FIFO； + * KAFKA_SCHD 走 flushSchd 批量——同一 FLID 未发事件按最新 STATE_VERSION 合并输出， + * TOMBSTONE 发 null 值消息。两主题间不保证顺序（§7.3）。 + * 批量闭环：队首退避未到期不 claim；发送失败整批 attempts+1 退避，达上限整批 DEAD/DLQ。 */ @Singleton class Dispatcher( @@ -45,9 +48,9 @@ class Dispatcher( @Volatile private var running = true - private var lastFlush: Instant = Instant.EPOCH + private var lastFlush: Instant? = null - /** 优雅停机：loop 收尾后退出；线程中断由 Runner 负责。 */ + /** 优雅停机：loop 收尾后退出；线程中断由 PipelineLifecycle 负责。 */ fun stop() { running = false } @@ -60,88 +63,84 @@ class Dispatcher( Thread.currentThread().interrupt() return } catch (e: Exception) { - // U08：投递循环只作最后防线；致命 Error 不吞。tick 内失败迁移已完成。 sleepQuietly(props.pipeline.pollInterval) } - // N18：轮询间隔取参数表（下限 50ms，避免退避节律被吞） - sleepQuietly(props.pipeline.pollInterval.coerceAtLeast(Duration.ofMillis(50))) } } internal fun tick() { - val targets = if (props.phase == PipelineProps.Phase.A) Targets.phaseA else Targets.phaseB - for (t in targets) { - if (t == Targets.KAFKA_SCHD) continue // N03：schd 唯一出口 flushSchd - val head = msgEvents.headUnsent(t) ?: continue - if (head.state == EventStatus.PENDING && head.nextAttemptAt != null && head.nextAttemptAt > scheduler.now()) { - // 队头退避未到期：等待，不跳过（保序）；超时升级归 U13（WP2） - continue - } - try { - deliver(t, head) - msgEvents.markSent(head.eventId!!) - log.debug("sent target={} eventId={}", t, head.eventId) - // ACM2-28：阶段 B Redis 投影废弃，读模型投递统一转由 Kafka 事件或 ES 处理 - } catch (e: Exception) { - retryOrDead(head, e.message ?: "unknown") - } - } - if (flushDue()) { - flushSchd() + val head = msgEvents.headUnsent(Targets.KAFKA_MSG) + if (head != null && (head.state == EventStatus.SENT || head.nextAttemptAt == null || head.nextAttemptAt <= scheduler.now())) { + deliver(Targets.KAFKA_MSG, head) } + if (flushDue()) flushSchd() + if (running) sleepQuietly(props.pipeline.pollInterval) } private fun flushDue(): Boolean = - Duration.between(lastFlush, scheduler.now()) >= props.schd.flushPeriod + lastFlush?.let { Duration.between(it, scheduler.now()) >= props.schd.flushPeriod } ?: false - /** 单条事件失败迁移：attempts+1；达上限 DEAD(EXHAUSTED)（DLQ，attempts 落库审计），否则退避重试。 */ - private fun retryOrDead(e: MsgEvent, lastError: String) { - val attempts = e.attempts + 1 - if (scheduler.exhausted(attempts)) { - log.error("event DEAD(DLQ) eventId={} attempts={} lastError={}", e.eventId, attempts, lastError) - msgEvents.markDead(e.eventId!!, ErrorClass.EXHAUSTED, lastError, attempts) - } else { - log.warn("event retry scheduled eventId={} attempts={} nextAttemptAt={}", e.eventId, attempts, scheduler.nextAttemptAt(attempts)) - msgEvents.scheduleRetry(e.eventId!!, scheduler.nextAttemptAt(attempts), attempts) + private fun deliver(target: String, e: MsgEvent) { + try { + when (target) { + Targets.KAFKA_MSG -> port.sendKafka("msg", e.partitionKey, e.payloadJson) + } + msgEvents.markSent(e.eventId ?: return) + } catch (ex: Exception) { + retryOrDead(e, ex.message ?: ex.javaClass.simpleName) } } - private fun deliver(target: String, e: MsgEvent) = when (target) { - Targets.KAFKA_MSG -> port.sendKafka("msg", e.payloadJson) - Targets.KAFKA_SCHD -> error("KAFKA_SCHD must go through flushSchd (N03)") - Targets.ES_FLIGHT_HTS -> port.indexFlightHts(e.payloadJson) - else -> error("unknown target $target") - } - - /** - * 流程 3 flushSchd：批上限 + 每 FLID 最新一态聚合（topic "schd"，wire=FLTR JSON 数组）。 - * U08 批量闭环：队首退避未到期不 claim；发送失败 → 整批 attempts+1（退避）或达上限整批 DEAD/DLQ； - * lastFlush 仅在成功（含空批）后推进。 - */ + /** flushSchd：同 FLID 未发事件按最新 STATE_VERSION 合并（§7.3）；TOMBSTONE 发 null。 */ internal fun flushSchd() { - val pending = msgEvents.claimBatch(Targets.KAFKA_SCHD, limit = props.schd.flushLimit) - if (pending.isEmpty()) { + val batch = try { + msgEvents.mergePendingSchd(props.schd.flushLimit) + } catch (e: Exception) { + log.warn("mergePendingSchd failed: {}", e.message) + return + } + if (batch.isEmpty()) { lastFlush = scheduler.now() return } - val due = pending.first() - if (due.nextAttemptAt != null && due.nextAttemptAt > scheduler.now()) { - return // 队首仍在退避：整批等待，不推进 lastFlush（到期再试） + val failures = mutableListOf() + for (e in batch) { + try { + when (e.eventType) { + EventType.TOMBSTONE -> port.sendKafkaNull("schd", e.partitionKey) + EventType.UPSERT -> port.sendKafkaSchd("schd", e.partitionKey, e.payloadJson) + } + } catch (ex: Exception) { + failures.add(e) + } } - val payload = SchdAggregation.latestPerFlight(pending).joinToString(",", "[", "]") - log.debug("flushSchd batch size={} payloadLen={}", pending.size, payload.length) - try { - port.sendKafka("schd", payload) - } catch (e: Exception) { - log.warn("flushSchd send failed batch={} -> batch retryOrDead: {}", pending.size, e.message) - pending.forEach { retryOrDead(it, "schd-send: ${e.message ?: "unknown"}") } - return // 不推进 lastFlush：整批退避（含 DEAD 出队）后到期重试 - } - msgEvents.markAllSent(pending.mapNotNull { it.eventId }) - log.info("flushSchd sent batch={}", pending.size) + val sentIds = batch.mapNotNull { it.eventId }.toSet() - failures.mapNotNull { it.eventId }.toSet() + if (sentIds.isNotEmpty()) msgEvents.markAllSent(sentIds.toList()) + // 被最新版本合并压掉的未发事件同样关闭（§7.3：同 FLID 只按最新 STATE_VERSION 输出一次） + val sentVersions = batch.associate { it.partitionKey to it.stateVersion } + runCatching { + while (true) { + val superseded = msgEvents.mergePendingSchd(props.schd.flushLimit) + .filter { sentVersions[it.partitionKey]?.let { v -> it.stateVersion < v } == true } + if (superseded.isEmpty()) break + msgEvents.markAllSent(superseded.mapNotNull { it.eventId }) + } + }.onFailure { log.warn("superseded cleanup failed: {}", it.message) } + failures.forEach { retryOrDead(it, it.lastError ?: "send-failed") } lastFlush = scheduler.now() } + /** 单条事件失败迁移：attempts+1；达上限 DEAD(EXHAUSTED)（DLQ，attempts 落库审计），否则退避重试。 */ + private fun retryOrDead(e: MsgEvent, lastError: String) { + val eventId = e.eventId ?: return + val attempts = e.attempts + 1 + if (scheduler.exhausted(attempts)) { + msgEvents.markDead(eventId, ErrorClass.EXHAUSTED, lastError, attempts) + } else { + msgEvents.scheduleRetry(eventId, scheduler.nextAttemptAt(attempts), attempts) + } + } + private fun sleepQuietly(d: Duration) { if (!d.isNegative && !d.isZero) Thread.sleep(d.toMillis().coerceAtLeast(1)) } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/delivery/SchdAggregation.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/delivery/SchdAggregation.kt deleted file mode 100644 index 32f17cd..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/delivery/SchdAggregation.kt +++ /dev/null @@ -1,22 +0,0 @@ -package com.gzzn.omms.msgexchange.delivery - -import com.gzzn.omms.msgexchange.domain.MsgEvent -import com.gzzn.omms.msgexchange.domain.SchdPush - -/** - * ACMA-8 流程 3 flushSchd 聚合（纯函数，可单测）： - * 按 PARTITION_KEY(=FLID) 分组、组内取 EVENT_ID 最大——v4 显式 max(EVENT_ID)， - * 不依赖集合遍历序（FIX：现役 buffer 无去重会重复投递旧值）。 - */ -object SchdAggregation { - fun latestPerFlight(pending: List): List = - pending - .groupBy { it.partitionKey ?: "" } - .map { (_, evs) -> evs.maxBy { it.eventId ?: 0 }.payloadJson } - - /** 由 Decision 直接构造时的等价聚合（测试与 dispatcher 共用语义）。 */ - fun latestPerFlightOfPush(pushes: List): List = - pushes - .groupBy { it.flid } - .map { (_, ps) -> ps.maxBy { it.eventSeq }.payloadJson } -} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/Decision.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/Decision.kt index 2681ab8..311580b 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/Decision.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/Decision.kt @@ -1,44 +1,31 @@ package com.gzzn.omms.msgexchange.domain -/** - * ACMA-8 流程 2：Handler 决策（纯函数）产物——状态与报文进，变更与事件出， - * 不直接触碰数据库或 Kafka。 - */ -data class Decision( - val flightChanges: List = emptyList(), - val msgNotifies: List = emptyList(), // → MSG_EVENT(KAFKA:msg) - val schdPush: List = emptyList(), // → MSG_EVENT(KAFKA:schd)，PARTITION_KEY=FLID - val outboundIntents: List = emptyList(), // → COUTMSGS（沿用既有列语义） - val refUpserts: List = emptyList(), // → 静态主数据（独立 PG reference 库，ACM2-11） -) +import com.gzzn.omms.msgexchange.domain.flight.MergeChange +import com.gzzn.omms.msgexchange.domain.flight.ScheduleRecord +import java.time.LocalDate /** - * 航班状态变更（阶段 A 落自有库 FLIGHT_SCHD 宽表，与事件同事务原子提交；ACM2-28 定案）。 - * fields = 本报文变更的字段集（field → value，与 legacy flightInfo hash 同构）， - * 仓储按「字段级合并」落库（仅新增/覆盖，不删除缺失字段，与 legacy hmset 同语义）。 + * 处理决策（docs/flight-state.md）——解码与校验进，落库计划出，纯数据不触碰 DB/Kafka。 + * 事务边界、PIPELINE_LOCK、事件登记由处理层执行（§5.1/§6）。 */ -data class FlightChange( - val flid: String, - val fields: Map, - val maid: String? = null, -) +sealed interface Decision { + /** + * SCHD DNLD/RESP：已过 §5.2 报文完整性校验的记录集。 + * §5.3 归属校验与 §5.4 upsert 在事务内执行（需读既有 OPERATION_DAY）。 + */ + data class Schedule( + val records: List, + /** 报文覆盖运营日范围（单日快照时两者相等；无法确定时为 null → 整包拒绝）。 */ + val scopeStart: LocalDate?, + val scopeEnd: LocalDate?, + ) : Decision -data class NotifyPayload(val payloadJson: String) + /** FLOP 增量合并（§6.1）/ ADFT（§2.1 语义待确认，按 MergeChange 承载）。 */ + data class Dynamic(val change: MergeChange) : Decision -data class SchdPush( - val flid: String, - val payloadJson: String, // KAFKA_SCHD 出站载荷（报文线格式，与库内展开存储无关） - val eventSeq: Long = 0, // 由事务插入时赋 EVENT_ID 语义序，聚合取 max -) + /** FDEL 标记删除（§6.2）。 */ + data class Delete(val flid: String) : Decision -data class OutboundIntent( - val coutmsgsXml: String, - val ackReqd: Boolean = true, -) - -data class RefUpsert( - val rtype: String, - val rkey: String, - val payloadJson: String, - val source: String, -) + /** 幂等无变化（重复/迟到等，记成功但不推进任何状态）。 */ + data object NoOp : Decision +} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/DecodedMessage.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/DecodedMessage.kt index b39b686..167a898 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/DecodedMessage.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/DecodedMessage.kt @@ -11,10 +11,14 @@ data class MetaFields( val dttm: Long, ) -/** 消息分派（sealed + 穷尽 when，ACMA-6 选型；取代 legacy 反射 get{TYPE}()）。 */ +/** 消息分派（sealed + 穷尽 when；FDEL 一等公民——终止航班实例 §6.2）。 */ sealed interface MsgKind { data class Schd(val subtype: SchdSubtype) : MsgKind - data class Flop(val subtype: String) : MsgKind // 29 类 STYP，阶段 2/3 逐类翻译 + data class Flop(val subtype: String) : MsgKind // 运行动态 STYP（FDEL 除外） + data object Fdel : MsgKind + + /** 未支持类型（§9：FAILED(UNSUPPORTED)，达阈值转 DEAD）。 */ + data class Unsupported(val tag: String) : MsgKind enum class SchdSubtype { RESP, DNLD, ADFT } } @@ -30,5 +34,7 @@ data class DecodedMessage( get() = when (val k = kind) { is MsgKind.Schd -> "SCHD-${k.subtype.name}" is MsgKind.Flop -> "FLOP-${k.subtype}" + MsgKind.Fdel -> "FDEL" + is MsgKind.Unsupported -> k.tag } } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/MsgEvent.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/MsgEvent.kt index 5117302..484e10c 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/MsgEvent.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/MsgEvent.kt @@ -1,29 +1,22 @@ package com.gzzn.omms.msgexchange.domain -/** - * ACMA-8 MSG_EVENT（统一投递事件 / outbox）。 - * TARGET 阶段化：KAFKA:msg、KAFKA:schd 自阶段 A；ES:flight_hts、REDIS:flightInfo 仅阶段 B 投影期。 - */ -object Targets { - const val KAFKA_MSG = "KAFKA:msg" - const val KAFKA_SCHD = "KAFKA:schd" - const val ES_FLIGHT_HTS = "ES:flight_hts" - @Deprecated("Retired in ACM2-28: Redis projection removed") - const val REDIS_FLIGHT_INFO = "REDIS:flightInfo" - - /** 阶段 A 投递目标（仅 Kafka）——ACM2-28：Delivery 阶段 A/B 均不写 Redis。 */ - val phaseA: List = listOf(KAFKA_MSG, KAFKA_SCHD) - - /** 阶段 B 追加投影目标（仅 ES 历史库；Redis 投影按 ACM2-28 废弃）。 */ - val phaseB: List = phaseA + listOf(ES_FLIGHT_HTS) -} +/** MSG_EVENT 事件形态（§7.3）。 */ +enum class EventType { UPSERT, TOMBSTONE } +/** MSG_EVENT 投递状态（outbox 状态机）。 */ enum class EventStatus { PENDING, SENT, DEAD } +/** + * MSG_EVENT（outbox，§3.2）：状态、变更、删除通知。 + * KAFKA_SCHD 整态 + KAFKA_MSG 变化通知；TOMBSTONE 仅在 ACTIVE→DELETED 时 + * 与删除同事务登记（§7.3），投递失败持续重试。 + */ data class MsgEvent( val eventId: Long? = null, val target: String, - val partitionKey: String? = null, // schd 事件恒为 FLID（v4 显式声明） + val partitionKey: String, // 恒为 FLID + val eventType: EventType = EventType.UPSERT, + val stateVersion: Long = 0, // 发布时航班版本；Dispatcher 合并同 FLID 未发事件取最新（§7.3） val payloadJson: String, val state: EventStatus = EventStatus.PENDING, val attempts: Int = 0, diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/OperationDay.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/OperationDay.kt new file mode 100644 index 0000000..301c7af --- /dev/null +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/OperationDay.kt @@ -0,0 +1,49 @@ +package com.gzzn.omms.msgexchange.domain + +import java.time.LocalDate +import java.time.LocalDateTime +import java.time.ZoneId + +/** + * 运营日计算（docs/flight-state.md §3.5）：由计划运行时间字段 SODT（ddMMMyyHHmm） + * 与机场时区计算；切日边界业务配置——默认 0 点为占位，待业务确认（§10）。 + */ +class OperationDayCalculator( + zone: ZoneId, + cutoffHour: Int, +) { + private val zone: ZoneId = zone + /** 切日边界：SODT 本地时刻早于该小时的归属前一运营日（0–23，越界按 0）。 */ + private val cutoffHour: Int = cutoffHour.coerceIn(0, 23) + + /** SODT → 运营日；输入 null/空/非法返回 null（不抛异常，调用方按"运营日不可计算"处理）。 */ + fun compute(sodt: String?): LocalDate? { + if (sodt.isNullOrBlank()) return null + val local = parseSodt(sodt.trim()) ?: return null + val day = local.atZone(zone).toLocalDate() + return if (local.hour < cutoffHour) day.minusDays(1) else day + } + + companion object { + /** + * SIS SODT 线格式：ddMMMyyHHmm（如 15DEC031723），月份英文三字母、大小写不敏感。 + * 两位年显式按 2000 基准展开（java.time 的 yy reduced-value 解析跨实现不一致， + * 显式展开保证 AODB 侧年份窗口唯一口径；基准年待真实报文验收确认 §10）。 + */ + private val SODT_REGEX = Regex("(\\d{1,2})([A-Za-z]{3})(\\d{2})(\\d{2})(\\d{2})") + + private val MONTHS = mapOf( + "JAN" to 1, "FEB" to 2, "MAR" to 3, "APR" to 4, "MAY" to 5, "JUN" to 6, + "JUL" to 7, "AUG" to 8, "SEP" to 9, "OCT" to 10, "NOV" to 11, "DEC" to 12, + ) + + internal fun parseSodt(raw: String): LocalDateTime? { + val m = SODT_REGEX.matchEntire(raw.uppercase()) ?: return null + val (d, mon, yy, hh, mm) = m.destructured + val month = MONTHS[mon] ?: return null + return runCatching { + LocalDateTime.of(2000 + yy.toInt(), month, d.toInt(), hh.toInt(), mm.toInt()) + }.getOrNull() + } + } +} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/ProcState.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/ProcState.kt index 503d663..07805d1 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/ProcState.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/ProcState.kt @@ -1,16 +1,18 @@ package com.gzzn.omms.msgexchange.domain /** - * ACMA-8 数据模型 / PROC_STATE 状态机（六迁移表）。 + * ACMA-8 数据模型 / PROC_STATE 状态机（docs/flight-state.md §3.2：每消息一行， + * 处理状态与重试结果；兼作快照重放判定 §5.1）。 */ enum class ProcStatus { PENDING, FAILED, SUCCEEDED, SKIPPED, DEAD } -enum class ErrorClass { MALFORMED, CODEC_ERROR, EXHAUSTED, INFRA, UNSUPPORTED } +/** §9 错误分类：PROTOCOL = 整包拒绝（归属日不符等），不重试交人工。 */ +enum class ErrorClass { MALFORMED, PROTOCOL, CODEC_ERROR, EXHAUSTED, INFRA, UNSUPPORTED } data class ProcState( - val cminmsgsId: Long, + val msgId: Long, val state: ProcStatus, - val identityKey: String? = null, // decode 后首次绑定；FAILED 重试不重绑（I3） + val identityKey: String? = null, // SNDR|TYPE|STYP|SEQN；decode 后首次绑定，FAILED 重试不重绑 val attempts: Int = 0, val nextAttemptAt: java.time.Instant? = null, val errorClass: ErrorClass? = null, @@ -20,6 +22,6 @@ data class ProcState( val isTerminal: Boolean get() = state == ProcStatus.SUCCEEDED || state == ProcStatus.SKIPPED || state == ProcStatus.DEAD - /** 终态皆可归档（矩阵 #12：SUCCEEDED ∪ SKIPPED ∪ DEAD）；PENDING/FAILED 不迁。 */ + /** 终态皆可归档；PENDING/FAILED 不迁。 */ val archivable: Boolean get() = isTerminal } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/SnapshotLog.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/SnapshotLog.kt new file mode 100644 index 0000000..45dc91f --- /dev/null +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/SnapshotLog.kt @@ -0,0 +1,26 @@ +package com.gzzn.omms.msgexchange.domain + +import java.time.Instant +import java.time.LocalDate + +/** + * SCHD 快照留痕模型（docs/flight-state.md §5.5）： + * RESULT 与 FLAGS 分列（可「成功且告警」）；一行 = 一次尝试，重放也记； + * 留痕不参与决策；写失败只记指标；保留 90 天，按 (SCOPE_END, RECV_AT) 清理。 + */ +enum class SnapshotResult { COMMITTED, REPLAY_SKIPPED, ROLLED_BACK } + +enum class SnapshotFlag { EMPTY, RECS_DROP, SEQN_REGRESSION, DAY_MISMATCH, SCHD_REVIVE_CONFLICT } + +data class SnapshotLogEntry( + val msgId: Long, + val recvAt: Instant, + val scopeStart: LocalDate, + val scopeEnd: LocalDate, + val recs: Int, + val upserted: Int, + val durationMs: Long, + val result: SnapshotResult, + val flags: Set = emptySet(), + val archiveKey: String? = null, // 证据层引用（尚未交付 §3.2） +) diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/Targets.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/Targets.kt new file mode 100644 index 0000000..b1fe80f --- /dev/null +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/Targets.kt @@ -0,0 +1,10 @@ +package com.gzzn.omms.msgexchange.domain + +/** + * MSG_EVENT 投递目标（§7.3：KAFKA_SCHD 发整态、KAFKA_MSG 只通知变化； + * 两主题间不保证顺序）。ES 投影属阶段 B，暂不登记目标。 + */ +object Targets { + const val KAFKA_MSG = "KAFKA:msg" + const val KAFKA_SCHD = "KAFKA:schd" +} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FieldCommand.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FieldCommand.kt deleted file mode 100644 index 5aaff93..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FieldCommand.kt +++ /dev/null @@ -1,23 +0,0 @@ -package com.gzzn.omms.msgexchange.domain.flight - -/** - * 显式字段命令（docs/flight-state.md §4）。 - */ -sealed interface ScalarCommand { - data object Unchanged : ScalarCommand - data class Set(val value: String) : ScalarCommand - data object Clear : ScalarCommand -} - -sealed interface CollectionCommand { - data object Unchanged : CollectionCommand - data class Replace(val items: List>) : CollectionCommand - data object Clear : CollectionCommand - data class Apply(val item: Map, val sourceSeq: String) : CollectionCommand -} - -data class FlightFieldCommands( - val flid: String, - val scalars: Map = emptyMap(), - val collections: Map = emptyMap(), -) diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightModel.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightModel.kt new file mode 100644 index 0000000..82a47a1 --- /dev/null +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightModel.kt @@ -0,0 +1,89 @@ +package com.gzzn.omms.msgexchange.domain.flight + +import java.time.Instant +import java.time.LocalDate + +/** + * 航班实例当前态模型（docs/flight-state.md §3）。 + * + * 身份：FLID 唯一关联键；OPERATION_DAY 一经确定不可变（§3.5/§5.3）； + * STATE 仅 ACTIVE/DELETED（§3.1，无 ARCHIVED——物理清除只发生在历史归档成功之后 §8.2）。 + */ +enum class FlightState { ACTIVE, DELETED } + +/** FLIGHT_SCHD 主行的身份与追踪字段（不含标量载荷）。 */ +data class FlightMainRow( + val flid: String, + val operationDay: LocalDate?, + val state: FlightState, + val stateVersion: Long, + val lastMsgId: Long?, + val updatedAt: Instant, +) + +/** + * SCHD DNLD/RESP 单条 FLTR 记录（解码产物，§5 入口）。 + * scalars/collections 只含报文中出现的字段——完整快照语义下 + * 出现 = Set/Replace，未出现 = 清除/替换空集（§5.4）。 + */ +data class ScheduleRecord( + val flid: String, + val scalars: Map, + val collections: Map>> = emptyMap(), + val seqn: Long = 0, +) + +/** + * 完整快照落库载荷（§5.1 步骤 6）：按记录整体替换映射内字段， + * 仓储层负责明细先删后插与 `OPERATION_DAY` 不可变条件更新（§7.4）。 + */ +data class SnapshotPatch( + val flid: String, + val operationDay: LocalDate, + val scalars: Map, + val collections: Map>>, +) + +/** 快照 upsert 结果（§5.1 步骤 6：DELETED 航班保持 DELETED 并告警，不恢复）。 */ +enum class UpsertOutcome { INSERTED, UPDATED, REVIVE_CONFLICT } + +/** + * FLOP/ADFT 增量载荷（§6.1/§2.1）。 + * scalars 出现 = Set，缺失 = 保留本地值；collections 出现 = Replace + * （FLOP 集合语义未定案前沿用 Replace，§10 当前偏差明示）。 + */ +data class MergeChange( + val flid: String, + val scalars: Map, + val collections: Map>> = emptyMap(), +) + +/** + * 完整当前态（§3.3）：主行 + 全部明细 = 完整当前态； + * 读取须在一致性读事务中，且展示层过滤 STATE = ACTIVE（§9）。 + */ +data class FlightSnapshot( + val flid: String, + val operationDay: LocalDate?, + val state: FlightState, + val stateVersion: Long, + val scalars: Map, + val collections: Map>>, +) + +/** §8.1 历史判定窗口（按机场时区计算；窗口值业务配置）。 */ +data class HistoryRules( + val cancelledHours: Long = 48, // CNCL 非空超过 N 小时 + val terminalHours: Long = 48, // NAAT/NEAT 终态超过 N 小时（字段含义待术语表确认 §10） + val deletedHours: Long = 48, // STATE = DELETED 超过 N 小时 + val idleHours: Long = 24 * 7, // 无终态字段：最后更新超过兜底期限 +) + +/** §8.1 命中历史判定的航班（归档 → 物理清除候选）。 */ +data class HistoryCandidate( + val flid: String, + val state: FlightState, + val stateVersion: Long, + /** 是否未经 FDEL 而被生命周期清除——清除前须补发一次删除事件（§7.3/§8.2）。 */ + val wasNeverFdel: Boolean, +) diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightNextState.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightNextState.kt deleted file mode 100644 index 43faa7c..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightNextState.kt +++ /dev/null @@ -1,24 +0,0 @@ -package com.gzzn.omms.msgexchange.domain.flight - -import com.fasterxml.jackson.databind.ObjectMapper -import com.gzzn.omms.msgexchange.infra.persistence.FlightFields - -data class FlightNextState( - val flid: String, - val scalars: Map, - val collections: Map>>, - val stateVersion: Long, - val lastMessageId: String, - /** 本次迁移中被显式清除的标量/异常/文本键（仓储据此写 NULL 列；快照全量替换下无意义）。 */ - val clearedKeys: Set = emptySet(), -) { - fun toFlightFields(mapper: ObjectMapper = ObjectMapper()): FlightFields { - val out = linkedMapOf() - out["FLID"] = flid - scalars.forEach { (k, v) -> out[k] = v } - collections.forEach { (key, items) -> - out[key] = mapper.writeValueAsString(items) - } - return out - } -} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightStateEngine.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightStateEngine.kt index e60c790..6ec0fa4 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightStateEngine.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightStateEngine.kt @@ -1,157 +1,134 @@ package com.gzzn.omms.msgexchange.domain.flight -import com.fasterxml.jackson.databind.JsonNode -import com.fasterxml.jackson.databind.ObjectMapper -import com.gzzn.omms.msgexchange.infra.persistence.FlightFields +import com.gzzn.omms.msgexchange.domain.OperationDayCalculator +import com.gzzn.omms.msgexchange.domain.SnapshotFlag +import java.time.LocalDate /** - * 从 legacy 字段视图解析/合并命令，并计算 nextState（v2 §4–§5）。 + * 航班状态引擎（docs/flight-state.md §3.3/§5.2/§5.4/§6.1）——纯函数， + * 内存生成完整新状态再落库；不触碰 DB/Kafka。 */ object FlightStateEngine { - private val mapper = ObjectMapper() - private val COLLECTION_KEYS = setOf( + /** 集合键白名单：10 类集合 ↔ 9 张明细表（ROUT/ERUT 共用 FLIGHT_ROUTE_POINT，§3.2）。 */ + val COLLECTION_KEYS: Set = setOf( "GTDT", "CKDT", "CLDT", "PSDT", "CHDT", "DELY", "ABTM", "CHOT", "ROUT", "ERUT", ) - /** 1:0..1 单值异常结构：主表前缀标量列承载；显式 null 载荷 = 清除（v2 §3.2/§9.1）。 */ - val EXCEPTION_KEYS = setOf("FDIV", "FRET", "FLAB") + /** + * §5.2 报文完整性五项校验：RECS 0–9999 且等于实收数；每条含合法数字型 FLID； + * 快照内不重复；每条记录运营日可计算且在报文覆盖范围内。任一失败整包不落地。 + */ + fun validateMessage( + recsDeclared: Int, + records: List, + scopeStart: LocalDate?, + scopeEnd: LocalDate?, + opDay: OperationDayCalculator, + ): SnapshotValidation { + val flags = linkedSetOf() + if (recsDeclared !in 0..9999) return SnapshotValidation.Invalid("RECS out of range: $recsDeclared", setOf(SnapshotFlag.RECS_DROP)) + if (records.size != recsDeclared) return SnapshotValidation.Invalid( + "RECS ($recsDeclared) != received FLTR count (${records.size})", + setOf(SnapshotFlag.RECS_DROP), + ) + if (records.isEmpty()) return SnapshotValidation.Ok(emptyMap(), setOf(SnapshotFlag.EMPTY)) - /** 异常/单值载荷的显式清除形态：空串、null 字面量、空对象（Oracle 空串即 NULL 语义的显式来源）。 */ - private fun isClearPayload(value: String): Boolean { - val trimmed = value.trim() - if (trimmed.isEmpty() || trimmed == "null") return true - val node = runCatching { mapper.readTree(trimmed) }.getOrNull() ?: return false - return node.isNull || (node.isObject && node.size() == 0) + val perRecordDay = linkedMapOf() + val seen = linkedSetOf() + for (record in records) { + if (!record.flid.matches(FLID_REGEX)) { + return SnapshotValidation.Invalid("illegal FLID: '${record.flid}'", emptySet()) + } + if (!seen.add(record.flid)) { + return SnapshotValidation.Invalid("duplicate FLID in snapshot: ${record.flid}", emptySet()) + } + val day = opDay.compute(record.scalars["SODT"]) + if (day == null) { + return SnapshotValidation.Invalid("operation day not computable, flid=${record.flid}", setOf(SnapshotFlag.DAY_MISMATCH)) + } + val inScope = (scopeStart == null || !day.isBefore(scopeStart)) && + (scopeEnd == null || !day.isAfter(scopeEnd)) + if (!inScope) { + return SnapshotValidation.Invalid( + "operation day $day outside coverage [$scopeStart, $scopeEnd], flid=${record.flid}", + setOf(SnapshotFlag.DAY_MISMATCH), + ) + } + perRecordDay[record.flid] = day + } + return SnapshotValidation.Ok(perRecordDay, flags) } - private val SEQ_ATTR = mapOf( - "GTDT" to "GTNO", - "CKDT" to "CKNO", - "CLDT" to "CLNO", - "PSDT" to "PSNO", - "CHDT" to "CHNO", - "ABTM" to "ASNO", - "CHOT" to "CSNO", - "ROUT" to "RTNO", - "ERUT" to "RTNO", - ) - // 注意：DELY 无协议序号属性（DLNO 非法）→ 不支持 Apply；清除走空数组 Replace(空集) - - /** DNLD/FLOP 字段集 → 命令（出现即 Set/Replace；未出现即 Unchanged；序号 0 条目 = 显式清除标记）。 */ - fun commandsFromFields(flid: String, fields: FlightFields): FlightFieldCommands { - val scalars = linkedMapOf() - val collections = linkedMapOf() - fields.forEach { (key, value) -> - if (key == "FLID") return@forEach - when { - key in COLLECTION_KEYS -> { - val seqAttr = SEQ_ATTR[key] - val parsed = parseCollection(value) - val markers = parsed.filter { seqAttr != null && it[seqAttr] == "0" } - collections[key] = when { - markers.isEmpty() -> CollectionCommand.Replace(parsed) - markers.size == parsed.size -> CollectionCommand.Clear - else -> throw IllegalArgumentException( - "$key mixes explicit clear marker (seq=0) with regular items; refusing to guess", - ) - } - } - key in EXCEPTION_KEYS && isClearPayload(value) -> scalars[key] = ScalarCommand.Clear - else -> scalars[key] = ScalarCommand.Set(value) + /** + * 完整快照语义（§5.4 DNLD/RESP）：标量出现 Set、缺失 Clear；集合出现 Replace、缺失 Replace 空集。 + * `keepDeleted = true` 时 STATE 保持 DELETED（§5.1 步骤 6：普通 SCHD 不恢复）。 + */ + fun snapshotState( + current: FlightSnapshot?, + record: ScheduleRecord, + operationDay: LocalDate, + keepDeleted: Boolean, + ): FlightSnapshot { + val state = when { + current == null -> FlightState.ACTIVE + keepDeleted -> FlightState.DELETED + else -> current.state + } + // §5.4 完整快照：标量出现 Set、缺失 Clear；集合出现 Replace、缺失 Replace 空集。 + // 完整替换 = 新状态只由记录决定，不从 current 继承任何字段。 + val scalars: Map = record.scalars + val collections = buildMap { + COLLECTION_KEYS.forEach { key -> put(key, emptyList()) } // 缺失 = Replace 空集（§5.4） + record.collections.forEach { (key, items) -> + if (key in COLLECTION_KEYS) put(key, items) } } - return FlightFieldCommands(flid, scalars, collections) - } - - fun apply( - current: FlightNextState?, - commands: FlightFieldCommands, - messageId: String, - bumpVersion: Boolean, - ): FlightNextState { - val baseScalars = current?.scalars?.toMutableMap() ?: mutableMapOf() - val baseCollections = current?.collections?.mapValues { it.value.toMutableList() } - ?.toMutableMap() ?: mutableMapOf() - val baseVersion = current?.stateVersion ?: 0L - val cleared = mutableSetOf() - - commands.scalars.forEach { (key, cmd) -> - when (cmd) { - ScalarCommand.Unchanged -> Unit - is ScalarCommand.Set -> { - baseScalars[key] = cmd.value - cleared.remove(key) - } - ScalarCommand.Clear -> { - baseScalars.remove(key) - cleared += key - } - } - } - - commands.collections.forEach { (key, cmd) -> - when (cmd) { - CollectionCommand.Unchanged -> Unit - is CollectionCommand.Replace -> baseCollections[key] = cmd.items.toMutableList() - CollectionCommand.Clear -> baseCollections.remove(key) - is CollectionCommand.Apply -> { - val seqAttr = SEQ_ATTR[key] - ?: throw IllegalArgumentException("$key does not support Apply: protocol defines no source sequence attribute") - val list = baseCollections.getOrPut(key) { mutableListOf() }.toMutableList() - val idx = list.indexOfFirst { it[seqAttr] == cmd.sourceSeq } - if (idx >= 0) { - list[idx] = cmd.item - } else { - list.add(cmd.item) - } - baseCollections[key] = list - } - } - } - - val nextVersion = if (bumpVersion) baseVersion + 1 else baseVersion - return FlightNextState( - flid = commands.flid, - scalars = baseScalars, - collections = baseCollections.mapValues { it.value.toList() }, - stateVersion = nextVersion, - lastMessageId = messageId, - clearedKeys = cleared.toSet(), + return FlightSnapshot( + flid = record.flid, + operationDay = operationDay, + state = state, + stateVersion = (current?.stateVersion ?: 0L) + 1, + scalars = scalars, + collections = collections, ) } - fun parseCollection(raw: String): List> { - val node = mapper.readTree(raw) - val items = when { - node.isNull -> emptyList() - node.isArray -> node.toList() - node.isObject -> listOf(node) - else -> throw IllegalArgumentException("collection value must be array or object") + /** + * FLOP 增量合并（§6.1）：标量出现覆盖、缺失保留；集合出现 Replace、缺失保留 + * ——FLOP 集合语义未定案，Replace 为当前偏差明示沿用（§10）。 + */ + fun mergedState(current: FlightSnapshot, change: MergeChange): FlightSnapshot { + val scalars = buildMap { + putAll(current.scalars) + putAll(change.scalars) } - return items.map { item -> - item.properties().associate { (k, v) -> - k to when { - v.isNull -> "" - v.isValueNode -> v.asText() - else -> v.toString() - } + val collections = buildMap { + putAll(current.collections) + change.collections.forEach { (key, items) -> + if (key in COLLECTION_KEYS) put(key, items) } } + return current.copy( + stateVersion = current.stateVersion + 1, + scalars = scalars, + collections = collections, + ) } - /** 当前库态 → FlightNextState（供增量合并）。 */ - fun fromFlightFields(flid: String, fields: FlightFields, stateVersion: Long = 0L, lastMessageId: String = ""): FlightNextState { - val scalars = linkedMapOf() - val collections = linkedMapOf>>() - fields.forEach { (key, value) -> - if (key == "FLID") return@forEach - if (key in COLLECTION_KEYS) { - collections[key] = parseCollection(value) - } else { - scalars[key] = value - } - } - return FlightNextState(flid, scalars, collections, stateVersion, lastMessageId) - } + /** FLID：数字型，SIS §3.16.2 Number(1-12)。 */ + private val FLID_REGEX = Regex("\\d{1,12}") +} + +/** §5.2 校验结果：Ok 携带每条记录的归属运营日与观测 flags；Invalid 整包拒绝。 */ +sealed interface SnapshotValidation { + data class Ok( + val perRecordDay: Map, + val flags: Set, + ) : SnapshotValidation + + data class Invalid( + val reason: String, + val flags: Set, + ) : SnapshotValidation } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/JacksonFactory.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/JacksonFactory.kt new file mode 100644 index 0000000..b73143d --- /dev/null +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/JacksonFactory.kt @@ -0,0 +1,13 @@ +package com.gzzn.omms.msgexchange.infra + +import com.fasterxml.jackson.databind.ObjectMapper +import com.fasterxml.jackson.module.kotlin.registerKotlinModule +import io.micronaut.context.annotation.Factory +import jakarta.inject.Singleton + +/** 统一 ObjectMapper 装配（处理器出站载荷 JSON 序列化共用）。 */ +@Factory +class JacksonFactory { + @Singleton + fun objectMapper(): ObjectMapper = ObjectMapper().registerKotlinModule() +} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/FlightFieldsJson.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/FlightFieldsJson.kt deleted file mode 100644 index f05acad..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/FlightFieldsJson.kt +++ /dev/null @@ -1,35 +0,0 @@ -package com.gzzn.omms.msgexchange.infra.persistence - -import com.fasterxml.jackson.databind.ObjectMapper - -/** - * 航班字段集（FLIGHT_SCHD 宽表字段）→ KAFKA_SCHD 事件载荷序列化。 - * 仅用于 MSG_EVENT 出站载荷（报文线格式，legacy 下游按 JSON 消费），与库内存储无关； - * 键按字典序输出保证同状态产出字节级稳定载荷（重放/幂等判据不因遍历序漂移）。 - */ -object FlightFieldsJson { - private val mapper = ObjectMapper() - - /** Fields whose legacy wire representation is a JSON array/object rather than a JSON string. */ - private val STRUCTURED_FIELDS = setOf( - "ROUT", "ERUT", "CHDT", "GTDT", "PSDT", "CKDT", "CLDT", "DELY", - "CHOT", "ABTM", "SRVT", "VIPF", "MAFL", "FDIV", "FRET", "FLAB", - ) - - fun toJson(fields: Map): String { - val node = mapper.createObjectNode() - fields.toSortedMap().forEach { (key, value) -> - if (key in STRUCTURED_FIELDS) { - val parsed = runCatching { mapper.readTree(value) }.getOrNull() - if (parsed != null && (parsed.isArray || parsed.isObject || parsed.isNull)) { - node.set(key, parsed) - } else { - node.put(key, value) - } - } else { - node.put(key, value) - } - } - return node.toString() - } -} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/Repositories.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/Repositories.kt index 340a4e3..14ba3e1 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/Repositories.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/Repositories.kt @@ -1,39 +1,55 @@ package com.gzzn.omms.msgexchange.infra.persistence import com.gzzn.omms.msgexchange.domain.ErrorClass -import com.gzzn.omms.msgexchange.domain.EventStatus import com.gzzn.omms.msgexchange.domain.MsgEvent import com.gzzn.omms.msgexchange.domain.ProcState import com.gzzn.omms.msgexchange.domain.ProcStatus -import com.gzzn.omms.msgexchange.domain.RefUpsert +import com.gzzn.omms.msgexchange.domain.SnapshotLogEntry +import com.gzzn.omms.msgexchange.domain.flight.FlightMainRow +import com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot +import com.gzzn.omms.msgexchange.domain.flight.HistoryCandidate +import com.gzzn.omms.msgexchange.domain.flight.HistoryRules import java.time.Instant +import java.time.LocalDate +import java.time.ZoneId -/** - * ACM2-28 仓储接口（接口驱动，主泵/调度循环可单测；Micronaut Data JDBC 实装属 U05 批次）。 - * 存储边界：自有 PostgreSQL（datasources.default）= 本文件除 CminmsgInboxRepository 外 - * 的全部接口（消息管道 PROC_STATE/MSG_EVENT、PUMP_JOB、REQ_TRACK、21 类 REF_MASTER）； - * 共享 MySQL 信箱（CMINMSGS / COUTMSGS）经信箱封装访问，仅 DML、不建表； - * 主路径=上游外部写 CMINMSGS → 本系统 JDBC 轮询读；compat=insertRaw HTTP 写； - * 自有库 = 消息管道 + 运营航班 FLIGHT_SCHD/SCHD_GEN + 静态数据；Redis 已退出阶段 A 权威与写路径。 - */ -/** 单航班字段集（field → value，与 legacy flightInfo hash 同构）。 */ -typealias FlightFields = Map +// ===================================================================== +// 仓储契约（docs/flight-state.md §3.2 表职责）。 +// 决策层写路径约定：所有 FLIGHT_SCHD 及明细写操作必须发生在 +// 「持有 PIPELINE_LOCK 的同一事务」内（§1：锁只串行化 DB 事务）。 +// ===================================================================== + +/** 自有 PG 单事务原子保障（§1 目标 3：状态、事件、处理终态同事务提交）。 */ +interface PipelineTransactionManager { + fun inTransaction(block: () -> T): T +} + +/** 单行锁（§3.2 PIPELINE_LOCK）：事务内第一步 SELECT ... FOR UPDATE，串行化状态写事务。 */ +interface PipelineLockRepository { + fun lock() +} + +/** PROC_STATE：每消息一行；兼作快照重放判定（§5.1 步骤 2：MSG_ID 已有成功终态 → 重放）。 */ interface ProcStateRepository { - fun insert(cminmsgsId: Long, state: ProcStatus = ProcStatus.PENDING) + fun insert(msgId: Long, state: ProcStatus = ProcStatus.PENDING) /** 主路径轮询/compat 入队前判重（PG 已有行则跳过）。 */ - fun exists(cminmsgsId: Long): Boolean + fun exists(msgId: Long): Boolean - /** I1：严格 FIFO 队头（最小未完成 CMINMSGS_ID）。 */ + fun find(msgId: Long): ProcState? + + fun findSuccessTerminal(msgId: Long): Boolean + + /** 严格 FIFO 队头（最小未完成 MSG_ID）。 */ fun headUnfinished(): ProcState? - /** I3：identity 首次绑定；返回 false = 另一条消息已持有该键。 */ - fun tryBindIdentity(cminmsgsId: Long, identityKey: String): Boolean + /** identity 首次绑定；返回 false = 另一条消息已持有该键。 */ + fun tryBindIdentity(msgId: Long, identityKey: String): Boolean fun ownerOfIdentity(identityKey: String): Long? fun update( - cminmsgsId: Long, + msgId: Long, state: ProcStatus, nextAttemptAt: Instant? = null, attempts: Int? = null, @@ -41,21 +57,20 @@ interface ProcStateRepository { lastError: String? = null, ) - /** - * U11 显式重放入口（ReplayService）：仅把给定 errorClass 集合中的行从 FAILED/DEAD 置回 PENDING， - * 以便主泵重新领取。实现约定：ATTEMPTS=0、NEXT_ATTEMPT_AT=NULL（立即重试）， - * ERROR_CLASS/LAST_ERROR 保留作审计。返回受影响行数。 - */ + /** 显式重放入口：仅把给定 errorClass 集合中的行从 FAILED/DEAD 置回 PENDING（ATTEMPTS=0）。 */ fun requeueByErrorClasses(errorClasses: List): Int } +/** MSG_EVENT outbox（§7.3）。KAFKA_SCHD 合并同 FLID 未发事件按最新 STATE_VERSION 输出。 */ interface MsgEventRepository { fun insertAll(events: List): List - /** I1 双层同策略：每 target 严格 FIFO 队头。 */ fun headUnsent(target: String): MsgEvent? - fun claimBatch(target: String, limit: Int): List // ORDER BY EVENT_ID ASC + fun claimBatch(target: String, limit: Int): List + + /** 同 FLID 未发 KAFKA_SCHD 事件合并：每 FLID 取最新 STATE_VERSION 一条（§7.3）。 */ + fun mergePendingSchd(limit: Int): List fun markSent(eventId: Long) @@ -64,108 +79,100 @@ interface MsgEventRepository { fun scheduleRetry(eventId: Long, nextAttemptAt: Instant, attempts: Int) fun markDead(eventId: Long, errorClass: ErrorClass, lastError: String, attempts: Int? = null) - - /** 阶段 B（定案 2）：Delivery 同线程在 ES 投递成功后同步 enqueue 删除事件。 */ - fun insertSync(events: List) } +/** 完整态落库结果：DAY_GUARD_VIOLATION = OPERATION_DAY 不可变条件更新未命中（§7.4）。 */ +enum class PersistOutcome { INSERTED, UPDATED, DAY_GUARD_VIOLATION } + /** - * 阶段 A 运营航班权威与日计划代（ACM2-28 定案 + ACM2-29 全宽表·零子表定案）： - * - 表 FLIGHT_SCHD：一行一航班的运营航班宽表（FLID 主键 + FDAY 可空所属代 + SCHD.FLTR 标量字段列 - * + 集合平铺标量列/紧凑 VARCHAR 字串列），零子表、零 CLOB，字段即列、天然可索引可直查， - * PG/Oracle 11g 方言一致；读侧视图由平铺列重建集合键，与 legacy flightInfo hash 同构； - * - 运营资源消息语义 = 单资源集合级全量快照替换；外层航班增量仍为字段级合并（仅新增/覆盖）； - * - 表 PIPELINE_LOCK：单写者行级互斥（SELECT ... FOR UPDATE，PG/11g 同构，无 DBMS_LOCK 特权依赖）； - * - 表 SCHD_GEN：各日代版本（SQL CAS）；表 SCHD_GEN_FLID：当前代有效航班 FLID 集合（差删依据）； - * - Redis 退出动态权威与全部写路径；事务 2 与快照发布全在自有库内单事务原子提交； - * - 增量更新（FLOP/ADFT）：新插 FDAY=NULL，已有行保留原 FDAY，按字段列更新 - * （与 legacy flightInfo hash「仅新增/覆盖、不删除缺失字段」同语义）； - * - 快照（DNLD）：声明/更新 FDAY 归属，航班字段整体替换； - * - 按代差删域化：DELETE FROM FLIGHT_SCHD WHERE FDAY = :day AND FLID IN (:diffSet) - * 仅删除仍属旧代的行，ADFT（FDAY=NULL）与已迁移至新代的同 FLID 行天然存活。 + * 航班当前态权威（§3.2 FLIGHT_SCHD + 9 张明细表）。 + * 唯一写路径 = persistFullState：写入前在内存生成完整新状态（引擎产物）再落库， + * 明细集合按组先删后插（§3.3）。 */ -interface FlightSchdRepository { - data class FlightRecord( - val flid: String, - val fday: String?, - val fields: FlightFields, - val createdAt: Instant, - val updatedAt: Instant, - ) +interface FlightStateRepository { + /** 主行点查（身份/版本判定；批量用于快照归属校验 §5.3）。 */ + fun findMainRow(flid: String): FlightMainRow? - data class GenMeta( - val fday: String, - val version: Long, - val flids: Set, - val lastMessageId: String? = null, - val updatedAt: Instant = Instant.now(), - ) + fun findMainRows(flids: Collection): Map - /** v2（P4 退场）：legacy upsertSnapshotBatch/upsertIncremental 已删除——唯一写路径是 persistNextStates。 */ - - /** 按代差删域化：仅删除 FDAY = day 且在 delFlids 中的记录（ADFT 与跨代已迁移行受保护）。 */ - fun deleteDiffByDay(day: String, delFlids: Collection): Int - - /** 点查单航班字段集；无字段行（含航班不存在）返回 null。 */ - fun findByFlid(flid: String): FlightFields? - - /** v2：点查单航班 nextState（含 state_version / last_message_id 追踪字段）。 */ - fun findNextStateByFlid(flid: String): com.gzzn.omms.msgexchange.domain.flight.FlightNextState? - - /** 点查多航班字段集：FLID → 字段集，仅含实际存在的航班。 */ - fun findByFlids(flids: Collection): Map - - /** 按计划日查询当前有效航班。 */ - fun findByDay(day: String): List> - - /** 全量查询（供影子对拍 / 一致性对账）。 */ - fun findAll(): Map + /** 完整当前态：主行 + 全部明细（一致性读边界由调用方事务保证，§9）。 */ + fun loadFullSnapshot(flid: String): FlightSnapshot? /** - * 历史清场删除：仅删除已确认归档至 ES 的 FLID 集合；空集合不执行；分批参数化删除。 - * 返回实际删除行数（允许重放时为 0）。 + * 完整当前态落库（§3.3）：主行 upsert + 全部明细按组先删后插； + * STATE_VERSION 以 snapshot.stateVersion 落库。 + * SCHD（整体替换/清除）、FLOP/ADFT（合并后全量写）共用此唯一写路径。 + * 条件更新带 `WHERE operation_day IS NULL OR operation_day = :day`（§7.4 不可变强化）。 */ - fun deleteByFlids(flids: Set): Int - - /** 读计划代（SCHD_GEN）。 */ - fun getGen(day: String): GenMeta? + fun persistFullState(snapshot: FlightSnapshot, msgId: Long, now: Instant): PersistOutcome /** - * 计划代 SQL 版本 CAS：仅当 expected 与数据库中当前版本一致（或不存在且 expected=0）时写入/推进新版本。 - * 返回 true 表示推进成功，false 表示发生 CAS 冲突。 + * FDEL（§6.2）：ACTIVE → 置 DELETED、推进版本、明细保留、返回 true（发布删除事件）； + * 已 DELETED 或不存在 → 返回 false（幂等成功，不推进版本不重复发布）。 */ - fun putGenIfVersion(day: String, expected: Long, newGen: GenMeta, now: Instant = Instant.now()): Boolean + fun markDeleted(flid: String, msgId: Long, now: Instant): Boolean + + /** ADFT 生命周期重激活（§6.3）：DELETED → ACTIVE，推进版本；非 DELETED 返回 false。 */ + fun revive(flid: String, msgId: Long, now: Instant): Boolean + + /** §8.1：按窗口规则选出历史候选（含 DELETED；候选时间按机场时区折算）。 */ + fun findHistoryCandidates(rules: HistoryRules, zone: ZoneId, now: Instant): List /** - * 历史代清理：清理 cutoffDay 之前的历史代记录（与 FLIGHT_SCHD 历史清场生命周期对齐）。 + * §8.2 步骤 3：物理删除主行与明细（仅历史存储确认成功后调用； + * 历史存储未接通时调用方必须传空集合——删 0 条）。 */ - fun deleteGenBefore(cutoffDay: String): Int + fun purgeArchived(flids: Collection): Int - /** - * v2 无损明细：将 nextState 写入宽表 + 明细表，并更新追踪字段。 - * 必须在 PipelineTransactionManager 事务内调用。 - */ - fun persistNextStates( - day: String?, - states: List, - snapshotReplace: Boolean, - now: Instant = Instant.now(), - ) + /** §8.3 观测：OPERATION_DAY 仍为 NULL 的航班数（只增不删，终止规则未定 §10）。 */ + fun countOperationDayNull(): Int } -/** 事务管理器抽象：自有 PG 单事务原子保障。 */ -interface PipelineTransactionManager { - fun inTransaction(block: () -> T): T +/** §5.5 SCHD_SNAP_LOG：事务外追加留痕，不参与决策；一行 = 一次尝试（重放也记）。 */ +interface SnapshotLogRepository { + fun append(entry: SnapshotLogEntry) +} + + +/** + * 请求状态机（§7.1）：只有 RESP 完成 RQFD 请求，按（运营日、发送方、请求类型） + * 匹配最新一条 PENDING；同类请求只留一条有效，新请求置旧为 EXPIRED。 + */ +interface ReqTrackRepository { + enum class ReqState { PENDING, SENT, DONE, EXPIRED } + + data class Req( + val reqId: Long, + val reqType: String, + val operationDay: LocalDate, + val sender: String, + val state: ReqState, + val coutmsgsId: Long? = null, + val sentAt: Instant? = null, + ) + + /** 登记新请求：同类（类型+运营日+发送方）旧有效请求先置 EXPIRED。 */ + fun insert(reqType: String, operationDay: LocalDate, sender: String): Long + + fun findLatest(reqType: String, operationDay: LocalDate, sender: String, states: List): Req? + + /** RESP 完成请求（§7.1）：匹配最新一条 PENDING/SENT；无匹配返回 false（迟到不报错）。 */ + fun completeLatest(reqType: String, operationDay: LocalDate, sender: String): Boolean + + fun linkCoutmsgs(reqId: Long, coutmsgsId: Long) + + fun markSent(reqId: Long, sentAt: Instant) + + fun expire(reqId: Long) } /** - * v2 §5（ACM2-29 P2-3）：提交后共享信箱回填的持久补偿待办。 - * 业务事务已提交（PROC_STATE=SUCCEEDED）后 backfillOnSuccess 失败 → 落本表重试； - * 回填失败不得把已成功的业务事务重新标记为失败，也不得重放业务变更。 + * 共享信箱回填补偿待办（§7.2）：业务事务内预登记（消除崩溃窗口 §10 偏差）， + * 提交后由 BackfillSweepJob 重试；回填失败不得把 SUCCEEDED 改回 FAILED。 */ interface BackfillTodoRepository { data class BackfillTask( - val cminmsgsId: Long, + val msgId: Long, val sndr: String, val type: String, val styp: String, @@ -176,104 +183,25 @@ interface BackfillTodoRepository { /** 失败即落库（幂等 upsert，同 ID 重复失败只刷新错误与重试时间）。 */ fun record(task: BackfillTask, lastError: String?, now: Instant = Instant.now()) - /** 到期待重试的补偿任务（next_attempt_at <= now，按到期时间升序）。 */ fun findDue(now: Instant = Instant.now(), limit: Int = 50): List - /** 重试失败：累加 attempts 并按退避推后 next_attempt_at。 */ - fun markFailed(cminmsgsId: Long, lastError: String?, nextAttemptAt: Instant, now: Instant = Instant.now()) + fun markFailed(msgId: Long, lastError: String?, nextAttemptAt: Instant, now: Instant = Instant.now()) - /** 重试成功：移除补偿待办。 */ - fun delete(cminmsgsId: Long) + fun delete(msgId: Long) - /** 当前待办数量（测试/对账用）。 */ fun count(): Int } -@Deprecated("Replaced by FlightSchdRepository in ACM2-28", ReplaceWith("FlightSchdRepository")) -interface RefDataRepository { - data class GenMeta(val flids: List, val version: Long) - fun getGen(day: String): GenMeta? - fun putGenIfVersion(day: String, expected: Long, new: GenMeta): Boolean -} /** - * 21 类静态主数据（航空公司/航线/机位/登机桥等）——自有 PostgreSQL `REF_MASTER` 表 - * （ACM2-12：与消息管道同自有库；SOURCE=ADMINAPI/AODB/PIPELINE，N19 对齐）。 - * 与主链弱事务耦合：写入者为 ReferenceService（21 类同步）与请求应答路径。 - */ -interface StaticRefRepository { - fun upsertAll(refs: List) - - fun findByType(type: String): List -} - -/** - * 15 类请求状态机——自有 PG `REQ_TRACK`（ACM2-12；Reference & Query 域）。 - * 与共享库 COUTMSGS（出站信箱）跨库：先 COUTMSGS 落库成功 → 再 markSent，补偿重扫，最终一致。 - */ -interface ReqTrackRepository { - data class Req( - val reqId: Long, - val reqType: String, - val state: String, // REGISTERED/SENT/WAITING/DONE/EXPIRED - val sentAt: Instant? = null, - ) - - fun findOpenByKind(kind: String): Req? - - fun forceExpireOpenOf(kind: String) - - fun insert(kind: String, paramsJson: String): Long - - fun linkCoutmsgs(reqId: Long, coutmsgsId: Long) - - fun markSent(reqId: Long, sentAt: Instant) - - fun expireIfWaiting(reqId: Long) - - fun markDone(reqId: Long) -} - -/** - * 泵作业调度记录——自有 PG `PUMP_JOB`（ACM2-12）。 - * 决策 1 修订：作业不插队，仅在消息队头空闲/退避窗口由主泵执行（跨库/异队列无全序）； - * 作业动作本身（归档写共享库 CMINMSGS_HST、清场删 ES+PG 等）仍在各自目标存储。 - */ -interface PumpJobRepository { - data class Job(val jobId: Long, val kind: String) // ARCHIVE/HISTORY_SWEEP/PROJECTION_REBUILD - - fun enqueue(kind: String) - - fun headQueued(): Job? - - fun markRunning(jobId: Long) - - fun markDone(jobId: Long) - - fun markFailed(jobId: Long, lastError: String) -} - -/** - * 共享信箱 CMINMSGS 访问（ACM2-12：库属他人系统，本系统不建表）。 - * - * **主路径（生产）**：`pollNew` / `rawOf` — JDBC 轮询/读取上游外部写入的报文（U05 InboxPoller）。 - * **compat 路径**：`insertRaw` — HTTP `POST /cminmsgs/send` 辅助写（手工/对拍，非主拓扑）。 - * - * 入队模型：发现或 compat 写得到 CMINMSGS_ID → 自有 PG 建 PROC_STATE(PENDING)； - * PG 建行失败以共享库 DATE_PROCESSED IS NULL 重扫补建。 - * `backfillOnSuccess` = 处理成功后的外部回填（DATE_PROCESSED/STATUS，最终一致）。 + * 共享 MySQL 信箱 CMINMSGS 访问（他人系统库，本系统不建表）。 + * 主路径 JDBC 轮询读 + 处理回填；compat HTTP 写；出站写 COUTMSGS 由出站适配层承担。 */ interface CminmsgInboxRepository { - /** compat HTTP 写路径：向共享信箱插入原文（U16 契约对拍）。 */ fun insertRaw(rawXml: String): Long - /** 主路径/处理：按 CMINMSGS_ID 读取上游已落信的原文 XML。 */ - fun rawOf(cminmsgsId: Long): String? + fun rawOf(msgId: Long): String? - /** - * 主路径 JDBC 轮询：共享库未处理报文 ID 列表（legacy `getNewMsgsAfterId` 同语义）。 - * @param afterId 下界（legacy 现役传 0）；仅返回 `DATE_PROCESSED IS NULL` 行。 - */ fun pollUnprocessed(afterId: Long, limit: Int): List - fun backfillOnSuccess(cminmsgsId: Long, sndr: String, type: String, styp: String, seqn: Long) + fun backfillOnSuccess(msgId: Long, sndr: String, type: String, styp: String, seqn: Long) } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlightDetailTables.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlightDetailTables.kt deleted file mode 100644 index cfc4ab4..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlightDetailTables.kt +++ /dev/null @@ -1,162 +0,0 @@ -package com.gzzn.omms.msgexchange.infra.persistence.jdbc - -import com.gzzn.omms.msgexchange.domain.flight.FlightNextState -import java.time.Instant -import javax.sql.DataSource - -/** - * v2 明细表读写（docs/flight-state.md §3.2）。 - * 主表保存标量，明细表保存重复集合；不存在旧槽位读写回退。 - */ -internal object FlightDetailTables { - private data class TableSpec( - val table: String, - val collectionKey: String, - val seqAttr: String, - val columns: List>, // json attr -> db column - ) - - private val SPECS = listOf( - TableSpec("flight_gate", "GTDT", "GTNO", listOf( - "GATE" to "gate", "PGOT" to "pgot", "PGCT" to "pgct", "GOTM" to "gotm", "GCTM" to "gctm", "GTYP" to "gtyp", - )), - TableSpec("flight_checkin", "CKDT", "CKNO", listOf( - "CHKC" to "chkc", "CCLS" to "ccls", "PCOT" to "pcot", "PCCT" to "pcct", "COTM" to "cotm", "CCTM" to "cctm", "CTYP" to "ctyp", - )), - TableSpec("flight_belt", "CLDT", "CLNO", listOf( - "BELT" to "belt", "BCLS" to "bcls", "PCOT" to "pcot", "PCCT" to "pcct", "FBAG" to "fbag", "LBAG" to "lbag", "BTYP" to "btyp", - )), - TableSpec("flight_stand_plan", "PSDT", "PSNO", listOf( - "PSST" to "psst", "STST" to "stst", "STET" to "stet", - )), - TableSpec("flight_chute", "CHDT", "CHNO", listOf( - "CHUT" to "chut", "CHCLS" to "chcls", "PCBT" to "pcbt", "PCET" to "pcet", "CBTM" to "cbtm", "CETM" to "cetm", "CHTYP" to "chtyp", - )), - TableSpec("flight_delay", "DELY", "DLNO", listOf( - "CODE" to "code", "STRT" to "strt", "DURA" to "dura", "REMC" to "remc", - )), - TableSpec("flight_bridge_op", "ABTM", "ASNO", listOf( - "ABDG" to "abdg", "ABOP" to "abop", "AOTM" to "aotm", - )), - TableSpec("flight_chock_op", "CHOT", "CSNO", listOf( - "CHID" to "chid", "CHST" to "chst", "CHTM" to "chtm", - )), - ) - - fun replaceAll(ds: DataSource, state: FlightNextState, now: Instant) { - val ts = now.toSqlTimestamp() - val conn = ds.obtainConnection() - try { - for (spec in SPECS) { - ds.update("DELETE FROM ${spec.table} WHERE flid = ?") { ps -> ps.setString(1, state.flid) } - val items = state.collections[spec.collectionKey] ?: continue - if (items.isEmpty()) continue - val colNames = listOf("flid", "ordinal", "source_seq", "record_version") + - spec.columns.map { it.second } + listOf("created_at", "updated_at") - val sql = "INSERT INTO ${spec.table} (${colNames.joinToString(", ")}) VALUES (${ - colNames.joinToString(", ") { "?" } - })" - conn.prepareStatement(sql).use { ps -> - items.forEachIndexed { index, item -> - var i = 1 - ps.setString(i++, state.flid) - ps.setInt(i++, index + 1) - ps.setString(i++, item[spec.seqAttr]) - ps.setLong(i++, state.stateVersion) - spec.columns.forEach { (attr, _) -> ps.setString(i++, item[attr]) } - ps.setTimestamp(i++, ts) - ps.setTimestamp(i++, ts) - ps.addBatch() - } - ps.executeBatch() - } - } - replaceRoutes(ds, state, now) - } finally { - conn.releaseIfNotInTransaction() - } - } - - private fun replaceRoutes(ds: DataSource, state: FlightNextState, now: Instant) { - val ts = now.toSqlTimestamp() - ds.update("DELETE FROM flight_route_point WHERE flid = ?") { ps -> ps.setString(1, state.flid) } - listOf("ROUT" to "ROUT", "ERUT" to "ERUT").forEach { (key, kind) -> - val items = state.collections[key] ?: return@forEach - if (items.isEmpty()) return@forEach - val conn = ds.obtainConnection() - try { - conn.prepareStatement( - "INSERT INTO flight_route_point (flid, ordinal, source_seq, record_version, route_kind, apcd, scat, scdt, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", - ).use { ps -> - items.forEachIndexed { index, item -> - ps.setString(1, state.flid) - ps.setInt(2, index + 1) - ps.setString(3, item["RTNO"]) - ps.setLong(4, state.stateVersion) - ps.setString(5, kind) - ps.setString(6, item["APCD"]) - ps.setString(7, item["SCAT"]) - ps.setString(8, item["SCDT"]) - ps.setTimestamp(9, ts) - ps.setTimestamp(10, ts) - ps.addBatch() - } - ps.executeBatch() - } - } finally { - conn.releaseIfNotInTransaction() - } - } - } - - fun deleteForFlids(ds: DataSource, flids: Collection) { - if (flids.isEmpty()) return - val tables = SPECS.map { it.table } + "flight_route_point" - for (table in tables) { - for (chunk in flids.chunked(200)) { - val placeholders = chunk.joinToString(",") { "?" } - ds.update("DELETE FROM $table WHERE flid IN ($placeholders)") { ps -> - chunk.forEachIndexed { i, flid -> ps.setString(i + 1, flid) } - } - } - } - } - - fun loadCollections(ds: DataSource, flid: String): Map { - val out = linkedMapOf() - val mapper = com.fasterxml.jackson.databind.ObjectMapper() - for (spec in SPECS) { - val rows = ds.query( - "SELECT * FROM ${spec.table} WHERE flid = ? ORDER BY ordinal ASC", - { ps -> ps.setString(1, flid) }, - ) { rs -> - // 只输出非空属性（缺失键语义），保证 write→read 线格式一致（评审 F5） - spec.columns.associate { (attr, col) -> attr to rs.getString(col) } - .filterValues { it != null } - .toMutableMap() - .apply { rs.getString("source_seq")?.let { put(spec.seqAttr, it) } } - } - if (rows.isNotEmpty()) { - out[spec.collectionKey] = mapper.writeValueAsString(rows) - } - } - listOf("ROUT" to "ROUT", "ERUT" to "ERUT").forEach { (key, kind) -> - val rows = ds.query( - "SELECT source_seq, apcd, scat, scdt FROM flight_route_point WHERE flid = ? AND route_kind = ? ORDER BY ordinal ASC", - { ps -> ps.setString(1, flid); ps.setString(2, kind) }, - ) { rs -> - buildMap { - rs.getString("source_seq")?.let { put("RTNO", it) } - rs.getString("apcd")?.let { put("APCD", it) } - rs.getString("scat")?.let { put("SCAT", it) } - rs.getString("scdt")?.let { put("SCDT", it) } - } - } - if (rows.isNotEmpty()) { - out[key] = mapper.writeValueAsString(rows) - } - } - return out - } - -} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlightSchdReadAssembler.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlightSchdReadAssembler.kt deleted file mode 100644 index ba7486d..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlightSchdReadAssembler.kt +++ /dev/null @@ -1,70 +0,0 @@ -package com.gzzn.omms.msgexchange.infra.persistence.jdbc - -import com.fasterxml.jackson.databind.ObjectMapper -import com.gzzn.omms.msgexchange.infra.persistence.FlightFields - -/** - * FLIGHT_SCHD 读侧视图组装（v2 权威读，P4 退场后唯一读路径）。 - * - * - 标量列直读 + 无界集合文本列（SRVT/VIPF/MAFL） - * - FDIV/FRET/FLAB 由主表前缀标量列重建对象 - * - 10 类重复集合由明细表重建（保序、保源序号），legacy 槽位/里程碑列已随 V1.4.0 退场 - * - * 过渡期三条读路径（assembleMerged/assembleLegacySlotsOnly）与 FS7 双读对拍 - * 已随回退窗口关闭一并移除（V1.4.0，ACM2-29 P4）。 - */ -object FlightSchdReadAssembler { - - /** SCHD.FLTR 标量列 + legacy 派生列（与 V1.1.0 迁移一致）。 */ - val SCALAR_COLUMNS: List = listOf( - "ALCD", "ALSC", "FLNO", "MVIN", "SODT", "FLTY", "FLIN", "ACFT", "RENO", - "TAOP", "TAFL", "TAID", "TRML", "MAXP", "CSOP", "CSFT", "MAID", - "ESTT", "ACTT", "STND", "PHAG", "CNCL", "REMC", "BOTM", "LACL", "FINT", - "APPT", "EGSR", "EGST", "FHAG", "MHAG", "VIPP", "VIPR", "LBNO", "LBWT", - "PAXC", "EXSC", "EXSR", "FTSS", "PEDT", "NEAT", "PADT", "NAAT", - "ABDG", "LPSDT", "ABN", - ) - - val EXCEPTION_COLUMNS: List = listOf( - "FDIV_DDES", "FDIV_DDIR", "FDIV_REMC", "FRET_REID", "FRET_RSN", - "FLAB_ARES", "FLAB_RSN", - ) - - val TEXT_COLUMNS: List = listOf("SRVT_TEXT", "VIPF_TEXT", "MAFL_TEXT") - - /** 库列名全集（读侧 SELECT 稳定顺序；与 V1.4.0 后表结构一致）。 */ - val ALL_COLUMNS: List = SCALAR_COLUMNS + EXCEPTION_COLUMNS + TEXT_COLUMNS - - private val mapper = ObjectMapper() - - /** v2 权威读：标量 + 异常 + 文本列 + 明细表集合。 */ - fun assembleDetailOnly( - flid: String, - row: Map, - detailCollections: Map, - ): FlightFields { - val fields = assembleScalarsAndExceptions(flid, row) - detailCollections.forEach { (key, value) -> fields[key] = value } - return fields - } - - private fun assembleScalarsAndExceptions(flid: String, row: Map): LinkedHashMap { - val fields = linkedMapOf() - fields["FLID"] = flid - SCALAR_COLUMNS.forEach { column -> row[column]?.let { fields[column] = it } } - TEXT_COLUMNS.forEach { column -> row[column]?.let { fields[column.removeSuffix("_TEXT")] = it } } - exceptionView("FDIV", listOf("DDES" to "FDIV_DDES", "DDIR" to "FDIV_DDIR", "REMC" to "FDIV_REMC"), row) - ?.let { fields["FDIV"] = it } - exceptionView("FRET", listOf("REID" to "FRET_REID", "RSN" to "FRET_RSN"), row) - ?.let { fields["FRET"] = it } - exceptionView("FLAB", listOf("ARES" to "FLAB_ARES", "RSN" to "FLAB_RSN"), row) - ?.let { fields["FLAB"] = it } - return fields - } - - private fun exceptionView(key: String, attrs: List>, row: Map): String? { - val node = mapper.createObjectNode() - attrs.forEach { (attr, column) -> row[column]?.let { node.put(attr, it) } } - return if (node.size() == 0) null else mapper.writeValueAsString(node) - } -} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/JdbcCminmsgInboxRepository.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/JdbcCminmsgInboxRepository.kt index 7fac118..940a6c4 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/JdbcCminmsgInboxRepository.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/JdbcCminmsgInboxRepository.kt @@ -27,10 +27,10 @@ class JdbcCminmsgInboxRepository( ps.setString(1, rawXml) } - override fun rawOf(cminmsgsId: Long): String? = + override fun rawOf(msgId: Long): String? = ds.queryOne( "SELECT CMINMSGS_CLOB_MSG FROM cminmsgs WHERE CMINMSGS_ID = ?", - { ps -> ps.setLong(1, cminmsgsId) }, + { ps -> ps.setLong(1, msgId) }, ) { rs -> rs.getString("CMINMSGS_CLOB_MSG") } override fun pollUnprocessed(afterId: Long, limit: Int): List = @@ -47,7 +47,7 @@ class JdbcCminmsgInboxRepository( }, ) { rs -> rs.getLong("CMINMSGS_ID") } - override fun backfillOnSuccess(cminmsgsId: Long, sndr: String, type: String, styp: String, seqn: Long) { + override fun backfillOnSuccess(msgId: Long, sndr: String, type: String, styp: String, seqn: Long) { ds.update( """ UPDATE cminmsgs @@ -57,7 +57,7 @@ class JdbcCminmsgInboxRepository( """.trimIndent(), ) { ps -> ps.setString(1, "PROCESSED") - ps.setLong(2, cminmsgsId) + ps.setLong(2, msgId) } } } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/JdbcPgRepositories.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/JdbcPgRepositories.kt index a818262..611ca93 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/JdbcPgRepositories.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/JdbcPgRepositories.kt @@ -1,96 +1,127 @@ package com.gzzn.omms.msgexchange.infra.persistence.jdbc -import com.fasterxml.jackson.databind.JsonNode -import com.fasterxml.jackson.databind.ObjectMapper -import com.fasterxml.jackson.databind.node.ObjectNode import com.gzzn.omms.msgexchange.domain.ErrorClass import com.gzzn.omms.msgexchange.domain.EventStatus +import com.gzzn.omms.msgexchange.domain.EventType import com.gzzn.omms.msgexchange.domain.MsgEvent import com.gzzn.omms.msgexchange.domain.ProcState import com.gzzn.omms.msgexchange.domain.ProcStatus -import com.gzzn.omms.msgexchange.domain.RefUpsert +import com.gzzn.omms.msgexchange.domain.flight.FlightMainRow +import com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot +import com.gzzn.omms.msgexchange.domain.flight.FlightState +import com.gzzn.omms.msgexchange.domain.flight.HistoryCandidate +import com.gzzn.omms.msgexchange.domain.flight.HistoryRules import com.gzzn.omms.msgexchange.infra.persistence.BackfillTodoRepository -import com.gzzn.omms.msgexchange.infra.persistence.FlightFields -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository +import com.gzzn.omms.msgexchange.infra.persistence.CminmsgInboxRepository +import com.gzzn.omms.msgexchange.infra.persistence.FlightStateRepository import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository +import com.gzzn.omms.msgexchange.infra.persistence.PersistOutcome +import com.gzzn.omms.msgexchange.infra.persistence.PipelineLockRepository import com.gzzn.omms.msgexchange.infra.persistence.PipelineTransactionManager import com.gzzn.omms.msgexchange.infra.persistence.ProcStateRepository -import com.gzzn.omms.msgexchange.infra.persistence.PumpJobRepository -import com.gzzn.omms.msgexchange.infra.persistence.RefDataRepository import com.gzzn.omms.msgexchange.infra.persistence.ReqTrackRepository -import com.gzzn.omms.msgexchange.infra.persistence.StaticRefRepository +import com.gzzn.omms.msgexchange.domain.SnapshotLogEntry +import com.gzzn.omms.msgexchange.infra.persistence.SnapshotLogRepository import io.micronaut.context.annotation.Requires import jakarta.inject.Singleton +import java.sql.ResultSet +import java.time.Duration import java.time.Instant +import java.time.LocalDate +import java.time.LocalDateTime +import java.time.ZoneId +import com.gzzn.omms.msgexchange.domain.OperationDayCalculator +import java.time.format.DateTimeFormatter +import java.util.Locale import javax.sql.DataSource +// ===================================================================== +// 自有 PostgreSQL 仓储实现（docs/flight-state.md §3.2 表职责；schema 见 +// db/migration/V1__flight_state_baseline.sql）。全部写路径约定在 +// withTransaction + PIPELINE_LOCK 内调用（§1 目标 3/§3.2）。 +// ===================================================================== + @Singleton -@Requires(property = "msgx.stubs", notEquals = "true") @Requires(property = "datasources.default.enabled", value = "true") +@Requires(missingProperty = "msgx.stubs") +class JdbcPipelineTransactionManager( + private val ds: DataSource, +) : PipelineTransactionManager { + override fun inTransaction(block: () -> T): T = ds.withTransaction(block) +} + +@Singleton +@Requires(property = "datasources.default.enabled", value = "true") +@Requires(missingProperty = "msgx.stubs") +class JdbcPipelineLockRepository( + private val ds: DataSource, +) : PipelineLockRepository { + /** 事务内第一步：单行 FOR UPDATE 串行化状态写事务（§3.2 PIPELINE_LOCK）。 */ + override fun lock() { + ds.queryOne("SELECT lock_id FROM pipeline_lock WHERE lock_id = 1 FOR UPDATE", {}) { 1 } ?: error("PIPELINE_LOCK row missing") + } +} + +@Singleton +@Requires(property = "datasources.default.enabled", value = "true") +@Requires(missingProperty = "msgx.stubs") class JdbcProcStateRepository( private val ds: DataSource, ) : ProcStateRepository { - - override fun insert(cminmsgsId: Long, state: ProcStatus) { - val now = Instant.now() + override fun insert(msgId: Long, state: ProcStatus) { ds.update( - """ - INSERT INTO proc_state (cminmsgs_id, state, attempts, updated_at) - VALUES (?, ?, 0, ?) - ON CONFLICT (cminmsgs_id) DO NOTHING - """.trimIndent(), - ) { ps -> - ps.setLong(1, cminmsgsId) - ps.setString(2, state.name) - ps.setTimestamp(3, now.toSqlTimestamp()) - } + "INSERT INTO proc_state (msg_id, state, updated_at) VALUES (?, ?, ?)", + { ps -> ps.setLong(1, msgId); ps.setString(2, state.name); ps.setTimestamp(3, Instant.now().toSqlTimestamp()) }, + ) } - override fun exists(cminmsgsId: Long): Boolean = + override fun exists(msgId: Long): Boolean = + ds.queryOne("SELECT 1 FROM proc_state WHERE msg_id = ?", { ps -> ps.setLong(1, msgId) }) { 1 } != null + + override fun find(msgId: Long): ProcState? = ds.queryOne( - "SELECT 1 FROM proc_state WHERE cminmsgs_id = ?", - { ps -> ps.setLong(1, cminmsgsId) }, + "SELECT msg_id, state, identity_key, attempts, next_attempt_at, error_class, last_error, updated_at FROM proc_state WHERE msg_id = ?", + { ps -> ps.setLong(1, msgId) }, + ::mapProcState, + ) + + override fun findSuccessTerminal(msgId: Long): Boolean = + ds.queryOne( + "SELECT 1 FROM proc_state WHERE msg_id = ? AND state = 'SUCCEEDED'", + { ps -> ps.setLong(1, msgId) }, ) { 1 } != null override fun headUnfinished(): ProcState? = ds.queryOne( - """ - SELECT cminmsgs_id, state, identity_key, attempts, next_attempt_at, error_class, last_error, updated_at - FROM proc_state - WHERE state IN ('PENDING', 'FAILED') - ORDER BY cminmsgs_id ASC - LIMIT 1 - """.trimIndent(), + "SELECT msg_id, state, identity_key, attempts, next_attempt_at, error_class, last_error, updated_at " + + "FROM proc_state WHERE state IN ('PENDING', 'FAILED') ORDER BY msg_id ASC LIMIT 1", {}, ::mapProcState, ) - override fun tryBindIdentity(cminmsgsId: Long, identityKey: String): Boolean { + override fun tryBindIdentity(msgId: Long, identityKey: String): Boolean { ownerOfIdentity(identityKey)?.let { owner -> - if (owner != cminmsgsId) return false - return true + return owner == msgId } val updated = ds.update( - """ - UPDATE proc_state SET identity_key = ?, updated_at = ? - WHERE cminmsgs_id = ? AND identity_key IS NULL - """.trimIndent(), - ) { ps -> - ps.setString(1, identityKey) - ps.setTimestamp(2, Instant.now().toSqlTimestamp()) - ps.setLong(3, cminmsgsId) - } + "UPDATE proc_state SET identity_key = ?, updated_at = ? WHERE msg_id = ? AND identity_key IS NULL", + { ps -> + ps.setString(1, identityKey) + ps.setTimestamp(2, Instant.now().toSqlTimestamp()) + ps.setLong(3, msgId) + }, + ) return updated == 1 } override fun ownerOfIdentity(identityKey: String): Long? = ds.queryOne( - "SELECT cminmsgs_id FROM proc_state WHERE identity_key = ?", + "SELECT msg_id FROM proc_state WHERE identity_key = ?", { ps -> ps.setString(1, identityKey) }, - ) { rs -> rs.getLong("cminmsgs_id") } + ) { rs -> rs.getLong("msg_id") } override fun update( - cminmsgsId: Long, + msgId: Long, state: ProcStatus, nextAttemptAt: Instant?, attempts: Int?, @@ -99,42 +130,36 @@ class JdbcProcStateRepository( ) { ds.update( """ - UPDATE proc_state SET - state = ?, - next_attempt_at = ?, - attempts = COALESCE(?, attempts), - error_class = COALESCE(?, error_class), - last_error = COALESCE(?, last_error), - updated_at = ? - WHERE cminmsgs_id = ? + UPDATE proc_state SET state = ?, next_attempt_at = ?, attempts = ?, error_class = ?, last_error = ?, updated_at = ? + WHERE msg_id = ? """.trimIndent(), - ) { ps -> - ps.setString(1, state.name) - ps.setTimestamp(2, nextAttemptAt?.toSqlTimestamp()) - if (attempts != null) ps.setInt(3, attempts) else ps.setNull(3, java.sql.Types.INTEGER) - ps.setString(4, errorClass?.name) - ps.setString(5, lastError) - ps.setTimestamp(6, Instant.now().toSqlTimestamp()) - ps.setLong(7, cminmsgsId) - } + { ps -> + ps.setString(1, state.name) + ps.setTimestamp(2, nextAttemptAt?.toSqlTimestamp()) + ps.setInt(3, attempts ?: 0) + ps.setString(4, errorClass?.name) + ps.setString(5, lastError) + ps.setTimestamp(6, Instant.now().toSqlTimestamp()) + ps.setLong(7, msgId) + }, + ) } override fun requeueByErrorClasses(errorClasses: List): Int { if (errorClasses.isEmpty()) return 0 val placeholders = errorClasses.joinToString(",") { "?" } return ds.update( - """ - UPDATE proc_state SET state = 'PENDING', attempts = 0, next_attempt_at = NULL, updated_at = ? - WHERE state IN ('FAILED', 'DEAD') AND error_class IN ($placeholders) - """.trimIndent(), - ) { ps -> - ps.setTimestamp(1, Instant.now().toSqlTimestamp()) - errorClasses.forEachIndexed { i, ec -> ps.setString(i + 2, ec.name) } - } + "UPDATE proc_state SET state = 'PENDING', attempts = 0, next_attempt_at = NULL, updated_at = ? " + + "WHERE state IN ('FAILED', 'DEAD') AND error_class IN ($placeholders)", + { ps -> + ps.setTimestamp(1, Instant.now().toSqlTimestamp()) + errorClasses.forEachIndexed { i, ec -> ps.setString(i + 2, ec.name) } + }, + ) } - private fun mapProcState(rs: java.sql.ResultSet) = ProcState( - cminmsgsId = rs.getLong("cminmsgs_id"), + private fun mapProcState(rs: ResultSet) = ProcState( + msgId = rs.getLong("msg_id"), state = ProcStatus.valueOf(rs.getString("state")), identityKey = rs.getString("identity_key"), attempts = rs.getInt("attempts"), @@ -146,100 +171,97 @@ class JdbcProcStateRepository( } @Singleton -@Requires(property = "msgx.stubs", notEquals = "true") @Requires(property = "datasources.default.enabled", value = "true") +@Requires(missingProperty = "msgx.stubs") class JdbcMsgEventRepository( private val ds: DataSource, ) : MsgEventRepository { - override fun insertAll(events: List): List = events.map { e -> ds.updateReturningLong( """ - INSERT INTO msg_event (target, partition_key, payload_json, state, attempts, next_attempt_at, created_at) - VALUES (?, ?, ?, ?, ?, ?, ?) + INSERT INTO msg_event (target, partition_key, event_type, state_version, payload_json, state, attempts, next_attempt_at, created_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) RETURNING event_id """.trimIndent(), - ) { ps -> - ps.setString(1, e.target) - ps.setString(2, e.partitionKey) - ps.setString(3, e.payloadJson) - ps.setString(4, e.state.name) - ps.setInt(5, e.attempts) - ps.setTimestamp(6, e.nextAttemptAt?.toSqlTimestamp()) - ps.setTimestamp(7, e.createdAt.toSqlTimestamp()) - } + { ps -> + ps.setString(1, e.target) + ps.setString(2, e.partitionKey) + ps.setString(3, e.eventType.name) + ps.setLong(4, e.stateVersion) + ps.setString(5, e.payloadJson) + ps.setString(6, e.state.name) + ps.setInt(7, e.attempts) + ps.setTimestamp(8, e.nextAttemptAt?.toSqlTimestamp()) + ps.setTimestamp(9, e.createdAt.toSqlTimestamp()) + }, + ) } override fun headUnsent(target: String): MsgEvent? = ds.queryOne( - """ - SELECT event_id, target, partition_key, payload_json, state, attempts, next_attempt_at, error_class, last_error, created_at - FROM msg_event - WHERE target = ? AND state = 'PENDING' - ORDER BY event_id ASC - LIMIT 1 - """.trimIndent(), + "SELECT * FROM msg_event WHERE target = ? AND state = 'PENDING' ORDER BY event_id ASC LIMIT 1", { ps -> ps.setString(1, target) }, ::mapEvent, ) override fun claimBatch(target: String, limit: Int): List = + ds.query( + "SELECT * FROM msg_event WHERE target = ? AND state = 'PENDING' ORDER BY event_id ASC LIMIT ?", + { ps -> ps.setString(1, target); ps.setInt(2, limit) }, + ::mapEvent, + ) + + /** §7.3：同 FLID 未发事件按最新 STATE_VERSION 合并；PG DISTINCT ON 方言（注释明示）。 */ + override fun mergePendingSchd(limit: Int): List = ds.query( """ - SELECT event_id, target, partition_key, payload_json, state, attempts, next_attempt_at, error_class, last_error, created_at - FROM msg_event - WHERE target = ? AND state = 'PENDING' - ORDER BY event_id ASC - LIMIT ? + SELECT * FROM ( + SELECT DISTINCT ON (partition_key) * + FROM msg_event WHERE target = 'KAFKA:schd' AND state = 'PENDING' + ORDER BY partition_key, state_version DESC, event_id DESC + ) latest ORDER BY event_id ASC LIMIT ? """.trimIndent(), - { ps -> - ps.setString(1, target) - ps.setInt(2, limit) - }, + { ps -> ps.setInt(1, limit) }, ::mapEvent, ) override fun markSent(eventId: Long) { - ds.update("UPDATE msg_event SET state = 'SENT' WHERE event_id = ?") { ps -> ps.setLong(1, eventId) } + ds.update("UPDATE msg_event SET state = 'SENT' WHERE event_id = ?", { ps -> ps.setLong(1, eventId) }) } override fun markAllSent(eventIds: List) { - eventIds.forEach(::markSent) + if (eventIds.isEmpty()) return + val placeholders = eventIds.joinToString(",") { "?" } + ds.update("UPDATE msg_event SET state = 'SENT' WHERE event_id IN ($placeholders)") { ps -> + eventIds.forEachIndexed { i, id -> ps.setLong(i + 1, id) } + } } override fun scheduleRetry(eventId: Long, nextAttemptAt: Instant, attempts: Int) { ds.update( - "UPDATE msg_event SET state = 'PENDING', next_attempt_at = ?, attempts = ? WHERE event_id = ?", - ) { ps -> - ps.setTimestamp(1, nextAttemptAt.toSqlTimestamp()) - ps.setInt(2, attempts) - ps.setLong(3, eventId) - } + "UPDATE msg_event SET next_attempt_at = ?, attempts = ? WHERE event_id = ?", + { ps -> ps.setTimestamp(1, nextAttemptAt.toSqlTimestamp()); ps.setInt(2, attempts); ps.setLong(3, eventId) }, + ) } override fun markDead(eventId: Long, errorClass: ErrorClass, lastError: String, attempts: Int?) { ds.update( - """ - UPDATE msg_event SET state = 'DEAD', error_class = ?, last_error = ?, - attempts = COALESCE(?, attempts) - WHERE event_id = ? - """.trimIndent(), - ) { ps -> - ps.setString(1, errorClass.name) - ps.setString(2, lastError) - if (attempts != null) ps.setInt(3, attempts) else ps.setNull(3, java.sql.Types.INTEGER) - ps.setLong(4, eventId) - } + "UPDATE msg_event SET state = 'DEAD', error_class = ?, last_error = ?, attempts = COALESCE(?, attempts) WHERE event_id = ?", + { ps -> + ps.setString(1, errorClass.name) + ps.setString(2, lastError.take(1000)) + attempts?.let { ps.setInt(3, it) } ?: ps.setNull(3, java.sql.Types.INTEGER) + ps.setLong(4, eventId) + }, + ) } - override fun insertSync(events: List) { - insertAll(events) - } - - private fun mapEvent(rs: java.sql.ResultSet) = MsgEvent( + private fun mapEvent(rs: ResultSet) = MsgEvent( eventId = rs.getLong("event_id"), target = rs.getString("target"), partitionKey = rs.getString("partition_key"), + eventType = EventType.valueOf(rs.getString("event_type")), + stateVersion = rs.getLong("state_version"), payloadJson = rs.getString("payload_json"), state = EventStatus.valueOf(rs.getString("state")), attempts = rs.getInt("attempts"), @@ -251,618 +273,464 @@ class JdbcMsgEventRepository( } @Singleton -@Requires(property = "msgx.stubs", notEquals = "true") @Requires(property = "datasources.default.enabled", value = "true") -class JdbcPumpJobRepository( +@Requires(missingProperty = "msgx.stubs") +class JdbcFlightStateRepository( private val ds: DataSource, -) : PumpJobRepository { +) : FlightStateRepository { - override fun enqueue(kind: String) { - val now = Instant.now() - ds.update( - "INSERT INTO pump_job (kind, state, created_at, updated_at) VALUES (?, 'QUEUED', ?, ?)", - ) { ps -> - ps.setString(1, kind) - ps.setTimestamp(2, now.toSqlTimestamp()) - ps.setTimestamp(3, now.toSqlTimestamp()) + private fun mapMainRow(rs: ResultSet) = FlightMainRow( + flid = rs.getString("flid"), + operationDay = rs.getDate("operation_day")?.toLocalDate(), + state = FlightState.valueOf(rs.getString("state")), + stateVersion = rs.getLong("state_version"), + lastMsgId = rs.getLong("last_msg_id").takeIf { !rs.wasNull() }, + updatedAt = rs.getInstant("updated_at") ?: Instant.now(), + ) + + override fun findMainRow(flid: String): FlightMainRow? = + ds.queryOne(SELECT_MAIN + " WHERE flid = ?", { ps -> ps.setString(1, flid) }, ::mapMainRow) + + override fun findMainRows(flids: Collection): Map { + if (flids.isEmpty()) return emptyMap() + val out = linkedMapOf() + flids.chunked(200).forEach { chunk -> + val placeholders = chunk.joinToString(",") { "?" } + ds.query( + SELECT_MAIN + " WHERE flid IN ($placeholders)", + { ps -> chunk.forEachIndexed { i, id -> ps.setString(i + 1, id) } }, + ::mapMainRow, + ).forEach { out[it.flid] = it } } + return out } - override fun headQueued(): PumpJobRepository.Job? = - ds.queryOne( - "SELECT job_id, kind FROM pump_job WHERE state = 'QUEUED' ORDER BY job_id ASC LIMIT 1", + override fun loadFullSnapshot(flid: String): FlightSnapshot? { + val main = findMainRow(flid) ?: return null + val scalars = ds.queryOne( + "SELECT ${SCALAR_COLUMNS.joinToString(", ")} FROM flight_schd WHERE flid = ?", + { ps -> ps.setString(1, flid) }, + ) { rs -> SCALAR_COLUMNS.associate { it.uppercase() to (rs.getString(it) ?: "") } } ?: emptyMap() + + val collections = linkedMapOf>>() + COLLECTIONS.forEach { (key, spec) -> + collections[key] = loadDetails(flid, key, spec) + } + return FlightSnapshot( + flid = main.flid, + operationDay = main.operationDay, + state = main.state, + stateVersion = main.stateVersion, + scalars = scalars.filterValues { it.isNotEmpty() }, + collections = collections, + ) + } + + override fun persistFullState(snapshot: FlightSnapshot, msgId: Long, now: Instant): PersistOutcome { + val existed = findMainRow(snapshot.flid) + val updated = ds.update( + """ + INSERT INTO flight_schd ( + flid, operation_day, state, state_version, last_msg_id, + ${SCALAR_COLUMNS.joinToString(", ")}, created_at, updated_at + ) VALUES (?, ?, ?, ?, ?, ${SCALAR_COLUMNS.joinToString(", ") { "?" }}, ?, ?) + ON CONFLICT (flid) DO UPDATE SET + operation_day = EXCLUDED.operation_day, + state = EXCLUDED.state, + state_version = EXCLUDED.state_version, + last_msg_id = EXCLUDED.last_msg_id, + ${SCALAR_COLUMNS.joinToString(", ") { "$it = EXCLUDED.$it" }}, + updated_at = EXCLUDED.updated_at + WHERE flight_schd.operation_day IS NULL OR flight_schd.operation_day = EXCLUDED.operation_day + """.trimIndent(), + { ps -> + bindMain(ps, snapshot, msgId, now) + }, + ) + if (updated == 0) { + // §7.4 不可变强化：条件更新未命中 = 归属日冲突 + return if (existed != null) PersistOutcome.DAY_GUARD_VIOLATION else PersistOutcome.UPDATED + } + replaceDetails(snapshot, now) + return if (existed == null) PersistOutcome.INSERTED else PersistOutcome.UPDATED + } + + override fun markDeleted(flid: String, msgId: Long, now: Instant): Boolean = + ds.update( + "UPDATE flight_schd SET state = 'DELETED', state_version = state_version + 1, last_msg_id = ?, updated_at = ? " + + "WHERE flid = ? AND state = 'ACTIVE'", + { ps -> ps.setLong(1, msgId); ps.setTimestamp(2, now.toSqlTimestamp()); ps.setString(3, flid) }, + ) == 1 + + override fun revive(flid: String, msgId: Long, now: Instant): Boolean = + ds.update( + "UPDATE flight_schd SET state = 'ACTIVE', state_version = state_version + 1, last_msg_id = ?, updated_at = ? " + + "WHERE flid = ? AND state = 'DELETED'", + { ps -> ps.setLong(1, msgId); ps.setTimestamp(2, now.toSqlTimestamp()); ps.setString(3, flid) }, + ) == 1 + + override fun findHistoryCandidates(rules: HistoryRules, zone: ZoneId, now: Instant): List { + // §8.1 四条判定在应用层执行（SIS 时间串解析无法下推 SQL）； + // 不按 updated_at 粗筛——取消/终态时间可能早于最近一次更新，粗筛会漏删。 + val rows = ds.query( + "SELECT flid, state, state_version, operation_day, cncl, naat, neat, updated_at FROM flight_schd", {}, - ) { rs -> PumpJobRepository.Job(rs.getLong("job_id"), rs.getString("kind")) } - - override fun markRunning(jobId: Long) = markState(jobId, "RUNNING", null) - - override fun markDone(jobId: Long) = markState(jobId, "DONE", null) - - override fun markFailed(jobId: Long, lastError: String) = markState(jobId, "FAILED", lastError) - - private fun markState(jobId: Long, state: String, lastError: String?) { - ds.update( - "UPDATE pump_job SET state = ?, updated_at = ?, last_error = ? WHERE job_id = ?", - ) { ps -> - ps.setString(1, state) - ps.setTimestamp(2, Instant.now().toSqlTimestamp()) - ps.setString(3, lastError) - ps.setLong(4, jobId) + ) { rs -> + object { + val flid = rs.getString("flid") + val state = FlightState.valueOf(rs.getString("state")) + val version = rs.getLong("state_version") + val cncl = rs.getString("cncl") + val naat = rs.getString("naat") + val neat = rs.getString("neat") + val updatedAt = rs.getInstant("updated_at") + } + } + return rows.mapNotNull { r -> + val cancelledAt = parseSisTime(r.cncl, zone) + val terminalAt = listOfNotNull(parseSisTime(r.naat, zone), parseSisTime(r.neat, zone)).maxOrNull() + val hasTerminalField = listOf(r.cncl, r.naat, r.neat).any { !it.isNullOrBlank() } + val hit = when { + cancelledAt != null && cancelledAt < now.minus(Duration.ofHours(rules.cancelledHours)) -> true + terminalAt != null && terminalAt < now.minus(Duration.ofHours(rules.terminalHours)) -> true + r.state == FlightState.DELETED && r.updatedAt != null && + r.updatedAt < now.minus(Duration.ofHours(rules.deletedHours)) -> true + !hasTerminalField && r.updatedAt != null && + r.updatedAt < now.minus(Duration.ofHours(rules.idleHours)) -> true + else -> false + } + // NAAT/NEAT 业务含义待术语表确认（§10）：解析失败视为不命中，不误删 + if (hit) HistoryCandidate(r.flid, r.state, r.version, wasNeverFdel = r.state == FlightState.ACTIVE) else null } } -} -/** 自有 PostgreSQL 单事务管理器（ACM2-28 事务 2 与快照发布原子提交）。 */ -@Singleton -@Requires(property = "msgx.stubs", notEquals = "true") -@Requires(property = "datasources.default.enabled", value = "true") -class JdbcPipelineTransactionManager( - private val ds: DataSource, -) : PipelineTransactionManager { - override fun inTransaction(block: () -> T): T = ds.withTransaction { - // Standard row lock works on PostgreSQL and Oracle 11g without DBMS_LOCK privileges. - // The lock is connection-scoped and is released automatically on commit, rollback, or crash. - ds.queryOne( - "SELECT lock_key FROM pipeline_lock WHERE lock_key = ? FOR UPDATE", - { ps -> ps.setString(1, "FLIGHT_SCHD_WRITER") }, - ) { rs -> rs.getString("lock_key") } - ?: error("missing FLIGHT_SCHD_WRITER row in PIPELINE_LOCK") - block() + override fun purgeArchived(flids: Collection): Int { + if (flids.isEmpty()) return 0 + var purged = 0 + flids.chunked(500).forEach { chunk -> + val placeholders = chunk.joinToString(",") { "?" } + DETAIL_TABLES.forEach { table -> + purged += ds.update( + "DELETE FROM $table WHERE flid IN ($placeholders)", + { ps -> chunk.forEachIndexed { i, id -> ps.setString(i + 1, id) } }, + ) + } + purged += ds.update( + "DELETE FROM flight_schd WHERE flid IN ($placeholders)", + { ps -> chunk.forEachIndexed { i, id -> ps.setString(i + 1, id) } }, + ) + } + return purged + } + + override fun countOperationDayNull(): Int = + ds.queryOne("SELECT count(*) AS n FROM flight_schd WHERE operation_day IS NULL", {}) { rs -> rs.getInt("n") } ?: 0 + + // ---- 内部实现 ---- + + private fun bindMain( + ps: java.sql.PreparedStatement, + snapshot: FlightSnapshot, + msgId: Long, + now: Instant, + ) { + var i = 1 + ps.setString(i++, snapshot.flid) + snapshot.operationDay?.let { ps.setDate(i++, java.sql.Date.valueOf(it)) } ?: ps.setNull(i++, java.sql.Types.DATE) + ps.setString(i++, snapshot.state.name) + ps.setLong(i++, snapshot.stateVersion) + ps.setLong(i++, msgId) + SCALAR_COLUMNS.forEach { col -> + val v = snapshot.scalars[col.uppercase()] + if (v.isNullOrBlank()) ps.setNull(i++, java.sql.Types.VARCHAR) else ps.setString(i++, v) + } + ps.setTimestamp(i++, now.toSqlTimestamp()) + ps.setTimestamp(i++, now.toSqlTimestamp()) + } + + private fun replaceDetails(snapshot: FlightSnapshot, now: Instant) { + val flid = snapshot.flid + COLLECTIONS.forEach { (key, spec) -> + ds.update("DELETE FROM ${spec.table} WHERE flid = ?", { ps -> ps.setString(1, flid) }) + val items = snapshot.collections[key].orEmpty() + items.forEachIndexed { ordinal, item -> + val cols = mutableListOf("flid", "ordinal", "source_seq") + val vals = mutableListOf(flid) + vals.add(ordinal + 1) // ORDINAL 保留输入顺序（§3.2） + vals.add(item[spec.seqAttr]) + spec.columns.forEach { col -> + cols.add(col) + vals.add(item[col.uppercase()]) + } + cols.add("created_at"); vals.add(now) + cols.add("updated_at"); vals.add(now) + val placeholders = cols.indices.joinToString(",") { "?" } + val extra = if (spec.routeKind != null) ", route_kind" else "" + val extraVals = if (spec.routeKind != null) { + vals.add(spec.routeKind); ", ?" + } else "" + ds.update( + "INSERT INTO ${spec.table} (${cols.joinToString(", ")}$extra) VALUES (${placeholders}$extraVals)", + { ps -> + vals.forEachIndexed { idx, v -> + when (v) { + null -> ps.setNull(idx + 1, java.sql.Types.VARCHAR) + is Int -> ps.setInt(idx + 1, v) + is Instant -> ps.setTimestamp(idx + 1, v.toSqlTimestamp()) + else -> ps.setString(idx + 1, v.toString()) + } + } + }, + ) + } + } + } + + private fun loadDetails(flid: String, key: String, spec: DetailSpec): List> { + val sql = if (spec.routeKind != null) { + // ROUT/ERUT 共表：按 ROUTE_KIND 过滤（§3.2） + "SELECT * FROM ${spec.table} WHERE flid = ? AND route_kind = ? ORDER BY ordinal ASC" + } else { + "SELECT * FROM ${spec.table} WHERE flid = ? ORDER BY ordinal ASC" + } + return ds.query(sql, { ps -> ps.setString(1, flid); spec.routeKind?.let { ps.setString(2, it) } }, ::detailItem) + } + + private fun detailItem(rs: ResultSet): Map { + val item = linkedMapOf() + val md = rs.metaData + for (i in 1..md.columnCount) { + val col = md.getColumnName(i).uppercase() + if (col in IGNORED_DETAIL_COLUMNS) continue + rs.getString(i)?.let { if (it.isNotEmpty()) item[col] = it } + } + return item + } + + + /** SIS 时间串解析（ddMMMyyHHmm，机场时区）——历史判定用；口径同 OperationDayCalculator。 */ + internal fun parseSisTime(raw: String?, zone: ZoneId): Instant? { + if (raw.isNullOrBlank()) return null + return OperationDayCalculator.parseSodt(raw.trim())?.atZone(zone)?.toInstant() + } + + internal data class DetailSpec( + val table: String, + val columns: List, + val seqAttr: String?, + val routeKind: String? = null, + ) + internal companion object { + private val log = org.slf4j.LoggerFactory.getLogger(JdbcFlightStateRepository::class.java) + + /** 明细读取时排除的追踪列（业务键以报文线格式大写键回传）。 */ + private val IGNORED_DETAIL_COLUMNS = + setOf("CREATED_AT", "UPDATED_AT", "FLID", "ORDINAL", "ROUTE_KIND") + + private const val SELECT_MAIN = + "SELECT flid, operation_day, state, state_version, last_msg_id, updated_at FROM flight_schd" + + /** FLIGHT_SCHD 标量列（与基线一致；不含身份/版本/追踪列）。 */ + internal val SCALAR_COLUMNS = listOf( + "alcd", "alsc", "flno", "mvin", "sodt", "flty", "flin", "acft", "reno", + "taop", "tafl", "taid", "trml", "maxp", "csop", "csft", "maid", + "estt", "actt", "stnd", "phag", "cncl", "remc", "botm", "lacl", "fint", + "appt", "egsr", "egst", "fhag", "mhag", "vipp", "vipr", "lbno", "lbwt", + "paxc", "exsc", "exsr", "ftss", "pedt", "naat", "neat", "padt", + ) + + internal val DETAIL_TABLES = listOf( + "flight_gate", "flight_checkin", "flight_belt", "flight_stand_plan", "flight_chute", + "flight_delay", "flight_bridge_op", "flight_chock_op", "flight_route_point", + ) + + /** 10 类集合 ↔ 明细表/列映射（§3.2；列名与基线一致）。 */ + internal val COLLECTIONS: Map = mapOf( + "GTDT" to DetailSpec("flight_gate", listOf("gate", "pgot", "pgct", "gotm", "gctm", "gtyp"), "GTNO"), + "CKDT" to DetailSpec("flight_checkin", listOf("chkc", "ccls", "pcot", "pcct", "cotm", "cctm", "ctyp"), "CKNO"), + "CLDT" to DetailSpec("flight_belt", listOf("belt", "bcls", "pcot", "pcct", "fbag", "lbag", "btyp"), "CLNO"), + "PSDT" to DetailSpec("flight_stand_plan", listOf("psst", "stst", "stet"), "PSNO"), + "CHDT" to DetailSpec("flight_chute", listOf("chut", "chcls", "pcbt", "pcet", "cbtm", "cetm", "chtyp"), "CHNO"), + "DELY" to DetailSpec("flight_delay", listOf("code", "strt", "dura", "remc"), null), + "ABTM" to DetailSpec("flight_bridge_op", listOf("abdg", "abop", "aotm"), "ASNO"), + "CHOT" to DetailSpec("flight_chock_op", listOf("chid", "chst", "chtm"), "CSNO"), + "ROUT" to DetailSpec("flight_route_point", listOf("apcd", "scat", "scdt"), "RTNO", routeKind = "ROUT"), + "ERUT" to DetailSpec("flight_route_point", listOf("apcd", "scat", "scdt"), "RTNO", routeKind = "ERUT"), + ) } } -/** v2 §5（ACM2-29 P2-3）：提交后共享信箱回填补偿待办的 PG 实现。 */ @Singleton -@Requires(property = "msgx.stubs", notEquals = "true") @Requires(property = "datasources.default.enabled", value = "true") +@Requires(missingProperty = "msgx.stubs") +class JdbcSnapshotLogRepository( + private val ds: DataSource, +) : SnapshotLogRepository { + /** §5.5 只追加；写失败由调用方捕获记指标（append 自身不抛出）。 */ + override fun append(entry: SnapshotLogEntry) { + ds.update( + """ + INSERT INTO schd_snap_log (msg_id, recv_at, scope_start, scope_end, recs, upserted, duration_ms, result, flags, archive_key, created_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + """.trimIndent(), + { ps -> + ps.setLong(1, entry.msgId) + ps.setTimestamp(2, entry.recvAt.toSqlTimestamp()) + ps.setDate(3, java.sql.Date.valueOf(entry.scopeStart)) + ps.setDate(4, java.sql.Date.valueOf(entry.scopeEnd)) + ps.setInt(5, entry.recs) + ps.setInt(6, entry.upserted) + ps.setLong(7, entry.durationMs) + ps.setString(8, entry.result.name) + ps.setString(9, entry.flags.joinToString(",") { it.name }) + ps.setString(10, entry.archiveKey) + ps.setTimestamp(11, Instant.now().toSqlTimestamp()) + }, + ) + } +} + +@Singleton +@Requires(property = "datasources.default.enabled", value = "true") +@Requires(missingProperty = "msgx.stubs") +class JdbcReqTrackRepository( + private val ds: DataSource, +) : ReqTrackRepository { + override fun insert(reqType: String, operationDay: LocalDate, sender: String): Long = + ds.updateReturningLong( + "INSERT INTO req_track (req_type, operation_day, sender, state, created_at) VALUES (?, ?, ?, 'PENDING', ?) RETURNING req_id", + { ps -> + ps.setString(1, reqType) + ps.setDate(2, java.sql.Date.valueOf(operationDay)) + ps.setString(3, sender) + ps.setTimestamp(4, Instant.now().toSqlTimestamp()) + }, + ) + + override fun findLatest( + reqType: String, + operationDay: LocalDate, + sender: String, + states: List, + ): ReqTrackRepository.Req? { + if (states.isEmpty()) return null + val placeholders = states.joinToString(",") { "?" } + return ds.queryOne( + "SELECT * FROM req_track WHERE req_type = ? AND operation_day = ? AND sender = ? AND state IN ($placeholders) " + + "ORDER BY created_at DESC, req_id DESC LIMIT 1", + { ps -> + ps.setString(1, reqType) + ps.setDate(2, java.sql.Date.valueOf(operationDay)) + ps.setString(3, sender) + states.forEachIndexed { i, s -> ps.setString(i + 4, s.name) } + }, + ::mapReq, + ) + } + + /** §7.1 RESP 完成请求：匹配最新一条 PENDING/SENT；无匹配返回 false（迟到不报错）。 */ + override fun completeLatest(reqType: String, operationDay: LocalDate, sender: String): Boolean = + ds.update( + """ + UPDATE req_track SET state = 'DONE', completed_at = ? + WHERE req_id = ( + SELECT req_id FROM req_track + WHERE req_type = ? AND operation_day = ? AND sender = ? AND state IN ('PENDING', 'SENT') + ORDER BY created_at DESC, req_id DESC LIMIT 1 + ) + """.trimIndent(), + { ps -> + ps.setTimestamp(1, Instant.now().toSqlTimestamp()) + ps.setString(2, reqType) + ps.setDate(3, java.sql.Date.valueOf(operationDay)) + ps.setString(4, sender) + }, + ) == 1 + + override fun linkCoutmsgs(reqId: Long, coutmsgsId: Long) { + ds.update("UPDATE req_track SET coutmsgs_id = ? WHERE req_id = ?", { ps -> ps.setLong(1, coutmsgsId); ps.setLong(2, reqId) }) + } + + override fun markSent(reqId: Long, sentAt: Instant) { + ds.update( + "UPDATE req_track SET state = 'SENT', sent_at = ? WHERE req_id = ?", + { ps -> ps.setTimestamp(1, sentAt.toSqlTimestamp()); ps.setLong(2, reqId) }, + ) + } + + override fun expire(reqId: Long) { + ds.update("UPDATE req_track SET state = 'EXPIRED' WHERE req_id = ?", { ps -> ps.setLong(1, reqId) }) + } + + private fun mapReq(rs: ResultSet) = ReqTrackRepository.Req( + reqId = rs.getLong("req_id"), + reqType = rs.getString("req_type"), + operationDay = rs.getDate("operation_day").toLocalDate(), + sender = rs.getString("sender"), + state = ReqTrackRepository.ReqState.valueOf(rs.getString("state")), + coutmsgsId = rs.getLong("coutmsgs_id").takeIf { !rs.wasNull() }, + sentAt = rs.getInstant("sent_at"), + ) +} + +@Singleton +@Requires(property = "datasources.default.enabled", value = "true") +@Requires(missingProperty = "msgx.stubs") class JdbcBackfillTodoRepository( private val ds: DataSource, ) : BackfillTodoRepository { - override fun record(task: BackfillTodoRepository.BackfillTask, lastError: String?, now: Instant) { - val ts = now.toSqlTimestamp() ds.update( """ - INSERT INTO backfill_todo (cminmsgs_id, sndr, type, styp, seqn, attempts, next_attempt_at, last_error, created_at, updated_at) + INSERT INTO backfill_todo (msg_id, sndr, type, styp, seqn, attempts, next_attempt_at, last_error, created_at, updated_at) VALUES (?, ?, ?, ?, ?, 0, ?, ?, ?, ?) - ON CONFLICT (cminmsgs_id) DO UPDATE SET - last_error = EXCLUDED.last_error, - next_attempt_at = EXCLUDED.next_attempt_at, - updated_at = EXCLUDED.updated_at + ON CONFLICT (msg_id) DO UPDATE SET last_error = EXCLUDED.last_error, next_attempt_at = EXCLUDED.next_attempt_at, updated_at = EXCLUDED.updated_at """.trimIndent(), - ) { ps -> - ps.setLong(1, task.cminmsgsId) - ps.setString(2, task.sndr) - ps.setString(3, task.type) - ps.setString(4, task.styp) - ps.setLong(5, task.seqn) - ps.setTimestamp(6, ts) - ps.setString(7, lastError?.take(512)) - ps.setTimestamp(8, ts) - ps.setTimestamp(9, ts) - } + { ps -> + ps.setLong(1, task.msgId) + ps.setString(2, task.sndr) + ps.setString(3, task.type) + ps.setString(4, task.styp) + ps.setLong(5, task.seqn) + ps.setTimestamp(6, now.toSqlTimestamp()) + ps.setString(7, lastError?.take(512)) + ps.setTimestamp(8, now.toSqlTimestamp()) + ps.setTimestamp(9, now.toSqlTimestamp()) + }, + ) } override fun findDue(now: Instant, limit: Int): List = ds.query( - """ - SELECT cminmsgs_id, sndr, type, styp, seqn, attempts FROM backfill_todo - WHERE next_attempt_at <= ? ORDER BY next_attempt_at ASC LIMIT ? - """.trimIndent(), - { ps -> - ps.setTimestamp(1, now.toSqlTimestamp()) - ps.setInt(2, limit) - }, + "SELECT msg_id, sndr, type, styp, seqn, attempts FROM backfill_todo WHERE next_attempt_at <= ? ORDER BY next_attempt_at ASC LIMIT ?", + { ps -> ps.setTimestamp(1, now.toSqlTimestamp()); ps.setInt(2, limit) }, ) { rs -> BackfillTodoRepository.BackfillTask( - cminmsgsId = rs.getLong("cminmsgs_id"), + msgId = rs.getLong("msg_id"), sndr = rs.getString("sndr"), type = rs.getString("type"), - styp = rs.getString("styp") ?: "", + styp = rs.getString("styp"), seqn = rs.getLong("seqn"), attempts = rs.getInt("attempts"), ) } - override fun markFailed(cminmsgsId: Long, lastError: String?, nextAttemptAt: Instant, now: Instant) { + override fun markFailed(msgId: Long, lastError: String?, nextAttemptAt: Instant, now: Instant) { ds.update( - "UPDATE backfill_todo SET attempts = attempts + 1, next_attempt_at = ?, last_error = ?, updated_at = ? WHERE cminmsgs_id = ?", - ) { ps -> - ps.setTimestamp(1, nextAttemptAt.toSqlTimestamp()) - ps.setString(2, lastError?.take(512)) - ps.setTimestamp(3, now.toSqlTimestamp()) - ps.setLong(4, cminmsgsId) - } + "UPDATE backfill_todo SET attempts = attempts + 1, last_error = ?, next_attempt_at = ?, updated_at = ? WHERE msg_id = ?", + { ps -> + ps.setString(1, lastError?.take(512)) + ps.setTimestamp(2, nextAttemptAt.toSqlTimestamp()) + ps.setTimestamp(3, now.toSqlTimestamp()) + ps.setLong(4, msgId) + }, + ) } - override fun delete(cminmsgsId: Long) { - ds.update("DELETE FROM backfill_todo WHERE cminmsgs_id = ?") { ps -> ps.setLong(1, cminmsgsId) } + override fun delete(msgId: Long) { + ds.update("DELETE FROM backfill_todo WHERE msg_id = ?", { ps -> ps.setLong(1, msgId) }) } override fun count(): Int = - ds.queryOne("SELECT COUNT(*) AS c FROM backfill_todo", {}) { rs -> rs.getInt("c") } ?: 0 -} - -@Singleton -@Requires(property = "msgx.stubs", notEquals = "true") -@Requires(property = "datasources.default.enabled", value = "true") -class JdbcFlightSchdRepository( - private val ds: DataSource, -) : FlightSchdRepository { - - /** 方言接缝（P3-B）：当前生产 PG；切 11g 需现场验收通过后显式替换（见 SqlDialect 激活门控）。 */ - private val dialect: com.gzzn.omms.msgexchange.infra.persistence.dialect.SqlDialect = - com.gzzn.omms.msgexchange.infra.persistence.dialect.PostgreSqlDialect - - companion object { - /** SCHD.FLTR 标量列 + legacy 派生列（与 V1.1.0 迁移一致，全部可空 VARCHAR）。 */ - private val SCALAR_COLUMNS: List = FlightSchdReadAssembler.SCALAR_COLUMNS - - /** v2 主表写入列：标量 + 异常 + 无界文本；集合由明细表承载（P4 起槽位/里程碑/航路列已退场）。 */ - private val SCALAR_WRITE_COLUMNS: List = FlightSchdReadAssembler.ALL_COLUMNS - - /** 库列名全集（读侧 SELECT 稳定顺序；P4 退场后 = 无损承载列，与 V1.4.0 后表结构一致）。 */ - private val ALL_COLUMNS: List = FlightSchdReadAssembler.ALL_COLUMNS - - private val SCALAR_KEY_SET: Set = SCALAR_COLUMNS.toSet() - } - - private val mapper = ObjectMapper() - - private fun toSqlDate(day: String): java.sql.Date = - java.sql.Date.valueOf(day.trim().take(10)) - - /** v2：仅标量/异常/文本列 flatten；集合键由明细表读写，不经过槽位列。 */ - private fun flattenScalarsOnly(scalars: Map): Map { - val out = mutableMapOf() - scalars.forEach { (key, value) -> - when { - key in SCALAR_KEY_SET -> out[key] = value - key in setOf("SRVT", "VIPF", "MAFL") -> out["${key}_TEXT"] = value - } - } - flattenExceptions(scalars, out) - return out - } - - /** FDIV/FRET/FLAB：1:0..1 单值异常对象 → 前缀标量列（自由文本取首个命中的文本键）。 */ - private fun flattenExceptions(fields: FlightFields, out: MutableMap) { - fun flatten(key: String, mappings: Map, textTarget: String) { - val raw = fields[key] ?: return - val node = runCatching { mapper.readTree(raw) }.getOrNull() ?: return - if (!node.isObject) return - mappings.forEach { (source, target) -> - node[source]?.takeUnless(JsonNode::isNull)?.asText()?.let { out[target] = it } - } - listOf("REMC", "RSN", "value", "#text").asSequence() - .mapNotNull { name -> node[name]?.takeUnless(JsonNode::isNull)?.asText() } - .firstOrNull() - ?.let { out[textTarget] = it } - } - flatten("FDIV", mapOf("DDES" to "FDIV_DDES", "DDIR" to "FDIV_DDIR"), "FDIV_REMC") - flatten("FRET", mapOf("REID" to "FRET_REID"), "FRET_RSN") - flatten("FLAB", mapOf("ARES" to "FLAB_ARES"), "FLAB_RSN") - } - - override fun deleteDiffByDay(day: String, delFlids: Collection): Int { - if (delFlids.isEmpty()) return 0 - var totalDeleted = 0 - val sqlDate = toSqlDate(day) - for (chunk in delFlids.chunked(200)) { - val placeholders = chunk.joinToString(",") { "?" } - // 与主表使用相同日期域；跨日迁移和 FDAY=NULL 的航班必须连同明细保留。 - val ownedFlids = ds.query( - "SELECT flid FROM flight_schd WHERE fday = ? AND flid IN ($placeholders)", - { ps -> - ps.setDate(1, sqlDate) - chunk.forEachIndexed { i, flid -> ps.setString(i + 2, flid) } - }, - ) { rs -> rs.getString("flid") } - FlightDetailTables.deleteForFlids(ds, ownedFlids) - val sql = "DELETE FROM flight_schd WHERE fday = ? AND flid IN ($placeholders)" - totalDeleted += ds.update(sql) { ps -> - ps.setDate(1, sqlDate) - chunk.forEachIndexed { i, flid -> ps.setString(i + 2, flid) } - } - } - return totalDeleted - } - - /** 行 → 字段视图：标量列直读，集合键由明细表重建（v2 §8：完整明细权威读，P2-2 起不再回退槽位列）。 */ - private fun mapFlightRow(rs: java.sql.ResultSet): Pair { - val flid = rs.getString("flid") - val row = linkedMapOf() - for (column in ALL_COLUMNS) { - row[column] = rs.getString(column) - } - val detailCollections = FlightDetailTables.loadCollections(ds, flid) - return flid to FlightSchdReadAssembler.assembleDetailOnly(flid, row, detailCollections) - } - - override fun findByFlid(flid: String): FlightFields? = - ds.queryOne( - "SELECT flid, ${ALL_COLUMNS.joinToString(", ")} FROM flight_schd WHERE flid = ?", - { ps -> ps.setString(1, flid) }, - ::mapFlightRow, - )?.second - - override fun findNextStateByFlid(flid: String): com.gzzn.omms.msgexchange.domain.flight.FlightNextState? = - ds.queryOne( - "SELECT flid, state_version, last_message_id, ${ALL_COLUMNS.joinToString(", ")} FROM flight_schd WHERE flid = ?", - { ps -> ps.setString(1, flid) }, - ) { rs -> - val (id, fields) = mapFlightRow(rs) - com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine.fromFlightFields( - id, - fields, - rs.getLong("state_version"), - rs.getString("last_message_id") ?: "", - ) - } - - override fun findByFlids(flids: Collection): Map { - if (flids.isEmpty()) return emptyMap() - val result = linkedMapOf() - val sql = "SELECT flid, ${ALL_COLUMNS.joinToString(", ")} FROM flight_schd WHERE flid IN (" - for (chunk in flids.chunked(200)) { - val placeholders = chunk.joinToString(",") { "?" } - ds.query( - sql + placeholders + ")", - { ps -> chunk.forEachIndexed { i, flid -> ps.setString(i + 1, flid) } }, - ) { rs -> mapFlightRow(rs) }.forEach { (flid, fields) -> result[flid] = fields } - } - return result - } - - override fun findByDay(day: String): List> = - ds.query( - "SELECT flid, ${ALL_COLUMNS.joinToString(", ")} FROM flight_schd WHERE fday = ? ORDER BY flid ASC", - { ps -> ps.setDate(1, toSqlDate(day)) }, - ::mapFlightRow, - ) - - override fun findAll(): Map = - ds.query( - "SELECT flid, ${ALL_COLUMNS.joinToString(", ")} FROM flight_schd ORDER BY flid ASC", - {}, - ) { rs -> mapFlightRow(rs) }.toMap() - - override fun deleteByFlids(flids: Set): Int { - if (flids.isEmpty()) return 0 - FlightDetailTables.deleteForFlids(ds, flids) - var totalDeleted = 0 - for (chunk in flids.chunked(200)) { - val placeholders = chunk.joinToString(",") { "?" } - val sql = "DELETE FROM flight_schd WHERE flid IN ($placeholders)" - totalDeleted += ds.update(sql) { ps -> - chunk.forEachIndexed { i, flid -> ps.setString(i + 1, flid) } - } - } - return totalDeleted - } - - override fun getGen(day: String): FlightSchdRepository.GenMeta? { - val meta = ds.queryOne( - "SELECT version, last_message_id, updated_at FROM schd_gen WHERE fday = ?", - { ps -> ps.setDate(1, toSqlDate(day)) }, - ) { rs -> - Triple(rs.getLong("version"), rs.getString("last_message_id"), rs.getInstant("updated_at") ?: Instant.now()) - } - ?: return null - val flids = ds.query( - "SELECT flid FROM schd_gen_flid WHERE fday = ? ORDER BY flid ASC", - { ps -> ps.setDate(1, toSqlDate(day)) }, - ) { rs -> rs.getString("flid") }.toSet() - return FlightSchdRepository.GenMeta( - fday = day, - version = meta.first, - flids = flids, - lastMessageId = meta.second, - updatedAt = meta.third, - ) - } - - /** 当前代 FLID 集合整体替换（与 SCHD_GEN 版本推进同事务）。 */ - private fun replaceGenFlids(day: String, flids: Set) { - ds.update("DELETE FROM schd_gen_flid WHERE fday = ?") { ps -> - ps.setDate(1, toSqlDate(day)) - } - if (flids.isEmpty()) return - val conn = ds.obtainConnection() - try { - conn.prepareStatement("INSERT INTO schd_gen_flid (fday, flid) VALUES (?, ?)").use { ps -> - var count = 0 - for (flid in flids.sorted()) { - ps.setDate(1, toSqlDate(day)) - ps.setString(2, flid) - ps.addBatch() - count++ - if (count % 500 == 0) { - ps.executeBatch() - } - } - if (count % 500 != 0) { - ps.executeBatch() - } - } - } finally { - conn.releaseIfNotInTransaction() - } - } - - override fun putGenIfVersion(day: String, expected: Long, newGen: FlightSchdRepository.GenMeta, now: Instant): Boolean { - val sqlDate = toSqlDate(day) - val sqlTimestamp = now.toSqlTimestamp() - if (expected == 0L) { - val inserted = ds.update( - """ - INSERT INTO schd_gen (fday, version, last_message_id, updated_at) - VALUES (?, ?, ?, ?) - ON CONFLICT (fday) DO NOTHING - """.trimIndent(), - ) { ps -> - ps.setDate(1, sqlDate) - ps.setLong(2, newGen.version) - ps.setString(3, newGen.lastMessageId) - ps.setTimestamp(4, sqlTimestamp) - } - if (inserted == 1) { - replaceGenFlids(day, newGen.flids) - return true - } - } - - val updated = ds.update( - """ - UPDATE schd_gen - SET version = ?, last_message_id = ?, updated_at = ? - WHERE fday = ? AND version = ? - """.trimIndent(), - ) { ps -> - ps.setLong(1, newGen.version) - ps.setString(2, newGen.lastMessageId) - ps.setTimestamp(3, sqlTimestamp) - ps.setDate(4, sqlDate) - ps.setLong(5, expected) - } - if (updated == 1) { - replaceGenFlids(day, newGen.flids) - return true - } - return false - } - - - /** v2 主表行存在性保障（增量路径新插 FDAY=NULL）；SQL 由方言接缝生成（P3-B）。 */ - private fun ensureFlightRow(flid: String, now: Instant) { - val ts = now.toSqlTimestamp() - ds.update(dialect.flightRowEnsureSql()) { ps -> - ps.setString(1, flid) - ps.setTimestamp(2, ts) - ps.setTimestamp(3, ts) - } - } - - /** v2 快照：标量列全量替换（集合不写槽位/里程碑列）；SQL 由方言接缝生成（P3-B）。 */ - private fun upsertSnapshotScalars(day: String, state: com.gzzn.omms.msgexchange.domain.flight.FlightNextState, now: Instant) { - val stored = flattenScalarsOnly(state.scalars) - val sqlDate = toSqlDate(day) - val sqlTimestamp = now.toSqlTimestamp() - val sql = dialect.flightSnapshotUpsertSql(SCALAR_WRITE_COLUMNS) - ds.update(sql) { ps -> - var i = 1 - ps.setString(i++, state.flid) - ps.setDate(i++, sqlDate) - for (column in SCALAR_WRITE_COLUMNS) { - ps.setString(i++, stored[column]) - } - ps.setString(i++, state.lastMessageId) - ps.setLong(i++, state.stateVersion) - ps.setTimestamp(i++, sqlTimestamp) - ps.setTimestamp(i++, sqlTimestamp) - } - } - - /** 显式清除键 → 需要置 NULL 的物理列（异常前缀列 / 文本列 / 普通标量列）。 */ - private fun clearColumnsFor(keys: Set): List = keys.flatMap { key -> - when { - key == "FDIV" -> listOf("FDIV_DDES", "FDIV_DDIR", "FDIV_REMC") - key == "FRET" -> listOf("FRET_REID", "FRET_RSN") - key == "FLAB" -> listOf("FLAB_ARES", "FLAB_RSN") - key == "SRVT" -> listOf("SRVT_TEXT") - key == "VIPF" -> listOf("VIPF_TEXT") - key == "MAFL" -> listOf("MAFL_TEXT") - key in SCALAR_KEY_SET -> listOf(key) - else -> emptyList() - } - } - - /** v2 增量：仅更新出现的标量列 + 显式清除列置 NULL + 追踪字段。 */ - private fun upsertIncrementalScalars(state: com.gzzn.omms.msgexchange.domain.flight.FlightNextState, now: Instant) { - val stored = flattenScalarsOnly(state.scalars) - val clearedColumns = clearColumnsFor(state.clearedKeys) - if (stored.isEmpty() && clearedColumns.isEmpty()) { - updateTrackingFields(state, now) - return - } - val assignments = stored.keys.map { "$it = ?" } + - clearedColumns.map { "$it = NULL" } + - listOf("last_message_id = ?", "state_version = ?", "updated_at = ?") - val sql = "UPDATE flight_schd SET ${assignments.joinToString(", ")} WHERE flid = ?" - val sqlTimestamp = now.toSqlTimestamp() - ds.update(sql) { ps -> - var i = 1 - stored.values.forEach { ps.setString(i++, it) } - ps.setString(i++, state.lastMessageId) - ps.setLong(i++, state.stateVersion) - ps.setTimestamp(i++, sqlTimestamp) - ps.setString(i, state.flid) - } - } - - private fun updateTrackingFields(state: com.gzzn.omms.msgexchange.domain.flight.FlightNextState, now: Instant) { - ds.update( - "UPDATE flight_schd SET last_message_id = ?, state_version = ?, updated_at = ? WHERE flid = ?", - ) { ps -> - ps.setString(1, state.lastMessageId) - ps.setLong(2, state.stateVersion) - ps.setTimestamp(3, now.toSqlTimestamp()) - ps.setString(4, state.flid) - } - } - - override fun persistNextStates( - day: String?, - states: List, - snapshotReplace: Boolean, - now: Instant, - ) { - if (states.isEmpty()) return - for (state in states) { - if (snapshotReplace && day != null) { - upsertSnapshotScalars(day, state, now) - } else { - ensureFlightRow(state.flid, now) - upsertIncrementalScalars(state, now) - } - FlightDetailTables.replaceAll(ds, state, now) - } - } - - override fun deleteGenBefore(cutoffDay: String): Int = - ds.update( - "DELETE FROM schd_gen WHERE fday < ?", - { ps -> ps.setDate(1, toSqlDate(cutoffDay)) }, - ) -} - -@Singleton -@Requires(property = "msgx.stubs", notEquals = "true") -@Requires(property = "datasources.default.enabled", value = "true") -class JdbcRefDataRepository( - private val flightSchd: FlightSchdRepository, -) : RefDataRepository { - override fun getGen(day: String): RefDataRepository.GenMeta? = - flightSchd.getGen(day)?.let { RefDataRepository.GenMeta(it.flids.toList(), it.version) } - - override fun putGenIfVersion(day: String, expected: Long, new: RefDataRepository.GenMeta): Boolean = - flightSchd.putGenIfVersion( - day, - expected, - FlightSchdRepository.GenMeta(day, new.version, new.flids.toSet()), - ) -} - -@Singleton -@Requires(property = "msgx.stubs", notEquals = "true") -@Requires(property = "datasources.default.enabled", value = "true") -class JdbcStaticRefRepository( - private val ds: DataSource, -) : StaticRefRepository { - - override fun upsertAll(refs: List) { - val now = Instant.now() - refs.forEach { r -> - ds.update( - """ - INSERT INTO ref_master (rtype, rkey, payload_json, source, refreshed_at) - VALUES (?, ?, ?, ?, ?) - ON CONFLICT (rtype, rkey) DO UPDATE SET - payload_json = EXCLUDED.payload_json, - source = EXCLUDED.source, - refreshed_at = EXCLUDED.refreshed_at - """.trimIndent(), - ) { ps -> - ps.setString(1, r.rtype) - ps.setString(2, r.rkey) - ps.setString(3, r.payloadJson) - ps.setString(4, r.source) - ps.setTimestamp(5, now.toSqlTimestamp()) - } - } - } - - override fun findByType(type: String): List = - ds.query( - "SELECT rtype, rkey, payload_json, source FROM ref_master WHERE rtype = ?", - { ps -> ps.setString(1, type) }, - ) { rs -> - RefUpsert( - rtype = rs.getString("rtype"), - rkey = rs.getString("rkey"), - payloadJson = rs.getString("payload_json"), - source = rs.getString("source"), - ) - } -} - -@Singleton -@Requires(property = "msgx.stubs", notEquals = "true") -@Requires(property = "datasources.default.enabled", value = "true") -class JdbcReqTrackRepository( - private val ds: DataSource, -) : ReqTrackRepository { - - override fun findOpenByKind(kind: String): ReqTrackRepository.Req? = - ds.queryOne( - """ - SELECT req_id, req_type, state, sent_at FROM req_track - WHERE req_type = ? AND state IN ('REGISTERED', 'SENT', 'WAITING') - ORDER BY req_id ASC LIMIT 1 - """.trimIndent(), - { ps -> ps.setString(1, kind) }, - ) { rs -> - ReqTrackRepository.Req( - reqId = rs.getLong("req_id"), - reqType = rs.getString("req_type"), - state = rs.getString("state"), - sentAt = rs.getInstant("sent_at"), - ) - } - - override fun forceExpireOpenOf(kind: String) { - ds.update( - """ - UPDATE req_track SET state = 'EXPIRED' - WHERE req_type = ? AND state IN ('REGISTERED', 'SENT', 'WAITING') - """.trimIndent(), - ) { ps -> ps.setString(1, kind) } - } - - override fun insert(kind: String, paramsJson: String): Long = - ds.updateReturningLong( - "INSERT INTO req_track (req_type, params_json, state) VALUES (?, ?, 'REGISTERED')", - ) { ps -> - ps.setString(1, kind) - ps.setString(2, paramsJson) - } - - override fun linkCoutmsgs(reqId: Long, coutmsgsId: Long) { - ds.update("UPDATE req_track SET coutmsgs_id = ? WHERE req_id = ?") { ps -> - ps.setLong(1, coutmsgsId) - ps.setLong(2, reqId) - } - } - - override fun markSent(reqId: Long, sentAt: Instant) { - ds.update("UPDATE req_track SET state = 'SENT', sent_at = ? WHERE req_id = ?") { ps -> - ps.setTimestamp(1, sentAt.toSqlTimestamp()) - ps.setLong(2, reqId) - } - } - - override fun expireIfWaiting(reqId: Long) { - ds.update("UPDATE req_track SET state = 'EXPIRED' WHERE req_id = ? AND state = 'WAITING'") { ps -> - ps.setLong(1, reqId) - } - } - - override fun markDone(reqId: Long) { - ds.update( - "UPDATE req_track SET state = 'DONE', completed_at = ? WHERE req_id = ?", - ) { ps -> - ps.setTimestamp(1, Instant.now().toSqlTimestamp()) - ps.setLong(2, reqId) - } - } + ds.queryOne("SELECT count(*) AS n FROM backfill_todo", {}) { rs -> rs.getInt("n") } ?: 0 } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/pipeline/PipelineHolderFactory.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/pipeline/PipelineHolderFactory.kt deleted file mode 100644 index dca40d4..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/pipeline/PipelineHolderFactory.kt +++ /dev/null @@ -1,23 +0,0 @@ -package com.gzzn.omms.msgexchange.infra.pipeline - -import com.gzzn.omms.msgexchange.codec.JacksonXmlCodec -import com.gzzn.omms.msgexchange.processing.CodecHolder -import com.gzzn.omms.msgexchange.processing.Handler -import com.gzzn.omms.msgexchange.processing.HandlerHolder -import com.gzzn.omms.msgexchange.processing.HandlerRegistry -import io.micronaut.context.annotation.Bean -import io.micronaut.context.annotation.Factory -import jakarta.inject.Singleton - -/** 主泵解码与 Handler 装配（stub / 实装模式共用真实 XmlCodec + 已注册 Handler）。 */ -@Factory -class PipelineHolderFactory { - - @Bean - @Singleton - fun codecHolder(codec: JacksonXmlCodec): CodecHolder = CodecHolder(codec) - - @Bean - @Singleton - fun handlerHolder(handlers: List): HandlerHolder = HandlerHolder(HandlerRegistry(handlers)) -} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/retry/ProcFailure.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/retry/ProcFailure.kt index 3c5208c..c421af9 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/retry/ProcFailure.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/retry/ProcFailure.kt @@ -20,14 +20,14 @@ class ProcFailure( val attempts = head.attempts + 1 if (scheduler.exhausted(attempts)) { procState.update( - head.cminmsgsId, ProcStatus.DEAD, + head.msgId, ProcStatus.DEAD, attempts = attempts, errorClass = ErrorClass.EXHAUSTED, lastError = "$reason; attempts=$attempts", ) } else { procState.update( - head.cminmsgsId, ProcStatus.FAILED, + head.msgId, ProcStatus.FAILED, attempts = attempts, nextAttemptAt = scheduler.nextAttemptAt(attempts), errorClass = ec, diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/stub/StubAdapters.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/stub/StubAdapters.kt index cf03809..89aa0f5 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/stub/StubAdapters.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/stub/StubAdapters.kt @@ -5,16 +5,28 @@ import io.micronaut.context.annotation.Requires import jakarta.inject.Singleton /** - * U07：stub 适配层——DeliveryPort 内存实现，仅在 msgx.stubs=true 时生效。 - * XmlCodec / Handler 由 [com.gzzn.omms.msgexchange.infra.pipeline.PipelineHolderFactory] 统一装配。 + * stub 适配层——DeliveryPort 内存实现，仅在 msgx.stubs=true 时生效。 + * 记录 (topic, key, payload)：payload = null 表示 TOMBSTONE（§7.3 键缺失=删除旧值）。 */ @Requires(property = "msgx.stubs", value = "true") @Singleton class StubDeliveryPort : DeliveryPort { - val sent = mutableListOf>() + data class Sent(val topic: String, val key: String, val payload: String?) - fun clear() { sent.clear() } + val sent = mutableListOf() + val tombstones: List get() = sent.filter { it.payload == null } - override fun sendKafka(topic: String, payloadJson: String) { sent += topic to payloadJson } - override fun indexFlightHts(payloadJson: String) = Unit + fun clear() = sent.clear() + + override fun sendKafka(topic: String, key: String, payloadJson: String) { + sent += Sent(topic, key, payloadJson) + } + + override fun sendKafkaSchd(topic: String, key: String, payloadJson: String) { + sent += Sent(topic, key, payloadJson) + } + + override fun sendKafkaNull(topic: String, key: String) { + sent += Sent(topic, key, null) + } } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/stub/StubRepositories.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/stub/StubRepositories.kt index c708c8a..f73ab68 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/infra/stub/StubRepositories.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/infra/stub/StubRepositories.kt @@ -2,79 +2,105 @@ package com.gzzn.omms.msgexchange.infra.stub import com.gzzn.omms.msgexchange.domain.ErrorClass import com.gzzn.omms.msgexchange.domain.EventStatus +import com.gzzn.omms.msgexchange.domain.EventType import com.gzzn.omms.msgexchange.domain.MsgEvent import com.gzzn.omms.msgexchange.domain.ProcState import com.gzzn.omms.msgexchange.domain.ProcStatus -import com.gzzn.omms.msgexchange.domain.RefUpsert +import com.gzzn.omms.msgexchange.domain.flight.FlightMainRow +import com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot +import com.gzzn.omms.msgexchange.domain.flight.FlightState +import com.gzzn.omms.msgexchange.domain.flight.HistoryCandidate +import com.gzzn.omms.msgexchange.domain.flight.HistoryRules import com.gzzn.omms.msgexchange.infra.persistence.BackfillTodoRepository import com.gzzn.omms.msgexchange.infra.persistence.CminmsgInboxRepository -import com.gzzn.omms.msgexchange.infra.persistence.FlightFields -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository +import com.gzzn.omms.msgexchange.infra.persistence.FlightStateRepository import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository +import com.gzzn.omms.msgexchange.infra.persistence.PersistOutcome +import com.gzzn.omms.msgexchange.infra.persistence.PipelineLockRepository import com.gzzn.omms.msgexchange.infra.persistence.PipelineTransactionManager import com.gzzn.omms.msgexchange.infra.persistence.ProcStateRepository -import com.gzzn.omms.msgexchange.infra.persistence.PumpJobRepository -import com.gzzn.omms.msgexchange.infra.persistence.RefDataRepository import com.gzzn.omms.msgexchange.infra.persistence.ReqTrackRepository -import com.gzzn.omms.msgexchange.infra.persistence.StaticRefRepository +import com.gzzn.omms.msgexchange.domain.SnapshotLogEntry +import com.gzzn.omms.msgexchange.infra.persistence.SnapshotLogRepository import io.micronaut.context.annotation.Requires import jakarta.inject.Singleton import java.time.Instant +import java.time.LocalDate +import java.time.ZoneId import java.util.concurrent.atomic.AtomicLong /** - * U07（U01 遗留 stub）：内存仓储装配——仅当 `msgx.stubs=true`（dev/影子冒烟）时生效 - * （@Requires），让「启动 + compat HTTP 写 + 主泵/投递循环 + /beans 端到端」在无 MySQL/Redis/Kafka 时可跑。 - * 与真实 Micronaut Data 实装按模块替换；切换点条件显式，不误入生产。 + * stub 仓储（msgx.stubs=true 时装配）——内存实现全部契约，测试与无库环境用。 + * 事务管理器直接执行 block（无嵌套语义）；锁为 no-op（单线程测试前提）。 */ -@Requires(property = "msgx.stubs", value = "true") @Singleton +@Requires(property = "msgx.stubs", value = "true") +class StubPipelineTx : PipelineTransactionManager { + override fun inTransaction(block: () -> T): T = block() +} + +@Singleton +@Requires(property = "msgx.stubs", value = "true") +class StubPipelineLock : PipelineLockRepository { + override fun lock() = Unit +} + +@Singleton +@Requires(property = "msgx.stubs", value = "true") class StubProcState : ProcStateRepository { - private val rows = linkedMapOf() - private val bound = mutableMapOf() + val rows = linkedMapOf() + val bound = linkedMapOf() - fun clear() { rows.clear(); bound.clear() } - - override fun insert(cminmsgsId: Long, state: ProcStatus) { - rows[cminmsgsId] = ProcState(cminmsgsId, state) + fun clear() { + rows.clear(); bound.clear() } - override fun exists(cminmsgsId: Long): Boolean = rows.containsKey(cminmsgsId) + override fun insert(msgId: Long, state: ProcStatus) { + rows.getOrPut(msgId) { ProcState(msgId, state) } + } + + override fun exists(msgId: Long): Boolean = rows.containsKey(msgId) + + override fun find(msgId: Long): ProcState? = rows[msgId] + + override fun findSuccessTerminal(msgId: Long): Boolean = rows[msgId]?.state == ProcStatus.SUCCEEDED override fun headUnfinished(): ProcState? = - rows.filterValues { it.state == ProcStatus.PENDING || it.state == ProcStatus.FAILED } - .minByOrNull { it.key }?.value + rows.values.filter { it.state == ProcStatus.PENDING || it.state == ProcStatus.FAILED }.minByOrNull { it.msgId } - override fun tryBindIdentity(cminmsgsId: Long, identityKey: String): Boolean { + override fun tryBindIdentity(msgId: Long, identityKey: String): Boolean { val owner = bound[identityKey] - if (owner != null && owner != cminmsgsId) return false - bound[identityKey] = cminmsgsId - rows[cminmsgsId] = (rows[cminmsgsId] ?: ProcState(cminmsgsId, ProcStatus.PENDING)).copy(identityKey = identityKey) + if (owner != null && owner != msgId) return false + bound[identityKey] = msgId + rows[msgId] = (rows[msgId] ?: ProcState(msgId, ProcStatus.PENDING)).copy(identityKey = identityKey) return true } override fun ownerOfIdentity(identityKey: String): Long? = bound[identityKey] override fun update( - cminmsgsId: Long, state: ProcStatus, nextAttemptAt: Instant?, attempts: Int?, - errorClass: ErrorClass?, lastError: String?, + msgId: Long, + state: ProcStatus, + nextAttemptAt: Instant?, + attempts: Int?, + errorClass: ErrorClass?, + lastError: String?, ) { - val old = rows[cminmsgsId] ?: ProcState(cminmsgsId, state) - rows[cminmsgsId] = old.copy( + val old = rows[msgId] ?: ProcState(msgId, state) + rows[msgId] = old.copy( state = state, nextAttemptAt = nextAttemptAt ?: old.nextAttemptAt, attempts = attempts ?: old.attempts, errorClass = errorClass ?: old.errorClass, lastError = lastError ?: old.lastError, + updatedAt = Instant.now(), ) } override fun requeueByErrorClasses(errorClasses: List): Int { var n = 0 - rows.keys.toList().forEach { id -> - val s = rows[id]!! - if (s.errorClass != null && s.errorClass in errorClasses && - (s.state == ProcStatus.FAILED || s.state == ProcStatus.DEAD)) { + rows.forEach { (id, s) -> + if (s.errorClass in errorClasses && (s.state == ProcStatus.FAILED || s.state == ProcStatus.DEAD)) { rows[id] = s.copy(state = ProcStatus.PENDING, attempts = 0, nextAttemptAt = null) n++ } @@ -82,360 +108,252 @@ class StubProcState : ProcStateRepository { return n } - /** 测试/运维观测用：读取当前状态行。 */ - fun snapshotOf(cminmsgsId: Long): ProcState? = rows[cminmsgsId] + fun snapshotOf(msgId: Long): ProcState? = rows[msgId] } -@Requires(property = "msgx.stubs", value = "true") @Singleton -class StubInbox : CminmsgInboxRepository { - private val raws = linkedMapOf() - private val processed = mutableSetOf() - - fun clear() { - raws.clear() - processed.clear() - } +@Requires(property = "msgx.stubs", value = "true") +class StubMsgEvents : MsgEventRepository { + val rows = linkedMapOf() private val ids = AtomicLong(0) - /** 模拟上游外部写信箱（不经 compat HTTP、不自动入队 PG）。 */ - fun simulateExternalWrite(rawXml: String): Long { + fun clear() = rows.clear() + + override fun insertAll(events: List): List = + events.map { e -> + val id = ids.incrementAndGet() + rows[id] = e.copy(eventId = id) + id + } + + override fun headUnsent(target: String): MsgEvent? = + rows.values.filter { it.target == target && it.state == EventStatus.PENDING }.minByOrNull { it.eventId!! } + + override fun claimBatch(target: String, limit: Int): List = + rows.values.filter { it.target == target && it.state == EventStatus.PENDING }.sortedBy { it.eventId!! }.take(limit) + + /** §7.3：同 FLID 取最新 STATE_VERSION，按事件序输出。 */ + override fun mergePendingSchd(limit: Int): List = + rows.values + .filter { it.target == "KAFKA:schd" && it.state == EventStatus.PENDING } + .groupBy { it.partitionKey } + .map { (_, group) -> group.maxBy { it.stateVersion } } + .sortedBy { it.eventId!! } + .take(limit) + + override fun markSent(eventId: Long) { + rows[eventId] = (rows[eventId] ?: return).copy(state = EventStatus.SENT) + } + + override fun markAllSent(eventIds: List) { + eventIds.forEach { markSent(it) } + } + + override fun scheduleRetry(eventId: Long, nextAttemptAt: Instant, attempts: Int) { + rows[eventId] = (rows[eventId] ?: return).copy(nextAttemptAt = nextAttemptAt, attempts = attempts) + } + + override fun markDead(eventId: Long, errorClass: ErrorClass, lastError: String, attempts: Int?) { + rows[eventId] = (rows[eventId] ?: return).copy( + state = EventStatus.DEAD, errorClass = errorClass, lastError = lastError, + attempts = attempts ?: rows[eventId]?.attempts ?: 0, + ) + } +} + +@Singleton +@Requires(property = "msgx.stubs", value = "true") +class StubFlightState : FlightStateRepository { + val mains = linkedMapOf() + val snapshots = linkedMapOf() // 内存全量态（主行+明细一体） + + fun clear() { + mains.clear(); snapshots.clear() + } + + override fun findMainRow(flid: String): FlightMainRow? = mains[flid] + + override fun findMainRows(flids: Collection): Map = + flids.mapNotNull { flid -> mains[flid]?.let { flid to it } }.toMap() + + override fun loadFullSnapshot(flid: String): FlightSnapshot? = snapshots[flid] + + /** §7.4 不可变条件：已有非空 OPERATION_DAY 且与新值不同 → DAY_GUARD_VIOLATION。 */ + override fun persistFullState(snapshot: FlightSnapshot, msgId: Long, now: Instant): PersistOutcome { + val existing = mains[snapshot.flid] + if (existing?.operationDay != null && existing.operationDay != snapshot.operationDay) { + return PersistOutcome.DAY_GUARD_VIOLATION + } + val main = FlightMainRow( + flid = snapshot.flid, + operationDay = snapshot.operationDay ?: existing?.operationDay, + state = snapshot.state, + stateVersion = snapshot.stateVersion, + lastMsgId = msgId, + updatedAt = now, + ) + mains[snapshot.flid] = main + snapshots[snapshot.flid] = snapshot.copy(operationDay = main.operationDay) + return if (existing == null) PersistOutcome.INSERTED else PersistOutcome.UPDATED + } + + override fun markDeleted(flid: String, msgId: Long, now: Instant): Boolean { + val main = mains[flid] ?: return false + if (main.state != FlightState.ACTIVE) return false + mains[flid] = main.copy(state = FlightState.DELETED, stateVersion = main.stateVersion + 1, lastMsgId = msgId, updatedAt = now) + snapshots[flid] = (snapshots[flid] ?: return true).copy(state = FlightState.DELETED, stateVersion = main.stateVersion + 1) + return true + } + + override fun revive(flid: String, msgId: Long, now: Instant): Boolean { + val main = mains[flid] ?: return false + if (main.state != FlightState.DELETED) return false + mains[flid] = main.copy(state = FlightState.ACTIVE, stateVersion = main.stateVersion + 1, lastMsgId = msgId, updatedAt = now) + snapshots[flid] = (snapshots[flid] ?: return true).copy(state = FlightState.ACTIVE, stateVersion = main.stateVersion + 1) + return true + } + + override fun findHistoryCandidates(rules: HistoryRules, zone: ZoneId, now: Instant): List = + mains.values.mapNotNull { main -> + val snap = snapshots[main.flid] + val cancelled = snap?.scalars?.get("CNCL") + val hasTerminalField = listOf("CNCL", "NAAT", "NEAT").any { !snap?.scalars?.get(it).isNullOrBlank() } + val idleHit = !hasTerminalField && main.updatedAt < now.minusSeconds(rules.idleHours * 3600) + val deletedHit = main.state == FlightState.DELETED && main.updatedAt < now.minusSeconds(rules.deletedHours * 3600) + if (idleHit || deletedHit) { + HistoryCandidate(main.flid, main.state, main.stateVersion, wasNeverFdel = main.state == FlightState.ACTIVE) + } else { + null + } + } + + override fun purgeArchived(flids: Collection): Int { + var n = 0 + flids.forEach { flid -> + if (mains.remove(flid) != null) n++ + snapshots.remove(flid) + } + return n + } + + override fun countOperationDayNull(): Int = mains.values.count { it.operationDay == null } +} + +@Singleton +@Requires(property = "msgx.stubs", value = "true") +class StubSnapshotLog : SnapshotLogRepository { + val entries = mutableListOf() + + fun clear() = entries.clear() + + override fun append(entry: SnapshotLogEntry) { + entries.add(entry) + } +} + +@Singleton +@Requires(property = "msgx.stubs", value = "true") +class StubReqTrack : ReqTrackRepository { + val rows = linkedMapOf() + private val ids = AtomicLong(0) + + fun clear() = rows.clear() + + override fun insert(reqType: String, operationDay: LocalDate, sender: String): Long { + // §7.1：同类（类型+运营日+发送方）旧有效请求先置 EXPIRED + rows.values.filter { + it.reqType == reqType && it.operationDay == operationDay && it.sender == sender && + (it.state == ReqTrackRepository.ReqState.PENDING || it.state == ReqTrackRepository.ReqState.SENT) + }.forEach { rows[it.reqId] = it.copy(state = ReqTrackRepository.ReqState.EXPIRED) } val id = ids.incrementAndGet() - raws[id] = rawXml + rows[id] = ReqTrackRepository.Req(id, reqType, operationDay, sender, ReqTrackRepository.ReqState.PENDING) return id } + override fun findLatest( + reqType: String, + operationDay: LocalDate, + sender: String, + states: List, + ): ReqTrackRepository.Req? = + rows.values.filter { + it.reqType == reqType && it.operationDay == operationDay && it.sender == sender && it.state in states + }.maxByOrNull { it.reqId } + + override fun completeLatest(reqType: String, operationDay: LocalDate, sender: String): Boolean { + val req = findLatest(reqType, operationDay, sender, listOf(ReqTrackRepository.ReqState.PENDING, ReqTrackRepository.ReqState.SENT)) + ?: return false + rows[req.reqId] = req.copy(state = ReqTrackRepository.ReqState.DONE) + return true + } + + override fun linkCoutmsgs(reqId: Long, coutmsgsId: Long) { + rows[reqId]?.let { rows[reqId] = it.copy(coutmsgsId = coutmsgsId) } + } + + override fun markSent(reqId: Long, sentAt: Instant) { + rows[reqId]?.let { rows[reqId] = it.copy(state = ReqTrackRepository.ReqState.SENT, sentAt = sentAt) } + } + + override fun expire(reqId: Long) { + rows[reqId]?.let { rows[reqId] = it.copy(state = ReqTrackRepository.ReqState.EXPIRED) } + } +} + +@Singleton +@Requires(property = "msgx.stubs", value = "true") +class StubBackfillTodo : BackfillTodoRepository { + val tasks = linkedMapOf() + private val errors = linkedMapOf() + private val due = linkedMapOf() + + fun clear() { tasks.clear(); errors.clear(); due.clear() } + + fun lastErrorOf(msgId: Long): String? = errors[msgId] + + override fun record(task: BackfillTodoRepository.BackfillTask, lastError: String?, now: Instant) { + errors[task.msgId] = lastError + due.putIfAbsent(task.msgId, now) + tasks[task.msgId] = task.copy(attempts = tasks[task.msgId]?.attempts ?: 0) + } + + override fun findDue(now: Instant, limit: Int): List = + tasks.keys.filter { (due[it] ?: Instant.EPOCH) <= now }.take(limit).mapNotNull { tasks[it] } + + override fun markFailed(msgId: Long, lastError: String?, nextAttemptAt: Instant, now: Instant) { + errors[msgId] = lastError + due[msgId] = nextAttemptAt + tasks[msgId]?.let { tasks[msgId] = it.copy(attempts = it.attempts + 1) } + } + + override fun delete(msgId: Long) { + tasks.remove(msgId); errors.remove(msgId); due.remove(msgId) + } + + override fun count(): Int = tasks.size +} + +@Singleton +@Requires(property = "msgx.stubs", value = "true") +class StubInbox : CminmsgInboxRepository { + val raws = linkedMapOf() + private val ids = AtomicLong(0) + + fun clear() = raws.clear() + override fun insertRaw(rawXml: String): Long { val id = ids.incrementAndGet() raws[id] = rawXml return id } - override fun rawOf(cminmsgsId: Long): String? = raws[cminmsgsId] + override fun rawOf(msgId: Long): String? = raws[msgId] override fun pollUnprocessed(afterId: Long, limit: Int): List = - raws.keys - .filter { it > afterId && it !in processed } - .sorted() - .take(limit) + raws.keys.filter { it > afterId }.sorted().take(limit) - override fun backfillOnSuccess(cminmsgsId: Long, sndr: String, type: String, styp: String, seqn: Long) { - processed += cminmsgsId - } -} + override fun backfillOnSuccess(msgId: Long, sndr: String, type: String, styp: String, seqn: Long) = Unit -@Requires(property = "msgx.stubs", value = "true") -@Singleton -class StubMsgEvents : MsgEventRepository { - private val rows = linkedMapOf() - - fun clear() { rows.clear() } - private val ids = AtomicLong(0) - - override fun insertAll(events: List): List = events.map { e -> - val id = ids.incrementAndGet() - rows[id] = e.copy(eventId = id) - id - } - - override fun headUnsent(target: String): MsgEvent? = - rows.values.filter { it.target == target && it.state == EventStatus.PENDING } - .minByOrNull { it.eventId ?: Long.MAX_VALUE } - - override fun claimBatch(target: String, limit: Int): List = - rows.values.filter { it.target == target && it.state == EventStatus.PENDING } - .sortedBy { it.eventId ?: Long.MAX_VALUE } - .take(limit) - - override fun markSent(eventId: Long) = mutate(eventId) { it.copy(state = EventStatus.SENT) } - - override fun markAllSent(eventIds: List) = eventIds.forEach(::markSent) - - override fun scheduleRetry(eventId: Long, nextAttemptAt: Instant, attempts: Int) = - mutate(eventId) { it.copy(state = EventStatus.PENDING, attempts = attempts, nextAttemptAt = nextAttemptAt) } - - override fun markDead(eventId: Long, errorClass: ErrorClass, lastError: String, attempts: Int?) = - mutate(eventId) { it.copy(state = EventStatus.DEAD, errorClass = errorClass, lastError = lastError, attempts = attempts ?: it.attempts) } - - override fun insertSync(events: List) { - insertAll(events) - } - - private fun mutate(eventId: Long, f: (MsgEvent) -> MsgEvent) { - val cur = rows[eventId] ?: return - rows[eventId] = f(cur) - } -} - -@Requires(property = "msgx.stubs", value = "true") -@Singleton -class StubPumpJobs : PumpJobRepository { - - fun clear() { rows.clear() } - data class JobRow(val job: PumpJobRepository.Job, var state: String, var lastError: String? = null) - - private val rows = linkedMapOf() - private val ids = AtomicLong(0) - - override fun enqueue(kind: String) { - val id = ids.incrementAndGet() - rows[id] = JobRow(PumpJobRepository.Job(id, kind), "QUEUED") - } - - override fun headQueued(): PumpJobRepository.Job? = - rows.values.firstOrNull { it.state == "QUEUED" }?.job - - override fun markRunning(jobId: Long) { rows[jobId]?.state = "RUNNING" } - override fun markDone(jobId: Long) { rows[jobId]?.state = "DONE" } - override fun markFailed(jobId: Long, lastError: String) { rows[jobId]?.state = "FAILED"; rows[jobId]?.lastError = lastError } -} - -@Requires(property = "msgx.stubs", value = "true") -@Singleton -class StubPipelineTransactionManager : PipelineTransactionManager { - override fun inTransaction(block: () -> T): T = block() -} - -@Requires(property = "msgx.stubs", value = "true") -@Singleton -class StubFlightSchd : FlightSchdRepository { - data class Record( - val flid: String, - val fday: String?, - val fields: FlightFields, - val stateVersion: Long = 0L, - val lastMessageId: String = "", - val createdAt: Instant, - val updatedAt: Instant, - ) - - private val records = linkedMapOf() - private val detailCollections = mutableMapOf>>>() - private val gens = mutableMapOf() - - fun clear() { - records.clear() - detailCollections.clear() - gens.clear() - } - - override fun deleteDiffByDay(day: String, delFlids: Collection): Int { - if (delFlids.isEmpty()) return 0 - val delSet = delFlids.toSet() - val toRemove = records.filter { (flid, rec) -> rec.fday == day && flid in delSet }.keys - toRemove.forEach { flid -> - records.remove(flid) - detailCollections.remove(flid) - } - return toRemove.size - } - - override fun findByFlid(flid: String): FlightFields? { - val rec = records[flid] ?: return null - return mergeFields(rec) - } - - private fun mergeFields(rec: Record): FlightFields { - val out = linkedMapOf() - out.putAll(rec.fields) - val mapper = com.fasterxml.jackson.databind.ObjectMapper() - detailCollections[rec.flid]?.forEach { (key, items) -> - out[key] = mapper.writeValueAsString(items) - } - return out - } - - override fun findNextStateByFlid(flid: String): com.gzzn.omms.msgexchange.domain.flight.FlightNextState? { - val rec = records[flid] ?: return null - return com.gzzn.omms.msgexchange.domain.flight.FlightNextState( - flid = rec.flid, - scalars = rec.fields.filterKeys { it != "FLID" }, - collections = detailCollections[flid] ?: emptyMap(), - stateVersion = rec.stateVersion, - lastMessageId = rec.lastMessageId, - ) - } - - override fun findByFlids(flids: Collection): Map = - flids.mapNotNull { flid -> records[flid]?.let { flid to mergeFields(it) } }.toMap() - - override fun findByDay(day: String): List> = - records.values.filter { it.fday == day } - .sortedBy { it.flid } - .map { it.flid to mergeFields(it) } - - override fun findAll(): Map = - records.mapValues { mergeFields(it.value) } - - override fun deleteByFlids(flids: Set): Int { - if (flids.isEmpty()) return 0 - var count = 0 - for (flid in flids) { - if (records.remove(flid) != null) count++ - detailCollections.remove(flid) - } - return count - } - - override fun getGen(day: String): FlightSchdRepository.GenMeta? = gens[day] - - override fun putGenIfVersion(day: String, expected: Long, newGen: FlightSchdRepository.GenMeta, now: Instant): Boolean { - val current = gens[day]?.version ?: 0L - if (current != expected) return false - gens[day] = newGen.copy(updatedAt = now) - return true - } - - override fun persistNextStates( - day: String?, - states: List, - snapshotReplace: Boolean, - now: Instant, - ) { - for (state in states) { - val scalarFields = linkedMapOf("FLID" to state.flid) - state.scalars.forEach { (k, v) -> scalarFields[k] = v } - val existing = records[state.flid] - if (snapshotReplace && day != null) { - records[state.flid] = Record( - flid = state.flid, - fday = day, - fields = scalarFields, - stateVersion = state.stateVersion, - lastMessageId = state.lastMessageId, - createdAt = existing?.createdAt ?: now, - updatedAt = now, - ) - detailCollections[state.flid] = state.collections - } else { - // nextState 由引擎在当前态上合并而来，是完整权威态：直接整体替换（含清除语义） - records[state.flid] = Record( - flid = state.flid, - fday = existing?.fday, - fields = scalarFields, - stateVersion = state.stateVersion, - lastMessageId = state.lastMessageId, - createdAt = existing?.createdAt ?: now, - updatedAt = now, - ) - detailCollections[state.flid] = state.collections - } - } - } - - override fun deleteGenBefore(cutoffDay: String): Int { - val toRemove = gens.keys.filter { it < cutoffDay } - toRemove.forEach { gens.remove(it) } - return toRemove.size - } -} - -/** 快照 gen 协议 stub：委托至 StubFlightSchd。 */ -@Requires(property = "msgx.stubs", value = "true") -@Singleton -class StubRefData( - private val flightSchd: FlightSchdRepository, -) : RefDataRepository { - override fun getGen(day: String): RefDataRepository.GenMeta? = - flightSchd.getGen(day)?.let { RefDataRepository.GenMeta(it.flids.toList(), it.version) } - - override fun putGenIfVersion(day: String, expected: Long, new: RefDataRepository.GenMeta): Boolean = - flightSchd.putGenIfVersion( - day, - expected, - FlightSchdRepository.GenMeta(day, new.version, new.flids.toSet()), - ) -} - -/** 21 类静态主数据 stub（独立 PG reference 库；内存实现，source 保留供审计断言）。 */ -@Requires(property = "msgx.stubs", value = "true") -@Singleton -class StubStaticRef : StaticRefRepository { - private val rows = mutableMapOf, RefUpsert>() - - fun clear() { rows.clear() } - - override fun upsertAll(refs: List) { - refs.forEach { rows[it.rtype to it.rkey] = it } - } - - override fun findByType(type: String): List = - rows.values.filter { it.rtype == type } -} - -@Requires(property = "msgx.stubs", value = "true") -@Singleton -class StubReqTrack : ReqTrackRepository { - private val rows = mutableMapOf() - - fun clear() { rows.clear() } - private val ids = AtomicLong(0) - - override fun findOpenByKind(kind: String): ReqTrackRepository.Req? = - rows.values.firstOrNull { it.reqType == kind && it.state in setOf("REGISTERED", "SENT", "WAITING") } - - override fun forceExpireOpenOf(kind: String) { - rows.keys.toList().forEach { id -> - val r = rows[id]!! - if (r.reqType == kind && r.state in setOf("REGISTERED", "SENT", "WAITING")) - rows[id] = r.copy(state = "EXPIRED") - } - } - - override fun insert(kind: String, paramsJson: String): Long { - val id = ids.incrementAndGet() - rows[id] = ReqTrackRepository.Req(id, kind, "REGISTERED") - return id - } - - override fun linkCoutmsgs(reqId: Long, coutmsgsId: Long) = Unit - override fun markSent(reqId: Long, sentAt: Instant) { rows[reqId]?.let { rows[reqId] = it.copy(state = "SENT", sentAt = sentAt) } } - override fun expireIfWaiting(reqId: Long) { rows[reqId]?.let { rows[reqId] = it.copy(state = "EXPIRED") } } - override fun markDone(reqId: Long) { rows[reqId]?.let { rows[reqId] = it.copy(state = "DONE") } } -} - -@Requires(property = "msgx.stubs", value = "true") -@Singleton -class StubBackfillTodo : BackfillTodoRepository { - private data class Row(val task: BackfillTodoRepository.BackfillTask, var nextAttemptAt: Instant) - - private val rows = linkedMapOf() - private val errors = linkedMapOf() - - @Synchronized - fun clear() { - rows.clear() - errors.clear() - } - - @Synchronized - fun tasks(): List = rows.values.map { it.task } - - @Synchronized - fun lastErrorOf(cminmsgsId: Long): String? = errors[cminmsgsId] - - @Synchronized - override fun record(task: BackfillTodoRepository.BackfillTask, lastError: String?, now: Instant) { - rows[task.cminmsgsId] = Row(task, now) - errors[task.cminmsgsId] = lastError - } - - @Synchronized - override fun findDue(now: Instant, limit: Int): List = - rows.values.filter { it.nextAttemptAt <= now }.map { it.task }.take(limit) - - @Synchronized - override fun markFailed(cminmsgsId: Long, lastError: String?, nextAttemptAt: Instant, now: Instant) { - rows[cminmsgsId]?.let { rows[cminmsgsId] = it.copy(task = it.task.copy(attempts = it.task.attempts + 1), nextAttemptAt = nextAttemptAt) } - errors[cminmsgsId] = lastError - } - - @Synchronized - override fun delete(cminmsgsId: Long) { - rows.remove(cminmsgsId) - errors.remove(cminmsgsId) - } - - @Synchronized - override fun count(): Int = rows.size + /** 测试辅助：模拟上游外部写入共享信箱（不经本系统）。 */ + fun simulateExternalWrite(rawXml: String): Long = insertRaw(rawXml) } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxController.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxController.kt index 94ec41b..cb8c694 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxController.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxController.kt @@ -18,7 +18,7 @@ class InboxController(private val inbox: InboxService) { @Produces(MediaType.TEXT_PLAIN) fun send(@Body rawXml: String): HttpResponse { val receipt = inbox.accept(rawXml) - return HttpResponse.ok(receipt.cminmsgsId.toString()) // TODO: 与现役响应体逐字对拍后固化 + return HttpResponse.ok(receipt.msgId.toString()) // TODO: 与现役响应体逐字对拍后固化 } // TODO(阶段2): /schd/sync、/all/flights、/kafka/topics/{name}/msgs 按契约冻结清单补齐。 diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxEnqueue.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxEnqueue.kt index 2fe04b2..7bcd81f 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxEnqueue.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxEnqueue.kt @@ -8,9 +8,9 @@ import jakarta.inject.Singleton class InboxEnqueue(private val procState: ProcStateRepository) { /** @return true 若新建 PENDING 行；false 若已存在（轮询重扫/compensate 幂等）。 */ - fun enqueue(cminmsgsId: Long): Boolean { - if (procState.exists(cminmsgsId)) return false - procState.insert(cminmsgsId) + fun enqueue(msgId: Long): Boolean { + if (procState.exists(msgId)) return false + procState.insert(msgId) return true } } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxPoller.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxPoller.kt index 120b4e9..07185d4 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxPoller.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxPoller.kt @@ -29,7 +29,7 @@ class InboxPoller( for (id in ids) { if (enqueue.enqueue(id)) { enqueued++ - log.info("polled cminmsgsId={}", id) + log.info("polled msgId={}", id) } } sweepBackfillTodos() diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxService.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxService.kt index acb7527..af31d13 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxService.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/ingress/InboxService.kt @@ -12,12 +12,12 @@ class InboxService( ) { private val log = org.slf4j.LoggerFactory.getLogger(InboxService::class.java) - data class Receipt(val cminmsgsId: Long, val receivedAt: Instant) + data class Receipt(val msgId: Long, val receivedAt: Instant) fun accept(rawXml: String): Receipt { val id = inbox.insertRaw(rawXml) enqueue.enqueue(id) - log.info("compat-accepted cminmsgsId={}", id) + log.info("compat-accepted msgId={}", id) return Receipt(id, Instant.now()) } } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/BackfillSweepJob.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/BackfillSweepJob.kt index 09dcc01..fed4059 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/BackfillSweepJob.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/BackfillSweepJob.kt @@ -40,11 +40,11 @@ class BackfillSweepJob( var failed = 0 for (task in due) { try { - inbox.backfillOnSuccess(task.cminmsgsId, task.sndr, task.type, task.styp, task.seqn) - todo.delete(task.cminmsgsId) + inbox.backfillOnSuccess(task.msgId, task.sndr, task.type, task.styp, task.seqn) + todo.delete(task.msgId) succeeded++ } catch (e: Exception) { - todo.markFailed(task.cminmsgsId, e.message, now.plus(backoffDelayFor(task.attempts + 1)), now) + todo.markFailed(task.msgId, e.message, now.plus(backoffDelayFor(task.attempts + 1)), now) failed++ } } diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/HistorySweepJob.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/HistorySweepJob.kt new file mode 100644 index 0000000..e2eefe7 --- /dev/null +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/HistorySweepJob.kt @@ -0,0 +1,80 @@ +package com.gzzn.omms.msgexchange.jobs + +import com.gzzn.omms.msgexchange.config.HistoryProps +import com.gzzn.omms.msgexchange.domain.EventType +import com.gzzn.omms.msgexchange.domain.MsgEvent +import com.gzzn.omms.msgexchange.domain.Targets +import com.gzzn.omms.msgexchange.domain.flight.HistoryCandidate +import com.gzzn.omms.msgexchange.domain.flight.HistoryRules +import com.gzzn.omms.msgexchange.infra.persistence.FlightStateRepository +import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository +import jakarta.inject.Singleton +import java.time.Duration +import java.time.Instant +import java.time.ZoneId + +/** + * 历史归档与物理清除（docs/flight-state.md §8.2，顺序不可颠倒）： + * 1. HISTORY_SWEEP 选出满足 §8.1 的航班（含 DELETED）； + * 2. 写入历史存储； + * 3. 历史存储返回成功的 FLID 集合 → 物理删除主行与明细（归档结果只记录在历史存储，当前态无 ARCHIVED 态）； + * 4. 未经 FDEL 的航班在清除前补发一次删除事件（§7.3）；其余不发； + * 5. 失败或不明确的保留重试。 + * 前提红线：历史存储未接通时必须删 0 条（§8.2）。 + */ +@Singleton +class HistorySweepJob( + private val flightState: FlightStateRepository, + private val msgEvents: MsgEventRepository, + private val props: HistoryProps, + /** 历史存储端口：返回归档成功的 FLID 集合；未接通时不注入（null）。 */ + private val historyStore: HistoryStore? = null, + /** §8.3 留痕清理端口（90 天，按 (SCOPE_END, RECV_AT)）；未接通时不注入。 */ + private val snapLogPurge: SnapshotLogPurge? = null, +) { + /** 历史存储端口（由部署侧适配实现；脚手架默认未接通）。 */ + fun interface HistoryStore { + /** 归档候选航班；返回归档确认成功的 FLID 集合（部分成功允许）。 */ + fun archive(candidates: List): Set + } + + fun interface SnapshotLogPurge { + fun purgeBefore(instant: Instant): Int + } + + data class SweepOutcome(val selected: Int, val archived: Int, val purged: Int, val snapLogPurged: Int = 0) + + fun run(now: Instant = Instant.now()): SweepOutcome { + if (!props.historyStoreEnabled || historyStore == null) { + // 红线：历史存储未接通必须删 0 条；绝不允许先删当前态再补历史（§8.2） + return SweepOutcome(selected = 0, archived = 0, purged = 0) + } + + val rules = HistoryRules(props.cancelledHours, props.terminalHours, props.deletedHours, props.idleHours) + val zone = ZoneId.of("Asia/Shanghai") // §8.1：窗口按机场时区计算 + val candidates = flightState.findHistoryCandidates(rules, zone, now) + if (candidates.isEmpty()) return SweepOutcome(0, 0, 0) + + val archivedFlids = historyStore.archive(candidates) + if (archivedFlids.isEmpty()) return SweepOutcome(candidates.size, archived = 0, purged = 0) + + val toPurge = candidates.filter { it.flid in archivedFlids } + // §8.2 步骤 4：未经 FDEL、由生命周期直接清除的航班，清除前补发一次删除事件 + val preDelete = toPurge.filter { it.wasNeverFdel } + if (preDelete.isNotEmpty()) { + msgEvents.insertAll(preDelete.map { tombstone(it) }) + } + val purged = flightState.purgeArchived(archivedFlids) + + val snapLogPurged = snapLogPurge?.purgeBefore(now.minus(Duration.ofDays(props.snapLogRetentionDays))) ?: 0 + return SweepOutcome(candidates.size, archivedFlids.size, purged, snapLogPurged) + } + + private fun tombstone(candidate: HistoryCandidate) = MsgEvent( + target = Targets.KAFKA_SCHD, + partitionKey = candidate.flid, + eventType = EventType.TOMBSTONE, + stateVersion = candidate.stateVersion, + payloadJson = """{"flid":"${candidate.flid}","stateVersion":${candidate.stateVersion},"deleted":true}""", + ) +} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/JobExecutor.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/JobExecutor.kt deleted file mode 100644 index e0d9539..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/JobExecutor.kt +++ /dev/null @@ -1,99 +0,0 @@ -package com.gzzn.omms.msgexchange.jobs - -import com.gzzn.omms.msgexchange.infra.persistence.FlightFields -import com.gzzn.omms.msgexchange.infra.persistence.FlightFieldsJson -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository -import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository -import com.gzzn.omms.msgexchange.infra.persistence.PumpJobRepository -import com.gzzn.omms.msgexchange.domain.MsgEvent -import com.gzzn.omms.msgexchange.domain.Targets -import jakarta.inject.Singleton - -/** - * ACMA-8：泵作业执行器。cron 触发入队（PUMP_JOB），主泵 FIFO 执行（决策 1）—— - * 产物不绕过队头顺序；阶段 A 的清场同步链与归档在此落地（I4/I5）。 - */ -@Singleton -class JobExecutor( - private val historySweep: HistorySweepJob, - private val archive: ArchiveJob, - private val projectionRebuild: ProjectionRebuildJob, - private val backfillSweep: BackfillSweepJob, -) { - fun execute(job: PumpJobRepository.Job) = when (job.kind) { - "HISTORY_SWEEP" -> historySweep.run() - "ARCHIVE" -> archive.run() - "PROJECTION_REBUILD" -> projectionRebuild.run() - "BACKFILL_SWEEP" -> backfillSweep.sweep() - else -> error("unknown job ${job.kind}") - } -} - -/** - * 流程 4 runJob(HISTORY_SWEEP)：3:30 清场——主泵作业内同步链（I4，KEEP 现役 - * FlightHisScheduled 同步语义）：判史 → 同步写 ES（成功集）→ 仅删成功集。 - */ -@Singleton -class HistorySweepJob( - private val flightSchd: FlightSchdRepository, -) { - companion object { - var historyPicker: ((Map) -> Map)? = null - var esArchiver: ((Map) -> Set)? = null - var cutoffProvider: (() -> String?)? = null - } - - fun run() { - val all = flightSchd.findAll() - val history = pickHistory(all) // U10/T07：saveSync 接线前判史为空集——禁止“全量可删”默认 - val success = esArchiver?.invoke(history) ?: emptySet() // 仅收集已确认写入 ES 的成功 FLID 集合 - if (success.isNotEmpty()) { - flightSchd.deleteByFlids(success) // FS2/FS4：仅删除已确认写入 ES 的成功集合，分批参数化删除 - } - val cutoff = cutoffProvider?.invoke() - if (cutoff != null) { - flightSchd.deleteGenBefore(cutoff) - } - } - - /** U10/T07（修订）：接入 ES success 集之前的门禁——占位默认返回空集； - * 现役五条判史规则（SODT 3 天 / CNCL 1 小时 / 备降 / 离港 / 到港）golden 通过后才允许接线。 */ - private fun pickHistory(all: Map): Map = - historyPicker?.invoke(all) ?: emptyMap() -} - -/** 流程 5 runJob(ARCHIVE)：3:00 归档——1 天前且仅终态可迁（矩阵 #12）。 */ -@Singleton -class ArchiveJob( - // TODO(阶段1后续): CMINMSGS⇆CMINMSGS_HST 迁移 SQL：JOIN PROC_STATE，STATE ∈ {SUCCEEDED,DEAD,SKIPPED} -) { - fun run() { - // TODO: 迁移（DEAD 带 ERROR_CLASS、SKIPPED 带 duplicate-of 审计随行保留） - } -} - -/** 流程 7：阶段 B 投影重建——切入阶段 B 时全量重建一次，此后增量走事件。 */ -@Singleton -class ProjectionRebuildJob( - private val flightSchd: FlightSchdRepository, - private val msgEvents: MsgEventRepository, -) { - fun run() { - for (day in activeDays()) { - val flights = flightSchd.findByDay(day) - flights.forEach { (flid, fields) -> - msgEvents.insertSync( - listOf( - MsgEvent( - target = Targets.KAFKA_SCHD, - partitionKey = flid, - payloadJson = FlightFieldsJson.toJson(fields), - ), - ), - ) - } - } - } - - private fun activeDays(): List = emptyList() // TODO(阶段B) -} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/JobRunner.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/JobRunner.kt new file mode 100644 index 0000000..695fac8 --- /dev/null +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/jobs/JobRunner.kt @@ -0,0 +1,75 @@ +package com.gzzn.omms.msgexchange.jobs + +import com.gzzn.omms.msgexchange.config.HistoryProps +import com.gzzn.omms.msgexchange.config.PipelineProps +import jakarta.inject.Singleton +import java.time.Duration +import java.time.Instant +import java.time.LocalDate +import java.time.ZoneId + +/** + * 维护作业调度（docs/flight-state.md §8）：单 daemon 线程，独立于主泵—— + * 旧「作业不插队 PUMP_JOB 队列」机制随审计口径移除（§3.2 表清单无 PUMP_JOB）； + * 历史归档/留痕清理均为内部清理路径，不参与 FIFO 消息序。 + * 触发：回填补偿 30s 固定间隔；历史归档/留痕清理每日 03:30（机场时区）后首个 tick。 + */ +@Singleton +class JobRunner( + private val backfillSweep: BackfillSweepJob, + private val historySweep: HistorySweepJob, + @Suppress("unused") private val pipelineProps: PipelineProps, + @Suppress("unused") private val historyProps: HistoryProps, +) { + private val log = org.slf4j.LoggerFactory.getLogger(JobRunner::class.java) + private val zone: ZoneId = ZoneId.of("Asia/Shanghai") + + @Volatile + private var running = true + + @Volatile + private var lastHistoryDay: LocalDate? = null + + private var thread: Thread? = null + + fun start() { + if (thread != null) return + running = true + thread = Thread.ofPlatform().name("msgx-jobs").daemon(true).start { loop() } + log.info("job runner started (backfill 30s, history daily 03:30 {})", zone) + } + + fun stop() { + running = false + thread?.interrupt() + thread = null + } + + internal fun loop() { + while (running) { + try { + backfillSweep.sweep() + maybeHistorySweep() + } catch (e: InterruptedException) { + Thread.currentThread().interrupt() + return + } catch (e: Exception) { + log.warn("job tick failed: {}", e.message ?: e.javaClass.simpleName) + } + if (running) runCatching { Thread.sleep(Duration.ofSeconds(30).toMillis()) } + } + } + + private fun maybeHistorySweep() { + val today = LocalDate.now(zone) + if (lastHistoryDay == today) return + val now = java.time.LocalTime.now(zone) + if ((now.hour == 3 && now.minute >= 30) || now.hour > 3) { + val outcome = historySweep.run() + lastHistoryDay = today + if (outcome.selected > 0 || outcome.snapLogPurged > 0) { + log.info("history sweep: {}", outcome) + } + } + } +} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/processing/DynamicProcessors.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/processing/DynamicProcessors.kt new file mode 100644 index 0000000..2eb3faf --- /dev/null +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/processing/DynamicProcessors.kt @@ -0,0 +1,214 @@ +package com.gzzn.omms.msgexchange.processing + +import com.fasterxml.jackson.databind.ObjectMapper +import com.gzzn.omms.msgexchange.codec.FlopPayload +import com.gzzn.omms.msgexchange.config.OperationDayProps +import com.gzzn.omms.msgexchange.domain.DecodedMessage +import com.gzzn.omms.msgexchange.domain.EventType +import com.gzzn.omms.msgexchange.domain.MsgEvent +import com.gzzn.omms.msgexchange.domain.OperationDayCalculator +import com.gzzn.omms.msgexchange.domain.ProcState +import com.gzzn.omms.msgexchange.domain.Targets +import com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot +import com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine +import com.gzzn.omms.msgexchange.domain.flight.MergeChange +import com.gzzn.omms.msgexchange.infra.persistence.BackfillTodoRepository +import com.gzzn.omms.msgexchange.infra.persistence.FlightStateRepository +import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository +import com.gzzn.omms.msgexchange.infra.persistence.PipelineLockRepository +import com.gzzn.omms.msgexchange.infra.persistence.PipelineTransactionManager +import jakarta.inject.Singleton +import java.time.Instant +import java.time.ZoneId + +/** + * FLOP（§6.1）：读取完整当前态 → 合并变化 → 保留运营日 → STATE_VERSION+1 → + * 同事务登记 KAFKA_MSG / KAFKA_SCHD 与处理终态。 + */ +@Singleton +class FlopProcessor( + private val txManager: PipelineTransactionManager, + private val lock: PipelineLockRepository, + private val flightState: FlightStateRepository, + private val msgEvents: MsgEventRepository, + private val backfillTodo: BackfillTodoRepository?, + private val mapper: ObjectMapper, +) { + fun apply(head: ProcState, msg: DecodedMessage, payload: FlopPayload): ApplyResult = txManager.inTransaction { + lock.lock() + val current = flightState.loadFullSnapshot(payload.flid) + ?: return@inTransaction idempotentAbsent(head, msg) // 迟到/未知航班：幂等成功，不创建 + + val change = MergeChange(flid = payload.flid, scalars = payload.scalars, collections = payload.collections) + val next = FlightStateEngine.mergedState(current, change) + flightState.persistFullState(next, msgId = head.msgId, now = Instant.now()) + msgEvents.insertAll(eventsFor(next, mapper)) + preRegisterBackfill(head, msg, backfillTodo) + ApplyResult.Succeeded + } +} + +/** + * FDEL（§6.2）：ACTIVE → 置 DELETED、推进版本、明细保留、发布 tombstone； + * 已 DELETED / 不存在 → 幂等成功，不推进版本、不重复发布。 + */ +@Singleton +class FdelProcessor( + private val txManager: PipelineTransactionManager, + private val lock: PipelineLockRepository, + private val flightState: FlightStateRepository, + private val msgEvents: MsgEventRepository, + private val backfillTodo: BackfillTodoRepository?, + private val mapper: ObjectMapper, +) { + fun apply(head: ProcState, msg: DecodedMessage, payload: FlopPayload): ApplyResult = txManager.inTransaction { + lock.lock() + val deleted = flightState.markDeleted(payload.flid, msgId = head.msgId, now = Instant.now()) + if (deleted) { + val current = flightState.loadFullSnapshot(payload.flid) + // tombstone 仅在 ACTIVE→DELETED 时登记（§7.3），与删除同事务 + msgEvents.insertAll( + listOf( + MsgEvent( + target = Targets.KAFKA_SCHD, + partitionKey = payload.flid, + eventType = EventType.TOMBSTONE, + stateVersion = current?.stateVersion ?: 0L, + payloadJson = mapper.writeValueAsString( + mapOf( + "flid" to payload.flid, + "stateVersion" to (current?.stateVersion ?: 0L), + "deleted" to true, + ), + ), + ), + MsgEvent( + target = Targets.KAFKA_MSG, + partitionKey = payload.flid, + stateVersion = current?.stateVersion ?: 0L, + payloadJson = mapper.writeValueAsString( + mapOf("flid" to payload.flid, "stateVersion" to (current?.stateVersion ?: 0L), "deleted" to true), + ), + ), + ), + ) + preRegisterBackfill(head, msg, backfillTodo) + } + ApplyResult.Succeeded // 未命中 = 迟到/重复，幂等成功（§6.2 步骤 3/4） + } +} + +/** + * ADFT（§6.3 + §2.1）：字段缺失语义待确认——确认前按保守 Set-only 处理 + * （出现字段覆盖，缺失不 Clear，不沿用 FLOP 全量合并规则）。 + * FLID 已存在且 DELETED → 生命周期重激活；不存在 → 新实例建立（含运营日计算 §3.5）。 + */ +@Singleton +class AdftProcessor( + private val txManager: PipelineTransactionManager, + private val lock: PipelineLockRepository, + private val flightState: FlightStateRepository, + private val msgEvents: MsgEventRepository, + private val backfillTodo: BackfillTodoRepository?, + operationDayProps: OperationDayProps, + private val mapper: ObjectMapper, +) { + private val opDay = OperationDayCalculator( + zone = runCatching { ZoneId.of(operationDayProps.zone) }.getOrElse { ZoneId.of("Asia/Shanghai") }, + cutoffHour = operationDayProps.cutoffHour, + ) + + fun apply(head: ProcState, msg: DecodedMessage, record: com.gzzn.omms.msgexchange.domain.flight.ScheduleRecord): ApplyResult = + txManager.inTransaction { + lock.lock() + val main = flightState.findMainRow(record.flid) + if (main != null && main.state == com.gzzn.omms.msgexchange.domain.flight.FlightState.DELETED) { + // §6.3 重激活：DELETED → ACTIVE，推进版本，登记状态事件 + if (flightState.revive(record.flid, msgId = head.msgId, now = Instant.now())) { + val current = flightState.loadFullSnapshot(record.flid) + if (current != null) { + val next = FlightStateEngine.mergedState(current, setOnly(record)) + flightState.persistFullState(next, msgId = head.msgId, now = Instant.now()) + msgEvents.insertAll(eventsFor(next, mapper)) + } + } + preRegisterBackfill(head, msg, backfillTodo) + return@inTransaction ApplyResult.Succeeded + } + + val current = flightState.loadFullSnapshot(record.flid) + val next: FlightSnapshot = if (current == null) { + // 新实例建立：ADFT 含 SODT 时直接计算运营日（§2.1），不可算则置 null 待快照收录 + val day = opDay.compute(record.scalars["SODT"]) + FlightSnapshot( + flid = record.flid, + operationDay = day, // 待确认项 §2.1：不可算时不得默认写接收日 + state = com.gzzn.omms.msgexchange.domain.flight.FlightState.ACTIVE, + stateVersion = 1L, + scalars = record.scalars, + collections = com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine.COLLECTION_KEYS.associateWith { key -> + record.collections[key] ?: emptyList() + }, + ) + } else { + FlightStateEngine.mergedState(current, setOnly(record)) + } + val outcome = flightState.persistFullState(next, msgId = head.msgId, now = Instant.now()) + check(outcome != com.gzzn.omms.msgexchange.infra.persistence.PersistOutcome.DAY_GUARD_VIOLATION) { + "operation-day guard violated flid=${record.flid}" + } + msgEvents.insertAll(eventsFor(next, mapper)) + preRegisterBackfill(head, msg, backfillTodo) + ApplyResult.Succeeded + } + + /** §2.1 保守语义：仅出现字段 Set；集合出现 Replace、缺失保留。 */ + private fun setOnly(record: com.gzzn.omms.msgexchange.domain.flight.ScheduleRecord) = MergeChange( + flid = record.flid, + scalars = record.scalars, + collections = record.collections, + ) +} + +// ===================================================================== +// 共享小工具（处理器层私有约定） +// ===================================================================== + +/** 航班不存在/迟到：幂等成功（§9 队头不阻塞；不创建实例——创建入口只有 SCHD/ADFT）。 */ +private fun idempotentAbsent(head: ProcState, msg: DecodedMessage): ApplyResult = ApplyResult.Succeeded + +/** KAFKA_SCHD 整态 + KAFKA_MSG 变化通知（§7.3）。 */ +internal fun eventsFor(next: FlightSnapshot, mapper: ObjectMapper): List { + val payload = linkedMapOf( + "flid" to next.flid, + "stateVersion" to next.stateVersion, + "scalars" to next.scalars, + "collections" to next.collections, + ) + return listOf( + MsgEvent( + target = Targets.KAFKA_SCHD, + partitionKey = next.flid, + stateVersion = next.stateVersion, + payloadJson = mapper.writeValueAsString(payload), + ), + MsgEvent( + target = Targets.KAFKA_MSG, + partitionKey = next.flid, + stateVersion = next.stateVersion, + payloadJson = mapper.writeValueAsString(mapOf("flid" to next.flid, "stateVersion" to next.stateVersion)), + ), + ) +} + +/** §7.2 业务事务内预登记回填待办（与终态同事务）。 */ +internal fun preRegisterBackfill(head: ProcState, msg: DecodedMessage, backfillTodo: BackfillTodoRepository?) { + backfillTodo?.record( + BackfillTodoRepository.BackfillTask( + msgId = head.msgId, sndr = msg.meta.sndr, type = msg.meta.type, + styp = msg.meta.styp, seqn = msg.meta.seqn, + ), + null, + ) +} + diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/processing/Handler.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/processing/Handler.kt deleted file mode 100644 index 7abe904..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/processing/Handler.kt +++ /dev/null @@ -1,37 +0,0 @@ -package com.gzzn.omms.msgexchange.processing - -import com.gzzn.omms.msgexchange.domain.DecodedMessage -import com.gzzn.omms.msgexchange.domain.Decision -import com.gzzn.omms.msgexchange.domain.MsgKind - -/** - * ACMA-8 流程 2：Handler = 纯函数（状态与报文进，Decision 出，不碰 Redis/Kafka）。 - * 32 个 Handler（flop 29 + schd 3）翻译属阶段 2/3，逐条对照 ACMA-4 基线与兼容矩阵 KEEP/FIX。 - */ -interface Handler { - val kind: MsgKind - - /** - * 在线状态以只读视图传入：FLID → 字段集（field → value，与 legacy flightInfo hash / - * hgetAllFlightInfo 同构；阶段 A 由调用方点查自有库 FLIGHT_SCHD 宽表装配，ACM2-28 定案）。 - */ - fun decide(flightView: Map>, msg: DecodedMessage): Decision -} - -/** - * sealed 穷尽分派（ACMA-6 选型：取代 legacy 反射 get{TYPE}()）。 - */ -class HandlerRegistry(handlers: List) { - private val byKind: Map = handlers.associateBy { keyOf(it.kind) } - - fun dispatcherFor(msg: DecodedMessage): Handler? = byKind[keyOf(msg.kind)] - - companion object { - fun keyOf(kind: MsgKind): String = when (kind) { - is MsgKind.Schd -> "SCHD-${kind.subtype.name}" - is MsgKind.Flop -> "FLOP-${kind.subtype}" - } - } -} - -// TODO(阶段2/3): 注册 3+29 Handler；本阶段仅含骨架（阶段 2 最小纵向链路先做 1 SCHD(DNLD) + 1 FLOP）。 diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/processing/Pump.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/processing/Pump.kt index 70eca56..9cf2ee7 100644 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/processing/Pump.kt +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/processing/Pump.kt @@ -1,37 +1,32 @@ package com.gzzn.omms.msgexchange.processing +import com.gzzn.omms.msgexchange.codec.FlopPayload +import com.gzzn.omms.msgexchange.codec.JacksonXmlCodec +import com.gzzn.omms.msgexchange.codec.ScheduleBody import com.gzzn.omms.msgexchange.config.PipelineProps import com.gzzn.omms.msgexchange.domain.DecodedMessage -import com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine import com.gzzn.omms.msgexchange.domain.ErrorClass -import com.gzzn.omms.msgexchange.domain.MsgEvent +import com.gzzn.omms.msgexchange.domain.MsgKind import com.gzzn.omms.msgexchange.domain.ProcState import com.gzzn.omms.msgexchange.domain.ProcStatus -import com.gzzn.omms.msgexchange.domain.Targets +import com.gzzn.omms.msgexchange.infra.log.TraceLog import com.gzzn.omms.msgexchange.infra.persistence.BackfillTodoRepository import com.gzzn.omms.msgexchange.infra.persistence.CminmsgInboxRepository -import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository import com.gzzn.omms.msgexchange.infra.persistence.ProcStateRepository -import com.gzzn.omms.msgexchange.infra.persistence.PumpJobRepository -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository -import com.gzzn.omms.msgexchange.infra.persistence.PipelineTransactionManager import com.gzzn.omms.msgexchange.infra.retry.ProcFailure -import com.gzzn.omms.msgexchange.jobs.JobExecutor import jakarta.inject.Singleton import java.time.Duration import java.time.Instant /** - * ACMA-8 流程 2：处理主泵——严格 FIFO + HOL + 毒丸 DEAD 升级（I1）； - * 阶段 A 运营航班 FLIGHT_SCHD 与待发事件、终态同事务原子提交（I2/I5，ACM2-28 定案）。 + * 处理主泵（docs/flight-state.md §1 目标 2）：单活动主泵严格 FIFO + HOL + 毒丸 DEAD 升级； + * 航班状态、事件、处理终态在处理器事务内原子提交（§1 目标 3）。 */ @Singleton class Pump( private val procState: ProcStateRepository, - private val pumpJobs: PumpJobRepository, private val inbox: CminmsgInboxRepository, private val processor: MessageProcessor, - private val jobExecutor: JobExecutor, private val props: PipelineProps, ) { private val log = org.slf4j.LoggerFactory.getLogger(Pump::class.java) @@ -39,7 +34,7 @@ class Pump( @Volatile private var running = true - /** 优雅停机：loop 在当前 tick 收尾后退出；线程中断由 Runner 负责。 */ + /** 优雅停机：loop 在当前 tick 收尾后退出；线程中断由 PipelineLifecycle 负责。 */ fun stop() { running = false } @@ -52,27 +47,23 @@ class Pump( Thread.currentThread().interrupt() return } catch (e: Exception) { - // U08：loop 只作最后防线——失败状态迁移已在 processOne/execute 的边界内完成； - // 致命 Error 不在此捕获（任其终止进程，保证“异常必可见”）。 + // 最后防线：失败状态迁移已在 processOne 边界内完成；致命 Error 不捕获 sleepQuietly(props.pipeline.pollInterval) } } } internal fun tick() { - val job = pumpJobs.headQueued() val head = procState.headUnfinished() - - // 统一 FIFO：JOB 与消息同队列语义（定时任务产物不绕过队头顺序，决策 1）。 - val nextJob = job?.takeIf { jobBefore(job, head) } when { - nextJob != null -> execute(nextJob) head == null -> sleepQuietly(props.pipeline.pollInterval) head.state == ProcStatus.FAILED && (head.nextAttemptAt ?: Instant.EPOCH) > Instant.now() -> if (poisoned(head)) { - log.error("poison -> DEAD id={} attempts={} lastError={}", head.cminmsgsId, head.attempts, head.lastError) - procState.update(head.cminmsgsId, ProcStatus.DEAD, - errorClass = ErrorClass.EXHAUSTED, lastError = head.lastError ?: "head-deadline-exceeded") + log.error("poison -> DEAD msgId={} attempts={} lastError={}", head.msgId, head.attempts, head.lastError) + procState.update( + head.msgId, ProcStatus.DEAD, + errorClass = ErrorClass.EXHAUSTED, lastError = head.lastError ?: "head-deadline-exceeded", + ) } else { sleepQuietly(Duration.between(Instant.now(), head.nextAttemptAt)) } @@ -85,69 +76,41 @@ class Pump( head.attempts >= props.pipeline.maxAttempts || Duration.between(head.updatedAt, Instant.now()) > props.pipeline.headDeadline - /** JOB 与队头消息的先后由入队时间近似；实装以统一序号列保证（阶段 1 后续）。 */ - private fun jobBefore(job: PumpJobRepository.Job, head: ProcState?) = - head == null || head.state == ProcStatus.FAILED - - private fun execute(job: PumpJobRepository.Job) { - pumpJobs.markRunning(job.jobId) - try { - jobExecutor.execute(job) - pumpJobs.markDone(job.jobId) - } catch (e: Exception) { - pumpJobs.markFailed(job.jobId, e.message ?: "unknown") - } - } - private fun sleepQuietly(d: Duration) { if (!d.isNegative && !d.isZero) Thread.sleep(d.toMillis().coerceAtLeast(1)) } } /** - * ACMA-8 流程 2 processOne：解码→绑定→纯函数决策→提交。 - * U08/U10/U11：失败状态迁移全部在“持有具体 head”的边界内完成—— - * 任何意外异常 → FAILED(INFRA)+退避（ProcFailure）；attempts 达上限 → DEAD(EXHAUSTED)； - * MALFORMED 直接 DEAD；InterruptedException 恢复中断位并上抛（不被普通恢复逻辑吞掉）。 + * processOne：解码 → 绑定 → 处理器（事务内决策+落库）→ 终态迁移 → 回填。 + * 边界化失败迁移（ProcFailure）：任何意外异常归于本条 head，FAILED(INFRA)+退避，不穿出杀泵； + * MALFORMED / PROTOCOL 直接 DEAD 不重试（§9）。 */ @Singleton class MessageProcessor( private val inbox: CminmsgInboxRepository, private val procState: ProcStateRepository, - private val msgEvents: MsgEventRepository, - private val codecHolder: CodecHolder, - private val handlers: HandlerHolder, - private val flightSchd: FlightSchdRepository, - private val snapshotFlow: SnapshotFlow, + private val codec: JacksonXmlCodec, + private val scheduleProcessor: ScheduleProcessor, + private val flopProcessor: FlopProcessor, + private val fdelProcessor: FdelProcessor, + private val adftProcessor: AdftProcessor, private val procFailure: ProcFailure, private val props: PipelineProps, - private val txManager: PipelineTransactionManager, private val backfillTodo: BackfillTodoRepository? = null, ) { private val log = org.slf4j.LoggerFactory.getLogger(MessageProcessor::class.java) - private val flidRegex = Regex("(.*?)", RegexOption.IGNORE_CASE) - - private fun extractCandidateFlids(decoded: DecodedMessage): Set { - val fromXml = flidRegex.findAll(decoded.rawXml).map { it.groupValues[1].trim() }.filter { it.isNotEmpty() }.toSet() - if (fromXml.isNotEmpty()) return fromXml - val b = decoded.body - if (b is Map<*, *>) { - val flid = b["FLID"] ?: b["flid"] - if (flid != null) return setOf(flid.toString()) - } - return emptySet() - } fun processOne(head: ProcState) { - com.gzzn.omms.msgexchange.infra.log.TraceLog.withTrace(head.cminmsgsId) { + TraceLog.withTrace(head.msgId) { try { processInternal(head) } catch (e: InterruptedException) { Thread.currentThread().interrupt() throw e } catch (e: Exception) { - // U08：边界化——异常归于本条 head，写 FAILED/DEAD，而不是穿出杀 pump - log.warn("processOne unexpected failure id={} ec=INFRA msg={}", head.cminmsgsId, e.message ?: e.javaClass.simpleName) + // 边界化：异常归于本条 head，写 FAILED(INFRA)/DEAD，而不是穿出杀 pump + log.warn("processOne unexpected failure msgId={} ec=INFRA msg={}", head.msgId, e.message ?: e.javaClass.simpleName) procFailure.fail(head, ErrorClass.INFRA, e.message ?: e.javaClass.simpleName) } } @@ -156,126 +119,134 @@ class MessageProcessor( private fun processInternal(head: ProcState) { // 守卫：手工/遗留 FAILED 行若 attempts 已达上限，直接终态（防止退避到期后无限重试） if (head.state == ProcStatus.FAILED && procFailure.scheduler.exhausted(head.attempts)) { - log.error("head exhausted at entry -> DEAD id={} attempts={}", head.cminmsgsId, head.attempts) - procState.update(head.cminmsgsId, ProcStatus.DEAD, errorClass = ErrorClass.EXHAUSTED, - lastError = head.lastError ?: "max-attempts") + log.error("head exhausted at entry -> DEAD msgId={} attempts={}", head.msgId, head.attempts) + procState.update(head.msgId, ProcStatus.DEAD, errorClass = ErrorClass.EXHAUSTED, lastError = head.lastError ?: "max-attempts") return } - val raw = inbox.rawOf(head.cminmsgsId) ?: run { - log.error("raw missing -> DEAD(MALFORMED) id={}", head.cminmsgsId) - procState.update(head.cminmsgsId, ProcStatus.DEAD, errorClass = ErrorClass.MALFORMED, lastError = "raw-missing") + val raw = inbox.rawOf(head.msgId) ?: run { + log.error("raw missing -> DEAD(MALFORMED) msgId={}", head.msgId) + procState.update(head.msgId, ProcStatus.DEAD, errorClass = ErrorClass.MALFORMED, lastError = "raw-missing") return } - val decoded = when (val r = codecHolder.codec.decode(raw)) { + val decoded = when (val r = codec.decode(raw)) { is com.gzzn.omms.msgexchange.codec.DecodeResult.Ok -> r.message is com.gzzn.omms.msgexchange.codec.DecodeResult.Err -> { - // T06（U11）：MALFORMED（报文非法）→ DEAD 不重试；CODEC_ERROR（可随 codec 修复重放）→ FAILED 退避 + // MALFORMED（报文非法）→ DEAD 不重试；CODEC_ERROR（可随 codec 修复重放）→ FAILED 退避 if (r.failure.errorClass == ErrorClass.MALFORMED) { - log.error("decode MALFORMED -> DEAD id={} detail={}", head.cminmsgsId, r.failure.detail) - procState.update(head.cminmsgsId, ProcStatus.DEAD, errorClass = ErrorClass.MALFORMED, - lastError = r.failure.detail) + log.error("decode MALFORMED -> DEAD msgId={} detail={}", head.msgId, r.failure.detail) + procState.update(head.msgId, ProcStatus.DEAD, errorClass = ErrorClass.MALFORMED, lastError = r.failure.detail) } else { - log.warn("decode {} -> FAILED id={} detail={}", r.failure.errorClass, head.cminmsgsId, r.failure.detail) + log.warn("decode {} -> FAILED msgId={} detail={}", r.failure.errorClass, head.msgId, r.failure.detail) procFailure.fail(head, r.failure.errorClass, r.failure.detail) } return } } - // I3：identity 仅首次绑定（head.identityKey == null）；FAILED 重试不重绑（N25：入参传递 head，不再二次查询） + // I3：identity 仅首次绑定（head.identityKey == null）；FAILED 重试不重绑 if (head.identityKey == null) { val identity = Identity.of(decoded, props.identity) - if (!procState.tryBindIdentity(head.cminmsgsId, identity)) { + if (!procState.tryBindIdentity(head.msgId, identity)) { val owner = procState.ownerOfIdentity(identity) ?: -1L - log.info("duplicate-of:{} -> SKIPPED id={}", owner, head.cminmsgsId) - procState.update(head.cminmsgsId, ProcStatus.SKIPPED, lastError = "duplicate-of:$owner") - // 重复报文自身也是一条信箱记录：同样回填共享信箱，防止被反复轮询（失败落补偿） + log.info("duplicate-of:{} -> SKIPPED msgId={}", owner, head.msgId) + procState.update(head.msgId, ProcStatus.SKIPPED, lastError = "duplicate-of:$owner") compensateBackfill(head, decoded) return } } - // 快照消息走专属流程（流程 4：staging → SQL 批处理/域内差删/CAS 同事务） - val handler = handlers.registry.dispatcherFor(decoded) - if (handler == null) { - // U10（N21）：未注册 ≠ 报文非法——写 FAILED（可重放），绝不写终态 - log.warn("no-handler:{} -> FAILED(UNSUPPORTED) id={}", decoded.typeTag, head.cminmsgsId) - procFailure.fail(head, ErrorClass.UNSUPPORTED, "no-handler:${decoded.typeTag}") - return - } - val schdKind = decoded.kind as? com.gzzn.omms.msgexchange.domain.MsgKind.Schd - if (schdKind != null && - schdKind.subtype == com.gzzn.omms.msgexchange.domain.MsgKind.SchdSubtype.DNLD - ) { - snapshotFlow.publishSnapshot(head, decoded) - return - } - val candidateFlids = extractCandidateFlids(decoded) - val flightView = if (candidateFlids.isNotEmpty()) { - flightSchd.findByFlids(candidateFlids) - } else { - emptyMap() - } - - val decision = handler.decide(flightView, decoded) // 纯函数 - - // 事务 2（自有 PG 单事务原子）： - // upsert FLIGHT_SCHD(decision.flightChanges) + insert MSG_EVENT + PROC_STATE → SUCCEEDED - val events = buildList { - decision.msgNotifies.forEach { add(MsgEvent(target = Targets.KAFKA_MSG, payloadJson = it.payloadJson)) } - decision.schdPush.forEach { add(MsgEvent(target = Targets.KAFKA_SCHD, partitionKey = it.flid, payloadJson = it.payloadJson)) } - } - - val messageId = head.cminmsgsId.toString() - txManager.inTransaction { - if (decision.flightChanges.isNotEmpty()) { - val nextStates = decision.flightChanges.map { change -> - val current = flightSchd.findNextStateByFlid(change.flid) - val commands = FlightStateEngine.commandsFromFields(change.flid, change.fields) - FlightStateEngine.apply(current, commands, messageId, bumpVersion = true) + // 处理器分派：SCHD 快照主链路（§5.1）/ FLOP / FDEL / ADFT；缺载荷按 MALFORMED 终态 + val result: ApplyResult = when (val kind = decoded.kind) { + is MsgKind.Schd -> { + val body = decoded.body as? ScheduleBody + if (body == null) { + deadMalformed(head, "missing-schd-body") + return + } + when (kind.subtype) { + MsgKind.SchdSubtype.DNLD, MsgKind.SchdSubtype.RESP -> + scheduleProcessor.applyScheduleRecords(head, decoded) + MsgKind.SchdSubtype.ADFT -> { + val record = body.records.singleOrNull() + if (record == null) { + deadMalformed(head, "adft-needs-single-fltr") + return + } + adftProcessor.apply(head, decoded, record) + } } - flightSchd.persistNextStates(null, nextStates, snapshotReplace = false) } - if (events.isNotEmpty()) { - msgEvents.insertAll(events) + MsgKind.Fdel -> { + val payload = decoded.body as? FlopPayload + if (payload == null) { + deadMalformed(head, "missing-fdel-flid") + return + } + fdelProcessor.apply(head, decoded, payload) + } + is MsgKind.Flop -> { + val payload = decoded.body as? FlopPayload + if (payload == null) { + deadMalformed(head, "missing-flop-body") + return + } + flopProcessor.apply(head, decoded, payload) + } + is MsgKind.Unsupported -> { + // §9：未支持类型 → FAILED(UNSUPPORTED) 退避重试，达阈值转 DEAD；绝不写终态 + log.warn("unsupported type -> FAILED(UNSUPPORTED) msgId={} tag={}", head.msgId, kind.tag) + procFailure.fail(head, ErrorClass.UNSUPPORTED, "no-handler:${kind.tag}") + return } - procState.update(head.cminmsgsId, ProcStatus.SUCCEEDED) } - try { - inbox.backfillOnSuccess(head.cminmsgsId, decoded.meta.sndr, decoded.meta.type, decoded.meta.styp, decoded.meta.seqn) - } catch (e: Exception) { - log.error("backfill failed after SUCCEEDED id={} (compensation required)", head.cminmsgsId, e) - recordBackfillTodo(head.cminmsgsId, decoded, e) + when (result) { + is ApplyResult.Succeeded, ApplyResult.ReplaySkipped -> + procState.update(head.msgId, ProcStatus.SUCCEEDED) + is ApplyResult.DeadProtocol -> { + // §9：整包拒绝 DEAD(PROTOCOL)，立即释放队头，交人工确认 + log.error("DEAD(PROTOCOL) msgId={} reason={} flags={}", head.msgId, result.reason, result.flags) + procState.update(head.msgId, ProcStatus.DEAD, errorClass = ErrorClass.PROTOCOL, lastError = result.reason.take(1000)) + compensateBackfill(head, decoded) // 拒绝包同样要回填信箱，防止反复轮询 + return + } } - log.info("SUCCEEDED id={} events={} flightChanges={}", head.cminmsgsId, events.size, decision.flightChanges.size) + + backfill(head, decoded) + log.info("SUCCEEDED msgId={} kind={}", head.msgId, decoded.typeTag) } - /** v2 §5：回填失败只落补偿待办（成功终态不降级，业务不重放）；由 BackfillSweepJob 到期重试。 */ + /** §7.2：提交后回填共享信箱；失败不得把 SUCCEEDED 改回 FAILED，待办已事务内预登记。 */ + private fun backfill(head: ProcState, decoded: DecodedMessage) { + try { + inbox.backfillOnSuccess(head.msgId, decoded.meta.sndr, decoded.meta.type, decoded.meta.styp, decoded.meta.seqn) + backfillTodo?.delete(head.msgId) + } catch (e: Exception) { + log.error("backfill failed after SUCCEEDED msgId={} (todo pre-registered, sweep will retry)", head.msgId, e) + } + } + + /** SKIPPED/DEAD(PROTOCOL) 包的回填（无预登记待办）：失败落补偿待办。 */ private fun compensateBackfill(head: ProcState, decoded: DecodedMessage) { try { - inbox.backfillOnSuccess(head.cminmsgsId, decoded.meta.sndr, decoded.meta.type, decoded.meta.styp, decoded.meta.seqn) + inbox.backfillOnSuccess(head.msgId, decoded.meta.sndr, decoded.meta.type, decoded.meta.styp, decoded.meta.seqn) } catch (e: Exception) { - log.error("backfill failed for SKIPPED duplicate id={} (compensation required)", head.cminmsgsId, e) - recordBackfillTodo(head.cminmsgsId, decoded, e) + log.error("backfill failed msgId={} (compensation required)", head.msgId, e) + backfillTodo?.record( + BackfillTodoRepository.BackfillTask( + msgId = head.msgId, + sndr = decoded.meta.sndr, + type = decoded.meta.type, + styp = decoded.meta.styp, + seqn = decoded.meta.seqn, + ), + e.message ?: e.javaClass.simpleName, + ) ?: log.warn("no backfill-todo repository bound; compensation NOT persisted msgId={}", head.msgId) } } - private fun recordBackfillTodo(cminmsgsId: Long, decoded: DecodedMessage, e: Exception) { - backfillTodo?.record( - BackfillTodoRepository.BackfillTask( - cminmsgsId = cminmsgsId, - sndr = decoded.meta.sndr, - type = decoded.meta.type, - styp = decoded.meta.styp, - seqn = decoded.meta.seqn, - ), - e.message ?: e.javaClass.simpleName, - ) ?: log.warn("no backfill-todo repository bound; compensation NOT persisted id={}", cminmsgsId) + private fun deadMalformed(head: ProcState, detail: String) { + procState.update(head.msgId, ProcStatus.DEAD, errorClass = ErrorClass.MALFORMED, lastError = detail) } } - -/** 延迟装配占位（阶段 1 后续以 Micronaut Bean 替换直连构造）。 */ -class CodecHolder(val codec: com.gzzn.omms.msgexchange.codec.XmlCodec) -class HandlerHolder(val registry: HandlerRegistry) diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/processing/ScheduleProcessor.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/processing/ScheduleProcessor.kt new file mode 100644 index 0000000..3c1b911 --- /dev/null +++ b/src/main/kotlin/com/gzzn/omms/msgexchange/processing/ScheduleProcessor.kt @@ -0,0 +1,207 @@ +package com.gzzn.omms.msgexchange.processing + +import com.fasterxml.jackson.databind.ObjectMapper +import com.gzzn.omms.msgexchange.codec.ScheduleBody +import com.gzzn.omms.msgexchange.config.OperationDayProps +import com.gzzn.omms.msgexchange.domain.DecodedMessage +import com.gzzn.omms.msgexchange.domain.MsgEvent +import com.gzzn.omms.msgexchange.domain.ProcState +import com.gzzn.omms.msgexchange.domain.ProcStatus +import com.gzzn.omms.msgexchange.domain.SnapshotFlag +import com.gzzn.omms.msgexchange.domain.SnapshotLogEntry +import com.gzzn.omms.msgexchange.domain.SnapshotResult +import com.gzzn.omms.msgexchange.domain.Targets +import com.gzzn.omms.msgexchange.domain.OperationDayCalculator +import com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot +import com.gzzn.omms.msgexchange.domain.flight.FlightState +import com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine +import com.gzzn.omms.msgexchange.domain.flight.ScheduleRecord +import com.gzzn.omms.msgexchange.domain.flight.SnapshotValidation +import com.gzzn.omms.msgexchange.infra.persistence.BackfillTodoRepository +import com.gzzn.omms.msgexchange.infra.persistence.FlightStateRepository +import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository +import com.gzzn.omms.msgexchange.infra.persistence.PersistOutcome +import com.gzzn.omms.msgexchange.infra.persistence.PipelineLockRepository +import com.gzzn.omms.msgexchange.infra.persistence.PipelineTransactionManager +import com.gzzn.omms.msgexchange.infra.persistence.ProcStateRepository +import com.gzzn.omms.msgexchange.infra.persistence.SnapshotLogRepository +import jakarta.inject.Singleton +import java.time.Instant +import java.time.LocalDate +import java.time.ZoneId + +/** 处理器执行结果——终态迁移由 MessageProcessor 统一落库。 */ +sealed interface ApplyResult { + /** 业务成功（含幂等成功）。 */ + data object Succeeded : ApplyResult + + /** §5.1 步骤 2：MSG_ID 已有成功终态 → 重放，直接记幂等成功。 */ + data object ReplaySkipped : ApplyResult + + /** §9：整包拒绝 DEAD(PROTOCOL)，不重试，交人工确认。 */ + data class DeadProtocol(val reason: String, val flags: Set = emptySet()) : ApplyResult +} + +/** §5.3 第四行：归属日不符 = 串日/错发/污染，整包拒绝。 */ +class ProtocolViolation(message: String) : RuntimeException(message) + +/** + * SCHD 快照主链路（docs/flight-state.md §5.1 applyScheduleRecords，同一事务）： + * 对单日快照与滚动窗口统一适用，不做名单层面的处理。 + */ +@Singleton +class ScheduleProcessor( + private val txManager: PipelineTransactionManager, + private val lock: PipelineLockRepository, + private val procState: ProcStateRepository, + private val flightState: FlightStateRepository, + private val msgEvents: MsgEventRepository, + private val snapshotLog: SnapshotLogRepository, + private val backfillTodo: BackfillTodoRepository?, + operationDayProps: OperationDayProps, + private val mapper: ObjectMapper, +) { + private val opDay = OperationDayCalculator( + zone = runCatching { ZoneId.of(operationDayProps.zone) }.getOrElse { ZoneId.of("Asia/Shanghai") }, + cutoffHour = operationDayProps.cutoffHour, + ) + + fun applyScheduleRecords(head: ProcState, msg: DecodedMessage): ApplyResult { + val body = msg.body as? ScheduleBody ?: return ApplyResult.DeadProtocol("missing-schd-body") + val started = System.nanoTime() + + // ② 重放判定：MSG_ID 已有成功终态 → 幂等成功（§5.1 步骤 2，重放也记留痕） + if (procState.findSuccessTerminal(head.msgId)) { + logSnapshot(head, body, SnapshotResult.REPLAY_SKIPPED, upserted = 0, flags = emptySet(), started) + return ApplyResult.ReplaySkipped + } + + // ③ 报文完整性（§5.2 五项）：任一失败整包不落地 → DEAD(PROTOCOL) + val validation = FlightStateEngine.validateMessage( + recsDeclared = body.recsDeclared, + records = body.records, + scopeStart = body.scopeStart, + scopeEnd = body.scopeEnd, + opDay = opDay, + ) + if (validation is SnapshotValidation.Invalid) { + logSnapshot(head, body, SnapshotResult.ROLLED_BACK, upserted = 0, validation.flags, started) + return ApplyResult.DeadProtocol(validation.reason, validation.flags) + } + val ok = validation as SnapshotValidation.Ok + + if (ok.perRecordDay.isEmpty()) { + // 空快照：合法但无写入（§5.5 EMPTY），仍算成功终态 + logSnapshot(head, body, SnapshotResult.COMMITTED, upserted = 0, setOf(SnapshotFlag.EMPTY), started) + return ApplyResult.Succeeded + } + + val flags = linkedSetOf() + return try { + // ①④⑤⑥⑦ 同一事务：锁 → 归属校验 → upsert → 版本/事件/待办预登记/终态 + val upserted = txManager.inTransaction { + lock.lock() + + // ⑤ 归属校验（§5.3）：OPERATION_DAY 不可变，批量点查避免逐航班往返 + val mains = flightState.findMainRows(ok.perRecordDay.keys) + ok.perRecordDay.forEach { (flid, day) -> + val existing = mains[flid] ?: return@forEach + if (existing.operationDay != null && existing.operationDay != day) { + throw ProtocolViolation( + "SAME_FLID_ACROSS_OPERATION_DAYS flid=$flid existing=${existing.operationDay} incoming=$day", + ) + } + } + + var written = 0 + val events = mutableListOf() + ok.perRecordDay.forEach { (flid, day) -> + val record = body.records.first { it.flid == flid } + val existingMain = mains[flid] + val keepDeleted = existingMain?.state == FlightState.DELETED + if (keepDeleted) flags.add(SnapshotFlag.SCHD_REVIVE_CONFLICT) // §5.1 步骤 6：不恢复 + val current = if (existingMain != null) flightState.loadFullSnapshot(flid) else null + val next = FlightStateEngine.snapshotState( + current = current, + record = record, + operationDay = day, + keepDeleted = keepDeleted, + ) + when (flightState.persistFullState(next, msgId = head.msgId, now = Instant.now())) { + PersistOutcome.DAY_GUARD_VIOLATION -> + throw ProtocolViolation("operation-day guard violated flid=$flid") + else -> written++ + } + events += snapshotEvents(next) + } + if (events.isNotEmpty()) msgEvents.insertAll(events) + // §7.2 目标形态：业务事务内预登记回填待办（提交后由 MessageProcessor 回填并删待办） + backfillTodo?.record( + BackfillTodoRepository.BackfillTask( + msgId = head.msgId, sndr = msg.meta.sndr, type = msg.meta.type, + styp = msg.meta.styp, seqn = msg.meta.seqn, + ), + null, + ) + written + } + logSnapshot(head, body, SnapshotResult.COMMITTED, upserted, flags, started) + ApplyResult.Succeeded + } catch (e: ProtocolViolation) { + logSnapshot(head, body, SnapshotResult.ROLLED_BACK, upserted = 0, flags, started) + ApplyResult.DeadProtocol(e.message ?: "protocol-violation", flags) + } + } + + /** 状态事件：整态出站（§7.3 KAFKA_SCHD）+ 变化通知（KAFKA_MSG）。 */ + private fun snapshotEvents(next: FlightSnapshot): List { + // KAFKA_SCHD 整态：缺失集合输出空集 → 消费者删除旧值（§5.4/§7.3） + val payload = linkedMapOf( + "flid" to next.flid, + "stateVersion" to next.stateVersion, + "scalars" to next.scalars, + "collections" to next.collections, + ) + val notify = mapper.writeValueAsString(mapOf("flid" to next.flid, "stateVersion" to next.stateVersion)) + return listOf( + MsgEvent( + target = Targets.KAFKA_SCHD, + partitionKey = next.flid, + stateVersion = next.stateVersion, + payloadJson = mapper.writeValueAsString(payload), + ), + MsgEvent(target = Targets.KAFKA_MSG, partitionKey = next.flid, stateVersion = next.stateVersion, payloadJson = notify), + ) + } + + /** §5.5 留痕：事务外追加，失败只记 error 不阻塞；scope 按记录归属运营日推导。 */ + private fun logSnapshot( + head: ProcState, + body: ScheduleBody, + result: SnapshotResult, + upserted: Int, + flags: Set, + startedNanos: Long, + ) { + runCatching { + val days = body.records.mapNotNull { opDay.compute(it.scalars["SODT"]) } + snapshotLog.append( + SnapshotLogEntry( + msgId = head.msgId, + recvAt = Instant.now(), + scopeStart = days.minOrNull() ?: LocalDate.now(), + scopeEnd = days.maxOrNull() ?: LocalDate.now(), + recs = body.records.size, + upserted = upserted, + durationMs = (System.nanoTime() - startedNanos) / 1_000_000, + result = result, + flags = flags, + ), + ) + }.onFailure { log.error("snap-log write failed (metric only) msgId={}", head.msgId, it) } + } + + private companion object { + private val log = org.slf4j.LoggerFactory.getLogger(ScheduleProcessor::class.java) + } +} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/processing/SnapshotFlow.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/processing/SnapshotFlow.kt deleted file mode 100644 index e366a7e..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/processing/SnapshotFlow.kt +++ /dev/null @@ -1,175 +0,0 @@ -package com.gzzn.omms.msgexchange.processing - -import com.gzzn.omms.msgexchange.domain.DecodedMessage -import com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine -import com.gzzn.omms.msgexchange.domain.ErrorClass -import com.gzzn.omms.msgexchange.domain.ProcState -import com.gzzn.omms.msgexchange.domain.ProcStatus -import com.gzzn.omms.msgexchange.domain.MsgEvent -import com.gzzn.omms.msgexchange.domain.MsgKind -import com.gzzn.omms.msgexchange.domain.Targets -import com.gzzn.omms.msgexchange.infra.persistence.BackfillTodoRepository -import com.gzzn.omms.msgexchange.infra.persistence.CminmsgInboxRepository -import com.gzzn.omms.msgexchange.infra.persistence.FlightFields -import com.gzzn.omms.msgexchange.infra.persistence.FlightFieldsJson -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository -import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository -import com.gzzn.omms.msgexchange.infra.persistence.PipelineTransactionManager -import com.gzzn.omms.msgexchange.infra.persistence.ProcStateRepository -import com.gzzn.omms.msgexchange.infra.persistence.ReqTrackRepository -import com.gzzn.omms.msgexchange.infra.retry.ProcFailure -import jakarta.inject.Singleton - -/** - * staging（内存瞬态，崩溃从 raw 整包重放）→ PG 单事务原子“覆盖+按代差删+SQL CAS 推进+事件入队+SUCCEEDED”（I4/I5，ACM2-28 定案）。 - * U10/T07 占位安全化 + U08 统一失败迁移（ProcFailure）：staging 未实装 → FAILED(UNSUPPORTED)+退避 - * （可重放，绝不写终态）；CAS 冲突 → FAILED(INFRA)+退避；达上限统一 DEAD(EXHAUSTED)。 - */ -@Singleton -class SnapshotFlow( - private val procState: ProcStateRepository, - private val flightSchd: FlightSchdRepository, - private val msgEvents: MsgEventRepository, - private val reqTrack: ReqTrackRepository, - private val procFailure: ProcFailure, - private val txManager: PipelineTransactionManager, - private val inbox: CminmsgInboxRepository, - private val backfillTodo: BackfillTodoRepository? = null, -) { - private val log = org.slf4j.LoggerFactory.getLogger(SnapshotFlow::class.java) - - fun publishSnapshot(head: ProcState, msg: DecodedMessage) { - val messageId = head.cminmsgsId.toString() - val staged = StageResult.stagingOf(msg) - if (staged is StageResult.Invalid) { - log.warn("staging not implemented -> FAILED(UNSUPPORTED) id={} reason={}", head.cminmsgsId, staged.reason) - procFailure.fail(head, ErrorClass.UNSUPPORTED, staged.reason) - return - } - val ok = staged as StageResult.Ok - val normalized = ok.flights - val day = ok.day - - if (normalized.size > MAX_FLIGHTS_PER_SNAPSHOT) { - log.error("snapshot flights exceed limit -> DEAD(MALFORMED) id={} size={}", head.cminmsgsId, normalized.size) - procState.update(head.cminmsgsId, ProcStatus.DEAD, errorClass = ErrorClass.MALFORMED, - lastError = "flights-exceed-limit:${normalized.size}") - return - } - - // v2 §5 事务流程：锁外只做解析与整包校验；取得 PIPELINE_LOCK 后在锁内 - // 复核快照身份 → 读取当前代与当前航班态 → 计算 nextState → 写入 → 提交。 - var replayNoOp = false - try { - txManager.inTransaction { - // 锁内复核快照身份：同一消息已成功提交 → 重放短路（不加版本、不重复发事件） - val gen = flightSchd.getGen(day) - if (gen?.lastMessageId == messageId) { - procState.update(head.cminmsgsId, ProcStatus.SUCCEEDED) - replayNoOp = true - return@inTransaction - } - - val expected = gen?.version ?: 0L - val newFlids = normalized.map { it.first }.toSet() - val delFields = gen?.flids?.minus(newFlids) ?: emptySet() - val newVersion = expected + 1L - - // 锁内读取当前航班态，计算 nextState（单写者互斥下读到的一定是已提交最新态） - val nextStates = normalized.map { (flid, fields) -> - val current = flightSchd.findNextStateByFlid(flid) - val commands = FlightStateEngine.commandsFromFields(flid, fields) - FlightStateEngine.apply(current, commands, messageId, bumpVersion = true) - } - - flightSchd.persistNextStates(day, nextStates, snapshotReplace = true) - - if (delFields.isNotEmpty()) { - flightSchd.deleteDiffByDay(day, delFields) - } - - // 锁内 CAS：expected 即锁内刚读到的最新版本，仍失败属数据异常—— - // 一律回滚进入可重试 FAILED(INFRA)，绝不凭版本号推断“已经是我的提交”（v2 §5） - val casSuccess = flightSchd.putGenIfVersion( - day = day, - expected = expected, - newGen = FlightSchdRepository.GenMeta( - fday = day, - version = newVersion, - flids = newFlids, - lastMessageId = messageId, - ), - ) - if (!casSuccess) { - throw CasConflictException( - "gen-cas-conflict: expected=$expected msg=$messageId (lock-held CAS must not fail)", - ) - } - - val schdKind = msg.kind as? MsgKind.Schd - if (schdKind?.subtype == MsgKind.SchdSubtype.RESP) { - reqTrack.findOpenByKind("SCHD")?.let { req -> - reqTrack.markDone(req.reqId) - } - } - - val events = nextStates.map { state -> - MsgEvent( - target = Targets.KAFKA_SCHD, - partitionKey = state.flid, - payloadJson = FlightFieldsJson.toJson(state.toFlightFields()), - ) - } - if (events.isNotEmpty()) { - msgEvents.insertAll(events) - } - - procState.update(head.cminmsgsId, ProcStatus.SUCCEEDED) - } - - // 提交后补偿：重放短路同样补做回填（其待办可能仍在重试中） - safeBackfill(head, msg) - if (replayNoOp) { - log.info("snapshot replay no-op id={} day={}", head.cminmsgsId, day) - } else { - log.info("snapshot SUCCEEDED id={} day={} flights={}", head.cminmsgsId, day, normalized.size) - } - } catch (e: CasConflictException) { - log.warn("gen CAS conflict -> FAILED(INFRA) id={} msg={}", head.cminmsgsId, e.message) - procFailure.fail(head, ErrorClass.INFRA, e.message ?: "gen-cas-conflict") - } - } - - private fun safeBackfill(head: ProcState, msg: DecodedMessage) { - try { - inbox.backfillOnSuccess(head.cminmsgsId, msg.meta.sndr, msg.meta.type, msg.meta.styp, msg.meta.seqn) - } catch (e: Exception) { - log.error("backfill failed after SUCCEEDED id={} (compensation required)", head.cminmsgsId, e) - backfillTodo?.record( - BackfillTodoRepository.BackfillTask( - cminmsgsId = head.cminmsgsId, - sndr = msg.meta.sndr, - type = msg.meta.type, - styp = msg.meta.styp, - seqn = msg.meta.seqn, - ), - e.message ?: e.javaClass.simpleName, - ) ?: log.warn("no backfill-todo repository bound; compensation NOT persisted id={}", head.cminmsgsId) - } - } - - /** staging 结果（骨架）。 */ - sealed interface StageResult { - data class Ok(val day: String, val flights: List>) : StageResult - data class Invalid(val reason: String) : StageResult - - companion object { - var parser: ((DecodedMessage) -> StageResult)? = null - - fun stagingOf(msg: DecodedMessage): StageResult = - parser?.invoke(msg) ?: Invalid("staging-not-implemented(${msg.typeTag})") // TODO(阶段2) - } - } -} -class CasConflictException(message: String) : RuntimeException(message) -const val MAX_FLIGHTS_PER_SNAPSHOT = 10000 diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/processing/handlers/GtdtHandler.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/processing/handlers/GtdtHandler.kt deleted file mode 100644 index a0d099f..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/processing/handlers/GtdtHandler.kt +++ /dev/null @@ -1,44 +0,0 @@ -package com.gzzn.omms.msgexchange.processing.handlers - -import com.fasterxml.jackson.databind.ObjectMapper -import com.gzzn.omms.msgexchange.codec.FlopPayload -import com.gzzn.omms.msgexchange.domain.DecodedMessage -import com.gzzn.omms.msgexchange.domain.Decision -import com.gzzn.omms.msgexchange.domain.FlightChange -import com.gzzn.omms.msgexchange.domain.MsgKind -import com.gzzn.omms.msgexchange.domain.SchdPush -import com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine -import com.gzzn.omms.msgexchange.infra.persistence.FlightFieldsJson -import com.gzzn.omms.msgexchange.processing.Handler -import jakarta.inject.Singleton - -/** - * FLOP-GTDT：登机门集合级快照替换（SIS §3.34；GTNO=0 清除）。 - * 产出 FlightChange.fields["GTDT"] 为 JSON 数组，供 FlightStateEngine → persistNextStates 写入明细表。 - */ -@Singleton -class GtdtHandler( - private val mapper: ObjectMapper = ObjectMapper(), -) : Handler { - override val kind: MsgKind = MsgKind.Flop("GTDT") - - override fun decide(flightView: Map>, msg: DecodedMessage): Decision { - val body = msg.body as? FlopPayload - ?: throw IllegalArgumentException("GTDT handler requires FlopPayload body") - val gtdtItems = body.collections["GTDT"] ?: emptyList() - val fields = linkedMapOf() - fields["FLID"] = body.flid - body.scalars.forEach { (k, v) -> fields[k] = v } - fields["GTDT"] = mapper.writeValueAsString(gtdtItems) - - val current = flightView[body.flid]?.let { FlightStateEngine.fromFlightFields(body.flid, it) } - val commands = FlightStateEngine.commandsFromFields(body.flid, fields) - val preview = FlightStateEngine.apply(current, commands, messageId = "", bumpVersion = false) - val payloadJson = FlightFieldsJson.toJson(preview.toFlightFields(mapper)) - - return Decision( - flightChanges = listOf(FlightChange(body.flid, fields)), - schdPush = listOf(SchdPush(body.flid, payloadJson)), - ) - } -} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/reference/ReferenceService.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/reference/ReferenceService.kt deleted file mode 100644 index c81f312..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/reference/ReferenceService.kt +++ /dev/null @@ -1,19 +0,0 @@ -package com.gzzn.omms.msgexchange.reference - -import com.gzzn.omms.msgexchange.infra.persistence.StaticRefRepository -import jakarta.inject.Singleton - -/** - * ACMA-8 决策 6：21 类 admin-api 基础数据同步（阶段 6）。 - * 落点 = 21 类静态主数据独立 PostgreSQL 参考库（StaticRefRepository，ACM2-11）； - * 只产静态主数据行（权威写唯一入口）；影子实例不运行本模块（v4 影子二分）。 - */ -@Singleton -class ReferenceService( - private val staticRef: StaticRefRepository, - // TODO(阶段6): @Client(id="ADMINAPI") 声明式客户端 + 21 类端点配置化清单 -) { - fun refresh(type: String) { - // TODO(阶段6): 拉取 → staticRef.upsertAll(rows, source=ADMINAPI)；失败旧数据可用（参考库保留旧值） - } -} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/reference/RequestCoordinator.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/reference/RequestCoordinator.kt deleted file mode 100644 index 15a5bc8..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/reference/RequestCoordinator.kt +++ /dev/null @@ -1,60 +0,0 @@ -package com.gzzn.omms.msgexchange.reference - -import com.gzzn.omms.msgexchange.domain.RefUpsert -import com.gzzn.omms.msgexchange.infra.persistence.ReqTrackRepository -import com.gzzn.omms.msgexchange.infra.persistence.StaticRefRepository -import jakarta.inject.Singleton -import java.time.Instant - -/** - * ACMA-8 流程 6：请求式查询状态机（15 类，决策树版 v4）。 - * 同类并发=1（注册新请求先强制旧请求 EXPIRED，终态不再被匹配）； - * 应答匹配：回显字段 CONFIRM（矩阵 #13）→ 精确匹配；否则退化模式（DTTM ≥ SENT 才应用 + 审计）。 - * 应答落库目标 = 21 类静态主数据（独立 PG 参考库，StaticRefRepository，ACM2-11）； - * 注：N04 量纲缺陷（dttm vs epochMillis）与 N16 死分支未修，接线阶段 6 前处理（U19–U21）。 - */ -@Singleton -class RequestCoordinator( - private val reqTrack: ReqTrackRepository, - private val staticRef: StaticRefRepository, -) { - fun request(kind: String, rangeJson: String): Long { - reqTrack.forceExpireOpenOf(kind) // 防护① - val reqId = reqTrack.insert(kind, rangeJson) - // 出站走既有 COUTMSGS outbox（TODO(阶段6): codec.encodeRqrd + coutmsgs 落库 + link） - reqTrack.markSent(reqId, Instant.now()) - return reqId - // TODO(阶段6): timer.at(kind.timeout) { expireIfWaiting(reqId) } - } - - /** 应答处理（决策树）：msg.dttm 与回显字段由 codec 解出后传入。 */ - fun onResp( - kind: String, - dttm: Long, - echoSeqn: Long?, - records: List, // 应答载荷 → 静态主数据（source=AODB） - ): String { - val req = reqTrack.findOpenByKind(kind) ?: return audit("late/unknown resp kind=$kind") - val echoConfirmed = false // CONFIRM（矩阵 #13）待机场方结论 - return when { - echoConfirmed && echoSeqn != null -> { // ① 精确匹配（TODO: matchBy echoSeqn） - applyResp(req.reqId, records); "matched" - } - dttm < epochMillis(req.sentAt) -> audit("stale resp kind=$kind") // ② DTTM < SENT → 丢弃 - else -> { // ③ 退化模式应用；残余跨代错配风险明示接受并审计 - applyResp(req.reqId, records); "applied-degraded" - } - } - } - - private fun applyResp(reqId: Long, records: List) { - staticRef.upsertAll(records) // 独立 PG 参考库（阶段 6 接线） - reqTrack.markDone(reqId) - } - - private fun audit(reason: String): String = reason.also { - // TODO(阶段1后续): 审计表/日志（跨代错配残余风险可观测） - } - - private fun epochMillis(i: Instant?): Long = i?.toEpochMilli() ?: 0 -} diff --git a/src/main/kotlin/com/gzzn/omms/msgexchange/tools/FlightStoreDiffTool.kt b/src/main/kotlin/com/gzzn/omms/msgexchange/tools/FlightStoreDiffTool.kt deleted file mode 100644 index 95d4778..0000000 --- a/src/main/kotlin/com/gzzn/omms/msgexchange/tools/FlightStoreDiffTool.kt +++ /dev/null @@ -1,342 +0,0 @@ -package com.gzzn.omms.msgexchange.tools - -import com.fasterxml.jackson.databind.JsonNode -import com.fasterxml.jackson.databind.ObjectMapper -import com.fasterxml.jackson.databind.node.ArrayNode -import com.fasterxml.jackson.databind.node.ObjectNode -import com.fasterxml.jackson.databind.node.ValueNode -import com.gzzn.omms.msgexchange.infra.persistence.FlightFields -import java.math.BigDecimal - -/** - * ACM2-28 FS7：影子对拍跨存储 Diff 工具。 - * - * 核心目标： - * 对齐 CMINMSGS_ID 水位后，比对 nextgen 自有库 FLIGHT_SCHD（11g 修订：宽表列值） - * vs legacy 现役 Redis (flightInfo)。 - * - * 字段归一化规则（ACM2-28 评论 4 项 4，随宽表存储形态修订）： - * 1. PG 侧为宽表列值映射（字段名 → 字符串值），legacy 侧解析 FLTR JSON 后按字段逐一比对， - * 天然规避 JSONB/文本的 key 排序与序列化形态差异，绝不裸字符串比对； - * 2. 数值精度归一化（PG "100.0" vs legacy 100 视为等价；PG 值不可解析为数值则判不一致）； - * 3. 空值规范化（legacy JSON null/字段缺失 与 PG 列 NULL 视为等价，规避无语义偏离）； - * 4. 嵌套/集合值（legacy 为对象或数组）：PG 侧序列化文本按 JSON 解析后递归比对。 - * - * 已知合法偏离声明（ACM2-28 方案六）： - * ① FDAY 域化差删对跨代迁移行的保护（legacy 误删、nextgen 不删，属于修正性偏离）。 - */ -class FlightStoreDiffTool( - private val mapper: ObjectMapper = ObjectMapper(), -) { - - enum class DeviationKind { - /** 已知合法偏离 ①：跨日迁移航班在 nextgen PG 中受 FDAY 保护未被误删，而 legacy 误删 */ - CROSS_DAY_PROTECTED, - /** 非预期偏离：字段值不匹配 */ - FIELD_MISMATCH, - /** 非预期偏离：PG 缺失该航班 */ - MISSING_IN_PG, - /** 非预期偏离：Redis 缺失且非已知迁移保护行 */ - UNEXPECTED_EXTRA_IN_PG, - } - - data class Deviation( - val flid: String, - val kind: DeviationKind, - val path: String? = null, - val pgValue: Any? = null, - val legacyValue: Any? = null, - val detail: String, - ) - - data class DiffReport( - val totalPg: Int, - val totalLegacy: Int, - val matchedCount: Int, - val knownDeviations: List, - val unexpectedDeviations: List, - ) { - val isGreen: Boolean - get() = unexpectedDeviations.isEmpty() - - fun formatSummary(): String { - val sb = StringBuilder() - sb.appendLine("========== 影子对拍跨存储 Diff 报告 ==========") - sb.appendLine("Nextgen PG 航班总数 : $totalPg") - sb.appendLine("Legacy Redis 航班总数 : $totalLegacy") - sb.appendLine("完全一致航班数 : $matchedCount") - sb.appendLine("已知合法偏离数 (偏差①) : ${knownDeviations.size}") - sb.appendLine("非预期异常数 : ${unexpectedDeviations.size}") - sb.appendLine("验收红绿灯状态 : ${if (isGreen) "GREEN (通过)" else "RED (未通过)"}") - sb.appendLine("----------------------------------------------") - if (knownDeviations.isNotEmpty()) { - sb.appendLine("[已知合法偏离]") - knownDeviations.forEach { d -> - sb.appendLine(" FLID=${d.flid}: ${d.detail}") - } - } - if (unexpectedDeviations.isNotEmpty()) { - sb.appendLine("[非预期异常偏离]") - unexpectedDeviations.forEach { d -> - sb.appendLine(" FLID=${d.flid} [${d.kind}] path=${d.path}: PG=${d.pgValue}, Legacy=${d.legacyValue} (${d.detail})") - } - } - sb.appendLine("==============================================") - return sb.toString() - } - } - - - /** - * FS7 v2 双读对拍已随回退窗口关闭移除（V1.4.0，ACM2-29 P4）： - * legacy 槽位/里程碑列退场后无对比基准，损失性差异已由 P2-1 对拍记录留档。 - */ - - /** 单航班 FlightFields 对称比对（集合值为 JSON 文本时递归解析）。 */ - internal fun compareFlightFields(flid: String, left: FlightFields, right: FlightFields): List { - val mismatches = mutableListOf() - val allKeys = (left.keys + right.keys).toSortedSet() - for (key in allKeys) { - compareFieldValues(flid, key, left[key], right[key], mismatches) - } - return mismatches - } - - private fun compareFieldValues(flid: String, path: String, left: String?, right: String?, acc: MutableList) { - val leftNode = left?.let { parseFieldValue(it) } - val rightNode = right?.let { parseFieldValue(it) } - compareNodes(flid, path, leftNode, rightNode, acc) - } - - private fun parseFieldValue(value: String): JsonNode = - runCatching { mapper.readTree(value) }.getOrNull() - ?: mapper.nodeFactory.textNode(value) - - /** - * 执行全量比对。 - * @param pgFlights nextgen FLIGHT_SCHD 行（flid -> 宽表列值映射） - * @param legacyFlights legacy Redis flightInfo 行（flid -> value string） - * @param crossDayMigratedFlids 已知在多日之间迁移的航班 FLID 集合（用于识别合法偏差 ①） - */ - fun diff( - pgFlights: Map, - legacyFlights: Map, - crossDayMigratedFlids: Set = emptySet(), - ): DiffReport { - val allFlids = (pgFlights.keys + legacyFlights.keys).toSortedSet() - var matched = 0 - val known = mutableListOf() - val unexpected = mutableListOf() - - for (flid in allFlids) { - val pgFields = pgFlights[flid] - val legacyRaw = legacyFlights[flid] - - if (pgFields == null) { - unexpected += Deviation( - flid = flid, - kind = DeviationKind.MISSING_IN_PG, - legacyValue = legacyRaw, - detail = "Flight present in Redis but missing in PG", - ) - continue - } - - if (legacyRaw == null) { - if (flid in crossDayMigratedFlids) { - known += Deviation( - flid = flid, - kind = DeviationKind.CROSS_DAY_PROTECTED, - pgValue = pgFields, - detail = "Known Deviation 1: Cross-day migrated flight protected by FDAY domain-scoped delete in nextgen PG, erroneously deleted in legacy Redis", - ) - } else { - unexpected += Deviation( - flid = flid, - kind = DeviationKind.UNEXPECTED_EXTRA_IN_PG, - pgValue = pgFields, - detail = "Flight present in PG but missing in Redis without cross-day migration justification", - ) - } - continue - } - - // 两侧均存在：逐字段归一化比对（PG 列值 vs legacy JSON 解析树） - val mismatches = compareFlight(flid, pgFields, legacyRaw) - if (mismatches.isEmpty()) { - matched++ - } else { - unexpected.addAll(mismatches) - } - } - - return DiffReport( - totalPg = pgFlights.size, - totalLegacy = legacyFlights.size, - matchedCount = matched, - knownDeviations = known, - unexpectedDeviations = unexpected, - ) - } - - /** - * 单航班比对：PG 宽表列值映射 vs legacy FLTR JSON 文本。 - * 遍历两侧字段名并集，按归一化规则逐字段判定。 - */ - internal fun compareFlight(flid: String, pgFields: FlightFields, legacyJson: String): List { - val legacyNode = try { - mapper.readTree(legacyJson) - } catch (e: Exception) { - return listOf(Deviation(flid, DeviationKind.FIELD_MISMATCH, detail = "Legacy JSON parse failed: ${e.message}")) - } - if (!legacyNode.isObject) { - return listOf(Deviation(flid, DeviationKind.FIELD_MISMATCH, detail = "Legacy JSON is not an object")) - } - val legacyObj = legacyNode as ObjectNode - - val mismatches = mutableListOf() - val allKeys = (pgFields.keys + legacyObj.fieldNames().asSequence().toSet()).toSortedSet() - for (key in allKeys) { - val pgValue: String? = pgFields[key] - val legacyChild = legacyObj.get(key) - compareFieldNode(flid, key, pgValue, legacyChild, mismatches) - } - return mismatches - } - - /** 单字段判定：PG 列值（字符串或缺失）vs legacy JSON 节点。 */ - private fun compareFieldNode(flid: String, path: String, pgValue: String?, legacyNode: JsonNode?, acc: MutableList) { - fun mismatch(pg: Any?, legacy: Any?, detail: String) { - acc += Deviation(flid, DeviationKind.FIELD_MISMATCH, path = path, pgValue = pg, legacyValue = legacy, detail = detail) - } - - val legacyAbsent = legacyNode == null || legacyNode.isNull - val pgAbsent = pgValue == null - - // 规范化：legacy JSON null/缺失 与 PG 列 NULL 等价 - if (legacyAbsent && pgAbsent) return - if (legacyAbsent != pgAbsent) { - mismatch(pgValue, legacyNode?.asText(), "One side is null/absent while other is present") - return - } - - val legacy = legacyNode!! - - // 数值归一化（PG 列值可解析为数值时按数值比较） - if (legacy.isNumber) { - val pgNumber = pgValue!!.trim().toBigDecimalOrNull() - val legacyNumber = BigDecimal(legacy.asText()).stripTrailingZeros() - if (pgNumber == null || pgNumber.stripTrailingZeros().compareTo(legacyNumber) != 0) { - mismatch(pgValue, legacy.asText(), "Numeric value mismatch: $pgValue != ${legacy.asText()}") - } - return - } - - // 嵌套/集合值：PG 序列化文本按 JSON 解析后递归比对 - if (legacy.isObject || legacy.isArray) { - val pgNode = try { - mapper.readTree(pgValue) - } catch (_: Exception) { - null - } - if (pgNode == null) { - mismatch(pgValue, "<${legacy.nodeType}>", "PG value is not parseable as JSON while legacy is ${legacy.nodeType}") - return - } - compareNodes(flid, path, pgNode, legacy, acc) - return - } - - // 基本类型：文本值比较 - if (pgValue != legacy.asText()) { - mismatch(pgValue, legacy.asText(), "Value mismatch: $pgValue != ${legacy.asText()}") - } - } - - private fun compareNodes(flid: String, path: String, n1: JsonNode?, n2: JsonNode?, acc: MutableList) { - if (n1 == null && n2 == null) return - - // 规范化：null 节点与缺失节点视为等价 - val isN1Empty = n1 == null || n1.isNull - val isN2Empty = n2 == null || n2.isNull - if (isN1Empty && isN2Empty) return - - if (isN1Empty != isN2Empty) { - acc += Deviation( - flid = flid, - kind = DeviationKind.FIELD_MISMATCH, - path = path, - pgValue = n1?.asText(), - legacyValue = n2?.asText(), - detail = "One side is null/absent while other is present", - ) - return - } - - val nonNull1 = n1!! - val nonNull2 = n2!! - - // 数值归一化比较 - if (nonNull1.isNumber && nonNull2.isNumber) { - val d1 = BigDecimal(nonNull1.asText()).stripTrailingZeros() - val d2 = BigDecimal(nonNull2.asText()).stripTrailingZeros() - if (d1.compareTo(d2) != 0) { - acc += Deviation( - flid = flid, - kind = DeviationKind.FIELD_MISMATCH, - path = path, - pgValue = nonNull1.numberValue(), - legacyValue = nonNull2.numberValue(), - detail = "Numeric value mismatch: $d1 != $d2", - ) - } - return - } - - // 对象类型：递归属性比较（忽略 key 顺序） - if (nonNull1.isObject && nonNull2.isObject) { - val obj1 = nonNull1 as ObjectNode - val obj2 = nonNull2 as ObjectNode - val allKeys = (obj1.fieldNames().asSequence().toSet() + obj2.fieldNames().asSequence().toSet()).toSortedSet() - - for (key in allKeys) { - val nextPath = if (path.isEmpty()) key else "$path.$key" - compareNodes(flid, nextPath, obj1.get(key), obj2.get(key), acc) - } - return - } - - // 数组类型：逐项比较 - if (nonNull1.isArray && nonNull2.isArray) { - val arr1 = nonNull1 as ArrayNode - val arr2 = nonNull2 as ArrayNode - if (arr1.size() != arr2.size()) { - acc += Deviation( - flid = flid, - kind = DeviationKind.FIELD_MISMATCH, - path = path, - pgValue = "size=${arr1.size()}", - legacyValue = "size=${arr2.size()}", - detail = "Array size mismatch", - ) - return - } - for (i in 0 until arr1.size()) { - compareNodes(flid, "$path[$i]", arr1.get(i), arr2.get(i), acc) - } - return - } - - // 基本类型：文本值比较 - if (nonNull1.asText() != nonNull2.asText()) { - acc += Deviation( - flid = flid, - kind = DeviationKind.FIELD_MISMATCH, - path = path, - pgValue = nonNull1.asText(), - legacyValue = nonNull2.asText(), - detail = "Value mismatch: ${nonNull1.asText()} != ${nonNull2.asText()}", - ) - } - } -} diff --git a/src/main/resources/application.yml b/src/main/resources/application.yml index 2393a35..02a42bb 100644 --- a/src/main/resources/application.yml +++ b/src/main/resources/application.yml @@ -5,12 +5,11 @@ # 口令/端点全部环境变量外置——零入库。 # 存储边界：自有 PostgreSQL（datasources.default，全部内部状态与运营航班权威 FLIGHT_SCHD/SCHD_GEN）+ # 共享 MySQL 信箱（mailbox.shared-mysql，仅 DML，不建表）+ -# Kafka + Eureka/logstash（见 ACM2-28 与 docs；Redis 彻底退出阶段 A 权威与写路径）。 +# Kafka + Eureka/logstash。权威设计：docs/flight-state.md。 # 键位口径：Micronaut 5.1（U03）：datasources.* / flyway.datasources.* / # kafka.producers..* / eureka.client.*（Micronaut 原生键）。 # ===================================================================== msgx: - phase: A # 航班动态权威落自有 PG（FLIGHT_SCHD，主泵单线程写，I5；ACM2-28 定案） service-name: msgexchangeapi # 契约冻结；影子实例用 msgexchangeapi-shadow register-eureka: true # 影子对拍期置 false 或独立服务名 pipeline: # —— ACMA-8 参数表初值（阶段 0 基线校准）—— @@ -24,11 +23,16 @@ msgx: schd: flush-period: 3s # KEEP 现役推送节律 flush-limit: 500 # 批上限，防积压尖峰 - identity: - include-day-boundary: false # CONFIRM（矩阵 #11）：SEQN 重置作用域确认前保持关闭 - consistency-check: # 一致性哨兵（实装属 U25） - on-startup: true - daily-sample-ratio: 0.01 + operation-day: # §3.5：SODT + 机场时区 + 切日边界（默认占位 0 点，待业务确认 §10） + zone: Asia/Shanghai + cutoff-hour: 0 + history: # §8：历史判定窗口与保留期 + cancelled-hours: 48 + terminal-hours: 48 + deleted-hours: 48 + idle-hours: 168 + snap-log-retention-days: 90 + history-store-enabled: false # 未接通历史存储时 HISTORY_SWEEP 删 0 条（§8.2） micronaut: application: @@ -37,8 +41,9 @@ micronaut: port: 8080 # 管理端点（U03/N32）：/env、/beans 默认 sensitive；仅开发/影子环境放开——见 application-dev.yml -# 自有 PostgreSQL：全部内部状态（消息管道 PROC_STATE/MSG_EVENT、PUMP_JOB、REQ_TRACK、 -# 21 类 REF_MASTER、运营航班表 FLIGHT_SCHD 与 SCHD_GEN；迁移 db/migration，ACM2-28）。数据层实装前 enabled=false。 +# 自有 PostgreSQL：全部内部状态（PIPELINE_LOCK/PROC_STATE/MSG_EVENT/REQ_TRACK/BACKFILL_TODO/ +# FLIGHT_SCHD 及明细表/SCHD_SNAP_LOG；迁移 db/migration/V1__flight_state_baseline.sql）。 +# 数据层实装前 enabled=false（stub 模式不建连）。 datasources: default: enabled: false diff --git a/src/main/resources/db/migration/V1.0.0__own_pg_pipeline.sql b/src/main/resources/db/migration/V1.0.0__own_pg_pipeline.sql deleted file mode 100644 index 530f2a8..0000000 --- a/src/main/resources/db/migration/V1.0.0__own_pg_pipeline.sql +++ /dev/null @@ -1,79 +0,0 @@ --- ===================================================================== --- 自有 PostgreSQL 库（唯一自有数据库）· ACM2-12 定稿口径 --- --------------------------------------------------------------------- --- 存储边界（ACM2-12，取代 ACMA-8 v4「MySQL 六辅助表」口径，原 MySQL V2.0.0 --- 六表脚本已删除，定义见 git 历史）： --- · 自有 PG：全部内部状态 —— 消息管道（PROC_STATE / MSG_EVENT）+ 调度与请求 --- （PUMP_JOB / REQ_TRACK）+ 21 类静态主数据（REF_MASTER）。本地事务只在此库。 --- · 共享 MySQL（cdairport，他人系统库）：仅信箱 DML —— 上游外部写 CMINMSGS； --- 本系统 JDBC 轮询读 + 处理回填；出站写 COUTMSGS（他人读取发送）； --- 本系统不在此建表/schema（迁移不覆盖）。 --- · Redis：航班动态 flightInfo + 快照 SCHD_GEN（Lua 原子，另行脚本）。 --- · FLIGHT_STATE（阶段 B 权威）：缓做，不落表（Redis 永续动态权威）。 --- 21 类静态表 REF_MASTER 为通用 rtype/rkey 形态（SOURCE 审计；gen/SCHD_GEN --- 已迁 Redis，不再需要 VERSION/CAS 列）。 --- 影子对拍：本库开独立 schema/实例；共享信箱只读水位/回放（见 ACM2-12 影响面）。 --- ===================================================================== - --- ① 处理伴生状态（取消息侧；与共享库 CMINMSGS 一一对应，CMINMSGS_ID 沿用其主键； --- 入队由 JDBC 轮询发现新信后写入，非本系统创建 CMINMSGS 原文） -CREATE TABLE PROC_STATE ( - CMINMSGS_ID BIGINT PRIMARY KEY, - STATE VARCHAR(16) NOT NULL, -- PENDING/FAILED/SUCCEEDED/SKIPPED/DEAD - IDENTITY_KEY VARCHAR(200), -- SNDR|TYPE|STYP|SEQN，decode 后首次绑定（I3） - ATTEMPTS INT NOT NULL DEFAULT 0, - NEXT_ATTEMPT_AT TIMESTAMP, - ERROR_CLASS VARCHAR(20), -- MALFORMED/CODEC_ERROR/EXHAUSTED/INFRA/UNSUPPORTED - LAST_ERROR VARCHAR(1000), - UPDATED_AT TIMESTAMP NOT NULL, - CONSTRAINT uk_proc_identity UNIQUE (IDENTITY_KEY) -); -CREATE INDEX idx_proc_head ON PROC_STATE (STATE, CMINMSGS_ID); -- 主泵队头查询（I1） - --- ② 统一投递事件 outbox（发消息侧） -CREATE TABLE MSG_EVENT ( - EVENT_ID BIGSERIAL PRIMARY KEY, - TARGET VARCHAR(30) NOT NULL, -- KAFKA:msg / KAFKA:schd（A起）；ES:flight_hts / REDIS:flightInfo（B起） - PARTITION_KEY VARCHAR(64), -- schd 事件恒为 FLID - PAYLOAD_JSON TEXT NOT NULL, - STATE VARCHAR(16) NOT NULL, -- PENDING/SENT/DEAD - ATTEMPTS INT NOT NULL DEFAULT 0, - NEXT_ATTEMPT_AT TIMESTAMP, - ERROR_CLASS VARCHAR(20), - LAST_ERROR VARCHAR(1000), - CREATED_AT TIMESTAMP NOT NULL -); -CREATE INDEX idx_evt_head ON MSG_EVENT (TARGET, STATE, EVENT_ID); -- 每 target 队头（I1 双层） - --- ③ 泵作业调度记录（作业不插队：仅在消息队头空闲/退避窗口由主泵执行，决策 1 修订） -CREATE TABLE PUMP_JOB ( - JOB_ID BIGSERIAL PRIMARY KEY, - KIND VARCHAR(20) NOT NULL, -- ARCHIVE / HISTORY_SWEEP / PROJECTION_REBUILD - STATE VARCHAR(16) NOT NULL, -- QUEUED/RUNNING/DONE/FAILED - CREATED_AT TIMESTAMP NOT NULL, - UPDATED_AT TIMESTAMP NOT NULL, - LAST_ERROR VARCHAR(1000) -); - --- ④ 15 类请求状态机（Reference & Query 域；与共享库 COUTMSGS 跨库： --- 先落 COUTMSGS 成功 → 再标 SENT，补偿重扫，最终一致） -CREATE TABLE REQ_TRACK ( - REQ_ID BIGSERIAL PRIMARY KEY, - REQ_TYPE VARCHAR(20) NOT NULL, - PARAMS_JSON TEXT NOT NULL, - STATE VARCHAR(16) NOT NULL, -- REGISTERED/SENT/WAITING/DONE/EXPIRED - COUTMSGS_ID BIGINT, -- U18 修正：BIGINT（原 MySQL 版 INT） - SENT_AT TIMESTAMP, - COMPLETED_AT TIMESTAMP -); -CREATE INDEX idx_req_open ON REQ_TRACK (REQ_TYPE, STATE); - --- ⑤ 21 类静态主数据（SOURCE=ADMINAPI/AODB/PIPELINE 审计；弱事务 upsert） -CREATE TABLE REF_MASTER ( - RTYPE VARCHAR(20) NOT NULL, -- AIRL/ARPT/... - RKEY VARCHAR(64) NOT NULL, - PAYLOAD_JSON TEXT NOT NULL, - SOURCE VARCHAR(20) NOT NULL, - REFRESHED_AT TIMESTAMP NOT NULL, - PRIMARY KEY (RTYPE, RKEY) -); diff --git a/src/main/resources/db/migration/V1.1.0__flight_schd.sql b/src/main/resources/db/migration/V1.1.0__flight_schd.sql deleted file mode 100644 index f11d947..0000000 --- a/src/main/resources/db/migration/V1.1.0__flight_schd.sql +++ /dev/null @@ -1,115 +0,0 @@ --- ===================================================================== --- 自有库 · 阶段 A 运营航班表与计划代迁移 · ACM2-28 定案（选项 C）· 11g 修订 --- --------------------------------------------------------------------- --- 存储边界（ACM2-28 定案，取代 ACM2-12「FLIGHT_STATE 缓做不落表、Redis 永续权威」口径）： --- · FLIGHT_SCHD：SCHD 日计划快照 + FLOP/ADFT 增量合并后的当前运营航班权威态，一行一航班。 --- · SCHD_GEN / SCHD_GEN_FLID：各日代版本（SQL CAS）与当前代航班全量集合（差删依据）。 --- · Redis：彻底退出动态权威与全部写路径；阶段 A 读写事务全在自有库原子完成。 --- · 增量 Upsert 语义： --- - 快照全量写入（DNLD）：声明/更新 FDAY 归属，航班字段整体替换； --- - 增量写入（FLOP/ADFT）：新插置 NULL FDAY，已有行保留原 FDAY，按字段列更新 --- （与 legacy flightInfo hash「仅新增/覆盖 field」同语义）； --- - 按代差删域化：DELETE FROM FLIGHT_SCHD WHERE FDAY = :day AND FLID IN (:diffSet) --- 仅删除仍属旧代的行，ADFT（FDAY=NULL）与已迁移至新代的同 FLID 行天然存活。 --- · 时间列规范：统一使用 TIMESTAMP(6) WITH TIME ZONE。 --- --------------------------------------------------------------------- --- 11g 修订（现场数据库版本 11g，废除 FLTR_JSON JSONB 整文档存储）： --- · 表形态 = 运营航班宽表：一行一航班，字段即列，天然可索引、可直查、可加约束； --- · 列清单锚定 legacy SIS 契约（unisysaodbsis.xsd SCHD.FLTR 记录，ACMA-4 基线， --- 32 个 Handler 的 KEEP/FIX 翻译即以此为闭集）： --- - 标量字段 → 真实列（约 45 列 + legacy 派生列 ABDG/LPSDT/ABN）； --- - 1:N 明细集合（登机口/柜台/转盘/桥/延误/航线等，单航班可达 99 条）各占一列 --- 存序列化文本（TEXT；11g 移植为 CLOB），阶段 2/3 语义钉死后按需升独立子表； --- · 值语义保持 legacy 字符串原样（不做库端类型转换），保证影子对拍逐字段保真； --- 范围查询需要的类型化列（如 SODT→DATE）按查询需求逐列后置提升。 --- ===================================================================== - --- ① 运营航班表（当前运营航班动态权威，FLID 与 legacy flightInfo 同键；一行一航班） -CREATE TABLE FLIGHT_SCHD ( - FLID VARCHAR(32) NOT NULL PRIMARY KEY, -- AODB 唯一航班标识 - FDAY DATE NULL, -- 所属日计划代；NULL = 非代所有（ADFT/增量增建） - -- ==== SCHD.FLTR 标量字段（unisysaodbsis.xsd，值保持 legacy 字符串原样）==== - ALCD VARCHAR(64) NULL, -- 航空公司代码 - ALSC VARCHAR(64) NULL, -- 航空公司简称 - FLNO VARCHAR(64) NULL, -- 航班号 - MVIN VARCHAR(64) NULL, -- 进离港标识（A=到港 D=离港） - SODT VARCHAR(64) NULL, -- 计划时间（ddMMMyyHHmm） - FLTY VARCHAR(64) NULL, -- 航班类型 - FLIN VARCHAR(64) NULL, -- 国内/国际/混合属性 - ACFT VARCHAR(64) NULL, -- 机型 - RENO VARCHAR(64) NULL, -- 机尾号 - TAOP VARCHAR(64) NULL, -- 实际承运人代码 - TAFL VARCHAR(64) NULL, -- 实际承运航班号 - TAID VARCHAR(64) NULL, -- 实际承运航班 FLID - TRML VARCHAR(64) NULL, -- 航站楼 - MAXP VARCHAR(64) NULL, -- 最大旅客数 - CSOP VARCHAR(64) NULL, -- 共享承运人代码 - CSFT VARCHAR(64) NULL, -- 共享航班号 - MAID VARCHAR(32) NULL, -- 共享主航班 FLID - ESTT VARCHAR(64) NULL, -- 预计时间 - ACTT VARCHAR(64) NULL, -- 实际时间 - STND VARCHAR(64) NULL, -- 备降站 - PHAG VARCHAR(64) NULL, -- 地服代理（值机） - CNCL VARCHAR(64) NULL, -- 取消时间 - REMC VARCHAR(80) NULL, -- 备注 - BOTM VARCHAR(64) NULL, -- 摆渡车时间 - LACL VARCHAR(64) NULL, -- 行李确认时间 - FINT VARCHAR(64) NULL, -- 完成时间 - APPT VARCHAR(64) NULL, -- 旅客到达时间 - EGSR VARCHAR(64) NULL, -- 保障开始时间 - EGST VARCHAR(64) NULL, -- 保障结束时间 - FHAG VARCHAR(64) NULL, -- 地服代理（货运） - MHAG VARCHAR(64) NULL, -- 地服代理（机务） - VIPP VARCHAR(64) NULL, -- VIP 旅客数 - VIPR VARCHAR(64) NULL, -- VIP 等级 - LBNO VARCHAR(64) NULL, -- 行李件数 - LBWT VARCHAR(64) NULL, -- 行李重量 - PAXC VARCHAR(64) NULL, -- 旅客计数 - EXSC VARCHAR(64) NULL, -- 例外代码 - EXSR VARCHAR(256) NULL, -- 例外原因 - FTSS VARCHAR(64) NULL, -- 航班状态 - PEDT VARCHAR(64) NULL, -- 前序估计时间 - NEAT VARCHAR(64) NULL, -- 下机完成时间 - PADT VARCHAR(64) NULL, -- 旅客登机时间 - NAAT VARCHAR(64) NULL, -- 关门时间 - -- ==== legacy flightInfo 派生标量字段（user-stories §SCHD.FLTR）==== - ABDG VARCHAR(64) NULL, -- 当前登机桥（派生） - LPSDT VARCHAR(64) NULL, -- 历史计划机位（派生） - ABN VARCHAR(64) NULL, -- 异常标识（派生） - -- ==== 1:N 明细集合（序列化文本，一列一集合；阶段 2/3 按需升独立子表）==== - ROUT_TXT TEXT NULL, -- 航线（ROUTEDAILY ×6） - ERUT_TXT TEXT NULL, -- 扩展航线（ROUTEDAILY ×2..7） - CHDT_TXT TEXT NULL, -- 廊桥/ chute 数据（×99） - GTDT_TXT TEXT NULL, -- 登机口（×99） - PSDT_TXT TEXT NULL, -- 机位（×9） - CKDT_TXT TEXT NULL, -- 值机柜台（×99） - CLDT_TXT TEXT NULL, -- 行李转盘（×99） - DELY_TXT TEXT NULL, -- 延误（不限） - CHOT_TXT TEXT NULL, -- 客桥桥载（×99） - ABTM_TXT TEXT NULL, -- 登机桥（×99） - SRVT_TXT TEXT NULL, -- 服务（不限） - VIPF_TXT TEXT NULL, -- VIP（不限） - MAFL_TXT TEXT NULL, -- 共享航班列表（legacy 派生集合） - FDIV_TXT TEXT NULL, -- 备降明细 - FRET_TXT TEXT NULL, -- 返航明细 - FLAB_TXT TEXT NULL, -- 中止明细 - CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, - UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL -); -CREATE INDEX idx_flight_schd_fday ON FLIGHT_SCHD (FDAY); -- 日代扫描与影子对拍分片 --- 运营查询索引按需逐列追加（11g/PG 均为普通 B-tree，零方言成本） - --- ② 日计划代与差删元数据表（代版本推进与版本 CAS） -CREATE TABLE SCHD_GEN ( - FDAY DATE NOT NULL PRIMARY KEY, -- 计划日（yyyy-MM-dd） - VERSION BIGINT NOT NULL, -- 代版本号（单调递增） - UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL -); - --- ③ 当前代有效航班 FLID 集合（差删依据） -CREATE TABLE SCHD_GEN_FLID ( - FDAY DATE NOT NULL, -- 所属计划日（FK 级联删除） - FLID VARCHAR(32) NOT NULL, -- 当前代有效航班 - CONSTRAINT pk_schd_gen_flid PRIMARY KEY (FDAY, FLID), - CONSTRAINT fk_schd_gen_flid FOREIGN KEY (FDAY) REFERENCES SCHD_GEN (FDAY) ON DELETE CASCADE -); diff --git a/src/main/resources/db/migration/V1.2.0__flight_resources.sql b/src/main/resources/db/migration/V1.2.0__flight_resources.sql deleted file mode 100644 index 27d4a9f..0000000 --- a/src/main/resources/db/migration/V1.2.0__flight_resources.sql +++ /dev/null @@ -1,118 +0,0 @@ --- ===================================================================== --- ACM2-29 · 全宽表·零子表（Zero-Subtable Wide Table）定案落地 --- --------------------------------------------------------------------- --- 废除「16 个明细集合保留 *_TXT 并逐个升独立子表」的原规划（对 XSD --- maxOccurs=99 的过度设计）。结合 docs/legacy/SIS_AODB_RMS-V0.1.md 报文 --- 样例与成都现场（双流/天府）地服实际规律——登机门 1~2 个、值机柜台专属 --- 1~3 个、转盘 1~2 个、延误单有效、靠撤桥/轮挡各 1 次——全部集合收敛为 --- FLIGHT_SCHD 宽表的标准 VARCHAR 标量列或紧凑格式化字串，零子表、零 CLOB： --- · 平铺标量列（属性名 + 出现槽位序号，可按需直接追加普通 B-tree， --- 11g/PG 零方言成本）： --- GTDT → GATE/PGOT/PGCT/GOTM/GCTM/GTYP ×2 组（登机门 1~2 个） --- CKDT → CHKC/CCLS/PCOT/PCCT/COTM/CCTM/CTYP ×3 组 --- （值机柜台 1~3 个；同柜台分舱复用条目在写侧抹平去重） --- CLDT → BELT/BCLS/BPCOT/BPCCT/FBAG/LBAG/BTYP ×2 组（转盘 1~2 个； --- B 前缀时刻 = BELT 专属，避免与值机 PCOT/PCCT 同名） --- PSDT → PSST/STST/STET ×2 组（计划机位 1~2 个） --- CHDT → CHUT/CHCLS/PCBT/PCET/CBTM/CETM/CHTYP ×2 组（离港通道 1~2 个； --- CH 前缀 = CHUTE 专属，避免与值机 CCLS/CTYP 同名） --- DELY → DELY_CODE/DELY_STRT/DELY_DURA/DELY_REMC（单有效覆盖语义） --- ABTM → ABTM_A/ABTM_D（靠桥/撤桥时刻；桥号沿用派生列 ABDG） --- CHOT → CHOT_ON/CHOT_OFF（上/下轮挡时刻；机位沿用派生列 STND） --- FDIV/FRET/FLAB → 前缀标量列（规范 1:0..1 单值异常） --- 超出现场定界的条目按定案丢弃（不再为 XSD 理论上限设计）。 --- · 紧凑 VARCHAR 字串列（无界/航路型集合，11g VARCHAR2 内联存储，非 CLOB）： --- ROUT/ERUT → ROUT_PATH/ERUT_PATH（≤7 站紧凑航路 "APCD/SCAT/SCDT,..."） --- SRVT/VIPF/MAFL → *_TEXT（服务/VIP/共享列表条数无 XSD 上限，存 JSON 数组； --- 超长写侧 fail fast，不允许静默截断） --- · 值语义保持 legacy 字符串原样；读侧视图由标量列重建集合键， --- 与 legacy flightInfo hash 同构（KAFKA_SCHD 线格式与 FS7 对拍不变）。 --- · 运营资源消息语义 = 单资源集合级全量快照替换；外层航班字段仍为 --- 字段级增量合并（集合键出现即整集合覆盖，缺槽位置 NULL）。 --- --------------------------------------------------------------------- --- 交叉实例单写者互斥（I5 修正定案）：PIPELINE_LOCK 行级锁。主泵写事务 --- 持有 FLIGHT_SCHD_WRITER 行 SELECT ... FOR UPDATE；连接断开、提交或回滚 --- 时由数据库释放，PG 与 Oracle 11g 同构，无 DBMS_LOCK DBA 特权依赖。 --- ===================================================================== - -ALTER TABLE FLIGHT_SCHD - -- 登机门 GTDT（现场 1~2 个） - ADD COLUMN GATE1 VARCHAR(64), ADD COLUMN PGOT1 VARCHAR(64), ADD COLUMN PGCT1 VARCHAR(64), - ADD COLUMN GOTM1 VARCHAR(64), ADD COLUMN GCTM1 VARCHAR(64), ADD COLUMN GTYP1 VARCHAR(64), - ADD COLUMN GATE2 VARCHAR(64), ADD COLUMN PGOT2 VARCHAR(64), ADD COLUMN PGCT2 VARCHAR(64), - ADD COLUMN GOTM2 VARCHAR(64), ADD COLUMN GCTM2 VARCHAR(64), ADD COLUMN GTYP2 VARCHAR(64), - -- 值机柜台 CKDT（现场专属 1~3 个，分舱复用条目写侧抹平） - ADD COLUMN CHKC1 VARCHAR(64), ADD COLUMN CCLS1 VARCHAR(64), - ADD COLUMN PCOT1 VARCHAR(64), ADD COLUMN PCCT1 VARCHAR(64), - ADD COLUMN COTM1 VARCHAR(64), ADD COLUMN CCTM1 VARCHAR(64), ADD COLUMN CTYP1 VARCHAR(64), - ADD COLUMN CHKC2 VARCHAR(64), ADD COLUMN CCLS2 VARCHAR(64), - ADD COLUMN PCOT2 VARCHAR(64), ADD COLUMN PCCT2 VARCHAR(64), - ADD COLUMN COTM2 VARCHAR(64), ADD COLUMN CCTM2 VARCHAR(64), ADD COLUMN CTYP2 VARCHAR(64), - ADD COLUMN CHKC3 VARCHAR(64), ADD COLUMN CCLS3 VARCHAR(64), - ADD COLUMN PCOT3 VARCHAR(64), ADD COLUMN PCCT3 VARCHAR(64), - ADD COLUMN COTM3 VARCHAR(64), ADD COLUMN CCTM3 VARCHAR(64), ADD COLUMN CTYP3 VARCHAR(64), - -- 行李转盘 CLDT（现场 1~2 个；B 前缀时刻 = BELT 专属） - ADD COLUMN BELT1 VARCHAR(64), ADD COLUMN BCLS1 VARCHAR(64), - ADD COLUMN BPCOT1 VARCHAR(64), ADD COLUMN BPCCT1 VARCHAR(64), - ADD COLUMN FBAG1 VARCHAR(64), ADD COLUMN LBAG1 VARCHAR(64), ADD COLUMN BTYP1 VARCHAR(64), - ADD COLUMN BELT2 VARCHAR(64), ADD COLUMN BCLS2 VARCHAR(64), - ADD COLUMN BPCOT2 VARCHAR(64), ADD COLUMN BPCCT2 VARCHAR(64), - ADD COLUMN FBAG2 VARCHAR(64), ADD COLUMN LBAG2 VARCHAR(64), ADD COLUMN BTYP2 VARCHAR(64), - -- 计划机位 PSDT（现场 1~2 个） - ADD COLUMN PSST1 VARCHAR(64), ADD COLUMN STST1 VARCHAR(64), ADD COLUMN STET1 VARCHAR(64), - ADD COLUMN PSST2 VARCHAR(64), ADD COLUMN STST2 VARCHAR(64), ADD COLUMN STET2 VARCHAR(64), - -- 离港行李通道 CHDT（现场 1~2 个；CH 前缀 = CHUTE 专属） - ADD COLUMN CHUT1 VARCHAR(64), ADD COLUMN CHCLS1 VARCHAR(64), - ADD COLUMN PCBT1 VARCHAR(64), ADD COLUMN PCET1 VARCHAR(64), - ADD COLUMN CBTM1 VARCHAR(64), ADD COLUMN CETM1 VARCHAR(64), ADD COLUMN CHTYP1 VARCHAR(64), - ADD COLUMN CHUT2 VARCHAR(64), ADD COLUMN CHCLS2 VARCHAR(64), - ADD COLUMN PCBT2 VARCHAR(64), ADD COLUMN PCET2 VARCHAR(64), - ADD COLUMN CBTM2 VARCHAR(64), ADD COLUMN CETM2 VARCHAR(64), ADD COLUMN CHTYP2 VARCHAR(64), - -- 延误 DELY（单有效覆盖语义） - ADD COLUMN DELY_CODE VARCHAR(64), - ADD COLUMN DELY_STRT VARCHAR(64), - ADD COLUMN DELY_DURA VARCHAR(64), - ADD COLUMN DELY_REMC VARCHAR(80), - -- 靠撤桥 ABTM / 轮挡 CHOT（保障里程碑时刻） - ADD COLUMN ABTM_A VARCHAR(64), - ADD COLUMN ABTM_D VARCHAR(64), - ADD COLUMN CHOT_ON VARCHAR(64), - ADD COLUMN CHOT_OFF VARCHAR(64), - -- 异常明细 FDIV/FRET/FLAB（规范 1:0..1 单值异常） - ADD COLUMN FDIV_DDES VARCHAR(64), - ADD COLUMN FDIV_DDIR VARCHAR(64), - ADD COLUMN FDIV_REMC VARCHAR(80), - ADD COLUMN FRET_REID VARCHAR(64), - ADD COLUMN FRET_RSN VARCHAR(80), - ADD COLUMN FLAB_ARES VARCHAR(64), - ADD COLUMN FLAB_RSN VARCHAR(80), - -- 紧凑航路字串（≤7 站，免除多表关联） - ADD COLUMN ROUT_PATH VARCHAR(256), - ADD COLUMN ERUT_PATH VARCHAR(256), - -- 无界集合紧凑 JSON 数组字串（11g VARCHAR2 内联，非 CLOB；超长写侧 fail fast） - ADD COLUMN SRVT_TEXT VARCHAR(2000), - ADD COLUMN VIPF_TEXT VARCHAR(512), - ADD COLUMN MAFL_TEXT VARCHAR(512); - --- 16 个集合的过渡 *_TXT 文本列全清场 -ALTER TABLE FLIGHT_SCHD - DROP COLUMN ROUT_TXT, DROP COLUMN ERUT_TXT, DROP COLUMN CHDT_TXT, - DROP COLUMN GTDT_TXT, DROP COLUMN PSDT_TXT, DROP COLUMN CKDT_TXT, - DROP COLUMN CLDT_TXT, DROP COLUMN DELY_TXT, DROP COLUMN CHOT_TXT, - DROP COLUMN ABTM_TXT, DROP COLUMN SRVT_TXT, DROP COLUMN VIPF_TXT, - DROP COLUMN MAFL_TXT, DROP COLUMN FDIV_TXT, DROP COLUMN FRET_TXT, - DROP COLUMN FLAB_TXT; - --- 运营查询索引按需逐列追加示例（11g/PG 均为普通 B-tree，零方言成本）： --- CREATE INDEX idx_flight_schd_gate1 ON FLIGHT_SCHD (GATE1); --- CREATE INDEX idx_flight_schd_chkc1 ON FLIGHT_SCHD (CHKC1); --- CREATE INDEX idx_flight_schd_dely_code ON FLIGHT_SCHD (DELY_CODE); - --- 交叉实例单写者互斥行（I5：SELECT ... FOR UPDATE 行级锁） -CREATE TABLE PIPELINE_LOCK ( - LOCK_KEY VARCHAR(32) NOT NULL PRIMARY KEY, - OWNER_INFO VARCHAR(128), - UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL -); -INSERT INTO PIPELINE_LOCK (LOCK_KEY, OWNER_INFO, UPDATED_AT) -VALUES ('FLIGHT_SCHD_WRITER', NULL, CURRENT_TIMESTAMP); diff --git a/src/main/resources/db/migration/V1.3.0__flight_detail_tables.sql b/src/main/resources/db/migration/V1.3.0__flight_detail_tables.sql deleted file mode 100644 index e80693c..0000000 --- a/src/main/resources/db/migration/V1.3.0__flight_detail_tables.sql +++ /dev/null @@ -1,146 +0,0 @@ --- docs/flight-state.md (ACM2-29): lossless detail tables + tracking columns - -ALTER TABLE flight_schd - ADD COLUMN IF NOT EXISTS last_message_id VARCHAR(64), - ADD COLUMN IF NOT EXISTS state_version BIGINT NOT NULL DEFAULT 0; - -ALTER TABLE schd_gen - ADD COLUMN IF NOT EXISTS last_message_id VARCHAR(64); - -CREATE TABLE IF NOT EXISTS flight_gate ( - flid VARCHAR(32) NOT NULL, - ordinal INTEGER NOT NULL, - source_seq VARCHAR(64), - record_version BIGINT NOT NULL DEFAULT 0, - gate VARCHAR(64), - pgot VARCHAR(64), - pgct VARCHAR(64), - gotm VARCHAR(64), - gctm VARCHAR(64), - gtyp VARCHAR(64), - created_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - updated_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - PRIMARY KEY (flid, ordinal) -); - -CREATE TABLE IF NOT EXISTS flight_checkin ( - flid VARCHAR(32) NOT NULL, - ordinal INTEGER NOT NULL, - source_seq VARCHAR(64), - record_version BIGINT NOT NULL DEFAULT 0, - chkc VARCHAR(64), - ccls VARCHAR(64), - pcot VARCHAR(64), - pcct VARCHAR(64), - cotm VARCHAR(64), - cctm VARCHAR(64), - ctyp VARCHAR(64), - created_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - updated_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - PRIMARY KEY (flid, ordinal) -); - -CREATE TABLE IF NOT EXISTS flight_belt ( - flid VARCHAR(32) NOT NULL, - ordinal INTEGER NOT NULL, - source_seq VARCHAR(64), - record_version BIGINT NOT NULL DEFAULT 0, - belt VARCHAR(64), - bcls VARCHAR(64), - pcot VARCHAR(64), - pcct VARCHAR(64), - fbag VARCHAR(64), - lbag VARCHAR(64), - btyp VARCHAR(64), - created_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - updated_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - PRIMARY KEY (flid, ordinal) -); - -CREATE TABLE IF NOT EXISTS flight_stand_plan ( - flid VARCHAR(32) NOT NULL, - ordinal INTEGER NOT NULL, - source_seq VARCHAR(64), - record_version BIGINT NOT NULL DEFAULT 0, - psst VARCHAR(64), - stst VARCHAR(64), - stet VARCHAR(64), - created_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - updated_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - PRIMARY KEY (flid, ordinal) -); - -CREATE TABLE IF NOT EXISTS flight_chute ( - flid VARCHAR(32) NOT NULL, - ordinal INTEGER NOT NULL, - source_seq VARCHAR(64), - record_version BIGINT NOT NULL DEFAULT 0, - chut VARCHAR(64), - chcls VARCHAR(64), - pcbt VARCHAR(64), - pcet VARCHAR(64), - cbtm VARCHAR(64), - cetm VARCHAR(64), - chtyp VARCHAR(64), - created_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - updated_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - PRIMARY KEY (flid, ordinal) -); - -CREATE TABLE IF NOT EXISTS flight_delay ( - flid VARCHAR(32) NOT NULL, - ordinal INTEGER NOT NULL, - source_seq VARCHAR(64), - record_version BIGINT NOT NULL DEFAULT 0, - code VARCHAR(64), - strt VARCHAR(64), - dura VARCHAR(64), - remc VARCHAR(80), - created_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - updated_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - PRIMARY KEY (flid, ordinal) -); - -CREATE TABLE IF NOT EXISTS flight_bridge_op ( - flid VARCHAR(32) NOT NULL, - ordinal INTEGER NOT NULL, - source_seq VARCHAR(64), - record_version BIGINT NOT NULL DEFAULT 0, - abdg VARCHAR(64), - abop VARCHAR(64), - aotm VARCHAR(64), - created_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - updated_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - PRIMARY KEY (flid, ordinal) -); - -CREATE TABLE IF NOT EXISTS flight_chock_op ( - flid VARCHAR(32) NOT NULL, - ordinal INTEGER NOT NULL, - source_seq VARCHAR(64), - record_version BIGINT NOT NULL DEFAULT 0, - chid VARCHAR(64), - chst VARCHAR(64), - chtm VARCHAR(64), - created_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - updated_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - PRIMARY KEY (flid, ordinal) -); - -CREATE TABLE IF NOT EXISTS flight_route_point ( - flid VARCHAR(32) NOT NULL, - ordinal INTEGER NOT NULL, - source_seq VARCHAR(64), - record_version BIGINT NOT NULL DEFAULT 0, - route_kind VARCHAR(8) NOT NULL, - apcd VARCHAR(64), - scat VARCHAR(64), - scdt VARCHAR(64), - created_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - updated_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - PRIMARY KEY (flid, ordinal) -); - -CREATE INDEX IF NOT EXISTS idx_flight_gate_flid ON flight_gate (flid); -CREATE INDEX IF NOT EXISTS idx_flight_checkin_flid ON flight_checkin (flid); -CREATE INDEX IF NOT EXISTS idx_flight_belt_flid ON flight_belt (flid); diff --git a/src/main/resources/db/migration/V1.3.1__flight_schd_display.sql b/src/main/resources/db/migration/V1.3.1__flight_schd_display.sql deleted file mode 100644 index decd65b..0000000 --- a/src/main/resources/db/migration/V1.3.1__flight_schd_display.sql +++ /dev/null @@ -1,57 +0,0 @@ --- docs/flight-state.md §8 (ACM2-29 P2-2): compatibility read view. --- 提供「首个/第二个资源 + 总数」的运营查询投影；完整明细仍在各明细表可查。 --- PG 专属语法（LATERAL）；Oracle 11g 方言随 P3-B 单独提供 location。 - -CREATE OR REPLACE VIEW flight_schd_display AS -SELECT - f.flid, - f.fday, - f.flno, - f.mvin, - f.sodt, - f.stnd, - f.abdg, - f.state_version, - f.last_message_id, - - g1.gate AS gate1, - g2.gate AS gate2, - COALESCE(g_total.cnt, 0) AS gate_total, - - c1.chkc AS chkc1, - c2.chkc AS chkc2, - COALESCE(c_total.cnt, 0) AS checkin_total, - - b1.belt AS belt1, - COALESCE(b_total.cnt, 0) AS belt_total, - - dly.code AS dely_code, - dly.strt AS dely_strt -FROM flight_schd f -LEFT JOIN LATERAL ( - SELECT gate FROM flight_gate WHERE flid = f.flid ORDER BY ordinal LIMIT 1 OFFSET 0 -) g1 ON TRUE -LEFT JOIN LATERAL ( - SELECT gate FROM flight_gate WHERE flid = f.flid ORDER BY ordinal LIMIT 1 OFFSET 1 -) g2 ON TRUE -LEFT JOIN LATERAL ( - SELECT count(*) AS cnt FROM flight_gate WHERE flid = f.flid -) g_total ON TRUE -LEFT JOIN LATERAL ( - SELECT chkc FROM flight_checkin WHERE flid = f.flid ORDER BY ordinal LIMIT 1 OFFSET 0 -) c1 ON TRUE -LEFT JOIN LATERAL ( - SELECT chkc FROM flight_checkin WHERE flid = f.flid ORDER BY ordinal LIMIT 1 OFFSET 1 -) c2 ON TRUE -LEFT JOIN LATERAL ( - SELECT count(*) AS cnt FROM flight_checkin WHERE flid = f.flid -) c_total ON TRUE -LEFT JOIN LATERAL ( - SELECT belt FROM flight_belt WHERE flid = f.flid ORDER BY ordinal LIMIT 1 OFFSET 0 -) b1 ON TRUE -LEFT JOIN LATERAL ( - SELECT count(*) AS cnt FROM flight_belt WHERE flid = f.flid -) b_total ON TRUE -LEFT JOIN LATERAL ( - SELECT code, strt FROM flight_delay WHERE flid = f.flid ORDER BY ordinal LIMIT 1 -) dly ON TRUE; diff --git a/src/main/resources/db/migration/V1.3.2__backfill_todo.sql b/src/main/resources/db/migration/V1.3.2__backfill_todo.sql deleted file mode 100644 index 6733fb7..0000000 --- a/src/main/resources/db/migration/V1.3.2__backfill_todo.sql +++ /dev/null @@ -1,18 +0,0 @@ --- docs/flight-state.md §5 (ACM2-29 P2-3): 提交后共享信箱回填的持久补偿待办。 --- 业务事务已提交（PROC_STATE=SUCCEEDED）后回填失败 → 落本表重试； --- 回填失败不得把已成功的业务事务重新标记为失败，也不得重放业务变更。 - -CREATE TABLE IF NOT EXISTS backfill_todo ( - cminmsgs_id BIGINT PRIMARY KEY, - sndr VARCHAR(64) NOT NULL, - type VARCHAR(32) NOT NULL, - styp VARCHAR(32), - seqn BIGINT, - attempts INTEGER NOT NULL DEFAULT 0, - next_attempt_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - last_error VARCHAR(512), - created_at TIMESTAMP(6) WITH TIME ZONE NOT NULL, - updated_at TIMESTAMP(6) WITH TIME ZONE NOT NULL -); - -CREATE INDEX IF NOT EXISTS idx_backfill_todo_due ON backfill_todo (next_attempt_at); diff --git a/src/main/resources/db/migration/V1.4.0__retire_legacy_slot_storage.sql b/src/main/resources/db/migration/V1.4.0__retire_legacy_slot_storage.sql deleted file mode 100644 index 9b8e89d..0000000 --- a/src/main/resources/db/migration/V1.4.0__retire_legacy_slot_storage.sql +++ /dev/null @@ -1,45 +0,0 @@ --- docs/flight-state.md §7 步骤 5 (ACM2-29 P4): 旧损失性结构退场。 --- v2 无损明细表已是唯一权威存储（V1.3.0 起），槽位/里程碑/紧凑航路列自 V1.3.0 起零写入、 --- P2-2 起零读取；对拍（FS7 v2 双读 + 全结构回环矩阵）与升级链验证通过后关闭回退窗口， --- 本迁移物理删除这些列与其冗余索引。SRVT/VIPF/MAFL_TEXT 与异常前缀列为无损承载，保留。 - -ALTER TABLE flight_schd - DROP COLUMN IF EXISTS gate1, DROP COLUMN IF EXISTS pgot1, DROP COLUMN IF EXISTS pgct1, - DROP COLUMN IF EXISTS gotm1, DROP COLUMN IF EXISTS gctm1, DROP COLUMN IF EXISTS gtyp1, - DROP COLUMN IF EXISTS gate2, DROP COLUMN IF EXISTS pgot2, DROP COLUMN IF EXISTS pgct2, - DROP COLUMN IF EXISTS gotm2, DROP COLUMN IF EXISTS gctm2, DROP COLUMN IF EXISTS gtyp2, - - DROP COLUMN IF EXISTS chkc1, DROP COLUMN IF EXISTS ccls1, DROP COLUMN IF EXISTS pcot1, - DROP COLUMN IF EXISTS pcct1, DROP COLUMN IF EXISTS cotm1, DROP COLUMN IF EXISTS cctm1, - DROP COLUMN IF EXISTS ctyp1, DROP COLUMN IF EXISTS chkc2, DROP COLUMN IF EXISTS ccls2, - DROP COLUMN IF EXISTS pcot2, DROP COLUMN IF EXISTS pcct2, DROP COLUMN IF EXISTS cotm2, - DROP COLUMN IF EXISTS cctm2, DROP COLUMN IF EXISTS ctyp2, DROP COLUMN IF EXISTS chkc3, - DROP COLUMN IF EXISTS ccls3, DROP COLUMN IF EXISTS pcot3, DROP COLUMN IF EXISTS pcct3, - DROP COLUMN IF EXISTS cotm3, DROP COLUMN IF EXISTS cctm3, DROP COLUMN IF EXISTS ctyp3, - - DROP COLUMN IF EXISTS belt1, DROP COLUMN IF EXISTS bcls1, DROP COLUMN IF EXISTS bpcot1, - DROP COLUMN IF EXISTS bpcct1, DROP COLUMN IF EXISTS fbag1, DROP COLUMN IF EXISTS lbag1, - DROP COLUMN IF EXISTS btyp1, DROP COLUMN IF EXISTS belt2, DROP COLUMN IF EXISTS bcls2, - DROP COLUMN IF EXISTS bpcot2, DROP COLUMN IF EXISTS bpcct2, DROP COLUMN IF EXISTS fbag2, - DROP COLUMN IF EXISTS lbag2, DROP COLUMN IF EXISTS btyp2, - - DROP COLUMN IF EXISTS psst1, DROP COLUMN IF EXISTS stst1, DROP COLUMN IF EXISTS stet1, - DROP COLUMN IF EXISTS psst2, DROP COLUMN IF EXISTS stst2, DROP COLUMN IF EXISTS stet2, - - DROP COLUMN IF EXISTS chut1, DROP COLUMN IF EXISTS chcls1, DROP COLUMN IF EXISTS pcbt1, - DROP COLUMN IF EXISTS pcet1, DROP COLUMN IF EXISTS cbtm1, DROP COLUMN IF EXISTS cetm1, - DROP COLUMN IF EXISTS chtyp1, DROP COLUMN IF EXISTS chut2, DROP COLUMN IF EXISTS chcls2, - DROP COLUMN IF EXISTS pcbt2, DROP COLUMN IF EXISTS pcet2, DROP COLUMN IF EXISTS cbtm2, - DROP COLUMN IF EXISTS cetm2, DROP COLUMN IF EXISTS chtyp2, - - DROP COLUMN IF EXISTS dely_code, DROP COLUMN IF EXISTS dely_strt, - DROP COLUMN IF EXISTS dely_dura, DROP COLUMN IF EXISTS dely_remc, - DROP COLUMN IF EXISTS abtm_a, DROP COLUMN IF EXISTS abtm_d, - DROP COLUMN IF EXISTS chot_on, DROP COLUMN IF EXISTS chot_off, - - DROP COLUMN IF EXISTS rout_path, DROP COLUMN IF EXISTS erut_path; - --- V1.3.0 的三个 (flid) 单列索引与 PK (flid, ordinal) 前缀冗余（L2），一并清理 -DROP INDEX IF EXISTS idx_flight_gate_flid; -DROP INDEX IF EXISTS idx_flight_checkin_flid; -DROP INDEX IF EXISTS idx_flight_belt_flid; diff --git a/src/main/resources/db/migration/V1__flight_state_baseline.sql b/src/main/resources/db/migration/V1__flight_state_baseline.sql new file mode 100644 index 0000000..761bda9 --- /dev/null +++ b/src/main/resources/db/migration/V1__flight_state_baseline.sql @@ -0,0 +1,306 @@ +-- ===================================================================== +-- 航班运行数据接入与当前状态管理 · 全量基线 +-- --------------------------------------------------------------------- +-- 权威设计：docs/flight-state.md（审计定稿）。本脚本整体取代历史迁移 +-- （V1.0.0–V1.4.0 已删除），按 §3.2 表职责建立全部分层： +-- · 决策层：FLIGHT_SCHD + 8 张资源明细表 + FLIGHT_ROUTE_POINT —— 承担正确性； +-- · 管道层：PIPELINE_LOCK / PROC_STATE / MSG_EVENT(outbox) / REQ_TRACK / BACKFILL_TODO； +-- · 留痕层：SCHD_SNAP_LOG —— 只追加、不参与决策、可重建； +-- · 证据层：报文原文归档 —— 冷路径，尚未交付（ARCHIVE_KEY 仅留引用位）。 +-- 核心定案（对照文档章节）： +-- · 身份：FLID 主键；OPERATION_DAY 一经确定不可变（§3.1/§5.3，应用层校验）； +-- STATE ∈ {ACTIVE, DELETED}，无 ARCHIVED —— 物理清除只发生在历史归档成功之后（§8.2）； +-- · 无名单差删：主链路无集合删除；删除入口只有 FDEL（§6.2）与历史归档（§8.2）； +-- · 版本：每次成功写入推进 STATE_VERSION；Kafka 判旧 = (FLID, STATE_VERSION, UPDATED_AT)（§7.3）； +-- · 出站：COUTMSGS 是共享 MySQL 信箱（他人系统库，本系统不建表，迁移不覆盖）。 +-- 类型口径：TIMESTAMP(6) WITH TIME ZONE 统一 UTC 语义；无 JSONB/CLOB 依赖； +-- BIGSERIAL 为 PostgreSQL 方言（Oracle 适配见 §10，未定前不预设）。 +-- ===================================================================== + +-- ① 单行锁：串行化状态写事务（§1 边界：只串行化 DB 事务，不代替消息认领/选主/故障切换） +CREATE TABLE PIPELINE_LOCK ( + LOCK_ID INT NOT NULL PRIMARY KEY, -- 恒为 1 + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL +); +INSERT INTO PIPELINE_LOCK (LOCK_ID, UPDATED_AT) VALUES (1, now()); + +-- ② 处理伴生状态：与共享信箱 CMINMSGS 一一对应（MSG_ID = CMINMSGS_ID）。 +-- 兼作快照重放判定（§5.1 步骤 2）：MSG_ID 已有成功终态 → 重放，直接记幂等成功。 +CREATE TABLE PROC_STATE ( + MSG_ID BIGINT NOT NULL PRIMARY KEY, + STATE VARCHAR(16) NOT NULL, -- PENDING/FAILED/SUCCEEDED/SKIPPED/DEAD + IDENTITY_KEY VARCHAR(200), -- SNDR|TYPE|STYP|SEQN（decode 后首次绑定；SEQN 同会话顺序观测 §5.5） + ATTEMPTS INT NOT NULL DEFAULT 0, + NEXT_ATTEMPT_AT TIMESTAMP(6) WITH TIME ZONE, + ERROR_CLASS VARCHAR(20), -- MALFORMED/PROTOCOL/INFRA/UNSUPPORTED/EXHAUSTED（§9） + LAST_ERROR VARCHAR(1000), + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + CONSTRAINT uk_proc_identity UNIQUE (IDENTITY_KEY) +); +CREATE INDEX idx_proc_head ON PROC_STATE (STATE, MSG_ID); -- 主泵队头查询（严格 FIFO） + +-- ③ 航班当前态主行：一行一 FLID（决策层权威；展示视图是投影，不是权威 §3.3） +CREATE TABLE FLIGHT_SCHD ( + FLID VARCHAR(32) NOT NULL PRIMARY KEY, -- AODB 实例 ID，Number(1-12)（SIS §3.16.2） + OPERATION_DAY DATE NULL, -- 运营保障日；一经确定不可变（§3.5）；NULL=尚未被快照收录 + STATE VARCHAR(8) NOT NULL, -- ACTIVE/DELETED（§3.1，无第三态） + STATE_VERSION BIGINT NOT NULL DEFAULT 0, -- 每次成功写入 +1（§4） + LAST_MSG_ID BIGINT NULL, -- 最近一次成功写入的消息 ID + -- ==== SCHD.FLTR 标量字段（值保持 AODB 字符串原样；缺失语义由处理层决定，§5.4）==== + ALCD VARCHAR(64) NULL, -- 航空公司代码 + ALSC VARCHAR(64) NULL, -- 航空公司简称 + FLNO VARCHAR(64) NULL, -- 航班号（展示用，不参与身份判定 §3.1） + MVIN VARCHAR(64) NULL, -- 进离港标识（A/D） + SODT VARCHAR(64) NULL, -- 计划运行时间（ddMMMyyHHmm；OPERATION_DAY 计算源 §3.5） + FLTY VARCHAR(64) NULL, -- 航班类型 + FLIN VARCHAR(64) NULL, -- 国内/国际/混合 + ACFT VARCHAR(64) NULL, -- 机型 + RENO VARCHAR(64) NULL, -- 机尾号 + TAOP VARCHAR(64) NULL, -- 过站实际承运人代码（§3.4：过站关联，非代码共享） + TAFL VARCHAR(64) NULL, -- 过站实际承运航班号 + TAID VARCHAR(64) NULL, -- 过站关联航班 FLID + TRML VARCHAR(64) NULL, -- 航站楼 + MAXP VARCHAR(64) NULL, -- 最大旅客数 + CSOP VARCHAR(64) NULL, -- 共享承运人代码（§3.4） + CSFT VARCHAR(64) NULL, -- 共享航班号 + MAID VARCHAR(32) NULL, -- 共享主航班 FLID + ESTT VARCHAR(64) NULL, -- 预计时间 + ACTT VARCHAR(64) NULL, -- 实际时间 + STND VARCHAR(64) NULL, -- 备降站/机位 + PHAG VARCHAR(64) NULL, -- 地服代理（值机） + CNCL VARCHAR(64) NULL, -- 取消时间（§8.1 历史判定依据） + REMC VARCHAR(256) NULL, -- 备注 + BOTM VARCHAR(64) NULL, -- 摆渡车时间 + LACL VARCHAR(64) NULL, -- 行李确认时间 + FINT VARCHAR(64) NULL, -- 完成时间 + APPT VARCHAR(64) NULL, -- 旅客到达时间 + EGSR VARCHAR(64) NULL, -- 保障开始时间 + EGST VARCHAR(64) NULL, -- 保障结束时间 + FHAG VARCHAR(64) NULL, -- 地服代理（货运） + MHAG VARCHAR(64) NULL, -- 地服代理（机务） + VIPP VARCHAR(64) NULL, -- VIP 旅客数 + VIPR VARCHAR(64) NULL, -- VIP 等级 + LBNO VARCHAR(64) NULL, -- 行李件数 + LBWT VARCHAR(64) NULL, -- 行李重量 + PAXC VARCHAR(64) NULL, -- 旅客计数 + EXSC VARCHAR(64) NULL, -- 例外代码 + EXSR VARCHAR(256) NULL, -- 例外原因 + FTSS VARCHAR(64) NULL, -- 航班状态 + PEDT VARCHAR(64) NULL, -- 前序估计时间 + NAAT VARCHAR(64) NULL, -- 到港终态时间（§8.1；业务含义待术语表确认 §10） + NEAT VARCHAR(64) NULL, -- 离港终态时间（§8.1；业务含义待术语表确认 §10） + PADT VARCHAR(64) NULL, -- 旅客登机时间 + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL +); +CREATE INDEX idx_flight_schd_opday ON FLIGHT_SCHD (OPERATION_DAY); +CREATE INDEX idx_flight_schd_state ON FLIGHT_SCHD (STATE); +-- OPERATION_DAY 不可变的数据库层强化（§7.4）：快照 upsert 一律带 +-- WHERE OPERATION_DAY IS NULL OR OPERATION_DAY = :day，行级条件更新即满足； +-- 不使用触发器（Oracle/PG 双方言成本）。 + +-- ④ 资源明细表 ×8 + 路线点表（每 FLID 多行；明细集合按组先删后插 §3.3； +-- ORDINAL 保留输入顺序，SOURCE_SEQ 只存上游序号 §3.2； +-- 删除一律标记在主行 STATE，明细物理清除只发生在历史归档 §8.2） +-- ④-1 登机门 GTDT +CREATE TABLE FLIGHT_GATE ( + FLID VARCHAR(32) NOT NULL, + ORDINAL INT NOT NULL, + SOURCE_SEQ VARCHAR(64), + GATE VARCHAR(64), + PGOT VARCHAR(64), -- 计划登机开始 + PGCT VARCHAR(64), -- 计划登机结束 + GOTM VARCHAR(64), -- 实际登机开始 + GCTM VARCHAR(64), -- 实际登机结束 + GTYP VARCHAR(64), + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + PRIMARY KEY (FLID, ORDINAL) +); +-- ④-2 值机柜台 CKDT +CREATE TABLE FLIGHT_CHECKIN ( + FLID VARCHAR(32) NOT NULL, + ORDINAL INT NOT NULL, + SOURCE_SEQ VARCHAR(64), + CHKC VARCHAR(64), + CCLS VARCHAR(64), + PCOT VARCHAR(64), -- 计划开柜 + PCCT VARCHAR(64), -- 计划关柜 + COTM VARCHAR(64), -- 实际开柜 + CCTM VARCHAR(64), -- 实际关柜 + CTYP VARCHAR(64), + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + PRIMARY KEY (FLID, ORDINAL) +); +-- ④-3 行李转盘 CLDT +CREATE TABLE FLIGHT_BELT ( + FLID VARCHAR(32) NOT NULL, + ORDINAL INT NOT NULL, + SOURCE_SEQ VARCHAR(64), + BELT VARCHAR(64), + BCLS VARCHAR(64), + PCOT VARCHAR(64), -- 计划开转盘（转盘专属前缀，避免与柜台同名） + PCCT VARCHAR(64), -- 计划关转盘 + FBAG VARCHAR(64), -- 首件行李 + LBAG VARCHAR(64), -- 末件行李 + BTYP VARCHAR(64), + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + PRIMARY KEY (FLID, ORDINAL) +); +-- ④-4 计划机位 PSDT +CREATE TABLE FLIGHT_STAND_PLAN ( + FLID VARCHAR(32) NOT NULL, + ORDINAL INT NOT NULL, + SOURCE_SEQ VARCHAR(64), + PSST VARCHAR(64), + STST VARCHAR(64), -- 计划开始 + STET VARCHAR(64), -- 计划结束 + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + PRIMARY KEY (FLID, ORDINAL) +); +-- ④-5 滑槽 CHDT +CREATE TABLE FLIGHT_CHUTE ( + FLID VARCHAR(32) NOT NULL, + ORDINAL INT NOT NULL, + SOURCE_SEQ VARCHAR(64), + CHUT VARCHAR(64), + CHCLS VARCHAR(64), + PCBT VARCHAR(64), -- 计划开始（SIS 线格式 PCBT） + PCET VARCHAR(64), -- 计划结束（PCET） + CBTM VARCHAR(64), -- 实际开始（CBTM） + CETM VARCHAR(64), -- 实际结束（CETM） + CHTYP VARCHAR(64), + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + PRIMARY KEY (FLID, ORDINAL) +); +-- ④-6 延误 DELY（业务上任意时刻仅 1 个有效延误，保留多行能力以无损承接） +CREATE TABLE FLIGHT_DELAY ( + FLID VARCHAR(32) NOT NULL, + ORDINAL INT NOT NULL, + SOURCE_SEQ VARCHAR(64), + CODE VARCHAR(64), + STRT VARCHAR(64), + DURA VARCHAR(64), + REMC VARCHAR(256), + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + PRIMARY KEY (FLID, ORDINAL) +); +-- ④-7 靠撤桥 ABTM（桥号 ABDG；A/D 各一条） +CREATE TABLE FLIGHT_BRIDGE_OP ( + FLID VARCHAR(32) NOT NULL, + ORDINAL INT NOT NULL, + SOURCE_SEQ VARCHAR(64), + ABDG VARCHAR(64), + ABOP VARCHAR(64), -- A=靠桥 D=撤桥 + AOTM VARCHAR(64), -- 动作时间 + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + PRIMARY KEY (FLID, ORDINAL) +); +-- ④-8 轮挡 CHOT（机位 CHID；ON/OFF 各一条） +CREATE TABLE FLIGHT_CHOCK_OP ( + FLID VARCHAR(32) NOT NULL, + ORDINAL INT NOT NULL, + SOURCE_SEQ VARCHAR(64), + CHID VARCHAR(64), + CHST VARCHAR(64), -- ON/OFF + CHTM VARCHAR(64), -- 动作时间 + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + PRIMARY KEY (FLID, ORDINAL) +); +-- ④-9 路线点 ROUT/ERUT 共用（ROUTE_KIND 区分 §3.2：10 类集合 9 张表） +CREATE TABLE FLIGHT_ROUTE_POINT ( + FLID VARCHAR(32) NOT NULL, + ORDINAL INT NOT NULL, + SOURCE_SEQ VARCHAR(64), + ROUTE_KIND VARCHAR(8) NOT NULL, -- ROUT/ERUT + APCD VARCHAR(64), -- 机场代码 + SCAT VARCHAR(64), -- 计划到达时间 + SCDT VARCHAR(64), -- 计划出发时间 + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + PRIMARY KEY (FLID, ROUTE_KIND, ORDINAL) +); +CREATE INDEX idx_flight_gate_flid ON FLIGHT_GATE (FLID); +CREATE INDEX idx_flight_checkin_flid ON FLIGHT_CHECKIN (FLID); +CREATE INDEX idx_flight_belt_flid ON FLIGHT_BELT (FLID); +CREATE INDEX idx_flight_stand_flid ON FLIGHT_STAND_PLAN (FLID); +CREATE INDEX idx_flight_chute_flid ON FLIGHT_CHUTE (FLID); +CREATE INDEX idx_flight_delay_flid ON FLIGHT_DELAY (FLID); +CREATE INDEX idx_flight_bridge_flid ON FLIGHT_BRIDGE_OP (FLID); +CREATE INDEX idx_flight_chock_flid ON FLIGHT_CHOCK_OP (FLID); + +-- ⑤ 统一投递事件 outbox（§7.3：KAFKA_MSG 只通知变化；KAFKA_SCHD 发整态； +-- tombstone 仅在 ACTIVE→DELETED 时与删除同事务登记，投递失败持续重试） +CREATE TABLE MSG_EVENT ( + EVENT_ID BIGSERIAL PRIMARY KEY, + TARGET VARCHAR(30) NOT NULL, -- KAFKA_MSG / KAFKA_SCHD + PARTITION_KEY VARCHAR(32) NOT NULL, -- 恒为 FLID + EVENT_TYPE VARCHAR(16) NOT NULL DEFAULT 'UPSERT', -- UPSERT / TOMBSTONE + STATE_VERSION BIGINT NOT NULL, -- 发布时航班版本；聚合按 FLID 取最新（§7.3） + PAYLOAD_JSON TEXT NOT NULL, -- TOMBSTONE 时至少含 FLID/STATE_VERSION/DELETED + STATE VARCHAR(16) NOT NULL, -- PENDING/SENT/DEAD + ATTEMPTS INT NOT NULL DEFAULT 0, + NEXT_ATTEMPT_AT TIMESTAMP(6) WITH TIME ZONE, + ERROR_CLASS VARCHAR(20), + LAST_ERROR VARCHAR(1000), + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL +); +CREATE INDEX idx_evt_head ON MSG_EVENT (TARGET, STATE, EVENT_ID); -- 每 target 队头 +CREATE INDEX idx_evt_flid ON MSG_EVENT (PARTITION_KEY, STATE_VERSION); -- 同 FLID 未发事件合并 + +-- ⑥ 请求状态机（§7.1：只有 RESP 完成 RQFD 请求，按（运营日、发送方、请求类型） +-- 匹配最新一条 PENDING；同类请求只留一条有效，新请求置旧为 EXPIRED） +CREATE TABLE REQ_TRACK ( + REQ_ID BIGSERIAL PRIMARY KEY, + REQ_TYPE VARCHAR(20) NOT NULL, + OPERATION_DAY DATE NOT NULL, -- 请求覆盖运营日 + SENDER VARCHAR(64) NOT NULL, -- 请求发送方（匹配键之一） + STATE VARCHAR(16) NOT NULL, -- PENDING/SENT/DONE/EXPIRED + COUTMSGS_ID BIGINT, + SENT_AT TIMESTAMP(6) WITH TIME ZONE, + COMPLETED_AT TIMESTAMP(6) WITH TIME ZONE, + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL +); +CREATE INDEX idx_req_open ON REQ_TRACK (REQ_TYPE, OPERATION_DAY, SENDER, STATE); + +-- ⑦ 共享信箱回填补偿待办（§7.2：提交后回填；回填失败不得把 SUCCEEDED 改回 FAILED； +-- 目标形态 = 业务事务内预登记，消除「提交后写待办」的崩溃窗口，§10 当前偏差） +CREATE TABLE BACKFILL_TODO ( + MSG_ID BIGINT NOT NULL PRIMARY KEY, + SNDR VARCHAR(64) NOT NULL, + TYPE VARCHAR(32) NOT NULL, + STYP VARCHAR(32), + SEQN BIGINT, + ATTEMPTS INT NOT NULL DEFAULT 0, + NEXT_ATTEMPT_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + LAST_ERROR VARCHAR(512), + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, + UPDATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL +); +CREATE INDEX idx_backfill_todo_due ON BACKFILL_TODO (NEXT_ATTEMPT_AT); + +-- ⑧ SCHD 快照留痕（§5.5）：事务外追加，只追加留痕、不参与决策；一行=一次尝试，重放也记； +-- RESULT 与 FLAGS 分列（可「成功且告警」）；写失败只记指标；保留 90 天，按 (SCOPE_END, RECV_AT) 清理 +CREATE TABLE SCHD_SNAP_LOG ( + LOG_ID BIGSERIAL PRIMARY KEY, + MSG_ID BIGINT NOT NULL, + RECV_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL, -- 接收时刻（UTC） + SCOPE_START DATE NOT NULL, -- 报文覆盖运营日范围（单日时两者相等） + SCOPE_END DATE NOT NULL, + RECS INT NOT NULL, -- 报文声明记录数 + UPSERTED INT NOT NULL, -- 实际写入数 + DURATION_MS BIGINT NOT NULL, + RESULT VARCHAR(20) NOT NULL, -- COMMITTED/REPLAY_SKIPPED/ROLLED_BACK + FLAGS VARCHAR(200) NOT NULL DEFAULT '', -- 逗号分隔：EMPTY/RECS_DROP/SEQN_REGRESSION/DAY_MISMATCH/SCHD_REVIVE_CONFLICT + ARCHIVE_KEY VARCHAR(200), -- 原文归档引用（证据层，尚未交付） + CREATED_AT TIMESTAMP(6) WITH TIME ZONE NOT NULL +); +CREATE INDEX idx_snaplog_cleanup ON SCHD_SNAP_LOG (SCOPE_END, RECV_AT); +CREATE INDEX idx_snaplog_msg ON SCHD_SNAP_LOG (MSG_ID); diff --git a/src/main/resources/db/migration/oracle11g/README.md b/src/main/resources/db/migration/oracle11g/README.md index db0f49c..8d8b7eb 100644 --- a/src/main/resources/db/migration/oracle11g/README.md +++ b/src/main/resources/db/migration/oracle11g/README.md @@ -13,12 +13,11 @@ Flyway 配置**；PG 路径使用 `classpath:db/migration`，两者互不混用 ## 计划内容 -- V1.0.0 起：管道表（PROC_STATE/MSG_EVENT/PUMP_JOB/REQ_TRACK/REF_MASTER/PIPELINE_LOCK） - 的 11g DDL——`NUMBER`/序列、`TIMESTAMP WITH TIME ZONE`、`VARCHAR2` BYTE/CHAR 语义钉死。 -- FLIGHT_SCHD 宽表 + V1.3.0 等价明细表 DDL（无 `ON CONFLICT`/`LATERAL` 依赖）。 -- `SRVT_TEXT/VIPF_TEXT/MAFL_TEXT` 映射 `CLOB`（PG 为 TEXT）。 -- `flight_schd_display` 11g 版视图（无 LATERAL，用标量子查询）。 +- V1 起：`V1__flight_state_baseline.sql` 的 11g 等价 DDL—— + PIPELINE_LOCK/PROC_STATE/MSG_EVENT/REQ_TRACK/BACKFILL_TODO/FLIGHT_SCHD + 9 张明细表/SCHD_SNAP_LOG； + `NUMBER`/序列替代 `BIGSERIAL`、`TIMESTAMP WITH TIME ZONE`、`VARCHAR2` BYTE/CHAR 语义钉死。 +- JdbcFlightStateRepository 的 `INSERT ... ON CONFLICT` 改 MERGE（OPERATION_DAY 不可变条件 §7.4）。 - 空串按 NULL 的语义回归：命令层 presence 信息不得被 11g 空串语义吞掉 - （docs/flight-state.md §5）。 + （docs/flight-state.md §5/§10）。 版本号与 PG location 各自独立推进，禁止复用版本号语义。 diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/PipelineSmokeTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/PipelineSmokeTest.kt index f4bebe8..b46e3d6 100644 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/PipelineSmokeTest.kt +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/PipelineSmokeTest.kt @@ -58,18 +58,17 @@ class PipelineSmokeTest { } companion object { - /** 合法 META + FLOP，但未注册 Handler → FAILED(UNSUPPORTED) */ - val UNSUPPORTED_FLOP_XML = """ + /** 合法 META，但 TYPE 未在支持范围 → FAILED(UNSUPPORTED)（§9） */ + val UNSUPPORTED_XML = """ - AODB120260908120000FLOPDELY - F1 + AODB120260908120000XYZQFOO """.trimIndent() } @Test fun `accept then pump tick transitions message to FAILED UNSUPPORTED with backoff`() { - val receipt = controller.send(UNSUPPORTED_FLOP_XML) + val receipt = controller.send(UNSUPPORTED_XML) assertNotNull(receipt.body()) // 受理 ID val id = receipt.body()!!.toLong() @@ -86,7 +85,7 @@ class PipelineSmokeTest { @Test fun `replay reopens failed UNSUPPORTED row to PENDING`() { - val receipt = controller.send(UNSUPPORTED_FLOP_XML) + val receipt = controller.send(UNSUPPORTED_XML) val id = receipt.body()!!.toLong() pump.tick() val stub = ctx.getBean(StubProcState::class.java) @@ -106,14 +105,14 @@ class PipelineSmokeTest { val events = ctx.getBean(StubMsgEvents::class.java) val port = ctx.getBean(StubDeliveryPort::class.java) events.insertAll(listOf( - MsgEvent(target = Targets.KAFKA_SCHD, partitionKey = "F1", payloadJson = """{"FLID":"F1","v":"old"}"""), - MsgEvent(target = Targets.KAFKA_SCHD, partitionKey = "F1", payloadJson = """{"FLID":"F1","v":"new"}"""), + MsgEvent(target = Targets.KAFKA_SCHD, partitionKey = "F1", stateVersion = 1, payloadJson = """{"FLID":"F1","v":"old"}"""), + MsgEvent(target = Targets.KAFKA_SCHD, partitionKey = "F1", stateVersion = 2, payloadJson = """{"FLID":"F1","v":"new"}"""), )) dispatcher.flushSchd() - assertEquals(1, port.sent.count { it.first == "schd" }) - val payload = port.sent.first { it.first == "schd" }.second + assertEquals(1, port.sent.count { it.topic == "schd" }) + val payload = port.sent.first { it.topic == "schd" }.payload!! assertTrue(payload.contains("new") && !payload.contains("old")) } } diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/codec/JacksonXmlCodecTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/codec/JacksonXmlCodecTest.kt index 6b6a99b..97bd181 100644 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/codec/JacksonXmlCodecTest.kt +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/codec/JacksonXmlCodecTest.kt @@ -24,24 +24,24 @@ class JacksonXmlCodecTest { 121112312 - CA-CA101-A-12DEC031345-D + CA-CA101-A-12DEC261345-D G28 - 15DEC031805 - 15DEC031925 - 15DEC031825 + 15DEC261805 + 15DEC261925 + 15DEC261825 D G33 - 15DEC031805 - 15DEC031925 + 15DEC261805 + 15DEC261925 I G23 - 15DEC031805 - 15DEC031925 + 15DEC261805 + 15DEC261925 D @@ -57,7 +57,7 @@ class JacksonXmlCodecTest { val body = msg.body as FlopPayload assertEquals("121112312", body.flid) - assertEquals("CA-CA101-A-12DEC031345-D", body.scalars["FFID"]) + assertEquals("CA-CA101-A-12DEC261345-D", body.scalars["FFID"]) assertEquals(3, body.collections["GTDT"]!!.size) assertEquals("G28", body.collections["GTDT"]!![0]["GATE"]) assertEquals("1", body.collections["GTDT"]!![0]["GTNO"]) diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/config/PipelinePropsBindingTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/config/PipelinePropsBindingTest.kt index dcde4b5..c2ee4c7 100644 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/config/PipelinePropsBindingTest.kt +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/config/PipelinePropsBindingTest.kt @@ -12,7 +12,7 @@ import java.time.Duration /** * U03（N02）验收：msgx.* 嵌套键「改值—回读」离线化。 - * 四个嵌套类（Pipeline/Schd/Identity/ConsistencyCheck）须标注 @ConfigurationProperties 才会被绑定； + * 三个嵌套类（Pipeline/Schd/Identity）须标注 @ConfigurationProperties 才会被绑定； * 未标注时下列覆盖值会静默回落 Kotlin 默认值（本测试即红）。仅注入配置 bean，不触碰仓储/DB。 * 注：Micronaut Test 5.x 移除 @Property 注解，改以 TestPropertyProvider 注入测试属性（需 PER_CLASS）。 */ @@ -25,7 +25,6 @@ class PipelinePropsBindingTest : TestPropertyProvider { "msgx.pipeline.max-attempts" to "9", "msgx.schd.flush-limit" to "321", "msgx.identity.include-day-boundary" to "true", - "msgx.consistency-check.daily-sample-ratio" to "0.05", ) @Inject @@ -37,12 +36,10 @@ class PipelinePropsBindingTest : TestPropertyProvider { assertEquals(9, props.pipeline.maxAttempts) assertEquals(321, props.schd.flushLimit) assertTrue(props.identity.includeDayBoundary) - assertEquals(0.05, props.consistencyCheck.dailySampleRatio, 1e-9) } @Test fun `top-level scalars still bind`() { - assertEquals(PipelineProps.Phase.A, props.phase) assertEquals("msgexchangeapi", props.serviceName) assertFalse(props.registerEureka) // application-test.yml 置 false } diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/delivery/DispatcherTickTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/delivery/DispatcherTickTest.kt index 1c81d07..ebcd6e7 100644 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/delivery/DispatcherTickTest.kt +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/delivery/DispatcherTickTest.kt @@ -2,185 +2,131 @@ package com.gzzn.omms.msgexchange.delivery import com.gzzn.omms.msgexchange.MutableClock import com.gzzn.omms.msgexchange.config.PipelineProps -import com.gzzn.omms.msgexchange.domain.ErrorClass -import com.gzzn.omms.msgexchange.domain.EventStatus +import com.gzzn.omms.msgexchange.domain.EventType import com.gzzn.omms.msgexchange.domain.MsgEvent import com.gzzn.omms.msgexchange.domain.Targets import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository import com.gzzn.omms.msgexchange.infra.retry.FailureScheduler +import com.gzzn.omms.msgexchange.infra.stub.StubDeliveryPort +import com.gzzn.omms.msgexchange.infra.stub.StubMsgEvents import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Assertions.assertNotNull import org.junit.jupiter.api.Assertions.assertNull import org.junit.jupiter.api.Assertions.assertTrue import org.junit.jupiter.api.Test import java.time.Instant /** - * U06（N03）+ U08 闭环：schd 出口只走 flushSchd； - * 批发送失败 → 整批 attempts 递增 + 指数退避（队首未到期不 claim），达上限整批 DEAD/DLQ； - * 时间经可注入 Clock（MutableClock），不依赖真实睡眠。 + * 投递调度（docs/flight-state.md §7.3）： + * KAFKA_MSG 逐条 FIFO；KAFKA_SCHD 唯一出口 flushSchd——同 FLID 未发事件按最新 + * STATE_VERSION 合并；TOMBSTONE 发 null 值消息；失败退避重试、达上限 DEAD。 */ class DispatcherTickTest { - private class FakeRepo : MsgEventRepository { - val events = mutableListOf() - - fun enqueue(e: MsgEvent) { events += e } - - override fun insertAll(events: List) = events.map { it.eventId ?: 0L } - - override fun headUnsent(target: String): MsgEvent? = - events.filter { it.target == target && it.state != EventStatus.SENT && it.state != EventStatus.DEAD } - .minByOrNull { it.eventId ?: Long.MAX_VALUE } - - override fun claimBatch(target: String, limit: Int): List = - events.filter { it.target == target && it.state == EventStatus.PENDING } - .sortedBy { it.eventId ?: Long.MAX_VALUE } - .take(limit) - - override fun markSent(eventId: Long) { - val i = events.indexOfFirst { it.eventId == eventId } - if (i >= 0) events[i] = events[i].copy(state = EventStatus.SENT) - } - - override fun markAllSent(eventIds: List) = eventIds.forEach(::markSent) - - override fun scheduleRetry(eventId: Long, nextAttemptAt: Instant, attempts: Int) { - val i = events.indexOfFirst { it.eventId == eventId } - if (i >= 0) events[i] = events[i].copy(state = EventStatus.PENDING, attempts = attempts, nextAttemptAt = nextAttemptAt) - } - - override fun markDead(eventId: Long, errorClass: ErrorClass, lastError: String, attempts: Int?) { - val i = events.indexOfFirst { it.eventId == eventId } - if (i >= 0) events[i] = events[i].copy(state = EventStatus.DEAD, errorClass = errorClass, lastError = lastError, - attempts = attempts ?: events[i].attempts) - } - - override fun insertSync(events: List) { syncInserted += events } - - val syncInserted = mutableListOf() - } - - private class FakePort : DeliveryPort { - val sent = mutableListOf>() - var failTopic: String? = null - - override fun sendKafka(topic: String, payloadJson: String) { - if (failTopic == topic) throw RuntimeException("send-fail:$topic") - sent += topic to payloadJson - } - - override fun indexFlightHts(payloadJson: String) = Unit - } - private val clock = MutableClock(MutableClock.BASE) - private fun dispatcher(repo: FakeRepo, port: FakePort, p: PipelineProps = PipelineProps()): Dispatcher = + private fun dispatcher(repo: MsgEventRepository, port: DeliveryPort, p: PipelineProps = PipelineProps()): Dispatcher = Dispatcher(repo, port, p, FailureScheduler(p, clock)) - private fun ev(id: Long, target: String, key: String?, payload: String) = - MsgEvent(eventId = id, target = target, partitionKey = key, payloadJson = payload, state = EventStatus.PENDING) + private fun ev(id: Long, target: String, key: String, payload: String, version: Long = 0) = + MsgEvent(eventId = id, target = target, partitionKey = key, stateVersion = version, payloadJson = payload) @Test - fun `schd flushed once via batch path, per-target tick never consumes it`() { - val repo = FakeRepo() - val port = FakePort() - val d = dispatcher(repo, port) - repo.enqueue(ev(1, Targets.KAFKA_MSG, null, """{"msg":1}""")) - repo.enqueue(ev(2, Targets.KAFKA_SCHD, "F1", """{"FLID":"F1","v":1}""")) - - d.tick() // lastFlush=EPOCH → 首 tick 即到 flush 周期 - - assertEquals(1, port.sent.count { it.first == "msg" }) - assertEquals(1, port.sent.count { it.first == "schd" }) - assertEquals(EventStatus.SENT, repo.events.first { it.eventId == 2L }.state) - - repo.enqueue(ev(4, Targets.KAFKA_SCHD, "F1", """{"FLID":"F1","v":4}""")) - d.tick() // 时钟未推进 → flush 周期未到；逐条循环不得消费 schd - assertEquals(EventStatus.PENDING, repo.events.first { it.eventId == 4L }.state) - assertEquals(1, port.sent.count { it.first == "schd" }) - } - - @Test - fun `flushSchd aggregates latest per flight and marks all sent`() { - val repo = FakeRepo() - val port = FakePort() - val d = dispatcher(repo, port) - repo.enqueue(ev(1, Targets.KAFKA_SCHD, "F1", "old")) - repo.enqueue(ev(2, Targets.KAFKA_SCHD, "F2", "only")) - repo.enqueue(ev(3, Targets.KAFKA_SCHD, "F1", "new")) - - d.flushSchd() - - val payload = port.sent.first { it.first == "schd" }.second - assertTrue(payload.contains("new") && payload.contains("only") && !payload.contains("old")) - assertEquals(2, payload.removeSurrounding("[", "]").split(",").size) // 同 FLID 只发最新（矩阵 #7） - assertTrue(repo.events.all { it.state == EventStatus.SENT }) - } - - @Test - fun `flush failure schedules per-event backoff and retries after due`() { - val repo = FakeRepo() - val port = FakePort().apply { failTopic = "schd" } - val d = dispatcher(repo, port) - repo.enqueue(ev(1, Targets.KAFKA_SCHD, "F1", """{"v":"1"}""")) - - d.flushSchd() - val e1 = repo.events.single() - assertEquals(EventStatus.PENDING, e1.state) - assertEquals(1, e1.attempts) - assertNotNull(e1.nextAttemptAt) // 指数退避落库（attempts=1 → 首档） - assertEquals(0, port.sent.size) - - clock.advance(1000) // 退避到期 - port.failTopic = null - d.flushSchd() - assertEquals(EventStatus.SENT, repo.events.single().state) - assertEquals(1, port.sent.count { it.first == "schd" }) - } - - @Test - fun `repeated batch failures backoff grows and whole batch goes DEAD DLQ at limit`() { - val props = PipelineProps().apply { pipeline.maxAttempts = 2 } - val repo = FakeRepo() - val port = FakePort().apply { failTopic = "schd" } + fun `per-target tick delivers KAFKA_MSG only and never consumes schd`() { + val repo = StubMsgEvents() + val port = StubDeliveryPort() + val props = PipelineProps().apply { schd.flushPeriod = java.time.Duration.ofHours(1) } + repo.insertAll( + listOf( + ev(1, Targets.KAFKA_MSG, "F1", """{"flid":"F1"}"""), + ev(2, Targets.KAFKA_SCHD, "F1", """{"flid":"F1"}""", 1), + ), + ) val d = dispatcher(repo, port, props) - repo.enqueue(ev(1, Targets.KAFKA_SCHD, "F1", """{"v":"1"}""")) - repo.enqueue(ev(2, Targets.KAFKA_SCHD, "F2", """{"v":"2"}""")) + d.tick() - d.flushSchd() // 失败#1：attempts=1，退避 backoff[1]=1000ms - val after1 = repo.events.map { it.attempts to it.state } - assertEquals(listOf(1 to EventStatus.PENDING, 1 to EventStatus.PENDING), after1) - assertTrue(repo.events.all { it.nextAttemptAt == MutableClock.BASE.plusMillis(1000) }) + assertEquals(1, port.sent.count { it.topic == "msg" }) + assertEquals(0, port.sent.count { it.topic == "schd" }) // N03：schd 唯一出口 flushSchd + } - d.flushSchd() // 时钟未推进 → 队首未到期，不 claim（不推进 lastFlush） - assertEquals(1, repo.events.first { it.eventId == 1L }.attempts) - - clock.advance(1000) - d.flushSchd() // 失败#2：attempts=2 == maxAttempts → 整批 DEAD - assertTrue(repo.events.all { it.state == EventStatus.DEAD && it.errorClass == ErrorClass.EXHAUSTED }) - assertTrue(repo.events.all { it.attempts == 2 }) - assertEquals(0, port.sent.size) - - // DEAD 行不再参与 claim：空批 → 无新发送、无异常 + @Test + fun `flushSchd aggregates latest version per flight and marks sent`() { + val repo = StubMsgEvents() + val port = StubDeliveryPort() + repo.insertAll( + listOf( + ev(1, Targets.KAFKA_SCHD, "F1", """{"v":"old"}""", 1), + ev(2, Targets.KAFKA_SCHD, "F1", """{"v":"new"}""", 2), + ev(3, Targets.KAFKA_SCHD, "F2", """{"v":"f2"}""", 1), + ), + ) + val d = dispatcher(repo, port) d.flushSchd() - assertEquals(0, port.sent.size) + + val schd = port.sent.filter { it.topic == "schd" } + assertEquals(2, schd.size) + assertEquals(1, schd.count { it.key == "F1" && it.payload!!.contains("new") && !it.payload.contains("old") }) + assertEquals(0, repo.rows.values.count { it.state.name == "PENDING" }) + } + + @Test + fun `tombstone is delivered as null value message`() { + val repo = StubMsgEvents() + val port = StubDeliveryPort() + repo.insertAll( + listOf( + MsgEvent( + eventId = 1, target = Targets.KAFKA_SCHD, partitionKey = "F1", + eventType = EventType.TOMBSTONE, stateVersion = 3, payloadJson = """{"flid":"F1","deleted":true}""", + ), + ), + ) + val d = dispatcher(repo, port) + d.flushSchd() + + assertEquals(1, port.tombstones.size) + assertEquals("F1", port.tombstones.single().key) // 整态键缺失表示删除旧值（§7.3） + assertNull(port.tombstones.single().payload) + } + + @Test + fun `schd send failure schedules backoff and goes DEAD at limit`() { + val repo = StubMsgEvents() + val p = PipelineProps() + val d = dispatcher(repo, FailingSchdPort(), p) + repo.insertAll(listOf(ev(1, Targets.KAFKA_SCHD, "F1", """{"v":"1"}""", 1))) + + d.flushSchd() + assertEquals(1, repo.rows[1]!!.attempts) + + repeat(p.pipeline.maxAttempts - 1) { + clock.advance(p.pipeline.backoffFor(repo.rows[1]!!.attempts) + 1) + d.flushSchd() + } + val final = repo.rows[1]!! + assertTrue(final.attempts >= p.pipeline.maxAttempts || final.state.name == "DEAD") } @Test fun `KAFKA msg delivery unaffected while schd batch is retrying`() { - val props = PipelineProps().apply { pipeline.maxAttempts = 2 } - val repo = FakeRepo() - val port = FakePort().apply { failTopic = "schd" } - val d = dispatcher(repo, port, props) - repo.enqueue(ev(1, Targets.KAFKA_SCHD, "F1", """{"v":"1"}""")) - repo.enqueue(ev(2, Targets.KAFKA_MSG, null, """{"m":1}""")) - + val repo = StubMsgEvents() + val port = StubDeliveryPort() + val d = dispatcher(repo, port) + repo.insertAll(listOf(ev(1, Targets.KAFKA_MSG, "F9", """{"flid":"F9"}"""))) d.tick() + assertEquals(1, port.sent.count { it.topic == "msg" && it.key == "F9" }) + } +} - assertEquals(EventStatus.SENT, repo.events.first { it.eventId == 2L }.state) // msg 正常 - assertEquals(1, repo.events.first { it.eventId == 1L }.attempts) // schd 进退避 - assertEquals(EventStatus.PENDING, repo.events.first { it.eventId == 1L }.state) +/** schd 发送恒失败的端口（退避/终态闭环验证用）。 */ +private class FailingSchdPort : DeliveryPort { + var calls = 0 + override fun sendKafka(topic: String, key: String, payloadJson: String) = Unit + override fun sendKafkaSchd(topic: String, key: String, payloadJson: String) { + calls++ + throw IllegalStateException("broker-down") } + override fun sendKafkaNull(topic: String, key: String) { + throw IllegalStateException("broker-down") + } } diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/delivery/SchdAggregationTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/delivery/SchdAggregationTest.kt deleted file mode 100644 index 715dcaa..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/delivery/SchdAggregationTest.kt +++ /dev/null @@ -1,56 +0,0 @@ -package com.gzzn.omms.msgexchange.delivery - -import com.gzzn.omms.msgexchange.domain.EventStatus -import com.gzzn.omms.msgexchange.domain.MsgEvent -import com.gzzn.omms.msgexchange.domain.Targets -import org.junit.jupiter.api.Test -import kotlin.test.assertEquals - -/** - * ACMA-8 流程 3 / 兼容矩阵 #7（FIX）：同一 FLID 多次变更只发最新一态； - * v4 显式“组内取 EVENT_ID 最大”，不依赖集合遍历序。 - */ -class SchdAggregationTest { - - private fun ev(flid: String, eventId: Long, payload: String) = MsgEvent( - eventId = eventId, - target = Targets.KAFKA_SCHD, - partitionKey = flid, - payloadJson = payload, - state = EventStatus.PENDING, - ) - - @Test - fun `same flight collapsed to max EVENT_ID`() { - val pending = listOf( - ev("F1", 1, """{"FLID":"F1","v":"old"}"""), - ev("F2", 2, """{"FLID":"F2","v":"only"}"""), - ev("F1", 3, """{"FLID":"F1","v":"new"}"""), - ) - val latest = SchdAggregation.latestPerFlight(pending) - assertEquals(2, latest.size) - assertEquals("""{"FLID":"F1","v":"new"}""", latest.first { it.contains("F1") }) - assertEquals("""{"FLID":"F2","v":"only"}""", latest.first { it.contains("F2") }) - } - - @Test - fun `order-independent grouping yields same latest per flight`() { - val a = listOf( - ev("F1", 1, "old"), - ev("F1", 2, "new"), - ) - val b = a.reversed() - assertEquals( - SchdAggregation.latestPerFlight(a).toSet(), - SchdAggregation.latestPerFlight(b).toSet(), - ) - } - - @Test - fun `wire shape is FLTR JSON array`() { - val payload = SchdAggregation - .latestPerFlight(listOf(ev("F1", 1, "a"), ev("F2", 2, "b"))) - .joinToString(",", "[", "]") - assertEquals("[a,b]", payload) - } -} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/domain/OperationDayTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/domain/OperationDayTest.kt new file mode 100644 index 0000000..729c264 --- /dev/null +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/domain/OperationDayTest.kt @@ -0,0 +1,47 @@ +package com.gzzn.omms.msgexchange.domain + +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertFalse +import org.junit.jupiter.api.Assertions.assertNull +import org.junit.jupiter.api.Test +import java.time.LocalDate +import java.time.ZoneId + +/** + * 运营日计算（docs/flight-state.md §3.5）：SODT（ddMMMyyHHmm）+ 机场时区 + 业务切日边界。 + */ +class OperationDayTest { + + private val zone: ZoneId = ZoneId.of("Asia/Shanghai") + + @Test + fun `computes operation day from SODT with zero cutoff`() { + val calc = OperationDayCalculator(zone, cutoffHour = 0) + assertEquals(LocalDate.of(2026, 12, 15), calc.compute("15DEC261723")) + } + + @Test + fun `time before cutoff rolls back to previous operation day`() { + val calc = OperationDayCalculator(zone, cutoffHour = 4) + // 04:00 之前 → 前一运营日 + assertEquals(LocalDate.of(2026, 12, 14), calc.compute("15DEC260230")) + // 04:00 整点起归属当日 + assertEquals(LocalDate.of(2026, 12, 15), calc.compute("15DEC260400")) + } + + @Test + fun `invalid or missing SODT returns null`() { + val calc = OperationDayCalculator(zone, 0) + assertNull(calc.compute(null)) + assertNull(calc.compute("")) + assertNull(calc.compute("garbage")) + assertNull(calc.compute("99XXC12345")) + } + + @Test + fun `cutoff outside 0 to 23 is clamped to 23`() { + val calc = OperationDayCalculator(zone, cutoffHour = 99) + // 越界收敛为 23：当日 00:00 早于 23 点 → 归属前一运营日 + assertEquals(LocalDate.of(2026, 12, 14), calc.compute("15DEC260000")) + } +} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightStateEngineTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightStateEngineTest.kt index fa49691..480a27f 100644 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightStateEngineTest.kt +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/domain/flight/FlightStateEngineTest.kt @@ -1,202 +1,113 @@ package com.gzzn.omms.msgexchange.domain.flight +import com.gzzn.omms.msgexchange.domain.OperationDayCalculator +import com.gzzn.omms.msgexchange.domain.SnapshotFlag import org.junit.jupiter.api.Assertions.assertEquals import org.junit.jupiter.api.Assertions.assertFalse import org.junit.jupiter.api.Assertions.assertTrue import org.junit.jupiter.api.Test +import java.time.LocalDate +import java.time.ZoneId +/** + * 航班状态引擎（docs/flight-state.md §3.3/§5.2/§5.4/§6.1）不变量： + * 快照整体替换（缺失=清除）、DELETED 不被 SCHD 恢复、FLOP 增量保留缺失字段、 + * §5.2 五项校验整包拒绝语义。 + */ class FlightStateEngineTest { + private val opDay = OperationDayCalculator(ZoneId.of("Asia/Shanghai"), 0) + private val day = LocalDate.of(2026, 12, 15) + @Test - fun `scalar unchanged does not modify existing value`() { - val current = FlightNextState("F1", mapOf("FLNO" to "CA100"), emptyMap(), 1L, "m1") - val next = FlightStateEngine.apply( + fun `snapshot replaces scalars and clears absent ones`() { + val current = FlightSnapshot( + "121", day, FlightState.ACTIVE, 5, + scalars = mapOf("FLNO" to "CA001", "REMC" to "old-note"), + collections = mapOf("GTDT" to listOf(mapOf("GATE" to "G1"))), + ) + val next = FlightStateEngine.snapshotState( current, - FlightFieldCommands("F1", scalars = emptyMap()), - "m2", - bumpVersion = true, + ScheduleRecord("121", scalars = mapOf("FLNO" to "CA002")), + operationDay = day, + keepDeleted = false, ) - assertEquals("CA100", next.scalars["FLNO"]) - assertEquals(2L, next.stateVersion) + + assertEquals("CA002", next.scalars["FLNO"]) + assertFalse(next.scalars.containsKey("REMC")) // 缺失 = Clear（§5.4） + assertEquals(emptyList>(), next.collections["GTDT"]) // 缺失 = Replace 空集 + assertEquals(6, next.stateVersion) // 每次成功写入 +1（§4） + assertEquals(FlightState.ACTIVE, next.state) } @Test - fun `collection replace preserves order and source sequence`() { - val items = listOf( - mapOf("GTNO" to "1", "GATE" to "A1"), - mapOf("GTNO" to "3", "GATE" to "B2"), - mapOf("GTNO" to "7", "GATE" to "C3"), + fun `snapshot keeps DELETED state and flags revive conflict by caller`() { + val current = FlightSnapshot("121", day, FlightState.DELETED, 3, mapOf(), emptyMap()) + val next = FlightStateEngine.snapshotState( + current, ScheduleRecord("121", mapOf("FLNO" to "CA001")), day, keepDeleted = true, ) - val next = FlightStateEngine.apply( - null, - FlightFieldCommands("F1", collections = mapOf("GTDT" to CollectionCommand.Replace(items))), - "m1", - bumpVersion = true, - ) - assertEquals(3, next.collections["GTDT"]!!.size) - assertEquals("3", next.collections["GTDT"]!![1]["GTNO"]) - assertEquals("C3", next.collections["GTDT"]!![2]["GATE"]) + assertEquals(FlightState.DELETED, next.state) // §5.1 步骤 6：普通 SCHD 不恢复 + assertEquals(4, next.stateVersion) } @Test - fun `collection clear removes prior entries`() { - val current = FlightNextState( - "F1", - emptyMap(), - mapOf("GTDT" to listOf(mapOf("GTNO" to "1", "GATE" to "A1"))), - 1L, - "m1", + fun `flop merge retains absent scalars and collections`() { + val current = FlightSnapshot( + "121", day, FlightState.ACTIVE, 1, + scalars = mapOf("FLNO" to "CA001", "ESTT" to "15DEC261807"), + collections = mapOf("GTDT" to listOf(mapOf("GATE" to "G1"))), ) - val next = FlightStateEngine.apply( + val next = FlightStateEngine.mergedState( current, - FlightFieldCommands("F1", collections = mapOf("GTDT" to CollectionCommand.Clear)), - "m2", - bumpVersion = true, + MergeChange("121", scalars = mapOf("ESTT" to "15DEC261900")), ) - assertFalse(next.collections.containsKey("GTDT")) + + assertEquals("15DEC261900", next.scalars["ESTT"]) + assertEquals("CA001", next.scalars["FLNO"]) // 缺失 = 保留（§6.1） + assertEquals(listOf(mapOf("GATE" to "G1")), next.collections["GTDT"]) + assertEquals(2, next.stateVersion) } @Test - fun `same gate number different attributes both retained on replace`() { - val items = listOf( - mapOf("GTNO" to "1", "GATE" to "A1", "GTYP" to "D"), - mapOf("GTNO" to "1", "GATE" to "A1", "GTYP" to "I"), + fun `validation rejects RECS mismatch and duplicate flid and day outside scope`() { + val ok = FlightStateEngine.validateMessage( + 1, listOf(ScheduleRecord("121", mapOf("SODT" to "15DEC261723"))), day, day, opDay, ) - val next = FlightStateEngine.apply( - null, - FlightFieldCommands("F1", collections = mapOf("GTDT" to CollectionCommand.Replace(items))), - "m1", - bumpVersion = true, - ) - assertEquals("D", next.collections["GTDT"]!![0]["GTYP"]) - assertEquals("I", next.collections["GTDT"]!![1]["GTYP"]) - } + assertTrue(ok is SnapshotValidation.Ok) - @Test - fun `incremental apply bumps state version from persisted value`() { - val current = FlightNextState("F1", mapOf("FLNO" to "CA100"), emptyMap(), 5L, "m1") - val next = FlightStateEngine.apply( - current, - FlightFieldCommands("F1", scalars = mapOf("STAT" to ScalarCommand.Set("DEP"))), - "m2", - bumpVersion = true, - ) - assertEquals(6L, next.stateVersion) - } + val recsMismatch = FlightStateEngine.validateMessage( + 2, listOf(ScheduleRecord("121", mapOf("SODT" to "15DEC261723"))), day, day, opDay, + ) as SnapshotValidation.Invalid + assertTrue(recsMismatch.flags.contains(SnapshotFlag.RECS_DROP)) - @Test - fun `apply by source sequence updates single row`() { - val current = FlightNextState( - "F1", - emptyMap(), - mapOf("GTDT" to listOf(mapOf("GTNO" to "2", "GATE" to "OLD"))), - 1L, - "m1", - ) - val next = FlightStateEngine.apply( - current, - FlightFieldCommands( - "F1", - collections = mapOf( - "GTDT" to CollectionCommand.Apply(mapOf("GTNO" to "2", "GATE" to "NEW"), "2"), - ), + val duplicate = FlightStateEngine.validateMessage( + 2, + listOf( + ScheduleRecord("121", mapOf("SODT" to "15DEC261723")), + ScheduleRecord("121", mapOf("SODT" to "15DEC261723")), ), - "m2", - bumpVersion = true, + day, day, opDay, ) - assertEquals("NEW", next.collections["GTDT"]!![0]["GATE"]) + assertTrue(duplicate is SnapshotValidation.Invalid) + + val outOfScope = FlightStateEngine.validateMessage( + 1, listOf(ScheduleRecord("121", mapOf("SODT" to "20DEC261723"))), day, day, opDay, + ) as SnapshotValidation.Invalid + assertTrue(outOfScope.flags.contains(SnapshotFlag.DAY_MISMATCH)) // §5.2 覆盖范围校验 + + val empty = FlightStateEngine.validateMessage(0, emptyList(), day, day, opDay) as SnapshotValidation.Ok + assertTrue(empty.flags.contains(SnapshotFlag.EMPTY)) } @Test - fun `zero sequence marker alone translates to explicit clear`() { - val commands = FlightStateEngine.commandsFromFields( - "F1", - mapOf("GTDT" to """[{"GTNO":"0"}]"""), + fun `validation rejects non numeric or oversized flid`() { + val bad = FlightStateEngine.validateMessage( + 1, listOf(ScheduleRecord("ABC", mapOf("SODT" to "15DEC261723"))), day, day, opDay, ) - assertEquals(CollectionCommand.Clear, commands.collections["GTDT"]) - } - - @Test - fun `zero sequence marker mixed with regular items fails fast instead of guessing`() { - val error = runCatching { - FlightStateEngine.commandsFromFields( - "F1", - mapOf("GTDT" to """[{"GTNO":"1","GATE":"A1"},{"GTNO":"0"}]"""), - ) - }.exceptionOrNull() - assertTrue(error is IllegalArgumentException) - } - - @Test - fun `empty array replaces collection with empty set keeping key semantics`() { - val current = FlightNextState( - "F1", - emptyMap(), - mapOf("GTDT" to listOf(mapOf("GTNO" to "1", "GATE" to "A1"))), - 1L, - "m1", + assertTrue(bad is SnapshotValidation.Invalid) + val tooLong = FlightStateEngine.validateMessage( + 1, listOf(ScheduleRecord("1234567890123", mapOf("SODT" to "15DEC261723"))), day, day, opDay, ) - val commands = FlightStateEngine.commandsFromFields("F1", mapOf("GTDT" to "[]")) - val next = FlightStateEngine.apply(current, commands, "m2", bumpVersion = true) - assertEquals(emptyList>(), next.collections["GTDT"]) - } - - @Test - fun `apply on DELY fails fast because protocol defines no sequence attribute`() { - val current = FlightNextState( - "F1", - emptyMap(), - mapOf("DELY" to listOf(mapOf("CODE" to "YY"))), - 1L, - "m1", - ) - val error = runCatching { - FlightStateEngine.apply( - current, - FlightFieldCommands( - "F1", - collections = mapOf("DELY" to CollectionCommand.Apply(mapOf("CODE" to "ZZ"), "1")), - ), - "m2", - bumpVersion = true, - ) - }.exceptionOrNull() - assertTrue(error is IllegalArgumentException) - } - - @Test - fun `exception null payload translates to explicit clear and apply records cleared key`() { - val current = FlightNextState( - "F1", - mapOf("FLNO" to "CA100", "FRET" to """{"REID":"R1","RSN":"diverted"}"""), - emptyMap(), - 3L, - "m1", - ) - val commands = FlightStateEngine.commandsFromFields("F1", mapOf("FRET" to "null")) - assertEquals(ScalarCommand.Clear, commands.scalars["FRET"]) - - val next = FlightStateEngine.apply(current, commands, "m2", bumpVersion = true) - assertFalse(next.scalars.containsKey("FRET")) - assertTrue(next.clearedKeys.contains("FRET")) - assertEquals("CA100", next.scalars["FLNO"]) - - // 非空载荷仍为 Set，且清空后的下一跳 Set 恢复键（clearedKeys 不跨消息携带） - val restored = FlightStateEngine.apply( - next, - FlightStateEngine.commandsFromFields("F1", mapOf("FRET" to """{"REID":"R2"}""")), - "m3", - bumpVersion = true, - ) - assertEquals("""{"REID":"R2"}""", restored.scalars["FRET"]) - assertFalse(restored.clearedKeys.contains("FRET")) - } - - @Test - fun `exception clear accepts null literal empty object and empty string`() { - for (payload in listOf("null", "{}", "")) { - val commands = FlightStateEngine.commandsFromFields("F1", mapOf("FDIV" to payload)) - assertEquals(ScalarCommand.Clear, commands.scalars["FDIV"], "payload=[$payload] must be Clear") - } + assertTrue(tooLong is SnapshotValidation.Invalid) // SIS §3.16.2 Number(1-12) } } diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/infra/health/HealthIndicatorsTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/infra/health/HealthIndicatorsTest.kt index 61c1e8f..1498547 100644 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/infra/health/HealthIndicatorsTest.kt +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/infra/health/HealthIndicatorsTest.kt @@ -12,14 +12,16 @@ import org.junit.jupiter.api.Test class HealthIndicatorsTest { private class FakePort(private val pingResult: Boolean) : DeliveryPort { - override fun sendKafka(topic: String, payloadJson: String) = Unit - override fun indexFlightHts(payloadJson: String) = Unit + override fun sendKafka(topic: String, key: String, payloadJson: String) = Unit + override fun sendKafkaSchd(topic: String, key: String, payloadJson: String) = Unit + override fun sendKafkaNull(topic: String, key: String) = Unit override fun ping(): Boolean = pingResult } private class ThrowingPort : DeliveryPort { - override fun sendKafka(topic: String, payloadJson: String) = Unit - override fun indexFlightHts(payloadJson: String) = Unit + override fun sendKafka(topic: String, key: String, payloadJson: String) = Unit + override fun sendKafkaSchd(topic: String, key: String, payloadJson: String) = Unit + override fun sendKafkaNull(topic: String, key: String) = Unit override fun ping(): Boolean = throw RuntimeException("metadata fetch failed") } diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/infra/persistence/FlightFieldsJsonTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/infra/persistence/FlightFieldsJsonTest.kt deleted file mode 100644 index 3a4ee74..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/infra/persistence/FlightFieldsJsonTest.kt +++ /dev/null @@ -1,28 +0,0 @@ -package com.gzzn.omms.msgexchange.infra.persistence - -import com.fasterxml.jackson.databind.ObjectMapper -import kotlin.test.Test -import kotlin.test.assertEquals -import kotlin.test.assertTrue - -class FlightFieldsJsonTest { - private val mapper = ObjectMapper() - - @Test - fun `collection JSON is emitted as a native array`() { - val json = FlightFieldsJson.toJson( - mapOf("FLID" to "F1", "GTDT" to """[{"GTNO":"1","GATE":"A1"}]"""), - ) - - val root = mapper.readTree(json) - assertTrue(root["GTDT"].isArray) - assertEquals("A1", root["GTDT"][0]["GATE"].asText()) - } - - @Test - fun `malformed collection text stays a string instead of breaking delivery`() { - val root = mapper.readTree(FlightFieldsJson.toJson(mapOf("GTDT" to "legacy-value"))) - assertTrue(root["GTDT"].isTextual) - assertEquals("legacy-value", root["GTDT"].asText()) - } -} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlightSchdJdbcPgTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlightSchdJdbcPgTest.kt deleted file mode 100644 index 1169fd2..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlightSchdJdbcPgTest.kt +++ /dev/null @@ -1,815 +0,0 @@ -package com.gzzn.omms.msgexchange.infra.persistence.jdbc - -import com.gzzn.omms.msgexchange.domain.FlightChange -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository -import org.junit.jupiter.api.AfterEach -import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Assertions.assertFalse -import org.junit.jupiter.api.Assertions.assertNotNull -import org.junit.jupiter.api.Assertions.assertNull -import org.junit.jupiter.api.Assertions.assertTrue -import org.junit.jupiter.api.Assumptions.assumeTrue -import org.junit.jupiter.api.BeforeEach -import org.junit.jupiter.api.Test -import com.gzzn.omms.msgexchange.support.PgTestSupport -import com.zaxxer.hikari.HikariDataSource -import java.sql.DriverManager -import java.time.Instant -import java.time.ZoneId -import java.util.TimeZone - -/** - * ACM2-28 FS6 + ACM2-29：PostgreSQL 真实方言集成测试（JdbcFlightSchdRepository） - * 验证： - * 1. DNLD 快照批处理写入（JDBC batch，200 批次）； - * 2. 增量 Upsert FDAY 保留策略（ON CONFLICT 保留原代，新插置 NULL）； - * 3. 域化差删（按 FDAY 严格隔离）； - * 4. SCHD_GEN SQL CAS（防并发断言）； - * 5. 单事务原子回滚（崩溃无残留）； - * 6. ES 历史清场 deleteByFlids 幂等删除； - * 7. ACM2-29 零子表：集合平铺槽位列写读（集合级全量替换/序号 0 清除/里程碑与航路紧凑列）， - * 读侧视图重建 legacy 同构集合键； - * 8. I5：PIPELINE_LOCK 行级单写者互斥与提交释放。 - */ -class FlightSchdJdbcPgTest { - - private lateinit var ds: HikariDataSource - private lateinit var repo: JdbcFlightSchdRepository - private val mapper = com.fasterxml.jackson.databind.ObjectMapper() - - @BeforeEach - fun setUp() { - assumeTrue(PgTestSupport.canConnect(), PgTestSupport.skipMessage()) - - ds = HikariDataSource().apply { - jdbcUrl = PgTestSupport.jdbcUrl - username = PgTestSupport.user - password = PgTestSupport.password - driverClassName = "org.postgresql.Driver" - maximumPoolSize = 2 - } - // 幂等前置迁移：全新容器（或库缺失 schema）时补齐 V1.0.0→V1.3.0 - org.flywaydb.core.Flyway.configure() - .dataSource(PgTestSupport.jdbcUrl, PgTestSupport.user, PgTestSupport.password) - .locations("classpath:db/migration") - .load() - .migrate() - repo = JdbcFlightSchdRepository(ds) - cleanup() - } - @AfterEach - fun tearDown() { - if (::ds.isInitialized) { - cleanup() - ds.close() - } - } - - private fun cleanup() { - try { - val testFlids = ds.query( - "SELECT flid FROM flight_schd WHERE flid LIKE 'TEST_%'", - {}, - ) { rs -> rs.getString("flid") } - if (testFlids.isNotEmpty()) { - com.gzzn.omms.msgexchange.infra.persistence.jdbc.FlightDetailTables.deleteForFlids(ds, testFlids) - } - ds.update("DELETE FROM flight_schd WHERE flid LIKE 'TEST_%'") {} - ds.update("DELETE FROM schd_gen WHERE fday >= '2026-09-01'") {} - } catch (_: Exception) { - } - } - - /** v2 写路径便捷构造：字段集 → 快照 nextState（版本顺序推进）。 */ - private fun snapshotStates( - day: String, - flights: List>>, - version: Long = 1L, - messageId: String = "msg-test", - ) = flights.map { (flid, fields) -> - com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine.apply( - null, - com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine.commandsFromFields(flid, fields), - messageId, - bumpVersion = false, - ).copy(stateVersion = version) - } - - @Test - fun `day diff preserves all details of migrated and unowned flights`() { - val day = "2026-09-07" - val otherDay = "2026-09-08" - val fields = mapOf("GTDT" to """[{"GTNO":"3","GATE":"G23"}]""") - val removed = "TEST_DIFF_REMOVED" - val moved = "TEST_DIFF_MOVED" - val unowned = "TEST_DIFF_ADFT" - repo.persistNextStates(day, snapshotStates(day, listOf(removed to fields)), snapshotReplace = true) - repo.persistNextStates(otherDay, snapshotStates(otherDay, listOf(moved to fields)), snapshotReplace = true) - repo.persistNextStates(null, snapshotStates(day, listOf(unowned to fields)), snapshotReplace = false) - val beforeMoved = repo.findByFlid(moved) - val beforeUnowned = repo.findByFlid(unowned) - - JdbcPipelineTransactionManager(ds).inTransaction { - assertEquals(1, repo.deleteDiffByDay(day, listOf(removed, moved, unowned))) - } - - assertNull(repo.findByFlid(removed)) - assertTrue(FlightDetailTables.loadCollections(ds, removed).isEmpty()) - assertEquals(beforeMoved, repo.findByFlid(moved)) - assertEquals(beforeUnowned, repo.findByFlid(unowned)) - } - - @Test - fun `PG dialect - snapshot batch upsert, find, and domain diff delete`() { - val day = "2026-09-07" - // 构造 250 条记录跨越 JDBC batch 200 门限 - val flights = (1..250).map { i -> - "TEST_FL_$i" to mapOf("FLID" to "TEST_FL_$i", "FLNO" to "CA$i") - } - repo.persistNextStates(day, snapshotStates(day, flights), snapshotReplace = true) - - // 验证全量写入成功 - val byDay = repo.findByDay(day) - assertEquals(250, byDay.size) - - // 点查测试 - val f1 = repo.findByFlid("TEST_FL_1") - assertNotNull(f1) - assertEquals("CA1", f1!!["FLNO"]) - - // 多键点查测试（分批查） - val subset = (1..50).map { "TEST_FL_$it" } - val map = repo.findByFlids(subset) - assertEquals(50, map.size) - - // 域内差删：删除 1..50 - val deleted = repo.deleteDiffByDay(day, subset) - assertEquals(50, deleted) - assertEquals(200, repo.findByDay(day).size) - assertNull(repo.findByFlid("TEST_FL_1")) - } - - @Test - fun `PG dialect - incremental upsert preserves existing FDAY and sets NULL for new`() { - val day = "2026-09-07" - // 1. 快照写入 REG_01，FDAY 为 2026-09-07 - repo.persistNextStates( - day, - snapshotStates(day, listOf("TEST_REG_01" to mapOf("FLID" to "TEST_REG_01", "REMC" to "1"))), - snapshotReplace = true, - ) - - // 2. 增量更新 REG_01（已有行）与 ADFT_01（新行） - repo.persistNextStates( - null, - listOf("TEST_REG_01" to "2", "TEST_ADFT_01" to "1").map { (flid, remc) -> - com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine.apply( - null, - com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine.commandsFromFields( - flid, mapOf("FLID" to flid, "REMC" to remc), - ), - "msg-inc-01", - bumpVersion = true, - ) - }, - snapshotReplace = false, - ) - - // 3. 验证 REG_01 内容更新但 FDAY 依然保留为 2026-09-07 - val reg01 = ds.queryOne("SELECT fday, remc FROM flight_schd WHERE flid = 'TEST_REG_01'", {}) { rs -> - rs.getDate("fday")?.toString() to rs.getString("remc") - } - assertNotNull(reg01) - assertEquals("2026-09-07", reg01!!.first) - assertEquals("2", reg01.second) - - // 4. 验证 ADFT_01 新插入行 FDAY 恒为 NULL - val adft01 = ds.queryOne("SELECT fday, remc FROM flight_schd WHERE flid = 'TEST_ADFT_01'", {}) { rs -> - rs.getDate("fday")?.toString() to rs.getString("remc") - } - assertNotNull(adft01) - assertNull(adft01!!.first) - - // 5. 执行 2026-09-07 的差删，ADFT_01 绝不被误删 - val deleted = repo.deleteDiffByDay(day, listOf("TEST_ADFT_01")) - assertEquals(0, deleted) - assertNotNull(repo.findByFlid("TEST_ADFT_01")) - } - - @Test - fun `resource collections replace as a whole and zero sequence clears them losslessly`() { - val day = "2026-09-07" - val flid = "TEST_RESOURCE" - val engine = com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine - // 快照：两个登机门 → 明细表两行（保序保 GTNO） - repo.persistNextStates( - day, - listOf( - engine.apply( - null, - engine.commandsFromFields(flid, mapOf("GTDT" to """[{"GTNO":"1","GATE":"A1"},{"GTNO":"2","GATE":"A2"}]""")), - "msg-res-1", bumpVersion = true, - ), - ), - snapshotReplace = true, - ) - val gates = ds.query( - "SELECT ordinal, source_seq, gate FROM flight_gate WHERE flid = ? ORDER BY ordinal", - { ps -> ps.setString(1, flid) }, - ) { rs -> Triple(rs.getInt("ordinal"), rs.getString("source_seq"), rs.getString("gate")) } - assertEquals(listOf(Triple(1, "1", "A1"), Triple(2, "2", "A2")), gates) - // 读侧明细权威视图：两个登机门完整读回 - val view = repo.findByFlid(flid)!!["GTDT"]?.let { mapper.readTree(it) } - assertEquals(2, view!!.size()) - assertEquals("A2", view[1]["GATE"].asText()) - assertEquals("2", view[1]["GTNO"].asText()) - - // 增量 = 集合级全量替换：1 个登机门（非连续 GTNO=3 保真） - repo.persistNextStates( - null, - listOf( - engine.apply( - repo.findNextStateByFlid(flid), - engine.commandsFromFields(flid, mapOf("GTDT" to """[{"GTNO":"3","GATE":"B1"}]""")), - "msg-res-2", bumpVersion = true, - ), - ), - snapshotReplace = false, - ) - val replaced = mapper.readTree(repo.findByFlid(flid)!!["GTDT"]) - assertEquals(1, replaced.size()) - assertEquals("B1", replaced[0]["GATE"].asText()) - assertEquals("3", replaced[0]["GTNO"].asText()) - - // 序号属性 "0" = 显式删除标记 → 集合清除，明细行删除，视图无 GTDT 键 - repo.persistNextStates( - null, - listOf( - engine.apply( - repo.findNextStateByFlid(flid), - engine.commandsFromFields(flid, mapOf("GTDT" to """[{"GTNO":"0"}]""")), - "msg-res-3", bumpVersion = true, - ), - ), - snapshotReplace = false, - ) - assertNull(repo.findByFlid(flid)!!["GTDT"]) - assertEquals(0, ds.query("SELECT COUNT(*) FROM flight_gate WHERE flid = ?", { ps -> ps.setString(1, flid) }) { rs -> rs.getInt(1) }.first()) - } - - @Test - fun `milestone delay route and exception collections persist losslessly in v2 storage`() { - val day = "2026-09-07" - val flid = "TEST_SCALAR" - val engine = com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine - repo.persistNextStates( - day, - listOf( - engine.apply( - null, - engine.commandsFromFields( - flid, - mapOf( - "DELY" to """[{"CODE":"YY","STRT":"07SEP261605","DURA":"0200","REMC":"Flight Delayed"}]""", - "ROUT" to """[{"RTNO":"1","APCD":"ORD","SCAT":"07SEP261125","SCDT":"07SEP261315"},{"RTNO":"2","APCD":"MSP"}]""", - "ABTM" to """[{"ASNO":"1","ABDG":"B01","ABOP":"A","AOTM":"07SEP261700"}]""", - "FDIV" to """{"DDES":"PEK","DDIR":"TO","REMC":"weather"}""", - ), - ), - "msg-scalar-1", bumpVersion = true, - ), - ), - snapshotReplace = true, - ) - // 库内明细表保序保源序号：延误、航路（route_kind 区分）、靠桥操作 - val delay = ds.queryOne( - "SELECT source_seq, code, strt, remc FROM flight_delay WHERE flid = ?", - { ps -> ps.setString(1, flid) }, - ) { rs -> listOf(rs.getString("source_seq"), rs.getString("code"), rs.getString("strt"), rs.getString("remc")) } - assertEquals(listOf(null, "YY", "07SEP261605", "Flight Delayed"), delay) - val routes = ds.query( - "SELECT route_kind, source_seq, apcd, scdt FROM flight_route_point WHERE flid = ? ORDER BY ordinal", - { ps -> ps.setString(1, flid) }, - ) { rs -> listOf(rs.getString("route_kind"), rs.getString("source_seq"), rs.getString("apcd"), rs.getString("scdt")) } - assertEquals( - listOf( - listOf("ROUT", "1", "ORD", "07SEP261315"), - listOf("ROUT", "2", "MSP", null), - ), - routes, - ) - val bridge = ds.queryOne( - "SELECT source_seq, abdg, abop, aotm FROM flight_bridge_op WHERE flid = ?", - { ps -> ps.setString(1, flid) }, - ) { rs -> listOf(rs.getString("source_seq"), rs.getString("abdg"), rs.getString("abop"), rs.getString("aotm")) } - assertEquals(listOf("1", "B01", "A", "07SEP261700"), bridge) - // 异常对象 = 主表前缀标量列；读侧视图重建 FDIV 键 - val fdiv = ds.queryOne( - "SELECT fdiv_ddes, fdiv_ddir, fdiv_remc FROM flight_schd WHERE flid = ?", - { ps -> ps.setString(1, flid) }, - ) { rs -> listOf(rs.getString("fdiv_ddes"), rs.getString("fdiv_ddir"), rs.getString("fdiv_remc")) } - assertEquals(listOf("PEK", "TO", "weather"), fdiv) - val fields = repo.findByFlid(flid)!! - assertEquals("YY", mapper.readTree(fields["DELY"])[0]["CODE"].asText()) - assertEquals(2, mapper.readTree(fields["ROUT"]).size()) - assertEquals("A", mapper.readTree(fields["ABTM"])[0]["ABOP"].asText()) - assertEquals("PEK", mapper.readTree(fields["FDIV"])["DDES"].asText()) - - // 增量：空延误数组 = 集合替换为空 → 明细行删除；航路整集合替换 - repo.persistNextStates( - null, - listOf( - engine.apply( - repo.findNextStateByFlid(flid), - engine.commandsFromFields( - flid, - mapOf( - "DELY" to "[]", - "ROUT" to """[{"RTNO":"1","APCD":"CTU"}]""", - ), - ), - "msg-scalar-2", bumpVersion = true, - ), - ), - snapshotReplace = false, - ) - assertEquals(0, ds.query("SELECT COUNT(*) FROM flight_delay WHERE flid = ?", { ps -> ps.setString(1, flid) }) { rs -> rs.getInt(1) }.first()) - assertNull(repo.findByFlid(flid)!!["DELY"]) - val replacedRoute = mapper.readTree(repo.findByFlid(flid)!!["ROUT"]) - assertEquals(1, replacedRoute.size()) - assertEquals("CTU", replacedRoute[0]["APCD"].asText()) - } - - @Test - fun `PIPELINE_LOCK row lock guards the writer transaction and releases on commit`() { - val txManager = JdbcPipelineTransactionManager(ds) - - // 持锁事务内：另一连接 NOWAIT 获取同行锁必须立即失败（互斥生效） - assertEquals("held", txManager.inTransaction { - DriverManager.getConnection(PgTestSupport.jdbcUrl, PgTestSupport.user, PgTestSupport.password).use { other -> - other.autoCommit = false - try { - other.prepareStatement( - "SELECT lock_key FROM pipeline_lock WHERE lock_key = 'FLIGHT_SCHD_WRITER' FOR UPDATE NOWAIT", - ).use { ps -> ps.executeQuery() } - throw AssertionError("expected PIPELINE_LOCK contention") - } catch (_: java.sql.SQLException) { - // 期望：行锁已被写者事务持有 - } finally { - other.rollback() - } - } - "held" - }) - - // 提交后锁释放：可再次进入，且种子行完整 - assertEquals("again", txManager.inTransaction { "again" }) - val owner = ds.queryOne( - "SELECT owner_info FROM pipeline_lock WHERE lock_key = 'FLIGHT_SCHD_WRITER'", - {}, - ) { rs -> rs.getString("owner_info") } - assertNull(owner) - } - - @Test - fun `PG dialect - SCHD_GEN SQL CAS version advance and conflict rejection`() { - val day = "2026-09-07" - - // 初始 CAS: expected = 0，版本推进至 1 - val gen1 = FlightSchdRepository.GenMeta(day, 1L, setOf("TEST_A", "TEST_B")) - val ok1 = repo.putGenIfVersion(day, 0L, gen1) - assertTrue(ok1) - - val stored1 = repo.getGen(day) - assertNotNull(stored1) - assertEquals(1L, stored1!!.version) - assertEquals(setOf("TEST_A", "TEST_B"), stored1.flids) - - // 成功 CAS: expected = 1，版本推进至 2 - val gen2 = FlightSchdRepository.GenMeta(day, 2L, setOf("TEST_A", "TEST_C")) - val ok2 = repo.putGenIfVersion(day, 1L, gen2) - assertTrue(ok2) - assertEquals(2L, repo.getGen(day)!!.version) - - // 冲突 CAS: expected 依然传 1（已过期的旧版本），应被拒并返回 false - val genConflict = FlightSchdRepository.GenMeta(day, 3L, setOf("TEST_CONFLICT")) - val conflictResult = repo.putGenIfVersion(day, 1L, genConflict) - assertFalse(conflictResult) - - // 确认版本未被非法篡改，仍为 2 - assertEquals(2L, repo.getGen(day)!!.version) - } - - @Test - fun `PG dialect - Transaction rollback leaves zero partial rows on failure`() { - val day = "2026-09-07" - try { - ds.withTransaction { - repo.persistNextStates( - day, - snapshotStates(day, listOf("TEST_TX_01" to mapOf("REMC" to "tx"))), - snapshotReplace = true, - ) - repo.putGenIfVersion(day, 0L, FlightSchdRepository.GenMeta(day, 1L, setOf("TEST_TX_01"))) - // 模拟事务内部抛出异常 - throw IllegalStateException("forced-abort-for-rollback-test") - } - } catch (_: IllegalStateException) { - // 异常捕获 - } - - // 断言：由于事务原子回滚，数据库中零残留 - assertNull(repo.findByFlid("TEST_TX_01")) - assertNull(repo.getGen(day)) - } - - @Test - fun `PG dialect - ES history sweep deleteByFlids idempotent batch execution`() { - val day = "2026-09-07" - val flids = (1..50).map { "TEST_SWEEP_$it" } - repo.persistNextStates( - day, - snapshotStates(day, flids.map { it to mapOf("FLID" to it, "GTDT" to """[{"GTNO":"1","GATE":"A"}]""") }), - snapshotReplace = true, - ) - assertEquals(50, repo.findByFlids(flids).size) - assertEquals(50, ds.query("SELECT COUNT(*) FROM flight_gate WHERE flid LIKE 'TEST_SWEEP_%'", {}) { rs -> rs.getInt(1) }.first()) - - // 第一次分批删除：主表与明细级联删除 - val deleted1 = repo.deleteByFlids(flids.toSet()) - assertEquals(50, deleted1) - assertEquals(0, repo.findByFlids(flids).size) - assertEquals(0, ds.query("SELECT COUNT(*) FROM flight_gate WHERE flid LIKE 'TEST_SWEEP_%'", {}) { rs -> rs.getInt(1) }.first()) - - // 重放删除：安全返回 0，无副作用 - val deleted2 = repo.deleteByFlids(flids.toSet()) - assertEquals(0, deleted2) - } - - @Test - fun `PG dialect - non-UTC JVM default timezone preserves exact Instant timestamps without drift`() { - val originalTz = TimeZone.getDefault() - try { - // 1. 设置 JVM 默认时区为东京 (UTC+09:00) - TimeZone.setDefault(TimeZone.getTimeZone(ZoneId.of("Asia/Tokyo"))) - val day = "2026-09-07" - val fixedInstant = Instant.parse("2026-09-07T08:30:15.123456Z") - - repo.persistNextStates( - day, - snapshotStates(day, listOf("TEST_TZ_01" to mapOf("FLID" to "TEST_TZ_01"))), - snapshotReplace = true, - now = fixedInstant, - ) - repo.putGenIfVersion(day, 0L, FlightSchdRepository.GenMeta(day, 1L, setOf("TEST_TZ_01")), fixedInstant) - - // 东京时区下回读 - val genTokyo = repo.getGen(day) - assertNotNull(genTokyo) - assertEquals(fixedInstant, genTokyo!!.updatedAt) - - // 2. 切换 JVM 默认时区为纽约 (UTC-05:00) - TimeZone.setDefault(TimeZone.getTimeZone(ZoneId.of("America/New_York"))) - - // 纽约时区下再次回读同一条记录 - val genNy = repo.getGen(day) - assertNotNull(genNy) - // 断言：TIMESTAMPTZ 映射到 Instant 完全相等，绝无 JVM 时区偏移 - assertEquals(fixedInstant, genNy!!.updatedAt) - - val flightNy = ds.queryOne("SELECT created_at, updated_at FROM flight_schd WHERE flid = 'TEST_TZ_01'", {}) { rs -> - rs.getInstant("created_at") to rs.getInstant("updated_at") - } - assertNotNull(flightNy) - assertEquals(fixedInstant, flightNy!!.first) - assertEquals(fixedInstant, flightNy.second) - } finally { - TimeZone.setDefault(originalTz) - } - } - - @Test - fun `PG dialect - non-UTC database session timezone produces zero Instant drift across sessions`() { - val day = "2026-09-07" - val writeInstant = Instant.parse("2026-09-07T14:45:00.654321Z") - - // 1. 在会话 1 中显式设置会话时区为 'Asia/Shanghai' (+08:00) 并写入数据 - ds.connection.use { conn -> - conn.createStatement().use { it.execute("SET TIME ZONE 'Asia/Shanghai'") } - conn.prepareStatement( - """ - INSERT INTO flight_schd (flid, fday, remc, created_at, updated_at) - VALUES ('TEST_SESS_01', '2026-09-07', 'shanghai', ?, ?) - """.trimIndent(), - ).use { ps -> - ps.setTimestamp(1, writeInstant.toSqlTimestamp()) - ps.setTimestamp(2, writeInstant.toSqlTimestamp()) - ps.executeUpdate() - } - } - - // 2. 在会话 2 中显式设置会话时区为 'America/Chicago' (-05:00) 并回读数据 - val instantChicago = ds.connection.use { conn -> - conn.createStatement().use { it.execute("SET TIME ZONE 'America/Chicago'") } - conn.prepareStatement("SELECT created_at, updated_at FROM flight_schd WHERE flid = 'TEST_SESS_01'").use { ps -> - ps.executeQuery().use { rs -> - assertTrue(rs.next()) - rs.getInstant("created_at") - } - } - } - - // 3. 在会话 3 中显式设置会话时区为 'UTC' 并回读数据 - val instantUtc = ds.connection.use { conn -> - conn.createStatement().use { it.execute("SET TIME ZONE 'UTC'") } - conn.prepareStatement("SELECT created_at, updated_at FROM flight_schd WHERE flid = 'TEST_SESS_01'").use { ps -> - ps.executeQuery().use { rs -> - assertTrue(rs.next()) - rs.getInstant("created_at") - } - } - } - - // 断言：跨不同数据库会话时区（+08:00 vs -05:00 vs UTC），读取出的 Instant 与写入的 UTC Instant 完全一致！ - assertEquals(writeInstant, instantChicago) - assertEquals(writeInstant, instantUtc) - assertEquals(instantChicago, instantUtc) - } - - @Test - fun `PG dialect - persistNextStates roundtrip preserves collections source_seq and state_version`() { - val day = "2026-09-07" - val flid = "TEST_V2_ROUNDTRIP" - val gates = listOf( - mapOf("GTNO" to "1", "GATE" to "G28", "GTYP" to "D"), - mapOf("GTNO" to "3", "GATE" to "G33", "GTYP" to "I"), - mapOf("GTNO" to "5", "GATE" to "G23", "GTYP" to "D"), - ) - val state = com.gzzn.omms.msgexchange.domain.flight.FlightNextState( - flid = flid, - scalars = mapOf("FLNO" to "CA888"), - collections = mapOf("GTDT" to gates), - stateVersion = 1L, - lastMessageId = "msg-dnld-001", - ) - - repo.persistNextStates(day, listOf(state), snapshotReplace = true) - - val roundtrip = repo.findNextStateByFlid(flid)!! - assertEquals(1L, roundtrip.stateVersion) - assertEquals("msg-dnld-001", roundtrip.lastMessageId) - // 明细表读回保序保源序号；未填属性以空串回填，按语义字段断言 - val roundtripGates = roundtrip.collections["GTDT"]!! - assertEquals(3, roundtripGates.size) - gates.forEachIndexed { i, expected -> - val actual = roundtripGates[i] - assertEquals(expected["GTNO"], actual["GTNO"]) - assertEquals(expected["GATE"], actual["GATE"]) - assertEquals(expected["GTYP"], actual["GTYP"]) - } - - val replaced = state.copy( - collections = mapOf("GTDT" to listOf(mapOf("GTNO" to "2", "GATE" to "Z9"))), - stateVersion = 2L, - lastMessageId = "msg-flop-002", - ) - repo.persistNextStates(null, listOf(replaced), snapshotReplace = false) - assertEquals("Z9", repo.findNextStateByFlid(flid)!!.collections["GTDT"]!![0]["GATE"]) - assertEquals(2L, repo.findNextStateByFlid(flid)!!.stateVersion) - val detailRows = ds.query( - "SELECT ordinal, source_seq, gate FROM flight_gate WHERE flid = ? ORDER BY ordinal", - { ps -> ps.setString(1, flid) }, - ) { rs -> Triple(rs.getInt("ordinal"), rs.getString("source_seq"), rs.getString("gate")) } - assertEquals(1, detailRows.size) - assertEquals(Triple(1, "2", "Z9"), detailRows[0]) - } - - @Test - fun `PG dialect - persistNextStates leaves all legacy collection storage columns null`() { - val day = "2026-09-07" - val flid = "TEST_V2_ALL_COLS" - val state = com.gzzn.omms.msgexchange.domain.flight.FlightNextState( - flid = flid, - scalars = mapOf("FLNO" to "MU123"), - collections = mapOf( - "GTDT" to listOf(mapOf("GTNO" to "1", "GATE" to "G1", "GTYP" to "D")), - "CKDT" to listOf(mapOf("CKNO" to "1", "CHKC" to "C1", "CTYP" to "Eco")), - "CLDT" to listOf(mapOf("CLNO" to "1", "BELT" to "B1", "BTYP" to "D")), - "DELY" to listOf(mapOf("DLNO" to "1", "CODE" to "01", "STRT" to "1200", "DURA" to "30")), - "ABTM" to listOf(mapOf("ASNO" to "1", "ABDG" to "B12", "ABOP" to "A", "AOTM" to "0730")), - "CHOT" to listOf(mapOf("CSNO" to "1", "CHID" to "ON", "CHTM" to "0800")), - "ROUT" to listOf(mapOf("RTNO" to "1", "APCD" to "CTU")), - ), - stateVersion = 1L, - lastMessageId = "msg-dnld-002", - ) - - repo.persistNextStates(day, listOf(state), snapshotReplace = true) - assertEquals("MU123", repo.findByFlid(flid)!!["FLNO"]) - assertEquals(1, ds.query("SELECT COUNT(*) FROM flight_gate WHERE flid = ?", { ps -> ps.setString(1, flid) }) { rs -> rs.getInt(1) }.first()) - assertEquals(1, ds.query("SELECT COUNT(*) FROM flight_delay WHERE flid = ?", { ps -> ps.setString(1, flid) }) { rs -> rs.getInt(1) }.first()) - } - - @Test - fun `PG dialect - schd_gen stores last_message_id and rejects stale CAS without advancing`() { - val day = "2026-09-07" - - // 初始提交：last_message_id = "m1"，版本 1 - val ok1 = repo.putGenIfVersion( - day, 0L, - FlightSchdRepository.GenMeta(day, 1L, setOf("TEST_CAS_M"), lastMessageId = "m1"), - ) - assertTrue(ok1) - assertEquals("m1", repo.getGen(day)!!.lastMessageId) - - // 重放判定：以过期 expected=0 重试必须失败（同一消息的身份凭 getGen().lastMessageId 判定，不是靠版本相等） - val replayAttempt = repo.putGenIfVersion( - day, 0L, - FlightSchdRepository.GenMeta(day, 1L, setOf("TEST_CAS_M"), lastMessageId = "m1"), - ) - assertFalse(replayAttempt) - assertEquals(1L, repo.getGen(day)!!.version) - - // 正常推进 expected=1 → 版本 2，last_message_id 换为 "m2" - val ok2 = repo.putGenIfVersion( - day, 1L, - FlightSchdRepository.GenMeta(day, 2L, setOf("TEST_CAS_M"), lastMessageId = "m2"), - ) - assertTrue(ok2) - val gen = repo.getGen(day)!! - assertEquals(2L, gen.version) - assertEquals("m2", gen.lastMessageId) - } - - @Test - fun `PG dialect - flight_schd_display view projects first second resources and totals from detail tables`() { - val day = "2026-09-07" - val flid = "TEST_DISPLAY" - val engine = com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine - repo.persistNextStates( - day, - listOf( - engine.apply( - null, - engine.commandsFromFields( - flid, - mapOf( - "FLNO" to "CA display", - "GTDT" to """[{"GTNO":"1","GATE":"G28"},{"GTNO":"2","GATE":"G33"},{"GTNO":"3","GATE":"G23"}]""", - "CKDT" to """[{"CKNO":"1","CHKC":"01","CCLS":"Y"},{"CKNO":"2","CHKC":"17","CCLS":"F"}]""", - "CLDT" to """[{"CLNO":"1","BELT":"B01"}]""", - "DELY" to """[{"CODE":"YY","STRT":"07SEP261605","DURA":"0200"}]""", - ), - ), - "msg-display-1", bumpVersion = true, - ), - ), - snapshotReplace = true, - ) - - val row = ds.queryOne( - """ - SELECT flno, gate1, gate2, gate_total, chkc1, chkc2, checkin_total, belt1, belt_total, dely_code, dely_strt - FROM flight_schd_display WHERE flid = ? - """.trimIndent(), - { ps -> ps.setString(1, flid) }, - ) { rs -> - listOf( - rs.getString("flno"), rs.getString("gate1"), rs.getString("gate2"), rs.getInt("gate_total"), - rs.getString("chkc1"), rs.getString("chkc2"), rs.getInt("checkin_total"), - rs.getString("belt1"), rs.getInt("belt_total"), - rs.getString("dely_code"), rs.getString("dely_strt"), - ) - } - // 视图提供首个/第二个资源 + 总数；完整明细仍在明细表可查（三门不丢失） - assertEquals( - listOf( - "CA display", "G28", "G33", 3, - "01", "17", 2, - "B01", 1, - "YY", "07SEP261605", - ), - row, - ) - - // 集合替换后视图随明细表同步（3 门 → 1 门） - repo.persistNextStates( - null, - listOf( - engine.apply( - repo.findNextStateByFlid(flid), - engine.commandsFromFields(flid, mapOf("GTDT" to """[{"GTNO":"5","GATE":"Z1"}]""")), - "msg-display-2", bumpVersion = true, - ), - ), - snapshotReplace = false, - ) - val after = ds.queryOne( - "SELECT gate1, gate2, gate_total FROM flight_schd_display WHERE flid = ?", - { ps -> ps.setString(1, flid) }, - ) { rs -> Triple(rs.getString("gate1"), rs.getString("gate2"), rs.getInt("gate_total")) } - assertEquals(Triple("Z1", null, 1), after) - } - - @Test - fun `PG dialect - exception object explicit clear writes null columns and omits wire key`() { - val day = "2026-09-07" - val flid = "TEST_EXC_CLEAR" - val engine = com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine - - // 1. 快照写入返航异常对象 → 主表前缀标量列 - repo.persistNextStates( - day, - listOf( - engine.apply( - null, - engine.commandsFromFields(flid, mapOf("FRET" to """{"REID":"CA108","RSN":"diverted"}""")), - "msg-exc-1", bumpVersion = true, - ), - ), - snapshotReplace = true, - ) - val stored1 = ds.queryOne( - "SELECT fret_reid, fret_rsn FROM flight_schd WHERE flid = ?", - { ps -> ps.setString(1, flid) }, - ) { rs -> listOf(rs.getString("fret_reid"), rs.getString("fret_rsn")) } - assertEquals(listOf("CA108", "diverted"), stored1) - - // 2. 增量显式清除（FRET="null"）→ 列置 NULL，读视图无 FRET 键（评审 F4 修复闭环） - repo.persistNextStates( - null, - listOf( - engine.apply( - repo.findNextStateByFlid(flid), - engine.commandsFromFields(flid, mapOf("FRET" to "null")), - "msg-exc-2", bumpVersion = true, - ), - ), - snapshotReplace = false, - ) - val stored2 = ds.queryOne( - "SELECT fret_reid, fret_rsn FROM flight_schd WHERE flid = ?", - { ps -> ps.setString(1, flid) }, - ) { rs -> listOf(rs.getString("fret_reid"), rs.getString("fret_rsn")) } - assertEquals(listOf(null, null), stored2) - assertNull(repo.findByFlid(flid)!!["FRET"]) - assertNull(repo.findNextStateByFlid(flid)!!.scalars["FRET"]) - } - - @Test - fun `PG dialect - readAfterWrite equals nextState across all structures and wire stays consistent`() { - val day = "2026-09-07" - val flid = "TEST_MATRIX" - val engine = com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine - val fields = mapOf( - "FLNO" to "CA_MATRIX", - "SODT" to "07SEP260800", - "GTDT" to """[{"GTNO":"1","GATE":"G28","GTYP":"D"},{"GTNO":"3","GATE":"G33"},{"GTNO":"3","GATE":"G23"}]""", - "CKDT" to """[{"CKNO":"1","CHKC":"01","CCLS":"Y"},{"CKNO":"2","CHKC":"01","CCLS":"F"}]""", - "CLDT" to """[{"CLNO":"1","BELT":"B01"},{"CLNO":"2","BELT":"B02"}]""", - "PSDT" to """[{"PSNO":"1","PSST":"S209"}]""", - "CHDT" to """[{"CHNO":"1","CHUT":"C1"}]""", - "DELY" to """[{"CODE":"YY","STRT":"07SEP261605","DURA":"0200","REMC":"delayed"},{"CODE":"WX","STRT":"07SEP261800","DURA":"0030"}]""", - "ABTM" to """[{"ASNO":"1","ABDG":"B01","ABOP":"A","AOTM":"07SEP261700"},{"ASNO":"2","ABDG":"B01","ABOP":"D","AOTM":"07SEP261900"}]""", - "CHOT" to """[{"CSNO":"1","CHID":"ON","CHTM":"07SEP261650"},{"CSNO":"2","CHID":"OFF","CHTM":"07SEP261920"}]""", - "ROUT" to """[{"RTNO":"1","APCD":"CTU"},{"RTNO":"2","APCD":"PEK","SCDT":"07SEP261100"}]""", - "FDIV" to """{"DDES":"PEK","DDIR":"TO"}""", - "SRVT" to """[{"SRTC":"WHEEL"}]""", - ) - val state = engine.apply(null, engine.commandsFromFields(flid, fields), "msg-matrix-1", bumpVersion = true) - - repo.persistNextStates(day, listOf(state), snapshotReplace = true) - - // 断言一：readAfterWrite == nextState（评审 F5 的持久化侧闭环） - val readBack = repo.findNextStateByFlid(flid)!! - assertEquals(state.stateVersion, readBack.stateVersion) - assertEquals(state.lastMessageId, readBack.lastMessageId) - assertEquals(state.scalars.keys, readBack.scalars.keys) - state.scalars.forEach { (k, v) -> assertJsonEquivalent(v, readBack.scalars[k], "scalar[$k]") } - assertEquals(state.collections.keys, readBack.collections.keys) - state.collections.forEach { (key, items) -> - val actual = readBack.collections[key]!! - assertEquals(items.size, actual.size, "collection[$key] row count") - items.forEachIndexed { i, item -> - item.forEach { (attr, v) -> assertEquals(v, actual[i][attr], "collection[$key][$i].$attr") } - } - } - - // 断言二：wire(nextState) == wire(readAfterWrite)（出站载荷与库态同源，F5 的线格式侧闭环） - val wireBefore = mapper.readTree(com.gzzn.omms.msgexchange.infra.persistence.FlightFieldsJson.toJson(state.toFlightFields())) - val wireAfter = mapper.readTree(com.gzzn.omms.msgexchange.infra.persistence.FlightFieldsJson.toJson(repo.findByFlid(flid)!!)) - assertEquals(wireBefore, wireAfter) - } - - /** JSON 文本按解析树等价比较（键序无关）；非 JSON 退化为字符串比较。 */ - private fun assertJsonEquivalent(expected: String?, actual: String?, path: String) { - if (expected == null || actual == null) { - assertEquals(expected, actual, path) - return - } - val e = runCatching { mapper.readTree(expected) }.getOrNull() - val a = runCatching { mapper.readTree(actual) }.getOrNull() - if (e != null && a != null) assertEquals(e, a, path) else assertEquals(expected, actual, path) - } - -} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlywayMigrationTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlywayMigrationTest.kt index 3539a2d..b825006 100644 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlywayMigrationTest.kt +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/infra/persistence/jdbc/FlywayMigrationTest.kt @@ -3,40 +3,43 @@ package com.gzzn.omms.msgexchange.infra.persistence.jdbc import com.gzzn.omms.msgexchange.support.PgTestSupport import org.flywaydb.core.Flyway import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Assertions.assertFalse -import org.junit.jupiter.api.Assertions.assertNull import org.junit.jupiter.api.Assertions.assertTrue import org.junit.jupiter.api.Assumptions.assumeTrue import org.junit.jupiter.api.Test import java.sql.DriverManager /** - * P2-1 验证测试：Flyway 迁移引擎在真实 PostgreSQL 上的端到端自动迁移验证。 - * 断言： - * 1. Flyway 自动执行 V1.0.0__own_pg_pipeline.sql 与 V1.1.0__flight_schd.sql； - * 2. 真实落库 flyway_schema_history 元数据表，且 success = true； - * 3. 运营航班表 FLIGHT_SCHD 与计划代 SCHD_GEN 结构完整就绪。 + * Flyway 迁移引擎端到端验证（docs/flight-state.md §3.2 表职责基线）： + * 1. V1__flight_state_baseline.sql 在真实 PostgreSQL 上自动迁移成功； + * 2. flyway_schema_history 落库且 success = true； + * 3. 决策层/管道层/留痕层全表就绪；PIPELINE_LOCK 单行种子就位。 */ class FlywayMigrationTest { + private val expectedTables = setOf( + "pipeline_lock", "proc_state", "flight_schd", + "flight_gate", "flight_checkin", "flight_belt", "flight_stand_plan", "flight_chute", + "flight_delay", "flight_bridge_op", "flight_chock_op", "flight_route_point", + "msg_event", "req_track", "backfill_todo", "schd_snap_log", + ) + @Test - fun `Flyway automatically migrates V1_0_0 and V1_1_0 onto real PostgreSQL`() { + fun `Flyway migrates flight-state baseline onto real PostgreSQL`() { assumeTrue(PgTestSupport.canConnect(), PgTestSupport.skipMessage()) val url = PgTestSupport.jdbcUrl val user = PgTestSupport.user val pass = PgTestSupport.password - val flyway = Flyway.configure() + val migrateResult = Flyway.configure() .dataSource(url, user, pass) .locations("classpath:db/migration") .load() - - val migrateResult = flyway.migrate() + .migrate() assertTrue(migrateResult.success, "Flyway migration must succeed") - // 验证 flyway_schema_history 元数据表与执行记录 DriverManager.getConnection(url, user, pass).use { conn -> conn.createStatement().use { stmt -> + // 迁移记录：单基线 V1 stmt.executeQuery( "SELECT version, script, success FROM flyway_schema_history ORDER BY installed_rank ASC", ).use { rs -> @@ -44,115 +47,38 @@ class FlywayMigrationTest { while (rs.next()) { records.add(Triple(rs.getString("version"), rs.getString("script"), rs.getBoolean("success"))) } - assertTrue(records.size >= 2, "At least 2 migrations must be recorded in flyway_schema_history") - assertEquals("1.0.0", records[0].first) - assertEquals("V1.0.0__own_pg_pipeline.sql", records[0].second) + assertTrue(records.isNotEmpty(), "flyway_schema_history must record migrations") + assertEquals("1", records[0].first) + assertEquals("V1__flight_state_baseline.sql", records[0].second) assertTrue(records[0].third) - - assertEquals("1.1.0", records[1].first) - assertEquals("V1.1.0__flight_schd.sql", records[1].second) - assertTrue(records[1].third) } - // 验证 FLIGHT_SCHD 与 SCHD_GEN 存在 + // §3.2 全表就绪 stmt.executeQuery( - "SELECT table_name FROM information_schema.tables WHERE table_schema = 'public' AND table_name IN ('flight_schd', 'schd_gen')", + "SELECT table_name FROM information_schema.tables WHERE table_schema = 'public'", ).use { rs -> val tables = mutableSetOf() - while (rs.next()) { - tables.add(rs.getString("table_name")) - } - assertTrue(tables.contains("flight_schd")) - assertTrue(tables.contains("schd_gen")) + while (rs.next()) tables.add(rs.getString("table_name")) + val missing = expectedTables - tables + assertTrue(missing.isEmpty(), "missing tables: $missing") + } + + // 身份不变量基础结构：FLIGHT_SCHD 主键 + STATE 列 + stmt.executeQuery( + "SELECT column_name FROM information_schema.columns WHERE table_name = 'flight_schd' " + + "AND column_name IN ('flid', 'operation_day', 'state', 'state_version', 'last_msg_id')", + ).use { rs -> + val cols = mutableSetOf() + while (rs.next()) cols.add(rs.getString("column_name")) + assertEquals(setOf("flid", "operation_day", "state", "state_version", "last_msg_id"), cols) + } + + // PIPELINE_LOCK 单行种子（§3.2） + stmt.executeQuery("SELECT count(*) FROM pipeline_lock WHERE lock_id = 1").use { rs -> + assertTrue(rs.next()) + assertEquals(1, rs.getInt(1)) } } } } - - /** - * P2-4：带数据的 V1.1.0 → 最新版本升级链验证（隔离 schema，不污染 public）。 - * V1.2.0 删除 *_TXT 集合列是「已知损失边界」（评审 F6）：标量列与结构完整存活， - * 集合文本不可恢复——升级前必须完整导出或重放（docs/flight-state.md）。 - */ - @Test - fun `upgrade chain from seeded V1_1_0 data keeps scalars and adds v2 tracking`() { - assumeTrue(PgTestSupport.canConnect(), PgTestSupport.skipMessage()) - val schema = "upgrade_chain_test" - // 清理历史遗留的隔离 schema - DriverManager.getConnection(PgTestSupport.jdbcUrl, PgTestSupport.user, PgTestSupport.password).use { conn -> - conn.createStatement().use { it.execute("DROP SCHEMA IF EXISTS $schema CASCADE") } - } - - // 1. 迁移到 V1.1.0（含 *_TXT 集合文本列形态）并写入带数据行 - Flyway.configure() - .dataSource(PgTestSupport.jdbcUrl, PgTestSupport.user, PgTestSupport.password) - .locations("classpath:db/migration") - .schemas(schema) - .defaultSchema(schema) - .target("1.1.0") - .load() - .migrate() - DriverManager.getConnection(PgTestSupport.jdbcUrl, PgTestSupport.user, PgTestSupport.password).use { conn -> - conn.createStatement().use { - it.execute( - """ - INSERT INTO $schema.flight_schd (flid, fday, flno, remc, gtdt_txt, created_at, updated_at) - VALUES ('UPG_FL_1', '2026-09-07', 'CA_UPG', 'legacy-note', '[{"GTNO":"1","GATE":"G1"}]', now(), now()) - """.trimIndent(), - ) - } - } - - // 2. 一路升级到最新版本 - val result = Flyway.configure() - .dataSource(PgTestSupport.jdbcUrl, PgTestSupport.user, PgTestSupport.password) - .locations("classpath:db/migration") - .schemas(schema) - .defaultSchema(schema) - .load() - .migrate() - assertTrue(result.success) - - DriverManager.getConnection(PgTestSupport.jdbcUrl, PgTestSupport.user, PgTestSupport.password).use { conn -> - conn.createStatement().use { stmt -> - // 3. 标量数据存活；v2 追踪列就位（存量行 state_version 默认 0） - stmt.executeQuery( - "SELECT flno, remc, state_version, last_message_id FROM $schema.flight_schd WHERE flid = 'UPG_FL_1'", - ).use { rs -> - assertTrue(rs.next()) - assertEquals("CA_UPG", rs.getString("flno")) - assertEquals("legacy-note", rs.getString("remc")) - assertEquals(0L, rs.getLong("state_version")) - assertNull(rs.getString("last_message_id")) - } - - // 4. 已知损失边界：*_TXT 集合列已不存在（升级前须导出/重放，评审 F6） - stmt.executeQuery( - "SELECT column_name FROM information_schema.columns WHERE table_schema = '$schema' AND table_name = 'flight_schd' AND column_name LIKE '%\\_TXT'", - ).use { rs -> - assertFalse(rs.next(), "legacy *_TXT columns must be gone after V1.2.0 (documented lossy boundary)") - } - - // 5. v2 结构就位：display 视图可查询（存量行 gate_total=0）、backfill_todo 与明细表存在 - stmt.executeQuery( - "SELECT flno, gate_total FROM $schema.flight_schd_display WHERE flid = 'UPG_FL_1'", - ).use { rs -> - assertTrue(rs.next()) - assertEquals("CA_UPG", rs.getString("flno")) - assertEquals(0, rs.getInt("gate_total")) - } - stmt.executeQuery("SELECT COUNT(*) FROM $schema.backfill_todo").use { rs -> - assertTrue(rs.next()) - } - stmt.executeQuery("SELECT COUNT(*) FROM $schema.flight_gate WHERE flid = 'UPG_FL_1'").use { rs -> - assertTrue(rs.next()) - assertEquals(0, rs.getInt(1)) - } - } - } - - DriverManager.getConnection(PgTestSupport.jdbcUrl, PgTestSupport.user, PgTestSupport.password).use { conn -> - conn.createStatement().use { it.execute("DROP SCHEMA IF EXISTS $schema CASCADE") } - } - } } diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/infra/retry/ReplayServiceTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/infra/retry/ReplayServiceTest.kt index 1736c0f..f4ebccd 100644 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/infra/retry/ReplayServiceTest.kt +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/infra/retry/ReplayServiceTest.kt @@ -23,13 +23,15 @@ class ReplayServiceTest { rows[id] = ProcState(id, status, identityKey = "k$id", attempts = 3, errorClass = ec, lastError = "x") } - override fun insert(cminmsgsId: Long, state: ProcStatus) = Unit - override fun exists(cminmsgsId: Long): Boolean = rows.containsKey(cminmsgsId) + override fun insert(msgId: Long, state: ProcStatus) = Unit + override fun exists(msgId: Long): Boolean = rows.containsKey(msgId) + override fun find(msgId: Long): ProcState? = rows[msgId] + override fun findSuccessTerminal(msgId: Long): Boolean = rows[msgId]?.state == ProcStatus.SUCCEEDED override fun headUnfinished(): ProcState? = null - override fun tryBindIdentity(cminmsgsId: Long, identityKey: String): Boolean = true + override fun tryBindIdentity(msgId: Long, identityKey: String): Boolean = true override fun ownerOfIdentity(identityKey: String): Long? = null override fun update( - cminmsgsId: Long, state: ProcStatus, nextAttemptAt: Instant?, attempts: Int?, + msgId: Long, state: ProcStatus, nextAttemptAt: Instant?, attempts: Int?, errorClass: ErrorClass?, lastError: String?, ) = Unit diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/jobs/BackfillSweepJobTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/jobs/BackfillSweepJobTest.kt index a043b4e..efa7ee3 100644 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/jobs/BackfillSweepJobTest.kt +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/jobs/BackfillSweepJobTest.kt @@ -56,10 +56,10 @@ class BackfillSweepJobTest { assertEquals(BackfillSweepJob.SweepOutcome(inspected = 1, succeeded = 0, failed = 1), outcome) assertEquals(1, todo.count()) - assertEquals(1, todo.tasks().single().attempts) + assertEquals(1, todo.tasks.values.single().attempts) assertEquals("mysql-down", todo.lastErrorOf(901L)) // 一次失败后 30s 内不再到期（退避生效） - assertEquals(emptyList(), todo.findDue(t0.plusSeconds(29))) + assertEquals(0, todo.findDue(t0.plusSeconds(29)).size) assertEquals(1, todo.findDue(t0.plusSeconds(30)).size) } diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/jobs/HistorySweepJobTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/jobs/HistorySweepJobTest.kt index 18551f8..35a2c92 100644 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/jobs/HistorySweepJobTest.kt +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/jobs/HistorySweepJobTest.kt @@ -1,168 +1,103 @@ package com.gzzn.omms.msgexchange.jobs -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository -import com.gzzn.omms.msgexchange.infra.stub.StubFlightSchd -import com.gzzn.omms.msgexchange.support.seedSnapshot -import org.junit.jupiter.api.AfterEach +import com.gzzn.omms.msgexchange.config.HistoryProps +import com.gzzn.omms.msgexchange.domain.EventType +import com.gzzn.omms.msgexchange.domain.flight.FlightState +import com.gzzn.omms.msgexchange.domain.flight.HistoryCandidate +import com.gzzn.omms.msgexchange.infra.stub.StubFlightState +import com.gzzn.omms.msgexchange.infra.stub.StubMsgEvents import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Assertions.assertNotNull -import org.junit.jupiter.api.Assertions.assertNull import org.junit.jupiter.api.Assertions.assertTrue -import org.junit.jupiter.api.BeforeEach import org.junit.jupiter.api.Test +import java.time.Instant /** - * ACM2-28 FS6 历史清场测试： - * 覆盖 ES 全成功、部分成功、全部失败、ES 成功后 PG 删除前崩溃、PG 删除重放五种场景； - * 断言失败项不丢、成功项最终删除、重复执行无副作用。 + * 历史归档与物理清除（docs/flight-state.md §8.2，顺序不可颠倒）： + * 历史存储未接通必须删 0 条；先归档确认再物理清除； + * 未经 FDEL 的航班在清除前补发删除事件。 */ class HistorySweepJobTest { - private lateinit var flightSchd: StubFlightSchd - private lateinit var sweepJob: HistorySweepJob + private val now: Instant = Instant.parse("2026-09-09T00:00:00Z") - @BeforeEach - fun setUp() { - flightSchd = StubFlightSchd() - sweepJob = HistorySweepJob(flightSchd) - HistorySweepJob.historyPicker = null - HistorySweepJob.esArchiver = null - HistorySweepJob.cutoffProvider = null - } - - @AfterEach - fun tearDown() { - HistorySweepJob.historyPicker = null - HistorySweepJob.esArchiver = null - HistorySweepJob.cutoffProvider = null - } - - // 1. 场景一：ES 全成功 - @Test - fun `Scenario 1 - ES All Success removes all confirmed flights and cleans old gens`() { - val flights = listOf( - "F1" to mapOf("FLID" to "F1"), - "F2" to mapOf("FLID" to "F2"), - "F3" to mapOf("FLID" to "F3"), + private fun seededFlight(flid: String, deleted: Boolean, idleDays: Long): StubFlightState { + val f = StubFlightState() + f.persistFullState( + com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot( + flid, java.time.LocalDate.of(2026, 9, 1), if (deleted) FlightState.DELETED else FlightState.ACTIVE, + 1, mapOf("FLNO" to "CA001"), emptyMap(), + ), + msgId = 1, + now = now.minusSeconds(idleDays * 86400), ) - seedSnapshot(flightSchd, "2026-09-01", flights) - flightSchd.putGenIfVersion("2026-09-01", 0L, FlightSchdRepository.GenMeta("2026-09-01", 1L, setOf("F1", "F2", "F3"))) - - // 模拟判史：F1, F2, F3 全部为历史候选 - HistorySweepJob.historyPicker = { it } - // 模拟 ES 写入：全部成功返回 - HistorySweepJob.esArchiver = { it.keys } - // 模拟历史代清理：清理 2026-09-02 之前的代 - HistorySweepJob.cutoffProvider = { "2026-09-02" } - - sweepJob.run() - - // 断言：PG 中 F1, F2, F3 全部被删除 - assertNull(flightSchd.findByFlid("F1")) - assertNull(flightSchd.findByFlid("F2")) - assertNull(flightSchd.findByFlid("F3")) - // 断言：过期代元数据被清理 - assertNull(flightSchd.getGen("2026-09-01")) + return f } - // 2. 场景二：ES 部分成功 - @Test - fun `Scenario 2 - ES Partial Success deletes only successful flights and keeps failed ones`() { - val flights = listOf( - "F1" to mapOf("FLID" to "F1"), - "F2" to mapOf("FLID" to "F2"), - "F3" to mapOf("FLID" to "F3"), - ) - seedSnapshot(flightSchd, "2026-09-01", flights) + private class RecordingHistoryStore : HistorySweepJob.HistoryStore { + val archivedBatches = mutableListOf>() + val confirmed = mutableSetOf() + var fail = false - HistorySweepJob.historyPicker = { it } - // 模拟 ES 写入：F1 和 F2 成功，F3 失败 - HistorySweepJob.esArchiver = { setOf("F1", "F2") } - - sweepJob.run() - - // 断言：成功项最终删除 - assertNull(flightSchd.findByFlid("F1")) - assertNull(flightSchd.findByFlid("F2")) - - // 断言：失败项 F3 绝不丢失，继续保留在 PG 中供下次重试 - assertNotNull(flightSchd.findByFlid("F3")) - } - - // 3. 场景三：ES 全部失败 - @Test - fun `Scenario 3 - ES All Failure deletes nothing and keeps all candidates in PG`() { - val flights = listOf( - "F1" to mapOf("FLID" to "F1"), - "F2" to mapOf("FLID" to "F2"), - ) - seedSnapshot(flightSchd, "2026-09-01", flights) - - HistorySweepJob.historyPicker = { it } - // 模拟 ES 全部写入失败 - HistorySweepJob.esArchiver = { emptySet() } - - sweepJob.run() - - // 断言：PG 未执行任何删除，全部候选均安全保留 - assertNotNull(flightSchd.findByFlid("F1")) - assertNotNull(flightSchd.findByFlid("F2")) - } - - // 4. 场景四：ES 成功后 PG 删除前崩溃重放 - @Test - fun `Scenario 4 - Crash after ES success before PG delete recovers idempotently on next run`() { - val flights = listOf( - "F1" to mapOf("FLID" to "F1"), - ) - seedSnapshot(flightSchd, "2026-09-01", flights) - - // 第一次运行模拟：ES 写入成功，但在调用 deleteByFlids 前崩溃（PG 依然保留 F1） - val esUpserted = mutableSetOf() - var crashed = false - - HistorySweepJob.historyPicker = { it } - HistorySweepJob.esArchiver = { - esUpserted.addAll(it.keys) - if (!crashed) { - crashed = true - throw RuntimeException("crash-before-pg-delete") - } - it.keys + override fun archive(candidates: List): Set { + archivedBatches.add(candidates) + if (fail) return emptySet() + return candidates.map { it.flid }.toSet().intersect(confirmed.ifEmpty { candidates.map { it.flid }.toSet() }) } - - try { - sweepJob.run() - } catch (_: RuntimeException) { - // 崩溃发生 - } - - // 崩溃后断言：ES 已有数据，但 PG 中 F1 仍然存在作为可恢复事实 - assertTrue(esUpserted.contains("F1")) - assertNotNull(flightSchd.findByFlid("F1")) - - // 下一次作业重试： - sweepJob.run() - - // 断言：重试成功，PG 中 F1 最终完成删除 - assertNull(flightSchd.findByFlid("F1")) } - // 5. 场景五：PG 删除重复执行（无副作用与幂等性） @Test - fun `Scenario 5 - Replaying deleteByFlids is completely idempotent with no side effects`() { - val flights = listOf("F1" to mapOf("FLID" to "F1")) - seedSnapshot(flightSchd, "2026-09-01", flights) + fun `history store not connected must delete zero rows`() { + val f = seededFlight("F1", deleted = true, idleDays = 30) + val events = StubMsgEvents() + val job = HistorySweepJob(f, events, HistoryProps().apply { historyStoreEnabled = false }) - // 首次删除 - val deletedCount1 = flightSchd.deleteByFlids(setOf("F1")) - assertEquals(1, deletedCount1) - assertNull(flightSchd.findByFlid("F1")) + val outcome = job.run(now) - // 重复执行删除（重放） - val deletedCount2 = flightSchd.deleteByFlids(setOf("F1")) - // 断言：允许重放时影响行数为 0，不抛出异常，无任何副作用 - assertEquals(0, deletedCount2) - assertNull(flightSchd.findByFlid("F1")) + assertEquals(0, outcome.purged) // §8.2 红线：先删历史再补当前态绝不允许 + assertTrue(f.findMainRow("F1") != null) + assertEquals(0, events.rows.size) + } + + @Test + fun `archive success precedes physical purge and confirmed rows only`() { + val f = seededFlight("F1", deleted = true, idleDays = 30) + f.persistFullState( + com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot( + "F2", java.time.LocalDate.of(2026, 9, 1), FlightState.DELETED, 1, mapOf("FLNO" to "CA002"), emptyMap(), + ), + msgId = 2, now = now.minusSeconds(30 * 86400), + ) + val events = StubMsgEvents() + val store = RecordingHistoryStore().apply { confirmed.add("F1") } // 只有 F1 归档确认成功 + val job = HistorySweepJob( + f, events, HistoryProps().apply { historyStoreEnabled = true }, historyStore = store, + ) + + val outcome = job.run(now) + + assertEquals(2, outcome.selected) + assertEquals(1, outcome.archived) + assertEquals(1, outcome.purged) + assertEquals(null, f.findMainRow("F1")) // §8.2 步骤 3：确认成功 → 物理删除 + assertTrue(f.findMainRow("F2") != null) // 失败或不明确的保留重试（步骤 5） + assertEquals(0, events.rows.size) // DELETED 航班清除不再发业务删除事件 + } + + @Test + fun `never-fdel lifecycle purge emits tombstone before deletion`() { + val f = seededFlight("F3", deleted = false, idleDays = 30) // ACTIVE 且超兜底窗（§8.1） + val events = StubMsgEvents() + val store = RecordingHistoryStore() + val job = HistorySweepJob( + f, events, HistoryProps().apply { historyStoreEnabled = true }, historyStore = store, + ) + + job.run(now) + + // §7.3/§8.2 步骤 4：未经 FDEL 的航班，清除前补发一次删除事件 + val tombstone = events.rows.values.single { it.eventType == EventType.TOMBSTONE } + assertEquals("F3", tombstone.partitionKey) + assertTrue(tombstone.payloadJson.contains("\"deleted\":true")) + assertEquals(null, f.findMainRow("F3")) } } diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/FdelAndAdftProcessorTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/processing/FdelAndAdftProcessorTest.kt new file mode 100644 index 0000000..53b95b5 --- /dev/null +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/processing/FdelAndAdftProcessorTest.kt @@ -0,0 +1,172 @@ +package com.gzzn.omms.msgexchange.processing + +import com.fasterxml.jackson.databind.ObjectMapper +import com.gzzn.omms.msgexchange.config.OperationDayProps +import com.gzzn.omms.msgexchange.codec.FlopPayload +import com.gzzn.omms.msgexchange.domain.DecodedMessage +import com.gzzn.omms.msgexchange.domain.EventType +import com.gzzn.omms.msgexchange.domain.MetaFields +import com.gzzn.omms.msgexchange.domain.MsgEvent +import com.gzzn.omms.msgexchange.domain.MsgKind +import com.gzzn.omms.msgexchange.domain.ProcState +import com.gzzn.omms.msgexchange.domain.ProcStatus +import com.gzzn.omms.msgexchange.domain.Targets +import com.gzzn.omms.msgexchange.domain.flight.FlightState +import com.gzzn.omms.msgexchange.infra.persistence.PersistOutcome +import com.gzzn.omms.msgexchange.infra.stub.StubBackfillTodo +import com.gzzn.omms.msgexchange.infra.stub.StubFlightState +import com.gzzn.omms.msgexchange.infra.stub.StubMsgEvents +import com.gzzn.omms.msgexchange.infra.stub.StubPipelineLock +import com.gzzn.omms.msgexchange.infra.stub.StubPipelineTx +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test +import java.time.LocalDate + +/** + * FDEL（§6.2）+ ADFT（§6.3/§2.1）不变量： + * ACTIVE→DELETED 发布一次 tombstone 且明细保留；重复/迟到 FDEL 幂等不推进版本； + * ADFT 重激活恢复 ACTIVE；新实例建立；缺失标量不误删（§2.1 保守语义）。 + */ +class FdelAndAdftProcessorTest { + + private val msgId = 7L + + private fun head() = ProcState(msgId, ProcStatus.PENDING) + + private fun msg() = DecodedMessage( + meta = MetaFields("AODB", "FDEL", "", 9L, 1L), + kind = MsgKind.Fdel, + rawXml = "", + body = FlopPayload("121"), + ) + + private fun flights(): StubFlightState { + val f = StubFlightState() + val outcome = f.persistFullState( + com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot( + "121", LocalDate.of(2026, 12, 15), FlightState.ACTIVE, 4, + mapOf("FLNO" to "CA001"), emptyMap(), + ), + msgId = 1, + now = java.time.Instant.now(), + ) + check(outcome != PersistOutcome.DAY_GUARD_VIOLATION) + return f + } + + @Test + fun `active flight deletion publishes single tombstone and keeps details`() { + val f = flights() + val events = StubMsgEvents() + val todo = StubBackfillTodo() + + val result = FdelProcessor(StubPipelineTx(), StubPipelineLock(), f, events, todo, ObjectMapper()) + .apply(head(), msg(), FlopPayload("121")) + + assertEquals(ApplyResult.Succeeded, result) + assertEquals(FlightState.DELETED, f.findMainRow("121")!!.state) + assertEquals(5L, f.findMainRow("121")!!.stateVersion) + val tombstones = events.rows.values.filter { it.eventType == EventType.TOMBSTONE } + assertEquals(1, tombstones.size) + assertEquals(Targets.KAFKA_SCHD, tombstones.single().target) + assertTrue(tombstones.single().payloadJson.contains("\"deleted\":true")) + assertEquals(1, todo.count()) + } + + @Test + fun `repeated FDEL is idempotent without version bump or duplicate event`() { + val f = flights() + val events = StubMsgEvents() + val proc = FdelProcessor(StubPipelineTx(), StubPipelineLock(), f, events, null, ObjectMapper()) + + proc.apply(head(), msg(), FlopPayload("121")) + val version = f.findMainRow("121")!!.stateVersion + val result = proc.apply(head(), msg(), FlopPayload("121")) + + assertEquals(ApplyResult.Succeeded, result) // §6.2 步骤 3：幂等成功 + assertEquals(version, f.findMainRow("121")!!.stateVersion) + assertEquals(1, events.rows.values.count { it.eventType == EventType.TOMBSTONE }) + } + + @Test + fun `FDEL of unknown flight is idempotent success`() { + val f = StubFlightState() + val events = StubMsgEvents() + + val result = FdelProcessor(StubPipelineTx(), StubPipelineLock(), f, events, null, ObjectMapper()) + .apply(head(), msg(), FlopPayload("999")) + + assertEquals(ApplyResult.Succeeded, result) // §6.2 步骤 4：迟到不报错 + assertEquals(0, events.rows.size) + } + + @Test + fun `ADFT revives deleted flight and sets provided scalars`() { + val f = flights() + f.markDeleted("121", msgId = 2, now = java.time.Instant.now()) + val events = StubMsgEvents() + val adft = AdftProcessor( + StubPipelineTx(), StubPipelineLock(), f, events, null, + OperationDayProps().apply { zone = "Asia/Shanghai" }, ObjectMapper(), + ) + val record = com.gzzn.omms.msgexchange.domain.flight.ScheduleRecord( + "121", mapOf("SODT" to "15DEC261723", "FLNO" to "CA002"), + ) + + val result = adft.apply(head(), msg(), record) + + assertEquals(ApplyResult.Succeeded, result) + val main = f.findMainRow("121")!! + assertEquals(FlightState.ACTIVE, main.state) // §6.3 生命周期重激活 + assertEquals(LocalDate.of(2026, 12, 15), main.operationDay) + assertEquals("CA002", f.loadFullSnapshot("121")!!.scalars["FLNO"]) + } + + @Test + fun `ADFT of unknown flight creates new instance with operation day from SODT`() { + val f = StubFlightState() + val events = StubMsgEvents() + val adft = AdftProcessor( + StubPipelineTx(), StubPipelineLock(), f, events, null, + OperationDayProps().apply { zone = "Asia/Shanghai" }, ObjectMapper(), + ) + + val result = adft.apply( + head(), msg(), + com.gzzn.omms.msgexchange.domain.flight.ScheduleRecord("555", mapOf("SODT" to "20DEC261723")), + ) + + assertEquals(ApplyResult.Succeeded, result) + val main = f.findMainRow("555")!! + assertEquals(FlightState.ACTIVE, main.state) + assertEquals(LocalDate.of(2026, 12, 20), main.operationDay) // §2.1：含 SODT 直接计算 + } + + @Test + fun `ADFT merge does not clear absent scalars`() { + val f = StubFlightState() + f.persistFullState( + com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot( + "555", LocalDate.of(2026, 12, 20), FlightState.ACTIVE, 2, + mapOf("FLNO" to "XX100", "REMC" to "keep-me"), emptyMap(), + ), + msgId = 1, now = java.time.Instant.now(), + ) + val adft = AdftProcessor( + StubPipelineTx(), StubPipelineLock(), f, StubMsgEvents(), null, + OperationDayProps().apply { zone = "Asia/Shanghai" }, ObjectMapper(), + ) + + adft.apply(head(), msg(), com.gzzn.omms.msgexchange.domain.flight.ScheduleRecord("555", mapOf("FLNO" to "XX200"))) + + val scalars = f.loadFullSnapshot("555")!!.scalars + assertEquals("XX200", scalars["FLNO"]) + assertEquals("keep-me", scalars["REMC"]) // §2.1 保守 Set-only：缺失不 Clear + } + + @Suppress("unused") + private fun unused() { + MsgEvent(target = "x", partitionKey = "y", payloadJson = "") + } +} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/FlightSchdInvariantTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/processing/FlightSchdInvariantTest.kt deleted file mode 100644 index 12f4e0d..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/FlightSchdInvariantTest.kt +++ /dev/null @@ -1,355 +0,0 @@ -package com.gzzn.omms.msgexchange.processing - -import com.gzzn.omms.msgexchange.MutableClock -import com.gzzn.omms.msgexchange.codec.DecodeResult -import com.gzzn.omms.msgexchange.codec.XmlCodec -import com.gzzn.omms.msgexchange.config.PipelineProps -import com.gzzn.omms.msgexchange.domain.DecodedMessage -import com.gzzn.omms.msgexchange.domain.Decision -import com.gzzn.omms.msgexchange.domain.ErrorClass -import com.gzzn.omms.msgexchange.domain.FlightChange -import com.gzzn.omms.msgexchange.domain.MetaFields -import com.gzzn.omms.msgexchange.domain.MsgEvent -import com.gzzn.omms.msgexchange.domain.MsgKind -import com.gzzn.omms.msgexchange.domain.NotifyPayload -import com.gzzn.omms.msgexchange.domain.ProcState -import com.gzzn.omms.msgexchange.domain.ProcStatus -import com.gzzn.omms.msgexchange.domain.SchdPush -import com.gzzn.omms.msgexchange.domain.Targets -import com.gzzn.omms.msgexchange.infra.persistence.CminmsgInboxRepository -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository -import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository -import com.gzzn.omms.msgexchange.infra.persistence.PipelineTransactionManager -import com.gzzn.omms.msgexchange.infra.persistence.ProcStateRepository -import com.gzzn.omms.msgexchange.infra.persistence.ReqTrackRepository -import com.gzzn.omms.msgexchange.infra.retry.FailureScheduler -import com.gzzn.omms.msgexchange.infra.retry.ProcFailure -import com.gzzn.omms.msgexchange.infra.stub.StubFlightSchd -import com.gzzn.omms.msgexchange.infra.stub.StubMsgEvents -import com.gzzn.omms.msgexchange.infra.stub.StubPipelineTransactionManager -import com.gzzn.omms.msgexchange.infra.stub.StubProcState -import com.gzzn.omms.msgexchange.infra.stub.StubReqTrack -import com.gzzn.omms.msgexchange.support.seedIncremental -import com.gzzn.omms.msgexchange.support.seedSnapshot -import org.junit.jupiter.api.AfterEach -import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Assertions.assertFalse -import org.junit.jupiter.api.Assertions.assertNotNull -import org.junit.jupiter.api.Assertions.assertNull -import org.junit.jupiter.api.Assertions.assertTrue -import org.junit.jupiter.api.BeforeEach -import org.junit.jupiter.api.Test -import java.time.Instant -import java.time.ZoneId -import java.util.TimeZone - -/** - * U09 / U29 不变量门禁测试（ACM2-28 FS6 核心验收门槛）： - * 1. I2 不变量：航班状态变更、MSG_EVENT 插入、PROC_STATE 终态在单事务原子提交；失败整体回滚重放； - * 2. 门槛 1（崩溃幂等无自增）：重放相同快照报文后 SCHD_GEN.VERSION 绝不发生二次自增； - * 3. 门槛 2（CAS 防并发）：伪造版本过期模拟并发快照，准确拦截 CAS 冲突并转入 FAILED(INFRA) 与退避； - * 4. 门槛 3（ADFT 存活保障）：增量 ADFT 航班在后续 DNLD 跨代替换后天然存活、不被差删；跨代迁移保护； - * 5. UTC 方言测试：非 UTC JVM 时区下读写与幂等判据无漂移。 - */ -class FlightSchdInvariantTest { - - private lateinit var flightSchd: StubFlightSchd - private lateinit var procState: StubProcState - private lateinit var msgEvents: StubMsgEvents - private lateinit var reqTrack: StubReqTrack - private lateinit var txManager: StubPipelineTransactionManager - private lateinit var inbox: FakeInbox - private lateinit var clock: MutableClock - private lateinit var scheduler: FailureScheduler - private lateinit var procFailure: ProcFailure - private lateinit var props: PipelineProps - - class FakeInbox : CminmsgInboxRepository { - val raws = mutableMapOf() - val backfilled = mutableListOf() - - override fun insertRaw(rawXml: String): Long = 1L - override fun rawOf(cminmsgsId: Long): String? = raws[cminmsgsId] - override fun pollUnprocessed(afterId: Long, limit: Int): List = emptyList() - override fun backfillOnSuccess(cminmsgsId: Long, sndr: String, type: String, styp: String, seqn: Long) { - backfilled.add(cminmsgsId) - } - } - - class SimpleCodec(private val decodedMsg: DecodedMessage) : XmlCodec { - override fun decode(rawXml: String): DecodeResult = DecodeResult.Ok(decodedMsg) - override fun encodeRqrd(kind: String, rangeJson: String): String = "" - } - - @BeforeEach - fun setUp() { - flightSchd = StubFlightSchd() - procState = StubProcState() - msgEvents = StubMsgEvents() - reqTrack = StubReqTrack() - txManager = StubPipelineTransactionManager() - inbox = FakeInbox() - clock = MutableClock(MutableClock.BASE) - props = PipelineProps() - scheduler = FailureScheduler(props, clock) - procFailure = ProcFailure(procState, scheduler) - SnapshotFlow.StageResult.parser = null - } - - @AfterEach - fun tearDown() { - SnapshotFlow.StageResult.parser = null - } - - // ========================================================================= - // I2 不变量测试：普通报文事务 2 扩展（变更 + 事件 + SUCCEEDED 原子提交） - // ========================================================================= - - @Test - fun `I2 - MessageProcessor atomically commits flightChanges, msgEvents, and SUCCEEDED`() { - val headId = 101L - inbox.raws[headId] = "F101" - procState.insert(headId, ProcStatus.PENDING) - - val meta = MetaFields("AODB", "FLOP", "DELY", 1L, 20260907120000L) - val decoded = DecodedMessage(meta, MsgKind.Flop("DELY"), "F101") - val codecHolder = CodecHolder(SimpleCodec(decoded)) - - val handler = object : Handler { - override val kind: MsgKind = MsgKind.Flop("DELY") - override fun decide(flightView: Map>, msg: DecodedMessage): Decision { - return Decision( - flightChanges = listOf(FlightChange("F101", mapOf("FLID" to "F101", "STAT" to "DELAYED"))), - msgNotifies = listOf(NotifyPayload("""{"flid":"F101","event":"DELAY"}""")), - schdPush = listOf(SchdPush("F101", """{"FLID":"F101","STAT":"DELAYED"}""")), - ) - } - } - val handlerHolder = HandlerHolder(HandlerRegistry(listOf(handler))) - val snapshotFlow = SnapshotFlow(procState, flightSchd, msgEvents, reqTrack, procFailure, txManager, inbox) - val processor = MessageProcessor( - inbox, procState, msgEvents, codecHolder, handlerHolder, - flightSchd, snapshotFlow, procFailure, props, txManager, - ) - - val head = procState.headUnfinished()!! - processor.processOne(head) - - // 断言：FLIGHT_SCHD 有更新（字段列 DELAYED 已落库） - val fields = flightSchd.findByFlid("F101") - assertNotNull(fields) - assertEquals("DELAYED", fields!!["STAT"]) - - // 断言：MSG_EVENT 写入了 KAFKA_MSG 和 KAFKA_SCHD 两条事件 - val evtMsg = msgEvents.headUnsent(Targets.KAFKA_MSG) - assertNotNull(evtMsg) - val evtSchd = msgEvents.headUnsent(Targets.KAFKA_SCHD) - assertNotNull(evtSchd) - - // 断言：伴生状态更新为 SUCCEEDED - val currentHead = procState.headUnfinished() - assertNull(currentHead) // 无未完成队头，说明 101 已终态 - - // 断言：共享信箱 backfill 成功调用 - assertTrue(inbox.backfilled.contains(headId)) - } - - // ========================================================================= - // FS6 门槛 1：崩溃幂等无自增（重放相同快照报文，SCHD_GEN.VERSION 绝不发生二次自增） - // ========================================================================= - - @Test - fun `Threshold 1 - Crash idempotency without version re-increment`() { - val headId = 201L - val day = "2026-09-07" - procState.insert(headId, ProcStatus.PENDING) - - val meta = MetaFields("AODB", "SCHD", "DNLD", 1L, 20260907030000L) - val decoded = DecodedMessage(meta, MsgKind.Schd(MsgKind.SchdSubtype.DNLD), "") - - // 配置 staging 解析模拟产出 2 条航班 - val flights = listOf( - "FL_01" to mapOf("FLID" to "FL_01", "FLNO" to "CA1234"), - "FL_02" to mapOf("FLID" to "FL_02", "FLNO" to "MU5678"), - ) - SnapshotFlow.StageResult.parser = { SnapshotFlow.StageResult.Ok(day, flights) } - - // 模拟快照单事务内崩溃注入（例如在提交前崩溃回滚） - var crashInjected = true - val rollbackTxManager = object : PipelineTransactionManager { - override fun inTransaction(block: () -> T): T { - if (crashInjected) { - try { - block() - } finally { - // 事务回滚：清除未提交修改并恢复状态 - flightSchd.clear() - procState.update(headId, ProcStatus.PENDING) - } - throw RuntimeException("crash-before-pg-commit") - } - return block() - } - } - - val snapshotFlowCrashing = SnapshotFlow(procState, flightSchd, msgEvents, reqTrack, procFailure, rollbackTxManager, inbox) - - // 首次运行：事务内崩溃注入 - val head1 = procState.headUnfinished()!! - try { - snapshotFlowCrashing.publishSnapshot(head1, decoded) - } catch (_: RuntimeException) { - // 崩溃发生 - } - - // 崩溃后断言：由于 PG 单事务回滚，SCHD_GEN 不存在半成品中间态，版本未推进 - assertNull(flightSchd.getGen(day)) - - // 重放相同报文（正常完成） - crashInjected = false - val snapshotFlowNormal = SnapshotFlow(procState, flightSchd, msgEvents, reqTrack, procFailure, txManager, inbox) - val headReplay = procState.headUnfinished()!! - snapshotFlowNormal.publishSnapshot(headReplay, decoded) - - // 门槛 1 断言：版本绝不发生二次自增，版本号精确为 1L（根除旧 Redis 两阶段二次自增缺陷） - val genAfterReplay = flightSchd.getGen(day) - assertNotNull(genAfterReplay) - assertEquals(1L, genAfterReplay!!.version) - assertEquals(setOf("FL_01", "FL_02"), genAfterReplay.flids) - } - // ========================================================================= - // FS6 门槛 2：CAS 防并发（伪造版本过期模拟并发快照，拦截 CAS 冲突并转入 FAILED(INFRA)） - // ========================================================================= - - @Test - fun `Threshold 2 - CAS prevents concurrency and enters FAILED INFRA`() { - val headId = 202L - val day = "2026-09-07" - procState.insert(headId, ProcStatus.PENDING) - - val meta = MetaFields("AODB", "SCHD", "DNLD", 2L, 20260907033000L) - val decoded = DecodedMessage(meta, MsgKind.Schd(MsgKind.SchdSubtype.DNLD), "") - - val flights = listOf("FL_01" to mapOf("FLID" to "FL_01")) - SnapshotFlow.StageResult.parser = { SnapshotFlow.StageResult.Ok(day, flights) } - - // 先预置日代版本为 5L（模拟另一并发实例已经推进了版本） - flightSchd.putGenIfVersion(day, 0L, FlightSchdRepository.GenMeta(day, 5L, setOf("FL_OLD"))) - - // 构造一个在读取版本后版本被篡改的场景（模拟读到 5L 后，外部并发变成了 6L） - val mockFlightSchd = object : FlightSchdRepository by flightSchd { - override fun getGen(day: String): FlightSchdRepository.GenMeta? { - // 模拟读取时返回版本 5L - return FlightSchdRepository.GenMeta(day, 5L, setOf("FL_OLD")) - } - - override fun putGenIfVersion(day: String, expected: Long, newGen: FlightSchdRepository.GenMeta, now: Instant): Boolean { - // 模拟 CAS 校验失败（数据库已被并发推进，expected 5 已过期） - return false - } - } - - val snapshotFlow = SnapshotFlow(procState, mockFlightSchd, msgEvents, reqTrack, procFailure, txManager, inbox) - val head = procState.headUnfinished()!! - snapshotFlow.publishSnapshot(head, decoded) - - // 门槛 2 断言：CAS 冲突被拦截，状态转换为 FAILED(INFRA)，带有退避与告警错误信息 - val failedHead = procState.headUnfinished()!! - assertEquals(ProcStatus.FAILED, failedHead.state) - assertEquals(ErrorClass.INFRA, failedHead.errorClass) - assertTrue(failedHead.lastError?.contains("gen-cas-conflict") == true) - assertNotNull(failedHead.nextAttemptAt) - } - - // ========================================================================= - // FS6 门槛 3：ADFT 存活保障（增量 ADFT 航班在后续 DNLD 跨代替换后天然存活、不被差删） - // ========================================================================= - - @Test - fun `Threshold 3 - ADFT flights survive subsequent DNLD replacement`() { - val day = "2026-09-07" - - // 1. 增量更新写入一条临时加飞航班 ADFT（FDAY 为 NULL） - val adftChange = FlightChange(flid = "ADFT_888", fields = mapOf("FLID" to "ADFT_888", "FLTY" to "ADFT")) - seedIncremental(flightSchd, listOf(adftChange)) - - // 2. 写入旧代的一条定期计划航班 REG_OLD（FDAY 为 2026-09-07） - seedSnapshot(flightSchd, day, listOf("REG_OLD" to mapOf("FLID" to "REG_OLD", "FLTY" to "REG"))) - flightSchd.putGenIfVersion(day, 0L, FlightSchdRepository.GenMeta(day, 1L, setOf("REG_OLD"))) - - // 3. 执行下一轮快照 DNLD，新代仅包含 REG_NEW（REG_OLD 不在新代中，属于待删差集；ADFT 也不在新代中） - val headId = 203L - procState.insert(headId, ProcStatus.PENDING) - val meta = MetaFields("AODB", "SCHD", "DNLD", 3L, 20260907040000L) - val decoded = DecodedMessage(meta, MsgKind.Schd(MsgKind.SchdSubtype.DNLD), "") - - val newFlights = listOf("REG_NEW" to mapOf("FLID" to "REG_NEW", "FLTY" to "REG")) - SnapshotFlow.StageResult.parser = { SnapshotFlow.StageResult.Ok(day, newFlights) } - - val snapshotFlow = SnapshotFlow(procState, flightSchd, msgEvents, reqTrack, procFailure, txManager, inbox) - val head = procState.headUnfinished()!! - snapshotFlow.publishSnapshot(head, decoded) - - // 门槛 3 核心断言： - // ① 旧代航班 REG_OLD 被按代差删清除 - assertNull(flightSchd.findByFlid("REG_OLD")) - - // ② 新代航班 REG_NEW 成功写入 - assertNotNull(flightSchd.findByFlid("REG_NEW")) - - // ③ 增量 ADFT_888 航班由于 FDAY=NULL 天然存活、绝不被误删！ - val adftRecord = flightSchd.findByFlid("ADFT_888") - assertNotNull(adftRecord) - assertEquals("ADFT_888", adftRecord!!["FLID"]) - } - - @Test - fun `Threshold 3 - Migrated flight across days is protected from diff deletion of old day`() { - // 场景：航班 FL_MIG 原属 2026-09-07，后来迁移到了 2026-09-08（FDAY 更新为 09-08） - val dayOld = "2026-09-07" - val dayNew = "2026-09-08" - - // 初始属旧代 - seedSnapshot(flightSchd, dayOld, listOf("FL_MIG" to mapOf("FLID" to "FL_MIG", "REMC" to "07"))) - flightSchd.putGenIfVersion(dayOld, 0L, FlightSchdRepository.GenMeta(dayOld, 1L, setOf("FL_MIG"))) - - // 随后 09-08 快照写入将 FDAY 更新为 2026-09-08 - seedSnapshot(flightSchd, dayNew, listOf("FL_MIG" to mapOf("FLID" to "FL_MIG", "REMC" to "08"))) - - // 此时 09-07 再次执行差删（差集中包含 FL_MIG） - val deleted = flightSchd.deleteDiffByDay(dayOld, listOf("FL_MIG")) - - // 断言：FL_MIG 虽在差集，但由于 FDAY 已迁移至 09-08，受到域化差删保护，删除数为 0，记录依然存活！ - assertEquals(0, deleted) - assertNotNull(flightSchd.findByFlid("FL_MIG")) - } - - // ========================================================================= - // UTC 时区规范测试：非 UTC JVM 默认时区与会话下写入读取无漂移 - // ========================================================================= - - @Test - fun `UTC Dialect - Timestamps and instant evaluations are immune to JVM timezone drift`() { - val originalTz = TimeZone.getDefault() - try { - // 切换 JVM 默认时区为非 UTC（东京 +09:00 与 纽约 -05:00） - TimeZone.setDefault(TimeZone.getTimeZone(ZoneId.of("Asia/Tokyo"))) - - val now = Instant.parse("2026-09-07T08:00:00.123456Z") - seedSnapshot(flightSchd, "2026-09-07", listOf("TZ_01" to mapOf("FLID" to "TZ_01")), now) - flightSchd.putGenIfVersion("2026-09-07", 0L, FlightSchdRepository.GenMeta("2026-09-07", 1L, setOf("TZ_01")), now) - - val gen = flightSchd.getGen("2026-09-07") - assertNotNull(gen) - assertEquals(now, gen!!.updatedAt) - - // 再次切换到西五区 - TimeZone.setDefault(TimeZone.getTimeZone(ZoneId.of("America/New_York"))) - val genNy = flightSchd.getGen("2026-09-07") - assertNotNull(genNy) - assertEquals(now, genNy!!.updatedAt) // 绝无时区漂移 - } finally { - TimeZone.setDefault(originalTz) - } - } -} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/GtdtPipelineTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/processing/GtdtPipelineTest.kt deleted file mode 100644 index 791ad59..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/GtdtPipelineTest.kt +++ /dev/null @@ -1,82 +0,0 @@ -package com.gzzn.omms.msgexchange.processing - -import com.gzzn.omms.msgexchange.MutableClock -import com.gzzn.omms.msgexchange.codec.DecodeResult -import com.gzzn.omms.msgexchange.codec.JacksonXmlCodec -import com.gzzn.omms.msgexchange.config.PipelineProps -import com.gzzn.omms.msgexchange.domain.ProcState -import com.gzzn.omms.msgexchange.domain.ProcStatus -import com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine -import com.gzzn.omms.msgexchange.infra.persistence.CminmsgInboxRepository -import com.gzzn.omms.msgexchange.infra.persistence.PipelineTransactionManager -import com.gzzn.omms.msgexchange.infra.retry.FailureScheduler -import com.gzzn.omms.msgexchange.infra.retry.ProcFailure -import com.gzzn.omms.msgexchange.infra.stub.StubFlightSchd -import com.gzzn.omms.msgexchange.infra.stub.StubMsgEvents -import com.gzzn.omms.msgexchange.infra.stub.StubProcState -import com.gzzn.omms.msgexchange.infra.stub.StubReqTrack -import com.gzzn.omms.msgexchange.processing.handlers.GtdtHandler -import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Test - -/** P2-0-1：GTDT 报文端到端 SUCCEEDED + 三门读回（真实 XmlCodec + GtdtHandler）。 */ -class GtdtPipelineTest { - - @Test - fun `GTDT XML end-to-end SUCCEEDED with three gates persisted`() { - val raw = """ - - - - AODB900120260908120000 - FLOPGTDT - - - FL_GTDT_E2E - A1D - B2I - C3D - - - """.trimIndent() - - val procState = StubProcState() - val flightSchd = StubFlightSchd() - val msgEvents = StubMsgEvents() - val reqTrack = StubReqTrack() - val txManager = object : PipelineTransactionManager { - override fun inTransaction(block: () -> T): T = block() - } - val inbox = object : CminmsgInboxRepository { - override fun insertRaw(rawXml: String) = 1L - override fun rawOf(cminmsgsId: Long) = raw - override fun pollUnprocessed(afterId: Long, limit: Int) = emptyList() - override fun backfillOnSuccess(cminmsgsId: Long, sndr: String, type: String, styp: String, seqn: Long) = Unit - } - val clock = MutableClock(MutableClock.BASE) - val procFailure = ProcFailure(procState, FailureScheduler(PipelineProps(), clock)) - val codec = JacksonXmlCodec() - val handler = GtdtHandler() - val snapshot = SnapshotFlow(procState, flightSchd, msgEvents, reqTrack, procFailure, txManager, inbox) - val processor = MessageProcessor( - inbox, procState, msgEvents, - CodecHolder(codec), - HandlerHolder(HandlerRegistry(listOf(handler))), - flightSchd, snapshot, procFailure, PipelineProps(), txManager, - ) - - val headId = 42L - procState.insert(headId, ProcStatus.PENDING) - processor.processOne(ProcState(headId, ProcStatus.PENDING)) - - assertEquals(ProcStatus.SUCCEEDED, procState.snapshotOf(headId)!!.state) - val next = flightSchd.findNextStateByFlid("FL_GTDT_E2E")!! - assertEquals(3, next.collections["GTDT"]!!.size) - assertEquals("3", next.collections["GTDT"]!![1]["GTNO"]) - assertEquals("C3", next.collections["GTDT"]!![2]["GATE"]) - assertEquals(1L, next.stateVersion) - - val roundtrip = FlightStateEngine.fromFlightFields("FL_GTDT_E2E", flightSchd.findByFlid("FL_GTDT_E2E")!!) - assertEquals(3, roundtrip.collections["GTDT"]!!.size) - } -} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/MessageProcessorTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/processing/MessageProcessorTest.kt deleted file mode 100644 index b0ff3a6..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/MessageProcessorTest.kt +++ /dev/null @@ -1,388 +0,0 @@ -package com.gzzn.omms.msgexchange.processing - -import com.gzzn.omms.msgexchange.MutableClock -import com.gzzn.omms.msgexchange.codec.DecodeFailure -import com.gzzn.omms.msgexchange.codec.DecodeResult -import com.gzzn.omms.msgexchange.codec.XmlCodec -import com.gzzn.omms.msgexchange.config.PipelineProps -import com.gzzn.omms.msgexchange.domain.DecodedMessage -import com.gzzn.omms.msgexchange.domain.Decision -import com.gzzn.omms.msgexchange.domain.ErrorClass -import com.gzzn.omms.msgexchange.domain.EventStatus -import com.gzzn.omms.msgexchange.domain.MsgEvent -import com.gzzn.omms.msgexchange.domain.MsgKind -import com.gzzn.omms.msgexchange.domain.MetaFields -import com.gzzn.omms.msgexchange.domain.NotifyPayload -import com.gzzn.omms.msgexchange.domain.ProcState -import com.gzzn.omms.msgexchange.domain.ProcStatus -import com.gzzn.omms.msgexchange.domain.SchdPush -import com.gzzn.omms.msgexchange.domain.Targets -import com.gzzn.omms.msgexchange.infra.persistence.BackfillTodoRepository -import com.gzzn.omms.msgexchange.infra.persistence.CminmsgInboxRepository -import com.gzzn.omms.msgexchange.infra.persistence.FlightFields -import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository -import com.gzzn.omms.msgexchange.infra.persistence.ProcStateRepository -import com.gzzn.omms.msgexchange.infra.persistence.PumpJobRepository -import com.gzzn.omms.msgexchange.infra.persistence.ReqTrackRepository -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository -import com.gzzn.omms.msgexchange.infra.persistence.PipelineTransactionManager -import com.gzzn.omms.msgexchange.infra.retry.FailureScheduler -import com.gzzn.omms.msgexchange.infra.retry.ProcFailure -import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Assertions.assertNotNull -import org.junit.jupiter.api.Assertions.assertNull -import org.junit.jupiter.api.Assertions.assertThrows -import org.junit.jupiter.api.Assertions.assertTrue -import org.junit.jupiter.api.Test -import java.time.Instant - -/** - * U08/U10/U11：processOne 失败闭环—— - * ① 意外异常在边界内写 FAILED(INFRA)+退避，重试达上限转 DEAD(EXHAUSTED)（Pump 内部异常不杀泵）； - * ② 未注册/CODEC_ERROR/staging 未实装 → FAILED（绝不直接终态）；MALFORMED → DEAD； - * ③ InterruptedException/致命 Error 不被普通恢复逻辑吞掉。 - */ -class MessageProcessorTest { - - private class FakeProcState : ProcStateRepository { - var record = mutableMapOf() - val bound = mutableMapOf() - - override fun insert(cminmsgsId: Long, state: ProcStatus) { record[cminmsgsId] = ProcState(cminmsgsId, state) } - - override fun exists(cminmsgsId: Long): Boolean = record.containsKey(cminmsgsId) - - override fun headUnfinished(): ProcState? = null - - override fun tryBindIdentity(cminmsgsId: Long, identityKey: String): Boolean { - val owner = bound[identityKey] - if (owner != null && owner != cminmsgsId) return false - bound[identityKey] = cminmsgsId - record[cminmsgsId] = record[cminmsgsId]?.copy(identityKey = identityKey) ?: ProcState(cminmsgsId, ProcStatus.PENDING, identityKey = identityKey) - return true - } - - override fun ownerOfIdentity(identityKey: String): Long? = bound[identityKey] - - override fun update( - cminmsgsId: Long, state: ProcStatus, nextAttemptAt: Instant?, attempts: Int?, - errorClass: ErrorClass?, lastError: String?, - ) { - val old = record[cminmsgsId] ?: ProcState(cminmsgsId, state) - record[cminmsgsId] = old.copy( - state = state, - nextAttemptAt = nextAttemptAt ?: old.nextAttemptAt, - attempts = attempts ?: old.attempts, - errorClass = errorClass ?: old.errorClass, - lastError = lastError ?: old.lastError, - ) - } - - override fun requeueByErrorClasses(errorClasses: List): Int { - var n = 0 - record.keys.toList().forEach { id -> - val s = record[id]!! - if (s.errorClass != null && s.errorClass in errorClasses && - (s.state == ProcStatus.FAILED || s.state == ProcStatus.DEAD)) { - record[id] = s.copy(state = ProcStatus.PENDING, attempts = 0, nextAttemptAt = null) - n++ - } - } - return n - } - - fun state(id: Long): ProcState = record.getValue(id) - } - - private class FakeInbox : CminmsgInboxRepository { - var raws = mutableMapOf() - var throwOnRawOf: Throwable? = null - var throwOnBackfill: Throwable? = null - val backfilled = mutableListOf>() - - override fun insertRaw(rawXml: String): Long = 0 - - override fun pollUnprocessed(afterId: Long, limit: Int): List = emptyList() - - override fun rawOf(cminmsgsId: Long): String? { - throwOnRawOf?.let { throw it } - return raws[cminmsgsId] - } - - override fun backfillOnSuccess(cminmsgsId: Long, sndr: String, type: String, styp: String, seqn: Long) { - throwOnBackfill?.let { throw it } - backfilled += listOf(cminmsgsId, sndr, type, styp, seqn) - } - } - - private class FakeEvents : MsgEventRepository { - val inserted = mutableListOf>() - - override fun insertAll(events: List): List { inserted += events; return events.indices.map { it.toLong() + 1 } } - - override fun headUnsent(target: String): MsgEvent? = null - override fun claimBatch(target: String, limit: Int): List = emptyList() - override fun markSent(eventId: Long) = Unit - override fun markAllSent(eventIds: List) = Unit - override fun scheduleRetry(eventId: Long, nextAttemptAt: Instant, attempts: Int) = Unit - override fun markDead(eventId: Long, errorClass: ErrorClass, lastError: String, attempts: Int?) = Unit - override fun insertSync(events: List) = Unit - } - - private class FakeCodec : XmlCodec { - var result: DecodeResult = DecodeResult.Ok( - DecodedMessage( - meta = MetaFields(sndr = "AODB", type = "FLOP", styp = "DELY", seqn = 1L, dttm = 20260906120000L), - kind = MsgKind.Flop("DELY"), - rawXml = "", - ), - ) - override fun decode(rawXml: String): DecodeResult = result - override fun encodeRqrd(kind: String, rangeJson: String): String = "" - } - - private class FakeReqTrack : ReqTrackRepository { - override fun findOpenByKind(kind: String): ReqTrackRepository.Req? = null - override fun forceExpireOpenOf(kind: String) = Unit - override fun insert(kind: String, paramsJson: String): Long = 1L - override fun linkCoutmsgs(reqId: Long, coutmsgsId: Long) = Unit - override fun markSent(reqId: Long, sentAt: Instant) = Unit - override fun expireIfWaiting(reqId: Long) = Unit - override fun markDone(reqId: Long) = Unit - } - - private class FakeFlightSchd : FlightSchdRepository { - val flights = mutableMapOf() - val gens = mutableMapOf() - val incrementalChanges = mutableListOf() - - override fun deleteDiffByDay(day: String, delFlids: Collection): Int = 0 - override fun findByFlid(flid: String): FlightFields? = flights[flid] - override fun findByFlids(flids: Collection): Map = - flids.mapNotNull { f -> flights[f]?.let { f to it } }.toMap() - override fun findByDay(day: String): List> = flights.map { it.key to it.value } - override fun findAll(): Map = flights.toMap() - override fun deleteByFlids(flids: Set): Int = 0 - override fun getGen(day: String): FlightSchdRepository.GenMeta? = gens[day] - override fun putGenIfVersion(day: String, expected: Long, newGen: FlightSchdRepository.GenMeta, now: Instant): Boolean { - val cur = gens[day]?.version ?: 0L - if (cur != expected) return false - gens[day] = newGen - return true - } - override fun deleteGenBefore(cutoffDay: String): Int = 0 - override fun findNextStateByFlid(flid: String): com.gzzn.omms.msgexchange.domain.flight.FlightNextState? = - flights[flid]?.let { com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine.fromFlightFields(flid, it) } - override fun persistNextStates( - day: String?, - states: List, - snapshotReplace: Boolean, - now: Instant, - ) { - states.forEach { state -> - val scalarFields = linkedMapOf("FLID" to state.flid) - state.scalars.forEach { (k, v) -> scalarFields[k] = v } - flights[state.flid] = scalarFields + state.collections.mapValues { (_, items) -> - com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(items) - } - } - } - } - - private class FakeTxManager : PipelineTransactionManager { - override fun inTransaction(block: () -> T): T = block() - } - // ---------- helpers ---------- - private val clock = MutableClock(MutableClock.BASE) - - private fun msg(seqn: String) = DecodedMessage( - meta = MetaFields(sndr = "AODB", type = "FLOP", styp = "DELY", seqn = seqn.toLong(), dttm = 20260906120000L), - kind = MsgKind.Flop("DELY"), - rawXml = "", - ) - - private fun head(id: Long = 1, status: ProcStatus = ProcStatus.PENDING, attempts: Int = 0, identityKey: String? = null) = - ProcState(id, status, identityKey = identityKey, attempts = attempts) - - private fun deliveHandler() = object : Handler { - override val kind: MsgKind = MsgKind.Flop("DELY") - override fun decide(flightView: Map>, msg: DecodedMessage): Decision = Decision( - msgNotifies = listOf(NotifyPayload("""{"n":1}""")), - schdPush = listOf(SchdPush(flid = "F1", payloadJson = """{"FLID":"F1","v":2}""")), - ) - } - - private fun processor( - procState: FakeProcState, inbox: FakeInbox, events: FakeEvents, codec: FakeCodec, - registry: HandlerRegistry = HandlerRegistry(emptyList()), - backfillTodo: BackfillTodoRepository? = null, - ): MessageProcessor { - val props = PipelineProps() - val scheduler = FailureScheduler(props, clock) - val procFailure = ProcFailure(procState, scheduler) - val flightSchd = FakeFlightSchd() - val txManager = FakeTxManager() - val snapshot = SnapshotFlow(procState, flightSchd, events, FakeReqTrack(), procFailure, txManager, inbox, backfillTodo = backfillTodo) - return MessageProcessor(inbox, procState, events, CodecHolder(codec), HandlerHolder(registry), flightSchd, snapshot, procFailure, props, txManager, backfillTodo) - } - // ---------- tests ---------- - @Test - fun `no handler is FAILED UNSUPPORTED with backoff - never terminal`() { - val ps = FakeProcState(); val inbox = FakeInbox(); val ev = FakeEvents() - ps.record[1] = head(); inbox.raws[1] = "" - processor(ps, inbox, ev, FakeCodec()).processOne(ps.state(1)) - - val s = ps.state(1) - assertEquals(ProcStatus.FAILED, s.state) - assertEquals(ErrorClass.UNSUPPORTED, s.errorClass) - assertEquals(1, s.attempts) - assertNotNull(s.nextAttemptAt) - assertTrue(s.lastError?.startsWith("no-handler:FLOP-DELY") == true) - } - - @Test - fun `CODEC_ERROR decode failure is FAILED retryable`() { - val ps = FakeProcState(); val inbox = FakeInbox(); val ev = FakeEvents() - ps.record[1] = head(); inbox.raws[1] = "" - val codec = FakeCodec().apply { result = DecodeResult.Err(DecodeFailure(ErrorClass.CODEC_ERROR, "boom")) } - processor(ps, inbox, ev, codec).processOne(ps.state(1)) - - val s = ps.state(1) - assertEquals(ProcStatus.FAILED, s.state) - assertEquals(ErrorClass.CODEC_ERROR, s.errorClass) - assertEquals(1, s.attempts) - assertNotNull(s.nextAttemptAt) - } - - @Test - fun `MALFORMED decode failure stays DEAD terminal`() { - val ps = FakeProcState(); val inbox = FakeInbox(); val ev = FakeEvents() - ps.record[1] = head(); inbox.raws[1] = "" - val codec = FakeCodec().apply { result = DecodeResult.Err(DecodeFailure(ErrorClass.MALFORMED, "bad-xml")) } - processor(ps, inbox, ev, codec).processOne(ps.state(1)) - - val s = ps.state(1) - assertEquals(ProcStatus.DEAD, s.state) - assertEquals(ErrorClass.MALFORMED, s.errorClass) - } - - @Test - fun `legacy failed head at max attempts upgrades to DEAD at entry`() { - val ps = FakeProcState(); val inbox = FakeInbox(); val ev = FakeEvents() - ps.record[1] = head(status = ProcStatus.FAILED, attempts = 5, identityKey = "k") - processor(ps, inbox, ev, FakeCodec()).processOne(ps.state(1)) - - assertEquals(ProcStatus.DEAD, ps.state(1).state) - assertEquals(ErrorClass.EXHAUSTED, ps.state(1).errorClass) - } - - @Test - fun `success path inserts events backfills and marks SUCCEEDED`() { - val ps = FakeProcState(); val inbox = FakeInbox(); val ev = FakeEvents() - ps.record[1] = head(); inbox.raws[1] = "" - val registry = HandlerRegistry(listOf(deliveHandler())) - processor(ps, inbox, ev, FakeCodec(), registry).processOne(ps.state(1)) - - assertEquals(ProcStatus.SUCCEEDED, ps.state(1).state) - val events = ev.inserted.single() - assertEquals(listOf(Targets.KAFKA_MSG, Targets.KAFKA_SCHD), events.map { it.target }) - assertEquals("F1", events.first { it.target == Targets.KAFKA_SCHD }.partitionKey) - assertEquals(listOf(1L, "AODB", "FLOP", "DELY", 1L), inbox.backfilled.single()) - } - - @Test - fun `duplicate identity leads to SKIPPED`() { - val ps = FakeProcState(); val inbox = FakeInbox(); val ev = FakeEvents() - ps.record[1] = head() - inbox.raws[1] = "" - ps.bound["AODB|FLOP|DELY|1"] = 2L // 另一条消息已持有该键 - - processor(ps, inbox, ev, FakeCodec()).processOne(ps.state(1)) - - val s = ps.state(1) - assertEquals(ProcStatus.SKIPPED, s.state) - assertTrue(s.lastError == "duplicate-of:2") - assertTrue(ev.inserted.isEmpty()) - } - - @Test - fun `skipped duplicate still compensates mailbox backfill and persists todo on failure`() { - val ps = FakeProcState(); val inbox = FakeInbox(); val ev = FakeEvents() - ps.record[1] = head() - inbox.raws[1] = "" - inbox.throwOnBackfill = IllegalStateException("mysql-down") - ps.bound["AODB|FLOP|DELY|1"] = 2L - val todo = com.gzzn.omms.msgexchange.infra.stub.StubBackfillTodo() - - processor(ps, inbox, ev, FakeCodec(), backfillTodo = todo).processOne(ps.state(1)) - - // SKIPPED 不变，但共享信箱回填的补偿待办必须落库（否则重复报文被反复轮询） - assertEquals(ProcStatus.SKIPPED, ps.state(1).state) - assertEquals(1, todo.count()) - assertEquals(1L, todo.tasks().single().cminmsgsId) - assertEquals("mysql-down", todo.lastErrorOf(1L)) - } - - // ---------- U08 边界化（评审要求①③） ---------- - @Test - fun `unexpected exception maps to FAILED INFRA with backoff at boundary`() { - val ps = FakeProcState(); val inbox = FakeInbox(); val ev = FakeEvents() - ps.record[1] = head() - inbox.throwOnRawOf = RuntimeException("db-down") - - processor(ps, inbox, ev, FakeCodec()).processOne(ps.state(1)) - - val s = ps.state(1) - assertEquals(ProcStatus.FAILED, s.state) - assertEquals(ErrorClass.INFRA, s.errorClass) - assertEquals(1, s.attempts) - assertNotNull(s.nextAttemptAt) - assertTrue(s.lastError == "db-down") - } - - @Test - fun `repeated unexpected exceptions escalate to DEAD EXHAUSTED at max attempts`() { - val ps = FakeProcState(); val inbox = FakeInbox(); val ev = FakeEvents() - ps.record[1] = head() - inbox.throwOnRawOf = RuntimeException("db-down") - val p = processor(ps, inbox, ev, FakeCodec()) - - var attempts = 0 - while (ps.state(1).state != ProcStatus.DEAD && attempts < 10) { - p.processOne(ps.state(1)) - clock.advance(60_000) // 推进时钟：让退避/毒丸时间语义确定 - attempts++ - } - - val s = ps.state(1) - assertEquals(ProcStatus.DEAD, s.state, "should exhaust within maxAttempts(=5)") - assertEquals(ErrorClass.EXHAUSTED, s.errorClass) - assertTrue(s.lastError?.contains("attempts=5") == true) - assertEquals(5, s.attempts) - } - - @Test - fun `fatal Error is not swallowed by exception recovery`() { - val ps = FakeProcState(); val inbox = FakeInbox(); val ev = FakeEvents() - ps.record[1] = head() - inbox.throwOnRawOf = AssertionError("fatal") - - assertThrows(AssertionError::class.java) { - processor(ps, inbox, ev, FakeCodec()).processOne(ps.state(1)) - } - // 未被普通恢复逻辑改写成 FAILED（Error 不落入 catch Exception） - assertEquals(ProcStatus.PENDING, ps.state(1).state) - } - - @Test - fun `InterruptedException restores flag and propagates`() { - val ps = FakeProcState(); val inbox = FakeInbox(); val ev = FakeEvents() - ps.record[1] = head() - inbox.throwOnRawOf = InterruptedException("stop") - - assertThrows(InterruptedException::class.java) { - processor(ps, inbox, ev, FakeCodec()).processOne(ps.state(1)) - } - assertTrue(Thread.interrupted(), "interrupt flag must be restored") - } -} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/ScheduleProcessorTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/processing/ScheduleProcessorTest.kt new file mode 100644 index 0000000..2414317 --- /dev/null +++ b/src/test/kotlin/com/gzzn/omms/msgexchange/processing/ScheduleProcessorTest.kt @@ -0,0 +1,205 @@ +package com.gzzn.omms.msgexchange.processing + +import com.fasterxml.jackson.databind.ObjectMapper +import com.gzzn.omms.msgexchange.codec.ScheduleBody +import com.gzzn.omms.msgexchange.config.OperationDayProps +import com.gzzn.omms.msgexchange.domain.DecodedMessage +import com.gzzn.omms.msgexchange.domain.MetaFields +import com.gzzn.omms.msgexchange.domain.MsgKind +import com.gzzn.omms.msgexchange.domain.ProcState +import com.gzzn.omms.msgexchange.domain.ProcStatus +import com.gzzn.omms.msgexchange.domain.SnapshotFlag +import com.gzzn.omms.msgexchange.domain.SnapshotResult +import com.gzzn.omms.msgexchange.domain.Targets +import com.gzzn.omms.msgexchange.domain.flight.FlightState +import com.gzzn.omms.msgexchange.infra.persistence.BackfillTodoRepository +import com.gzzn.omms.msgexchange.infra.persistence.FlightStateRepository +import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository +import com.gzzn.omms.msgexchange.infra.persistence.PersistOutcome +import com.gzzn.omms.msgexchange.infra.persistence.PipelineLockRepository +import com.gzzn.omms.msgexchange.infra.persistence.PipelineTransactionManager +import com.gzzn.omms.msgexchange.infra.persistence.ProcStateRepository +import com.gzzn.omms.msgexchange.infra.persistence.SnapshotLogRepository +import com.gzzn.omms.msgexchange.infra.stub.StubFlightState +import com.gzzn.omms.msgexchange.infra.stub.StubMsgEvents +import com.gzzn.omms.msgexchange.infra.stub.StubProcState +import com.gzzn.omms.msgexchange.infra.stub.StubSnapshotLog +import com.gzzn.omms.msgexchange.infra.stub.StubBackfillTodo +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertNull +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test +import java.time.LocalDate + +/** + * §5.1 applyScheduleRecords 不变量：重放幂等、完整性校验整包拒绝 DEAD(PROTOCOL)、 + * 归属日冲突不落地、DELETED 不恢复且记冲突告警、成功路径版本推进+事件+留痕+待办预登记。 + */ +class ScheduleProcessorTest { + + private val msgId = 42L + + private fun head() = ProcState(msgId, ProcStatus.PENDING) + + private fun message(body: ScheduleBody) = DecodedMessage( + meta = MetaFields("AODB", "SCHD", "DNLD", 100L, 1L), + kind = MsgKind.Schd(MsgKind.SchdSubtype.DNLD), + rawXml = "", + body = body, + ) + + private fun makeBody(vararg recs: Pair) = ScheduleBody( + recsDeclared = recs.size, + records = recs.map { (flid, sodt) -> + com.gzzn.omms.msgexchange.domain.flight.ScheduleRecord(flid, mapOf("SODT" to sodt)) + }, + ) + + private class TxRunner(private val failOn: () -> Boolean) : PipelineTransactionManager { + var entered = 0 + override fun inTransaction(block: () -> T): T { + entered++ + if (failOn()) throw IllegalStateException("db-down") + return block() + } + } + + private fun processor( + proc: StubProcState = StubProcState(), + flights: StubFlightState = StubFlightState(), + events: StubMsgEvents = StubMsgEvents(), + log: StubSnapshotLog = StubSnapshotLog(), + todo: StubBackfillTodo? = StubBackfillTodo(), + tx: PipelineTransactionManager? = null, + ): ScheduleProcessor { + val txRunner = tx ?: object : PipelineTransactionManager { + override fun inTransaction(block: () -> T): T = block() + } + return ScheduleProcessor( + txManager = txRunner, + lock = object : PipelineLockRepository { override fun lock() = Unit }, + procState = proc, + flightState = flights, + msgEvents = events, + snapshotLog = log, + backfillTodo = todo, + operationDayProps = OperationDayProps().apply { zone = "Asia/Shanghai"; cutoffHour = 0 }, + mapper = ObjectMapper(), + ) + } + + @Test + fun `happy path persists snapshot with version bump, events, snap log and backfill todo`() { + val flights = StubFlightState() + val events = StubMsgEvents() + val log = StubSnapshotLog() + val todo = StubBackfillTodo() + + val result = processor(flights = flights, events = events, log = log, todo = todo) + .applyScheduleRecords(head(), message(makeBody("121" to "15DEC261723"))) + + assertEquals(ApplyResult.Succeeded, result) + val main = flights.findMainRow("121")!! + assertEquals(1L, main.stateVersion) + assertEquals(LocalDate.of(2026, 12, 15), main.operationDay) + assertEquals(FlightState.ACTIVE, main.state) + assertEquals(2, events.rows.size) // KAFKA_SCHD 整态 + KAFKA_MSG 通知 + assertEquals(1, log.entries.size) + assertEquals(SnapshotResult.COMMITTED, log.entries.single().result) + assertEquals(1, log.entries.single().upserted) + assertEquals(1, todo.count()) // §7.2 事务内预登记 + } + + @Test + fun `replay of succeeded message records idempotent success without writes`() { + val proc = StubProcState() + proc.insert(msgId, ProcStatus.SUCCEEDED) + val flights = StubFlightState() + val log = StubSnapshotLog() + + val result = processor(proc = proc, flights = flights, log = log) + .applyScheduleRecords(head(), message(makeBody("121" to "15DEC261723"))) + + assertEquals(ApplyResult.ReplaySkipped, result) // §5.1 步骤 2 + assertNull(flights.findMainRow("121")) + assertEquals(SnapshotResult.REPLAY_SKIPPED, log.entries.single().result) + } + + @Test + fun `validation failure is DEAD protocol without any persistence`() { + val flights = StubFlightState() + val log = StubSnapshotLog() + val schdBody = ScheduleBody(recsDeclared = 2, records = makeBody("121" to "15DEC261723").records) + + val result = processor(flights = flights, log = log).applyScheduleRecords(head(), message(schdBody)) as ApplyResult.DeadProtocol + assertTrue(result.flags.contains(SnapshotFlag.RECS_DROP)) // §5.2 RECS = 实收数 + assertNull(flights.findMainRow("121")) + assertEquals(SnapshotResult.ROLLED_BACK, log.entries.single().result) + } + + @Test + fun `same flid across operation days is rejected whole-batch without writes`() { + val flights = StubFlightState() + // 预置 121 已归属 12-15；报文声称 12-16 → §5.3 第四行 + flights.persistFullState( + com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot( + "121", LocalDate.of(2026, 12, 15), FlightState.ACTIVE, 1, mapOf("SODT" to "15DEC261723"), emptyMap(), + ), + msgId = 1, + now = java.time.Instant.now(), + ) + val log = StubSnapshotLog() + + val result = processor(flights = flights, log = log).applyScheduleRecords( + head(), message(makeBody("121" to "16DEC261723")), + ) + + assertTrue(result is ApplyResult.DeadProtocol) + assertEquals(LocalDate.of(2026, 12, 15), flights.findMainRow("121")!!.operationDay) // 不迁移归属 + assertEquals(1L, flights.findMainRow("121")!!.stateVersion) // 不改写 + assertEquals(SnapshotResult.ROLLED_BACK, log.entries.single().result) + } + + @Test + fun `DELETED flight stays deleted with revive conflict flag`() { + val flights = StubFlightState() + flights.persistFullState( + com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot( + "121", LocalDate.of(2026, 12, 15), FlightState.DELETED, 7, + mapOf("SODT" to "15DEC261723"), emptyMap(), + ), + msgId = 1, + now = java.time.Instant.now(), + ) + val log = StubSnapshotLog() + + val result = processor(flights = flights, log = log).applyScheduleRecords( + head(), message(makeBody("121" to "15DEC261723")), + ) + + assertEquals(ApplyResult.Succeeded, result) + assertEquals(FlightState.DELETED, flights.findMainRow("121")!!.state) // §5.1 步骤 6 不恢复 + assertEquals(8L, flights.findMainRow("121")!!.stateVersion) + assertEquals(SnapshotResult.COMMITTED, log.entries.single().result) + assertTrue(log.entries.single().flags.contains(SnapshotFlag.SCHD_REVIVE_CONFLICT)) + } + + @Test + fun `infra failure inside transaction throws and leaves nothing persisted`() { + val proc = StubProcState() + val flights = StubFlightState() + val log = StubSnapshotLog() + val todo = StubBackfillTodo() + proc.insert(msgId) + val p = processor(proc = proc, flights = flights, log = log, todo = todo, tx = TxRunner { true }) + + // §9：数据库/内部故障 → 异常上抛，MessageProcessor 记 FAILED(INFRA) 退避；不写任何终态 + org.junit.jupiter.api.Assertions.assertThrows(IllegalStateException::class.java) { + p.applyScheduleRecords(head(), message(makeBody("121" to "15DEC261723"))) + } + assertNull(flights.findMainRow("121")) + assertEquals(0, todo.count()) + assertEquals(0, log.entries.size) + assertEquals(ProcStatus.PENDING, proc.find(msgId)!!.state) + } +} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/SnapshotFlowBackfillTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/processing/SnapshotFlowBackfillTest.kt deleted file mode 100644 index 0405884..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/SnapshotFlowBackfillTest.kt +++ /dev/null @@ -1,118 +0,0 @@ -package com.gzzn.omms.msgexchange.processing - -import com.gzzn.omms.msgexchange.domain.DecodedMessage -import com.gzzn.omms.msgexchange.domain.MetaFields -import com.gzzn.omms.msgexchange.domain.MsgKind -import com.gzzn.omms.msgexchange.domain.ProcState -import com.gzzn.omms.msgexchange.domain.ProcStatus -import com.gzzn.omms.msgexchange.infra.persistence.CminmsgInboxRepository -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository -import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository -import com.gzzn.omms.msgexchange.infra.persistence.PipelineTransactionManager -import com.gzzn.omms.msgexchange.infra.persistence.ProcStateRepository -import com.gzzn.omms.msgexchange.infra.persistence.ReqTrackRepository -import com.gzzn.omms.msgexchange.infra.retry.FailureScheduler -import com.gzzn.omms.msgexchange.infra.retry.ProcFailure -import com.gzzn.omms.msgexchange.config.PipelineProps -import com.gzzn.omms.msgexchange.MutableClock -import com.gzzn.omms.msgexchange.infra.stub.StubFlightSchd -import com.gzzn.omms.msgexchange.infra.stub.StubMsgEvents -import com.gzzn.omms.msgexchange.infra.stub.StubPipelineTransactionManager -import com.gzzn.omms.msgexchange.infra.stub.StubProcState -import com.gzzn.omms.msgexchange.infra.stub.StubReqTrack -import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Assertions.assertTrue -import org.junit.jupiter.api.Test - -/** v2 §2.6 / §5：提交后回填失败不得回滚 PROC_STATE。 */ -class SnapshotFlowBackfillTest { - - private class FailingInbox : CminmsgInboxRepository { - override fun insertRaw(rawXml: String): Long = 1L - override fun rawOf(cminmsgsId: Long): String? = null - override fun pollUnprocessed(afterId: Long, limit: Int): List = emptyList() - override fun backfillOnSuccess(cminmsgsId: Long, sndr: String, type: String, styp: String, seqn: Long) { - throw RuntimeException("mysql-down") - } - } - - @Test - fun `backfill failure after commit leaves PROC_STATE SUCCEEDED`() { - val flightSchd = StubFlightSchd() - val procState = StubProcState() - val msgEvents = StubMsgEvents() - val reqTrack = StubReqTrack() - val txManager = StubPipelineTransactionManager() - val inbox = FailingInbox() - val clock = MutableClock(MutableClock.BASE) - val procFailure = ProcFailure(procState, FailureScheduler(PipelineProps(), clock)) - - val headId = 901L - procState.insert(headId, ProcStatus.PENDING) - val day = "2026-09-07" - val meta = MetaFields("AODB", "SCHD", "DNLD", 9L, 20260907090000L) - val decoded = DecodedMessage(meta, MsgKind.Schd(MsgKind.SchdSubtype.DNLD), "") - SnapshotFlow.StageResult.parser = { - SnapshotFlow.StageResult.Ok(day, listOf("FL_X" to mapOf("FLID" to "FL_X"))) - } - - // v2 §5（P2-3）：回填失败必须落持久补偿待办（否则重启后无从对账） - val todo = com.gzzn.omms.msgexchange.infra.stub.StubBackfillTodo() - val flow = SnapshotFlow(procState, flightSchd, msgEvents, reqTrack, procFailure, txManager, inbox, backfillTodo = todo) - flow.publishSnapshot(ProcState(headId, ProcStatus.PENDING), decoded) - - assertEquals(ProcStatus.SUCCEEDED, procState.snapshotOf(headId)!!.state) - assertTrue(flightSchd.findByFlid("FL_X") != null) - assertEquals(1, todo.count()) - val recorded = todo.tasks().single() - assertEquals(901L, recorded.cminmsgsId) - assertEquals("SCHD", recorded.type) - assertEquals("DNLD", recorded.styp) - assertEquals("mysql-down", todo.lastErrorOf(901L)) - } - - @Test - fun `CAS replay requires matching lastMessageId not version alone`() { - val base = StubFlightSchd() - val procState = StubProcState() - val msgEvents = StubMsgEvents() - val reqTrack = StubReqTrack() - val txManager = StubPipelineTransactionManager() - val inbox = object : CminmsgInboxRepository { - override fun insertRaw(rawXml: String) = 1L - override fun rawOf(cminmsgsId: Long) = null - override fun pollUnprocessed(afterId: Long, limit: Int): List = emptyList() - override fun backfillOnSuccess(cminmsgsId: Long, sndr: String, type: String, styp: String, seqn: Long) = Unit - } - val clock = MutableClock(MutableClock.BASE) - val procFailure = ProcFailure(procState, FailureScheduler(PipelineProps(), clock)) - - val headId = 902L - val day = "2026-09-07" - procState.insert(headId, ProcStatus.PENDING) - - val conflictingGen = FlightSchdRepository.GenMeta(day, 2L, setOf("OTHER"), lastMessageId = "999") - base.putGenIfVersion(day, 0L, FlightSchdRepository.GenMeta(day, 1L, setOf("OTHER"), lastMessageId = "999")) - val mockFlightSchd = object : FlightSchdRepository by base { - override fun putGenIfVersion( - day: String, - expected: Long, - newGen: FlightSchdRepository.GenMeta, - now: java.time.Instant, - ): Boolean { - base.putGenIfVersion(day, 1L, conflictingGen) - return false - } - } - - SnapshotFlow.StageResult.parser = { - SnapshotFlow.StageResult.Ok(day, listOf("FL_Y" to mapOf("FLID" to "FL_Y"))) - } - val meta = MetaFields("AODB", "SCHD", "DNLD", 10L, 20260907100000L) - val decoded = DecodedMessage(meta, MsgKind.Schd(MsgKind.SchdSubtype.DNLD), "") - val flow = SnapshotFlow(procState, mockFlightSchd, msgEvents, reqTrack, procFailure, txManager, inbox) - flow.publishSnapshot(ProcState(headId, ProcStatus.PENDING), decoded) - - assertEquals(ProcStatus.FAILED, procState.snapshotOf(headId)!!.state) - } -} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/V2PipelineWritePathContractTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/processing/V2PipelineWritePathContractTest.kt deleted file mode 100644 index 8c262b5..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/V2PipelineWritePathContractTest.kt +++ /dev/null @@ -1,111 +0,0 @@ -package com.gzzn.omms.msgexchange.processing - -import com.gzzn.omms.msgexchange.MutableClock -import com.gzzn.omms.msgexchange.config.PipelineProps -import com.gzzn.omms.msgexchange.domain.DecodedMessage -import com.gzzn.omms.msgexchange.domain.Decision -import com.gzzn.omms.msgexchange.domain.FlightChange -import com.gzzn.omms.msgexchange.domain.MetaFields -import com.gzzn.omms.msgexchange.domain.MsgKind -import com.gzzn.omms.msgexchange.domain.ProcState -import com.gzzn.omms.msgexchange.domain.ProcStatus -import com.gzzn.omms.msgexchange.infra.persistence.CminmsgInboxRepository -import com.gzzn.omms.msgexchange.infra.retry.FailureScheduler -import com.gzzn.omms.msgexchange.infra.retry.ProcFailure -import com.gzzn.omms.msgexchange.infra.stub.StubMsgEvents -import com.gzzn.omms.msgexchange.infra.stub.StubPipelineTransactionManager -import com.gzzn.omms.msgexchange.infra.stub.StubProcState -import com.gzzn.omms.msgexchange.infra.stub.StubReqTrack -import com.gzzn.omms.msgexchange.codec.JacksonXmlCodec -import com.gzzn.omms.msgexchange.processing.handlers.GtdtHandler -import com.gzzn.omms.msgexchange.support.GuardedFlightSchd -import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Assertions.assertFalse -import org.junit.jupiter.api.Assertions.assertTrue -import org.junit.jupiter.api.Test -import java.nio.file.Paths -import kotlin.io.path.readText - -class V2PipelineWritePathContractTest { - - @Test - fun `SnapshotFlow and MessageProcessor sources must not call legacy upsert helpers`() { - val processingDir = Paths.get("src/main/kotlin/com/gzzn/omms/msgexchange/processing") - val snapshotSource = processingDir.resolve("SnapshotFlow.kt").readText() - val pumpSource = processingDir.resolve("Pump.kt").readText() - - assertFalse(snapshotSource.contains("upsertSnapshotBatch")) - assertFalse(snapshotSource.contains("upsertIncremental")) - assertTrue(snapshotSource.contains("persistNextStates")) - - assertFalse(pumpSource.contains("upsertSnapshotBatch")) - assertFalse(pumpSource.contains("upsertIncremental")) - assertTrue(pumpSource.contains("persistNextStates")) - } - - @Test - fun `SnapshotFlow DNLD path calls persistNextStates not legacy upsert helpers`() { - val flightSchd = GuardedFlightSchd() - val procState = StubProcState() - val msgEvents = StubMsgEvents() - val reqTrack = StubReqTrack() - val txManager = StubPipelineTransactionManager() - val inbox = object : CminmsgInboxRepository { - override fun insertRaw(rawXml: String) = 1L - override fun rawOf(cminmsgsId: Long) = null - override fun pollUnprocessed(afterId: Long, limit: Int): List = emptyList() - override fun backfillOnSuccess(cminmsgsId: Long, sndr: String, type: String, styp: String, seqn: Long) = Unit - } - val procFailure = ProcFailure(procState, FailureScheduler(PipelineProps(), MutableClock(MutableClock.BASE))) - - val headId = 1001L - procState.insert(headId, ProcStatus.PENDING) - val day = "2026-09-07" - SnapshotFlow.StageResult.parser = { - SnapshotFlow.StageResult.Ok(day, listOf("FL_DNLD" to mapOf("FLID" to "FL_DNLD", "FLNO" to "CA100", "GTDT" to """[{"GTNO":"1","GATE":"A1"}]"""))) - } - val decoded = DecodedMessage(MetaFields("AODB", "SCHD", "DNLD", 11L, 20260907110000L), MsgKind.Schd(MsgKind.SchdSubtype.DNLD), "") - SnapshotFlow(procState, flightSchd, msgEvents, reqTrack, procFailure, txManager, inbox) - .publishSnapshot(ProcState(headId, ProcStatus.PENDING), decoded) - - assertEquals(ProcStatus.SUCCEEDED, procState.snapshotOf(headId)!!.state) - assertEquals(1, flightSchd.persistNextStatesCalls) - } - - @Test - fun `MessageProcessor FLOP path calls persistNextStates not legacy upsert helpers`() { - val flightSchd = GuardedFlightSchd() - val procState = StubProcState() - val msgEvents = StubMsgEvents() - val reqTrack = StubReqTrack() - val txManager = StubPipelineTransactionManager() - val raws = mutableMapOf() - val inbox = object : CminmsgInboxRepository { - override fun insertRaw(rawXml: String) = 1L - override fun rawOf(cminmsgsId: Long) = raws[cminmsgsId] - override fun pollUnprocessed(afterId: Long, limit: Int): List = emptyList() - override fun backfillOnSuccess(cminmsgsId: Long, sndr: String, type: String, styp: String, seqn: Long) = Unit - } - val procFailure = ProcFailure(procState, FailureScheduler(PipelineProps(), MutableClock(MutableClock.BASE))) - val handler = GtdtHandler() - val snapshot = SnapshotFlow(procState, flightSchd, msgEvents, reqTrack, procFailure, txManager, inbox) - val processor = MessageProcessor( - inbox, procState, msgEvents, - CodecHolder(JacksonXmlCodec()), - HandlerHolder(HandlerRegistry(listOf(handler))), - flightSchd, snapshot, procFailure, PipelineProps(), txManager, - ) - val headId = 1002L - procState.insert(headId, ProcStatus.PENDING) - raws[headId] = """ - - AODB120260907120000FLOPGTDT - FL_FLOPB9 - - """.trimIndent() - processor.processOne(ProcState(headId, ProcStatus.PENDING)) - - assertEquals(ProcStatus.SUCCEEDED, procState.snapshotOf(headId)!!.state) - assertEquals(1, flightSchd.persistNextStatesCalls) - } -} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/handlers/GtdtHandlerTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/processing/handlers/GtdtHandlerTest.kt deleted file mode 100644 index b50151e..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/processing/handlers/GtdtHandlerTest.kt +++ /dev/null @@ -1,41 +0,0 @@ -package com.gzzn.omms.msgexchange.processing.handlers - -import com.gzzn.omms.msgexchange.codec.FlopPayload -import com.gzzn.omms.msgexchange.domain.DecodedMessage -import com.gzzn.omms.msgexchange.domain.MetaFields -import com.gzzn.omms.msgexchange.domain.MsgKind -import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Test - -class GtdtHandlerTest { - - private val handler = GtdtHandler() - - @Test - fun `decide replaces GTDT collection for three gates`() { - val msg = DecodedMessage( - meta = MetaFields("AODB", "FLOP", "GTDT", 1L, 20260908120000L), - kind = MsgKind.Flop("GTDT"), - rawXml = "", - body = FlopPayload( - flid = "121112312", - scalars = mapOf("FFID" to "CA-CA101-A-12DEC031345-D"), - collections = mapOf( - "GTDT" to listOf( - mapOf("GTNO" to "1", "GATE" to "G28", "GTYP" to "D"), - mapOf("GTNO" to "2", "GATE" to "G33", "GTYP" to "I"), - mapOf("GTNO" to "3", "GATE" to "G23", "GTYP" to "D"), - ), - ), - ), - ) - - val decision = handler.decide(emptyMap(), msg) - assertEquals(1, decision.flightChanges.size) - assertEquals("121112312", decision.flightChanges[0].flid) - val gtdtJson = decision.flightChanges[0].fields["GTDT"]!! - assertEquals(true, gtdtJson.contains("\"G28\"")) - assertEquals(1, decision.schdPush.size) - assertEquals("121112312", decision.schdPush[0].flid) - } -} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/support/SeedHelpers.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/support/SeedHelpers.kt deleted file mode 100644 index 6d780a8..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/support/SeedHelpers.kt +++ /dev/null @@ -1,44 +0,0 @@ -package com.gzzn.omms.msgexchange.support - -import com.gzzn.omms.msgexchange.domain.FlightChange -import com.gzzn.omms.msgexchange.domain.flight.FlightStateEngine -import com.gzzn.omms.msgexchange.infra.persistence.FlightFields -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository -import java.time.Instant - -/** 测试种子助手：v2 写路径（persistNextStates）等价替代已退场的 legacy upsert（P4）。 */ -fun seedSnapshot( - repo: FlightSchdRepository, - day: String, - flights: List>, - now: Instant = Instant.now(), -) { - repo.persistNextStates( - day, - flights.map { (flid, fields) -> - FlightStateEngine.apply( - null, - FlightStateEngine.commandsFromFields(flid, fields), - "seed-$day", - bumpVersion = false, - ).copy(stateVersion = 1L) - }, - snapshotReplace = true, - now = now, - ) -} - -fun seedIncremental(repo: FlightSchdRepository, changes: List, messageId: String = "seed-inc") { - repo.persistNextStates( - null, - changes.map { change -> - FlightStateEngine.apply( - repo.findNextStateByFlid(change.flid), - FlightStateEngine.commandsFromFields(change.flid, change.fields), - messageId, - bumpVersion = true, - ) - }, - snapshotReplace = false, - ) -} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/support/V2WritePathGuard.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/support/V2WritePathGuard.kt deleted file mode 100644 index 8a1bf1e..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/support/V2WritePathGuard.kt +++ /dev/null @@ -1,31 +0,0 @@ -package com.gzzn.omms.msgexchange.support - -import com.gzzn.omms.msgexchange.domain.flight.FlightNextState -import com.gzzn.omms.msgexchange.infra.persistence.FlightFields -import com.gzzn.omms.msgexchange.infra.persistence.FlightSchdRepository -import com.gzzn.omms.msgexchange.infra.stub.StubFlightSchd -import java.time.Instant - -/** v2 写路径守卫（P4 退场后）：persistNextStates 是唯一写路径，legacy upsert 入口已从接口删除。 */ -class GuardedFlightSchd( - private val inner: FlightSchdRepository = StubFlightSchd(), -) : FlightSchdRepository { - var persistNextStatesCalls = 0 - - override fun persistNextStates(day: String?, states: List, snapshotReplace: Boolean, now: Instant) { - persistNextStatesCalls++ - inner.persistNextStates(day, states, snapshotReplace, now) - } - - override fun deleteDiffByDay(day: String, delFlids: Collection): Int = inner.deleteDiffByDay(day, delFlids) - override fun findByFlid(flid: String): FlightFields? = inner.findByFlid(flid) - override fun findNextStateByFlid(flid: String): FlightNextState? = inner.findNextStateByFlid(flid) - override fun findByFlids(flids: Collection): Map = inner.findByFlids(flids) - override fun findByDay(day: String): List> = inner.findByDay(day) - override fun findAll(): Map = inner.findAll() - override fun deleteByFlids(flids: Set): Int = inner.deleteByFlids(flids) - override fun getGen(day: String): FlightSchdRepository.GenMeta? = inner.getGen(day) - override fun putGenIfVersion(day: String, expected: Long, newGen: FlightSchdRepository.GenMeta, now: Instant): Boolean = - inner.putGenIfVersion(day, expected, newGen, now) - override fun deleteGenBefore(cutoffDay: String): Int = inner.deleteGenBefore(cutoffDay) -} diff --git a/src/test/kotlin/com/gzzn/omms/msgexchange/tools/FlightStoreDiffToolTest.kt b/src/test/kotlin/com/gzzn/omms/msgexchange/tools/FlightStoreDiffToolTest.kt deleted file mode 100644 index a19a79a..0000000 --- a/src/test/kotlin/com/gzzn/omms/msgexchange/tools/FlightStoreDiffToolTest.kt +++ /dev/null @@ -1,115 +0,0 @@ -package com.gzzn.omms.msgexchange.tools - -import org.junit.jupiter.api.Assertions.assertEquals -import org.junit.jupiter.api.Assertions.assertFalse -import org.junit.jupiter.api.Assertions.assertTrue -import org.junit.jupiter.api.Test - -class FlightStoreDiffToolTest { - - private val tool = FlightStoreDiffTool() - - @Test - fun `Field map comparison ignores legacy JSON key ordering differences`() { - val pg = mapOf( - "F1" to mapOf( - "flid" to "F1", - "airline" to "CA", - "flightNo" to "123", - "status" to "SCHD", - ), - ) - val legacy = mapOf( - "F1" to """{"status":"SCHD","flightNo":"123","airline":"CA","flid":"F1"}""", - ) - - val report = tool.diff(pg, legacy) - assertTrue(report.isGreen) - assertEquals(1, report.matchedCount) - assertEquals(0, report.unexpectedDeviations.size) - } - - @Test - fun `Field map comparison normalizes numeric precision and null equivalents`() { - val pg = mapOf( - "F1" to mapOf( - "flid" to "F1", - "weight" to "100.0", - "delay" to "0", - ), - ) - val legacy = mapOf( - "F1" to """{"flid":"F1","weight":100,"delay":0.0,"extra":null}""", - ) - - val report = tool.diff(pg, legacy) - assertTrue(report.isGreen) - assertEquals(1, report.matchedCount) - assertEquals(0, report.unexpectedDeviations.size) - } - - @Test - fun `Recognizes known deviation 1 - Cross-day migrated flights protected in PG`() { - // PG 中包含跨日迁移未被误删的航班，Legacy Redis 中已被误删 - val pg = mapOf( - "F_STABLE" to mapOf("flid" to "F_STABLE"), - "F_MIGRATED" to mapOf("flid" to "F_MIGRATED", "FDAY" to "2026-09-08"), - ) - val legacy = mapOf( - "F_STABLE" to """{"flid":"F_STABLE"}""", - // F_MIGRATED 在 legacy 中被按旧代差删误删 - ) - - val report = tool.diff(pg, legacy, crossDayMigratedFlids = setOf("F_MIGRATED")) - assertTrue(report.isGreen) // 属于已知合法偏差，红绿灯依然为 GREEN - assertEquals(1, report.matchedCount) - assertEquals(1, report.knownDeviations.size) - assertEquals(FlightStoreDiffTool.DeviationKind.CROSS_DAY_PROTECTED, report.knownDeviations[0].kind) - assertEquals(0, report.unexpectedDeviations.size) - } - - @Test - fun `Detects real field mismatch and missing flight as unexpected deviation`() { - val pg = mapOf( - "F1" to mapOf("flid" to "F1", "status" to "BOARDING"), - "F_EXTRA" to mapOf("flid" to "F_EXTRA"), - ) - val legacy = mapOf( - "F1" to """{"flid":"F1","status":"DEPARTED"}""", - "F_MISSING" to """{"flid":"F_MISSING"}""", - ) - - val report = tool.diff(pg, legacy) - assertFalse(report.isGreen) - assertEquals(0, report.matchedCount) - assertEquals(3, report.unexpectedDeviations.size) - - val kinds = report.unexpectedDeviations.map { it.kind } - assertTrue(kinds.contains(FlightStoreDiffTool.DeviationKind.FIELD_MISMATCH)) - assertTrue(kinds.contains(FlightStoreDiffTool.DeviationKind.MISSING_IN_PG)) - assertTrue(kinds.contains(FlightStoreDiffTool.DeviationKind.UNEXPECTED_EXTRA_IN_PG)) - - val summary = report.formatSummary() - assertTrue(summary.contains("RED (未通过)")) - assertTrue(summary.contains("F1")) - } - - @Test - fun `Nested collection value is compared as parsed JSON tree`() { - val pg = mapOf( - "F1" to mapOf( - "flid" to "F1", - // 1:N 明细集合列：序列化文本（JSON 数组），按解析树与 legacy 递归比对 - "GTDT" to """[{"GATE":"A1","PGOT":"07SEP260800"}]""", - ), - ) - val legacy = mapOf( - "F1" to """{"flid":"F1","GTDT":[{"PGOT":"07SEP260800","GATE":"A1"}]}""", - ) - - val report = tool.diff(pg, legacy) - assertTrue(report.isGreen) - assertEquals(1, report.matchedCount) - assertEquals(0, report.unexpectedDeviations.size) - } -}