fix(flight): 历史清理判据对齐 US-14 AC2 五类已结束条件(ACM2-91)

- HistoryRules.isEnded 共享求值器:计划超 3 天/取消超 1h/离港/到港;JDBC 与 stub 同源不漂移
- 删除无依据的 idle/deleted/terminal 三个窗口(reference.md 四行冲突参数重写为 AC2 口径)
- 备降条件依赖 FDIV(DDES/DDIR),落地(Q3、ACM2-100)前显式不命中,参数先行登记
- 修「当日」取系统时钟的缺陷,改由传入 now 推导(测试抓出)
- implementation.md 12.5 与配置、yml 同批对齐
This commit is contained in:
windyboy
2026-09-21 13:39:18 +08:00
parent 1d95926e06
commit f9d51399da
11 changed files with 178 additions and 63 deletions
+1 -1
View File
@@ -407,7 +407,7 @@ ADFTSet-only`US-04` AC2),未带字段不清。有 `SODT` 则算 `OPERA
### 12.5 生命周期
运营日过去 ≠ 航班结束。历史清理候选按 `US-14` AC2 的五类已结束条件选取;先写 ES 再删,失败删 0 行(`D1`)。当前配置的判据与窗口尚未对齐该验收,见 [reference.md](reference.md) 依据列标「与需求冲突」的四项
运营日过去 ≠ 航班结束。历史清理候选按 `US-14` AC2 的五类已结束条件选取(共享求值器 `HistoryRules.isEnded``SODT`/`CNCL`/`NAAT`/`NEAT` 解析不出一律不命中,宁漏删不误删);先写 ES 再删,失败删 0 行(`D1`)。备降条件依赖 `FDIV` 段(`DDES`/`DDIR`),该段落地(`Q3`、ACM2-100)前恒不命中
## 13. 静态参考数据
+4 -4
View File
@@ -56,10 +56,10 @@
| `msgx.identity.include-day-boundary` | `false` | 去重时是否把日期算进消息身份;受 `C-3` 约束,不能随意改变 | 默认关闭 |
| `msgx.health.backlog-cache-ttl-ms` | `30000` ms | `/health``/metrics` 共用的未处理消息统计最多缓存多久;`0` 表示每次重算 | 暂定 |
| `msgx.history.history-store-enabled` | `false` | 是否启用历史航班写入;关闭时不删除实时航班 | 默认关闭 |
| `msgx.history.cancelled-hours` | `48` 小时 | 与 `US-14` AC2 的取消航班清理条件冲突 | 与需求冲突 |
| `msgx.history.terminal-hours` | `48` 小时 | `US-14` AC2 的清理条件不包含这个等待时间 | 与需求冲突 |
| `msgx.history.deleted-hours` | `48` 小时 | 已删除航班还要等待的时间;`US-14` AC2 没有此条件 | 与需求冲突 |
| `msgx.history.idle-hours` | `168` 小时 | 长时间未更新的航班也会被清理;`US-14` AC2 没有此条件 | 与需求冲突 |
| `msgx.history.planned-age-days` | `3` 天 | `US-14` AC2 条件 1:计划时间早于当前超过该天数视为已结束 | `US-14` AC2 |
| `msgx.history.cancelled-hours` | `1` 小时 | `US-14` AC2 条件 2:取消时间早于当前超过该窗口视为已结束 | `US-14` AC2 |
| `msgx.history.diverted-hours` | `1` 小时 | `US-14` AC2 条件 3:备降且计划时间早于当前超过该窗口;备降依据 `FDIV``DDES`/`DDIR`)落地前不参与判定(`Q3`、ACM2-100 | `US-14` AC2 |
| `msgx.history.arrived-hours` | `1` 小时 | `US-14` AC2 条件 5:实际到港时间早于当前超过该窗口视为已结束(条件 4 离港不设窗口) | `US-14` AC2 |
| `msgx.history.snap-log-retention-days` | `90` 天 | 日计划处理记录保存多久 | 暂定 |
### 信箱与外部依赖(成组登记)
@@ -2,20 +2,24 @@ package com.gzzn.omms.msgexchange.config
import io.micronaut.context.annotation.ConfigurationProperties
/** 航班归档删除的时间窗口和留痕保留期,对应配置里的 `msgx.history.*`;窗口按机场时区算。 */
/** 历史清理候选的时间窗口(`US-14` AC2 五类已结束条件)和留痕保留期,对应 `msgx.history.*`;窗口按机场时区算。 */
@ConfigurationProperties("msgx.history")
class HistoryProps {
/** 取消时间字段(CNCL)非空,且已经过去这么多小时。 */
var cancelledHours: Long = 48
/** `US-14` AC2 条件 1:计划时间(SODT)早于当前超过这么多。 */
var plannedAgeDays: Long = 3
/** 到达/离开终态字段(NAAT/NEAT)非空且已过去这么多小时;这两个字段的确切含义还要跟业务确认。 */
var terminalHours: Long = 48
/** `US-14` AC2 条件 2:取消时间(CNCL)早于当前超过这么多小时。 */
var cancelledHours: Long = 1
/** 航班已标记删除(STATE = DELETED)并且过了这么多小时。 */
var deletedHours: Long = 48
/**
* `US-14` AC2 条件 3 的窗口:备降(应降本场、实际降其他机场)且计划时间早于当前超过这么多小时。
* 备降依据是 `FDIV` 段(`DDES`/`DDIR`),该段尚未解码落库(`Q3`、ACM2-100);
* 落地前本条件不参与候选判定,参数先行登记。
*/
var divertedHours: Long = 1
/** 兜底期限:航班既没取消也没到终态时,最后一次有效更新超过这么多小时就算静默,可以清理。 */
var idleHours: Long = 24 * 7
/** `US-14` AC2 条件 5:实际到港时间(NAAT)早于当前超过这么多小时。 */
var arrivedHours: Long = 1
/** 日计划留痕表 SCHD_SNAP_LOG 保留多少天,按(快照覆盖截止日, 接收时间)删旧行。 */
var snapLogRetentionDays: Long = 90
@@ -64,13 +64,49 @@ data class FlightSnapshot(
val collections: Map<String, List<Map<String, String>>>,
)
/** 判定一个航班可以归档删除的四条时间窗口,单位是小时,按机场所在时区算;具体取值走业务配置。 */
/**
* 历史清理候选判据(`US-14` AC2 的五类已结束条件;窗口按机场时区算,取值走业务配置):
*
* 1. 计划时间(SODT)早于当前超过 [plannedAgeDays] 天;
* 2. 取消时间(CNCL)早于当前超过 [cancelledHours] 小时;
* 3. 备降(应降本场、实际降其他机场)且计划时间早于当前超过 [divertedHours] 小时
* ——**当前不可判**:备降依据是 `FDIV` 段的 `DDES`/`DDIR`,该段不在现行 FLOP 白名单、
* 也无 DTO 与存储列(`Q3`、ACM2-100),落地前本条件恒不命中(漏删优于误删);
* 4. 离港:计划日期早于当日、有实际离港时间(NEAT)且不晚于当前;
* 5. 到港:计划日期早于当日、有实际到港时间(NAAT)且早于当前超过 [arrivedHours] 小时。
*/
data class HistoryRules(
val cancelledHours: Long = 48, // 取消时间字段(CNCL)非空且已过 N 小时
val terminalHours: Long = 48, // 到达/离开终态字段(NAAT/NEAT)非空且已过 N 小时
val deletedHours: Long = 48, // 已标记删除(STATE = DELETED)且已过 N 小时
val idleHours: Long = 24 * 7, // 既没有终态也没有取消:最后更新距今超过这个兜底期限
)
val plannedAgeDays: Long = 3,
val cancelledHours: Long = 1,
val divertedHours: Long = 1,
val arrivedHours: Long = 1,
) {
/**
* 五类条件的共享求值:JDBC 候选查询与内存 stub 都走这里,避免两份实现漂移。
* 时间字段解析不出来(空、格式错)一律视为条件不满足——宁漏删不误删。
*/
fun isEnded(
plannedAt: java.time.Instant?,
cancelledAt: java.time.Instant?,
arrivalAt: java.time.Instant?,
departureAt: java.time.Instant?,
zone: java.time.ZoneId,
now: java.time.Instant,
): Boolean {
val plannedDate = plannedAt?.atZone(zone)?.toLocalDate()
// 「当日」由调用方传入的 now 推导(运营日时区),不用系统时钟——测试与回放需要确定性行为
val beforeToday = plannedDate != null && plannedDate.isBefore(now.atZone(zone).toLocalDate())
return when {
plannedAt != null && plannedAt.isBefore(now.minusSeconds(plannedAgeDays * 86400)) -> true // 条件 1
cancelledAt != null && cancelledAt.isBefore(now.minusSeconds(cancelledHours * 3600)) -> true // 条件 2
// 条件 3(备降)待 FDIV 落地,见类注释
beforeToday && departureAt != null && !departureAt.isAfter(now) -> true // 条件 4
beforeToday && arrivalAt != null &&
arrivalAt.isBefore(now.minusSeconds(arrivedHours * 3600)) -> true // 条件 5
else -> false
}
}
}
/**
* 被生命周期判定选中、准备归档并物理删除的航班。顺序固定是"先归档、再删除",
@@ -718,37 +718,34 @@ class JdbcFlightStateRepository(
) == 1
override fun findHistoryCandidates(rules: HistoryRules, zone: ZoneId, now: Instant): List<HistoryCandidate> {
// 四条清理判据在应用层算:里面的时间字段是 SIS 格式字符串,没法交给 SQL 比较;
// 判据在应用层算:时间字段是 SIS 格式字符串ddMMMyyHHmm,交给 SQL 比较不了
// 不按 updated_at 粗筛——取消/终态时间可能早于最近一次更新,粗筛会漏删。
val rows = ds.query(
"SELECT flid, state, state_version, operation_day, cncl, naat, neat, updated_at FROM flight_schd",
"SELECT flid, state, state_version, sodt, cncl, naat, neat FROM flight_schd",
{},
) { rs ->
object {
val flid = rs.getString("flid")
val state = FlightState.valueOf(rs.getString("state"))
val version = rs.getLong("state_version")
val sodt = rs.getString("sodt")
val cncl = rs.getString("cncl")
val naat = rs.getString("naat")
val neat = rs.getString("neat")
val updatedAt = rs.getInstant("updated_at")
}
}
return rows.mapNotNull { r ->
val cancelledAt = parseSisTime(r.cncl, zone)
val terminalAt = listOfNotNull(parseSisTime(r.naat, zone), parseSisTime(r.neat, zone)).maxOrNull()
val hasTerminalField = listOf(r.cncl, r.naat, r.neat).any { !it.isNullOrBlank() }
val hit = when {
cancelledAt != null && cancelledAt < now.minus(Duration.ofHours(rules.cancelledHours)) -> true
terminalAt != null && terminalAt < now.minus(Duration.ofHours(rules.terminalHours)) -> true
r.state == FlightState.DELETED && r.updatedAt != null &&
r.updatedAt < now.minus(Duration.ofHours(rules.deletedHours)) -> true
!hasTerminalField && r.updatedAt != null &&
r.updatedAt < now.minus(Duration.ofHours(rules.idleHours)) -> true
else -> false
}
// NAAT/NEAT 的业务含义还没确认,所以解析不出来时一律当成"不满足条件",宁可漏删
if (hit) HistoryCandidate(r.flid, r.state, r.version, wasNeverFdel = r.state == FlightState.ACTIVE) else null
// US-14 AC2 五类已结束条件共享求值(HistoryRules.isEnded);备降条件待 FDIV 落地,
// 见 HistoryRules 注释。解析不出来一律不命中:宁漏删不误删。
val ended = rules.isEnded(
plannedAt = parseSisTime(r.sodt, zone),
cancelledAt = parseSisTime(r.cncl, zone),
arrivalAt = parseSisTime(r.naat, zone),
departureAt = parseSisTime(r.neat, zone),
zone = zone,
now = now,
)
if (ended) HistoryCandidate(r.flid, r.state, r.version, wasNeverFdel = r.state == FlightState.ACTIVE) else null
}
}
@@ -385,16 +385,21 @@ class StubFlightState : FlightStateRepository {
override fun findHistoryCandidates(rules: HistoryRules, zone: ZoneId, now: Instant): List<HistoryCandidate> =
mains.values.mapNotNull { main ->
val snap = snapshots[main.flid]
val cancelled = snap?.scalars?.get("CNCL")
val hasTerminalField = listOf("CNCL", "NAAT", "NEAT").any { !snap?.scalars?.get(it).isNullOrBlank() }
val idleHit = !hasTerminalField && main.updatedAt < now.minusSeconds(rules.idleHours * 3600)
val deletedHit = main.state == FlightState.DELETED && main.updatedAt < now.minusSeconds(rules.deletedHours * 3600)
if (idleHit || deletedHit) {
HistoryCandidate(main.flid, main.state, main.stateVersion, wasNeverFdel = main.state == FlightState.ACTIVE)
} else {
null
val snap = snapshots[main.flid] ?: return@mapNotNull null
fun sisTime(key: String): Instant? {
val raw = snap.scalars[key]?.trim()?.takeIf { it.isNotEmpty() } ?: return null
return com.gzzn.omms.msgexchange.domain.OperationDayCalculator.parseSodt(raw)?.atZone(zone)?.toInstant()
}
// 与 JDBC 候选查询同走 HistoryRules.isEndedUS-14 AC2 五类条件)
val ended = rules.isEnded(
plannedAt = sisTime("SODT"),
cancelledAt = sisTime("CNCL"),
arrivalAt = sisTime("NAAT"),
departureAt = sisTime("NEAT"),
zone = zone,
now = now,
)
if (ended) HistoryCandidate(main.flid, main.state, main.stateVersion, wasNeverFdel = main.state == FlightState.ACTIVE) else null
}
override fun purgeArchived(candidates: List<HistoryCandidate>): Int {
@@ -58,7 +58,7 @@ class HistorySweepJob(
return SweepOutcome(selected = 0, archived = 0, purged = 0, snapLogPurged = snapLogPurged)
}
val rules = HistoryRules(props.cancelledHours, props.terminalHours, props.deletedHours, props.idleHours)
val rules = HistoryRules(props.plannedAgeDays, props.cancelledHours, props.divertedHours, props.arrivedHours)
val zone = operationDayProps.zoneId() // 保留期窗口按 PARAM:msgx.operation-day.zone 算,不用 UTC
val candidates = flightState.findHistoryCandidates(rules, zone, now)
if (candidates.isEmpty()) return SweepOutcome(0, 0, 0, snapLogPurged = snapLogPurged)
+5 -5
View File
@@ -31,11 +31,11 @@ msgx:
operation-day: # SODT + 机场时区 + 切日边界(默认占位 0 点,待业务确认)
zone: Asia/Shanghai
cutoff-hour: 0
history: # 历史判定窗口与保留期
cancelled-hours: 48
terminal-hours: 48
deleted-hours: 48
idle-hours: 168
history: # 历史清理判据窗口(US-14 AC2 五类已结束条件)与留痕保留期
planned-age-days: 3 # 条件 1:计划时间早于当前超过 3 天
cancelled-hours: 1 # 条件 2:取消时间早于当前超过 1 小时
diverted-hours: 1 # 条件 3:备降窗口;FDIV(DDES/DDIR)落地前不参与判定(Q3、ACM2-100)
arrived-hours: 1 # 条件 5:实际到港早于当前超过 1 小时
snap-log-retention-days: 90
history-store-enabled: false # 未接通历史存储时 HISTORY_SWEEP 删 0 条
@@ -0,0 +1,67 @@
package com.gzzn.omms.msgexchange.domain.flight
import org.junit.jupiter.api.Assertions.assertFalse
import org.junit.jupiter.api.Assertions.assertTrue
import org.junit.jupiter.api.Test
import java.time.Instant
import java.time.ZoneId
/**
* `US-14` AC2 五类已结束条件的共享求值边界(`HistoryRules.isEnded`):
* JDBC 候选查询与内存 stub 都走这一个实现,这里守住每一类的开/关边界。
* 时间字段解析不出(缺席)一律不命中:宁漏删不误删。
*/
class HistoryRulesIsEndedTest {
private val zone: ZoneId = ZoneId.of("Asia/Shanghai")
/** 2026-09-09T00:00Z = 上海本地 2026-09-09T08:00。 */
private val now: Instant = Instant.parse("2026-09-09T00:00:00Z")
private val rules = HistoryRules(plannedAgeDays = 3, cancelledHours = 1, divertedHours = 1, arrivedHours = 1)
private fun ended(
plannedAt: Instant? = null,
cancelledAt: Instant? = null,
arrivalAt: Instant? = null,
departureAt: Instant? = null,
): Boolean = rules.isEnded(plannedAt, cancelledAt, arrivalAt, departureAt, zone, now)
@Test
fun `condition 1 planned time older than three days`() {
assertTrue(ended(plannedAt = now.minusSeconds(3 * 86400 + 1))) // 超过 3 天 → 已结束
assertFalse(ended(plannedAt = now.minusSeconds(3 * 86400 - 1))) // 恰好不满 3 天 → 未结束
}
@Test
fun `condition 2 cancelled more than one hour ago`() {
assertTrue(ended(cancelledAt = now.minusSeconds(3600 + 1)))
assertFalse(ended(cancelledAt = now.minusSeconds(3600 - 1)))
}
@Test
fun `condition 4 departed yesterday-or-earlier plan and actual departure not after now`() {
// SODT 2026-09-08T10:00 上海(前一天),NEAT = 当前时刻 → 已结束
assertTrue(ended(plannedAt = Instant.parse("2026-09-08T02:00:00Z"), departureAt = now))
// 实际起飞晚于当前(还没起飞)→ 未结束
assertFalse(ended(plannedAt = Instant.parse("2026-09-08T02:00:00Z"), departureAt = now.plusSeconds(60)))
}
@Test
fun `condition 4 and 5 require planned date before today`() {
// SODT 2026-09-09T07:00 上海(当日),即使到港/离港时间都满足也不算结束
assertFalse(ended(plannedAt = Instant.parse("2026-09-08T23:00:00Z"), departureAt = now.minusSeconds(3600)))
assertFalse(ended(plannedAt = Instant.parse("2026-09-08T23:00:00Z"), arrivalAt = now.minusSeconds(7200)))
}
@Test
fun `condition 5 arrived more than one hour ago`() {
val yesterday = Instant.parse("2026-09-08T02:00:00Z")
assertTrue(ended(plannedAt = yesterday, arrivalAt = now.minusSeconds(7200))) // 2 小时前到港
assertFalse(ended(plannedAt = yesterday, arrivalAt = now.minusSeconds(1800))) // 30 分钟前到港
}
@Test
fun `absent time fields never qualify`() {
assertFalse(ended())
assertFalse(ended(plannedAt = now, cancelledAt = now, arrivalAt = now, departureAt = now))
}
}
@@ -24,15 +24,16 @@ class HistorySweepJobTest {
private val now: Instant = Instant.parse("2026-09-09T00:00:00Z")
private fun seededFlight(flid: String, deleted: Boolean, idleDays: Long): StubFlightState {
/** 播种一条已结束(US-14 AC2 条件 1:SODT 早于当前超过 3 天)的航班。 */
private fun seededFlight(flid: String, deleted: Boolean): StubFlightState {
val f = StubFlightState()
f.persistFullState(
com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot(
flid, java.time.LocalDate.of(2026, 9, 1), if (deleted) FlightState.DELETED else FlightState.ACTIVE,
1, mapOf("FLNO" to "CA001"), emptyMap(),
1, mapOf("FLNO" to "CA001", "SODT" to "01Sep260000"), emptyMap(),
),
msgId = 1,
now = now.minusSeconds(idleDays * 86400),
now = now.minusSeconds(30 * 86400),
)
return f
}
@@ -51,7 +52,7 @@ class HistorySweepJobTest {
@Test
fun `history store not connected must delete zero rows`() {
val f = seededFlight("F1", deleted = true, idleDays = 30)
val f = seededFlight("F1", deleted = true)
val events = StubMsgEvents()
val job = HistorySweepJob(f, events, HistoryProps().apply { historyStoreEnabled = false }, OperationDayProps(), StubPipelineTx(), StubPipelineLock())
@@ -64,10 +65,11 @@ class HistorySweepJobTest {
@Test
fun `archive success precedes physical purge and confirmed rows only`() {
val f = seededFlight("F1", deleted = true, idleDays = 30)
val f = seededFlight("F1", deleted = true)
f.persistFullState(
com.gzzn.omms.msgexchange.domain.flight.FlightSnapshot(
"F2", java.time.LocalDate.of(2026, 9, 1), FlightState.DELETED, 1, mapOf("FLNO" to "CA002"), emptyMap(),
"F2", java.time.LocalDate.of(2026, 9, 1), FlightState.DELETED, 1,
mapOf("FLNO" to "CA002", "SODT" to "01Sep260000"), emptyMap(),
),
msgId = 2, now = now.minusSeconds(30 * 86400),
)
@@ -89,7 +91,7 @@ class HistorySweepJobTest {
@Test
fun `never-fdel lifecycle purge emits msg deletion notice before deletion`() {
val f = seededFlight("F3", deleted = false, idleDays = 30) // 还在用,但已经静默超过兜底期限,命中清理条件
val f = seededFlight("F3", deleted = false) // 还在用,但计划时间早于当前超过 3 天(US-14 AC2 条件 1,命中清理条件
val events = StubMsgEvents()
val store = RecordingHistoryStore()
val job = HistorySweepJob(
@@ -107,7 +109,7 @@ class HistorySweepJobTest {
@Test
fun `a flight updated after archival is not purged and emits no deletion notice`() {
val f = seededFlight("F4", deleted = false, idleDays = 30)
val f = seededFlight("F4", deleted = false)
val events = StubMsgEvents()
val store = object : HistorySweepJob.HistoryStore {
override fun archive(candidates: List<HistoryCandidate>): Set<String> {
@@ -131,7 +133,7 @@ class HistorySweepJobTest {
@Test
fun `snap log purge runs independently of the history store switch`() {
val f = seededFlight("F1", deleted = true, idleDays = 30)
val f = seededFlight("F1", deleted = true)
var cutoff: Instant? = null
val purge = com.gzzn.omms.msgexchange.infra.persistence.SnapshotLogPurge { instant -> cutoff = instant; 7 }
val job = HistorySweepJob(
@@ -41,9 +41,13 @@ class HistorySweepPurgePgTest {
val events = JdbcMsgEventRepository(ds, clock)
val flid = "PG-" + UUID.randomUUID().toString().take(8)
flights.persistFullState(
FlightSnapshot(flid, LocalDate.of(2026, 8, 1), FlightState.ACTIVE, 1, emptyMap(), emptyMap()),
FlightSnapshot(
flid, LocalDate.of(2026, 8, 1), FlightState.ACTIVE, 1,
mapOf("SODT" to "01Aug260000"), // 计划时间早于当前超过 3 天 → US-14 AC2 条件 1 命中
emptyMap(),
),
msgId = 1,
now = t0.minusSeconds(30L * 86400), // 静默超过 idle-hours,命中生命周期清退
now = t0.minusSeconds(30L * 86400),
)
// 删除阶段注入失败:删除通知已登记,删除抛错 → 整个事务必须回滚。