refactor(ingress): 信箱边界层契约重构——发现与处理标记解耦、回填事实并入 PROC_STATE
收报扫描不再以 DATE_PROCESSED 为谓词：终态而未回填的行（解码失败死信等）会永久占据 有限批次，累积到 claim-batch 后收报整体停摆（US-01 条目 3 / message-lifecycle §5.3）。 ingress/InboxPoller.kt：按 ID 区间升序有界读取（ID > W），水位落 INBOX_CURSOR 并与入队 同一 PG 事务推进（中断后重扫补建）；遇空洞即停，空洞超过 pipeline.max-commit-delay 判定 为永久并放行——否则水位永久停摆于一次自增回滚留下的空位。删除 InboxEnqueue（改由 insertIfAbsent 幂等入队）与 poller 内的回填扫描（消除 ingress→jobs 反向依赖）。 infra/persistence：端口按事实重画为 readRange/maxId/markProcessedIfUnmarked，标记 UPDATE 带 DATE_PROCESSED IS NULL 守卫，只把空标写为已处理（§11 单调，重复执行无副作用）。 回填事实并入 PROC_STATE（RECEIVED_AT/BACKFILL_AT/NEXT_AT/ATTEMPTS/ERROR），BACKFILL_TODO 随 V2 迁移下线；终态与回填意图是同一条 UPDATE，由处理器在自己的业务事务内落库， message-lifecycle §4 登记的两个崩溃窗口（提交后回填前崩溃、待办二次落账失败）不再是缺口。 processing/BackfillService.kt（取代 BackfillSweepJob）：终态提交后立即尝试一次，失败按 30s→15min 指数退避重试；扫描条件「终态 + 未确认标记 +（已到期 或 接收时间早于 NOW − R）」 使 §5.2 的超期期限 R 覆盖退避，中间态永不补写。死信同样可补写——回填只需消息 ID， 不再依赖 META。Pump 改用可注入 Clock。 infra/health：InboxLifecycleHealthIndicator 输出积压条数、最老未处理信龄、未回填终态数与 水位滞后（OPS-2 / §5.3 验收）。预计消化时长需吞吐采样，留待接入指标注册表时补。 配置：pipeline.max-commit-delay / overdue-backfill / backfill-batch、mailbox.processed-value （Q2/Q6/Q7 未书面确认前取保守初值，不得为提速下调）。 不变量回归测试：死信不阻断后续发现、水位遇空洞即停与老化放行、终态+意图同事务、 超期 R 覆盖退避、中间态不补写、标记单调；InboxLifecycleJdbcSqlTest 以 H2 的 PostgreSQL 兼容模式直连验证上述 SQL 语义（不依赖 docker）。libs.h2 由 testRuntimeOnly 提为 testImplementation 以支持该用例。 验证：gradle clean test --offline → 78 tests / 0 failures / 1 skipped （PG Testcontainers 集成用例在本机无 docker 时按既有约定 assumeTrue 跳过）。
This commit is contained in:
windyboy
@@ -45,6 +45,7 @@ class PipelineSmokeTest {
|
||||
fun cleanStubs() {
|
||||
ctx.getBean(StubProcState::class.java).clear()
|
||||
ctx.getBean(com.gzzn.omms.msgexchange.infra.stub.StubInbox::class.java).clear()
|
||||
ctx.getBean(com.gzzn.omms.msgexchange.infra.stub.StubInboxCursor::class.java).clear()
|
||||
ctx.getBean(StubMsgEvents::class.java).clear()
|
||||
ctx.getBean(StubDeliveryPort::class.java).clear()
|
||||
}
|
||||
@@ -100,6 +101,30 @@ class PipelineSmokeTest {
|
||||
assertEquals(0, reopened.attempts)
|
||||
}
|
||||
|
||||
/**
|
||||
* 回归（message-lifecycle §5.2/§5.3）：死信到达终态时同时登记回填意图并立即回填——
|
||||
* 否则永不回填的行会永久占据发现窗口，累积到批大小后收报整体停摆。
|
||||
*/
|
||||
@Test
|
||||
fun `dead letter reaches terminal state, gets marked and cannot block later discovery`() {
|
||||
val inbox = ctx.getBean(com.gzzn.omms.msgexchange.infra.stub.StubInbox::class.java)
|
||||
val proc = ctx.getBean(StubProcState::class.java)
|
||||
val dead = inbox.simulateExternalWrite("<MSG/>")
|
||||
ctx.getBean(com.gzzn.omms.msgexchange.ingress.InboxPoller::class.java).pollOnce()
|
||||
|
||||
pump.tick() // 解码 MALFORMED → DEAD + 回填意图（同一条 UPDATE）→ 提交后立即回填
|
||||
|
||||
val row = proc.find(dead)!!
|
||||
assertEquals(ProcStatus.DEAD, row.state)
|
||||
assertEquals(ErrorClass.MALFORMED, row.errorClass)
|
||||
assertNotNull(row.backfillAt)
|
||||
assertTrue(inbox.isMarked(dead))
|
||||
|
||||
val fresh = inbox.simulateExternalWrite("<MSG/>")
|
||||
ctx.getBean(com.gzzn.omms.msgexchange.ingress.InboxPoller::class.java).pollOnce()
|
||||
assertNotNull(proc.find(fresh)) // 死信不阻断后续发现
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `dispatcher flushes schd batch through stub port`() {
|
||||
val events = ctx.getBean(StubMsgEvents::class.java)
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
package com.gzzn.omms.msgexchange.infra.health
|
||||
|
||||
import com.gzzn.omms.msgexchange.MutableClock
|
||||
import com.gzzn.omms.msgexchange.delivery.DeliveryPort
|
||||
import com.gzzn.omms.msgexchange.domain.ProcStatus
|
||||
import com.gzzn.omms.msgexchange.infra.persistence.InboxCursorRepository
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubInbox
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubInboxCursor
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubProcState
|
||||
import io.micronaut.health.HealthStatus
|
||||
import org.junit.jupiter.api.Assertions.assertEquals
|
||||
import org.junit.jupiter.api.Test
|
||||
@@ -32,4 +38,35 @@ class HealthIndicatorsTest {
|
||||
assertEquals(HealthStatus.DOWN, kafkaHealth(ThrowingPort()).status)
|
||||
assertEquals(HealthStatus.DOWN, kafkaHealth(null).status) // bean 缺失
|
||||
}
|
||||
|
||||
/** OPS-2（message-lifecycle §5.3）：积压、最老未处理信龄、未回填终态与水位滞后。 */
|
||||
@Test
|
||||
fun `inbox lifecycle reports backlog, oldest age, unmarked terminals and watermark lag`() {
|
||||
val proc = StubProcState()
|
||||
val inbox = StubInbox().apply { clear() }
|
||||
val cursor = StubInboxCursor()
|
||||
val oldest = MutableClock.BASE.minusSeconds(3600)
|
||||
proc.insertIfAbsent(1L, oldest)
|
||||
proc.insertIfAbsent(2L, MutableClock.BASE)
|
||||
proc.markTerminal(2L, ProcStatus.SUCCEEDED)
|
||||
val newest = inbox.insertRaw("<MSG/>")
|
||||
cursor.save(InboxCursorRepository.Cursor(committedUpTo = newest - 2))
|
||||
|
||||
val result = lifecycleHealth(proc, cursor, inbox, now = MutableClock.BASE)
|
||||
|
||||
assertEquals(HealthStatus.UP, result.status)
|
||||
@Suppress("UNCHECKED_CAST")
|
||||
val details = result.details as Map<String, Any>
|
||||
assertEquals(1, details["backlog"])
|
||||
assertEquals(3600L, details["oldestUnprocessedSeconds"])
|
||||
assertEquals(1, details["unmarkedTerminal"])
|
||||
assertEquals(2L, details["watermarkLag"])
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `inbox lifecycle stays up when ports are not bound`() {
|
||||
val result = lifecycleHealth(procState = null, cursor = null, mailbox = null)
|
||||
|
||||
assertEquals(HealthStatus.UP, result.status) // 可用性由依赖自身指示器承担
|
||||
}
|
||||
}
|
||||
|
||||
+35
-10
@@ -9,10 +9,12 @@ import org.junit.jupiter.api.Test
|
||||
import java.sql.DriverManager
|
||||
|
||||
/**
|
||||
* Flyway 迁移引擎端到端验证（docs/flight-state.md §2 权威模型 + design.md §2.1）：
|
||||
* 1. V1__flight_state_baseline.sql 在真实 PostgreSQL 上自动迁移成功；
|
||||
* Flyway 迁移引擎端到端验证（docs/flight-state.md §2 权威模型 + design.md §2.1 +
|
||||
* message-lifecycle.md §5.1/§5.2）：
|
||||
* 1. V1 基线 + V2 信箱生命周期迁移在真实 PostgreSQL 上自动成功；
|
||||
* 2. flyway_schema_history 落库且 success = true；
|
||||
* 3. 决策层/管道层/留痕层全表就绪；PIPELINE_LOCK 单行种子就位。
|
||||
* 3. 决策层/管道层/留痕层全表就绪；PIPELINE_LOCK 与 INBOX_CURSOR 单行种子就位；
|
||||
* 4. V2 收敛结果成立：回填事实并入 PROC_STATE，BACKFILL_TODO 下线。
|
||||
*/
|
||||
class FlywayMigrationTest {
|
||||
|
||||
@@ -20,11 +22,11 @@ class FlywayMigrationTest {
|
||||
"pipeline_lock", "proc_state", "flight_schd",
|
||||
"flight_gate", "flight_checkin", "flight_belt", "flight_stand_plan", "flight_chute",
|
||||
"flight_delay", "flight_bridge_op", "flight_chock_op", "flight_route_point",
|
||||
"msg_event", "req_track", "backfill_todo", "schd_snap_log",
|
||||
"msg_event", "req_track", "inbox_cursor", "schd_snap_log",
|
||||
)
|
||||
|
||||
@Test
|
||||
fun `Flyway migrates flight-state baseline onto real PostgreSQL`() {
|
||||
fun `Flyway migrates flight-state baseline and inbox lifecycle onto real PostgreSQL`() {
|
||||
assumeTrue(PgTestSupport.canConnect(), PgTestSupport.skipMessage())
|
||||
val url = PgTestSupport.jdbcUrl
|
||||
val user = PgTestSupport.user
|
||||
@@ -39,7 +41,7 @@ class FlywayMigrationTest {
|
||||
|
||||
DriverManager.getConnection(url, user, pass).use { conn ->
|
||||
conn.createStatement().use { stmt ->
|
||||
// 迁移记录：单基线 V1
|
||||
// 迁移记录：V1 基线 + V2 信箱生命周期
|
||||
stmt.executeQuery(
|
||||
"SELECT version, script, success FROM flyway_schema_history ORDER BY installed_rank ASC",
|
||||
).use { rs ->
|
||||
@@ -47,13 +49,15 @@ class FlywayMigrationTest {
|
||||
while (rs.next()) {
|
||||
records.add(Triple(rs.getString("version"), rs.getString("script"), rs.getBoolean("success")))
|
||||
}
|
||||
assertTrue(records.isNotEmpty(), "flyway_schema_history must record migrations")
|
||||
assertTrue(records.size >= 2, "flyway_schema_history must record both migrations")
|
||||
assertEquals("1", records[0].first)
|
||||
assertEquals("V1__flight_state_baseline.sql", records[0].second)
|
||||
assertTrue(records[0].third)
|
||||
assertEquals("2", records[1].first)
|
||||
assertEquals("V2__inbox_lifecycle.sql", records[1].second)
|
||||
assertTrue(records.all { it.third })
|
||||
}
|
||||
|
||||
// §3.2 全表就绪
|
||||
// 全表就绪（V2 后 backfill_todo 下线）
|
||||
stmt.executeQuery(
|
||||
"SELECT table_name FROM information_schema.tables WHERE table_schema = 'public'",
|
||||
).use { rs ->
|
||||
@@ -61,6 +65,7 @@ class FlywayMigrationTest {
|
||||
while (rs.next()) tables.add(rs.getString("table_name"))
|
||||
val missing = expectedTables - tables
|
||||
assertTrue(missing.isEmpty(), "missing tables: $missing")
|
||||
assertTrue("backfill_todo" !in tables, "BACKFILL_TODO 已由 PROC_STATE 回填列取代")
|
||||
}
|
||||
|
||||
// 身份不变量基础结构：FLIGHT_SCHD 主键 + STATE 列
|
||||
@@ -73,11 +78,31 @@ class FlywayMigrationTest {
|
||||
assertEquals(setOf("flid", "operation_day", "state", "state_version", "last_msg_id"), cols)
|
||||
}
|
||||
|
||||
// PIPELINE_LOCK 单行种子（§3.2）
|
||||
// 回填事实并入 PROC_STATE（§5.2 判据 + 回填待办）
|
||||
stmt.executeQuery(
|
||||
"SELECT column_name FROM information_schema.columns WHERE table_name = 'proc_state' " +
|
||||
"AND column_name IN ('received_at', 'backfill_at', 'backfill_next_at', " +
|
||||
"'backfill_attempts', 'backfill_error')",
|
||||
).use { rs ->
|
||||
val cols = mutableSetOf<String>()
|
||||
while (rs.next()) cols.add(rs.getString("column_name"))
|
||||
assertEquals(
|
||||
setOf("received_at", "backfill_at", "backfill_next_at", "backfill_attempts", "backfill_error"),
|
||||
cols,
|
||||
)
|
||||
}
|
||||
|
||||
// PIPELINE_LOCK 与 INBOX_CURSOR 单行种子（§3.2 / §5.1）
|
||||
stmt.executeQuery("SELECT count(*) FROM pipeline_lock WHERE lock_id = 1").use { rs ->
|
||||
assertTrue(rs.next())
|
||||
assertEquals(1, rs.getInt(1))
|
||||
}
|
||||
stmt.executeQuery("SELECT committed_up_to, hole_since FROM inbox_cursor WHERE cursor_id = 1").use { rs ->
|
||||
assertTrue(rs.next(), "INBOX_CURSOR 单行种子必须就位")
|
||||
assertEquals(0L, rs.getLong("committed_up_to"))
|
||||
rs.getTimestamp("hole_since")
|
||||
assertTrue(rs.wasNull(), "初始无空洞观测")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+221
@@ -0,0 +1,221 @@
|
||||
package com.gzzn.omms.msgexchange.infra.persistence.jdbc
|
||||
|
||||
import com.gzzn.omms.msgexchange.domain.ErrorClass
|
||||
import com.gzzn.omms.msgexchange.domain.ProcStatus
|
||||
import com.gzzn.omms.msgexchange.infra.persistence.InboxCursorRepository
|
||||
import org.h2.jdbcx.JdbcDataSource
|
||||
import org.junit.jupiter.api.Assertions.assertEquals
|
||||
import org.junit.jupiter.api.Assertions.assertFalse
|
||||
import org.junit.jupiter.api.Assertions.assertNotNull
|
||||
import org.junit.jupiter.api.Assertions.assertNull
|
||||
import org.junit.jupiter.api.Assertions.assertTrue
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import java.sql.Connection
|
||||
import java.sql.Timestamp
|
||||
import java.time.Duration
|
||||
import java.time.Instant
|
||||
import java.util.UUID
|
||||
import javax.sql.DataSource
|
||||
|
||||
/**
|
||||
* 信箱生命周期 SQL 语义（message-lifecycle §4/§5.1/§5.2/§11）在真实 JDBC 上的验证。
|
||||
* 以 H2 的 PostgreSQL 兼容模式承载与 V1+V2 等价的表结构——不依赖 docker/外接库，
|
||||
* 覆盖 PG 侧（终态与回填意图同体、到期/超期筛选、积压观测、水位游标）与 MySQL 侧
|
||||
* （区间发现、标记单调）的实际语句行为。
|
||||
*
|
||||
* 说明：H2 不支持 `INSERT ... ON CONFLICT DO NOTHING`，
|
||||
* [JdbcProcStateRepository.insertIfAbsent] 的入队幂等由 PG 语义与
|
||||
* `InboxPollerTest`（重复轮询不再入队）分别保证。
|
||||
*/
|
||||
class InboxLifecycleJdbcSqlTest {
|
||||
|
||||
private lateinit var ds: DataSource
|
||||
private lateinit var proc: JdbcProcStateRepository
|
||||
private lateinit var mailbox: JdbcCminmsgInboxRepository
|
||||
private lateinit var cursor: JdbcInboxCursorRepository
|
||||
|
||||
private val t0: Instant = Instant.parse("2026-09-08T03:00:00Z")
|
||||
private val overdue: Instant = t0.minus(Duration.ofDays(31))
|
||||
|
||||
@BeforeEach
|
||||
fun setUp() {
|
||||
ds = JdbcDataSource().apply {
|
||||
setURL("jdbc:h2:mem:inbox-${UUID.randomUUID()};DB_CLOSE_DELAY=-1;MODE=PostgreSQL")
|
||||
setUser("sa")
|
||||
setPassword("")
|
||||
}
|
||||
ds.connection.use { conn ->
|
||||
conn.createStatement().use { st ->
|
||||
st.execute(PROC_STATE_DDL)
|
||||
st.execute(CURSOR_DDL)
|
||||
st.execute(MAILBOX_DDL)
|
||||
st.execute(
|
||||
"INSERT INTO inbox_cursor (cursor_id, committed_up_to, hole_since, updated_at) " +
|
||||
"VALUES (1, 0, NULL, CURRENT_TIMESTAMP)",
|
||||
)
|
||||
}
|
||||
}
|
||||
proc = JdbcProcStateRepository(ds)
|
||||
mailbox = JdbcCminmsgInboxRepository(ds)
|
||||
cursor = JdbcInboxCursorRepository(ds)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `terminal write carries the backfill intent in the same statement`() {
|
||||
seed(11L, t0)
|
||||
|
||||
proc.markTerminal(11L, ProcStatus.DEAD, errorClass = ErrorClass.MALFORMED, lastError = "raw-missing", now = t0)
|
||||
|
||||
val row = proc.find(11L)!!
|
||||
assertEquals(ProcStatus.DEAD, row.state)
|
||||
assertEquals(ErrorClass.MALFORMED, row.errorClass)
|
||||
assertEquals(t0, row.backfillNextAt) // 终态与回填意图同一条 UPDATE（§2/§4）
|
||||
assertNull(row.backfillAt)
|
||||
assertEquals(t0, row.receivedAt)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `backfill failure is recorded on the same row without touching the terminal state`() {
|
||||
seed(11L, t0)
|
||||
proc.markTerminal(11L, ProcStatus.SUCCEEDED, now = t0)
|
||||
|
||||
proc.recordBackfillFailure(11L, "mysql-down", attempts = 2, nextAttemptAt = t0.plusSeconds(120), now = t0)
|
||||
|
||||
val row = proc.find(11L)!!
|
||||
assertEquals(ProcStatus.SUCCEEDED, row.state) // §11 终态不可逆
|
||||
assertEquals(2, row.backfillAttempts)
|
||||
assertEquals(t0.plusSeconds(120), row.backfillNextAt)
|
||||
assertEquals("mysql-down", row.backfillError)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `due query honours due time, the overdue deadline and terminal-only filter`() {
|
||||
// ① 到期
|
||||
seed(1L, t0)
|
||||
proc.markTerminal(1L, ProcStatus.SUCCEEDED, now = t0)
|
||||
// ② 未到期（退避推后）
|
||||
seed(2L, t0)
|
||||
proc.markTerminal(2L, ProcStatus.SUCCEEDED, now = t0)
|
||||
proc.recordBackfillFailure(2L, "mysql-down", 1, t0.plus(Duration.ofMinutes(15)), t0)
|
||||
// ③ 未到期但接收时间已达超期期限 R（§5.2 覆盖退避）
|
||||
seed(3L, overdue)
|
||||
proc.markTerminal(3L, ProcStatus.DEAD, errorClass = ErrorClass.MALFORMED, now = t0)
|
||||
proc.recordBackfillFailure(3L, "mysql-down", 1, t0.plus(Duration.ofMinutes(15)), t0)
|
||||
// ④ 中间态：永不补写（§5.2）
|
||||
seed(4L, overdue)
|
||||
// ⑤ 已确认标记
|
||||
seed(5L, t0)
|
||||
proc.markTerminal(5L, ProcStatus.SUCCEEDED, now = t0)
|
||||
proc.markBackfilled(5L, t0)
|
||||
|
||||
val due = proc.findBackfillDue(t0, t0.minus(Duration.ofDays(30)), limit = 100).map { it.msgId }
|
||||
|
||||
assertEquals(listOf(1L, 3L), due)
|
||||
assertEquals(listOf(1L), proc.findBackfillDue(t0, t0.minus(Duration.ofDays(30)), limit = 1).map { it.msgId })
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `backlog reports unfinished messages, oldest receive time and unmarked terminals`() {
|
||||
seed(1L, overdue)
|
||||
seed(2L, t0)
|
||||
proc.markTerminal(2L, ProcStatus.SUCCEEDED, now = t0)
|
||||
|
||||
val backlog = proc.backlog()
|
||||
|
||||
assertEquals(1, backlog.unfinished)
|
||||
assertEquals(overdue, backlog.oldestReceivedAt) // OPS-2 最老未处理信龄锚点
|
||||
assertEquals(1, backlog.unmarkedTerminal)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `watermark round trips with and without a pending hole`() {
|
||||
assertEquals(InboxCursorRepository.Cursor(0L, null), cursor.load())
|
||||
|
||||
cursor.save(InboxCursorRepository.Cursor(42L, t0))
|
||||
|
||||
assertEquals(InboxCursorRepository.Cursor(42L, t0), cursor.load())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `mailbox range read ignores processing marks and marking never overwrites`() {
|
||||
val first = mailbox.insertRaw("<A/>")
|
||||
val second = mailbox.insertRaw("<B/>")
|
||||
val third = mailbox.insertRaw("<C/>")
|
||||
|
||||
// 发现按 ID 区间；处理标记不参与扫描谓词
|
||||
assertEquals(listOf(first, second, third), mailbox.readRange(0L, 50).map { it.msgId })
|
||||
assertNotNull(mailbox.readRange(0L, 50).first().receivedAt)
|
||||
assertEquals(listOf(second, third), mailbox.readRange(first, 50).map { it.msgId })
|
||||
assertEquals(third, mailbox.maxId())
|
||||
|
||||
assertTrue(mailbox.markProcessedIfUnmarked(second, "PROCESSED"))
|
||||
assertFalse(mailbox.markProcessedIfUnmarked(second, "OTHER")) // §11 只把空标写为已处理
|
||||
assertEquals("PROCESSED", statusOf(second))
|
||||
// 已标记行仍出现在区间读结果中（发现与标记彻底解耦）
|
||||
assertEquals(listOf(first, second, third), mailbox.readRange(0L, 50).map { it.msgId })
|
||||
assertFalse(mailbox.markProcessedIfUnmarked(999L, "PROCESSED"))
|
||||
}
|
||||
|
||||
private fun seed(msgId: Long, receivedAt: Instant?, state: ProcStatus = ProcStatus.PENDING) {
|
||||
ds.connection.use { conn: Connection ->
|
||||
conn.prepareStatement(
|
||||
"INSERT INTO proc_state (msg_id, state, received_at, backfill_attempts, updated_at) VALUES (?, ?, ?, 0, ?)",
|
||||
).use { ps ->
|
||||
ps.setLong(1, msgId)
|
||||
ps.setString(2, state.name)
|
||||
ps.setTimestamp(3, receivedAt?.let { Timestamp.from(it) })
|
||||
ps.setTimestamp(4, Timestamp.from(t0))
|
||||
ps.executeUpdate()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun statusOf(msgId: Long): String? =
|
||||
ds.connection.use { conn: Connection ->
|
||||
conn.prepareStatement("SELECT CMINMSGS_STATUS FROM cminmsgs WHERE CMINMSGS_ID = ?").use { ps ->
|
||||
ps.setLong(1, msgId)
|
||||
ps.executeQuery().use { rs -> if (rs.next()) rs.getString(1) else null }
|
||||
}
|
||||
}
|
||||
|
||||
private companion object {
|
||||
const val PROC_STATE_DDL = """
|
||||
CREATE TABLE proc_state (
|
||||
msg_id BIGINT PRIMARY KEY,
|
||||
state VARCHAR(16) NOT NULL,
|
||||
identity_key VARCHAR(200),
|
||||
attempts INT NOT NULL DEFAULT 0,
|
||||
next_attempt_at TIMESTAMP WITH TIME ZONE,
|
||||
error_class VARCHAR(20),
|
||||
last_error VARCHAR(1000),
|
||||
received_at TIMESTAMP WITH TIME ZONE,
|
||||
backfill_at TIMESTAMP WITH TIME ZONE,
|
||||
backfill_next_at TIMESTAMP WITH TIME ZONE,
|
||||
backfill_attempts INT NOT NULL DEFAULT 0,
|
||||
backfill_error VARCHAR(512),
|
||||
updated_at TIMESTAMP WITH TIME ZONE NOT NULL,
|
||||
CONSTRAINT uk_proc_identity UNIQUE (identity_key)
|
||||
)
|
||||
"""
|
||||
|
||||
const val CURSOR_DDL = """
|
||||
CREATE TABLE inbox_cursor (
|
||||
cursor_id INT PRIMARY KEY,
|
||||
committed_up_to BIGINT NOT NULL,
|
||||
hole_since TIMESTAMP WITH TIME ZONE,
|
||||
updated_at TIMESTAMP WITH TIME ZONE NOT NULL
|
||||
)
|
||||
"""
|
||||
|
||||
const val MAILBOX_DDL = """
|
||||
CREATE TABLE cminmsgs (
|
||||
CMINMSGS_ID BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
CMINMSGS_CLOB_MSG CLOB,
|
||||
CMINMSGS_DATE_RECEIVED TIMESTAMP,
|
||||
CMINMSGS_DATE_PROCESSED TIMESTAMP,
|
||||
CMINMSGS_STATUS VARCHAR(32)
|
||||
)
|
||||
"""
|
||||
}
|
||||
}
|
||||
@@ -23,8 +23,7 @@ class ReplayServiceTest {
|
||||
rows[id] = ProcState(id, status, identityKey = "k$id", attempts = 3, errorClass = ec, lastError = "x")
|
||||
}
|
||||
|
||||
override fun insert(msgId: Long, state: ProcStatus) = Unit
|
||||
override fun exists(msgId: Long): Boolean = rows.containsKey(msgId)
|
||||
override fun insertIfAbsent(msgId: Long, receivedAt: Instant?): Boolean = rows.putIfAbsent(msgId, ProcState(msgId, ProcStatus.PENDING, receivedAt = receivedAt)) == null
|
||||
override fun find(msgId: Long): ProcState? = rows[msgId]
|
||||
override fun findSuccessTerminal(msgId: Long): Boolean = rows[msgId]?.state == ProcStatus.SUCCEEDED
|
||||
override fun headUnfinished(): ProcState? = null
|
||||
@@ -35,6 +34,20 @@ class ReplayServiceTest {
|
||||
errorClass: ErrorClass?, lastError: String?,
|
||||
) = Unit
|
||||
|
||||
override fun markTerminal(
|
||||
msgId: Long, state: ProcStatus, errorClass: ErrorClass?, lastError: String?,
|
||||
attempts: Int?, now: Instant,
|
||||
) = Unit
|
||||
|
||||
override fun markBackfilled(msgId: Long, now: Instant) = Unit
|
||||
|
||||
override fun recordBackfillFailure(msgId: Long, error: String?, attempts: Int, nextAttemptAt: Instant, now: Instant) = Unit
|
||||
|
||||
override fun findBackfillDue(now: Instant, overdueBefore: Instant, limit: Int) =
|
||||
emptyList<com.gzzn.omms.msgexchange.infra.persistence.BackfillDue>()
|
||||
|
||||
override fun backlog() = com.gzzn.omms.msgexchange.infra.persistence.Backlog(0, null, 0)
|
||||
|
||||
override fun requeueByErrorClasses(errorClasses: List<ErrorClass>): Int {
|
||||
requeueCalls += errorClasses
|
||||
var n = 0
|
||||
|
||||
@@ -1,47 +1,120 @@
|
||||
package com.gzzn.omms.msgexchange.ingress
|
||||
|
||||
import com.gzzn.omms.msgexchange.config.PipelineProps
|
||||
import com.gzzn.omms.msgexchange.domain.ErrorClass
|
||||
import com.gzzn.omms.msgexchange.domain.ProcStatus
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubInbox
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubInboxCursor
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubPipelineTx
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubProcState
|
||||
import com.gzzn.omms.msgexchange.processing.Pump
|
||||
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
|
||||
import jakarta.inject.Inject
|
||||
import org.junit.jupiter.api.Assertions.assertEquals
|
||||
import org.junit.jupiter.api.Assertions.assertFalse
|
||||
import org.junit.jupiter.api.Assertions.assertNotNull
|
||||
import org.junit.jupiter.api.Assertions.assertNull
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import java.time.Instant
|
||||
|
||||
/** 主路径：JDBC 轮询语义（stub 下 InboxPoller + 外部写信箱模拟）。 */
|
||||
@MicronautTest
|
||||
/**
|
||||
* 收报发现权不变量（docs/message-lifecycle.md §5.1/§5.3 + architecture.md §5 严格 FIFO）：
|
||||
* - 扫描按 ID 区间，**不受处理标记影响**：终态而未回填的行不得占据批次、不得阻断新信发现；
|
||||
* - 水位只随成功入队推进，且与入队同事务（中断后由重扫补建）；
|
||||
* - 遇空洞即停（较小 ID 未入队时不得被后续消息越过）；空洞老化后放行（水位不得永久停摆）；
|
||||
* - 收报层不写处理标记。
|
||||
*/
|
||||
class InboxPollerTest {
|
||||
|
||||
@Inject lateinit var poller: InboxPoller
|
||||
@Inject lateinit var pump: Pump
|
||||
@Inject lateinit var stubInbox: StubInbox
|
||||
@Inject lateinit var stubProc: StubProcState
|
||||
private val t0: Instant = Instant.parse("2026-09-08T03:00:00Z")
|
||||
private val props = PipelineProps()
|
||||
|
||||
private lateinit var inbox: StubInbox
|
||||
private lateinit var proc: StubProcState
|
||||
private lateinit var cursor: StubInboxCursor
|
||||
private lateinit var poller: InboxPoller
|
||||
|
||||
@BeforeEach
|
||||
fun clean() {
|
||||
stubInbox.clear()
|
||||
stubProc.clear()
|
||||
fun setUp() {
|
||||
inbox = StubInbox().apply { clear() }
|
||||
proc = StubProcState().apply { clear() }
|
||||
cursor = StubInboxCursor().apply { clear() }
|
||||
poller = InboxPoller(inbox, proc, cursor, StubPipelineTx(), props)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `poller enqueues externally written mailbox rows`() {
|
||||
val id = stubInbox.simulateExternalWrite("<MSG/>")
|
||||
assertEquals(1, poller.pollOnce())
|
||||
assertNotNull(stubProc.snapshotOf(id))
|
||||
assertEquals(ProcStatus.PENDING, stubProc.snapshotOf(id)!!.state)
|
||||
fun `external rows are enqueued in id order and advance the watermark without marking the mailbox`() {
|
||||
val first = inbox.simulateExternalWrite("<MSG/>")
|
||||
val second = inbox.simulateExternalWrite("<MSG/>")
|
||||
|
||||
assertEquals(2, poller.pollOnce(t0))
|
||||
|
||||
assertEquals(ProcStatus.PENDING, proc.find(first)!!.state)
|
||||
assertEquals(ProcStatus.PENDING, proc.find(second)!!.state)
|
||||
assertEquals(second, cursor.cursor.committedUpTo)
|
||||
assertNull(cursor.cursor.holeSince)
|
||||
// §5.3：消化阶段只写自有 PG，不触碰信箱标记
|
||||
assertFalse(inbox.isMarked(first))
|
||||
assertEquals(0, poller.pollOnce(t0)) // 重复扫描幂等
|
||||
}
|
||||
|
||||
/**
|
||||
* 回归（US-01 条目 3 / §5.3「积压挡批」）：终态且永不回填的行（解码失败死信等）曾占满
|
||||
* 有限批次使收报整体停摆——发现必须与处理标记彻底解耦。
|
||||
*/
|
||||
@Test
|
||||
fun `terminal rows without a mailbox mark do not block discovery of later messages`() {
|
||||
props.pipeline.claimBatch = 3
|
||||
val dead = (1..3).map { inbox.simulateExternalWrite("<MSG/>") }
|
||||
assertEquals(3, poller.pollOnce(t0))
|
||||
dead.forEach {
|
||||
proc.markTerminal(it, ProcStatus.DEAD, errorClass = ErrorClass.MALFORMED, lastError = "raw-missing")
|
||||
}
|
||||
|
||||
val fresh = inbox.simulateExternalWrite("<MSG/>")
|
||||
|
||||
assertEquals(1, poller.pollOnce(t0))
|
||||
assertEquals(ProcStatus.PENDING, proc.find(fresh)!!.state)
|
||||
assertEquals(fresh, cursor.cursor.committedUpTo)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `poller is idempotent for already enqueued rows`() {
|
||||
val id = stubInbox.simulateExternalWrite("<MSG/>")
|
||||
poller.pollOnce()
|
||||
assertEquals(0, poller.pollOnce())
|
||||
pump.tick()
|
||||
assertEquals(ProcStatus.DEAD, stubProc.snapshotOf(id)!!.state)
|
||||
assertEquals(ErrorClass.MALFORMED, stubProc.snapshotOf(id)!!.errorClass)
|
||||
fun `watermark stops at a hole so later ids cannot overtake a missing smaller id`() {
|
||||
val first = inbox.simulateExternalWrite("<MSG/>")
|
||||
val hole = inbox.simulateExternalWrite("<MSG/>")
|
||||
val afterHole = inbox.simulateExternalWrite("<MSG/>")
|
||||
inbox.removeRow(hole)
|
||||
|
||||
assertEquals(1, poller.pollOnce(t0))
|
||||
|
||||
assertEquals(first, cursor.cursor.committedUpTo)
|
||||
assertNotNull(cursor.cursor.holeSince)
|
||||
assertNull(proc.find(afterHole)) // 不得越过空洞入队（FIFO）
|
||||
assertEquals(0, poller.pollOnce(t0.plusSeconds(60))) // 宽限期内水位不推进
|
||||
assertEquals(first, cursor.cursor.committedUpTo)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `an aged hole is released and later ids resume enqueuing`() {
|
||||
val hole = inbox.simulateExternalWrite("<MSG/>")
|
||||
val afterHole = inbox.simulateExternalWrite("<MSG/>")
|
||||
inbox.removeRow(hole)
|
||||
poller.pollOnce(t0) // 记录空洞观测时刻
|
||||
|
||||
val agedOut = t0.plus(props.pipeline.maxCommitDelay)
|
||||
assertEquals(0, poller.pollOnce(agedOut)) // 空洞判永久：推进水位但不越过入队
|
||||
|
||||
assertNull(cursor.cursor.holeSince)
|
||||
assertEquals(afterHole - 1, cursor.cursor.committedUpTo)
|
||||
assertEquals(1, poller.pollOnce(agedOut)) // 下一轮恢复发现
|
||||
assertEquals(afterHole, cursor.cursor.committedUpTo)
|
||||
assertNotNull(proc.find(afterHole))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `compat http path and poller do not double enqueue the same message`() {
|
||||
val receipt = InboxService(inbox, proc).accept("<MSG/>", t0)
|
||||
|
||||
assertEquals(0, poller.pollOnce(t0))
|
||||
assertEquals(receipt.msgId, cursor.cursor.committedUpTo) // 已在 PG：读取进度照常推进
|
||||
assertNotNull(proc.find(receipt.msgId))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,90 +0,0 @@
|
||||
package com.gzzn.omms.msgexchange.jobs
|
||||
|
||||
import com.gzzn.omms.msgexchange.config.PipelineProps
|
||||
import com.gzzn.omms.msgexchange.infra.persistence.BackfillTodoRepository
|
||||
import com.gzzn.omms.msgexchange.infra.persistence.CminmsgInboxRepository
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubBackfillTodo
|
||||
import org.junit.jupiter.api.Assertions.assertEquals
|
||||
import org.junit.jupiter.api.Assertions.assertTrue
|
||||
import org.junit.jupiter.api.Test
|
||||
import java.time.Duration
|
||||
import java.time.Instant
|
||||
|
||||
/**
|
||||
* v2 §5（ACM2-29 P2-3）：提交后回填补偿——失败落库、到期重试、成功即清、退避推后。
|
||||
* 重启对账语义：待办持久化后，任意时刻新的 sweep 都能继续推进（不依赖内存状态）。
|
||||
*/
|
||||
class BackfillSweepJobTest {
|
||||
|
||||
private class FakeInbox(var fail: Boolean = false) : CminmsgInboxRepository {
|
||||
val backfilled = mutableSetOf<Long>()
|
||||
override fun insertRaw(rawXml: String) = 1L
|
||||
override fun rawOf(cminmsgsId: Long): String? = null
|
||||
override fun pollUnprocessed(afterId: Long, limit: Int): List<Long> = emptyList()
|
||||
override fun backfillOnSuccess(cminmsgsId: Long, sndr: String, type: String, styp: String, seqn: Long) {
|
||||
if (fail) throw IllegalStateException("mysql-down")
|
||||
backfilled += cminmsgsId
|
||||
}
|
||||
}
|
||||
|
||||
private val t0: Instant = Instant.parse("2026-09-08T03:00:00Z")
|
||||
|
||||
private fun task(id: Long = 901L) = BackfillTodoRepository.BackfillTask(id, "AODB", "SCHD", "DNLD", 9L)
|
||||
|
||||
@Test
|
||||
fun `due task is retried and removed after successful backfill`() {
|
||||
val todo = StubBackfillTodo()
|
||||
val inbox = FakeInbox(fail = false)
|
||||
todo.record(task(), "mysql-down", t0)
|
||||
val job = BackfillSweepJob(todo, inbox, PipelineProps())
|
||||
|
||||
val outcome = job.sweep(t0.plusSeconds(60))
|
||||
|
||||
assertEquals(BackfillSweepJob.SweepOutcome(inspected = 1, succeeded = 1, failed = 0), outcome)
|
||||
assertEquals(0, todo.count())
|
||||
assertTrue(901L in inbox.backfilled)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `failed retry increments attempts and applies exponential backoff`() {
|
||||
val todo = StubBackfillTodo()
|
||||
val inbox = FakeInbox(fail = true)
|
||||
todo.record(task(), "mysql-down", t0)
|
||||
val job = BackfillSweepJob(todo, inbox, PipelineProps())
|
||||
|
||||
val outcome = job.sweep(t0)
|
||||
|
||||
assertEquals(BackfillSweepJob.SweepOutcome(inspected = 1, succeeded = 0, failed = 1), outcome)
|
||||
assertEquals(1, todo.count())
|
||||
assertEquals(1, todo.tasks.values.single().attempts)
|
||||
assertEquals("mysql-down", todo.lastErrorOf(901L))
|
||||
// 一次失败后 30s 内不再到期（退避生效）
|
||||
assertEquals(0, todo.findDue(t0.plusSeconds(29)).size)
|
||||
assertEquals(1, todo.findDue(t0.plusSeconds(30)).size)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `recovered inbox completes compensation on a later sweep as if restarted`() {
|
||||
val todo = StubBackfillTodo()
|
||||
val inbox = FakeInbox(fail = true)
|
||||
todo.record(task(), "mysql-down", t0)
|
||||
val job = BackfillSweepJob(todo, inbox, PipelineProps())
|
||||
job.sweep(t0)
|
||||
job.sweep(t0.plusSeconds(30)) // 第二次仍失败 → attempts=2
|
||||
|
||||
inbox.fail = false // 共享信箱恢复（模拟重启后对账）
|
||||
val outcome = job.sweep(t0.plusSeconds(90))
|
||||
|
||||
assertEquals(BackfillSweepJob.SweepOutcome(inspected = 1, succeeded = 1, failed = 0), outcome)
|
||||
assertEquals(0, todo.count())
|
||||
assertTrue(901L in inbox.backfilled)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `backoff delay doubles per attempt and caps at fifteen minutes`() {
|
||||
assertEquals(Duration.ofSeconds(30), BackfillSweepJob.backoffDelayFor(1))
|
||||
assertEquals(Duration.ofMinutes(2), BackfillSweepJob.backoffDelayFor(3))
|
||||
assertEquals(Duration.ofMinutes(15), BackfillSweepJob.backoffDelayFor(10))
|
||||
assertEquals(Duration.ofMinutes(15), BackfillSweepJob.backoffDelayFor(50))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,170 @@
|
||||
package com.gzzn.omms.msgexchange.processing
|
||||
|
||||
import com.gzzn.omms.msgexchange.config.MailboxProps
|
||||
import com.gzzn.omms.msgexchange.config.PipelineProps
|
||||
import com.gzzn.omms.msgexchange.domain.ErrorClass
|
||||
import com.gzzn.omms.msgexchange.domain.ProcStatus
|
||||
import com.gzzn.omms.msgexchange.infra.persistence.CminmsgInboxRepository
|
||||
import com.gzzn.omms.msgexchange.infra.persistence.MailboxRow
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubInbox
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubProcState
|
||||
import org.junit.jupiter.api.Assertions.assertEquals
|
||||
import org.junit.jupiter.api.Assertions.assertFalse
|
||||
import org.junit.jupiter.api.Assertions.assertNotNull
|
||||
import org.junit.jupiter.api.Assertions.assertNull
|
||||
import org.junit.jupiter.api.Assertions.assertTrue
|
||||
import org.junit.jupiter.api.Test
|
||||
import java.time.Clock
|
||||
import java.time.Duration
|
||||
import java.time.Instant
|
||||
import java.time.ZoneOffset
|
||||
|
||||
/**
|
||||
* 回填通道不变量（docs/message-lifecycle.md §3/§4/§5.2/§11）：
|
||||
* 终态即刻补写、失败退避重试、超期期限 R 覆盖退避、标记单调只写空标、
|
||||
* 中间态不适用、回填失败不回改终态、重启后扫描不依赖内存状态。
|
||||
*/
|
||||
class BackfillServiceTest {
|
||||
|
||||
private val t0: Instant = Instant.parse("2026-09-08T03:00:00Z")
|
||||
private val props = PipelineProps()
|
||||
|
||||
/** 可注入故障的信箱：验证回填失败路径（§3 失败处理）。 */
|
||||
private class FakeMailbox(var fail: Boolean = false) : CminmsgInboxRepository {
|
||||
val marked = linkedSetOf<Long>()
|
||||
override fun insertRaw(rawXml: String): Long = 1L
|
||||
override fun rawOf(msgId: Long): String? = null
|
||||
override fun readRange(fromExclusive: Long, limit: Int): List<MailboxRow> = emptyList()
|
||||
override fun maxId(): Long? = null
|
||||
override fun markProcessedIfUnmarked(msgId: Long, value: String): Boolean {
|
||||
if (fail) throw IllegalStateException("mysql-down")
|
||||
return marked.add(msgId)
|
||||
}
|
||||
}
|
||||
|
||||
private fun service(proc: StubProcState, mailbox: CminmsgInboxRepository, now: Instant = t0) =
|
||||
BackfillService(proc, mailbox, MailboxProps(), props, Clock.fixed(now, ZoneOffset.UTC))
|
||||
|
||||
/** 终态 + 回填意图（固定时刻，避免依赖真实时钟）。 */
|
||||
private fun succeeded(proc: StubProcState, id: Long) {
|
||||
proc.insertIfAbsent(id, t0)
|
||||
proc.markTerminal(id, ProcStatus.SUCCEEDED, now = t0)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `terminal message is marked immediately and the intent is cleared`() {
|
||||
val proc = StubProcState()
|
||||
val inbox = StubInbox().apply { clear() }
|
||||
val id = inbox.insertRaw("<MSG/>")
|
||||
succeeded(proc, id)
|
||||
|
||||
service(proc, inbox).attempt(id)
|
||||
|
||||
assertEquals("PROCESSED", inbox.markOf(id))
|
||||
val row = proc.find(id)!!
|
||||
assertNotNull(row.backfillAt)
|
||||
assertNull(row.backfillNextAt)
|
||||
assertNull(row.backfillError)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `marking is monotonic - an existing value is never overwritten`() {
|
||||
val inbox = StubInbox().apply { clear() }
|
||||
val id = inbox.insertRaw("<MSG/>")
|
||||
|
||||
assertTrue(inbox.markProcessedIfUnmarked(id, "PROCESSED"))
|
||||
assertFalse(inbox.markProcessedIfUnmarked(id, "OTHER")) // §11 只把空标写为已处理
|
||||
assertEquals("PROCESSED", inbox.markOf(id))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `already marked rows are idempotent and never recorded as failures`() {
|
||||
val proc = StubProcState()
|
||||
val inbox = StubInbox().apply { clear() }
|
||||
val id = inbox.insertRaw("<MSG/>")
|
||||
succeeded(proc, id)
|
||||
|
||||
val backfill = service(proc, inbox)
|
||||
backfill.attempt(id)
|
||||
backfill.attempt(id) // 重复执行无副作用（§5.2 补写只针对空标记）
|
||||
|
||||
assertNull(proc.find(id)!!.backfillError)
|
||||
assertEquals(0, proc.find(id)!!.backfillAttempts)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `failed attempt records backoff and never touches the terminal state`() {
|
||||
val proc = StubProcState()
|
||||
val mailbox = FakeMailbox(fail = true)
|
||||
succeeded(proc, 901L)
|
||||
|
||||
service(proc, mailbox).attempt(901L)
|
||||
|
||||
val row = proc.find(901L)!!
|
||||
assertEquals(ProcStatus.SUCCEEDED, row.state) // §11 终态不可逆
|
||||
assertEquals(1, row.backfillAttempts)
|
||||
assertEquals("mysql-down", row.backfillError)
|
||||
assertEquals(t0.plus(Duration.ofSeconds(30)), row.backfillNextAt)
|
||||
assertNull(row.backfillAt)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `sweep retries due rows and completes once the mailbox recovers`() {
|
||||
val proc = StubProcState()
|
||||
val mailbox = FakeMailbox(fail = true)
|
||||
succeeded(proc, 901L)
|
||||
val backfill = service(proc, mailbox)
|
||||
|
||||
assertEquals(1, backfill.sweep(t0)) // 到期 → 失败 → 退避
|
||||
assertEquals(0, backfill.sweep(t0.plusSeconds(29))) // 未到期
|
||||
assertEquals(1, backfill.sweep(t0.plusSeconds(30))) // 到期再试 → 仍失败（重启后同样收敛）
|
||||
|
||||
mailbox.fail = false
|
||||
assertEquals(1, backfill.sweep(t0.plusSeconds(90)))
|
||||
|
||||
assertTrue(901L in mailbox.marked)
|
||||
assertNotNull(proc.find(901L)!!.backfillAt)
|
||||
}
|
||||
|
||||
/** §5.2：超期期限 R 覆盖退避，保证库方清除前提「边界内无未标记行」在有限时间内成立。 */
|
||||
@Test
|
||||
fun `overdue rows bypass the retry backoff`() {
|
||||
val proc = StubProcState()
|
||||
val inbox = StubInbox().apply { clear() }
|
||||
val id = inbox.insertRaw("<MSG/>")
|
||||
val old = t0.minus(props.pipeline.overdueBackfill).minusSeconds(60)
|
||||
proc.insertIfAbsent(id, old) // 接收时间早于 R
|
||||
proc.markTerminal(id, ProcStatus.SUCCEEDED, now = t0)
|
||||
proc.recordBackfillFailure(id, "mysql-down", 5, t0.plus(Duration.ofMinutes(15)), t0) // 退避推到很远之后
|
||||
|
||||
assertEquals(1, service(proc, inbox).sweep(t0))
|
||||
|
||||
assertNotNull(proc.find(id)!!.backfillAt)
|
||||
}
|
||||
|
||||
/** §5.2：中间态（PENDING / FAILED）不适用超期补写——处理未完成时不打标。 */
|
||||
@Test
|
||||
fun `mid states are never marked even when far past the deadline`() {
|
||||
val proc = StubProcState()
|
||||
val inbox = StubInbox().apply { clear() }
|
||||
val pending = inbox.insertRaw("<MSG/>")
|
||||
val failed = inbox.insertRaw("<MSG/>")
|
||||
val old = t0.minus(props.pipeline.overdueBackfill).minusSeconds(60)
|
||||
proc.insertIfAbsent(pending, old)
|
||||
proc.insertIfAbsent(failed, old)
|
||||
proc.update(failed, ProcStatus.FAILED, errorClass = ErrorClass.INFRA)
|
||||
|
||||
assertEquals(0, service(proc, inbox).sweep(t0))
|
||||
|
||||
assertFalse(inbox.isMarked(pending))
|
||||
assertFalse(inbox.isMarked(failed))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `backoff delay doubles per attempt and caps at fifteen minutes`() {
|
||||
assertEquals(Duration.ofSeconds(30), BackfillService.backoffDelayFor(1))
|
||||
assertEquals(Duration.ofMinutes(2), BackfillService.backoffDelayFor(3))
|
||||
assertEquals(Duration.ofMinutes(15), BackfillService.backoffDelayFor(10))
|
||||
assertEquals(Duration.ofMinutes(15), BackfillService.backoffDelayFor(50))
|
||||
}
|
||||
}
|
||||
@@ -13,12 +13,13 @@ import com.gzzn.omms.msgexchange.domain.ProcStatus
|
||||
import com.gzzn.omms.msgexchange.domain.Targets
|
||||
import com.gzzn.omms.msgexchange.domain.flight.FlightState
|
||||
import com.gzzn.omms.msgexchange.infra.persistence.PersistOutcome
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubBackfillTodo
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubProcState
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubFlightState
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubMsgEvents
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubPipelineLock
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubPipelineTx
|
||||
import org.junit.jupiter.api.Assertions.assertEquals
|
||||
import org.junit.jupiter.api.Assertions.assertNotNull
|
||||
import org.junit.jupiter.api.Assertions.assertTrue
|
||||
import org.junit.jupiter.api.Test
|
||||
import java.time.LocalDate
|
||||
@@ -59,9 +60,10 @@ class FdelAndAdftProcessorTest {
|
||||
fun `active flight deletion publishes single tombstone and keeps details`() {
|
||||
val f = flights()
|
||||
val events = StubMsgEvents()
|
||||
val todo = StubBackfillTodo()
|
||||
val proc = StubProcState()
|
||||
proc.insertIfAbsent(msgId, null)
|
||||
|
||||
val result = FdelProcessor(StubPipelineTx(), StubPipelineLock(), f, events, todo, ObjectMapper())
|
||||
val result = FdelProcessor(StubPipelineTx(), StubPipelineLock(), f, events, proc, ObjectMapper())
|
||||
.apply(head(), msg(), FlopPayload("121"))
|
||||
|
||||
assertEquals(ApplyResult.Succeeded, result)
|
||||
@@ -71,14 +73,17 @@ class FdelAndAdftProcessorTest {
|
||||
assertEquals(1, tombstones.size)
|
||||
assertEquals(Targets.KAFKA_SCHD, tombstones.single().target)
|
||||
assertTrue(tombstones.single().payloadJson.contains("\"deleted\":true"))
|
||||
assertEquals(1, todo.count())
|
||||
// 终态 + 回填意图由处理器在自己的事务内落库（message-lifecycle §2/§4）
|
||||
val row = proc.find(msgId)!!
|
||||
assertEquals(ProcStatus.SUCCEEDED, row.state)
|
||||
assertNotNull(row.backfillNextAt)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `repeated FDEL is idempotent without version bump or duplicate event`() {
|
||||
val f = flights()
|
||||
val events = StubMsgEvents()
|
||||
val proc = FdelProcessor(StubPipelineTx(), StubPipelineLock(), f, events, null, ObjectMapper())
|
||||
val proc = FdelProcessor(StubPipelineTx(), StubPipelineLock(), f, events, StubProcState(), ObjectMapper())
|
||||
|
||||
proc.apply(head(), msg(), FlopPayload("121"))
|
||||
val version = f.findMainRow("121")!!.stateVersion
|
||||
@@ -94,7 +99,7 @@ class FdelAndAdftProcessorTest {
|
||||
val f = StubFlightState()
|
||||
val events = StubMsgEvents()
|
||||
|
||||
val result = FdelProcessor(StubPipelineTx(), StubPipelineLock(), f, events, null, ObjectMapper())
|
||||
val result = FdelProcessor(StubPipelineTx(), StubPipelineLock(), f, events, StubProcState(), ObjectMapper())
|
||||
.apply(head(), msg(), FlopPayload("999"))
|
||||
|
||||
assertEquals(ApplyResult.Succeeded, result) // §3.3：迟到/不存在幂等成功
|
||||
@@ -107,7 +112,7 @@ class FdelAndAdftProcessorTest {
|
||||
f.markDeleted("121", msgId = 2, now = java.time.Instant.now())
|
||||
val events = StubMsgEvents()
|
||||
val adft = AdftProcessor(
|
||||
StubPipelineTx(), StubPipelineLock(), f, events, null,
|
||||
StubPipelineTx(), StubPipelineLock(), f, events, StubProcState(),
|
||||
OperationDayProps().apply { zone = "Asia/Shanghai" }, ObjectMapper(),
|
||||
)
|
||||
val record = com.gzzn.omms.msgexchange.domain.flight.ScheduleRecord(
|
||||
@@ -128,7 +133,7 @@ class FdelAndAdftProcessorTest {
|
||||
val f = StubFlightState()
|
||||
val events = StubMsgEvents()
|
||||
val adft = AdftProcessor(
|
||||
StubPipelineTx(), StubPipelineLock(), f, events, null,
|
||||
StubPipelineTx(), StubPipelineLock(), f, events, StubProcState(),
|
||||
OperationDayProps().apply { zone = "Asia/Shanghai" }, ObjectMapper(),
|
||||
)
|
||||
|
||||
@@ -154,7 +159,7 @@ class FdelAndAdftProcessorTest {
|
||||
msgId = 1, now = java.time.Instant.now(),
|
||||
)
|
||||
val adft = AdftProcessor(
|
||||
StubPipelineTx(), StubPipelineLock(), f, StubMsgEvents(), null,
|
||||
StubPipelineTx(), StubPipelineLock(), f, StubMsgEvents(), StubProcState(),
|
||||
OperationDayProps().apply { zone = "Asia/Shanghai" }, ObjectMapper(),
|
||||
)
|
||||
|
||||
|
||||
@@ -12,7 +12,6 @@ import com.gzzn.omms.msgexchange.domain.SnapshotFlag
|
||||
import com.gzzn.omms.msgexchange.domain.SnapshotResult
|
||||
import com.gzzn.omms.msgexchange.domain.Targets
|
||||
import com.gzzn.omms.msgexchange.domain.flight.FlightState
|
||||
import com.gzzn.omms.msgexchange.infra.persistence.BackfillTodoRepository
|
||||
import com.gzzn.omms.msgexchange.infra.persistence.FlightStateRepository
|
||||
import com.gzzn.omms.msgexchange.infra.persistence.MsgEventRepository
|
||||
import com.gzzn.omms.msgexchange.infra.persistence.PersistOutcome
|
||||
@@ -24,8 +23,8 @@ import com.gzzn.omms.msgexchange.infra.stub.StubFlightState
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubMsgEvents
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubProcState
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubSnapshotLog
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubBackfillTodo
|
||||
import org.junit.jupiter.api.Assertions.assertEquals
|
||||
import org.junit.jupiter.api.Assertions.assertNotNull
|
||||
import org.junit.jupiter.api.Assertions.assertNull
|
||||
import org.junit.jupiter.api.Assertions.assertTrue
|
||||
import org.junit.jupiter.api.Test
|
||||
@@ -70,7 +69,6 @@ class ScheduleProcessorTest {
|
||||
flights: StubFlightState = StubFlightState(),
|
||||
events: StubMsgEvents = StubMsgEvents(),
|
||||
log: StubSnapshotLog = StubSnapshotLog(),
|
||||
todo: StubBackfillTodo? = StubBackfillTodo(),
|
||||
tx: PipelineTransactionManager? = null,
|
||||
): ScheduleProcessor {
|
||||
val txRunner = tx ?: object : PipelineTransactionManager {
|
||||
@@ -83,20 +81,20 @@ class ScheduleProcessorTest {
|
||||
flightState = flights,
|
||||
msgEvents = events,
|
||||
snapshotLog = log,
|
||||
backfillTodo = todo,
|
||||
operationDayProps = OperationDayProps().apply { zone = "Asia/Shanghai"; cutoffHour = 0 },
|
||||
mapper = ObjectMapper(),
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `happy path persists snapshot with version bump, events, snap log and backfill todo`() {
|
||||
fun `happy path persists snapshot, events, terminal state and backfill intent in one transaction`() {
|
||||
val proc = StubProcState()
|
||||
proc.insertIfAbsent(msgId, null)
|
||||
val flights = StubFlightState()
|
||||
val events = StubMsgEvents()
|
||||
val log = StubSnapshotLog()
|
||||
val todo = StubBackfillTodo()
|
||||
|
||||
val result = processor(flights = flights, events = events, log = log, todo = todo)
|
||||
val result = processor(proc = proc, flights = flights, events = events, log = log)
|
||||
.applyScheduleRecords(head(), message(makeBody("121" to "15DEC261723")))
|
||||
|
||||
assertEquals(ApplyResult.Succeeded, result)
|
||||
@@ -108,13 +106,17 @@ class ScheduleProcessorTest {
|
||||
assertEquals(1, log.entries.size)
|
||||
assertEquals(SnapshotResult.COMMITTED, log.entries.single().result)
|
||||
assertEquals(1, log.entries.single().upserted)
|
||||
assertEquals(1, todo.count()) // design.md §3.3/§6.1 事务内预登记
|
||||
// message-lifecycle §2/§4：终态与回填意图随业务写入在同一事务提交（不依赖主泵补写）
|
||||
val row = proc.find(msgId)!!
|
||||
assertEquals(ProcStatus.SUCCEEDED, row.state)
|
||||
assertNotNull(row.backfillNextAt)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `replay of succeeded message records idempotent success without writes`() {
|
||||
val proc = StubProcState()
|
||||
proc.insert(msgId, ProcStatus.SUCCEEDED)
|
||||
proc.insertIfAbsent(msgId, null)
|
||||
proc.markTerminal(msgId, ProcStatus.SUCCEEDED)
|
||||
val flights = StubFlightState()
|
||||
val log = StubSnapshotLog()
|
||||
|
||||
@@ -190,17 +192,16 @@ class ScheduleProcessorTest {
|
||||
val proc = StubProcState()
|
||||
val flights = StubFlightState()
|
||||
val log = StubSnapshotLog()
|
||||
val todo = StubBackfillTodo()
|
||||
proc.insert(msgId)
|
||||
val p = processor(proc = proc, flights = flights, log = log, todo = todo, tx = TxRunner { true })
|
||||
proc.insertIfAbsent(msgId, null)
|
||||
val p = processor(proc = proc, flights = flights, log = log, tx = TxRunner { true })
|
||||
|
||||
// design.md §2.3：数据库/内部故障 → 异常上抛，MessageProcessor 记 FAILED(INFRA) 退避；不写任何终态
|
||||
org.junit.jupiter.api.Assertions.assertThrows(IllegalStateException::class.java) {
|
||||
p.applyScheduleRecords(head(), message(makeBody("121" to "15DEC261723")))
|
||||
}
|
||||
assertNull(flights.findMainRow("121"))
|
||||
assertEquals(0, todo.count())
|
||||
assertEquals(0, log.entries.size)
|
||||
assertEquals(ProcStatus.PENDING, proc.find(msgId)!!.state)
|
||||
assertNull(proc.find(msgId)!!.backfillNextAt) // 事务回滚：终态与回填意图都不落库
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,6 +4,8 @@
|
||||
# U07：stubs=true 提供内存仓储使全链路（Controller→Inbox→Pump→Dispatcher）可装配；
|
||||
# autostart=false：测试内不自动拉起后台循环（避免泄漏线程），按需手动 tick。
|
||||
# ACM2-12：main 的 datasources.default（自有 PG）默认 enabled=false，测试以 H2 内存替代并显式启用。
|
||||
# V2：BackfillService 注入 MailboxProps，其 shared-mysql.* 占位符必须在测试环境可解析
|
||||
# （信箱不启用：enabled=false，仅需占位符取值）。
|
||||
msgx:
|
||||
register-eureka: false
|
||||
stubs: true
|
||||
@@ -18,6 +20,15 @@ datasources:
|
||||
password: ""
|
||||
driver-class-name: org.h2.Driver
|
||||
|
||||
mailbox:
|
||||
processed-value: PROCESSED
|
||||
shared-mysql:
|
||||
enabled: false
|
||||
url: jdbc:mysql://127.0.0.1:3306/cdairport
|
||||
username: msgx_test
|
||||
password: msgx_test
|
||||
driver-class-name: com.mysql.cj.jdbc.Driver
|
||||
|
||||
flyway:
|
||||
datasources:
|
||||
default:
|
||||
|
||||
Reference in New Issue
Block a user