refactor(ingress): 信箱边界层契约重构——发现与处理标记解耦、回填事实并入 PROC_STATE
收报扫描不再以 DATE_PROCESSED 为谓词：终态而未回填的行（解码失败死信等）会永久占据 有限批次，累积到 claim-batch 后收报整体停摆（US-01 条目 3 / message-lifecycle §5.3）。 ingress/InboxPoller.kt：按 ID 区间升序有界读取（ID > W），水位落 INBOX_CURSOR 并与入队 同一 PG 事务推进（中断后重扫补建）；遇空洞即停，空洞超过 pipeline.max-commit-delay 判定 为永久并放行——否则水位永久停摆于一次自增回滚留下的空位。删除 InboxEnqueue（改由 insertIfAbsent 幂等入队）与 poller 内的回填扫描（消除 ingress→jobs 反向依赖）。 infra/persistence：端口按事实重画为 readRange/maxId/markProcessedIfUnmarked，标记 UPDATE 带 DATE_PROCESSED IS NULL 守卫，只把空标写为已处理（§11 单调，重复执行无副作用）。 回填事实并入 PROC_STATE（RECEIVED_AT/BACKFILL_AT/NEXT_AT/ATTEMPTS/ERROR），BACKFILL_TODO 随 V2 迁移下线；终态与回填意图是同一条 UPDATE，由处理器在自己的业务事务内落库， message-lifecycle §4 登记的两个崩溃窗口（提交后回填前崩溃、待办二次落账失败）不再是缺口。 processing/BackfillService.kt（取代 BackfillSweepJob）：终态提交后立即尝试一次，失败按 30s→15min 指数退避重试；扫描条件「终态 + 未确认标记 +（已到期 或 接收时间早于 NOW − R）」 使 §5.2 的超期期限 R 覆盖退避，中间态永不补写。死信同样可补写——回填只需消息 ID， 不再依赖 META。Pump 改用可注入 Clock。 infra/health：InboxLifecycleHealthIndicator 输出积压条数、最老未处理信龄、未回填终态数与 水位滞后（OPS-2 / §5.3 验收）。预计消化时长需吞吐采样，留待接入指标注册表时补。 配置：pipeline.max-commit-delay / overdue-backfill / backfill-batch、mailbox.processed-value （Q2/Q6/Q7 未书面确认前取保守初值，不得为提速下调）。 不变量回归测试：死信不阻断后续发现、水位遇空洞即停与老化放行、终态+意图同事务、 超期 R 覆盖退避、中间态不补写、标记单调；InboxLifecycleJdbcSqlTest 以 H2 的 PostgreSQL 兼容模式直连验证上述 SQL 语义（不依赖 docker）。libs.h2 由 testRuntimeOnly 提为 testImplementation 以支持该用例。 验证：gradle clean test --offline → 78 tests / 0 failures / 1 skipped （PG Testcontainers 集成用例在本机无 docker 时按既有约定 assumeTrue 跳过）。
This commit is contained in:
windyboy
@@ -1,47 +1,120 @@
|
||||
package com.gzzn.omms.msgexchange.ingress
|
||||
|
||||
import com.gzzn.omms.msgexchange.config.PipelineProps
|
||||
import com.gzzn.omms.msgexchange.domain.ErrorClass
|
||||
import com.gzzn.omms.msgexchange.domain.ProcStatus
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubInbox
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubInboxCursor
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubPipelineTx
|
||||
import com.gzzn.omms.msgexchange.infra.stub.StubProcState
|
||||
import com.gzzn.omms.msgexchange.processing.Pump
|
||||
import io.micronaut.test.extensions.junit5.annotation.MicronautTest
|
||||
import jakarta.inject.Inject
|
||||
import org.junit.jupiter.api.Assertions.assertEquals
|
||||
import org.junit.jupiter.api.Assertions.assertFalse
|
||||
import org.junit.jupiter.api.Assertions.assertNotNull
|
||||
import org.junit.jupiter.api.Assertions.assertNull
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import java.time.Instant
|
||||
|
||||
/** 主路径：JDBC 轮询语义（stub 下 InboxPoller + 外部写信箱模拟）。 */
|
||||
@MicronautTest
|
||||
/**
|
||||
* 收报发现权不变量（docs/message-lifecycle.md §5.1/§5.3 + architecture.md §5 严格 FIFO）：
|
||||
* - 扫描按 ID 区间，**不受处理标记影响**：终态而未回填的行不得占据批次、不得阻断新信发现；
|
||||
* - 水位只随成功入队推进，且与入队同事务（中断后由重扫补建）；
|
||||
* - 遇空洞即停（较小 ID 未入队时不得被后续消息越过）；空洞老化后放行（水位不得永久停摆）；
|
||||
* - 收报层不写处理标记。
|
||||
*/
|
||||
class InboxPollerTest {
|
||||
|
||||
@Inject lateinit var poller: InboxPoller
|
||||
@Inject lateinit var pump: Pump
|
||||
@Inject lateinit var stubInbox: StubInbox
|
||||
@Inject lateinit var stubProc: StubProcState
|
||||
private val t0: Instant = Instant.parse("2026-09-08T03:00:00Z")
|
||||
private val props = PipelineProps()
|
||||
|
||||
private lateinit var inbox: StubInbox
|
||||
private lateinit var proc: StubProcState
|
||||
private lateinit var cursor: StubInboxCursor
|
||||
private lateinit var poller: InboxPoller
|
||||
|
||||
@BeforeEach
|
||||
fun clean() {
|
||||
stubInbox.clear()
|
||||
stubProc.clear()
|
||||
fun setUp() {
|
||||
inbox = StubInbox().apply { clear() }
|
||||
proc = StubProcState().apply { clear() }
|
||||
cursor = StubInboxCursor().apply { clear() }
|
||||
poller = InboxPoller(inbox, proc, cursor, StubPipelineTx(), props)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `poller enqueues externally written mailbox rows`() {
|
||||
val id = stubInbox.simulateExternalWrite("<MSG/>")
|
||||
assertEquals(1, poller.pollOnce())
|
||||
assertNotNull(stubProc.snapshotOf(id))
|
||||
assertEquals(ProcStatus.PENDING, stubProc.snapshotOf(id)!!.state)
|
||||
fun `external rows are enqueued in id order and advance the watermark without marking the mailbox`() {
|
||||
val first = inbox.simulateExternalWrite("<MSG/>")
|
||||
val second = inbox.simulateExternalWrite("<MSG/>")
|
||||
|
||||
assertEquals(2, poller.pollOnce(t0))
|
||||
|
||||
assertEquals(ProcStatus.PENDING, proc.find(first)!!.state)
|
||||
assertEquals(ProcStatus.PENDING, proc.find(second)!!.state)
|
||||
assertEquals(second, cursor.cursor.committedUpTo)
|
||||
assertNull(cursor.cursor.holeSince)
|
||||
// §5.3：消化阶段只写自有 PG，不触碰信箱标记
|
||||
assertFalse(inbox.isMarked(first))
|
||||
assertEquals(0, poller.pollOnce(t0)) // 重复扫描幂等
|
||||
}
|
||||
|
||||
/**
|
||||
* 回归（US-01 条目 3 / §5.3「积压挡批」）：终态且永不回填的行（解码失败死信等）曾占满
|
||||
* 有限批次使收报整体停摆——发现必须与处理标记彻底解耦。
|
||||
*/
|
||||
@Test
|
||||
fun `terminal rows without a mailbox mark do not block discovery of later messages`() {
|
||||
props.pipeline.claimBatch = 3
|
||||
val dead = (1..3).map { inbox.simulateExternalWrite("<MSG/>") }
|
||||
assertEquals(3, poller.pollOnce(t0))
|
||||
dead.forEach {
|
||||
proc.markTerminal(it, ProcStatus.DEAD, errorClass = ErrorClass.MALFORMED, lastError = "raw-missing")
|
||||
}
|
||||
|
||||
val fresh = inbox.simulateExternalWrite("<MSG/>")
|
||||
|
||||
assertEquals(1, poller.pollOnce(t0))
|
||||
assertEquals(ProcStatus.PENDING, proc.find(fresh)!!.state)
|
||||
assertEquals(fresh, cursor.cursor.committedUpTo)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `poller is idempotent for already enqueued rows`() {
|
||||
val id = stubInbox.simulateExternalWrite("<MSG/>")
|
||||
poller.pollOnce()
|
||||
assertEquals(0, poller.pollOnce())
|
||||
pump.tick()
|
||||
assertEquals(ProcStatus.DEAD, stubProc.snapshotOf(id)!!.state)
|
||||
assertEquals(ErrorClass.MALFORMED, stubProc.snapshotOf(id)!!.errorClass)
|
||||
fun `watermark stops at a hole so later ids cannot overtake a missing smaller id`() {
|
||||
val first = inbox.simulateExternalWrite("<MSG/>")
|
||||
val hole = inbox.simulateExternalWrite("<MSG/>")
|
||||
val afterHole = inbox.simulateExternalWrite("<MSG/>")
|
||||
inbox.removeRow(hole)
|
||||
|
||||
assertEquals(1, poller.pollOnce(t0))
|
||||
|
||||
assertEquals(first, cursor.cursor.committedUpTo)
|
||||
assertNotNull(cursor.cursor.holeSince)
|
||||
assertNull(proc.find(afterHole)) // 不得越过空洞入队（FIFO）
|
||||
assertEquals(0, poller.pollOnce(t0.plusSeconds(60))) // 宽限期内水位不推进
|
||||
assertEquals(first, cursor.cursor.committedUpTo)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `an aged hole is released and later ids resume enqueuing`() {
|
||||
val hole = inbox.simulateExternalWrite("<MSG/>")
|
||||
val afterHole = inbox.simulateExternalWrite("<MSG/>")
|
||||
inbox.removeRow(hole)
|
||||
poller.pollOnce(t0) // 记录空洞观测时刻
|
||||
|
||||
val agedOut = t0.plus(props.pipeline.maxCommitDelay)
|
||||
assertEquals(0, poller.pollOnce(agedOut)) // 空洞判永久：推进水位但不越过入队
|
||||
|
||||
assertNull(cursor.cursor.holeSince)
|
||||
assertEquals(afterHole - 1, cursor.cursor.committedUpTo)
|
||||
assertEquals(1, poller.pollOnce(agedOut)) // 下一轮恢复发现
|
||||
assertEquals(afterHole, cursor.cursor.committedUpTo)
|
||||
assertNotNull(proc.find(afterHole))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `compat http path and poller do not double enqueue the same message`() {
|
||||
val receipt = InboxService(inbox, proc).accept("<MSG/>", t0)
|
||||
|
||||
assertEquals(0, poller.pollOnce(t0))
|
||||
assertEquals(receipt.msgId, cursor.cursor.committedUpTo) // 已在 PG：读取进度照常推进
|
||||
assertNotNull(proc.find(receipt.msgId))
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user