Files
ciimsproxy/plans/2026-07-08-ciimsproxy-codebase-analysis.md
T
zhiqiang feng ce4c2b088f fix: comprehensive bug fixes, hardening, and test coverage
Bug fixes:
- Fix nil-pointer panic in sendMessage SOAP fault handler (used err.Error() on nil)
- Fix missing return after SOAP fault response (caused fall-through to 200 OK)
- Fix timeout=0 shadowing: removed package-level constant, use configured value
- Fix receiveMessage not checking SOAP faults from CIIMS

Codec hardening:
- Replace fragile regex parsing with encoding/xml.Decoder for namespace-agnostic XML
- XML-escape all user inputs (user, pass, event) — not just message body
- Remove magic-number-based string slicing (split, GetErrMsg)

Transport improvements:
- Extract Client type with connection pooling (reuse http.Client across requests)
- Add CIIMSResponse domain type with IsFault(), ErrorMessage(), Messages()
- Add HTTPStatusError and ResponseTooLargeError typed errors
- Replace deprecated ioutil.ReadAll with io.ReadAll
- Add response body size limit (64 MiB)

Security hardening:
- Add MaxBytesReader (1MB request body limit) on both endpoints
- Add URL allowlist via CIIMS_ALLOWED_SERVERS env var
- Add normalizeBaseURL with scheme/host/query validation
- Add count bounds validation (1-1000) on /receive
- Add server-level timeouts (ReadHeaderTimeout, ReadTimeout, IdleTimeout)

Configuration:
- Introduce Config struct to replace package-level globals
- Add loadConfig() with full validation and error propagation
- Add getIntEnv() with positive-value enforcement

Test coverage (75 tests, 35 new):
- Phase 1: 8 internal tests (nil receiver, error types, SOAP+HTTP500, malformed XML)
- Phase 2: 19 pure function tests (normalizeBaseURL, parseAllowedServers, getIntEnv)
- Phase 3: 16 handler tests (send/receive success, SOAP fault, network error, HTTP 500,
  body too large, invalid JSON, missing fields, URL allowlist)
- Phase 4: 8 config/router tests (loadConfig, newServer, newRouter)

Toolchain:
- Upgrade Go 1.13 → 1.22, gin 1.6.3 → 1.10.0, testify 1.5.1 → 1.10.0

Cleanup:
- Remove dead code (unused post() function, commented-out defaults)
- Replace println with structured logging
- Add .gitignore
- Rewrite README with API docs, env vars, security considerations
2026-07-08 16:23:00 +08:00

400 lines
19 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# CIIMS Proxy — Codebase Analysis
> Generated 2026-07-08 · Go 1.22 · 4 source files + 2 test files
---
## 1. Source Code Analysis
### 1.1 Package Structure & Module Layout
The codebase is split across two packages:
| Package | Files | Lines | Role |
|---|---|---|---|
| `main` (`cmd/main/`) | `main.go` | 145 | Entry point, HTTP routing, configuration, request validation |
| `internal` | `codec.go` (139), `http.go` (43) | 182 | SOAP envelope construction, response parsing, HTTP transport |
This is a **minimal two-layer split**: presentation (HTTP handlers) in `main`, and domain logic (SOAP encoding, transport) in `internal`. The separation is functional but thin—the `internal` package has no types of its own; everything is functions operating on raw strings.
### 1.2 Entry Point: `cmd/main/main.go`
**Configuration** (`cmd/main/main.go:14-22`):
| Constant | Value | Purpose |
|---|---|---|
| `servicePrefix` | `"/services/ExchangeService"` | SOAP endpoint path suffix appended to every CIIMS URL |
| `defaultTimeout` | `240` | Fallback timeout in seconds when `CIIMS_TIMEOUT` is unset |
| `maxMsgLen` | `1048576` (1 MB) | Request body size cap enforced via `http.MaxBytesReader` |
| `maxCount` | `1000` | Maximum value accepted for the `count` field on `/receive` |
Two package-level variables hold runtime configuration:
| Variable | Source | Default |
|---|---|---|
| `defaultURL` | `CIIMS_SERVER` env | `""` |
| `timeout` | `CIIMS_TIMEOUT` env | `240` |
**Note:** `timeout` is still a package-level global—handlers are coupled to shared mutable state. A struct-based config injected via closure would be more idiomatic and testable.
**`main()`** (`cmd/main/main.go:24-51`):
- Reads three environment variables: `CIIMS_SERVER`, `PROXY_LISTEN`, `CIIMS_TIMEOUT`.
- `getIntEnv` returns `(int, error)`—the error is handled cleanly with `log.Errorf` + `os.Exit(1)`.
- If `CIIMS_TIMEOUT` is unset (`t == 0`), the package-level `timeout` retains its default of 240. If set and valid, it overrides.
- Three routes: `GET /ping`, `POST /send`, `POST /receive`.
- Uses `gin.Default()` which includes Logger and Recovery middleware.
**`sendMessage()`** (`cmd/main/main.go:54-89`):
- Body size limited via `http.MaxBytesReader` to 1 MB (`cmd/main/main.go:56`). Gin returns 413 if exceeded.
- Anonymous struct with `binding:"required"` tags on `User`, `Pass`, `Event`, `Msg`.
- Calls `internal.CreateSend()``internal.Send()``internal.GetErrMsg()`.
- On SOAP fault: logs the fault message, returns `500` with the fault text, and **returns** (no fall-through to 200 OK).
- On network error: logs the error, returns `500`.
- On success: returns `200 {"error":""}`.
**`receiveMessage()`** (`cmd/main/main.go:91-123`):
- Same body size limit.
- Additional validation: `count` must be 11000 (`cmd/main/main.go:108-111`).
- Calls `internal.CreateReceive()``internal.Send()``internal.GetMsgs()`.
- Returns `200 {"msgs": [...]}` on success.
- **Gap:** Does not check for SOAP faults in the response. If CIIMS returns a fault, the proxy returns 200 with an empty message list.
**`getURL()`** (`cmd/main/main.go:125-133`):
- If the request provides a `url`, uses it + `servicePrefix`; otherwise uses `defaultURL` + `servicePrefix`.
- This means the per-request `url` field can override the global CIIMS server—useful for multi-tenant or failover scenarios.
**`getIntEnv()`** (`cmd/main/main.go:135-144`):
- Returns `(int, error)`, caller handles the error.
- Returns `(0, nil)` when unset—caller interprets `0` as "use default."
### 1.3 SOAP Codec: `internal/codec.go`
**Template constants** (`internal/codec.go:12-88`):
- `sendtpl` (lines 12-53): A complete SOAP 1.1 envelope with `BHIA_CIIMS:AuthenticationToken` header and `ns1:send` body. Contains six `##placeholder##` tokens.
- `receivetpl` (lines 56-84): Similar envelope but with `ns1:receive` body and a single `##count##` token.
- Both templates have inconsistent indentation: `sendtpl` uses spaces, `receivetpl` uses tabs.
- The templates contain blank lines between every XML element—this bloats the wire payload unnecessarily.
**`CreateSend()`** (`internal/codec.go:91-101`):
- Six `strings.Replace` calls, one per placeholder.
- `user`, `pass`, `event` are escaped via `xmlEscape()`.
- `message` is escaped via `xml.Escape` directly into a `bytes.Buffer`.
- `priority` and `valXML` are formatted with `strconv.Itoa` and `strconv.FormatBool`—these are numeric/boolean so escaping is unnecessary.
- Returns the complete SOAP envelope as a string.
**`CreateReceive()`** (`internal/codec.go:104-109`):
- Three `strings.Replace` calls.
- `user` and `pass` escaped; `count` formatted with `strconv.Itoa`.
**`xmlEscape()`** (`internal/codec.go:112-116`):
- Thin wrapper around `xml.Escape` that returns a string instead of writing to a buffer.
- Allocates a new `bytes.Buffer` per call—three allocations per `CreateSend` invocation.
**`GetMsgs()`** (`internal/codec.go:118-128`):
- Compiles the `msgExp` regex at call time via `regexp.MustCompile`. This is wasteful—the regex is constant and should be compiled once at package init.
- Uses `FindAllStringSubmatch` with capture group 1 (the inner content).
- Applies `html.UnescapeString` to decode XML entities back to raw characters.
- The regex `<\w+:string>(.*?)</\w+:string>` is namespace-agnostic—it matches any prefix like `ns1`, `ns2`, etc.
**`GetErrMsg()`** (`internal/codec.go:131-138`):
- Same pattern: compiles regex at call time, uses capture group 1, applies `html.UnescapeString`.
- The regex `<errorMessage[^>]*>(.*?)</errorMessage>` handles attributes on the opening tag.
### 1.4 HTTP Transport: `internal/http.go`
**`postSim()`** (`internal/http.go:15-37`):
- Creates a new `http.Client` per call with a per-call timeout. This is inefficient—`http.Client` is designed to be reused (connection pooling). A single client should be created at startup.
- The timeout calculation `time.Duration(time.Duration(sec) * time.Second)` has a redundant double `time.Duration` cast.
- Sets three headers: `Content-Type: text/xml; charset=UTF-8`, `User-Agent` (masquerading as XFire/IE6 from 2005), and `SOAPAction: ""`.
- Returns `(err.Error(), err)` on failure—this duplicates the error message in both return values.
- Uses `ioutil.ReadAll` which is deprecated since Go 1.16 (now `io.ReadAll`).
**`Send()`** (`internal/http.go:40-42`):
- A one-line passthrough to `postSim`. This is a redundant abstraction layer—`postSim` could be renamed to `Send` directly.
---
## 2. Test Analysis
### 2.1 Test Structure
Two test files, both in `package internal`:
| File | Tests | Focus |
|---|---|---|
| `codec_test.go` | 5 tests | Unit tests for SOAP construction and response parsing |
| `handler_test.go` | 11 tests | Integration-style tests for HTTP transport + codec |
### 2.2 `codec_test.go` — Legacy Tests (unchanged)
| Test | What It Verifies | Approach |
|---|---|---|
| `TestSend` | `CreateSend` produces exact expected XML | Golden-file string comparison |
| `TestReceive` | `CreateReceive` produces exact expected XML | Golden-file string comparison |
| `TestGetMsg` | `GetMsgs` extracts 2 messages from `ReceiveResp` | Count + content assertion |
| `TestGetErrMsg` | `GetErrMsg` extracts fault text from `ErrMsg` | Exact string match |
| `TestSendOk` | `GetErrMsg` returns `""` for successful response | Empty string check |
**Assessment:** The golden-file tests are brittle—any whitespace change in the template breaks them. However, they serve as a strong regression safety net for the SOAP format.
### 2.3 `handler_test.go` — New Tests
| Test | What It Verifies | Approach |
|---|---|---|
| `TestSend_SOAPFault` | Full round-trip: mock CIIMS returns fault → `GetErrMsg` extracts it | `httptest.Server` |
| `TestSend_Success` | Mock CIIMS returns success → `GetErrMsg` returns `""` | `httptest.Server` |
| `TestSend_ReceiveMessages` | Mock CIIMS returns messages → `GetMsgs` extracts them | `httptest.Server` |
| `TestSend_NetworkTimeout` | Client timeout triggers when server never responds | Raw `net.Listener` |
| `TestSend_ServerError` | Connection refused produces error | Invalid port (127.0.0.1:1) |
| `TestGetMsgs_Empty` | Empty SOAP body returns zero messages | Direct call |
| `TestGetMsgs_DifferentNamespacePrefix` | `ns2:string` works (namespace-agnostic regex) | Direct call |
| `TestGetErrMsg_NoError` | Successful response returns `""` | Direct call |
| `TestGetErrMsg_Empty` | Empty SOAP body returns `""` | Direct call |
| `TestCreateSend_XMLEscapes` | XML metacharacters in user/pass are escaped | `assert.Contains` |
| `TestCreateReceive_XMLEscapes` | XML metacharacters in user/pass are escaped | `assert.Contains` |
### 2.4 Test Coverage Gaps
| Area | Covered? | Notes |
|---|---|---|
| SOAP send construction | Yes | Golden file + XML escape tests |
| SOAP receive construction | Yes | Golden file + XML escape tests |
| Message extraction | Yes | Happy path, empty, different namespace |
| Error extraction | Yes | Fault, success, empty |
| HTTP timeout | Yes | Raw listener approach |
| HTTP connection error | Yes | Invalid port |
| Handler-level (Gin endpoints) | **No** | No tests for `/send` or `/receive` HTTP handlers |
| Body size limit | **No** | Not tested |
| Count validation | **No** | Not tested |
| Malformed JSON | **No** | Not tested |
| Missing required fields | **No** | Not tested |
| SOAP fault in receive | **No** | Not tested (and not implemented) |
---
## 3. Abstraction Analysis
### 3.1 Current Abstraction Layers
```
┌─────────────────────────────────────────┐
│ HTTP Layer (cmd/main/main.go) │
│ - Routing (Gin) │
│ - JSON binding/validation │
│ - Body size limiting │
│ - Error → HTTP status mapping │
├─────────────────────────────────────────┤
│ Codec Layer (internal/codec.go) │
│ - JSON → SOAP XML (CreateSend/Receive) │
│ - SOAP XML → domain data (GetMsgs/ │
│ GetErrMsg) │
├─────────────────────────────────────────┤
│ Transport Layer (internal/http.go) │
│ - HTTP POST with timeout │
│ - SOAP header management │
└─────────────────────────────────────────┘
```
### 3.2 Abstraction Quality
**Strengths:**
- The three layers have clear responsibilities: routing, encoding, transport.
- The `internal` package hides SOAP complexity from the HTTP handlers.
- The handlers don't know about SOAP; the codec doesn't know about HTTP routing.
**Weaknesses:**
1. **Anemic domain model**: There are no types representing CIIMS messages. Everything is `string` in, `string` out. This means:
- No compile-time guarantees about message structure.
- No way to add methods or validation to message types.
- `CreateSend` has 6 positional parameters—easy to misorder.
2. **Leaky transport abstraction**: `Send()` returns `(string, error)` where the string is raw XML. The caller (`sendMessage`) then calls `GetErrMsg(resp)` to check for SOAP faults. The transport layer should either:
- Return a parsed response struct, or
- Have the codec layer wrap the transport call entirely.
3. **Redundant `Send` wrapper**: `Send()` is a one-line call to `postSim()`. This is unnecessary indirection.
4. **No configuration type**: `defaultURL` and `timeout` are package-level globals in `main`. There's no `Config` struct, making the code harder to test and reason about.
5. **Regex compiled at call time**: `GetMsgs` and `GetErrMsg` compile their regex patterns on every invocation. For a proxy that may handle many requests, this is wasteful. The patterns should be `var` declarations compiled at init time.
### 3.3 Suggested Abstraction Improvements
```go
// A typed request would prevent parameter ordering bugs:
type SendRequest struct {
User string
Pass string
Priority int
Event string
ValXML bool
Message string
}
// A typed response would encapsulate parsing:
type CIIMSResponse struct {
RawXML string
}
func (r *CIIMSResponse) IsFault() bool { ... }
func (r *CIIMSResponse) ErrorMessage() string { ... }
func (r *CIIMSResponse) Messages() []string { ... }
// A Config struct would eliminate globals:
type Config struct {
ServerURL string
Timeout time.Duration
Listen string
}
```
---
## 4. Logic Analysis
### 4.1 Request Flow: `POST /send`
```
Client POST /send {"user":"FIMS","pass":"x","event":"E1","msg":"<M/>"}
sendMessage()
├─ http.MaxBytesReader (1MB limit) [cmd/main/main.go:56]
├─ c.Bind(&message) [cmd/main/main.go:67]
│ └─ Gin validates binding:"required" fields
├─ internal.CreateSend(user,pass,pri,evt,val,msg)
│ │ [internal/codec.go:91-101]
│ ├─ xml.Escape(message) → &lt;M/&gt;
│ ├─ xmlEscape(user) → FIMS (no change)
│ ├─ xmlEscape(pass) → x (no change)
│ ├─ xmlEscape(event) → E1 (no change)
│ └─ strings.Replace × 6 → full SOAP envelope
├─ getURL(message.URL) [cmd/main/main.go:125-133]
│ └─ url + "/services/ExchangeService" or defaultURL + prefix
├─ internal.Send(url, msg, timeout) [internal/http.go:40-42]
│ └─ postSim(url, msg, timeout)
│ │ [internal/http.go:15-37]
│ ├─ http.Client{Timeout: timeout}
│ ├─ POST with SOAP headers
│ └─ ioutil.ReadAll → raw XML string
├─ internal.GetErrMsg(resp) [internal/codec.go:131-138]
│ ├─ regexp: <errorMessage[^>]*>(.*?)</errorMessage>
│ └─ html.UnescapeString(capture group)
├─ [if errMsg != ""] → 500 {"error": errMsg}
└─ [else] → 200 {"error": ""}
```
### 4.2 Request Flow: `POST /receive`
```
Client POST /receive {"user":"FIMS","pass":"x","count":5}
receiveMessage()
├─ http.MaxBytesReader (1MB limit) [cmd/main/main.go:93]
├─ c.Bind(&message) [cmd/main/main.go:101]
├─ count validation: 1 ≤ count ≤ 1000 [cmd/main/main.go:108-111]
├─ internal.CreateReceive(user, pass, count) [internal/codec.go:104-109]
├─ getURL(message.URL)
├─ internal.Send(url, msg, timeout)
└─ internal.GetMsgs(resp) [internal/codec.go:118-128]
├─ regexp: <\w+:string>(.*?)</\w+:string>
├─ FindAllStringSubmatch → capture groups
└─ html.UnescapeString each → []string
```
### 4.3 Error Handling Matrix
| Error Scenario | Detection | HTTP Status | Response Body | Logged? |
|---|---|---|---|---|
| Invalid JSON / missing required field | `c.Bind` returns error | 400 | `{"error":"..."}` | Yes |
| Body exceeds 1MB | `MaxBytesReader` triggers | 413 (Gin) | Gin default | No |
| `count` out of range | Manual check | 400 | `{"error":"count must be..."}` | No |
| Invalid `CIIMS_TIMEOUT` env | `getIntEnv` returns error | N/A (`os.Exit`) | N/A | Yes |
| Network error (timeout, DNS, refused) | `Send` returns error | 500 | `{"error":"..."}` | Yes |
| CIIMS SOAP fault (send) | `GetErrMsg` returns non-empty | 500 | `{"error":"<fault>"}` | Yes |
| CIIMS SOAP fault (receive) | Not checked | 200 | `{"msgs":[]}` (empty) | **No** |
| HTTP non-200 from CIIMS | Not checked | 200 | Depends on body | **No** |
**Two gaps remain:**
1. **`receiveMessage` does not check for SOAP faults**—if CIIMS returns a fault for a receive request, the proxy returns 200 with an empty message list instead of an error.
2. **HTTP status codes from CIIMS are ignored**`postSim` reads the body regardless of status code. A 500 from CIIMS would be treated as a successful response.
### 4.4 Concurrency & Thread Safety
- `defaultURL` and `timeout` are set in `main()` before `r.Run()` starts the server. Since Gin handlers read these values after they're set and they're never written again, there is **no data race** in practice.
- However, if someone added a reload endpoint that modified these variables at runtime, it would be racy. A `sync/atomic` or `sync.RWMutex` would be needed.
- `http.Client` is created per request in `postSim`—this is safe but inefficient (no connection pooling).
### 4.5 Memory & Allocation Profile
Per `POST /send` request, the following allocations occur:
1. Gin request parsing (framework overhead)
2. `bytes.Buffer` for XML-escaped message (`CreateSend`)
3. Three `bytes.Buffer` allocations for `xmlEscape(user)`, `xmlEscape(pass)`, `xmlEscape(event)`
4. Six `strings.Replace` calls, each allocating a new string (the SOAP template is ~1.5 KB)
5. `http.NewRequest` + `strings.NewReader(msg)` (another copy)
6. `ioutil.ReadAll` for the response body
7. `regexp.MustCompile` + match allocations in `GetErrMsg`
For a low-throughput proxy this is fine. For high throughput, the repeated regex compilation and multiple string copies per request would benefit from optimization (pre-compiled regex, `strings.Builder`, reusable buffers).
---
## 5. Summary Assessment
| Dimension | Rating | Key Points |
|---|---|---|
| **Correctness** | Good | Critical bugs (nil panic, missing return, timeout=0) are fixed. Two gaps remain: receive fault detection and HTTP status checking. |
| **Security** | Adequate | Body size limits, XML escaping, input validation added. No caller authentication—delegated to reverse proxy. |
| **Testability** | Fair | 16 tests cover codec and transport well. Handler-level (Gin endpoint) tests are missing. |
| **Abstraction** | Basic | Three-layer split is clean but uses primitive types exclusively. No domain types, no Config struct. |
| **Performance** | Adequate | Per-request regex compilation and multiple string copies are suboptimal but acceptable for low-volume use. |
| **Maintainability** | Fair | Dead code removed, dependencies upgraded. Template-based SOAP and regex parsing are inherently fragile. |
---
## 6. Remaining Recommendations
### High Priority
1. **Add SOAP fault detection to `receiveMessage`** — mirror the `GetErrMsg` check from `sendMessage` to avoid silently returning empty results on CIIMS errors.
2. **Check HTTP status codes from CIIMS** — in `postSim`, check `resp.StatusCode` and return an error for non-2xx responses.
3. **Pre-compile regex patterns** — move `regexp.MustCompile` calls to package-level `var` declarations so they run once at init, not per request.
### Medium Priority
4. **Reuse `http.Client`** — create a single client at startup with the configured timeout rather than allocating one per request.
5. **Replace `ioutil.ReadAll`** with `io.ReadAll` — the former is deprecated since Go 1.16.
6. **Add handler-level tests** — use Gin's `httptest` to test the `/send` and `/receive` endpoints end-to-end, including body size limits, count validation, and malformed JSON.
### Low Priority
7. **Introduce domain types**`SendRequest`, `ReceiveRequest`, `CIIMSResponse` structs to replace raw string passing.
8. **Introduce a `Config` struct** — eliminate package-level globals for `defaultURL` and `timeout`.
9. **Normalize template indentation** — make `sendtpl` and `receivetpl` use consistent whitespace.
10. **Remove the redundant `Send` wrapper** — rename `postSim` to `Send` and delete the passthrough.