Repository navigation
id(groups): a parseable torn final line in groups.jsonl is accepted, glued onto by the next append, and then bricks every group command #2947
Description
Summary
id(groups): GroupLog::open (pkgs/id/src/groups/log.rs) accepts a parseable torn final line — a complete event JSON whose trailing newline was lost to a crash mid-append — and the next record() append glues the new event onto that same line. The resulting file can never be opened again: the glued line is newline-terminated and unparseable, so GroupLog::open fails fatally with "line N is not an event" and every group command bricks until someone hand-edits groups.jsonl.
This is exactly the failure #1620 fixed for the envelope outbox ("a parseable torn tail is accepted, then merged into the next record") and #1521 fixed for the directory journal. The sibling rule those fixes established — an unterminated final line is torn, parseable or not, and is dropped and truncated at open — was never applied to GroupLog. #1522 fixed GroupLog only for the unparseable fragment case.
Evidence (main b90f91a9)
GroupLog::append(log.rs:172-183) writes one JSON line +\nin a singlewrite_allfollowed bysync_data, so a crash/power loss during that write can leave the full JSON with only its\nmissing.GroupLog::open(log.rs:60-92):completeis computed (line.ends_with('\n'), :68) but is only consulted on the error arm (Err(_) if !complete→ drop torn fragment). On theOk(event)arm the event is pushed andkept += line.len()regardless ofcomplete— so a parseable unterminated tail is kept,kept == text.len(), and no truncation/repair happens.record()→append()(log.rs:155-183) then appends the next event's line directly after the unterminated one, merging two JSON objects onto one line.- On the next open, that merged line is newline-terminated, so it takes the fatal arm:
"{path}: line {N} is not an event". Every group command starts withGroupLog::open(commands/group.rs), so all of them fail the same way. - Contrast the established sibling rule:
Outbox::open(envelope_outbox.rs:141-150) drops an unterminated final line "Parseable or not … Keeping a parseable torn line would let the next append merge two records onto one line and brick the queue (id(outbox): a parseable torn tail is accepted, then merged into the next record — Outbox::open only drops unterminated final lines that fail to parse #1620)";Directory::replay(directory.rs:514-525, id(directory): a torn final line in directory.jsonl bricks the directory — Directory::replay refuses the crash-torn tail that ArtifactLog (loaded in the same open) drops and truncates #1521) drops and truncates the same way.
Verified (laptop probe, real id crate)
Probe: ~/git/fleet/probe-2026-10-30-laneB-grouplog/ (path-dep on the real id crate from a laptop worktree at origin/main b90f91a9; log.rs md5 e70afe7a2262351c65dfd69a701ae0f4, identical in both checkouts):
control healthy: events=2
case first outcome: Applied
case reopen-after-tear: ACCEPTED events=1
case second record: Ok(Applied)
case file lines after append: 1
case final reopen: BRICKED /home/user/tmp/.tmp2e4gPn/groups.jsonl: line 1 is not an event: trailing characters at line 1 column 427
- Control: a healthy newline-terminated log records two events and reopens with
events=2. - Case: after stripping only the trailing
\nfrom a one-event log (the crash cut), reopen accepts the event; the nextrecord()succeeds (Applied) but the file has 1 line for 2 events (glued); the final reopen is bricked.
Why it matters
groups.jsonl is the node's only group record; every group command opens it first. The one-byte tear (a complete JSON missing its newline) is precisely the crash outcome append's single write_all + sync_data can leave, and the current open converts that recoverable state into permanent, fatal corruption on the next ordinary write — silently (the append reports success). Two sibling stores in the same crate already encode the correct rule with regression tests; GroupLog is the remaining store that doesn't.
Acceptance criteria
-
GroupLog::opentreats an unterminated final line as torn whether or not it parses: the line is dropped (with the same warn log as the fragment case) and the file is truncated to the last complete line — mirroringOutbox::open(id(outbox): a parseable torn tail is accepted, then merged into the next record — Outbox::open only drops unterminated final lines that fail to parse #1620) andDirectory::replay(id(directory): a torn final line in directory.jsonl bricks the directory — Directory::replay refuses the crash-torn tail that ArtifactLog (loaded in the same open) drops and truncates #1521). - Regression test in
groups/log.rsmirroring the existing torn-fragment tests and the outbox'sa_parseable_torn_last_line_is_also_dropped: write a valid event, strip only its trailing newline, reopen (event dropped, file truncated to empty/previous complete lines), record a new event, reopen again and fold successfully. - Existing behavior unchanged: a newline-terminated line that does not parse stays fatal; blank lines still skipped; healthy logs record/reopen exactly as before.
- Related, do not duplicate: id(groups): a torn final line in groups.jsonl bricks every group command — GroupLog::open refuses the crash-torn tail that FileJournal and Outbox drop and truncate #1522 (GroupLog unparseable torn tail — closed, this is its parseable sibling case), id(outbox): a parseable torn tail is accepted, then merged into the next record — Outbox::open only drops unterminated final lines that fail to parse #1620 (outbox parseable torn tail — closed), id(directory): a torn final line in directory.jsonl bricks the directory — Directory::replay refuses the crash-torn tail that ArtifactLog (loaded in the same open) drops and truncates #1521 (directory torn tail — closed), id(artifact): a blank line in the artifact journal bricks Directory::open — ArtifactLog refuses blank lines its sibling loader skips, and truncates torn tails silently #2128 (artifact journal blank lines — distinct).
Activity
dezren39 commentedon Oct 31, 2026
🤖 Agent note · filed
idcrate: GroupLog::open accepts a parseable torn final line (complete event JSON, only its trailing newline lost to a crash mid-append), the next record() glues its event onto that line (file holds 1 line for 2 events), and the following open bricks fatally — "line 1 is not an event: trailing characters at line 1 column 427" — for every group command. This is the id(outbox): a parseable torn tail is accepted, then merged into the next record — Outbox::open only drops unterminated final lines that fail to parse #1620 (outbox) / id(directory): a torn final line in directory.jsonl bricks the directory — Directory::replay refuses the crash-torn tail that ArtifactLog (loaded in the same open) drops and truncates #1521 (directory) parseable-torn case; id(groups): a torn final line in groups.jsonl bricks every group command — GroupLog::open refuses the crash-torn tail that FileJournal and Outbox drop and truncate #1522 fixed GroupLog only for unparseable fragments. Full evidence in the issue body.24 remaining items
dezren39 commentedon Nov 2, 2026
agent-note:v1 — state: parked at 2026-11-02T09:04:55Z (OpenChamber scheduled lane)
dezren39 commentedon Nov 2, 2026
agent: repo-worker-code (lane C)
state: picked-up
task: 5480
issue: 2947
pr: 4287
Lane C claimed task 5480 (resume from awaiting_ci). Fresh revalidation 2026-11-02T09:53Z: issue #2947 OPEN, no hard-stop labels; PR #4287 OPEN at 8509b3a on the task's own branch queue/5480-2947-grouplog-torn-tail, head unchanged since the prior pass. Diff verified: GroupLog::open now drops ANY unterminated final line (with the existing warn) before parsing, instead of only unparseable ones — a parseable torn tail was previously kept and appended onto. New regression test truncates the final newline and asserts the torn entry is dropped and a subsequent append round-trips cleanly. Prior-pass laptop evidence stands: cargo test -p id groups::log 7/7 on this head, rustfmt and diff --check clean, single-file diff. Codex: no Codex review or findings exist on PR #4287. CI run 41416889387: basic guard and native arm64 still pending (runner backlog), full flake check skipping — proceeding under standing merge authority with the laptop-verified tests.
dezren39 commentedon Nov 2, 2026
agent: repo-worker-code (lane C)
state: done
task: 5480
issue: 2947
pr: 4287
Task 5480 complete: PR #4287 squash-merged at 2026-11-02T09:44Z (merge commit 84af61b). GroupLog::open now drops any unterminated final line — including parseable torn tails — before parsing, so a later append can no longer glue records together; newline-terminated corrupt records remain fatal. Regression test covers drop, truncate, append and reopen. Verification: cargo test -p id groups::log 7/7 on the merged head (laptop), rustfmt + diff --check clean, diff re-verified this shift, no Codex findings on the PR; CI was still pending at merge time (runner backlog) — merged under standing merge authority. The PR body carried no closing keyword, so this note closes the issue manually with the merge evidence above. In-flight labels removed.