Skip to content

Commit 82cb80c

Browse files
Scope argsSchema docs to the run_factory path and widen the E2E fixture
Enforcement lives behind toolRunFactoryValidateArgs, which the runtime calls only from runFactoryTool. session.factory.run does not validate, so the docs and JSDoc now say which caller is checked instead of implying all of them are. The argument-echo fixture declared ["object","null"] while its contract is to echo any JsonValue, and it is invoked with an array. Nothing broke, because the SDK path does not validate, but the narrow declaration was dishonest and would have become load-bearing if that path ever gained validation. Widened it to the factory's real contract and corrected the comment, which claimed the fixture exercised enforcement when it only exercises registration. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent b5fe21a commit 82cb80c

4 files changed

Lines changed: 33 additions & 12 deletions

File tree

nodejs/docs/factories.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,13 +52,15 @@ Factory metadata contains a stable `name`, a human-readable `description`, decla
5252

5353
## Declaring an argument shape
5454

55-
A factory that reads `ctx.args` should declare `meta.argsSchema`, as the example above does. The CLI validates the caller's `args` against it **before** the run starts.
55+
A factory that reads `ctx.args` should declare `meta.argsSchema`, as the example above does. When the model invokes the factory through the `run_factory` tool, the CLI validates `args` against the declaration **before** the run starts.
5656

5757
Declaring one turns an expensive failure into a cheap one. With a schema, a malformed call is rejected up front — the model gets a correction hint and retries, and no run row, permission prompt, or credit spend happens. Without one, nothing validates: the run starts, takes a user approval, spends credits, and then dies inside the factory body with a confusing error. Agents can read the declared shape with `factories_manage` using `operation: "inspect"`.
5858

5959
Enforcement covers structure — types, required properties, and enum or const values. Finer constraints such as `minLength`, `pattern`, or `additionalProperties` are recorded in the declaration but not enforced. The accepted vocabulary is the `FactoryJsonSchema` subset also used for subagent structured output: `type`, `required`, `enum`, `const`, recursive `properties`/`items`, and `anyOf`/`oneOf`/`allOf`. A `type` is one of `null`, `boolean`, `integer`, `number`, `string`, `array`, or `object`, or a non-empty array of those such as `["object", "null"]`. A declaration outside that subset is rejected at registration.
6060

61-
`argsSchema` is optional and backward compatible. A factory that omits it behaves exactly as before, so **the `description` is then the only thing telling an agent what arguments to supply** — state the expected shape there. Arguments supplied by an extension calling `session.factory.run(...)` directly are typed through `defineFactory<TArgs>`, but that typing does not reach the model. A factory that reads `ctx.args` should still validate it rather than assume a shape, because the declared subset does not enforce every constraint.
61+
`argsSchema` is optional and backward compatible. A factory that omits it behaves exactly as before, so **the `description` is then the only thing telling an agent what arguments to supply** — state the expected shape there.
62+
63+
Validation covers the model's `run_factory` path only. An extension calling `session.factory.run(...)` directly is not validated against `argsSchema`; those arguments are typed through `defineFactory<TArgs>` instead, and that typing does not reach the model. So a factory that reads `ctx.args` should still validate it rather than assume a shape — the declared subset does not enforce every constraint, and it does not run at all on the SDK path.
6264

6365
`defineFactory<TArgs, TResult>` accepts a `run(context)` function returning `Promise<TResult>`, where `TResult` is `JsonValue | void`. Objects, arrays, strings, numbers, booleans, and `null` are valid results. Returning `undefined` completes the factory with no result. Other non-JSON values are rejected.
6466

nodejs/src/types.ts

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2011,13 +2011,18 @@ export interface FactoryMeta {
20112011
* Optional declared shape of the arguments this factory expects as `ctx.args`.
20122012
*
20132013
* Declaring one is strongly recommended for any factory that reads `ctx.args`.
2014-
* The CLI validates the caller's `args` against it **before** the run starts, so a
2015-
* malformed call from the model is rejected with a correction hint and retried
2016-
* without ever creating a run row, prompting the user for permission, or spending
2017-
* credits. A factory that declares nothing is never validated: a malformed call
2018-
* starts, takes an approval, spends credits, and then fails inside the factory
2019-
* body. `factories_manage` with `operation: "inspect"` reports the declared shape
2020-
* so an agent can read it before invoking.
2014+
* When the model invokes the factory through the `run_factory` tool, the CLI
2015+
* validates `args` against this declaration **before** the run starts, so a
2016+
* malformed call is rejected with a correction hint and retried without ever
2017+
* creating a run row, prompting the user for permission, or spending credits. A
2018+
* factory that declares nothing is never validated: a malformed call starts,
2019+
* takes an approval, spends credits, and then fails inside the factory body.
2020+
* `factories_manage` with `operation: "inspect"` reports the declared shape so an
2021+
* agent can read it before invoking.
2022+
*
2023+
* This covers the model's `run_factory` path only. `session.factory.run(...)` is
2024+
* not validated against the declaration, so a factory should still check
2025+
* `ctx.args` rather than assume the declared shape held.
20212026
*
20222027
* Enforcement covers structure — types, required properties, and enum/const
20232028
* values. Finer constraints such as `minLength`, `pattern`, and

nodejs/test/e2e/fixtures/factory-extension.mjs

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,15 @@ const argumentEcho = defineFactory({
1818
name: "argument-echo",
1919
description: "Return the invocation arguments verbatim.",
2020
phases: [],
21-
// A declared shape has to survive the SDK boundary and reach the runtime,
22-
// which validates `args` against it before a run row exists.
23-
argsSchema: { type: ["object", "null"] },
21+
// Proves a declared shape survives the SDK boundary and registers against a
22+
// real runtime. It does not exercise enforcement: `argsSchema` is checked by
23+
// the model's `run_factory` tool, and these tests invoke `session.factory.run`,
24+
// which does not validate. The declaration stays as wide as this factory's
25+
// actual contract — it echoes any JsonValue, and is called with an array, an
26+
// object, and nothing — so it cannot constrain the runs below.
27+
argsSchema: {
28+
type: ["object", "array", "string", "number", "integer", "boolean", "null"],
29+
},
2430
},
2531
run: async ({ args }) => args,
2632
});

nodejs/test/factory.test.ts

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -546,6 +546,14 @@ describe("factories", () => {
546546
);
547547
}
548548
expect(normalizeJSDoc(publicTypes)).toContain("before** the run starts");
549+
// Enforcement is tool-path only: `toolRunFactoryValidateArgs` is called from
550+
// the runtime's runFactoryTool, and never from `session.factory.run`. Both
551+
// surfaces must keep saying so, or authors will assume their own SDK-initiated
552+
// runs are checked.
553+
expect(normalizeJSDoc(publicTypes)).toContain(
554+
"`session.factory.run(...)` is not validated against the declaration"
555+
);
556+
expect(guide).toContain("Validation covers the model's `run_factory` path only");
549557
expect(normalizeJSDoc(publicApi)).toContain(
550558
"`null`, `boolean`, `integer`, `number`, `string`, `array`, or `object`"
551559
);

0 commit comments

Comments
 (0)