Skip to content

ci: validate GraphQL operations in Python sources #12784

Description

@kyle-sexton

The GraphQL operations test from #12734 (.github/scripts/graphql-operations/graphql-operations.test.cjs) scans tracked JS, shell and workflow sources only. Python sources are not scanned, so the operations in 8 files under plugins/source-control/skills/babysit-prs/scripts/ are never validated against GitHub's schema, including the resolvedBy{__typename login} paginator in babysit_gh.py, the same field whose ... on Bot spread broke the review lanes.

Found by the claude-review lane on #12734: #12734 (comment)

What is needed

  • Add py to SOURCE and teach the extractor Python's string shapes: implicit concatenation of adjacent literals inside parentheses, f-string {expr} substitutions (placeholder per position, as the JS path does), and triple-quoted literals.
  • Add an unextractable.txt row for each Python site that still cannot be checked, with its reason.
  • Remove the "Python sources are not scanned" notes from the test header and unextractable.txt.

Acceptance

Changing resolvedBy{__typename login} to resolvedBy{... on Bot{login}} in babysit_gh.py makes node --test graphql-operations.test.cjs fail.

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 3, 2026
  2. github-actions commented on Nov 3, 2026

    @github-actions
    Contributor

    Proposed work class: work-class: scoped
    Basis: automated intake triage; the lanes App applied this class.

    Size: Medium
    Complexity: Medium
    Uncertainty: Known
    Flags: none
    Blocked by: none

    I understand this as extending the GraphQL operations test to scan Python sources under the babysit-prs scripts. The extractor needs to handle adjacent-literal concatenation, f-string placeholders and triple-quoted strings. Sites it can't check get unextractable.txt rows. The "not scanned" notes come out, and changing resolvedBy{__typename login} in babysit_gh.py must fail the test.

    Decision recorded.
    Agent choice, not an owner answer.
    Answers to the refinement's three questions. (1) This issue builds on the graphql-operations test and unextractable.txt that #12734 added; this record is posted after #12734 merged, so both are on the base branch. (2) The scan covers the production scripts under plugins/source-control/skills/babysit-prs/scripts/ and not their tests/ directory, whose files hold partial operations on purpose. (3) Yes: each Python site the extractor cannot resolve gets an unextractable.txt row with its reason, rather than a refactor of the Python.

    • The graphql-operations test's SOURCE includes py, and its extractor handles implicit concatenation of adjacent literals inside parentheses, f-string {expr} substitutions (one placeholder per position, as the JS path does) and triple-quoted literals.
    • Each Python production site under plugins/source-control/skills/babysit-prs/scripts/ that still cannot be checked has an unextractable.txt row with its reason; files under that directory's tests/ are not scanned.
    • The "Python sources are not scanned" notes are removed from the test header and unextractable.txt.
    • Changing resolvedBy{__typename login} to resolvedBy{... on Bot{login}} in babysit_gh.py makes node --test graphql-operations.test.cjs fail.

    #12784 (comment)

    This decision settles the open questions. I found no likely duplicates.

    Automated triage, applied by the lanes App.

  3. added
    priority: mediumReal value, no hard deadline; normal backlog flow.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 3, 2026
  4. added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.
    on Nov 3, 2026
  5. melodic-automation-lanes commented on Nov 3, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane yet: no trusted-human acceptance criteria: a listed human must put the confirmed - [ ] criteria in the body or post them as their own comment, or, on an issue a listed human filed, intake's newest refinement comment must hold them unedited; intake's refinement of this body is due and may supply them.

  6. added
    status: needs-decisionAwaiting a human or maintainer judgment call.
    and removed
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    on Nov 3, 2026
  7. melodic-automation-lanes commented on Nov 3, 2026

    @melodic-automation-lanes
    Contributor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

    Intended outcome: The GraphQL operations test scans Python sources under babysit-prs/scripts, so changing resolvedBy{__typename login} in babysit_gh.py fails it, with unextractable.txt rows for sites it cannot check.

    Open questions

    1. The checkout has no .github/scripts directory, no graphql-operations.test.cjs and no unextractable.txt. Has #​12734 merged to the base branch, or is .github just absent from this checkout?
      RECOMMENDED: Treat #​12734 as a dependency and work this issue once its test and unextractable.txt are on the base branch. Basis: Glob and Grep of .base found no graphql-operations files and no 'unextractable' text anywhere
    2. Should the scan cover only the 8 production scripts, or also the files under babysit-prs/scripts/tests, which hold many query and mutation strings?
      RECOMMENDED: Scan production scripts only and exclude tests/, since tests may hold intentionally partial operations; follow whatever scope the JS path already uses. Basis: The issue says 8 files, but grep finds query/mutation text in many more .py files, e.g. tests/guard_contract.py with 33 matches
    3. Is it acceptable to record Python sites the extractor cannot resolve, such as f-strings that build field selections, as unextractable.txt rows instead of refactoring the Python?
      RECOMMENDED: Yes, add a row with a reason for each, as the issue states. Basis: The issue's What is needed section, third bullet

    Sources read

    • plugins/source-control/skills/babysit-prs/scripts/babysit_gh.py:2271: the review-threads query contains resolvedBy{__typename login}, the field named in the acceptance check
    • plugins/source-control/skills/babysit-prs/scripts/tests/test_babysit_gh.py:399: a test asserts the query contains resolvedBy{__typename login}

    To answer, post a comment that starts with this line:

    **Decision recorded.** <answer>
    

    Automated refinement by the lanes App. Answer with a comment whose first line is **Decision recorded.**.

  8. kyle-sexton commented on Nov 3, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded.
    Agent choice, not an owner answer.
    Answers to the refinement's three questions. (1) This issue builds on the graphql-operations test and unextractable.txt that #12734 added; this record is posted after #12734 merged, so both are on the base branch. (2) The scan covers the production scripts under plugins/source-control/skills/babysit-prs/scripts/ and not their tests/ directory, whose files hold partial operations on purpose. (3) Yes: each Python site the extractor cannot resolve gets an unextractable.txt row with its reason, rather than a refactor of the Python.

    • The graphql-operations test's SOURCE includes py, and its extractor handles implicit concatenation of adjacent literals inside parentheses, f-string {expr} substitutions (one placeholder per position, as the JS path does) and triple-quoted literals.
    • Each Python production site under plugins/source-control/skills/babysit-prs/scripts/ that still cannot be checked has an unextractable.txt row with its reason; files under that directory's tests/ are not scanned.
    • The "Python sources are not scanned" notes are removed from the test header and unextractable.txt.
    • Changing resolvedBy{__typename login} to resolvedBy{... on Bot{login}} in babysit_gh.py makes node --test graphql-operations.test.cjs fail.
  9. 15 remaining items

  10. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    and removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 4, 2026
  11. github-actions commented on Nov 4, 2026

    @github-actions
    Contributor
    No description provided.
  12. github-actions commented on Nov 4, 2026

    @github-actions
    Contributor
    No description provided.
  13. melodic-automation-lanes commented on Nov 4, 2026

    @melodic-automation-lanes
    Contributor

    The work-resolve-issue run that agent-ready started waited in its slot past a change to its lane workflow files on main, so it stopped before any model turn. The run's report-outcome removed and added agent-ready back to start a fresh run on main's workflow files. If this run ends the same way, it re-fires the issue again, waiting longer each time.

  14. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    and removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 4, 2026
  15. melodic-automation-lanes commented on Nov 4, 2026

    @melodic-automation-lanes
    Contributor

    This records a re-fire of an earlier refusal, not a new refusal. The lanes kill switch stopped run 41756522409, the work-resolve-issue run that agent-ready started, before it did any work; that run ended at 2026-11-04T01:50:00Z. work-unblock has now removed and added agent-ready back to start a fresh run. A kill-switch refusal is no fault in this issue or the lanes, so it never counts against later re-fires; after limits.limit-stops-per-issue of them since a person last added agent-ready or a run last reported, work-unblock waits longer before each one, up to 24 hours.

  16. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    and removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 4, 2026
  17. github-actions commented on Nov 4, 2026

    @github-actions
    Contributor
    No description provided.
  18. github-actions commented on Nov 4, 2026

    @github-actions
    Contributor
    No description provided.
  19. removed
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: mediumReal value, no hard deadline; normal backlog flow.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

    Type

    Fields

    Work status

    agent-ready

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions