Skip to content

research(licensing): may we ingest external GitHub comments/discussions into Atlas? — gate for the external corpus #6024

New issue
New issue
Open
Open
research(licensing): may we ingest external GitHub comments/discussions into Atlas? — gate for the external corpus#6024
Assignees
dezren39
Labels
area/infraInfrastructure, deployment, and operationskind/designA design proposal (design-architect): multi-step, phased, decision-bearingqueue/parkedQueue task is parked (waiting on CI, time, or an event)status/parkedDeliberately paused with a recorded resume conditiontask/triageQueue packet kind: verify, reproduce, and route this itemwork/assignedQueue worker lifecycle state

Description

@dezren39
dezren39
opened on Nov 5, 2026

Parent: #6018 (Atlas ingestion program). Statements S13–S14 of the 2026-11-04 brief. Family: tonight's licensing program — #5977 (taxonomy/classifier), #5979 (SPDX tooling), #5984 (policy/ToS), #5986 (AGPL/GPL hosting research). This issue is the ingestion-side counterpart: copying OTHER people's words IN, where the licensing program mostly covers our code and user uploads.

The gate (Drew, in substance)

Before Atlas ingests comments from external repos, determine whether we may: can I copy a comment from a GitHub discussion? Is it Creative Commons? Drew's sequencing: figure this out before the external corpus is built, but after it is somewhat designed — so this research runs parallel to design and gates the build of the external half of the GitHub-corpus extraction (sibling task under #6018). Our own repos are not gated.

Questions the research must answer

  1. What license, if any, covers a GitHub issue/PR comment or Discussion post? Facts to establish from primary sources (GitHub Terms of Service, esp. the user-content sections): users retain copyright; users grant GitHub a broad license; what, exactly, do OTHER users/third parties get (viewing + forking rights inside GitHub are granted — what about copying OUT)? Is any of it Creative Commons (short answer expected: no — but establish it, don't assume; note the contrast with Stack Overflow's CC BY-SA).
  2. Distinctions that may change the answer:
    • Our own comments on external repos vs third parties' comments.
    • Issues/PRs vs Discussions vs gists vs wiki pages.
    • Storing + embedding for a private, personal search corpus (never redistributed) vs redistributing/publishing the text — does private, non-distributive copying for personal retrieval sit differently (fair use / implied license / API terms) than republication?
    • GitHub API Terms constraints on storing API-served content, independent of copyright.
    • A repo whose project license is copyleft (GPL/AGPL) — does the project license reach comments filed in its tracker? (Generally no, but confirm and note the exceptions people assert.)
    • Attribution obligations if any use is permitted.
  3. Practical postures available: e.g. ingest-own-comments-only externally; ingest-all-but-never-redistribute (internal tier, no hydration into shared contexts); ingest metadata + embeddings only, fetch text live from GitHub at resolution time (store-no-text pattern — cf. #3125's resolve-at-read design); exclude Discussions while allowing issues/PRs. For each: what it costs the corpus and what risk it retires.

Done looks like

A sourced research note (primary sources linked, no vibes) ending in a clear gate ruling recommendation per posture, recorded on this issue and linked from the extraction task and #5986. The extraction build does not start its external half until Drew accepts a posture.

Activity

  1. dezren39
    added
    area/infraInfrastructure, deployment, and operations
    kind/designA design proposal (design-architect): multi-step, phased, decision-bearing
    on Nov 5, 2026

  2. developing-today commented on Nov 5, 2026

    @developing-today
    developing-todayBot
    on Nov 5, 2026 – with developing.today
    Contributor

    CODE-1690

  3. dezren39 commented on Nov 5, 2026

    @dezren39
    dezren39
    on Nov 5, 2026
    MemberAuthor

    Gate research result: ANSWERED (provisional) — the external build may proceed on zone rules

    The research lane read the GitHub Terms of Service in full (Section D + Section H, effective 2026-04-27) and answered this gate from primary text. Bottom line:

    • Copyright stays with the commenter. "You own the content you post on GitHub." Comments are Content, expressly named in the ToS.
    • D.4's license runs to GitHub only — store/host/parse, build a search index, even train AI Features, and share with other Users through the Service. Atlas acquires no rights from D.4; "GitHub can index it, so I can too" borrows a license granted to someone else.
    • D.5's license to other users is narrow: view, and use/display/perform/reproduce (by forking) content "through the Service as permitted by GitHub's functionality." Copying comment text off the Service into an external corpus is outside the four corners of that grant.
    • D.6 inbound=outbound is the strongest pro-reuse provision and is genuinely contested for comments: "Whenever you add Content to a repository containing notice of a license, you license that Content under the same terms." Whether issue/PR/discussion comment text is "add[ing] Content to a repository" (vs. conversation about it) is undefined in the ToS — Lawyer Question 1.
    • Comments are NOT Creative Commons. The ToS document itself is CC0 and github/docs is CC BY 4.0 (content) / MIT (code) — GitHub applied CC to its own materials, not to user content. The common belief conflates GitHub-the-author with GitHub-the-host.
    • D.8 defers to copyright law: GitHub's terms "do not restrict lawful access to or use" by third parties — public accessibility is not a license; no license → all rights reserved. Section H (API) grants nothing relevant. Also noted: D.5's licenses end when content is removed (unless forked within the Service) — an external corpus should honor source deletions.

    The three zones (the operative answer)

    Zone Posture
    Our own repos (Drew's + org; his comments anywhere) Clear — ingest freely (subject to classification tiering)
    External repos where Drew participated Private personal search defensible; no redistribution. His comments are his; others' replies ride on D.6 (where the repo is OSS-licensed) or untested fair-use-style arguments
    External public repos generally (bulk) Private personal search only; serving others or redistributing the corpus is not defensible on current sources. Factual metadata (titles, URLs, timestamps, counts, hashes) is not copyrightable expression and is always safe

    Reframe of this issue

    Status: answered-provisional. External design may proceed on the zone rules above. External full-text build for Zones 2–3 stays personal-search-only until the lawyer packet returns; the metadata layer of Zones 2–3 and all of Zone 1 are ungated. The five questions the ToS does not settle (D.6 reach; private-index fair use incl. whether embeddings change it; the redistribution line if anyone else ever queries; EU/UK TDM exposure; deletion/takedown process) are packaged as a follow-up task: #6055.

    Wave-2 children filed under this gate

    • #6052 — the three zones as an executable decision doc
    • #6053 — per-repo license + D.6 flag in provenance
    • #6054 — source deleted / repo privated → next-sync purge + trace tombstone
    • #6055 — the five open questions, packaged
    • #6056 — verify Doe v. GitHub docket state before the policy cites it

    — Wave-2 research fold-in, 2026-11-04 (prior-art doc: ~/workspace/scratch/atlas-ingestion-brief-2026-11-04/prior-art-research.md)

  4. dezren39
    added
    queue/queuedHas a queued task in the Supabase agent queue
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 5, 2026

  5. dezren39 commented on Nov 5, 2026

    @dezren39
    dezren39
    on Nov 5, 2026
    MemberAuthor

    🤖 Agent note · handed off

  6. dezren39
    added
    work/assignedQueue worker lifecycle state
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 5, 2026
  7. github-actions
    assigned
    dezren39
    on Nov 5, 2026

  8. dezren39 commented on Nov 5, 2026

    @dezren39
    dezren39
    on Nov 5, 2026
    MemberAuthor

    🤖 Agent note · handed off

  9. dezren39
    added
    queue/claimedQueue worker lifecycle state
    status/in-progressQueue worker lifecycle state
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/in-progressQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    on Nov 5, 2026

  10. developing-today commented on Nov 5, 2026

    @developing-today
    developing-todayBot
    on Nov 5, 2026 – with developing.today
    Contributor

    🤖 Agent note · handed off

  11. developing-today commented on Nov 5, 2026

    @developing-today
    developing-todayBot
    on Nov 5, 2026 – with developing.today
    Contributor

    agent-note:v1
    task: 10391
    lane: code-j
    state: handed_off
    execution_lane: opencode
    note: Claimed by lane code-j (attempt 3/6, repeat handoff); issue #6024 verified OPEN with no hard-stop labels via developing.today app token (gh user token rate-limited). Handed off to opencode claimant to launch the bound session; parking 1800s pending claimant launch.

  12. dezren39 commented on Nov 5, 2026

    @dezren39
    dezren39
    on Nov 5, 2026
    MemberAuthor

    Agent note - handed off

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/infraInfrastructure, deployment, and operationskind/designA design proposal (design-architect): multi-step, phased, decision-bearingqueue/parkedQueue task is parked (waiting on CI, time, or an event)status/parkedDeliberately paused with a recorded resume conditiontask/triageQueue packet kind: verify, reproduce, and route this itemwork/assignedQueue worker lifecycle state

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions