Skip to content

Document structure of parseable documentation markdown #402

Description

@Ethan-Arrowood

Even though this project's original goal is focussed around parsing the existing Node.js documentation, it would be valuable to start defining the specific ruleset it supports. For example, what is to a Class or Function definition? What goes into the section header, what are the necessary vs option subsections, etc.

I imagine that once we define the foundational ruleset for the existing Node.js docs, then we can iterate on it and adjust to support a different format and even become more general purpose. This would go hand in hand with linter rules and more specifically what kind of AST structure we want to declare.

In the past I experimented with this idea. What we are defining is really a subset of markdown. Since markdown itself is not very well defined we should make some declarations upfront like are we starting from GitHub Flavored Markdown or something else? I called my previous experiment "markdown documentation language" or "mddl" for short. You can see my WIP idea here.

It'd be my pleasure to start contributing to this initiative instead of continuing to create my own.

Activity

  1. self-assigned this
    on Aug 15, 2025
  2. ovflowd commented on Aug 15, 2025

    @ovflowd
    Member

    Related to #332

  3. Ethan-Arrowood commented on Sep 29, 2025

    @Ethan-Arrowood
    ContributorAuthor

    Btw I have not forgotten about this. Likely will complete by end of October

  4. avivkeller commented on Sep 29, 2025

    @avivkeller
    Member

    Take all the time you need... you are helping us just by starting it

  5. avivkeller commented on Dec 23, 2025

    @avivkeller
    Member

    @Ethan-Arrowood What's the progress here?

    (Also, note that this document can differ from the current format, ideally a mix of easily parse-able and write-able).

    For example, right now the documentation is easy to write, but not so easy to parse (since we have so many differing parts)

  6. Ethan-Arrowood commented on Jan 12, 2026

    @Ethan-Arrowood
    ContributorAuthor

    Hey! Progress is okay.

    I've been using the existing tools/doc/generate.mjs and tools/doc/json.mjs as starting points to try and understand what the "standard" structure is for the docs/api/*.md pages.

    The good news is that there is already lots of consistency here. The bad news is that it is quite complicated; and that is what I'm having the hardest time with. Creating a style guide for these is proving a little difficult even though I conceptually understand the details. It is hard for me to put into words, but I'm still hacking at it.

  7. avivkeller commented on Jan 12, 2026

    @avivkeller
    Member

    Makes sense, yes, the docs are quite confusing format wise. In case it wasn't clear earlier, I think you should be able to change anything to make an ideal format, and the docs will be updated accordingly.

    Let me know if you need any help :-)

  8. Ethan-Arrowood commented on Jan 12, 2026

    @Ethan-Arrowood
    ContributorAuthor

    I had AI help me articulate my analysis and thoughts... don't take the "What We Need" too seriously; but there is some truth to it.

    The Challenge of Standardizing Node.js API Documentation Structure

    The Node.js API documentation has a consistent implicit structure, but it's difficult to standardize because the relationship between documentation files and modules is not 1:1 and is declared contextually rather than explicitly.

    The Core Issue: Implicit Module Scoping

    The mapping between .md files and Node.js modules is not defined anywhere - it's inferred from the content itself through naming conventions:

    Pattern 1: File = Module (with prefixed references)

    • http.md documents the node:http module
    • APIs are prefixed: Class: http.Agent, Class: http.Server
    • The http. prefix tells you these are exported from the module

    Pattern 2: File = Module (without prefixed references - GLOBALS)

    • buffer.md documents the node:buffer module
    • But Class: Buffer and Class: Blob have no prefix
    • This could mean they're globals (and they are!)
    • They're available as: import buffer from 'node:buffer'; buffer.Buffer
    • AND as globals: Buffer, Blob
    • The globals.md file maintains the canonical list of what's actually global

    Pattern 3: Unprefixed but NOT Global (orphaned types)

    • fs.md documents Class: FileHandle (no prefix)
    • It's NOT a global
    • It's NOT directly exported from node:fs or node:fs/promises
    • It's only obtainable via: await fsPromises.open()
    • These are "return-only" types with no import path

    Pattern 4: Multiple Modules per File

    • fs.md documents both node:fs and node:fs/promises
    • Section ## Promises API contains fs/promises APIs
    • Section ## Callback API contains fs APIs
    • The module path fs/promises is only mentioned in prose: "The node:fs/promises API provides asynchronous file system methods..."
    • No explicit header like ## node:fs/promises module APIs (though some files do this)

    Why This Matters for Standardization

    To build proper tooling, we need to answer:

    1. What module does this API belong to?

      • Current answer: Parse the prefix or infer from section context
    2. How do I import this API?

      • http.Agent → import { Agent } from 'node:http'
      • Buffer → import { Buffer } from 'node:buffer' OR just use globally
      • FileHandle → Not importable, only returned by fsPromises.open()
    3. Is this a global?

      • Current answer: Check if it appears in globals.md or if it lacks a prefix in its defining file
    4. Which file documents which module(s)?

      • Current answer: No explicit mapping exists

    What We Need

    To standardize, we need explicit metadata that currently lives only in prose:

    # File system
    <!--introduced_in=v0.10.0-->
    <!--modules=fs,fs/promises-->  <!-- NEW: explicit module list -->
    
    > Stability: 2 - Stable
    
    ## Promises API
    <!--module=fs/promises-->  <!-- NEW: section declares module scope -->
    
    ### Class: `FileHandle`
    <!--module=fs/promises-->  <!-- NEW: this class belongs to fs/promises -->
    <!--exported=false-->       <!-- NEW: not directly importable -->
    <!--global=false-->          <!-- NEW: not a global -->

    OR we formalize the implicit rules:

    1. Heading prefix = module membership (e.g., http.Agent belongs to http)
    2. No prefix + listed in globals.md = global
    3. No prefix + not in globals.md = return-only type
    4. Section heading determines module scope when multiple modules per file

    The Question

    Should we:

    Option A: Add explicit metadata to make the implicit structure explicit?

    Option B: Formalize and document the implicit rules, then build tooling around them?

    Option C: Restructure to 1 file = 1 module, split files like fs.md into fs.md + fs-promises.md?

  9. avivkeller commented on Jan 15, 2026

    @avivkeller
    Member

    I think the best approach is to infer the meaning of headings based on the documentation file’s type. Each file in the nodejs/node doc/api directory has an associated type. For example, C++ Addons is classified as misc, since it doesn’t describe a module, and Assert is module by default.

    Under this model, if a file’s type is module, headings would be treated as exported by default unless explicitly marked otherwise1. If the type is misc2, headings would instead be assumed to be global unless stated otherwise.

    While we have specific differences in how we render globals vs exported methods, we don't have any differences in how we render one exported method from another module's exported method. It shouldn't really matter too much if fs.md has both fs and fs.promises / fsPromises exports.

    In a place where those little differences did matter, I'd say that we'd add a <!-- module=node:fs/promises --> under the Promises API header, to identify that all sub-headers are from the separate module.

    Footnotes

    1. This could be handled via YAML metadata (e.g., type: global) on individual headings. I believe doc-kit already supports this. ↩

    2. Or potentially a more specific classification, such as globals. ↩

  10. Ethan-Arrowood commented on Jan 15, 2026

    @Ethan-Arrowood
    ContributorAuthor

    Okay great so that the type metadata will be useful for differentiating.

    With that I think I can put together some structure style guides 🤔

    I think yes we may need to introduce a new metadata for globals. What do you mean doc-kit already supports that? Are you parsing for a certain metadata for globals? I see it totally reasonable for us to make upstream node docs changes if its to the benefit of the docs tooling. The existing globals.md page currently uses type: misc too.

    I think the best solution may be to just introduce more metadata. Make the docs more declarative rather than imperative I guess.

  11. avivkeller commented on Jan 27, 2026

    @avivkeller
    Member

    What do you mean doc-kit already supports that? Are you parsing for a certain metadata for globals?

    I meant that doc-kit supports supplying metadata on a heading-level basis. Our parser can parse YAML metadata at any depth.

  12. removed this from the Web Generator Migration milestone on Feb 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions