Skip to content

Collector self-logs: promote stanza operator_id / operator_type #27435

Open

Description

@kroepke

Follow-up from #27258.

The stanza framework attaches operator identity to every log record its operators emit (pkg/stanza/operator/helper/operator.go:65):

set.Logger = set.Logger.With(zap.String("operator_id", c.ID()), zap.String("operator_type", c.Type()))

operator_id also appears standalone in pkg/stanza/operator/parser/regex/config.go:86.
Both are zap.String, so promoteString handles them directly.

Neither is promoted today — they hit the default arm in CollectorLogRecordProcessor and end up in collector_log_attributes (enabled: false), so they are not searchable. They identify which configured input is failing, which is exactly what is needed when debugging a broken filelog or journald input.

Proposal

Restore collector_operator_id / collector_operator_type as top-level keyword fields (both existed before #27258) and add them to CollectorLogsIndexMapping.

collector_* rather than agent_* is deliberate: the GIM agent entity added in Illuminate #4048 is explicitly scoped to agent identity, and its only Graylog-specific additions are fleet, source-routing and receiver type. Operator identity is collector internals, so it stays in our own namespace.

Worth checking first

BasicConfig.ID() returns OperatorType when no explicit ID is configured (operator/helper/operator.go:29-33), so the two values are identical unless something named the operator. If Graylog's generated collector configs don't assign operator IDs, operator_type alone may be sufficient and operator_id can stay preserved.

Out of scope: component

component is ambiguous across producers and needs separate analysis before anyone promotes it. Two emitters exist in the current dependency tree, with different semantics:

  • pkg/stanza/fileconsumer/config.go:165 — zap.String("component", "fileconsumer"), a constant bound to the logger.
  • extension/storage/filestorage/extension.go:86 — zap.String("component", rawName), the actual name of the component whose storage client is being created. filestorage is in our distribution (builder/builder-config.yaml:12).

Promoting the bare key would mix a constant tag with real identifiers in one keyword field. If we want it, it probably needs disambiguating via otelcol.component.id/kind. Graylog's own receivers (macosunifiedloggingreceiver, windowseventlogreceiver) emit no component attribute at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions