You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add explicit date mappings for event_created / event_received_time to the default index mapping #27437
IndexMapping.fieldProperties (graylog2-server/src/main/java/org/graylog2/indexer/template/IndexMapping.java:111-136) maps timestamp, streams, source, message, full_message and the gl2_* fields, but not the GIM event timestamps. On index sets without a custom mapping, those fields are left to dynamic mapping.
Since #27258, the collector codec writes these as DateTime objects rather than preformatted strings, and since #27428 all three storage backends serialize them identically as ISO-8601. Graylog never sets date_detection anywhere, so OpenSearch's default (true, with strict_date_optional_time among dynamic_date_formats) applies, and IndexMapping7.dynamicStrings() only matches match_mapping_type: "string". Date detection resolves the type before dynamic templates are consulted, so on a new index thesefields now get mapped date automatically.
That works, but it is implicit and fragile.
Why make it explicit
Dynamic detection is first-document-wins per index.
If the first document in a fresh index carries a non-ISO value for one of these fields — set by a pipeline rule, or by another input writing the same GIM field — the whole index maps it keyword instead, and range queries silently stop working for that index.
It depends on an OpenSearch default we never assert.
Nothing in our templates pins date_detection, so a future default change or a cluster-level override would silently turn these back into keywords.
It protects against the inverse of the bug Change Collector system logs mapping #27258 fixed.
A user-set custom Date mapping renders as bare {"type": "date"} with no format (IndexMapping.java:126), which rejects Graylog's yyyy-MM-dd HH:mm:ss.SSS format. Any writer still emitting the legacy format into such a field produces mapper_parsing_exception and drops the whole document.
Proposal
Add event_created and event_received_time as explicit date fields. Worth considering the wider set of GIM date fields at the same time rather than these two in isolation.
Migration note
Index sets that already hold these as keyword will show a field type conflict once they rotate and new indices map them as date. Nothing is rejected and it clears as old indices age out, but it should be mentioned in the changelog.
Activity