Fixedstring type for SpanId and ParentId traces for better compression

What are you trying to do?

In the current recommended Clickhouse table structure for Traces SpanId and TraceId have type String

CREATE TABLE IF NOT EXISTS ${DATABASE}.otel_traces
(
    `Timestamp` DateTime64(9) CODEC(Delta(8), ZSTD(1)),
    `TraceId` String CODEC(ZSTD(1)),
    `SpanId` String CODEC(ZSTD(1)),
    `ParentSpanId` String CODEC(ZSTD(1)),
    `TraceState` String CODEC(ZSTD(1)),

These two fields are compressing very badly and basically make up half of the table space usage on disk by themselves:

2. │ SpanId             │ 34.68 GiB       │ 100.74 GiB        │     2.9 │        17.55 │

What would you like to see?

Would it make sense to migrate them to FixedString type?

@czephyr

@czephyr can you share the bytes per row you're currently seeing? Given SpanId is largely unique per row, there really isn't much compression we can do to improve beyond the 8byte per row. With zstd - I'm seeing it should compress to that.

@MikeShi42

@MikeShi42 Here you are:

SELECT
    c.name AS column_name,
    c.type,
    t.total_rows,
    formatReadableSize(c.data_compressed_bytes) AS compressed_size,
    formatReadableSize(c.data_uncompressed_bytes) AS uncompressed_size,
    round(c.data_compressed_bytes / t.total_rows, 2) AS compressed_bytes_per_row,
    round(c.data_uncompressed_bytes / t.total_rows, 2) AS uncompressed_bytes_per_row,
    round(c.data_uncompressed_bytes / nullif(c.data_compressed_bytes, 0), 2) AS compression_ratio
FROM system.columns AS c
INNER JOIN system.tables AS t ON (c.database = t.database) AND (c.`table` = t.name)
WHERE (c.`table` = 'otel_traces') AND (c.database = currentDatabase()) AND (t.total_rows > 0)
   ┌─column_name────────┬─type───────────────────────────────────────┬─total_rows─┬─compressed_size─┬─uncompressed_size─┬─compressed_bytes_per_row─┬─uncompressed_bytes_per_row─┬─compression_ratio─┐
 1. │ Timestamp          │ DateTime64(9)                              │ 4940078211 │ 19.55 GiB       │ 36.81 GiB         │                     4.25 │                          8 │              1.88 │
 2. │ TraceId            │ String                                     │ 4940078211 │ 17.93 GiB       │ 184.03 GiB        │                      3.9 │                         40 │             10.26 │
 3. │ SpanId             │ String                                     │ 4940078211 │ 38.01 GiB       │ 110.42 GiB        │                     8.26 │                         24 │               2.9 │
 4. │ ParentSpanId       │ String                                     │ 4940078211 │ 20.64 GiB       │ 106.31 GiB        │                     4.49 │                      23.11 │              5.15 │
 5. │ TraceState         │ String                                     │ 4940078211 │ 8.07 MiB        │ 216.07 MiB        │                        0 │                       0.05 │             26.78 │
 6. │ SpanName           │ LowCardinality(String)                     │ 4940078211 │ 11.78 MiB       │ 6.86 GiB          │                        0 │                       1.49 │            595.79 │
 7. │ SpanKind           │ LowCardinality(String)                     │ 4940078211 │ 15.97 MiB       │ 4.62 GiB          │                        0 │                          1 │             296.3 │
 8. │ ServiceName        │ LowCardinality(String)                     │ 4940078211 │ 9.58 MiB        │ 4.61 GiB          │                        0 │                          1 │            492.96 │
 9. │ ResourceAttributes │ Map(LowCardinality(String), String)        │ 4940078211 │ 43.37 GiB       │ 1.41 TiB          │                     9.43 │                     313.14 │             33.22 │
10. │ ScopeName          │ String                                     │ 4940078211 │ 1.22 GiB        │ 128.62 GiB        │                     0.27 │                      27.96 │            105.18 │
11. │ ScopeVersion       │ String                                     │ 4940078211 │ 599.99 MiB      │ 61.90 GiB         │                     0.13 │                      13.45 │            105.65 │
12. │ SpanAttributes     │ Map(LowCardinality(String), String)        │ 4940078211 │ 36.96 GiB       │ 1.35 TiB          │                     8.03 │                     299.98 │             37.35 │
13. │ Duration           │ UInt64                                     │ 4940078211 │ 15.26 GiB       │ 36.81 GiB         │                     3.32 │                          8 │              2.41 │
14. │ StatusCode         │ LowCardinality(String)                     │ 4940078211 │ 19.96 MiB       │ 4.62 GiB          │                        0 │                          1 │            237.09 │
15. │ StatusMessage      │ String                                     │ 4940078211 │ 19.04 MiB       │ 552.51 MiB        │                        0 │                       0.12 │             29.02 │
16. │ Events.Timestamp   │ Array(DateTime64(9))                       │ 4940078211 │ 14.93 GiB       │ 63.41 GiB         │                     3.24 │                      13.78 │              4.25 │
17. │ Events.Name        │ Array(LowCardinality(String))              │ 4940078211 │ 91.98 MiB       │ 4.87 GiB          │                     0.02 │                       1.06 │             54.22 │
18. │ Events.Attributes  │ Array(Map(LowCardinality(String), String)) │ 4940078211 │ 4.24 GiB        │ 150.90 GiB        │                     0.92 │                       32.8 │             35.61 │
19. │ Links.TraceId      │ Array(String)                              │ 4940078211 │ 15.80 MiB       │ 36.81 GiB         │                        0 │                          8 │           2386.15 │
20. │ Links.SpanId       │ Array(String)                              │ 4940078211 │ 0.00 B          │ 0.00 B            │                        0 │                          0 │              ᴺᵁᴸᴸ │
21. │ Links.TraceState   │ Array(String)                              │ 4940078211 │ 0.00 B          │ 0.00 B            │                        0 │                          0 │              ᴺᵁᴸᴸ │
22. │ Links.Attributes   │ Array(Map(LowCardinality(String), String)) │ 4940078211 │ 15.80 MiB       │ 36.81 GiB         │                        0 │                          8 │           2385.69 │
    └────────────────────┴────────────────────────────────────────────┴────────────┴─────────────────┴───────────────────┴──────────────────────────┴────────────────────────────┴───────────────────┘

Given the IDs are a fixed size (are them?) by using FixedString we should at least gain a bit of space avoiding delimiters no?

Navigation Menu

Repository navigation

Fixedstring type for SpanId and ParentId traces for better compression #3454

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Fixedstring type for SpanId and ParentId traces for better compression #3454

Description

Activity

MikeShi42 commented on Oct 23, 2026

czephyr commented on Oct 26, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions