[Security] Image Layer Cache Poisoning via Unverified DiffID Keying

This is a result of an internal vuln finding run. Many of these are known issues, but don't appear to have tracking issues, so I'm filing them anyway.

[Security] Image Layer Cache Poisoning via Unverified DiffID Keying

Severity: HIGH (4.48) | CWE: CWE-345

Problem Description

atelet unpacks each container image layer into a node-wide shared pool directory keyed by the layer's unverified diffID (internal/imagecache/layerpool.go).

During unpacking:

If a layer directory matching the diffID already exists on disk, atelet skips unpacking and reuses the existing cached layer directory.
The contents of the cached directory are not cryptographically re-verified, nor is the mapping between layer digest and unpacked content authenticated before mounting.

Attack Vector

A malicious tenant can publish an image with a collided or manipulated layer:

Tenant uploads an image that declares a common diffID but contains malicious backdoored binaries or files.
When the node pulls this image, it populates the shared layer cache for that diffID.
When another tenant or system service later runs an image sharing that same diffID, atelet mounts the poisoned cached directory.
The victim workload executes attacker-controlled code or libraries from the poisoned cache.

Impact

Cross-tenant and node-wide image layer cache poisoning leading to code execution inside victim containers sharing common base image layers.

Proposed Remediation

Cryptographically verify unpacked layer content against secure cryptographic digests (e.g. SHA-256 chain) before mounting.
Isolate layer caches per tenant or atespace rather than maintaining a shared node-wide layer directory.
Validate permissions and integrity of cache directories using read-only mounts.

On first read this sounded like it required actortemplate control. But gemini convinced me it still matters:

Even if operational policy restricts ActorTemplate creation exclusively to cluster admins, the issue breaches fundamental multi-tenancy invariants:

Cross-Template / Co-Tenant Blast Radius Collapse: If an admin configures Template A for an untrusted/partner agent (Image A) and Template B for a high-security internal agent (Image B, based on python:3.11), running both on the same node allows Image A to silently poison the shared layer cache for Image B. Kernel isolation (gVisor/microVM) is bypassed because the victim mounts a corrupted rootfs lowerdir before the sandbox executes.
Node Infrastructure Poisoning (pause image): atelet pulls the infrastructure pause image into the exact same node-wide pool (cmd/atelet/main.go:1571). An image claiming the pause image's diffID poisons the init/reaper container across all actors on that node.
Upstream Supply Chain / Tag Mutability (A7 Persona): Admins commonly reference public or vendor base images. If an upstream registry or image tag is tampered with to declare false diff_ids, pulling that image poisons standard base layers for all other templates on the node.

Remediation Strategy

This is gemini's proposed fix, which avoids reading the same bytes twice and should be fairly performant:

In internal/imagecache/imagecache.go:678-740, stream the uncompressed tar through a SHA-256 hasher during unpack:

hasher := sha256.New()
tr := io.TeeReader(rc, hasher)
cr := &countingReader{r: tr}
wh, err := unpackLayer(ctx, cr, root)
if err != nil {
    return err
}

actualHash := hex.EncodeToString(hasher.Sum(nil))
if actualHash != diffID.Hex {
    return fmt.Errorf("unpacked layer hash sha256:%s does not match claimed diffID %s", actualHash, diffID)
}

If the hash does not match, the temporary unpack directory is discarded immediately, aborting the pull and preventing any unverified files from entering the shared node pool.

igooch

igooch Could you please take this one given your contributions to internal/imagecache?

Proposal: verify layers before they enter the shared pool

The problem. The layer pool is shared by every image on a node and keyed by diffID. Today we never check that a layer's content matches the diffID it's stored under, so whichever image unpacks a diffID first decides what every later image with that diffID mounts. Both gaps reproduce on main:

The diffID is never checked. It's taken from the image config, which the image's author controls, and the comparison at imagecache.go:617-621 checks the config against itself. A malicious image can claim a popular base layer's diffID and plant its own content.
The compressed digest isn't checked either. go-containerregistry checks it only at EOF, and our untar stops before EOF. So we don't even confirm that the registry served the bytes the manifest names.

containerd, containers/image, and moby all treat the config's diffIDs as untrusted until they've hashed the content themselves. We should too.

PR 1 (issue fix): verify on write

The goal is that every layer in the pool holds content we hashed ourselves, so reusing a layer by diffID is safe.

Read every layer stream to EOF, so the existing compressed digest check runs and a tampered blob is rejected.
Compare the uncompressed hash to the claimed diffID before the layer becomes visible. An image that lies about its diffID then fails its own pull instead of poisoning the pool.
Stop trusting layers cached before this change, since none of them were checked (see the open question).

Cache hits are unchanged. Misses pay one SHA-256 pass alongside decompression.

PR 2 (additional hardening): reuse a layer only for blobs we've verified

PR 1 checks an image's content only when it downloads it. A node that already has the claimed layer reuses it without downloading anything, so an image whose content doesn't match its diffID fails on some nodes and runs on others.

PR 2 records, for each cached layer, which compressed blobs have been verified to produce it. When an image's blob isn't on that list, the node downloads and hashes the blob (without unpacking it) before reusing the layer. An image whose content doesn't match its diffID then fails on every node, which is what containers/image and moby do.

Open question

Layers already cached were never verified, so they can't be trusted after this change. Do we support in-place atelet upgrades before v1.0.0?

No: bump the layout version, and upgrading requires a reinstall (Reclaim actor node state when a worker pod goes away #1706's uninstall wipe makes that clean).
Yes: skip the bump and replace old layers with verified copies as images need them.

Navigation Menu

Repository navigation

[Security] Image Layer Cache Poisoning via Unverified DiffID Keying #2082

[Security] Image Layer Cache Poisoning via Unverified DiffID Keying

Problem Description

Attack Vector

Impact

Proposed Remediation

Remediation Strategy

Proposal: verify layers before they enter the shared pool

PR 1 (issue fix): verify on write

PR 2 (additional hardening): reuse a layer only for blobs we've verified

Open question

Related

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Security] Image Layer Cache Poisoning via Unverified DiffID Keying #2082

Description

[Security] Image Layer Cache Poisoning via Unverified DiffID Keying

Problem Description

Attack Vector

Impact

Proposed Remediation

Activity

destijl commented on Oct 1, 2026

Remediation Strategy

zlammerts-svg commented on Oct 2, 2026

igooch commented on Oct 5, 2026

Proposal: verify layers before they enter the shared pool

PR 1 (issue fix): verify on write

PR 2 (additional hardening): reuse a layer only for blobs we've verified

Open question

Related

BenTheElder commented on Oct 6, 2026

zlammerts-svg commented on Oct 16, 2026

BenTheElder commented on Oct 20, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions