[Bug]: microVM: an actor that writes more than its memory limit is killed, though its processes use a few MiB #3720

What happened?

On microVM, an actor is killed when the page cache of the files it writes fills the allocated memory.

Note that I couldn't reproduce the issue using only reads. The writes were necessary.

E.g If an actor that's allocated 1GB and writes 2GB to the filesystem then it gets killed and atelet logs show this after about 25 secs:
Killing a microVM over its memory limit.

The actor then goes to ACTOR_STATE_CRASHED with killed for exceeding its memory limit.

Expected Behavior

The actor shouldn't be killed. Page cache should be given back and not count against an actor. An actor should be killed only when its processes, cloud-hypervisor or virtiofsd use more than is allocated.

Steps to Reproduce

1. Deploy Substrate on kind: go run ./cmd/ate-setup --kind deploy ate-system

2. Create a micro-VM worker pool with one worker.
   Save this as workerpool.yaml, and apply it from the Substrate checkout, with your registry:
   KO_DOCKER_REPO=<registry> hack/run-tool.sh ko apply -f workerpool.yaml.

   apiVersion: v1
   kind: Namespace
   metadata:
     name: foobar
   ---
   apiVersion: ate.dev/v1
   kind: WorkerPool
   metadata:
     name: foobar
     namespace: foobar
     labels:
       workload: foobar
   spec:
     replicas: 1
     workerImage: ko://go.ate.dev/substrate/cmd/ateworker-microvm
     sandboxClasses:
     - name: microvm

3. Save this ActorTemplate as writer.yaml.
   It runs busybox 1.37 with a 1 GiB limit and a durable directory, and writes a 2 GiB file again
   and again.
   It waits 15 s first, so that the template's first snapshot holds none of the writing.

   metadata:
     atespace: write-repro
     name: writer
   workerSelector:
     matchLabels:
       workload: foobar
   containers:
   - name: writer
     image: docker.io/library/busybox@sha256:a7dbc319b1438115a207e11930dc4bb3e643dcb3ec0bdd694bb36cd8fad289ec
     command:
     - sh
     - -c
     - |
       mkdir -p /data/www && echo ok > /data/www/index.html && httpd -p 80 -h /data/www
       sleep 15
       while true; do dd if=/dev/zero of=/data/big bs=1M count=2048; done
     volumeMounts:
     - name: data
       mountPath: /data
     wakeupProbe:
       httpGet:
         path: /
         port: 80
   volumes:
   - name: data
     durableDir: {}
   resources:
     limits:
       cpuMillicores: 1000
       memoryBytes: 1073741824

4. Create and resume the actor:

   kubectl ate create atespace write-repro
   kubectl ate create actor-template -f writer.yaml
   kubectl ate create actor writer-1 -a write-repro --template writer
   kubectl ate resume actor writer-1 -a write-repro

5. Watch it: kubectl ate get actor writer-1 -a write-repro.
   About 30 s after the resume, the state is ACTOR_STATE_CRASHED, and
   kubectl ate get actor writer-1 -a write-repro -o yaml shows
   message: killed for exceeding its memory limit.

6. The control: replace the while line with these two lines, and the actor runs on.
   The file is written once with direct I/O, past the guest's cache, and then only read.

   dd if=/dev/zero of=/data/big bs=1M count=2048 oflag=direct
   while true; do cat /data/big > /dev/null; done

Sandbox Runtime

microVM (kata / cloud-hypervisor)

Agent Substrate Version / Commit SHA

v1.0.0 / b34fbee7

Kubernetes Version & Environment

kind v0.33.0 on rootless Podman 5.7.0, Kubernetes v1.37.0

Host OS & Architecture

Ubuntu 26.04.1 LTS, Linux 7.0.0-38-generic (x86_64)

Relevant Logs and Diagnostic Output

# atelet killed actor (from the worker pod's log):

{"time":"2026-11-07T01:08:57.884037854Z","level":"WARN","msg":"Killing a microVM over its memory limit","ate.atespace":"guest-cache-repro","ate.actor.name":"reader-1","ate.actor.uid":"5b92e258-3ec0-425c-bf04-6ae431ae2ec8","ate.template.atespace":"guest-cache-repro","ate.template.name":"reader","memoryLimitBytes":1086324736,"ate.stats.memory.usage":1094565888}
{"time":"2026-11-07T01:08:58.029735927Z","level":"ERROR","msg":"Failed to kill a microVM over its memory limit","ate.atespace":"guest-cache-repro","ate.actor.name":"reader-1","ate.actor.uid":"5b92e258-3ec0-425c-bf04-6ae431ae2ec8","ate.template.atespace":"guest-cache-repro","ate.template.name":"reader","memoryLimitBytes":1086324736,"ate.stats.memory.usage":1094565888,"err":"while waiting for cgroup \"/sys/fs/cgroup/5b92e258-3ec0-425c-bf04-6ae431ae2ec8\" to empty: open /sys/fs/cgroup/5b92e258-3ec0-425c-bf04-6ae431ae2ec8/cgroup.events: no such file or directory"}


# actor lines just before the kill:

01:08:52 start: write, 2048 MiB
01:08:55 pass 1: used 27 MiB, cache 930 MiB of 982

Additional Context

Hypothesis:

1. The microVM gets as much RAM as the actor's memory limit.
2. The worker kills the actor when its memory on the host exceeds the limit + some margin.
3. The worker counts the VM's RAM as the actor's memory, whatever the guest uses it for.
4. When the actor writes files, its memory on the host reaches the limit although the guest's processes only used a few MBs. The file cache dominates the RAM.

Possible fix:
The guest should get less RAM than the limit so its kernel drops the cache before hitting the cgroup's memory.high.

Related:

• A microVM actor using more host memory than it booked no longer takes down its worker #3429 added the code that kills the actor.
• [bug-bash] microVM: OOM kill inside the guest is silent, actor stays RUNNING #3615: an OOM kill inside the guest is slient. This is the opposite case.
• ateom-microvm: Tune guest memory & vCPUs #1724: tune the guest's memory and vCPUs.
• OOMKill can take down entire worker #3092: an OOM can take down entire worker.
• keep a microVM container's rootfs upper on a virtio-blk ext4 disk #3153: keep a microVM container's rootfs upper on a virtio-blk ext4 disk

Confirmation

• I have searched existing issues and verified that this is not a duplicate.
• I have verified that this issue occurs on the latest commit on main.