Should actor rootfs be read-only? #235

Raised in the review discussion on #184 (#184 (comment)): should sandboxed actor workloads get a read-only root filesystem?

atelet currently builds actor OCI specs with Root.Readonly: false. If we harden this, a few things need working through:

• Workloads that write scratch data (/tmp, /run, app-specific paths) would need writable tmpfs mounts in the spec; which paths to provide by default needs deciding.
• The actor identity directory is unaffected: it is already its own read-only bind mount at /run/ate, independent of rootfs writability.
• Interaction with checkpoint/restore needs checking: gVisor checkpoints include filesystem deltas, and tmpfs contents are part of sentry memory, so the snapshot impact of a read-only rootfs plus tmpfs scratch needs verifying.
• Worth deciding alongside the broader projection design in Add ActorID to the injected actor's process #178, since both shape what the guest filesystem contract looks like.

Counterpoints worth weighing:

• Each actor's rootfs is already private and freshly extracted per restore, behind the sandbox. A read-only rootfs is defense in depth against in-guest tampering and persistence, not host protection, so the win is smaller than in unsandboxed runtimes.
• Kubernetes makes readOnlyRootFilesystem a per-container opt-in because many images expect writable paths. An ActorTemplate field may fit better than changing the default for all workloads.
• The golden-snapshot pattern encourages expensive init before checkpoint, and init that writes to the filesystem is a legitimate form of that. Forcing those writes into tmpfs moves them from filesystem deltas into the memory image, with unverified snapshot-size consequences.

Related: #220 and #232 (working-space and external volume mounts), which a read-only rootfs would turn from convenience into requirement.