feat(view): add per-container snapshot views for boot artifacts#639
feat(view): add per-container snapshot views for boot artifacts#639sidneychang wants to merge 5 commits into
Conversation
✅ Deploy Preview for urunc ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
b529bb2 to
af30554
Compare
549974e to
731f713
Compare
8b4d20d to
ef929cd
Compare
cmainas
left a comment
There was a problem hiding this comment.
Thank you @sidneychang for this PR. I have added several comments regarding the urunc side and let's go together over the shim changes in today's sync.
| func (u *Unikontainer) Exec(metrics m.Writer) error { | ||
| metrics.Capture(m.TS15) | ||
|
|
||
| // Reload annotations written by the shim after Create. |
There was a problem hiding this comment.
Why do we need to reload the spec? The urunc process starts after shim has done its work.
There was a problem hiding this comment.
I further validated this with a real container run after removing the runtime reload and rebuilding. The result shows that the reload is required rather than redundant.
The key reason is that the shim patches the bundle config.json only after the inner Create returns:
func (s *taskService) Create(ctx context.Context, r *taskAPI.CreateTaskRequest) (*taskAPI.CreateTaskResponse, error) {
resp, err := s.TaskService.Create(ctx, r)
if err != nil {
return resp, err
}
if err := containerdShim.PatchConfigJSON(r.Bundle, shimAnnotations); err != nil {
return nil, err
}
return resp, nil
}This means runtime/reexec has already loaded and kept an in-memory spec, namely u.Spec, before the shim persists com.urunc.internal.rootfs.params into the bundle config.json. The persisted config.json is updated successfully, but u.Spec is not automatically refreshed.
I also added a diagnostic log at the exact place where Exec() consumes the annotation:
if rootfsParamsJSON := u.Spec.Annotations[annotRootfsParams]; rootfsParamsJSON != "" {
uniklog.WithField("len", len(rootfsParamsJSON)).
Debugf("runtime: found %s in in-memory spec", annotRootfsParams)
} else {
uniklog.WithField("present", false).
Warnf("runtime: missing %s in in-memory spec", annotRootfsParams)
}The runtime logs confirm this ordering:
urunc(shim): bundle spec missing com.urunc.internal.rootfs.params before persist
runtime: missing com.urunc.internal.rootfs.params in in-memory spec
I also checked the bundle config.json for the same container and confirmed that com.urunc.internal.rootfs.params was present there. Therefore, the patch itself succeeded; the issue is that runtime/reexec was still using a stale in-memory spec.
So the runtime reload is needed here to make runtime/reexec reload the patched config.json, ensuring that Exec() can see com.urunc.internal.rootfs.params.
| return types.RootfsParams{}, "", err | ||
| } | ||
|
|
||
| encoded, err := json.Marshal(rootfsParams) |
There was a problem hiding this comment.
If we refactor this function like that. There is no reason to create the json inside here. We can do it later. In that way we do not need to return an extra value.
| if err := b.rebindRootfsViewBootAfterPrepareRoot(); err != nil { | ||
| return fmt.Errorf("boot artifact setup after prepareRoot failed: %w", err) | ||
| } | ||
| } |
There was a problem hiding this comment.
This should be moved in the respective setup step for the block based rootfs.
| } | ||
| } | ||
|
|
||
| if uerr := mount.Unmount(mountpoint, 0); uerr != nil && !os.IsNotExist(uerr) && uerr != unix.EINVAL { |
There was a problem hiding this comment.
Please simplify this line.
| } | ||
|
|
||
| func rootfsViewRelPath(p string) string { | ||
| return strings.TrimPrefix(filepath.Clean(p), "/") |
There was a problem hiding this comment.
Please use full paths for bind mounts.
|
|
||
| // probeRootfsViewBootArtifacts keeps the legacy extract fallback available: | ||
| // preSetup still has mountedPath, but does not keep boot bind mounts. | ||
| func probeRootfsViewBootArtifacts(rootfsViewState *rootfsViewState, unikernelPath, initrdPath, uruncJSON string) (useView bool, err error) { |
There was a problem hiding this comment.
I do not really understand the purpose of this function. Can you help me?
|
|
||
| // rebindRootfsViewBootAfterPrepareRoot binds boot artifacts into the rootfs | ||
| // tree that qemu sees after chroot. | ||
| func (b blockRootfs) rebindRootfsViewBootAfterPrepareRoot() error { |
There was a problem hiding this comment.
THe logic of this fucntion should be part of one of the setup steps for block based rootfs.
|
|
||
| // prepareRootfsViewBootBinds runs after prepareRoot, so the binds live in the | ||
| // monitor mount namespace and are released with it. | ||
| func prepareRootfsViewBootBinds(rootfsViewState *rootfsViewState, monRootfs, unikernelPath, initrdPath, uruncJSON string) (useView bool, err error) { |
There was a problem hiding this comment.
Do not use booleans as a return value for the success or failure of a function. Use errors instead
|
|
||
| bindErr := bindBootArtifactsFromView(mountpoint, monRootfs, unikernelPath, initrdPath, uruncJSON, &bindTargets) | ||
|
|
||
| uerr := mount.Unmount(mountpoint, 0) |
There was a problem hiding this comment.
We can not unmount the source of the files we later bind mount. This may lead to corrupted data for the boot files.
| return files | ||
| } | ||
|
|
||
| func bindBootArtifactsFromView(viewRoot, monRootfs, unikernelPath, initrdPath, uruncJSON string, bindTargets *[]string) error { |
There was a problem hiding this comment.
This function could simply be part of the caller and hence avoid the bindTarget hack.
23e52a1 to
f281afe
Compare
Add a rootfs_view.enabled configuration knob and document it. The flag keeps rootfs view preparation disabled unless deployments explicitly opt in. Signed-off-by: sidneychang <2190206983@qq.com>
Add bundle state for shim-prepared rootfs views and containerd helpers to prepare and clean those views. Keep the accessor internal so shim code only passes a session and persisted cleanup state. Signed-off-by: sidneychang <2190206983@qq.com>
Load shim-prepared rootfs view state for block rootfs setup. Probe the view before unmounting the container rootfs, then bind the boot artifacts after prepareRoot in the block postSetup step. Signed-off-by: sidneychang <2190206983@qq.com>
Choose guest rootfs parameters after inner task creation and persist them for runtime Exec. When enabled, prepare a rootfs view during Create, roll it back on persistence failures, and clean it during Delete. Signed-off-by: sidneychang <2190206983@qq.com>
Wire a custom shim manager Stop path that reads persisted rootfs view state from the bundle and removes the view snapshot and lease. This covers cleanup paths where task Delete is not the final teardown hook. Signed-off-by: sidneychang <2190206983@qq.com>
Description
Add a shim-managed per-container snapshot-view path for block-backed rootfs setups so urunc can reuse a prepared read-only view of the container image when retrieving boot artifacts.
The shim now wraps task Create/Delete to prepare a snapshot view ahead of container startup, persist the view metadata and mounts into the bundle, and clean up the containerd view and lease during deletion. On the runtime side, unikontainers consume that shim-written state to bind the unikernel binary, initrd, and
urunc.jsonfrom the prepared view into the monitor rootfs, while keeping the legacy extraction path as a fallback when no per-container view is available.The PR also documents the new
com.urunc.unikernel.snapshotViewruntime annotation and its interaction withmountRootfsfor supported block snapshotters.Related issues
How was this tested?
LLM usage
Codex
Checklist
make lint).make test_ctr,make test_nerdctl,make test_docker,make test_crictl).