configure_hypervisor: add rbd helpers for quadlet containers#986
Open
insatomcat wants to merge 1 commit into
Open
configure_hypervisor: add rbd helpers for quadlet containers#986insatomcat wants to merge 1 commit into
insatomcat wants to merge 1 commit into
Conversation
Context
-------
SEAPATH containers (Podman quadlets) that use Ceph RBD for persistent storage
must today embed a long sequence of ExecStartPre= and ExecStopPost= steps
directly in the quadlet file. A typical unit needs to:
1. Detect and clean up dirty state from a previous interrupted run
(unmount stale mountpoints, unmap all stale RBD device mappings).
2. Create the RBD image on first use (rbd create).
3. Map the block device (rbd map).
4. Create the mountpoint directory if absent.
5. Detect and format the device on first use (mkfs.ext4).
6. Mount the device.
7. On stop: unmount all stacked mounts and unmap all device mappings.
This boilerplate is identical across every container that uses RBD — only
the image name changes. Writing it by hand is error-prone (especially the
dirty-state cleanup) and produces quadlet files that are hard to read and
maintain.
What this commit adds
---------------------
Two bash scripts installed to /usr/local/bin/ by the configure_hypervisor role.
Both take only the image name; pool (rbd) and mountpoint (/mnt/rbd/<name>)
are fixed by convention.
seapath-rbd-mount <image-name> [<size>]
Idempotent mount script. Handles the full lifecycle:
- Dirty-state cleanup: loops umount until the mountpoint is fully clear
(interrupted previous runs can stack multiple mounts at the same path),
then enumerates ALL device nodes mapped to the image via rbd device list
and unmaps them. The stable symlink /dev/rbd/<pool>/<image> points only
to the most recent mapping; earlier stale mappings must be found by
iterating the device list.
- First-use image creation: calls rbd create with --image-feature
layering and the caller-supplied size (default 1G) only if rbd info
reports the image does not exist.
- Device mapping: rbd map creates both /dev/rbdN and the stable symlink
/dev/rbd/rbd/<image>.
- First-use filesystem: blkid is used to detect whether a filesystem
already exists. If not (freshly created image), mkfs.ext4 -q formats
it silently. Subsequent runs skip this step.
- Mount.
seapath-rbd-unmount <image-name>
Best-effort unmount/unmap script for ExecStopPost=. Loops umount until
the mountpoint is fully clear (handles stacked mounts from repeated
interrupted starts), then force-unmaps (-o force) all device nodes still
mapped to the image. Force-unmapping handles devices held by udevd,
blkid, or other kernel references after a container crash. Exits 0
regardless.
Usage in a quadlet
------------------
The ExecStartPre / ExecStopPost block in the quadlet file is reduced to:
ExecStartPre=/usr/local/bin/seapath-rbd-mount mycontainer
ExecStopPost=/usr/local/bin/seapath-rbd-unmount mycontainer
Any app-specific subdirectories that the container needs inside the
mountpoint are the caller's responsibility (mkdir -p in a separate
ExecStartPre= line) — they are inherently application-specific and out of
scope for a generic helper.
Deployment
----------
The two scripts are installed by a new rbd_helpers.yml task file included
from configure_hypervisor/tasks/main.yml, following the same pattern as
the other helpers in that role (ansible.builtin.copy, mode 0755).
The task is skipped on Yocto (seapath_distro != "Yocto"): the root
filesystem is read-only there, so the scripts must be shipped by the image
itself. Porting them to meta-seapath is required before quadlets relying
on these helpers can be used on Yocto.
Signed-off-by: Florent Carli <florent.carli@rte-france.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
SEAPATH containers (Podman quadlets) that use Ceph RBD for persistent storage must today embed a long sequence of ExecStartPre= and ExecStopPost= steps directly in the quadlet file. A typical unit needs to:
This boilerplate is identical across every container that uses RBD, only the image name changes. Writing it by hand is error-prone (especially the dirty-state cleanup) and produces quadlet files that are hard to read and maintain.
What this commit adds
Two bash scripts installed to /usr/local/bin/ by the configure_hypervisor role. Both take only the image name; pool (rbd) and mountpoint (/mnt/rbd/) are fixed by convention.
seapath-rbd-mount []
seapath-rbd-unmount
Usage in a quadlet
The ExecStartPre / ExecStopPost block in the quadlet file is reduced to:
ExecStartPre=/usr/local/bin/seapath-rbd-mount mycontainer
ExecStopPost=/usr/local/bin/seapath-rbd-unmount mycontainer
Any app-specific subdirectories that the container needs inside the mountpoint are the caller's responsibility (mkdir -p in a separate ExecStartPre= line) — they are inherently application-specific and out of scope for a generic helper.
Deployment
The two scripts are installed by a new rbd_helpers.yml task file included from configure_hypervisor/tasks/main.yml, following the same pattern as the other helpers in that role (ansible.builtin.copy, mode 0755).
The task is skipped on Yocto (seapath_distro != "Yocto"): the root filesystem is read-only there, so the scripts must be shipped by the image itself. Porting them to meta-seapath is required before quadlets relying on these helpers can be used on Yocto.