Protecting /var and /etc from drift

In an atomic system based on bootc, the file system is divided into immutable (/usr) and persistent (/var, /etc) parts. The /var directory is not updated with the image — it is preserved as-is between updates. The /etc directory is managed via a 3-way merge, but not all changes are guaranteed to be applied.

This creates the problem of drift — the real state of the file system diverging from what the image expects.

Why drift occurs

The ALT Linux Sisyphus repository is a classic distribution not designed for atomic updates. Most packages do not ship tmpfiles.d configs for their directories and files. On a regular system this is not a problem — the package manager creates all the necessary files during installation. But in an atomic system:

• /var is not updated when the image changes. If a new package in the image expects a directory in /var and it is not there, the service will not start.
• /etc goes through a 3-way merge during the update, but destructive actions may have been performed on the files by the user.
• Permissions and owners can drift apart: for example, the GID of a group changed in the new image, but files in /var still have the old owner.

Besides updates, drift can occur due to manual user actions or failures.

How APM solves the problem

APM includes a lint module that scans the image file system and generates declarative tmpfiles.d configs. On every boot, systemd applies these configs, bringing /var and /etc to the declared state.

What is generated

The command apm system image lint --fix (hidden command, executed automatically during image build, do not run on a live system!) analyzes rootfs and creates /usr/lib/tmpfiles.d/apm-lint.conf with three types of entries:

• d — directories with permissions and owner. If the directory is missing, systemd will create it at boot.
• L — symlinks with the correct target. If the symlink is missing or points to the wrong place, systemd will restore it.
• z — permissions and owner for existing files. If the file exists but the permissions or owner differ, systemd will fix them.

Example of a generated config

# Auto-generated by apm lint
d /var/lib/chrony 0755 _chrony _chrony - -
d /var/lib/libvirt/qemu 0750 _libvirt vmusers - -
d /var/spool/cron 0730 root crontab - -
L /etc/mtab - - - - /proc/mounts
L /etc/init.d - - - - rc.d/init.d
z /etc/openssh/sshd_config 0600 root root - -
z /etc/shadow 0400 root root - -

What is not covered

The z type does not create files — it only sets permissions on existing ones. If a configuration file is missing from /etc, tmpfiles.d will not restore its contents. Such files must be shipped by the package and restored via the ostree 3-way merge. However, configs from /etc that are part of the image are always available on the live system and are located in /usr/etc; this can be considered the reference state. In the future, we will write full recovery tools for /etc.

During image build

When building an image via apm system image apply, lint is run automatically — the config is generated and included in the image. On a live system, nothing needs to be run manually.

Applying at boot

On every boot, systemd automatically executes systemd-tmpfiles --create, which reads all configs from /usr/lib/tmpfiles.d/, including apm-lint.conf. This happens at an early stage of boot.

Exceptions

Some directories are intentionally excluded from recursive analysis because their contents are managed by other mechanisms:

Directory	Reason
/var/home, /var/root	User home directories
/var/tmp	Temporary files
/var/cache/apt	Package manager cache
/etc/rc.d	SysVinit symlinks, managed by chkconfig
/etc/tcb	Managed by the authentication system
/etc/alternatives/links	Managed by the alternatives system
/etc/skel*	Templates for creating users