This commit adds support for providing custom CA certificates
during the build process. It allows a user to specify one ore
more CA file(s) via the new <certificates> section and/or
via the command line. The certificate files will be imported
directly after the image bootstrap has finished (where the
required CA update tools are available), but before any
further packages are retrieved, solving for situations where
the chroot environment needs certificates, e.g. when there's
a proxy server in the build environment.
Co-authored-by: Marcus Schaefer <marcus.schaefer@gmail.com>
Allow to specify a custom partition ID in the Disk
class interface and API. As of today the new parameter
partition_id in the ptable_entry_type record is not
in use by the current kiwi image description schema.
This change will be a followup to the API change and
needs further conversation upstream first. The code
changes here are based on the work from:
Co-authored-by: Caleb Wolfe <cwolfe651@gmail.com>
Co-authored-by: Caleb Wolfe (Amazon) <xcawolfe@amazon.com>
Assisted-by: Claude Opus 4.5 <noreply@anthropic.com>
This is related to #2917
The new root directory (/) for an image build is created
by taking the current system umask into account. However,
if that umask setting is not appropriate for the needs
of the image root directory some hard to debug after
effects can appear on the later run of the image binary.
As such this commit changes the permissions for the root
directory explicitly to the expected value.
This Fixes#2920
Add optional root_is_mountpoint parameter to the _call_script
method. When set the provided root path is turned into a bind
mount such that it appears as / being a mountpoint in the
chrooted call. Apply the argument to the call_disk_script
method such that is becomes effective for the disk.sh
script hook.
Some OS don't support the way systemd is dealing with keymaps (e.g.
Debian, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=790955).
If we try to set the keymap using `systemd-firstboot --keymap`, this
will fail, and the error message from `systemd-firstboot` is misleading:
it states that the keymap is not installed.
I couldn't find out how to reliably detect if systemd's keymap
management is supported, so instead, we provide a hint in the error
message if `systemd-firstboot --keymap` fails that the reason could be
that it's unsupported.
Fixes#2904
More generic keymap/systemd error message + dedent
Adds test of warning if keymap setting unsupported
Fix linter complaints about formatting
Run keymap check under chroot
We're interested in the capabilities of the target system, since we also
run `systemd-firstboot --keymap` in there.
Fixup for setup_test after chroot'ing localectl
Remove localectl call
It talks to systemd.
Fix tests for keymap
The partition id metadata file is used in the kiwi-repart
module. If a user wants to use the kiwi repart module
permanently, this metadata file needs to stay in the system.
Therefore it should not be automatically deleted by the
cleanup. A disk.sh hook script can be used to force the
deletion of the file though. This is related #2851
Allow to specify a filesystem label as part of a <partition>
definition. So far the label was set by the name of the
partition. With the new label attribute, a filesystem label
different from the partition name can be set. This commit
also updates/fixes the documentation in this regard.
Log the value of SDE so it is available to review,
even if the build system does not tell about it.
Update the tests to cover the new code-path.
Co-Authored-By: Marcus Schäfer <marcus.schaefer@gmail.com>
Make sure the final image rootfs does not contain unneeded
metadata files used during build time. The respective cleanup
call is performed after the root sync and after all initrd/boot
processing has been done. This is because up to that point it's
still possible that the information is required. This means
when building images with a read-only rootfs, it might not be
possible that the metadata can be deleted due to a chicken&egg
situation. Furthermore the cleanup is applied to the disk
builder only as other builders do not really suffer from
this data and for the container builder the metadata can
also be used for the stackbuild feature when building images
derived from containers. This Fixes#2668
Make sure to perform setfiles only on a writable target. In case
of a read-only root it is expected that the security context set
by kiwi in an earlier stage is complete. As there is no way to
modify data when root is read-only, there is also no way to change
the security context of any file such that we skip setfiles
in this case. Should there be a read-only system that has writable
partitions such as /boot and their content changes while the rest
of the root system is read-only it is in the responsibility of
the author of the image description to call setfiles only on
the affected and still writable files via a custom disk.sh
script. Along with the fix the respective integration test was
modified to enable selinux such that this change is actually
integration tested. This Fixes#2805
The ImageSystem.mount() method implemented its own handling
for mounting the volumes of a volume manager based system.
First and foremost this duplicates code that already exists
in the respective VolumeManager implementation and second
the code behaved wrong in case of btrfs when there is no
default subvolume configured
So far it was only possible to specify the size of the ramdisk
via the kernel commandline option: ramdisk_size. In a remote
deployment it was therefore required to carry this size as a
mandatory information to the deployment server. With this commit
we allow to specify the size for the ramdisk to be configured as
part of the image configuration which makes this information
also available inside of the initrd. If provided the ramdisk_size
kernel commandline option still takes precedence over the
<oem-ramdisk-size> setting to avoid any behavior change and to
still allow dynamic overrides of the ramdisk size.
In the effort of adapting to the latest snapper in Issue #2697
we overlooked the after effect of a different variable name
in the profile environment with regards to $kiwi_btrfs_root_is_snapshot
and $kiwi_btrfs_root_is_snapper_snapshot. Image builds that
references the former variable name would be broken by the change.
This commit makes sure no regression is introduced by providing
both variants. This Fixes bsc#1237772
Add a test case with absolute path in the target_dir
to make sure we never unpack the archive to the host
system. The actual issue was resolved together with
the implementation in #1953 and commit
78238a993c
This Fixes#2701
Rename btrfs_root_is_snapshot to btrfs_root_is_snapper_snapshot.
This happens in preparation for the changes suggested in #2697
where we want to get rid of snapper specific btrfs code which
will be available in snapper natively soon. To make sure a btrfs
layout specific to snapper(and SUSE), the implicitly used attribute
named btrfs_root_is_snapshot now becomes explicit and its new
name will indicate that snapper sits behind it. Along with the
rename a XSLT stylesheet to automatically convert the old name
into the new name for schema v8.3 will be performed.
Decommission the Checksum.md5() method and move all places
in code to sha256(). The md5 digest is considered insecure
and has also been removed from hashlib as a supported digest.
This Fixes#2696
Instead of providing a static list of filenames, provide a list
of package names. It is expected that the pilot of flake-pilot
resolves this list against the local package database to build
up the filelist for provisioning
The attribute provide_system_files creates a meta file in the
root tree named 'systemfiles'. The contents of this file were
produced by just a dump of the package database so far. For
a more generic use of this data some adaptions were needed.
First we allow to skip packages matching a pattern from being
part of the system files. Next we do not put ghost and doc
files into account. And last we handle library files in a different
file named 'systemfiles.libs' where we do not add symlink targets
if the target path is also part of the package. The consumer
of this information is flake-pilot which syncs that library system
files from the host via --copy-links. This allows a more generic
use with regards to versioned libraries e.g. libc
This has created havoc in the Fedora build environments by
fully unmounting /dev/pts and breaking the builders for
subsquent tasks.
This is a partial revert of commit daf1323c5d.
During runtime several kernel filesystems are bind mounted into
the image root system such that programs expecting it can work.
/dev/pts was not needed so far but seems to be a good addition
to the list to make tools like sudo to work properly when called
e.g. from a config.sh script. This Fixes#2686
Added the attributes provide_system_files and require_system_files
to control the provider and requester of system files in
container image builds. systemfiles is a metadata file which
contains all files from the package database at call time.
It is used in flake-pilot to provision the systemfiles data
from the host into the container instance. One possible use
case for this data is a flake registration which uses a
base container that is derived from a runtime container but
all data from the runtime should be provisioned from the
host. Using this feature tightly couples the flake to the
host OS distribution and version.
Add support for multi inheritance to the derived_from attribute
In the order of a comma seperated list of docker source URI's
a base tree is created. This was possible only with one container
so far and Fixes#2680 as well as jira#OBS-354
Some kiwi env vars are initialized with an empty value
and not overwritten if another value is provided. For
the selected variables an empty value setting is not
allowed because the schema also enforces the value to
be set at least once. In addition a helpful option
named --print-kiwi-env was added to the 'image info'
command which allows to print the environment variables
and their values.
Allow to specify references to OCI containers in the
image description like in the following example:
<containers source="registry.suse.com" backend="podman">
<container name="some" tag="some" path="/some/path"/>
</containers>
During the kiwi process the containers are fetched into a
temporary location and a systemd service is configured to
one time load the containers into the local registry at
first boot of the system. This Fixes#2663
When apt resolves packages on a multiarch repo it can happen
that dependencies for packages from other architectures are
pulled into the solver process but are not provided by any
repository. To overcome this behavior the repository can
be setup to serve packages only for a specified architecture
or list of architectures. This is related to
OSInside/kiwi-descriptions#102
There are descriptions out in the wild that use "non-numeric" versions
in their descriptions, particularly without separators for splitting.
This change switches all of this to strings rather than assuming
numbers and gracefully handles the single word case.
The former method provided a static value but there are huge
differences for the minimum size requirement of a filesystem.
For example extX is fine with 30MB whereas XFS requires 300MB.
This commit adds a more dynamic default value based on the
used filesystem.
So far 30MB was set as default volume size which is by far
too small for a number of filesystems, e.g btrfs and also XFS.
This commit increases the default volume size such that all
modern filesystems builds if the default volume size is used.
Usually custom files are managed by placing them as overlay
files or archives. However, overlay files must be structured
inside of a root/ subdirectory and archive files are binary
data. It is therefore not straight forward to just reference
one or more files as source files to the image description
to be placed into the image. This commit adds a new <file>
element which allows to do this. This Fixes#1953
The support for delta_root allows to build a delta container
image from a given base container. Due to the refactoring of
the kiwi code base using context managers no explicit deletion
of instances happens anymore. This uncovered a weakness of
the delta root code at the level of the overlay mount. At
the time of the umount there are still active temporary
mount handlers which keeps the mountpoint busy. In order to
fix this properly also the PackageManager factory is now
a context manager and the Repository factory received a
cleanup method which is called when the PackageManager goes
out of scope. This refactoring also fixes the busy state
when building deltas
Change the SystemPrepare class to context manager.
All code using SystemPrepare was updated to the following
with statement:
with SystemPrepare(...) as system_prepare:
system_prepare.some_member()
This completes the refactoring from finalizers to
context managers and Fixes#2412
Change the Repository Factory to be a context manager.
All code using Repository was updated to the following
with statement:
with Repository(...).new as repo:
repo.some_member()
This is related to Issue #2412
Command.run() currently has a bit of a confusing behavior: if raise_on_error is
False and the executable is not found, then a weird CommandT is returned (return
code is -1 and stdout+stderr is None). This makes it possible to hanlde command
not found errors separately, but it makes that needlessly verbose. So instead,
let's just return None in *this* special case.
That in turn uncovered, that in most cases when we set `raise_on_error=True`, we
actually want an error if the command is not present but no error if the command
fails to execute (e.g. because it returns -1 if you run `$cmd --version`). Hence we
introduce the flag `raise_on_command_not_found`, which causes an exception to
be raised if the command is not found. This makes it independent of the
`raise_on_error` flag.
Additionally, we add a small optimization: if command starts with /, then we
assume it's a full path and we omit the call to which (and just check whether it
exists).
Co-authored-by: Marcus Schäfer <marcus.schaefer@gmail.com>
Change the ImageSystem class to context manager.
All code using ImageSystem was updated to the following
with statement:
with ImageSystem(...) as image_system:
image_system.some_member()
This is related to Issue #2412
Change the OCI Factory to be a context manager.
All code using OCI was updated to the following
with statement:
with OCI(...).new as oci:
oci.some_member()
This is related to Issue #2412
The grub terminal setup is divided into the setting for the output
and the input console. For both settings different parameters exists.
So far kiwi did not differentiate between the two parts of the
console setup and that could lead to a wrong setting if only one
value is provided in kiwi's console= attribute which lead to the
grub setting, GRUB_TERMINAL=value. If value is set to e.g gfxterm
grub takes this for both input and output and it's obviously
wrong for the input. To make this less error prune the kiwi code
changes with this commit to set GRUB_TERMINAL_INPUT and
GRUB_TERMINAL_OUTPUT rather than GRUB_TERMINAL and also runs sanity
checks on the provided values if they are applicable. The information
for setting up the console in the schema stays untouched though.
That's because it's used for all bootloaders and also because grub
supports multiple values for the console in/out setting in one
GRUB_TERMINAL variable even though kiwi does no longer use it.
To make this clear for the users also the documentation for the
console attribute setup has been updated. If we want to wish two
distinct attributes for input and output console settings a schema
change and also differentiation between bootloaders is needed and
that I only see for the kiwi-10 branch if at all. This Fixes#2419
setup_selinux_file_contexts is now called after the config.sh
script. This makes sure that eventual policy related changes
done in the optional config.sh are covered by a late setfiles
call. In addition setup_selinux_file_contexts is called again
at the end of any chroot based script hook. So we assume that
any optional script target can change the system in a way that
a new setfiles call might be required. It can happen that
setfiles is called more often than required but as we cannot
know what custom scripts does, it's better to call it more
often compared to not often enough. This Fixes bsc#1210604
As part of the profile environment only the kiwi_install_volid
variable existed with the default value 'INSTALL'. This information
is correct when building an oem image with installiso set to
true. The resulting .install.iso image has the configured volid
or the mentioned default 'INSTALL'. However, when building a live
ISO image the default volid value is set to 'CDROM' but the profile
environment contains kiwi_install_volid which is misleading and
in case no volid is provided in the configuration file, the
default value would also be a wrong information. This commit cleans
up the mess and provides kiwi_install_volid for install ISO media
and kiwi_live_volid for Live ISO media with their respective
default values
