Change the DiskFormat Factory to be a context manager.
All code using DiskFormat was updated to the following
with statement:
with DiskFormat(...).new as disk_format:
disk_format.some_member()
This is related to Issue #2412
Change the VolumeManager Factory to be a context manager.
All code using VolumeManager was updated to the following
with statement:
with VolumeManager(...).new as volume_manager:
volume_manager.some_member()
This is related to Issue #2412
Change the Disk class to be a context manager.
All code using Disk was updated to the following
with statement:
with Disk(...) as disk:
disk.some_member()
This is related to Issue #2412
Change the BootLoaderConfig class to be a context manager.
All code using BootLoaderConfig was updated to the following
with statement:
with BootLoaderConfig.new(...) as bootloader_config:
bootloader_config.some_member()
This is related to Issue #2412
In preparation to further context manager related changes
in VolumeManager, LuksDevice, RaidDevice and more the
disk builder code which uses these classes needs to be
refactored beforehand to allow switching to context
manager based cascading of the storage device classes.
This commit does the refactoring and is related to
Issue #2412
Change the FileSystem class to be a context manager. All code using
FileSystem was updated to the following with statement:
with FileSystem.new(...) as filesystem:
filesystem.some_member()
This is related to Issue #2412
Change the LoopDevice class to be a context manager.
All code using LoopDevice was updated to the following
with statement:
with LoopDevice(...) as loop_provider:
loop_provider.some_member()
This is related to Issue #2412
A root (/) entry in fstab for btrfs is only required if no
default volume is configured. This commit adapts the code
path which adds an fstab entry for (/) to be effective only
if btrfs_set_default_volume is set to false.
Additionally, the documentation around btrfs_set_default_volume
was clarified to describe the behavior the switch triggers.
This Fixes#2366
In case of btrfs and if btrfs_set_default_volume is explicitly
switched off, we create the correct rootflags= kernel cmdline
entry to tell the system about the root volume for booting
The keyfile path was not correctly set in etc/crypttab which
caused systemd not being able to read the keyfile, consequently
asking for a passphrase. This commit fixes the writing of the
crypttab and also fixes a python name clash with the "os"
namespace.
In some configuration (eg on a transactional system), the root
file system is read-only, which means we will not be able to remove
the stale .root.keyfile after firstboot.
This patch changes the location of the keyfile and places it in
/root/.root.keyfile instead, because root's home directory is much
more likely to be read/write.
Also fixed the associated unit test (thanks to schaefi).
Signed-off-by: Olaf Kirch <okir@suse.com>
In case an alternative partition table start sector is
configured, the check to effectively apply it should be
based on the partition table type not on the firmware
name
The device selection for the read-only and read-write devices
in an overlayroot setup was implicitly done in the kiwi-overlay
dracut module by reading the root= information. This is a
concept which barely works and should be refactored in a way
that the root= information always points to the root device
and that an eventually existing write location is explicitly
provided via the existing rd.root.overlay.write parameter
which is allowed to be overwritten for providing an alternative
write location. Making this information consistent and explicit
on the kernel commandline simplifies the dracut code to
consume this information correctly. This Fixes#2251
Allow to specify an optional <bootloadersettings> element
inside of the <bootloader> section. The information is used to
specify custom arguments for the tools called in a bootloader
setup procedure, e.g shim-install, grub-install or grub-mkconfig.
kiwi does not judge on the given parameters and if the provided
data is effectively used depends on the individual bootloader
implementation. In this commit the data will be effectively
handled if bootloader="grub2" is configured. More precisely
the custom additions to support SUSE's TPM model with grub2
can be configured as follows:
<bootloadersettings>
<shimoption name="--suse-enable-tpm"/>
<installoption name="--suse-enable-tpm"/>
<configoption name="--set-trusted-boot"/>
</bootloadersettings>
This Fixes#2224
So far the last partition (typically root) in the partition table
takes all the rest space of the partition table in the image file.
At deployment/boot time users had several options to let that
partition grow to a custom size. However, during build time of the
image it was not possible to specify a specific fixed size for the
root partition as we don't wanted to produce gaps of unpartitioned
space in the image file. It has turned out that there is hardware
available which requires a partition to be an exact multiple of
some blocksize. As kiwi supports size constraints for all other
partitions but not for root this commit now allows for it. The
oem-systemsize element is now also taken into account at build
time of the image if the oem-resize which would do that at
deployment/boot time is switched off like the following example
shows:
```xml
<oemconfig>
<!-- set root partition to 2048MB -->
<oem-systemsize>2048</oem-systemsize>
<oem-resize>false</oem-resize>
</oemconfig>
```
This Fixes#2203
Add a new attribute integrity_legacy_hmac="true|false" which
allows to use old flawed HMAC calculation (does not protect superblock).
Add a new attribute integrity_legacy_padding="true|false" which
allows to use inefficient legacy padding. Do not use these attributes
until compatibility with a specific old kernel is required!
In case a clone should be created from a partition we need
to make sure to umount the device after sync and prior cloning.
Otherwise the clone operation is not safe because the rsynced
data might still be in memory and not synced out to the block
device.
The attribute disk_start_sector allows to specify a custom
start sector for the first partition of the disk. On GPT
tables everything works nicely, on DOS tables the used tools
fdisk/sfdisk are not able to manage the start/end values of
subsequent partitions if the first partition doesn't start
with the tooling default. This patch allows to set the
start sector after the partition table has been created
When using squashfs in a custom partitions setup like the following:
<partitions>
<partition ... filesystem="squashfs"/>
</partitions>
The build fails because the filesystem needs to be created
using the create_on_file() API and not the create_on_device()
API. In addition the size estimation is bogus when using
squashfs and cannot be pre-calculated because we only know
how much space the filesystem really needs after mksquashfs
as worked on the data and the compression. Thus this commit
also relaxes the required size check in case of squashfs.
Last but not least a squashfs filesystem does not provide
label or UUID and can only be referenced by the PARTUUID
it gets dumped on or by the native unix device node. As
the unix node is a loop during build time of the image and
meaningless this commit also forces by-partuuid mapping in
fstab when mounting the squashfs based device.
Support creating block level clones of certain partitions
used in the image. Clones can be created from the root, boot
and any partition listed in the <partitions> element.
There is support in kiwi to use dm_integrity in combination
with the LUKS header and dm_crypt. However there is also the
use case to setup dm_integrity in standalone mode. This commit
allows to create the dm_integrity layer outside of LUKS using
/etc/integritytab to activate the map through a systemd
generator if systemd is used.
Regarding systemd it's required to use a version of system which
provides: system-generators/systemd-integritysetup-generator.
If this generator does not exist in the distribution it will
also be missing in the dracut generated initrd and the boot
will not be able to succeed. It's mentioned here because even
newer distributions might be missing the generator
Along with the implementation there are two new optional
attributes in the <type> section:
standalone_integrity="true|false"
embed_integrity_metadata="true|false"
standalone_integrity activates/deactivates the dm_integrity map
on top of the root filesystem. Similar to the veritysetup support
there is the opportunity to create an embedded magic metadata
block at the end of the device containing the root
filesystem via embed_integrity_metadata
The Disk class provides methods to create partition(s)
and map names according to its scope and independent of
the actual partition tools. For example: create_root_partition().
This commit adds an additional optional clone parameter to
all methods for which we want to allow partition clones
Specifies to write a binary block at the end of the
partition serving the root filesystem, containing information
for dm_verity verification and to construct the device map
In addition to by-label and by-uuid also support mounting
by PARTUUID. Please note kiwi also makes sure that the grub
generated config file uses the root=PARTUUID= notation and it's
not clear if grub-mkconfig will persist making use of it.
Nevertheless there are also systems which uses different
methods to boot and it makes sense to support partuuid
mappings as well
Added new luksformat element which allows to pass
along options to the luksFormat call. This allows users to
switch between LUKS and LUKS2 via e.g
<luksformat>
<option name="--type" value="luks2"/>
</luksformat>
It also allows to pass along a set of options only available
to LUKS2, e.g
<luksformat>
<option name="--type" value="luks2"/>
<option name="--cipher" value="aes-gcm-random"/>
<option name="--integrity" value="aead"/>
</luksformat>
In addition to the new attribute the existing luks attribute
can also be specified to read credentials from a keyfile by
using the file:// source locator, e.g
<type ... luks="file:///path/to/a/keyfile"/>
This Fixes#1898
So far the verity support was only available with the
overlayroot layout and the read-only squashfs root. This
commit adds a new attribute: verity_blocks="number|all"
which allows to create the verity setup also on the
standard root partition
In addition to the change it was needed to extend the
Filesystem API with an additional optional paramter to
allow setup of the filesystem UUID. Having the opportunity
to set the UUID at filesystem creation is generally useful
and with regards to this particular change it became also
required
veritysetup was called as part of the disk builder. However,
the veritysetup should be a responsibility of the classes which
implements the sync_data method. This allows to use the creation
of a verity hash format right after sync as a feature to these
classes and in a broader scope. In addition to that change the
VeritySetup::format method now returns the metadata from the
format call and stores it as debug information to the log file.
A concept to persistently store the verification metadata as part
of the partition(s) will be a next step.
Setting this attribute to a number or 'all' in an overlayroot
configuration will create a dm verity hash from the number of
given blocks (or all) placed at the end of the squashfs compressed
read-only root filesystem. For later verification of the device,
and without further image description settings, the credentials
information produced by veritysetup from the cryptsetup tools, is
created as a file in /boot/overlayroot.verity and is stored as
such into the image by default.
The documentation explains the disk.sh script to be called
from inside of the image root as it exists on the block layer.
The disk.sh script is therefore also called after the sync
of the unpacked image root tree to the block layer. The
implementation however, was only partially calling disk.sh
from such an environment. In fact the environment was only
the mountpoint of the root partition but this is not the
complete system regarding layouts that uses extra partitions
and/or volumes. This commit introduces the use of the new
class ImageSystem and calls disk.sh in the way it was
designed and documented.
The modifications in this commit allows the unit tests
to run on both, pytest 6.x (nose test layout) and the new
pytest 7.x (xunit test layout). This Fixes#2072 in a
much nicer way. Thanks much to @smarlowucf
This reverts commit 0dc2e803e0.
The pytest interface from version v6 to v7 has received
changes which requires the tests to be adapted to work for
either the old or the new interface. As there are still many
distributions which uses v6 as the standard we decided to
revert back the adaptions done to support v7 and create
a version requirement to v6 in .virtualenv.dev-requirements.txt
This Fixes#2072
For the oem type only, allows to specify if the extra read-write
partition in an overlayroot setup should be created or not.
By default the partition is created and the kiwi-overlay dracut
module also expect it to be present. However, the overlayroot
feature can also be used without an initrd and under certain
circumstances it is handy to configure if the partition table
should contain the read-write partition or not.
The optional pre_disk_sync.sh script is executed for the
disk image type oem only and runs right before the synchronisation
of the root tree into the disk image loop file. The script hook
can be used to change content of the root tree as a last action
before the sync to the disk image is performed. This is useful
for example to delete components from the system which were
needed before or cannot be modified afterwards when syncing
into a read-only filesystem.
The pytest interface setup() method call has changed
in a way that an additional parameter is passed to
the method which leads to a python error at invocation
time if the setup method does not define it.
In addition to the volume volume management settings also
allow to setup low level table entries like in the following
example:
<partitions>
<partition name="var" size="100" mountpoint="/var" filesystem="ext3"/>
</partitions>
Allow to specify a bundle_format per <type> which is used
at the time of the result bundle creation to create image
output names matching the provided pattern.
This Fixes#1943
Moving use of mkdtemp, NamedTemporaryFile and TemporaryDirectory
into its own class called Temporary: By default all temporary
data is created below /var/tmp but can be changed via the
global commandline option --temp-dir. This Fixes#1870
Previously, kiwi created staging image files as plain temporary files
in /tmp, which causes issues on operating systems where /tmp is tmpfs.
Notably, image builds would fail with "no space left on the device"
because the tmpfs was not big enough for everything to exist there.
To fix this, we change to use /var/tmp, and additionally add a prefix
for our temporary files so that the user knows which ones kiwi created.
Fixes: https://github.com/OSInside/kiwi/issues/1866
Embedded systems and other customer use cases sometimes
doesn't require an initrd. So far the initrd creation was
a mandatory step in the process. With this commit it's
possible to configure <type ... initrd_system="none"/>
and therefore skip the creation and setup of an initrd.
Using this feature comes with a price. Without an initrd
the task of mounting the specified root=DEVICE_SPEC now
becomes a task of the kernel. If the kernel doesn't have
the required filesystem driver compiled in or the mount
process of the device is not just a simple mount
action, the boot of such an appliance will fail
