Commit Graph

1164 Commits

Author SHA1 Message Date
Marcus Schäfer
05f2d40836
Refactor VolumeManager
This commit refactors VolumeManager to turn it into a proper
factory class and to also include type hints to facilitate it's
use from an API POV. Related to #1498
2020-11-02 18:05:21 +01:00
Marcus Schäfer
a2d77a0a11
Refactor RootImport
This commit refactors RootImport to turn it into a proper
factory class and to also include type hints to facilitate it's
use from an API POV. Related to #1498
2020-11-02 16:51:56 +01:00
Marcus Schäfer
3600314322
Merge pull request #1601 from OSInside/refactor_solver_and_storage_factory
Refactor solver and storage factory
2020-11-02 16:13:29 +01:00
Marcus Schäfer
223abea7c4
Merge pull request #1598 from OSInside/improve_debootstap_execution
Improve debootstap execution
2020-11-02 15:41:09 +01:00
Marcus Schäfer
7b479cbd9c
Refactor DiskFormat
This commit refactors DiskFormat to turn it into a proper
factory class and to also include type hints to facilitate it's
use from an API POV. Related to #1498
2020-11-02 14:53:36 +01:00
David Cassany
6383ea6c1f
Better integrate debootstrap in KIWI process
This commit integrates debootstrap process in KIWI in a more transparent
way. This refactor makes debootstrap to operate directly over the image
target root tree. This way KIWI can show and parse debootstrap stdout
and also avoids having to copy the created root tree from one temporary
folder to the actual target root tree path.

Related to #1587
2020-11-02 14:09:39 +01:00
Marcus Schäfer
ff213dfdd5
Refactor SolverRepository
This commit refactors SolverRepository to turn it into a proper
factory class and to also include type hints to facilitate it's
use from an API POV. Related to #1498
2020-11-02 10:43:26 +01:00
Marcus Schäfer
226be19c91
Merge pull request #1596 from OSInside/custom_root_volume_name
Allow custom root volume name setup
2020-10-28 12:43:15 +01:00
Marcus Schäfer
a124d82173
Allow custom root volume name setup
In addition to the custom size of the root volume it's now
also possible to setup the name of the root volume as follows:

    <volume name="@root=rootlv"/>

If no name for the root volume is specified the default
name: LVRoot applies as before. This Fixes #1530
2020-10-27 16:17:47 +01:00
Marcus Schäfer
a5425fe575
Add support for s390 CDL DASD disks
On s390 and in CDL mode (4k DASD) the call of grub2-install
does not work because grub2-install is not able to identify
a 4k fdasd partitioned loop device as a grub supported device
and fails. As grub2-install is only used to invoke
grub2-zipl-setup and has no other job to do we can
circumvent this problem by directly calling grub2-zipl-setup
instead. Also delete LDL mode support, IBM no longer supports
this
2020-10-27 11:06:14 +01:00
David Cassany
c5d51ec86e
Include boostrap packages list into deboostrap call
This commit includes the packages listed in boostrap section to the
debootstrap call instead chrooting into the new root tree and call
apt.

Fixes #1587
2020-10-19 22:08:04 +02:00
Marcus Schäfer
31e0ce96cd
Enhance scope of _fix_grub_root_device_reference
In addition to the wrong root=/dev/mapper/loop... reference
fixing, written by grub2-mkconfig when used in obs there is
also the case that grub2-mkconfig writes root=PARTUUID which
is also unwanted and needs fixing. To solve this properly
and also with respect to more unexpected grub2-mkconfig
data, the code changes to use a regular expression for the
root= replacement. The expression matches all root= cases which
we want to fixup:

1. If the local used root device appears in the grub config file.
2. If a linux by-id setting is used in the grub config file.

The replacement happens for overlayroot disk images where we
know grub2-mkconfig cannot handle the layout as well as if kiwi
runs in obs where we know grub2-mkconfig fails due to the absence
of udev.
2020-10-12 09:08:08 +02:00
Marcus Schäfer
259b11bdc3
Merge pull request #1568 from OSInside/support_creation_of_package_change_info
Add support for package changelog info
2020-10-02 14:03:42 +02:00
Marcus Schäfer
d102c6af8d
Allow to configure .changes creation and bundling
Provide config option has_package_changes in the runtime config
file and set a useful default. For building outside obs the
default for the .changes creation is switched on, for building
in obs it's switched off because obs creates its own info file
2020-10-02 11:18:55 +02:00
Marcus Schäfer
836633c7b2
Fix/Refactor s390 support
This changes the s390 support on several stages:

1) On s390 the boot process is based on zipl which boots into an
   initrd from which a userspace grub process is started to support
   the grub capabilities. The implementation of this concept is
   provided via the grub2-s390x-emu package. Once installed the
   setup of the bootloader is done via the grub2-mkconfig and
   grub2-install commands and therefore from a caller perspective
   the same as with any other grub2 setup process. For kiwi this
   means no extra zipl bootloader target code is needed. Therefore
   this commit deletes the zipl setup from kiwi and puts on
   the standard grub2 process. This Fixes bsc#1170863

2) To support different targettypes the grub2-s390x-emu provided
   zipl template must be adapted. Parts of the former zipl bootloader
   setup therefore now applies to an update of the zipl2grub
   template file

3) Support for CDL/LDL DASD targets has been disabled in the schema
   When testing 4k devices and a respective zipl2grub template
   setup for CDL/LDL targettype it has turned out that grub2-install
   is not able to run on such a device. My assumption is that
   the device code in grub2-install does not work for 4k devices
   with an fdasd created partition table. As this needs further
   investigations and most probably adaptions on the grub toolchain
   for s390, we disabled the setup of these modes for now.
   emulated DASD (FBA) and SCSI targets stays supported.
2020-09-30 17:14:30 +02:00
David Cassany
a6f56179b7
Fix compat link for rpmdb location
This commit fixes the symlink creation for `/var/lib/rpm`. More specific
for derived container images in which the base root tree already
included the `/var/lib/rpm` the link, the `ln` command was creating a
symlink inside the `/var/lib/rpm` folder givent that it was following
the already existing symlink. Adding the `--no-target-directory` force
`ln` command to treat `/var/lib/rpm` path as the fully qualified symlink name.

Fixes bsc#1176977
2020-09-29 13:37:13 +02:00
Marcus Schäfer
490a9db564
Use pragma: no cover to skip non reachable code
Instead of placing the file into .coveragerc use the
no cover pragma to skip code paths that can never be
reached from being counted in the coverage report.
This affects the abstract constructors in the Factory
classes
2020-09-29 09:58:22 +02:00
Marcus Schäfer
3c940e5439
Merge pull request #1569 from OSInside/refactor_bootloaderconfig
Refactor BootLoaderConfig
2020-09-29 09:51:51 +02:00
David Cassany
2abcce803b
Refactor BootLoaderInstall
This commit refactors BootLoaderInstall class to make it a proper
factory class. In addition type hints are added for the constructor
method.

Related to #1498
2020-09-28 09:37:05 +02:00
David Cassany
c06a424258
Refactor BootLoaderConfig
This commit refactors BootLoaderConfig to turn it into a proper factory
class and to also include type hints to facilitate it's use from an API
POV.

Related to #1498
2020-09-28 09:35:17 +02:00
Marcus Schäfer
76ad67e82c
Add support for package changelog info
In addition to the .packages file which shows details about
the installed packages in terms of version, license, etc...
we now also create a .changes file which contains the changelog
information of the installed packages. The file can be used
to compare the package changelogs between image builds.
2020-09-26 14:46:13 +02:00
Marcus Schäfer
b7bdcb2bb1
Merge pull request #1566 from OSInside/refactor_boot_image_factory
Refactor BootImage Factory
2020-09-24 10:30:23 +02:00
Marcus Schäfer
362cb807fd
Merge pull request #1563 from OSInside/refactor_builder_factory
Refactor ImageBuilder Factory
2020-09-24 10:30:05 +02:00
Marcus Schäfer
0b8abf1c7c
Refactor BootImage Factory
This commit refactors the BootImage factory to be a real
factory and to add type hints such that its use from an api
perspective is clear and enforced. Related to Issue #1498
2020-09-23 15:48:21 +02:00
Marcus Schäfer
3e8e6c8608
Followup fix to handle one disk type better
The vmx type is auto converted into an oem type with rootfs
resize disabled such that all disk images can be handled
under one disk type. However people who run kiwi on the
commandline and have selected --type vmx before now end
with an error message saying that there is no vmx type
because it was converted into an oem type. To handle this
more gracefully this commit changes the commandline
option --type vmx into --type oem if provided and prints
a warning message.
2020-09-23 10:21:40 +02:00
Marcus Schäfer
d1d0444abd
Refactor ImageBuilder Factory
This commit refactors the ImageBuilder factory to be a real
factory and to add type hints such that its use from an api
perspective is clear and enforced. Related to Issue #1498
2020-09-22 16:34:44 +02:00
Marcus Schäfer
38482cc70c
Merge pull request #1560 from OSInside/set_default_values_for_containers
Get default maintainer and author from image description
2020-09-21 15:50:22 +02:00
Marcus Schäfer
8967c29a12
Added consistency runtime check for the type setup
multiple type sections within one preferences section is allowed
in a kiwi image description. However, if multiple type sections
for the same image attribute are configured only the last type
configuration will be ever reachable. The proposed runtime check
in this commit detects this situation and raises an exception
showing the conflicting types including a solution suggestion
which needs to be based on profiles to distinguish between
types of the same image type name.
2020-09-21 15:25:58 +02:00
David Cassany
ac7bed3221
Get default maintainer and author from image description
This commit sets the maintainer and author metadata from the description
section of the image in they are not explicitly specified in
container-config section.

In addition it sets the default container name to `system-container`
instead of `systemContainer` as uppercase letters are not valid for
docker container references.

Fixes #1419
2020-09-21 10:24:31 +02:00
Marcus Schäfer
7506c651c6
Merge pull request #1552 from OSInside/oem_superset_of_vmx
Make oem be a superset of vmx
2020-09-16 15:00:37 +02:00
Marcus Schäfer
8d2d9c214c
Make oem be a superset of vmx
A vmx image is the same disk as an oem just without the dracut
repart/resize feature. This difference is better handled with
an oemconfig parameter <oem-resize> which allows to switch resize
on or off. The extra image type vmx will be dropped and an XSLT
stylesheet automatically transforms a vmx description to be a
oem image with the resize feature switched off.
This Fixes #1425
2020-09-16 12:56:14 +02:00
David Cassany
cb7b7673e8
Fix the early boot grub.cfg file
This commit makes sure that the early boot configuration files
for grub make use of the proper boot path and omiting the `/boot`
prefix if there is a dedicated boot partition.

Fixes #1553
2020-09-16 11:35:00 +02:00
David Cassany
3bb2ab77fb
Include loadenv as a default module for grub images
This commit includes the 'loadenv' module to the list of basic grub
modules. This makes sure the module is included in any grub-mkimage
that KIWI does.

Fixes #1547
2020-09-14 22:07:23 +02:00
Marcus Schäfer
54ab18c96c
Make oem image use dracut by default
Before this commit an oem image still had the kiwi initrd_system
set as default. As we are decommission the custom kiwi initrd
concept the default should be changed. It is still possible to
use a custom kiwi initrd but it needs to be explicitly requested
via the initrd_system="kiwi" attribute. In addition to the
changed default a runtime check was introduced that checks
the presence of the boot= attribute which only makes sense
in combination with the kiwi initrd_system. If boot= is
set but initrd_system="kiwi" is not, a message is raised that
explains the situation and either requests setting initrd_system
properly or deleting the boot attribute. The change only
affects people who still use oem with a boot="oemboot/..."
setting and no explicit selection of kiwi as the initrd_system.
As these image type configurations should not be in use anyway
because this is all legacy and announced to go away, we
need to make the next step and enforce a new default in
code. This is related to Issue #1299
2020-09-11 09:49:17 +02:00
Marcus Schäfer
988404ff9f
Support dynamic linux/linuxefi in any case
Instead of restricting the dynamic linux vs. linuxefi setup
to a specific grub version, support this setup for any version
of grub. This Fixes bsc#1175729 and bsc#1176134
2020-09-08 18:14:15 +02:00
Marcus Schäfer
5ec3386afd
Merge pull request #1521 from OSInside/grub_loader_entries_option_handling
Make sure kernel options are respected by grub
2020-08-10 15:11:31 +02:00
David Cassany Viladomat
fc85191564
Merge pull request #1519 from OSInside/fix_scope_of_remove_hierarchy
Fixed scope of remove_hierarchy
2020-07-27 12:20:04 +02:00
Marcus Schäfer
d7cd5b555d
Make sure kernel options are respected by grub
Set options variable in grub loader/entries to match the
kiwi image description setup. This adaption currently
happens only when building in OBS because in this env
the options setup done by grub is a complete mess.
2020-07-22 22:33:15 +02:00
Marcus Schäfer
1c5fcfaecd
Merge pull request #1516 from OSInside/fix_grub_setup_in_efi_path
Fixed grub setup in EFI/BOOT directory
2020-07-22 14:16:44 +02:00
Marcus Schäfer
3698e31bdf
Fixed scope of remove_hierarchy
The remove_hierarchy method walked through the entire path
it was given. That included the root path which is beyond
its scope. This Fixes #1515
2020-07-22 11:32:28 +02:00
Marcus Schäfer
b9824f7b62
Fixed corrections in BootLoaderSpec grub config
Distributions like Fedora RawHide or CentOS8 applies to the
grub BootLoaderSpec as described here:

    https://www.freedesktop.org/wiki/Specifications/BootLoaderSpec

Part of the spec is that menu entries are handled in extra files
below /boot/loader/entries. Unfortunately the grub2-mkconfig code
has still no clue how to find the correct root device in special
environments like obs workers or in overlay systems. To fixup the
result of grub2-mkconfig there is code in kiwi which needs to be
adapted because the file that contains the wrong information is
now no longer grub.cfg but some /boot/loader/entries/*.conf file.
This commit solves the issue.
2020-07-22 10:36:24 +02:00
Marcus Schäfer
3255264d4c
Fixed grub setup in EFI/BOOT directory
kiwi copied the same grub.cfg file as it exists in boot/grub2
to the efi path. This is wrong as the setup in the efi boot
directory is used to enable normal grub loading and not providing
the user grub configuration. In addition the changes here makes
sure that the early grub boot code is placed into the system
in any EFI case except for secure boot when shim-install is
present. If shim-install is present it also creates the early
grub boot setup such that kiwi doesn't have to do it.
This Fixes #1491 and Fixes bsc#1172908
2020-07-22 09:41:08 +02:00
Marcus Schäfer
8a7d2d8678
Merge pull request #1514 from OSInside/rsync_inline
Use rsync in inplace transfer mode
2020-07-20 10:36:47 +02:00
Marcus Schäfer
47167fd486
Use rsync in inplace transfer mode
Using the --inplace option in rsync helps to save space on
syncing the rootfs data and prevents e.g OBS workers from
running out of VM space when transfering root filesystem
data. Also using --inplace allows to keep hardlinks intact.
This is related to bsc#1096738
2020-07-15 16:56:28 +02:00
Marcus Schäfer
6b2c4f88bb
Don't keep copy of grub2-install in the system
To prevent shim-install from calling grub2-install in uefi mode
kiwi temporary replaces the tool by a noop. This acts as a
workaround for an issue in shim-install. However the workaround
left a file copy of grub2-install in the system which should
not happen. This commit Fixes bsc#1173226 and Fixes #1490
2020-07-15 14:38:47 +02:00
David Cassany
4f0bdddddc
Do not create a md5 checksum output file for archive images
This commit removes the creation of the md5 file that includes a
checksum of the image binary for the `tbz` image type. Removing it
as this is the only image type that includes it as part of the result
and because the bundle procedure already creates a sha256 file out of
the results, so there is still the chance to produce validation
checksums.
2020-07-08 06:53:11 +02:00
Marcus Schäfer
ed22674468
Merge pull request #1495 from OSInside/factory_refactor
Refactor FileSystem Factory
2020-07-07 10:57:58 +02:00
Christian Schneemann
61803a30bd
Fixed oem recovery tar parameter order
Reorder tar parameters

When building an oem-image with oem-recovery set to true a tar error appears saying:
"The following options were used after any non-optional arguments in archive create or
update mode". This commit fixes the order of arguments to tar to avoid the error condition.
Fixes #1501
2020-07-03 10:33:05 +02:00
Marcus Schäfer
0fee3bbb60
Refactor FileSystem Factory
With regards to Issue #1486 a discussion came up that the
way factories are implemented are questionable when thinking
about strong typing for the public kiwi interface. This
commit refactors the FileSystem factory to be a real factory
and to add type hints such that its use from an api perspective
is clear and enforced.
2020-06-29 22:38:31 +02:00
Marcus Schäfer
301abaaffc
Merge pull request #1493 from OSInside/fix_live_ISOs
Fixes live ISOs
2020-06-25 16:42:30 +02:00