Commit Graph

918 Commits

Author SHA1 Message Date
David Cassany
4c3945c4d1
Update bootstrap macro management
This commit extends the bootstrap macro management to also consider
the case where image was already bootstrap. Note this is a common case
for building derived container images and also the situation when
`--allow-existing-root` flag is in use.
2019-04-16 14:40:57 +02:00
Marcus Schäfer
7b29751b92
Fix rpm metadata creation
There are rpm versions that require access to /dev and fail
with the error message 'Failed to initialize NSS library'
For details see: https://bugs.centos.org/view.php?id=14767
Due to the rpm location checkup fix from Issue #1037 this
change impacts builds for distributions that uses an rpm
version with the above behavior (e.g CentOS) and fails as
consequence of the rpm call error. This commit fixes this
in a way that we make sure the host /dev is available at
the time of the call.
2019-04-11 09:34:28 +02:00
Marcus Schäfer
720d34ddd8
Delete check_grub_efi_installed_for_efi_firmware
The motivation is nice to check if the required grub module
package is part of the package list if the efi firmware is
requested, but as long as there is no distribution wide
standard for packaging grub this check will cause more trouble
than it is of help. Currently it failed for the arm architecture
and the grub2-arm64-efi package. We decided to prevent checking
against static lists and dropped this runtime check. Missing
grub modules will be recognized at the grub stage when we
search for them.
2019-04-09 12:08:33 +02:00
Marcus Schäfer
c148d661b9
Deleted pxedeploy section
There is no further demand in the client config creation for the
legacy netboot code. Customers using the netboot initrd already
create the client config.<MAC> file manually or through another
system. With the next generation kiwi we also recommend the disk
based network deployment using the dracut capabilities and marked
the netboot code as still supported but on the legacy stream.
Along with the deletion in the schema the following changes
were also done:

* Deleted use of pxedeploy section in implementation
* Increase schema version to v7.0
* Update documentation on schema version update
* Added xsl stylesheet conversion from v6.9 to v7.0
  Automatic schema version upgrade from v6.9 to v7.0 deletes
  the pxedeploy section from any type specification

This Fixes #19
2019-04-05 14:26:54 +02:00
Marcus Schäfer
4c9037e12e
Merge pull request #1041 from SUSE/disable_runtime_checks
Add the possibility to disable runtime checks
2019-04-02 17:03:36 +02:00
David Cassany
c21570ac7b
Add the possibility to disable runtime checks
This commit adds runtime configuration parameters to disable the runtime
checks. This is specially handy if someone does not want to use the kiwi
dracut modules and provide custom ones instead. In orde disable some
runtime check consider a runtime config yaml like:

runtime_checks:
    - disable:
        - check_dracut_module_for_oem_install_in_package_list
        - check_dracut_module_for_live_iso_in_package_list

This commit fixes #893
2019-04-02 16:00:03 +02:00
Marcus Schäfer
0296808e24
Update compression flag for qcow2 format
In case of a qcow2 format we store the result uncompressed
Since the format conversion only takes the real bytes into
account such that the sparseness of the raw disk will not
result in the output format and can be taken one by one
This Fixes bsc#1128146

In addition the commit includes a refactoring for the
evaluation of the compress flag in the runtime config.
Instead of the global overwrite, the flag gets evaluated
individually at the time the result metadata is created
2019-04-02 11:24:35 +02:00
Marcus Schäfer
840a872507
Fixup has_rpm method from RpmDataBase class
The method checked for the presence of /usr/bin/rpm. But
that binary is also provided by another toolkit named
busybox. Thus to check if the rpm we are aiming for is
present the check has been modified to look for /usr/bin/rpmdb
which is exclusively provided by rpm only. This Fixes #1037
2019-04-01 14:52:33 +02:00
Marcus Schäfer
b9c0815e58
Fixed creation of image metadata files
At the end of a build process the metadata information files .packages
and .verified are created. On rpm based systems, rpm is invoked as
chrooted operation inside the new root tree. For images that gets
installed exclusively from the bootstrap phase there is no rpm inside
of the image and the call fails. The result are empty metadata files.
This patch prevents calling rpm inside of the image root tree if it's
not installed and also uses the RpmDataBase interface instead of
directly calling rpm. This Fixes #1037
2019-03-31 18:52:08 +02:00
David Cassany
0baa783965
Add buildah tool support for OCI and Docker types
This commit adds buildah tool support for OCI and Docker types. It
requires buildah and skopeo to be installed in the host. The use of
umoci (kept as default choice) or buildah is configured using the
runtime configuration file; consider the following structure:

```
oci:
    - archive_tool: buildah
```
2019-03-29 10:15:48 +01:00
Marcus Schäfer
ba3d455979
Fixed bootstrap only building
Image descriptions that define packages in the bootstrap section
only don't need a package manager inside of the image. However
the code to update the location of the rpm database did not
check the presence of rpm and failed on such image descriptions.
This Fixes #1030
2019-03-28 11:20:01 +01:00
Dan Čermák
e1ad0a33c7
Only execute scripts via bash when they are not user executable
Currently we call scripts directly through bash, which has the unfortunate
disadvantage, that the shebang line is completely ignored.
Now we instead check whether the owner of the script is allowed to execute it
and if yes, we let the OS execute it (which takes the shebang into account) or
otherwise call it through bash.
2019-03-26 15:20:56 +01:00
Dan Čermák
d28a7e935c
New method: Path.access - thin wrapper arround os.access
This function calls os.access, but first checks whether the input parameters are
sane, instead of simply returning false when the file in question does not exist
or an invalid mode was specified.
2019-03-26 15:20:49 +01:00
Marcus Schäfer
da2d1c8b1a
Support optional fstab.patch file
In addition to the support for fstab.append, users can now also
provide a patch file to change the contents of the fstab file
as it got written by kiwi. The feature is probably rarely used
but needed in the area of suse's transactional update mechanism.
This Fixes bsc#1129566 and Fixes #945
2019-03-25 11:32:24 +01:00
Dan Čermák
fecf715019
Fix libvirt vagrant box Vagrantfile to use rsync by default
The documentation is suggesting that rsync is used by default to synchronize the
shared folders, yet the libvirt vagrant boxes used the system default (=nfs).
2019-03-21 12:21:53 +01:00
Stefan Seyfried
1dd2cc1b96 relax the live iso root dir permissions
The root of the live ISO has permissions 0700, which is inconvenient if
you mount it to explore its contents. Relax to 0755 instead.
Without this, the mounted ISO FS can only be accessed by root.
2019-03-19 09:45:12 +01:00
David Cassany
8f1048f840
Refactor OCI images packing
This commit refactors the OCI images support:

  * added import_container_image and export_container_image methods
    to oci_tools classes. 'umoci' and 'buildah' consume different
    formats thus the inital skopeo call to import a container is tool
    dependent.

  * use oci-archive transport for packing the OCI images, this causes
    docker and oci operations to just diverge on transport type.

  * add_tag method no longer needed in oci_tools/base, skopeo is used
    for that matter.

  * container/docker.py class is no longer needed. The difference
    between docker and OCI images is just on packing format which is just
    a parameter in skopeo. It does not deserve a dedicated class

  * system/root_import/docker.py class no longer needed. The difference
    between OCI and Docker class was just the transport type for the
    skopeo call. It does not deserve a dedicated class
2019-03-11 13:54:42 +01:00
282e6b7823 Update generated configuration file for DNF
The dnf.conf file generated by KIWI contained some obsolete
options as well as options that have been renamed.

This change updates the options set so that they match what
is expected by DNF and configures the appropriate behavior.
2019-03-07 20:55:20 -05:00
Marcus Schäfer
eecfd489ae
Merge pull request #992 from SUSE/runtime_check_multiple_allsize_volumes
Check for multiple allsize volumes setup
2019-03-07 15:32:43 +01:00
Marcus Schäfer
b43ac0f24f
Cleanup example_dot_profile_config.xml
Don't use multiple all size volumes
2019-03-07 14:02:55 +01:00
Marcus Schäfer
00b8213ac9
Check for multiple allsize volumes setup
Added runtime check check_volume_setup_defines_multiple_fullsize_volumes
which reads the volume setup and counts the number of volumes that
define a 'all' volume. Only one is allowed
2019-03-07 11:28:30 +01:00
Marcus Schäfer
4899da09c6
Complete test coverage on iso tools testing
If there is no mkisofs/genisoimage present on the system, the
tests based on real example iso files are skipped. The code
tested in this case is now covered in a mock style to run
through the logic even without the tools in place
2019-03-06 17:44:45 +01:00
Marcus Schäfer
2cd080d0bb
Follow up fix for zypper compat link
Move the code handling the compat setup of the rpm database
to the correct method of the repository API. Call the
import of the signing keys only if there are signing
keys
2019-03-06 16:25:01 +01:00
Marcus Schäfer
d70df02aa3
Care for update alternatives on rpmdb move
In set_database_to_image_path we also have to care for the
move of the alternatives path to the new rpmdb location
2019-03-06 12:09:15 +01:00
Marcus Schäfer
d92606e46f
Fixup zypper/suse link to rpm database
The compat code generating the /var/lib/rpm link was only
effective if a signing key was specified, however it should
be effective in any case
2019-03-06 11:45:02 +01:00
Marcus Schäfer
b20545897b
Merge pull request #982 from SUSE/fix_unit_test_run_for_cdrtools
Run cdrtools based unit tests on demand
2019-03-06 10:12:50 +01:00
Marcus Schäfer
5112ec7121
Run cdrtools based unit tests on demand
Only if the underlaying system provides legacy cdrtools,
run the unit tests which actually reads the test data iso
using those tools.
2019-03-05 15:54:25 +01:00
Dan Čermák
2b17684d2c
Add support to build vagrant boxes with the Virtualbox provider
This commits adds initial support to build Vagrant boxes with the Virtualbox
provider.
- The attribute 'provider' of the option <vagrantconfig> now supports the option
  "virtualbox".
- A new boolean attribute 'virtualbox_guest_additions_present' was added to
  <vagrantconfig>. When set to true, KIWI will assume that guest additions have
  been installed into the base image and configure the shared folders to use
  vboxfs, otherwise rsync will be used.

This fixes #532 (https://github.com/SUSE/kiwi/issues/532).
2019-03-05 15:33:18 +01:00
e33f53aa45 Rework package manager selection to use DNF when YUM is requested
YUM v3 is no longer developed, having been superseded by DNF for
several years. With DNF now available as a usable package manager
in Red Hat Enterprise Linux 7 through the Extras channel and
SUSE Linux Enterprise 15 through PackageHub, there is no reason for
keeping support for YUM v3 around.

We are keeping support for requesting YUM because in Red Hat Enterprise
Linux 7, DNF is referred to as "YUM v4", and it is simply referred to
as "YUM" in Red Hat Enterprise Linux 8. To avoid confusion from people,
we're just going to leave it in place as an alias to the DNF package
manager.

As for why this is happening now, Fedora is retiring YUM v3 in
Fedora 31, so we might as well get it over with and cut over now.

Reference: https://fedoraproject.org/wiki/Changes/Retire_YUM_3
2019-02-28 15:09:21 -05:00
Marcus Schäfer
c4f0855d1f
Fixed iso tools unit test
Test was not environment independent and behaved differently
compared to which iso tool change is installed.
2019-02-28 11:11:58 +01:00
Marcus Schäfer
3c55023c9e
Write sha256sum --check compatible shasum format
Change the output format of the bundler shasum file to be
compatible with a 'sha256sum --check' call.
This fixes bsc#1127173
2019-02-27 17:33:53 +01:00
Marcus Schäfer
541cf9c3d1
Fixed import of signing keys
In reference to bsc#1112357 it was required to add the compat
symlink /var/lib/rpm such that zypper can read the signing
keys. Unfortunately zypper does not use the configured rpmdb
from the rpm macro setup.
2019-02-27 14:24:55 +01:00
Marcus Schäfer
ed6c11f7d6
Added get_description method to XMLState
In preparation to access the contents of the <description>
section this pull requests adds a simple method to read
the contents.
2019-02-26 15:20:21 +01:00
Marcus Schäfer
c1534a3563
Update vagrant libvirt unit test
Take extra config from DiskFormatVagrantLibVirt as base for the
test and don't mock that information as it was done before
in the base class
2019-02-22 16:01:16 +01:00
Marcus Schäfer
0e422e9447
Extend the vagrant base class test
Take an example for additional vagrant config settings into account
2019-02-22 11:30:46 +01:00
Marcus Schäfer
3f414dada8
Refactor unit tests for vagrant classes
Better use of pytest capabilities, avoid mock rewrite of
implementation classes
2019-02-22 09:53:57 +01:00
Marcus Schäfer
d70d8d85cc
Fixed losetup call due to move in option names
util-linux dropped losetup-Add-support-for-setting-logical-blocksize.patch
because different implementations exists in the new kernel, and it has
a conflicting implementation in util-linux. This caused a change in the
option to specify the logical sector size. The option --logical-blocksize
was replaced by --sector-size. We adapt this now in kiwi too
2019-02-21 15:49:11 +01:00
Marcus Schäfer
a50c251f5f
Refactor vagrant storage subformat
In preparation to support other vagrant providers a base class
for Vagrant operations has been created. Original Code written
by DanČermák <dcermak@suse.com>
2019-02-21 15:48:05 +01:00
Marcus Schäfer
9725153258
Fixed isolinux fallback path
In case isolinux-config failed or does not exist on the
distribution a fallback path is called. That code hardlinks
the files to the isolinux compiled in standard path. However
due to the move of the grub unicode file for iso images
the path contains a directory. Directories can't be hardlinked
thus this patch uses 'cp -l' instead of the 'ln' command to
create the linked target contents.
2019-02-14 17:37:38 +01:00
Marcus Schäfer
2de5dace61
Merge pull request #944 from SUSE/fix_location_of_unicode_font_in_iso_images
Fix location of grub unicode font
2019-02-14 11:06:32 +01:00
Marcus Schäfer
f9aafe7f35
Fix location of grub unicode font
This is a follow up patch for #f5bac4495d34. The change of the
location of the font file was not applied if an iso target, live
or install image is being built. This patch completes the change
and Fixes bsc#1124885
2019-02-13 18:34:40 +01:00
Marcus Schäfer
81bf8241cb
Fix handling of zypper return codes
The following codes should be handled as errors:
104 - ZYPPER_EXIT_INF_CAP_NOT_FOUND
105 - ZYPPER_EXIT_ON_SIGNAL
106 - ZYPPER_EXIT_INF_REPOS_SKIPPED
2019-02-13 17:08:19 +01:00
Bernhard M. Wiedemann
51a9015b34 Sort filesystem listings
so that kiwi works in a reproducible way
in spite of indeterministic filesystem readdir order
and http://bugs.python.org/issue30461

See https://reproducible-builds.org/ for why this is good.
2019-02-11 10:41:55 +01:00
Marcus Schäfer
84fcbc60fd
Sort file and dir entries in iso sortfile 2019-02-11 09:18:55 +01:00
Marcus Schäfer
07369d7c9b
Merge pull request #936 from SUSE/rpm_install_langs
Added support for %_install_langs rpm macro
2019-02-09 15:52:38 +01:00
Marcus Schäfer
f8265bb607
Added new preferences subsection
In the preferences section the following optional subsection
can be configured:

   <rpm-locale-filtering>true|false</rpm-locale-filtering>

If set to true the default locales POSIX, C, and C.UTF-8 are
applied as rpm install_lang macro. If the locale section is
configured in addition the list is extended by that information
too
2019-02-08 16:56:56 +01:00
Marcus Schäfer
e8b66e5da3
Extend the .packages file by the license field
For rpm based builds the License field from the rpm metadata
is extracted into the .packages file. For Debian based build
the license information is in an extra file and not taken
into account for the moment.
2019-02-08 15:58:29 +01:00
Marcus Schäfer
fad1c0f1ef
Added support for %_install_langs rpm macro
During the image build the custom rpm macro %_install_langs
is configured with the <locale> setup from the KIWI XML
description. This allows to filter language specific packages
on the rpm level and Fixes #771
2019-02-07 16:05:25 +01:00
Marcus Schäfer
892c6332f2
Add API for package manager returncode validation
Allow to validate the return code from a package manager
operation. In case of zypper the standard UNIX return
code validation does not apply. Return codes from zypper
which are >= 100 are not treated as an error anymore
2019-02-06 17:13:56 +01:00
Marcus Schäfer
95a287eaf5
Merge pull request #933 from SUSE/workaround_zypper_failed_to_cache_rpm_database
Fix Failed to cache rpm database on zypper addrepo
2019-02-06 15:15:09 +01:00