Commit Graph

50 Commits

Author SHA1 Message Date
Marcus Schäfer
8f39d7f6d0
Make excludes in bootstrap effective for dnf
Followup fix to really use the ignore information in the dnf
package manager implementation. This Fixes #2499
2024-04-03 12:27:02 +02:00
Marcus Schäfer
a7b984115d
Fixed delta_root build
The support for delta_root allows to build a delta container
image from a given base container. Due to the refactoring of
the kiwi code base using context managers no explicit deletion
of instances happens anymore. This uncovered a weakness of
the delta root code at the level of the overlay mount. At
the time of the umount there are still active temporary
mount handlers which keeps the mountpoint busy. In order to
fix this properly also the PackageManager factory is now
a context manager and the Repository factory received a
cleanup method which is called when the PackageManager goes
out of scope. This refactoring also fixes the busy state
when building deltas
2024-03-07 15:50:58 +01:00
Alexandre Detiste
fb69627ad3
Use unittest.mock from core python everywhere
mock was an independent module that has been merged into the Python standard library.
2024-02-18 22:15:30 +01:00
Marcus Schäfer
ae6effd5de
Change package manager dnf to dnf4
With dnf5 there is a successor for dnf but there will also
be a transition period where there will be both, the former
dnf and the new dnf5 available. For a clear distinction
between the two we got the recommendation from the RedHat
team to support both in different namespaces. This commit
now implements a backward compatible change for kiwi which
includes the following modifications:

* XSL stylesheet for automatic schema transformation from

  <packagemanager>dnf</packagemanager> to
  <packagemanager>dnf4</packagemanager>

* Code copy of dnf API interface from

  PackageManagerDnf -> PackageManagerDnf4
  RepositoryDnf -> RepositoryDnf4

* Deprecation of former Dnf API interface

The code change here will force developers to adapt
their code if they used RepositoryDnf / PackageManagerDnf
classes in their python code. After this change developers
will be dropped into a raise condition which exits kiwi
at the time of the call. Related to Issue #2300
and Issue #2262
2023-06-12 17:44:06 +02:00
Marcus Schäfer
e8bb1cd04b
Add support for dnf5 package manager
dnf5 is the next generation of the dnf package manager.
dnf5 is not a drop in replacement to dnf which is the reason
why this commit adds a new interface class for package and
repository management under a new name: dnf5. Beginning with
Fedora >= 39 dnf5 will be the standard. Thus the requirements
on the packaging level were adapted too.
This Fixes #2300
This Fixes #2262
2023-06-12 13:58:36 +02:00
Marcus Schäfer
41875f57ea
Add support for creating delta containers
Add new attribute delta_root="true|false" which in combination with
derived_from="OCI-base-image" allows to create a delta root tree
only containing the differences between the OCI-base-image and the
actions taken by the kiwi prepare step. Based on the delta root tree
the subsequent image creation process starts. Such an incomplete
root tree is only useful under certain circumstances which is also
the reason why the feature is only available to the oci and docker
image types.
2023-03-16 17:06:15 +01:00
Marcus Schäfer
40c16412f2
Make sure to rebuild rpm database
For rpm based distributions make sure to call
rpm --rebuilddb unconditionally prior using rpm
with the chroot. This Fixes #2165
2022-06-23 16:32:59 +02:00
Marcus Schäfer
4f47b1d94b
Add support for prebuilt bootstrap package for apt
When using the apt packagemanager kiwi required the use of
debootstrap to create the initial rootfs. This works as long
as there is always a main distribution repository available
which follows the structure of the official debian mirrors.
However if such a main distribution is not present or an
alternative layout like e.g OBS repos is used, debootstrap
will refuse to work. To allow for an alternative and without
the dependency to debootstrap kiwi supports using a prebuilt
bootstrap package providing the mini rootfs to serve as
the bootstrap result. As all other package managers properly
supports installation into an empty new root, this feature
was only added when using the apt packagemanager
2022-05-18 11:19:51 +02:00
Marcus Schäfer
76fecc2300
Merge pull request #2073 from OSInside/debootstrap_logging
Added debootstrap log info to exception message
2022-02-26 20:46:04 +01:00
Marcus Schäfer
9c9250ebc4
Support nose and xunit style tests
The modifications in this commit allows the unit tests
to run on both, pytest 6.x (nose test layout) and the new
pytest 7.x (xunit test layout). This Fixes #2072 in a
much nicer way. Thanks much to @smarlowucf
2022-02-26 20:26:18 +01:00
Marcus Schäfer
66e7b5bb74
Added debootstrap log info to exception message
In case debootstrap fails there is more detailed information
in a logfile written by debootstrap itself. This commit changes
the exception information to contain this log information if
present. Related to Issue #1800
2022-02-22 16:19:19 +01:00
Marcus Schäfer
8a32094f09
Follow up fix on force deleting debs
Also remove eventual post scripting prior force removal
of deb packages. Similar inconsistencies as with the pre
scripts can occur on force removal. We want the operation
to be successful in force mode even if that means to
leave a dirty state.
2022-02-14 10:46:19 +01:00
Marcus Schäfer
b80e238395
Followup fix for force deleting debian packages
The force uninstall deletes pre scripts prior removal
because if they fail the package will not be removed.
For a force uninstall we consider this ok. However,
the deletion of the scripts did not happen in the
image root. This patch fixes it
2022-01-29 14:18:14 +01:00
Marcus Schäfer
8d3129a97e
Followup fix for force deleting debian packages
Pass --force-depends to allow uninstall even if the
dependency checker complains
2022-01-26 08:27:29 +01:00
Marcus Schäfer
334895c740
Fix the uninstall(force) on Debian based dists
Packages marked for uninstall via <package name="delete|uninstall"/>
failed to become removed for several reasons. The way this was done
in kiwi did not work because dpkg needs to be called differently
and with some nasty pre-processing in order to allow for force
deletion. In force mode we also allow to remove packages marked as
essential. In gracefull uninstall mode this commit makes sure the
environment is prepared and does not fail for false-positive
reasons.
2022-01-24 16:05:46 +01:00
Marcus Schäfer
e2f32e8c8a
Added support for collection modules
In CentOS Stream 8 and Red Hat Enterprise Linux 8, there are
Application Streams that are offered in the form of modules
(using Fedora Modularity technology). To build images that use
this content KIWI needs to support to enable/disable various
modules. This commit allows to configure collection modules
in a new element as shown below

<packages type="bootstrap">
    <collectionModule name="module" stream="stream" enable="true|false"/>
</packages>

This Fixes Issue #1999
2021-12-14 22:55:05 +01:00
Marcus Schäfer
7e3297d9d3
Added support for setting up release version
Currently the release version is not set or set to '0'
for package managers which requires a value to operate.
However, in order to support leveraging the same description
across different releases it is important to have the
opportunity to specify a setting for the release version.
This commit adds a new optional attribute to the preferences
section which allows to specify a custom value which
serves as the release version:

<preferences>
    <release-version>TEXT</release-version>
</preferences>

If not specified the default setting as before applies.
Please note the release version information is currently
used in dnf and microdnf package managers only. It might
happen that it gets applied to the other package manager
backends as well. This will happen on demand though.
Related to Issue #1918. This Fixes #1927
2021-12-06 09:54:41 +01:00
Marcus Schäfer
79080d1c97
Fixed uninstall handling via dnf, microdnf, zypper
The above package managers supports uninstall instructions
like 'iwl*'. In kiwi there was code checking via rpm if
the packages given to uninstall actually exists. That code
does not work if the given package to uninstall is an
instruction that matches a pattern. Therefore if we use
the uninstall section in the kiwi image description, just
pass the provided information to the package manager and
don't try to be clever in kiwi itself.
2021-09-10 23:49:37 +02:00
Marcus Schäfer
f807a630dc
Merge pull request #1860 from OSInside/no_shell_for_dnf_calls
Don't shell out for calling dnf
2021-06-21 15:15:13 +02:00
Marcus Schäfer
d8d0fa38f2
Merge pull request #1857 from OSInside/no_shell_for_pacman_calls
Prevent calling pacman through a shell
2021-06-21 14:53:01 +02:00
Marcus Schäfer
89b5a6f526
Don't shell out for calling dnf
refactor the dnf call to install packages and groups in
one call. This allows to prevent calling dnf through a
shell. For installing of a package group the group ID
name is expected. This Fixes #1856
2021-06-21 14:50:59 +02:00
Marcus Schäfer
47e4a6709c
Do not shell out for calling microdnf.
In fact it can be counter productive if the shell
evaluates eventually existing package name/instruction
patterns. This is related to Issue #1856
2021-06-21 11:59:04 +02:00
Marcus Schäfer
89f18b02b5
Prevent calling pacman through a shell
There is no reason to shell out for calling pacman.
In fact it can be counter productive if the shell
evaluates eventually existing package name/instruction
patterns. This is related to Issue #1856
2021-06-21 11:51:43 +02:00
Marcus Schäfer
7d854ed813
Change zypper download mode to in-advance
In relation to upcoming zypper changes e.g to make use of librpm on
single transaction operations there is the possibility that file
triggers start being used. To ensure zypper behaves consistently
DownloadInAdvance mode should be used, this way the transaction
happens as a whole and with the upcoming zypper changes zypper
will still be capable to handle the file triggers.
This Fixes #1789
2021-04-19 09:40:34 +02:00
Marcus Schäfer
3e43d0aadc
Decommission obsolete code reaching EOL
Use the @decommissioned decorator to raise for API methods
that a over the obsoletion period
2021-04-10 20:04:39 +02:00
Marcus Schäfer
964b353413
Delete yum from packagemanager in schema
Auto convert yum to dnf if set as packagemanager. This allows
to delete the yum handling from code parts in kiwi where this
was still present. In addition this fixes the inclusion of yum
into the packagelist. This Fixes #1768
2021-03-19 15:45:11 +01:00
Marcus Schäfer
93d72e78dc
Change packagemanager setting from apt-get to apt
In kiwi we use the name of the section as package name to install this
package manager capability. However on Debian based distros there is
no package named apt-get. There is only a package named apt which
provides a tool named apt-get. To avoid inconsistencies like we had it
in Issue #1735 and to bring this setting in line with all other
packagemanager settings the setting was moved to just apt.
This Fixes #1738
2021-03-15 16:13:02 +01:00
David Cassany
01c05f63b1
Remove files or folders that conflict with debootstrap
This commit makes use of Path.wipe method instead of os.unlink. This way
we are certain the path is removed regardless being a directory or a
file.

fixes #1718
2021-02-10 22:27:50 +01:00
Marcus Schäfer
7725c1b1f7
Fixed debootstrap device node conflict
debootstrap creates its own device node tree and fails
if a node it creates itself already exists. This commit
introduces a list of conflicting device nodes and deletes
them prior kiwi calling debootstrap. This Fixes #1675
2021-02-01 09:29:16 +01:00
Marcus Schäfer
46053bcaca
Refactor PackageManager
This commit refactors the PackageManager class and turns it into
a proper factory class which also include type hints to facilitate
it's use from an API POV. Related to #1498
2020-11-23 15:35:03 +01:00
Marcus Schäfer
cb4e0c8096
Set --releasever=0 for microdnf
To allow microdnf to work from an empty root directory
we need to set the release version to zero
2020-11-19 14:18:30 +01:00
Marcus Schäfer
a22ec18999
Fixed microdnf support
The installroot argument must be used together with --config
and additionally with --noplugins, as well as --setopt for
cachedir, reposdir and varsdir. Related to #1625
2020-11-18 12:24:28 +01:00
7ebeeca10b Fix setopt argument for install_weak_deps for microdnf
Micro DNF does not support "True"/"False", only "1"/"0"...
2020-11-16 11:59:06 -05:00
a0912d526d Use --config instead of -c for DNF and Micro DNF
The -c option is not supported in Micro DNF, but --config is, and
it is supported with DNF as well.
2020-11-13 09:54:41 -05:00
e811d793c7 Drop 'microdnf makecache' call for microdnf package manager
This subcommand does not exist and is not needed. Instead, we
need to use '--refresh' where this is needed.
2020-11-13 09:49:48 -05:00
388eeef349 Add support for the Micro DNF package manager
Micro DNF is a minimal C implementation of DNF that is usable for
minimal appliances and containers. While it is not at parity with
DNF, it implements enough functionality that it is mostly usable
for building appliance images.
2020-11-12 10:09:23 -05:00
David Cassany
6383ea6c1f
Better integrate debootstrap in KIWI process
This commit integrates debootstrap process in KIWI in a more transparent
way. This refactor makes debootstrap to operate directly over the image
target root tree. This way KIWI can show and parse debootstrap stdout
and also avoids having to copy the created root tree from one temporary
folder to the actual target root tree path.

Related to #1587
2020-11-02 14:09:39 +01:00
David Cassany
c5d51ec86e
Include boostrap packages list into deboostrap call
This commit includes the packages listed in boostrap section to the
debootstrap call instead chrooting into the new root tree and call
apt.

Fixes #1587
2020-10-19 22:08:04 +02:00
Marcus Schäfer
47167fd486
Use rsync in inplace transfer mode
Using the --inplace option in rsync helps to save space on
syncing the rootfs data and prevents e.g OBS workers from
running out of VM space when transfering root filesystem
data. Also using --inplace allows to keep hardlinks intact.
This is related to bsc#1096738
2020-07-15 16:56:28 +02:00
David Cassany
542769c882
Fix signing keys management for APT
This commit fixes the management of the trusted keyring for apt
repositories. It creates a `trusted.gpg` keyring with the provided
signing keys so APT can check against that the configured repositories.

Fixes #1440
2020-05-18 12:17:56 +02:00
Marcus Schäfer
5faa984f83
Fixed zypper error code validation
The error code 127 - command not found, was not treated as
an error. This commit adds 127 to be an error condition
along with the other 1xx error codes from zypper that are
handled as errors. This Fixes #1430
2020-05-13 12:25:42 +02:00
David Cassany
feb574a913 Adding support for Pacman package manager
This commit adds support for pacman package manager and, in
consequence, support for Arch Linux images. It also adds
the package definition for Arch Linux.
2020-05-03 16:18:02 +02:00
Marcus Schäfer
fd08da5291
Merge pull request #1423 from OSInside/clean_AnonymousUniqueId
Clean AnonymousUniqueId from the image
2020-05-01 17:12:03 +02:00
David Cassany
469f71144e
Clean AnonymousUniqueId from the image
This commit is two fold.

For one side does a small refactor to move the deletion of custom RPM
macros to package manager level inside a cleaning method. This way only
RPM based package managers run RPM specific code and each package manager
can apply its own specific logic.

On the other hand for the zypper package manager the deletion of
/var/lib/zypp/AnonymousUniqueId file has been added as part of the
new cleaning method.

Fixes #1396
2020-04-30 15:19:08 +02:00
David Cassany
f7c805f156
Add flags and argument separator for zypper
This commits adds a `--` argument in zypper install calls right after
the flags and before the arguments (in this case packages) list.

Fixes #1407
2020-04-23 17:20:49 +02:00
David Cassany
686dafc9e4
Remove RootBind.move_to_root method
This commit removes the RootBind.move_to_root method as this can all be
done by using the Path.move_to_root utility method. This allows
to drop the RootBind attribute in PackageManager classes and focus
path manipulation methods into a common utility.

Related to #1281
2020-01-20 16:17:24 +01:00
David Cassany
479769dc4b
Do not rsync /dev on debootstrap
This commit does not rsync /dev on debootstrap and instead it uses
the bind mount for /dev the same way it is done for other non apt
based bootstrap processes.
2019-11-27 09:29:13 +01:00
David Cassany
c7e804b829
Do not rsync kernel filesystems on debootstrap
This commit ensures that /proc and /sys are not rsynched when
debootstrapping an apt based image.

Fixes #1270
2019-11-26 17:17:21 +01:00
Marcus Schäfer
bdb7123fa1
Refactor use of logging facility
Use getLogger method instead of a global log object
Also use caplog fixture to capture log messages in
unit tests. This Fixes #1244
2019-10-23 17:54:27 +02:00
David Cassany
4b3a105026
Re-structure unit tests folders
This commit relocates unit tests to a folder structure that matches
the source code structure.

Fixes #1128
2019-10-21 14:00:05 +02:00