Commit Graph

139 Commits

Author SHA1 Message Date
Marcus Schäfer
94de1336d8
Support custom partitions
In addition to the volume volume management settings also
allow to setup low level table entries like in the following
example:

<partitions>
    <partition name="var" size="100" mountpoint="/var" filesystem="ext3"/>
</partitions>
2021-11-03 13:31:17 +01:00
Marcus Schäfer
0aa490c654
Fixed use of LUKS encrypted images with empty pass
For initial provisioning of LUKS encrypted disk images an
empty passphrase key is handy to avoid interaction in the
deployment process. However, the dracut kiwi modules were
lacking the information that the luks keyfile could be an
empty passphrase key which must not be opened with the
potential risk to get prompted for input. This commit
introduces a new profile environment variable evaluated
by the dracut kiwi lib code to open the LUKS pool and
allows to distinguish the situation on key files with
or without a passphrase
2021-10-14 11:27:16 +02:00
Marcus Schäfer
ef05ecc36e
Add support for toplevel include directive(s)
On the toplevel of an image description you can now
specify include directive(s) like in the following
example

<image ...>
    ...
    <include from="filename_a.xml"/>
    ...
    <include from="filename_b.xml"/>
</image>

At the place of their occurrence the include statement
will be replaced with the contents of the given filename.
The implementation is based on a XSLT stylesheet and
applies very early in the process. The stylesheet reads
the contents of the given file as XML document().
Thus only valid XML documents gets accepted by this
include concept. This Fixes #1929 and is related to
Issue #1918
2021-10-07 09:50:38 +02:00
Jesus Bermudez Velazquez
fd52237478
Allow target dir for archive
- Add the option to specify a target directory
  to unpack the archive
- Update doc for target dir attribute

This Fixes #1794
2021-09-07 15:13:24 +01:00
Marcus Schäfer
e3e0d888c8
Added support for repo customization script
repo files allows for several customization options
which could not be set by kiwi through the current
repository schema. As the options used do not follow
any standard and are not compatible between package
managers and distributions the only generic way to
handle this is through a script which is invoked
with the repo file as parameter for each file created
to describe a repo for the selected package manager.
This allows users to update/change the repo file content
on their individual needs. In the kiwi description the
path to the custom script can be specified as follows

    <repository ... customize="/path/to/custom_script">
        <source path="..."/>
    </repository>

This Fixes #1896
2021-08-10 10:10:01 +02:00
Marcus Schäfer
7bea6c9203
Add support for a custom exclude file
The new optional metadata file image/exclude_files.yaml can
be placed inside of the local image root tree. At creation time of
the image binary the file contents are used to extend the default
exclude list with additional information. The structure of the
file must be as follows:

```yaml
exclude:
  - exclude-name-used-in-rsync
```
2021-04-16 10:23:33 +02:00
Marcus Schäfer
bc9f37fcf0
Allow to enable volumes for filesystem check
The new attribute <volume ... filesystem_check="true|false"/>
allows to change the default value for the fs_passno field in
the generated fstab file. By default kiwi sets "0" in this
field and leaves it up to the user to customize this as
appropriate via script code. Coding changes to the fstab file
via scripts are not very user friendly and with respect that
systemd takes over control and generates checkers depending on
the value of fs_passno it would be good if there is a way to
explicitly specify if checks to the filesystem are wanted or
not. Therefore the new attribute now exists. If set to: true
this results in a value of "2" for the fs_passno field. Please
note the root/boot and efi entries are excluded from this
setup. This Fixes #1728
2021-03-30 15:10:03 +02:00
Marcus Schäfer
47d48c1a89
Update unit- integration tests to schema v74 2021-03-15 16:13:02 +01:00
Marcus Schäfer
aae7da639d
Added deb support to sat solver module
Add support to create SAT solvables from Debian repos via
deb2solv tool from libsolv. This allows image info --resolve-package-list
to work with Debian/Ubuntu image descriptions. Please note
by default libsolv is not compiled with support for Debian.
Therefore the following compile flags must be set on libsolv

* -DENABLE_DEBIAN=1
* -DMULTI_SEMANTICS=1

If libsolv does not provide the needed capabilities kiwi will
fail on either the repository solvable creation due to missing
deb2solv or on call of setdisttype() due to missing MULTI_SEMANTICS
feature in libsolv
2021-02-21 18:50:16 +01:00
Marcus Schäfer
76e0b868fe
Allow to specify OBS credentials in runtime config
In preparation to the new obs kiwi plugin this commit adds
an opportunity to store obs credentials such that the plugin
could also be used in a non-interactive way
2021-02-12 11:21:03 +01:00
Marcus Schäfer
7a5ab282b6
Refactor RuntimeConfig class
The former implementation read the runtime config file every
time a new RuntimeConfig instance was created. The runtime
config is however static during the runtime of an image build
process and not supposed to change. Therefore the file should
be read in once and any new instance of RuntimConfig should
just use what has been read in at the first invocation. This
commit refactors the RuntimeConfig class to hold an application
global RUNTIME_CONFIG variable and allows to reread the
config on explicit request.
2021-02-08 12:41:20 +01:00
Marcus Schäfer
276d4d92d0
Added option to setup OBS API server in config
Allow to specify the open build service API server
in the kiwi runtime configuration file
2021-02-06 17:14:42 +01:00
David Cassany
43bec23db2
Add the schematron rules for installinitrd
This commit adds an schematron rule to limit the presence of
installinitrd element to disk builds including installation media.
2021-01-26 22:21:31 +01:00
David Cassany
6366624ef1
Add the installinitrd element
This commit adds the `<installinitrd>` element in kiwi schema to
facilitate the dracut modules configuration for the installation initrds
on OEM images. Within installinitrd element is possible list the dracut
modules to append, to omit or to just set an static list of dracut
modules to include.

Fixes #1676
Fixes #1683
2021-01-21 16:33:43 +01:00
David Cassany
a01b83a361
Add 'arch' attribute to preferences
This commits adds the attribute 'arch' to preferences. It works
as any other 'arch' attribute within the schema. Preferences defined
with architectures that do not match the host are ignored. If no 'arch'
is provided it matches all any host architecture.

Fixes #1640
2020-12-16 14:00:24 +01:00
Marcus Schäfer
930745268d
Added check_swap_name_used_with_lvm runtime check
The optional oem-swapname is only effective if used together
with the LVM volume manager. A name for the swap space can
only be set if it is created as a LVM volume. In any other
case the name does not apply to the system. This condition
should be checked to avoid useless settings in the image
description.
2020-12-11 15:26:38 +01:00
Marcus Schäfer
7957374de2
Allow to customize swap volume name
Added a new element <oem-swapname> below the <oemconfig>
section which allows to specify a name for the swap volume
in case the LVM volume manager is used. The default if
not specified continuous to stay at: LVSwap. This
Fixes #1638
2020-12-11 13:13:03 +01:00
Marcus Schäfer
a124d82173
Allow custom root volume name setup
In addition to the custom size of the root volume it's now
also possible to setup the name of the root volume as follows:

    <volume name="@root=rootlv"/>

If no name for the root volume is specified the default
name: LVRoot applies as before. This Fixes #1530
2020-10-27 16:17:47 +01:00
Marcus Schäfer
836633c7b2
Fix/Refactor s390 support
This changes the s390 support on several stages:

1) On s390 the boot process is based on zipl which boots into an
   initrd from which a userspace grub process is started to support
   the grub capabilities. The implementation of this concept is
   provided via the grub2-s390x-emu package. Once installed the
   setup of the bootloader is done via the grub2-mkconfig and
   grub2-install commands and therefore from a caller perspective
   the same as with any other grub2 setup process. For kiwi this
   means no extra zipl bootloader target code is needed. Therefore
   this commit deletes the zipl setup from kiwi and puts on
   the standard grub2 process. This Fixes bsc#1170863

2) To support different targettypes the grub2-s390x-emu provided
   zipl template must be adapted. Parts of the former zipl bootloader
   setup therefore now applies to an update of the zipl2grub
   template file

3) Support for CDL/LDL DASD targets has been disabled in the schema
   When testing 4k devices and a respective zipl2grub template
   setup for CDL/LDL targettype it has turned out that grub2-install
   is not able to run on such a device. My assumption is that
   the device code in grub2-install does not work for 4k devices
   with an fdasd created partition table. As this needs further
   investigations and most probably adaptions on the grub toolchain
   for s390, we disabled the setup of these modes for now.
   emulated DASD (FBA) and SCSI targets stays supported.
2020-09-30 17:14:30 +02:00
Marcus Schäfer
8967c29a12
Added consistency runtime check for the type setup
multiple type sections within one preferences section is allowed
in a kiwi image description. However, if multiple type sections
for the same image attribute are configured only the last type
configuration will be ever reachable. The proposed runtime check
in this commit detects this situation and raises an exception
showing the conflicting types including a solution suggestion
which needs to be based on profiles to distinguish between
types of the same image type name.
2020-09-21 15:25:58 +02:00
Marcus Schäfer
8d2d9c214c
Make oem be a superset of vmx
A vmx image is the same disk as an oem just without the dracut
repart/resize feature. This difference is better handled with
an oemconfig parameter <oem-resize> which allows to switch resize
on or off. The extra image type vmx will be dropped and an XSLT
stylesheet automatically transforms a vmx description to be a
oem image with the resize feature switched off.
This Fixes #1425
2020-09-16 12:56:14 +02:00
Marcus Schäfer
54ab18c96c
Make oem image use dracut by default
Before this commit an oem image still had the kiwi initrd_system
set as default. As we are decommission the custom kiwi initrd
concept the default should be changed. It is still possible to
use a custom kiwi initrd but it needs to be explicitly requested
via the initrd_system="kiwi" attribute. In addition to the
changed default a runtime check was introduced that checks
the presence of the boot= attribute which only makes sense
in combination with the kiwi initrd_system. If boot= is
set but initrd_system="kiwi" is not, a message is raised that
explains the situation and either requests setting initrd_system
properly or deleting the boot attribute. The change only
affects people who still use oem with a boot="oemboot/..."
setting and no explicit selection of kiwi as the initrd_system.
As these image type configurations should not be in use anyway
because this is all legacy and announced to go away, we
need to make the next step and enforce a new default in
code. This is related to Issue #1299
2020-09-11 09:49:17 +02:00
Marcus Schäfer
feaf6b0863
Refactor setup of bootloader parameters
The bootloader settings are handled through attributes in the
type element. Over the years some attributes were added and
there are requests for more settings. Therefore the bootloader
setup deservers its own section. With this commit the schema
changes from v7.1 to v7.2 and moves bootloader, bootloader_console
boottimeout and zipl_targettype into a new bootloader subsection
below type. The commit also adds an auto transformation template
such that customers don't have to change their image descriptions.
This is related to Issue #1401
2020-05-08 13:30:43 +02:00
Marcus Schäfer
211e446f39
Fixed ovf Machine section setup for virtualbox
Required attributes/sections name, ostype, uuid and StorageControllers
were missing according to IVirtualBox::createMachine documentation
from here:

    https://www.virtualbox.org/sdkref/interface_i_virtual_box.html

This Fixes #1322
2020-04-30 13:38:20 +02:00
Marcus Schäfer
5b8ac6692f
Disable AudioAdapter for vagrant
The kiwi template for vagrant images assumes the box files to
be used in non graphics mode and for the purpose of automation
or server deployment. Thus by default we disable Audio.
For details see the API reference:

    https://www.virtualbox.org/sdkref/interface_i_audio_adapter.html

This Fixes #1322
2020-04-22 12:17:12 +02:00
Marcus Schäfer
44a1d78e96
Added new image type: wsl
Support for building appx container images for
Windows Subsystem Linux (WSL). This Fixes #1235
2020-03-26 19:42:25 +01:00
Marcus Schäfer
3b7f5be918
Complete refactoring of fstab handling
With the new Fstab class from prior pull request there is an
opportunity to handle all fstab related actions to be done
by that class. This commit extends the Fstab class with an
add_entry method such that we can avoid the extra lists
holding raw fstab lines in e.g the disk builder. In the end
all fstab related data is stored in an instance of the Fstab
class. This also extends the KIWI api by an fstab management
class. Related to #1329 and #1349
2020-03-18 16:01:13 +01:00
Marcus Schäfer
bfcb277fe4
Added Fstab class
Handling of fstab should be done in its own namespace and class.
The current handling of fstab entries is spread at several places.
There should be only one code that writes the fstab entries and
that code should also care for the correct canonical order of
the mountpoints
2020-03-17 11:25:56 +01:00
Marcus Schäfer
04b345e7aa
Added support for spare partition fs attributes
Added new type attribute:

```xml
<type ... spare_part_fs_attributes="..."/>
```

which can be a comma separated list of the following currently
supported filesystem attributes:

* no-copy-on-write
* synchronous-updates

See chattr and filesystem manual pages for details on those
attributes. More attributes for the spare part context can be
added on request. This Fixes #1233
2020-01-15 16:15:48 +01:00
Marcus Schäfer
46a97c96c7
Added check_volume_setup_defines_reserved_labels
Check to prevent using reserved filesystem labels in LVM
volume definitions
2019-11-25 12:05:57 +01:00
Marcus Schäfer
ca75086128
Prevent swap partition to be the last one
In an OEM deployment that requested the creation of a swap
partition via <oem-swap> that swap partition was created
at first boot and was always the last partition on the disk.
This was required because it could not be placed before
any other partition without destroying those partition
contents. This process leaves the system in an inflexible
condition if the storage device can change its geometry
dynamicly as it's the case for SAN systems. The typical
deployment target for OEM images are SAN storage clusters
and it's cumbersome to resize the root partition if swap
is last.

This commit Fixes #1231 and changes the handling of swap if
requested via <oem-swap> as follows:

1. The swap space is created as part of the image build process
   and no longer on first boot of the image via dracut code.
   This increases the size of the non compressed .raw disk image
   by the configured swap space size or the default. The
   compressed versions are not affected since zero initialized
   swap space compresses to almost no space. Deployment of
   the image however also deploys the swap partition which
   increases deployment time. For big swap configurations
   it's advisable to switch off image verification via
   oem-skip-verify. For very big swap configurations it's
   also recommended to prevent kiwi from adding them as part
   of the image and let them be created on first boot via
   a systemd service that e.g places a swap file, or creates
   a swap volume when possible such that the fexibility to
   resize the rootfs is still available.

2. The setup of the swap space is now explicit. It's no longer
   calculated by twice times RAM size because on newer machines
   this could lead to huge numbers. Either the kiwi encoded
   default swap size applies or the user configured value.

3. LVM based oem disks creates the swap space as logical volume.
   The volume is created as part of the image build process
   and no longer on first boot. The swap volume at build time
   of the image is of a minimal size and gets resized on first
   boot.

4. The move of the swap creation into the builder code also
   handles swap per configured device persistency schema like
   any other devices. This means by default swap is mounted
   via by-uuid name and thus also Fixes #1259
2019-11-25 12:05:57 +01:00
David Cassany
80e791d2ef
Make home attribute optional on users definition
This commit makes home attribute of users definition optional. This way
the home directory is created according to the distribution defaults.
Note that `-m` option is passed to user add, thus the home directory is
requested to be created even without having an explicit home path
defined.

Fixes #1025
2019-10-23 15:57:51 +02:00
Marcus Schäfer
defdc23fcf
Follow up patch for SourceType info
The message that display the selected source type was
using the former metalink text which is wrong. This
patch corrects the message and only display it if a
sourcetype was specified
2019-09-30 16:39:16 +02:00
Marcus Schäfer
30e18d1398
Added support for fscreateoptions attribute
Along with the fsmountoptions attribute there is now also the
fscreateoptions attribute which allows to control the options
used for creating the filesystem. Please note, it's not kiwi's
task to validate that the given option string is supported
by the selected filesystem. This means providing wrong values
here will cause the build to fail at the time the filesystem
gets created. This Fixes #1109
2019-07-18 15:55:39 +02:00
Marcus Schäfer
dab7b8732a
Refactor ISO boot, use grub in EFI mode
Before this commit isolinux was used to boot an ISO and
thus forms a hard requirement. isolinux however only
exists for the x86 platform. This limitation did not
allow to create ISO images for other platforms. With this
commit a refactoring of the ISO boot setup is introduced.

* isolinux is only used if the bios firmware is requested
  and the platform matches the x86 architecture. A runtime
  check will check for this condition and exits early if
  not applicable

* in case of the EFI firmware we already used grub in
  EFI mode but still had isolinux in place for the legacy/CSM
  boot. That part is now also replaced by a platform specific
  grub eltorito image and grub's boot_hybrid.img for hybrid
  boot. On platforms that do not provide those modules the
  support for it will be skipped

With this change in place it's possible to control the ISO
boot layout through the firmware setup and all platform
specific modules are handled as such. Therefore we also
deleted the syslinux requirement. This Fixes #1092
2019-07-02 11:56:02 +02:00
Marcus Schäfer
ab59d4d217
Delete obsolete repository types
Deleted red-carpet, slack-site, up2date-mirrors, urpmi and yast2
from the allowed values list of the repository type attribute.
This Fixes #1029
2019-04-22 23:15:42 +02:00
Marcus Schäfer
4a21fcfb05
Auto upgrade all image descriptions to v7.0 schema 2019-04-05 14:28:55 +02:00
Marcus Schäfer
c148d661b9
Deleted pxedeploy section
There is no further demand in the client config creation for the
legacy netboot code. Customers using the netboot initrd already
create the client config.<MAC> file manually or through another
system. With the next generation kiwi we also recommend the disk
based network deployment using the dracut capabilities and marked
the netboot code as still supported but on the legacy stream.
Along with the deletion in the schema the following changes
were also done:

* Deleted use of pxedeploy section in implementation
* Increase schema version to v7.0
* Update documentation on schema version update
* Added xsl stylesheet conversion from v6.9 to v7.0
  Automatic schema version upgrade from v6.9 to v7.0 deletes
  the pxedeploy section from any type specification

This Fixes #19
2019-04-05 14:26:54 +02:00
David Cassany
c21570ac7b
Add the possibility to disable runtime checks
This commit adds runtime configuration parameters to disable the runtime
checks. This is specially handy if someone does not want to use the kiwi
dracut modules and provide custom ones instead. In orde disable some
runtime check consider a runtime config yaml like:

runtime_checks:
    - disable:
        - check_dracut_module_for_oem_install_in_package_list
        - check_dracut_module_for_live_iso_in_package_list

This commit fixes #893
2019-04-02 16:00:03 +02:00
Marcus Schäfer
b43ac0f24f
Cleanup example_dot_profile_config.xml
Don't use multiple all size volumes
2019-03-07 14:02:55 +01:00
Marcus Schäfer
00b8213ac9
Check for multiple allsize volumes setup
Added runtime check check_volume_setup_defines_multiple_fullsize_volumes
which reads the volume setup and counts the number of volumes that
define a 'all' volume. Only one is allowed
2019-03-07 11:28:30 +01:00
Dan Čermák
2b17684d2c
Add support to build vagrant boxes with the Virtualbox provider
This commits adds initial support to build Vagrant boxes with the Virtualbox
provider.
- The attribute 'provider' of the option <vagrantconfig> now supports the option
  "virtualbox".
- A new boolean attribute 'virtualbox_guest_additions_present' was added to
  <vagrantconfig>. When set to true, KIWI will assume that guest additions have
  been installed into the base image and configure the shared folders to use
  vboxfs, otherwise rsync will be used.

This fixes #532 (https://github.com/SUSE/kiwi/issues/532).
2019-03-05 15:33:18 +01:00
Marcus Schäfer
ed6c11f7d6
Added get_description method to XMLState
In preparation to access the contents of the <description>
section this pull requests adds a simple method to read
the contents.
2019-02-26 15:20:21 +01:00
Marcus Schäfer
f8265bb607
Added new preferences subsection
In the preferences section the following optional subsection
can be configured:

   <rpm-locale-filtering>true|false</rpm-locale-filtering>

If set to true the default locales POSIX, C, and C.UTF-8 are
applied as rpm install_lang macro. If the locale section is
configured in addition the list is extended by that information
too
2019-02-08 16:56:56 +01:00
Marcus Schäfer
fad1c0f1ef
Added support for %_install_langs rpm macro
During the image build the custom rpm macro %_install_langs
is configured with the <locale> setup from the KIWI XML
description. This allows to filter language specific packages
on the rpm level and Fixes #771
2019-02-07 16:05:25 +01:00
David Cassany
0f4b0b34bb
Add a runtime check for preferences metadata
This commit adds a runtime check for preferences metadata. More
specfic verifies there is a packagemanager defined and an image version
defined.

Fixes #925
2019-01-28 17:16:31 +01:00
Marcus Schäfer
3346f3028f
Make result compression in the bundler optional
Calling kiwi result bundle will take the image build results
and bundle the relevant image files according to their image
type. Depending on the result configuration this could instruct
the bundler to compress one or more files from the result.
By default this compression is switched off in the bundler but
can be activated to save storage space and speedup download
of the image with the following runtime configuration:

bundle:
  - compress: true|false

If compression is activated the result image has to be
uncompressed before it can be used. This Fixes #901
2019-01-10 16:13:02 +01:00
David Cassany
9601ab669c
Fix disk size calculation for VMX
Disk size calculation must take into account the empty volumes that
are to be mounted in a directory that does not exist in the root tree
otherwise there is KeyError. The result of
storate/setup._calculate_volume_mbytes must be a dict including all
defined volumes.

Fixes #904
2018-12-18 14:56:54 +01:00
David Cassany
f0612486dd
Add history metadata for container builds
This commit adds the history section in contianerconfig. With it
'author', 'created_by' and 'comment' can be customized. In addition
'created' is always included with the image creation date time.
'created_by' entry is set to 'KIWI __version__' by default if nothing
is provided.

Fixes #852
2018-11-08 15:33:04 +01:00
Marcus Schäfer
e51b91cb38
Added OCI tooling interface class
An initiative to formulate industry standards around container
formats and runtime is available at https://www.opencontainers.org
Different tools to implement the specifications had been
created. The purpose of this class and its sub-classes is
to provide a common interface in kiwi to allow using all
tools such that the container support in kiwi covers every
linux distribution no matter what tooling was preferred.
Related to Issue #859
2018-10-30 09:35:58 +01:00