=============== Configuration =============== Please read `productmd documentation `_ for `terminology `_ and other release and compose related details. Minimal Config Example ====================== :: # RELEASE release_name = "Fedora" release_short = "Fedora" release_version = "23" # GENERAL SETTINGS comps_file = "comps-f23.xml" variants_file = "variants-f23.xml" # KOJI koji_profile = "koji" runroot = False # PKGSET sigkeys = [None] pkgset_source = "koji" pkgset_koji_tag = "f23" # GATHER gather_method = "deps" greedy_method = "build" check_deps = False # BUILDINSTALL buildinstall_method = "lorax" Release ======= Following **mandatory** options describe a release. Options ------- **release_name** [mandatory] (*str*) -- release name **release_short** [mandatory] (*str*) -- release short name, without spaces and special characters **release_version** [mandatory] (*str*) -- release version **release_type** = "ga" (*str*) -- release type, for example ``ga``, ``updates`` or ``updates-testing``. See `list of all valid values `_ in productmd documentation. **release_internal** = False (*bool*) -- whether the compose is meant for public consumption **treeinfo_version** (*str*) Version to display in ``.treeinfo`` files. If not configured, the value from ``release_version`` will be used. Example ------- :: release_name = "Fedora" release_short = "Fedora" release_version = "23" # release_type = "ga" Base Product ============ Base product options are **optional** and we need to them only if we're composing a layered product built on another (base) product. Options ------- **base_product_name** (*str*) -- base product name **base_product_short** (*str*) -- base product short name, without spaces and special characters **base_product_version** (*str*) -- base product **major** version **base_product_type** = "ga" (*str*) -- base product type, "ga", "updates" etc., for full list see documentation of *productmd*. Example ------- :: release_name = "RPM Fusion" release_short = "rf" release_version = "23.0" base_product_name = "Fedora" base_product_short = "Fedora" base_product_version = "23" General Settings ================ Options ------- **comps_file** [mandatory] (:ref:`scm_dict `, *str* or None) -- reference to comps XML file with installation groups **variants_file** [mandatory] (:ref:`scm_dict ` or *str*) -- reference to variants XML file that defines release variants and architectures **module_defaults_dir** [optional] (:ref:`scm_dict ` or *str*) -- reference the module defaults directory containing modulemd-defaults YAML documents. Files relevant for modules included in the compose will be embedded in the generated repodata and available for DNF. :: module_defaults_dir = { "scm": "git", "repo": "https://pagure.io/releng/fedora-module-defaults.git", "dir": ".", } **failable_deliverables** [optional] (*list*) -- list which deliverables on which variant and architecture can fail and not abort the whole compose. This only applies to ``buildinstall`` and ``iso`` parts. All other artifacts can be configured in their respective part of configuration. Please note that ``*`` as a wildcard matches all architectures but ``src``. **comps_filter_environments** [optional] (*bool*) -- When set to ``False``, the comps files for variants will not have their environments filtered to match the variant. **tree_arches** ([*str*]) -- list of architectures which should be included; if undefined, all architectures from variants.xml will be included **tree_variants** ([*str*]) -- list of variants which should be included; if undefined, all variants from variants.xml will be included **repoclosure_strictness** (*list*) -- variant/arch mapping describing how repoclosure should run. Possible values are * ``off`` -- do not run repoclosure * ``lenient`` -- (default) run repoclosure and write results to logs, but detected errors are only reported in logs * ``fatal`` -- abort compose when any issue is detected When multiple blocks in the mapping match a variant/arch combination, the last value will win. **repoclosure_backend** (*str*) -- Select which tool should be used to run repoclosure over created repositories. By default ``yum`` is used, but you can switch to ``dnf``. Please note that when ``dnf`` is used, the build dependencies check is skipped. On Python 3, only ``dnf`` backend is available. See also: the ``gather_backend`` setting for Pungi's gather phase. **cts_url** (*str*) -- URL to Compose Tracking Service. If defined, Pungi will add the compose to Compose Tracking Service and ge the compose ID from it. For example ``https://cts.localhost.tld/`` **cts_keytab** (*str*) -- Path to Kerberos keytab which will be used for Compose Tracking Service Kerberos authentication. If not defined, the default Kerberos principal is used. **compose_type** (*str*) -- Allows to set default compose type. Type set via a command-line option overwrites this. **mbs_api_url** (*str*) -- URL to Module Build Service (MBS) API. For example ``https://mbs.example.com/module-build-service/2``. This is required by ``pkgset_scratch_modules``. Example ------- :: comps_file = { "scm": "git", "repo": "https://git.fedorahosted.org/git/comps.git", "branch": None, "file": "comps-f23.xml.in", } variants_file = { "scm": "git", "repo": "https://pagure.io/pungi-fedora.git ", "branch": None, "file": "variants-fedora.xml", } failable_deliverables = [ ('^.*$', { # Buildinstall can fail on any variant and any arch '*': ['buildinstall'], 'src': ['buildinstall'], # Nothing on i386 blocks the compose 'i386': ['buildinstall', 'iso', 'live'], }) ] tree_arches = ["x86_64"] tree_variants = ["Server"] repoclosure_strictness = [ # Make repoclosure failures fatal for compose on all variants … ('^.*$', {'*': 'fatal'}), # … except for Everything where it should not run at all. ('^Everything$', {'*': 'off'}) ] Image Naming ============ Both image name and volume id are generated based on the configuration. Since the volume id is limited to 32 characters, there are more settings available. The process for generating volume id is to get a list of possible formats and try them sequentially until one fits in the length limit. If substitutions are configured, each attempted volume id will be modified by it. For layered products, the candidate formats are first ``image_volid_layered_product_formats`` followed by ``image_volid_formats``. Otherwise, only ``image_volid_formats`` are tried. If no format matches the length limit, an error will be reported and compose aborted. Options ------- There a couple common format specifiers available for both the options: * ``compose_id`` * ``release_short`` * ``version`` * ``date`` * ``respin`` * ``type`` * ``type_suffix`` * ``label`` * ``label_major_version`` * ``variant`` * ``arch`` * ``disc_type`` **image_name_format** [optional] (*str|dict*) -- Python's format string to serve as template for image names. The value can also be a dict mapping variant UID regexes to the format string. The pattern should not overlap, otherwise it is undefined which one will be used. This format will be used for all phases generating images. Currently that means ``createiso``, ``live_images`` and ``buildinstall``. Available extra keys are: * ``disc_num`` * ``suffix`` **image_volid_formats** [optional] (*list*) -- A list of format strings for generating volume id. The extra available keys are: * ``base_product_short`` * ``base_product_version`` **image_volid_layered_product_formats** [optional] (*list*) -- A list of format strings for generating volume id for layered products. The keys available are the same as for ``image_volid_formats``. **restricted_volid** = False (*bool*) -- New versions of lorax replace all non-alphanumerical characters with dashes (underscores are preserved). This option will mimic similar behaviour in Pungi. **volume_id_substitutions** [optional] (*dict*) -- A mapping of string replacements to shorten the volume id. **disc_types** [optional] (*dict*) -- A mapping for customizing ``disc_type`` used in image names. Available keys are: * ``boot`` -- for ``boot.iso`` images created in *buildinstall* phase * ``live`` -- for images created by *live_images* phase * ``dvd`` -- for images created by *createiso* phase * ``ostree`` -- for ostree installer images Default values are the same as the keys. Example ------- :: # Image name respecting Fedora's image naming policy image_name_format = "%(release_short)s-%(variant)s-%(disc_type)s-%(arch)s-%(version)s%(suffix)s" # Use the same format for volume id image_volid_formats = [ "%(release_short)s-%(variant)s-%(disc_type)s-%(arch)s-%(version)s" ] # No special handling for layered products, use same format as for regular images image_volid_layered_product_formats = [] # Replace "Cloud" with "C" in volume id etc. volume_id_substitutions = { 'Cloud': 'C', 'Alpha': 'A', 'Beta': 'B', 'TC': 'T', } disc_types = { 'boot': 'netinst', 'live': 'Live', 'dvd': 'DVD', } Signing ======= If you want to sign deliverables generated during pungi run like RPM wrapped images. You must provide few configuration options: **signing_command** [optional] (*str*) -- Command that will be run with a koji build as a single argument. This command must not require any user interaction. If you need to pass a password for a signing key to the command, do this via command line option of the command and use string formatting syntax ``%(signing_key_password)s``. (See **signing_key_password_file**). **signing_key_id** [optional] (*str*) -- ID of the key that will be used for the signing. This ID will be used when crafting koji paths to signed files (``kojipkgs.fedoraproject.org/packages/NAME/VER/REL/data/signed/KEYID/..``). **signing_key_password_file** [optional] (*str*) -- Path to a file with password that will be formatted into **signing_command** string via ``%(signing_key_password)s`` string format syntax (if used). Because pungi config is usually stored in git and is part of compose logs we don't want password to be included directly in the config. Note: If ``-`` string is used instead of a filename, then you will be asked for the password interactivelly right after pungi starts. Example ------- :: signing_command = '~/git/releng/scripts/sigulsign_unsigned.py -vv --password=%(signing_key_password)s fedora-24' signing_key_id = '81b46521' signing_key_password_file = '~/password_for_fedora-24_key' .. _git-urls: Git URLs ======== In multiple places the config requires URL of a Git repository to download some file from. This URL is passed on to *Koji*. It is possible to specify which commit to use using this syntax: :: git://git.example.com/git/repo-name.git?# The ```` pattern can be replaced with actual commit SHA, a tag name, ``HEAD`` to indicate that tip of default branch should be used or ``origin/`` to use tip of arbitrary branch. If the URL specifies a branch or ``HEAD``, *Pungi* will replace it with the actual commit SHA. This will later show up in *Koji* tasks and help with tracing what particular inputs were used. .. note:: The ``origin`` must be specified because of the way *Koji* works with the repository. It will clone the repository then switch to requested state with ``git reset --hard REF``. Since no local branches are created, we need to use full specification including the name of the remote. Createrepo Settings =================== Options ------- **createrepo_checksum** (*str*) -- specify checksum type for createrepo; expected values: ``sha512``, ``sha256``, ``sha1``. Defaults to ``sha256``. **createrepo_c** = True (*bool*) -- use createrepo_c (True) or legacy createrepo (False) **createrepo_deltas** = False (*list*) -- generate delta RPMs against an older compose. This needs to be used together with ``--old-composes`` command line argument. The value should be a mapping of variants and architectures that should enable creating delta RPMs. Source and debuginfo repos never have deltas. **createrepo_use_xz** = False (*bool*) -- whether to pass ``--xz`` to the createrepo command. This will cause the SQLite databases to be compressed with xz. **createrepo_num_threads** (*int*) -- how many concurrent ``createrepo`` process to run. The default is to use one thread per CPU available on the machine. **createrepo_num_workers** (*int*) -- how many concurrent ``createrepo`` workers to run. Value defaults to 3. **createrepo_database** (*bool*) -- whether to create SQLite database as part of the repodata. This is only useful as an optimization for clients using Yum to consume to the repo. Default value depends on gather backend. For DNF it's turned off, for Yum the default is ``True``. **createrepo_extra_args** (*[str]*) -- a list of extra arguments passed on to ``createrepo`` or ``createrepo_c`` executable. This could be useful for enabling zchunk generation and pointing it to correct dictionaries. **createrepo_extra_modulemd** (*dict*) -- a mapping of variant UID to :ref:`an scm dict `. If specified, it should point to a directory with extra module metadata YAML files that will be added to the repository for this variant. The cloned files should be split into subdirectories for each architecture of the variant. **createrepo_enable_cache** = True (*bool*) -- whether to use ``--cachedir`` option of ``createrepo``. It will cache and reuse checksum vaules to speed up createrepo phase. The cache dir is located at ``/var/cache/pungi/createrepo_c/$release_short-$uid`` e.g. /var/cache/pungi/createrepo_c/Fedora-1000 **product_id** = None (:ref:`scm_dict `) -- If specified, it should point to a directory with certificates ``*--*.pem``. Pungi will copy each certificate file into the relevant Yum repositories as a ``productid`` file in the ``repodata`` directories. The purpose of these ``productid`` files is to expose the product data to `subscription-manager `_. subscription-manager includes a "product-id" Yum plugin that can read these ``productid`` certificate files from each Yum repository. **product_id_allow_missing** = False (*bool*) -- When ``product_id`` is used and a certificate for some variant and architecture is missing, Pungi will exit with an error by default. When you set this option to ``True``, Pungi will ignore the missing certificate and simply log a warning message. **product_id_allow_name_prefix** = True (*bool*) -- Allow arbitrary prefix for the certificate file name (see leading ``*`` in the pattern above). Setting this option to ``False`` will make the pattern more strict by requiring the file name to start directly with variant name. Example ------- :: createrepo_checksum = "sha" createrepo_deltas = [ # All arches for Everything should have deltas. ('^Everything$', {'*': True}), # Also Server.x86_64 should have them (but not on other arches). ('^Server$', {'x86_64': True}), ] createrepo_extra_modulemd = { "Server": { "scm": "git", "repo": "https://example.com/extra-server-modulemd.git", "dir": ".", # The directory should have this layout. Each architecture for the # variant should be included (even if the directory is empty. # . # ├── aarch64 # │ ├── some-file.yaml # │ └ ... # └── x86_64 } } Package Set Settings ==================== Options ------- **sigkeys** ([*str* or None]) -- priority list of signing key IDs. These key IDs match the key IDs for the builds in Koji. Pungi will choose signed packages according to the order of the key IDs that you specify here. Use one single key in this list to ensure that all RPMs are signed by one key. If the list includes an empty string or *None*, Pungi will allow unsigned packages. If the list only includes *None*, Pungi will use all unsigned packages. **pkgset_source** [mandatory] (*str*) -- "koji" (any koji instance) or "repos" (arbitrary yum repositories) **pkgset_koji_tag** (*str|[str]*) -- tag(s) to read package set from. This option can be omitted for modular composes. **pkgset_koji_builds** (*str|[str]*) -- extra build(s) to include in a package set defined as NVRs. **pkgset_koji_scratch_tasks** (*str|[str]*) -- RPM scratch build task(s) to include in a package set, defined as task IDs. This option can be used only when ``compose_type`` is set to ``test``. The RPM still needs to have higher NVR than any other RPM with the same name coming from other sources in order to appear in the resulting compose. **pkgset_koji_module_tag** (*str|[str]*) -- tags to read module from. This option works similarly to listing tags in variants XML. If tags are specified and variants XML specifies some modules via NSVC (or part of), only modules matching that list will be used (and taken from the tag). Inheritance is used automatically. **pkgset_koji_module_builds** (*dict*) -- A mapping of variants to extra module builds to include in a package set: ``{variant: [N:S:V:C]}``. **pkgset_koji_inherit** = True (*bool*) -- inherit builds from parent tags; we can turn it off only if we have all builds tagged in a single tag **pkgset_koji_inherit_modules** = False (*bool*) -- the same as above, but this only applies to modular tags. This option applies to the content tags that contain the RPMs. **pkgset_repos** (*dict*) -- A mapping of architectures to repositories with RPMs: ``{arch: [repo]}``. Only use when ``pkgset_source = "repos"``. **pkgset_scratch_modules** (*dict*) -- A mapping of variants to scratch module builds: ``{variant: [N:S:V:C]}``. Requires ``mbs_api_url``. **pkgset_exclusive_arch_considers_noarch** = True (*bool*) -- If a package includes ``noarch`` in its ``ExclusiveArch`` tag, it will be included in all architectures since ``noarch`` is compatible with everything. Set this option to ``False`` to ignore ``noarch`` in ``ExclusiveArch`` and always consider only binary architectures. **pkgset_allow_reuse** = True (*bool*) -- When set to ``True``, *Pungi* will try to reuse pkgset data from the old composes specified by ``--old-composes``. When enabled, this option can speed up new composes because it does not need to calculate the pkgset data from Koji. However, if you block or unblock a package in Koji (for example) between composes, then Pungi may not respect those changes in your new compose. **signed_packages_retries** = 0 (*int*) -- In automated workflows, you might start a compose before Koji has completely written all signed packages to disk. In this case you may want Pungi to wait for the package to appear in Koji's storage. This option controls how many times Pungi will retry looking for the signed copy. **signed_packages_wait** = 30 (*int*) -- Interval in seconds for how long to wait between attempts to find signed packages. This option only makes sense when ``signed_packages_retries`` is set higher than 0. Example ------- :: sigkeys = [None] pkgset_source = "koji" pkgset_koji_tag = "f23" Buildinstall Settings ===================== Script or process that creates bootable images with Anaconda installer is historically called `buildinstall `_. Options ------- **buildinstall_method** (*str*) -- "lorax" (f16+, rhel7+) or "buildinstall" (older releases) **lorax_options** (*list*) -- special options passed on to *lorax*. Format: ``[(variant_uid_regex, {arch|*: {option: name}})]``. Recognized options are: * ``bugurl`` -- *str* (default ``None``) * ``nomacboot`` -- *bool* (default ``True``) * ``noupgrade`` -- *bool* (default ``True``) * ``add_template`` -- *[str]* (default empty) * ``add_arch_template`` -- *[str]* (default empty) * ``add_template_var`` -- *[str]* (default empty) * ``add_arch_template_var`` -- *[str]* (default empty) * ``rootfs_size`` -- [*int*] (default empty) * ``version`` -- [*str*] (default from ``treeinfo_version`` or ``release_version``) -- used as ``--version`` and ``--release`` argument on the lorax command line * ``dracut_args`` -- [*[str]*] (default empty) override arguments for dracut. Please note that if this option is used, lorax will not use any other arguments, so you have to provide a full list and can not just add something. * ``skip_branding`` -- *bool* (default ``False``) * ``squashfs_only`` -- *bool* (default ``False``) pass the --squashfs_only to Lorax. * ``configuration_file`` -- (:ref:`scm_dict `) (default empty) pass the specified configuration file to Lorax using the -c option. **lorax_extra_sources** (*list*) -- a variant/arch mapping with urls for extra source repositories added to Lorax command line. Either one repo or a list can be specified. **lorax_use_koji_plugin** = False (*bool*) -- When set to ``True``, the Koji pungi_buildinstall task will be used to execute Lorax instead of runroot. Use only if the Koji instance has the pungi_buildinstall plugin installed. **buildinstall_kickstart** (:ref:`scm_dict `) -- If specified, this kickstart file will be copied into each file and pointed to in boot configuration. **buildinstall_topdir** (*str*) -- Full path to top directory where the runroot buildinstall Koji tasks output should be stored. This is useful in situation when the Pungi compose is not generated on the same storage as the Koji task is running on. In this case, Pungi can provide input repository for runroot task using HTTP and set the output directory for this task to ``buildinstall_topdir``. Once the runroot task finishes, Pungi will copy the results of runroot tasks to the compose working directory. **buildinstall_skip** (*list*) -- mapping that defines which variants and arches to skip during buildinstall; format: ``[(variant_uid_regex, {arch|*: True})]``. This is only supported for lorax. **buildinstall_allow_reuse** = False (*bool*) -- When set to ``True``, *Pungi* will try to reuse buildinstall results from old compose specified by ``--old-composes``. **buildinstall_packages** (list) – Additional packages to be installed in the runroot environment where lorax will run to create installer. Format: ``[(variant_uid_regex, {arch|*: [package_globs]})]``. Example ------- :: buildinstall_method = "lorax" # Enables macboot on x86_64 for all variants and builds upgrade images # everywhere. lorax_options = [ ("^.*$", { "x86_64": { "nomacboot": False } "*": { "noupgrade": False } }) ] # Don't run buildinstall phase for Modular variant buildinstall_skip = [ ('^Modular', { '*': True }) ] # Add another repository for lorax to install packages from lorax_extra_sources = [ ('^Simple$', { '*': 'https://example.com/repo/$basearch/', }) ] # Additional packages to be installed in the Koji runroot environment where # lorax will run. buildinstall_packages = [ ('^Simple$', { '*': ['dummy-package'], }) ] .. note:: It is advised to run buildinstall (lorax) in koji, i.e. with **runroot enabled** for clean build environments, better logging, etc. .. warning:: Lorax installs RPMs into a chroot. This involves running %post scriptlets and they frequently run executables in the chroot. If we're composing for multiple architectures, we **must** use runroot for this reason. Gather Settings =============== Options ------- **gather_method** [mandatory] (*str*|*dict*) -- Options are ``deps``, ``nodeps`` and ``hybrid``. Specifies whether and how package dependencies should be pulled in. Possible configuration can be one value for all variants, or if configured per-variant it can be a simple string ``hybrid`` or a a dictionary mapping source type to a value of ``deps`` or ``nodeps``. Make sure only one regex matches each variant, as there is no guarantee which value will be used if there are multiple matching ones. All used sources must have a configured method unless hybrid solving is used. **gather_fulltree** = False (*bool*) -- When set to ``True`` all RPMs built from an SRPM will always be included. Only use when ``gather_method = "deps"``. **gather_selfhosting** = False (*bool*) -- When set to ``True``, *Pungi* will build a self-hosting tree by following build dependencies. Only use when ``gather_method = "deps"``. **gather_allow_reuse** = False (*bool*) -- When set to ``True``, *Pungi* will try to reuse gather results from old compose specified by ``--old-composes``. **greedy_method** = none (*str*) -- This option controls how package requirements are satisfied in case a particular ``Requires`` has multiple candidates. * ``none`` -- the best packages is selected to satisfy the dependency and only that one is pulled into the compose * ``all`` -- packages that provide the symbol are pulled in * ``build`` -- the best package is selected, and then all packages from the same build that provide the symbol are pulled in .. note:: As an example let's work with this situation: a package in the compose has ``Requires: foo``. There are three packages with ``Provides: foo``: ``pkg-a``, ``pkg-b-provider-1`` and ``pkg-b-provider-2``. The ``pkg-b-*`` packages are build from the same source package. Best match determines ``pkg-b-provider-1`` as best matching package. * With ``greedy_method = "none"`` only ``pkg-b-provider-1`` will be pulled in. * With ``greedy_method = "all"`` all three packages will be pulled in. * With ``greedy_method = "build"`` ``pkg-b-provider-1`` and ``pkg-b-provider-2`` will be pulled in. **gather_backend** (*str*) --This changes the entire codebase doing dependency solving, so it can change the result in unpredictable ways. On Python 2, the choice is between ``yum`` or ``dnf`` and defaults to ``yum``. On Python 3 ``dnf`` is the only option and default. Particularly the multilib work is performed differently by using ``python-multilib`` library. Please refer to ``multilib`` option to see the differences. See also: the ``repoclosure_backend`` setting for Pungi's repoclosure phase. **multilib** (*list*) -- mapping of variant regexes and arches to list of multilib methods Available methods are: * ``none`` -- no package matches this method * ``all`` -- all packages match this method * ``runtime`` -- packages that install some shared object file (``*.so.*``) will match. * ``devel`` -- packages whose name ends with ``-devel`` or ``--static`` suffix will be matched. When ``dnf`` is used, this method automatically enables ``runtime`` method as well. With ``yum`` backend this method also uses a hardcoded blacklist and whitelist. * ``kernel`` -- packages providing ``kernel`` or ``kernel-devel`` match this method (only in ``yum`` backend) * ``yaboot`` -- only ``yaboot`` package on ``ppc`` arch matches this (only in ``yum`` backend) .. _additional_packages: **additional_packages** (*list*) -- additional packages to be included in a variant and architecture; format: ``[(variant_uid_regex, {arch|*: [package_globs]})]`` In contrast to the ``comps_file`` setting, the ``additional_packages`` setting merely adds the list of packages to the compose. When a package is in a comps group, it is visible to users via ``dnf groupinstall`` and Anaconda's Groups selection, but ``additional_packages`` does not affect DNF groups. The packages specified here are matched against RPM names, not any other provides in the package nor the name of source package. Shell globbing is used, so wildcards are possible. The package can be specified as name only or ``name.arch``. With ``dnf`` gathering backend, you can specify a debuginfo package to be included. This is meant to include a package if autodetection does not get it. If you add a debuginfo package that does not have anything else from the same build included in the compose, the sources will not be pulled in. If you list a package in ``additional_packages`` but Pungi cannot find it (for example, it's not available in the Koji tag), Pungi will log a warning in the "work" or "logs" directories and continue without aborting. *Example*: This configuration will add all packages in a Koji tag to an "Everything" variant:: additional_packages = [ ('^Everything$', { '*': [ '*', ], }) ] **filter_packages** (*list*) -- packages to be excluded from a variant and architecture; format: ``[(variant_uid_regex, {arch|*: [package_globs]})]`` See :ref:`additional_packages ` for details about package specification. **filter_modules** (*list*) -- modules to be excluded from a variant and architecture; format: ``[(variant_uid_regex, {arch|*: [name:stream]})]`` Both name and stream can use shell-style globs. If stream is omitted, all streams are removed. This option only applies to modules taken from Koji tags, not modules explicitly listed in variants XML without any tags. **filter_system_release_packages** (*bool*) -- for each variant, figure out the best system release package and filter out all others. This will not work if a variant needs more than one system release package. In such case, set this option to ``False``. **gather_prepopulate** = None (:ref:`scm_dict `) -- If specified, you can use this to add additional packages. The format of the file pointed to by this option is a JSON mapping ``{variant_uid: {arch: {build: [package]}}}``. Packages added through this option can not be removed by ``filter_packages``. **multilib_blacklist** (*dict*) -- multilib blacklist; format: ``{arch|*: [package_globs]}``. See :ref:`additional_packages ` for details about package specification. **multilib_whitelist** (*dict*) -- multilib blacklist; format: ``{arch|*: [package_names]}``. The whitelist must contain exact package names; there are no wildcards or pattern matching. **gather_lookaside_repos** = [] (*list*) -- lookaside repositories used for package gathering; format: ``[(variant_uid_regex, {arch|*: [repo_urls]})]`` The repo_urls are passed to the depsolver, which can use packages in the repos for satisfying dependencies, but the packages themselves are not pulled into the compose. The repo_urls can contain $basearch variable, which will be substituted with proper value by the depsolver. The repo_urls are used by repoclosure too, but it can't parse $basearch currently and that will cause Repoclosure phase crashed. *repoclosure_strictness* option could be used to stop running repoclosure. Please note that * as a wildcard matches all architectures but src. **hashed_directories** = False (*bool*) -- put packages into "hashed" directories, for example ``Packages/k/kernel-4.0.4-301.fc22.x86_64.rpm`` **check_deps** = True (*bool*) -- Set to ``False`` if you don't want the compose to abort when some package has broken dependencies. **require_all_comps_packages** = False (*bool*) -- Set to ``True`` to abort compose when package mentioned in comps file can not be found in the package set. When disabled (the default), such cases are still reported as warnings in the log. **gather_source_mapping** (*str*) -- JSON mapping with initial packages for the compose. The value should be a path to JSON file with following mapping: ``{variant: {arch: {rpm_name: [rpm_arch|None]}}}``. Relative paths are interpreted relative to the location of main config file. **gather_profiler** = False (*bool*) -- When set to ``True`` the gather tool will produce additional performance profiling information at the end of its logs. Only takes effect when ``gather_backend = "dnf"``. **variant_as_lookaside** (*list*) -- a variant/variant mapping that tells one or more variants in compose has other variant(s) in compose as a lookaside. Only top level variants are supported (not addons/layered products). Format: ``[(variant_uid, variant_uid)]`` Example ------- :: gather_method = "deps" greedy_method = "build" check_deps = False hashed_directories = True gather_method = { "^Everything$": { "comps": "deps" # traditional content defined by comps groups }, "^Modular$": { "module": "nodeps" # Modules do not need dependencies }, "^Mixed$": { # Mixed content in one variant "comps": "deps", "module": "nodeps" } "^OtherMixed$": "hybrid", # Using hybrid depsolver } additional_packages = [ # bz#123456 ('^(Workstation|Server)$', { '*': [ 'grub2', 'kernel', ], }), ] filter_packages = [ # bz#111222 ('^.*$', { '*': [ 'kernel-doc', ], }), ] multilib = [ ('^Server$', { 'x86_64': ['devel', 'runtime'] }) ] multilib_blacklist = { "*": [ "gcc", ], } multilib_whitelist = { "*": [ "alsa-plugins-*", ], } # gather_lookaside_repos = [ # ('^.*$', { # '*': [ # "https://dl.fedoraproject.org/pub/fedora/linux/releases/22/Everything/$basearch/os/", # ], # 'x86_64': [ # "https://dl.fedoraproject.org/pub/fedora/linux/releases/22/Everything/source/SRPMS/", # ] # }), # ] .. note:: It is a good practice to attach bug/ticket numbers to additional_packages, filter_packages, multilib_blacklist and multilib_whitelist to track decisions. Koji Settings ============= Options ------- **koji_profile** (*str*) -- koji profile name. This tells Pungi how to communicate with your chosen Koji instance. See `Koji's documentation about profiles `_ for more information about how to set up your Koji client profile. In the examples, the profile name is "koji", which points to Fedora's koji.fedoraproject.org. **global_runroot_method** (*str*) -- global runroot method to use. If ``runroot_method`` is set per Pungi phase using a dictionary, this option defines the default runroot method for phases not mentioned in the ``runroot_method`` dictionary. **runroot_method** (*str*|*dict*) -- Runroot method to use. It can further specify the runroot method in case the ``runroot`` is set to True. Available methods are: * ``local`` -- runroot tasks are run locally * ``koji`` -- runroot tasks are run in Koji * ``openssh`` -- runroot tasks are run on remote machine connected using OpenSSH. The ``runroot_ssh_hostnames`` for each architecture must be set and the user under which Pungi runs must be configured to login as ``runroot_ssh_username`` using the SSH key. The runroot method can also be set per Pungi phase using the dictionary with phase name as key and runroot method as value. The default runroot method is in this case defined by the ``global_runroot_method`` option. Example ------- :: global_runroot_method = "koji" runroot_method = { "createiso": "local" } **runroot_channel** (*str*) -- name of koji channel **runroot_tag** (*str*) -- name of koji **build** tag used for runroot **runroot_weights** (*dict*) -- customize task weights for various runroot tasks. The values in the mapping should be integers, the keys can be selected from the following list. By default no weight is assigned and Koji picks the default one according to policy. * ``buildinstall`` * ``createiso`` * ``ostree`` * ``ostree_installer`` Example ------- :: koji_profile = "koji" runroot_channel = "runroot" runroot_tag = "f23-build" Runroot "openssh" method settings ================================= Options ------- **runroot_ssh_username** (*str*) -- For ``openssh`` runroot method, configures the username used to login the remote machine to run the runroot task. Defaults to "root". **runroot_ssh_hostnames** (*dict*) -- For ``openssh`` runroot method, defines the hostname for each architecture on which the runroot task should be running. Format: ``{"x86_64": "runroot-x86-64.localhost.tld", ...}`` **runroot_ssh_init_template** (*str*) [optional] -- For ``openssh`` runroot method, defines the command to initializes the runroot task on the remote machine. This command is executed as first command for each runroot task executed. The command can print a string which is then available as ``{runroot_key}`` for other SSH commands. This string might be used to keep the context across different SSH commands executed for single runroot task. The goal of this command is setting up the environment for real runroot commands. For example preparing the unique mock environment, mounting the desired file-systems, ... The command string can contain following variables which are replaced by the real values before executing the init command: * ``{runroot_tag}`` - Tag to initialize the runroot environment from. When not set, no init command is executed. **runroot_ssh_install_packages_template** (*str*) [optional] -- For ``openssh`` runroot method, defines the template for command to install the packages requested to run the runroot task. The template string can contain following variables which are replaced by the real values before executing the install command: * ``{runroot_key}`` - Replaced with the string returned by ``runroot_ssh_init_template`` if used. This can be used to keep the track of context of SSH commands belonging to single runroot task. * ``{packages}`` - White-list separated list of packages to install. Example (The ``{runroot_key}`` is expected to be set to mock config file using the ``runroot_ssh_init_template`` command.): ``"mock -r {runroot_key} --install {packages}"`` When not set, no command to install packages on remote machine is executed. **runroot_ssh_run_template** (*str*) [optional] -- For ``openssh`` runroot method, defines the template for the main runroot command. The template string can contain following variables which are replaced by the real values before executing the install command: * ``{runroot_key}`` - Replaced with the string returned by ``runroot_ssh_init_template`` if used. This can be used to keep the track of context of SSH commands belonging to single runroot task. * ``{command}`` - Command to run. Example (The ``{runroot_key}`` is expected to be set to mock config file using the ``runroot_ssh_init_template`` command.): ``"mock -r {runroot_key} chroot -- {command}"`` When not set, the runroot command is run directly. Extra Files Settings ==================== Options ------- **extra_files** (*list*) -- references to external files to be placed in os/ directory and media; format: ``[(variant_uid_regex, {arch|*: [scm_dict]})]``. See :ref:`scm_support` for details. If the dict specifies a ``target`` key, an additional subdirectory will be used. Example ------- :: extra_files = [ ('^.*$', { '*': [ # GPG keys { "scm": "rpm", "repo": "fedora-repos", "branch": None, "file": [ "/etc/pki/rpm-gpg/RPM-GPG-KEY-22-fedora", ], "target": "", }, # GPL { "scm": "git", "repo": "https://pagure.io/pungi-fedora", "branch": None, "file": [ "GPL", ], "target": "", }, ], }), ] Extra Files Metadata -------------------- If extra files are specified a metadata file, ``extra_files.json``, is placed in the ``os/`` directory and media. The checksums generated are determined by ``media_checksums`` option. This metadata file is in the format: :: { "header": {"version": "1.0}, "data": [ { "file": "GPL", "checksums": { "sha256": "8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643" }, "size": 18092 }, { "file": "release-notes/notes.html", "checksums": { "sha256": "82b1ba8db522aadf101dca6404235fba179e559b95ea24ff39ee1e5d9a53bdcb" }, "size": 1120 } ] } CreateISO Settings ================== Options ------- **createiso_skip** = False (*list*) -- mapping that defines which variants and arches to skip during createiso; format: [(variant_uid_regex, {arch|*: True})] **createiso_max_size** (*list*) -- mapping that defines maximum expected size for each variant and arch. If the ISO is larger than the limit, a warning will be issued. Format: ``[(variant_uid_regex, {arch|*: number})]`` **createiso_max_size_is_strict** (*list*) -- Set the value to ``True`` to turn the warning from ``createiso_max_size`` into a hard error that will abort the compose. If there are multiple matches in the mapping, the check will be strict if at least one match says so. Format: ``[(variant_uid_regex, {arch|*: bool})]`` **create_jigdo** = False (*bool*) -- controls the creation of jigdo from ISO **create_optional_isos** = False (*bool*) -- when set to ``True``, ISOs will be created even for ``optional`` variants. By default only variants with type ``variant`` or ``layered-product`` will get ISOs. **createiso_break_hardlinks** = False (*bool*) -- when set to ``True``, all files that should go on the ISO and have a hardlink will be first copied into a staging directory. This should work around a bug in ``genisoimage`` including incorrect link count in the image, but it is at the cost of having to copy a potentially significant amount of data. The staging directory is deleted when ISO is successfully created. In that case the same task to create the ISO will not be re-runnable. **createiso_use_xorrisofs** = False (*bool*) -- when set to True, use ``xorrisofs`` for creating ISOs instead of ``genisoimage``. **iso_size** = 4700000000 (*int|str*) -- size of ISO image. The value should either be an integer meaning size in bytes, or it can be a string with ``k``, ``M``, ``G`` suffix (using multiples of 1024). **iso_level** (*int*) [optional] -- Set the ISO9660 conformance level. Valid numbers are 1 to 4. **split_iso_reserve** = 10MiB (*int|str*) -- how much free space should be left on each disk. The format is the same as for ``iso_size`` option. **iso_hfs_ppc64le_compatible** = True (*bool*) -- when set to False, the Apple/HFS compatibility is turned off for ppc64le ISOs. This option only makes sense for bootable products, and affects images produced in *createiso* and *extra_isos* phases. .. note:: Source architecture needs to be listed explicitly. Excluding '*' applies only on binary arches. Jigdo causes significant increase of time to ISO creation. Example ------- :: createiso_skip = [ ('^Workstation$', { '*': True, 'src': True }), ] .. _auto-version: Automatic generation of version and release =========================================== Version and release values for certain artifacts can be generated automatically based on release version, compose label, date, type and respin. This can be used to shorten the config and keep it the same for multiple uses. +----------------------------+-------------------+--------------+--------------+--------+------------------+ | Compose ID | Label | Version | Date | Respin | Release | +============================+===================+==============+==============+========+==================+ | ``F-Rawhide-20170406.n.0`` | ``-`` | ``Rawhide`` | ``20170406`` | ``0`` | ``20170406.n.0`` | +----------------------------+-------------------+--------------+--------------+--------+------------------+ | ``F-26-20170329.1`` | ``Alpha-1.6`` | ``26_Alpha`` | ``20170329`` | ``1`` | ``1.6`` | +----------------------------+-------------------+--------------+--------------+--------+------------------+ | ``F-Atomic-25-20170407.0`` | ``RC-20170407.0`` | ``25`` | ``20170407`` | ``0`` | ``20170407.0`` | +----------------------------+-------------------+--------------+--------------+--------+------------------+ | ``F-Atomic-25-20170407.0`` | ``-`` | ``25`` | ``20170407`` | ``0`` | ``20170407.0`` | +----------------------------+-------------------+--------------+--------------+--------+------------------+ All non-``RC`` milestones from label get appended to the version. For release either label is used or date, type and respin. Common options for Live Images, Live Media and Image Build ========================================================== All images can have ``ksurl``, ``version``, ``release`` and ``target`` specified. Since this can create a lot of duplication, there are global options that can be used instead. For each of the phases, if the option is not specified for a particular deliverable, an option named ``_