almalinux/pungi
14
1
Fork 0
Code Issues Pull Requests Releases Activity
The utility for building of AlmaLinux distributions (repos, ISO images).
2,368 Commits 3 Branches 366 Tags 9.2 MiB
Python 99.4%
XSLT 0.3%
Shell 0.2%
Makefile 0.1%
a209bda73c
Go to file
Jan Kaluza a209bda73c Create MaterializedPackageSets in threads to make pkgset faster. 
When modules are used, there are lot of small package sets. These
package sets have usually less than 500 packages. The createrepo
part of `MaterializedPackageSet.create` executed for such small
set of packages takes around 1 second. Most of this time
the createrepo_c runs in single thread. It does the initialization,
it writes the XML files, ...

The parts of createrepo which can be run in parallel and therefore
would use all the CPUs are quite small for very small package sets.

This commit therefore executes multiple threads with
`MaterializedPackageSet.create` for these very small package sets.

This saves around 40 seconds from pkgset phase for RHEL compose.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2020-03-13 12:51:02 +01:00
contrib/yum-dnf-compare gather: Only parse pungi log once 2017-08-09 11:04:14 +02:00
doc Allow reusing gather phase results. 2020-03-13 10:17:59 +01:00
pungi Create MaterializedPackageSets in threads to make pkgset faster. 2020-03-13 12:51:02 +01:00
pungi_utils Fix other flake8 complaints 2020-02-07 16:14:09 +08:00
share Allow setting <kojitag/> in <modules/> in variants.xml to get the modules from this Koji tag. 2018-03-21 14:33:45 +01:00
tests pkgset: Reuse pkgset repos 2020-03-13 17:29:53 +08:00
.gitignore Run flake8 and black via tox 2020-02-07 16:14:45 +08:00
AUTHORS extra-files: Write a metadata file enumerating extra files 2016-09-07 13:02:48 +02:00
COPYING Remove FSF address from comments 2016-09-23 10:26:43 +02:00
git-changelog git-changelog: Fix running on Python 3 2019-07-02 15:05:25 +02:00
GPL Update GPL to latest version from https://www.gnu.org/licenses/gpl-2.0.txt 2015-06-25 07:50:03 -04:00
Makefile Allow customizing nosetests command 2019-06-20 13:26:53 +02:00
MANIFEST.in Include all test fixtures in source tarball 2018-10-17 10:09:46 +02:00
pungi.spec 4.2.0 release 2020-02-03 08:26:02 +01:00
README.md README: add link to documentation 2019-02-18 14:36:48 -07:00
setup.py Remove dependency on dict.sorted 2020-02-17 12:45:49 +01:00
TODO Initial code merge for Pungi 4.0. 2015-02-10 08:19:34 -05:00
tox.ini Display black output in tests 2020-03-05 09:51:47 +01:00

README.md

Pungi

Pungi is a distribution compose tool.

Composes are release snapshots that contain release deliverables such as:

  • installation trees
    • RPMs
    • repodata
    • comps
  • (bootable) ISOs
  • kickstart trees
    • anaconda images
    • images for PXE boot

Tool overview

Pungi consists of multiple separate executables backed by a common library.

The main entry-point is the pungi-koji script. It loads the compose configuration and kicks off the process. Composing itself is done in phases. Each phase is responsible for generating some artifacts on disk and updating the compose object that is threaded through all the phases.

Pungi itself does not actually do that much. Most of the actual work is delegated to separate executables. Pungi just makes sure that all the commands are invoked in the appropriate order and with correct arguments. It also moves the artifacts to correct locations.

Links