pungi/pungi
Jan Kaluza a209bda73c Create MaterializedPackageSets in threads to make pkgset faster.
When modules are used, there are lot of small package sets. These
package sets have usually less than 500 packages. The createrepo
part of `MaterializedPackageSet.create` executed for such small
set of packages takes around 1 second. Most of this time
the createrepo_c runs in single thread. It does the initialization,
it writes the XML files, ...

The parts of createrepo which can be run in parallel and therefore
would use all the CPUs are quite small for very small package sets.

This commit therefore executes multiple threads with
`MaterializedPackageSet.create` for these very small package sets.

This saves around 40 seconds from pkgset phase for RHEL compose.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2020-03-13 12:51:02 +01:00
..
compose_metadata Format code base with black 2020-02-05 17:35:47 +08:00
ostree Format code base with black 2020-02-05 17:35:47 +08:00
phases Create MaterializedPackageSets in threads to make pkgset faster. 2020-03-13 12:51:02 +01:00
scripts pungi-koji: Allow user to specify version component count in latest symlink 2020-03-13 10:50:08 +01:00
wrappers Fix flake8 complaints - E501 2020-02-07 14:36:46 +08:00
__init__.py Format code base with black 2020-02-05 17:35:47 +08:00
arch.py Format code base with black 2020-02-05 17:35:47 +08:00
arch_utils.py Fix other flake8 complaints 2020-02-07 16:14:09 +08:00
checks.py Allow reusing gather phase results. 2020-03-13 10:17:59 +01:00
common.py Format code base with black 2020-02-05 17:35:47 +08:00
compose.py Fix other flake8 complaints 2020-02-07 16:14:09 +08:00
config.py Format code base with black 2020-02-05 17:35:47 +08:00
createiso.py Fix flake8 complaints - E501 2020-02-07 14:36:46 +08:00
dnf_wrapper.py Format code base with black 2020-02-05 17:35:47 +08:00
gather.py Fix other flake8 complaints 2020-02-07 16:14:09 +08:00
gather_dnf.py gather: Make additional_packages accept debuginfo packages 2020-02-12 16:38:08 +08:00
graph.py Fix flake8 complaints - E501 2020-02-07 14:36:46 +08:00
ks.py Format code base with black 2020-02-05 17:35:47 +08:00
linker.py Format code base with black 2020-02-05 17:35:47 +08:00
media_split.py Format code base with black 2020-02-05 17:35:47 +08:00
metadata.py Fix flake8 complaints - E501 2020-02-07 14:36:46 +08:00
module_util.py Move import of modulemd to a separate module 2019-10-03 10:40:31 +02:00
multilib_dnf.py Format code base with black 2020-02-05 17:35:47 +08:00
multilib_yum.py Format code base with black 2020-02-05 17:35:47 +08:00
notifier.py Format code base with black 2020-02-05 17:35:47 +08:00
pathmatch.py Remove FSF address from comments 2016-09-23 10:26:43 +02:00
paths.py pkgset: Reuse pkgset repos 2020-03-13 17:29:53 +08:00
profiler.py Format code base with black 2020-02-05 17:35:47 +08:00
runroot.py Format code base with black 2020-02-05 17:35:47 +08:00
util.py Create MaterializedPackageSets in threads to make pkgset faster. 2020-03-13 12:51:02 +01:00