almalinux/pungi
14
1
Fork 0
Code Issues Pull Requests Releases Activity
1,561 Commits 3 Branches 366 Tags 9.2 MiB
c5fcab2aa5
Commit Graph

1561 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Lubomír Sedlář
3a6653fbc2 Clean up main loop 
Greatly reduce duplication by moving common code into a method.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
0a44b2fd07 Filter source packages 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
5ef630d6d4 Simplify code 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
99699b85aa Use same tests for yum and dnf depsolving 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
a155674269 Set persistdir to a temporary location 
We don't want to even touch any system directory.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
9b2f0349de Define dnf_gather option in schema 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
e753891e27 Remove FSF address from copyright headers 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
b9fd755a05 Add only one handler to logger 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
4f4fd845d9 Fix PEP8 complaints 
* Remove dead imports
* Remove dead code
* Fix extra empty lines
* Fix formatting
* Break too long lines

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Daniel Mach
fdda8fe491 Fix multilib blacklist/whitelist when no multilib_method is set. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Daniel Mach
22a6291c8a Remove hardcoded blacklists and whitelists from multilib_dnf. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Daniel Mach
26cac77639 Add default logging handler to gather_dnf. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
94d16d8c32 Rewrite gathering to use new hawkey code that's part of libhif. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Lubos Kocman
dfef1837ea gather_dnf: don't add srpm deps if --nodeps is set 
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
 2017-03-24 09:24:47 +01:00
Lubos Kocman
14784847d4 add dnf_arch wrapper to workaround dnf.arch.basearch -> dnf.rpm.basearch change 
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
 2017-03-24 09:24:47 +01:00
Lubos Kocman
1a8f0bb0f9 gather_dnf.py: use self.logger for printing output 
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
320d2a8ae2 Boost add_initial_packages() by using query cache for 'build' greedy method. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
c02d801988 Boost prepopulate by using query cache. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
321d471125 Additional profiling in Gather.add_initial_packages(). 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
5a2c4f5e0f Improve working with query cache. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
06635a3917 Add profiler to pungi-gather. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
61e76d5824 Improve depsolving speed by pre-computing provides query. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Michael Mraka
026e49e259 speedup filter(sourcerpm=XXX) 
originally 6519 calls 18, i.e. 2% out of 15 mins

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Michael Mraka
835460ce8f speedup filter(name=XXX, version=YYY, release=ZZZ) 
originally 3109 calls 54s, i.e. 6% out of 15 mins

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Michael Mraka
110fe7647c speedup filter(pkg=XXX, arch__neq=WWW) 
originally 17976 calls 157s, i.e. 17% out of 15 mins

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:46 +01:00
Michael Mraka
ec56ee75e2 speedup filter(name=XXX, version=YYY, release=ZZZ, arch__neq=WWW) 
the second most demanding filter
it is called 8297 times and gets 189s out of 15min run which is 21%
spent here

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:46 +01:00
Michael Mraka
d8fcaf95f7 speedup filter(sourcerpm=XXX, arch__neq=YYY) 
this is most demanding filter
it is called 10826 times and gets 210.277010s out of 15min run which is
23% spent here

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:46 +01:00
Lubomír Sedlář
36f57b26b8 Add scripts to compare YUM and DNF gathering 
There are scripts to re-run depsolving with any backend based on config
and log file.

There is a new script to compare logs from two runs and report
differences. This script will be installed system wide in final RPM.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:46 +01:00
Daniel Mach
99b68ca96b Simplify ks group handling. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
7cb9e9dc20 Add support for mirrorlists in DNF gathering code. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
ddba26edd7 Don't run fulltree on packages added via fulltree. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
a5d302d036 Enable --fulltree, --selfhosting and --nodeps in pungi-gather. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
aff6ac906c Enable langpack gathering in DNF gather. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
43d8d02cb1 Integrate DNF gathering code with Pungi. 
Set 'dnf_gather = True' (temporary option) in config file to enable DNF
gathering.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
9764acbc4d Cache pkg.sourcerpm to get additional >5% boost. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 08:51:38 +01:00
Daniel Mach
13ed3f2b1d Performance improvements of DNF gathering. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 08:51:38 +01:00
Daniel Mach
caf11f55d4 New gathering (depsolving) module based on DNF. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 08:51:38 +01:00
Jan Kaluza
9fcd71f831 Add support for modular composes 
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
 2017-03-22 15:55:52 +01:00
Lubomír Sedlář
4b90822115 Add a script for modifying ISO images 
With this script it's possible to add additional files into an ISO file.
If the file happens to be ks.cfg, the boot configs are tweaked so that
the kickstart is actually used.

Resolves: #503
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-21 07:55:19 +01:00
Lubomír Sedlář
ae5ee3d856 iso-wrapper: Refactor functions 
Split the function for getting implant MD5 into two functions, so that
we can have access to other information retrieved by checkisomd5
command. A test case is added for both functions.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-21 07:55:19 +01:00
Lubomír Sedlář
306f7e69b0 iso-wrapper: Add utility for mounting images 
This patch refactors logic for creating a temporary mount point,
mounting an image, running arbitrary code on it, unmounting the image
and removing the mount point. It immediately uses it in the buildinstall
phase.

Similar mounting is present in product_img phase as well, but due to
different usage pattern it's not changed yet.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-21 07:55:19 +01:00
Lubomír Sedlář
05a666fb3b buildinstall: Move tweaking configs into a function 
This will allow us to more easily test the code and also reuse it in
other places.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-21 07:55:19 +01:00
Lubomír Sedlář
55035487de image-build: Correctly write can_fail option 
Koji expects the value as comma separated list, otherwise it will ignore
it. This makes it possible for the image building to fail on per arch
basis.

Relates: #128
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-20 09:05:03 +01:00
Qixiang Wan
79e97dc845 pungi-koji: new cmd option '--latest-link-status' 
Add a new option 'latest-link-status' to pungi-koji, if this is
specified, pungi will only create the latest symbol link to the compose
when compose's status matches the specified statuses. The status name is
case insensitive. If the option is not specified it will act as before.

Example:

pungi-koji --target-dir=_composes --config=data/dummy-pungi.conf \
--test --latest-link-status=finished --latest-link-status=finished_incomplete

Signed-off-by: Qixiang Wan <qwan@redhat.com>
 2017-03-20 11:15:30 +08:00
Lubomír Sedlář
23e53219eb live-cd: Print task ID on success 
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-15 17:37:00 +01:00
Lubomír Sedlář
69af0a6a65 ostree-installer: Print task ID on success 
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-15 17:37:00 +01:00
Lubomír Sedlář
263a990489 ostree: Print task ID on success 
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-15 17:37:00 +01:00
Lubomír Sedlář
5ee285cc24 live-media: Print task ID on success 
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-15 17:37:00 +01:00
Lubomír Sedlář
b10d275745 image-build: Print task ID on success 
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-15 17:37:00 +01:00
Dennis Gilmore
b09641a708 Merge #568 buildinstall: Print debug info if unmount fails 2017-03-15 16:07:33 +00:00