Lubomír Sedlář
b53e4a84d3
Fix fulltree when greedy
...
When greedy, fulltree should pull in all native and multilib packages.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář
3a6653fbc2
Clean up main loop
...
Greatly reduce duplication by moving common code into a method.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář
0a44b2fd07
Filter source packages
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář
5ef630d6d4
Simplify code
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář
9b2f0349de
Define dnf_gather option in schema
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář
e753891e27
Remove FSF address from copyright headers
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář
b9fd755a05
Add only one handler to logger
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář
4f4fd845d9
Fix PEP8 complaints
...
* Remove dead imports
* Remove dead code
* Fix extra empty lines
* Fix formatting
* Break too long lines
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Daniel Mach
fdda8fe491
Fix multilib blacklist/whitelist when no multilib_method is set.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Daniel Mach
22a6291c8a
Remove hardcoded blacklists and whitelists from multilib_dnf.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Daniel Mach
26cac77639
Add default logging handler to gather_dnf.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach
94d16d8c32
Rewrite gathering to use new hawkey code that's part of libhif.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Lubos Kocman
dfef1837ea
gather_dnf: don't add srpm deps if --nodeps is set
...
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2017-03-24 09:24:47 +01:00
Lubos Kocman
14784847d4
add dnf_arch wrapper to workaround dnf.arch.basearch -> dnf.rpm.basearch change
...
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2017-03-24 09:24:47 +01:00
Lubos Kocman
1a8f0bb0f9
gather_dnf.py: use self.logger for printing output
...
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach
320d2a8ae2
Boost add_initial_packages() by using query cache for 'build' greedy method.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach
c02d801988
Boost prepopulate by using query cache.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach
321d471125
Additional profiling in Gather.add_initial_packages().
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach
5a2c4f5e0f
Improve working with query cache.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach
06635a3917
Add profiler to pungi-gather.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach
61e76d5824
Improve depsolving speed by pre-computing provides query.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Michael Mraka
026e49e259
speedup filter(sourcerpm=XXX)
...
originally 6519 calls 18, i.e. 2% out of 15 mins
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Michael Mraka
835460ce8f
speedup filter(name=XXX, version=YYY, release=ZZZ)
...
originally 3109 calls 54s, i.e. 6% out of 15 mins
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Michael Mraka
110fe7647c
speedup filter(pkg=XXX, arch__neq=WWW)
...
originally 17976 calls 157s, i.e. 17% out of 15 mins
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:46 +01:00
Michael Mraka
ec56ee75e2
speedup filter(name=XXX, version=YYY, release=ZZZ, arch__neq=WWW)
...
the second most demanding filter
it is called 8297 times and gets 189s out of 15min run which is 21%
spent here
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:46 +01:00
Michael Mraka
d8fcaf95f7
speedup filter(sourcerpm=XXX, arch__neq=YYY)
...
this is most demanding filter
it is called 10826 times and gets 210.277010s out of 15min run which is
23% spent here
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:46 +01:00
Daniel Mach
99b68ca96b
Simplify ks group handling.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:12:23 +01:00
Daniel Mach
7cb9e9dc20
Add support for mirrorlists in DNF gathering code.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:12:23 +01:00
Daniel Mach
ddba26edd7
Don't run fulltree on packages added via fulltree.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:12:23 +01:00
Daniel Mach
aff6ac906c
Enable langpack gathering in DNF gather.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:12:23 +01:00
Daniel Mach
43d8d02cb1
Integrate DNF gathering code with Pungi.
...
Set 'dnf_gather = True' (temporary option) in config file to enable DNF
gathering.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:12:23 +01:00
Daniel Mach
9764acbc4d
Cache pkg.sourcerpm to get additional >5% boost.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 08:51:38 +01:00
Daniel Mach
13ed3f2b1d
Performance improvements of DNF gathering.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 08:51:38 +01:00
Daniel Mach
caf11f55d4
New gathering (depsolving) module based on DNF.
...
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 08:51:38 +01:00
Jan Kaluza
9fcd71f831
Add support for modular composes
...
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2017-03-22 15:55:52 +01:00
Lubomír Sedlář
ae5ee3d856
iso-wrapper: Refactor functions
...
Split the function for getting implant MD5 into two functions, so that
we can have access to other information retrieved by checkisomd5
command. A test case is added for both functions.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-21 07:55:19 +01:00
Lubomír Sedlář
306f7e69b0
iso-wrapper: Add utility for mounting images
...
This patch refactors logic for creating a temporary mount point,
mounting an image, running arbitrary code on it, unmounting the image
and removing the mount point. It immediately uses it in the buildinstall
phase.
Similar mounting is present in product_img phase as well, but due to
different usage pattern it's not changed yet.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-21 07:55:19 +01:00
Lubomír Sedlář
05a666fb3b
buildinstall: Move tweaking configs into a function
...
This will allow us to more easily test the code and also reuse it in
other places.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-21 07:55:19 +01:00
Lubomír Sedlář
55035487de
image-build: Correctly write can_fail option
...
Koji expects the value as comma separated list, otherwise it will ignore
it. This makes it possible for the image building to fail on per arch
basis.
Relates: #128
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-20 09:05:03 +01:00
Lubomír Sedlář
23e53219eb
live-cd: Print task ID on success
...
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 17:37:00 +01:00
Lubomír Sedlář
69af0a6a65
ostree-installer: Print task ID on success
...
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 17:37:00 +01:00
Lubomír Sedlář
263a990489
ostree: Print task ID on success
...
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 17:37:00 +01:00
Lubomír Sedlář
5ee285cc24
live-media: Print task ID on success
...
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 17:37:00 +01:00
Lubomír Sedlář
b10d275745
image-build: Print task ID on success
...
Relates: #570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 17:37:00 +01:00
Dennis Gilmore
b09641a708
Merge #568 buildinstall: Print debug info if unmount fails
2017-03-15 16:07:33 +00:00
Lubomír Sedlář
94b1159829
ostree-installer: Fix logging directory
...
Currently if there are multiple ostree installers for the same
architecture, the logfiles all end up in the same location and overwrite
each other.
This patch moves the logs to logs/<arch>/<variant>/ostree_installer-X/
for a unique value of X so that there can be multiple runs even for the
same tree.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-15 15:54:45 +01:00
Lubomír Sedlář
f0dc15bdc6
buildinstall: Print debug info if unmount fails
...
When even after retries the unmounting still fails, this patch runs `ls
-lA`, `fuser -vm` and `lsof +D` on the directory to give some idea of
what's blocking it.
Relates: #559
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-14 15:06:20 +01:00
Qixiang Wan
d081a4eda8
pkgset: report all unsigned packages
...
Fixes : #552
Merges: #567
Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-14 09:50:28 +01:00
Qixiang Wan
ee21663c8b
default createrepo_checksum to sha256
...
Signed-off-by: Qixiang Wan <qwan@redhat.com>
2017-03-14 13:15:43 +08:00
Lubomír Sedlář
5cd1c22e2d
buildinstall: Retry unmounting image
...
If the image can not be unmounted because the device is busy, we should
retry. There will be increasing pauses between the attempts. At most 10
attempts will be done before giving up.
Fixes : #559
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-10 08:29:54 +01:00