almalinux/pungi
14
1
Fork 0
Code Issues Pull Requests Releases Activity
1,725 Commits 3 Branches 366 Tags 7.4 MiB
b51a639277
Commit Graph

1725 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Lubomír Sedlář
35e72df99f Explicitly remove all temporary files 
There is no guarantee __del__ will ever be called, and we were leaving a
ton of stuff in /tmp. With this patch we pass the temporary directories
explictly and make sure they are deleted at the end.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
fe026bb588 Handle Requires(pre|post) 
These requires are accessible from a separate attribute, but we want to
handle them the same ways as regular Requires.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
2f40a45708 Honor multilib_blacklist for initial packages matching a glob 
We want to exclude all multilib packages matching the blacklist (that
are not noarch) from the package set.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
d4f8e32a80 Fix handling globs in input packages 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
a9415eb0e5 Simplify add_multilib_package method 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
cd80d6ef5a Stop caching provides of added packages 
This only works in non-greedy mode. When greedy, the same provides can
be linked to multiple packages that should be pulled in.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
7343daba24 Clean up logging excluded packages 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
e06e9165fc Add checking of flags for dnf depsolving 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
8cc912c5aa Install new executable 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
ec67eac1cc Use enum for flags 
This adds an extra dependency on python-enum34.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
d5e6639a6d Reduce duplication in code 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
9fd8e6319f Handle fulltree_excludes list 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:49 +01:00
Lubomír Sedlář
9a1674f3bf Improve error message for non-matching patterns 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
b53e4a84d3 Fix fulltree when greedy 
When greedy, fulltree should pull in all native and multilib packages.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
3a6653fbc2 Clean up main loop 
Greatly reduce duplication by moving common code into a method.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
0a44b2fd07 Filter source packages 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
5ef630d6d4 Simplify code 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
99699b85aa Use same tests for yum and dnf depsolving 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
a155674269 Set persistdir to a temporary location 
We don't want to even touch any system directory.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
9b2f0349de Define dnf_gather option in schema 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
e753891e27 Remove FSF address from copyright headers 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
b9fd755a05 Add only one handler to logger 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Lubomír Sedlář
4f4fd845d9 Fix PEP8 complaints 
* Remove dead imports
* Remove dead code
* Fix extra empty lines
* Fix formatting
* Break too long lines

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Daniel Mach
fdda8fe491 Fix multilib blacklist/whitelist when no multilib_method is set. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Daniel Mach
22a6291c8a Remove hardcoded blacklists and whitelists from multilib_dnf. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:48 +01:00
Daniel Mach
26cac77639 Add default logging handler to gather_dnf. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
94d16d8c32 Rewrite gathering to use new hawkey code that's part of libhif. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Lubos Kocman
dfef1837ea gather_dnf: don't add srpm deps if --nodeps is set 
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
 2017-03-24 09:24:47 +01:00
Lubos Kocman
14784847d4 add dnf_arch wrapper to workaround dnf.arch.basearch -> dnf.rpm.basearch change 
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
 2017-03-24 09:24:47 +01:00
Lubos Kocman
1a8f0bb0f9 gather_dnf.py: use self.logger for printing output 
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
320d2a8ae2 Boost add_initial_packages() by using query cache for 'build' greedy method. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
c02d801988 Boost prepopulate by using query cache. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
321d471125 Additional profiling in Gather.add_initial_packages(). 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
5a2c4f5e0f Improve working with query cache. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
06635a3917 Add profiler to pungi-gather. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Daniel Mach
61e76d5824 Improve depsolving speed by pre-computing provides query. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Michael Mraka
026e49e259 speedup filter(sourcerpm=XXX) 
originally 6519 calls 18, i.e. 2% out of 15 mins

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Michael Mraka
835460ce8f speedup filter(name=XXX, version=YYY, release=ZZZ) 
originally 3109 calls 54s, i.e. 6% out of 15 mins

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:47 +01:00
Michael Mraka
110fe7647c speedup filter(pkg=XXX, arch__neq=WWW) 
originally 17976 calls 157s, i.e. 17% out of 15 mins

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:46 +01:00
Michael Mraka
ec56ee75e2 speedup filter(name=XXX, version=YYY, release=ZZZ, arch__neq=WWW) 
the second most demanding filter
it is called 8297 times and gets 189s out of 15min run which is 21%
spent here

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:46 +01:00
Michael Mraka
d8fcaf95f7 speedup filter(sourcerpm=XXX, arch__neq=YYY) 
this is most demanding filter
it is called 10826 times and gets 210.277010s out of 15min run which is
23% spent here

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:46 +01:00
Lubomír Sedlář
36f57b26b8 Add scripts to compare YUM and DNF gathering 
There are scripts to re-run depsolving with any backend based on config
and log file.

There is a new script to compare logs from two runs and report
differences. This script will be installed system wide in final RPM.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:24:46 +01:00
Daniel Mach
99b68ca96b Simplify ks group handling. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
7cb9e9dc20 Add support for mirrorlists in DNF gathering code. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
ddba26edd7 Don't run fulltree on packages added via fulltree. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
a5d302d036 Enable --fulltree, --selfhosting and --nodeps in pungi-gather. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
aff6ac906c Enable langpack gathering in DNF gather. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
43d8d02cb1 Integrate DNF gathering code with Pungi. 
Set 'dnf_gather = True' (temporary option) in config file to enable DNF
gathering.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 09:12:23 +01:00
Daniel Mach
9764acbc4d Cache pkg.sourcerpm to get additional >5% boost. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 08:51:38 +01:00
Daniel Mach
13ed3f2b1d Performance improvements of DNF gathering. 
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
 2017-03-24 08:51:38 +01:00