Commit Graph

46 Commits

Author SHA1 Message Date
Lubomír Sedlář fcd4c231c4 Stop importing hawkey directly
It's API is considered deprecated.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:50 +01:00
Lubomír Sedlář 27b24a489f Handle noarch debuginfo packages
Pull debuginfo packages for noarch packages as well, but only when the
debuginfo is noarch as well.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:50 +01:00
Lubomír Sedlář d23a2f4548 Fix multilib exclude pattern to not match noarch
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:50 +01:00
Lubomír Sedlář 5533bf7ca3 Simplify add_binary_package_deps
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:50 +01:00
Lubomír Sedlář c32bdce46d Work around a bug in DNF
https://bugzilla.redhat.com/show_bug.cgi?id=1418298

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:50 +01:00
Lubomír Sedlář 73b8928013 Expand multilib blacklist and whitelist
This way we pay once for the expansion, and lookups are then done in
constant time instead of iterating over the list over and over again.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:50 +01:00
Lubomír Sedlář 5c1d04eb00 Add matching multilib method to log
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:50 +01:00
Lubomír Sedlář 08fbdec494 Add detailed reason why package is added
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:49 +01:00
Lubomír Sedlář 919e4d748e Remove extra section for adding greedy=build packages
It does not change any result for tests, and makes it work a lot faster.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:49 +01:00
Lubomír Sedlář fe026bb588 Handle Requires(pre|post)
These requires are accessible from a separate attribute, but we want to
handle them the same ways as regular Requires.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:49 +01:00
Lubomír Sedlář 2f40a45708 Honor multilib_blacklist for initial packages matching a glob
We want to exclude all multilib packages matching the blacklist (that
are not noarch) from the package set.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:49 +01:00
Lubomír Sedlář d4f8e32a80 Fix handling globs in input packages
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:49 +01:00
Lubomír Sedlář a9415eb0e5 Simplify add_multilib_package method
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:49 +01:00
Lubomír Sedlář cd80d6ef5a Stop caching provides of added packages
This only works in non-greedy mode. When greedy, the same provides can
be linked to multiple packages that should be pulled in.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:49 +01:00
Lubomír Sedlář 7343daba24 Clean up logging excluded packages
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:49 +01:00
Lubomír Sedlář ec67eac1cc Use enum for flags
This adds an extra dependency on python-enum34.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:49 +01:00
Lubomír Sedlář 9fd8e6319f Handle fulltree_excludes list
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:49 +01:00
Lubomír Sedlář 9a1674f3bf Improve error message for non-matching patterns
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář b53e4a84d3 Fix fulltree when greedy
When greedy, fulltree should pull in all native and multilib packages.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář 3a6653fbc2 Clean up main loop
Greatly reduce duplication by moving common code into a method.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář 0a44b2fd07 Filter source packages
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář 5ef630d6d4 Simplify code
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář e753891e27 Remove FSF address from copyright headers
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář b9fd755a05 Add only one handler to logger
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Lubomír Sedlář 4f4fd845d9 Fix PEP8 complaints
* Remove dead imports
* Remove dead code
* Fix extra empty lines
* Fix formatting
* Break too long lines

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Daniel Mach fdda8fe491 Fix multilib blacklist/whitelist when no multilib_method is set.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:48 +01:00
Daniel Mach 26cac77639 Add default logging handler to gather_dnf.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach 94d16d8c32 Rewrite gathering to use new hawkey code that's part of libhif.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Lubos Kocman dfef1837ea gather_dnf: don't add srpm deps if --nodeps is set
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2017-03-24 09:24:47 +01:00
Lubos Kocman 1a8f0bb0f9 gather_dnf.py: use self.logger for printing output
Signed-off-by: Lubos Kocman <lkocman@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach 320d2a8ae2 Boost add_initial_packages() by using query cache for 'build' greedy method.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach c02d801988 Boost prepopulate by using query cache.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach 321d471125 Additional profiling in Gather.add_initial_packages().
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach 5a2c4f5e0f Improve working with query cache.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach 06635a3917 Add profiler to pungi-gather.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Daniel Mach 61e76d5824 Improve depsolving speed by pre-computing provides query.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Michael Mraka 026e49e259 speedup filter(sourcerpm=XXX)
originally 6519 calls 18, i.e. 2% out of 15 mins

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Michael Mraka 835460ce8f speedup filter(name=XXX, version=YYY, release=ZZZ)
originally 3109 calls 54s, i.e. 6% out of 15 mins

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:47 +01:00
Michael Mraka 110fe7647c speedup filter(pkg=XXX, arch__neq=WWW)
originally 17976 calls 157s, i.e. 17% out of 15 mins

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:46 +01:00
Michael Mraka ec56ee75e2 speedup filter(name=XXX, version=YYY, release=ZZZ, arch__neq=WWW)
the second most demanding filter
it is called 8297 times and gets 189s out of 15min run which is 21%
spent here

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:46 +01:00
Michael Mraka d8fcaf95f7 speedup filter(sourcerpm=XXX, arch__neq=YYY)
this is most demanding filter
it is called 10826 times and gets 210.277010s out of 15min run which is
23% spent here

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:46 +01:00
Daniel Mach ddba26edd7 Don't run fulltree on packages added via fulltree.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:12:23 +01:00
Daniel Mach 43d8d02cb1 Integrate DNF gathering code with Pungi.
Set 'dnf_gather = True' (temporary option) in config file to enable DNF
gathering.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:12:23 +01:00
Daniel Mach 9764acbc4d Cache pkg.sourcerpm to get additional >5% boost.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 08:51:38 +01:00
Daniel Mach 13ed3f2b1d Performance improvements of DNF gathering.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 08:51:38 +01:00
Daniel Mach caf11f55d4 New gathering (depsolving) module based on DNF.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 08:51:38 +01:00