Commit Graph

50 Commits

Author SHA1 Message Date
Frédéric Pierret (fepitre)
86314fdc83 pungi-legacy: expose lorax's --rootfs-size argument
Merges: https://pagure.io/pungi/pull-request/1112
Fixes: https://pagure.io/pungi/issue/1107
Signed-off-by: Frédéric Pierret (fepitre) <frederic.pierret@qubes-os.org>
2019-01-14 10:41:37 +01:00
Frédéric Pierret (fepitre)
682f959ee0 Support for pungi-legacy with productmd format
Merges: https://pagure.io/pungi/pull-request/1099
Signed-off-by: Frédéric Pierret (fepitre) <frederic.pierret@qubes-os.org>
2019-01-04 14:11:23 +01:00
Frédéric Pierret (fepitre)
f41c32e413 gather.py: use createrepo_c for creating repodata instead of obsolete createrepo python library
Fixes: https://pagure.io/pungi/issue/1094
Merges: https://pagure.io/pungi/pull-request/1097
Signed-off-by: Frédéric Pierret (fepitre) <frederic.pierret@qubes-os.org>
2019-01-03 10:37:59 +01:00
Frédéric Pierret (fepitre)
c69bc13068 Fix import of ConfigParser for NoSectionError and NoOptionError
Merges: https://pagure.io/pungi/pull-request/1100
Signed-off-by: Frédéric Pierret (fepitre) <frederic.pierret@qubes-os.org>
2019-01-03 10:18:05 +01:00
Lubomír Sedlář
e78f8d1f13 Remove extra dependencies
There is a sorted dict implementation in Productmd used to achieve the
exact same thing as Pungi does here. No need for an extra dependency.
While we're at it, we can also sort the imports.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-10-08 14:15:00 +02:00
Marek Marczykowski-Górecki
d6e72c8e61 Set repodata mtime to SOURCE_DATE_EPOCH
repodata/repomd.xml include timestamps of all the other repodata files.
Even when those files are created reproducibly, they have current
modification time. In general case this is a good thing (ease checking
if repodata cache is up to date). But in case of composing installation
image, it breaks reproducibility.
Avoid this by instructing createrepo to set mtime and revision to
$SOURCE_DATE_EPOCH.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2018-10-08 13:33:29 +02:00
Marek Marczykowski-Górecki
2aeb8de459 Make sure .treeinfo file is sorted
OrderedDict used by default by ConfigParser isn't enough because order
of entries being added may not be deterministic (depends on directory
list order). To solve this problem, use SortedDict as a base.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2018-10-08 13:33:29 +02:00
Marek Marczykowski-Górecki
d469cbfef5 Use constant MBR ID for isohybrid
If not set explicitly, isohybrid choose it randomly, which harm
reproducibility.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2018-10-08 13:33:29 +02:00
Marek Marczykowski-Górecki
a12c2b9ea0 Use xorriso instead of genisoimage
xorriso make the image reproducible (given the same input files),
including support for SOURCE_DATE_EPOCH in various metadata.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2018-10-08 13:33:29 +02:00
Lubomír Sedlář
a5fa4457c3 gather: Print full unresolved dependency
We should not lose the flag and version. The name is not sufficient to
fully determine the problem.

The log information printed in `pungi/gather.py` is parsed by regex in
`pungi/wrappers/pungi.py`. It handles the dependency with spaces and
versions fine.

JIRA: COMPOSE-2880
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-08-24 08:38:22 +02:00
Lubomír Sedlář
4ecf75295a gather: Allow empty result for gather
This only affects Yum backend, and allows to mention packages in comps
groups before they become available. This means a compose can be set up
ahead of all packages being available, and it will simply produce empty
repos.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-07-23 09:38:39 -04:00
Lubomír Sedlář
9b576bf976 pungi-legacy: Add --joliet-long option
Using `-J` each path component can only be 64 characters long. This is
not sufficient for some packages. Adding `--joliet-long` increases the
limit to 103.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1605103
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2018-07-20 10:17:38 +02:00
Lubomír Sedlář
eead6ccc44 gather(yum): Don't exclude packages from lookaside
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-09-12 08:38:08 +02:00
Lubomír Sedlář
439a7ce348 Use Py3-compatible exception handling
In once case we can completely drop try-except statements: it only
logs the exception details but that will be logged again anyway.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-08-24 14:51:27 +02:00
Lubomír Sedlář
471e369d23 gather: Fix excluding debugsource packages from input list
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-07-27 08:50:21 +02:00
Till Maas
29bedf2ccc Use pungi.util.pkg_is_debug() instead of pungi.gather.is_debug()
There is no reason to have two functions for the same job.

Signed-off-by: Till Maas <opensource@till.name>
2017-07-26 23:18:15 +02:00
Lubomír Sedlář
d92390b80b Add support for debugsource packages
These packages should behave like regular debuginfo packages (at least
for now).

Fixes: https://pagure.io/pungi/issue/684
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-07-26 10:34:18 +02:00
Lubomír Sedlář
4fb28979cf gather: Display source repo of packages
When multiple repos are configured in pkgset_repos, the logs should
contain information on where exactly the package was pulled from. The
log file in question should be in
`logs/<arch>/pkgset_source.<arch>.log`.

Fixes: #545
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-06-16 12:39:04 +02:00
Lubomír Sedlář
3130d837c0 gather: Don't resolve dependencies in lookaside
When looking at a package in a lookaside repo, it does not make much
sense to process its dependencies. We should just assume that the
lookaside can satisfy them.

In the worst case, this could result in packages being pulled into the
compose just so that they could satisfy a dep of something in lookaside.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-05-29 12:55:38 +02:00
Dennis Gilmore
ccbd246edb Merge #605 gather: Process dependencies sorted 2017-04-27 15:31:28 +00:00
Lubomír Sedlář
ebe25a3717 gather: Print specific Requires which pulls a package in
When dependencies are pulled in, it's useful to know not only the
package that requires them, but also the specific requires.

This patch only implement this for the YUM version.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-26 09:52:07 +02:00
Lubomír Sedlář
75430a95b9 gather: Process dependencies sorted
Instead of going in random order, we should do it deterministically and
process the dependencies in order.

This should actually fix a problem where `glibc-langpack` packages are
missing on some architectures for Fedora Rawhide.

Fixes: https://pagure.io/pungi-fedora/issue/214
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-04-26 09:49:00 +02:00
Lubomír Sedlář
76cf4a7540 Update to use python-multilib
We also rename the old multilib module used by dnf code to multilib_yum
to make it clear what is imported where.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2017-03-24 09:24:50 +01:00
Lubomír Sedlář
ec6206b064 pungi: Fix --nosource option
When running with this option, if a debuginfo package is included in the
final package set, there would be a crash due to missing SRPM. To
replicate this issue, just use the argument with input kickstart listing
a single package which has debuginfo.

This is not an issue for Pungi 4, as it never uses the option in the
first place.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-12-13 10:45:05 +01:00
Lubomír Sedlář
56147f2e4d pungi: Handle missing SRPM
Source packages can be excluded while binary packages should still go
into the compose. This patch makes it so that the mapping from binary
packages to source packages contains None in such case. The code is
already capable of handling that. A warning will be emitted for each
binary package without source.

This also allows us to remove some code from `createSourceHashes` that
is now unused.

A test for excluding source package is added as well.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-12-13 10:45:04 +01:00
Lubomír Sedlář
deb8623bd1 pungi: Include noarch debuginfo
The check for arch compatibility should not be performed if the
debuginfo package is noarch. Such packages should be included
unconditionally.

Fixes: #450
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-16 14:50:03 +01:00
Lubomír Sedlář
ea1bcf625b pungi: Only add logger once
The Pungi object reuses the same logger for all its instances. Every
time a new instance is created, a stream handler would be added. This
means that a second instance will print everything twice. When running
tests, tens of instances would be created.

This patch only adds the handler if there are no handlers configured
yet.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-10 13:28:02 +01:00
Lubomír Sedlář
4a7257f550 pungi: Connect yum callback to logger
This patch changes the format a little and makes sure the output goes to
the same logger as the rest of the output. The output no longer contains
control characters.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-10 13:27:58 +01:00
Lubomír Sedlář
adaed6345c pungi: Fulltree should not apply for input multilib package
When the input explicitly lists a package as multilib, we should not
automatically add native version just because of fulltree.

The tests for this use case are now enabled and passing.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-11-09 09:36:23 +01:00
Lubomír Sedlář
a3a415e398 pungi: Avoid removing from list
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-26 12:51:02 +02:00
Daniel Mach
48bd3e6d2d pungi: Allow globs in %multilib-whitelist
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-26 12:51:02 +02:00
Daniel Mach
2cd2e0e15f pungi: Exclude RPMs that are in lookaside
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-26 12:51:02 +02:00
Daniel Mach
3e7f2dfa9d pungi: Fix excluding SRPMs
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-26 12:51:02 +02:00
Daniel Mach
e6342ede18 pungi: Speed up blacklist processing
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-26 12:51:02 +02:00
Mark Montague
4be3c72bd5 pungi: Replace kickstart repo url
Old Pungi should replace yum baseurl/mirrorlist when doing buildinstall.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1264570
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-10-07 07:56:06 +02:00
Lubomír Sedlář
d6dc269ef4 Remove FSF address from comments
The address is no longer correct. We can just as well simply point to
the web page describing the license.

Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
2016-09-23 10:26:43 +02:00
Daniel Mach
15811107bb gather: Check if txmbr.isDep is set for conditional deps. 2015-09-10 09:38:09 -04:00
Adam Miller
e8f4b7334e pull in gather.py patches from dmach for test compose 2015-06-10 10:03:57 -05:00
Dennis Gilmore
1d87fca380 fix up bad += from early test of implementing different iso labels
based on if there is a variant or not

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-05-20 16:33:54 -05:00
Dennis Gilmore
ca17987de9 make sure we treat the isfinal option as a boolean when fetching it
It will make sure that we only pass --isfinal to lorax when it is
passed in on the CLI

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-05-20 15:14:39 -05:00
Dennis Gilmore
f5eaa7326f if there is a variant use it in the volume id and shorten it. this
will make each producst install tree have different volume ids for
their isos

Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-05-20 15:14:00 -05:00
Dennis Gilmore
ad18e21d1a fix up typo in getting arch for the lorax log file
Signed-off-by: Dennis Gilmore <dennis@ausil.us>
2015-03-14 12:20:44 -05:00
Dennis Gilmore
18d4d2ecf8 fix up the pungi logging by putting the arch in the log file name 2015-03-12 17:29:59 -05:00
Dennis Gilmore
07e90f0f96 change pypungi imports to pungi 2015-03-12 16:12:38 -05:00
Brian C. Lane
320724ed98 Add the option to pass a custom path for the multilib config files
The default is /usr/share/pungi/multilib/, pass --multilibconf to
override this.

This also adds multilib.init() so that an import of multilib doesn't
immediately setup the classes.

(cherry picked from commit 234524296fd53871aed64690cf6a7d5849ca154a)
2015-03-12 11:40:23 -05:00
Brian C. Lane
13526d1c49 Call lorax as a process not a library
Doing this allows lorax to move to DNF (and Python3) without needing to
wait for pungi to be updated.
2015-03-12 11:29:58 -05:00
Dennis Gilmore
64b6c8065c switch to BSD style hashes for the iso checksums 2015-03-12 10:16:09 -05:00
Dennis Gilmore
0633eb29d3 refactor to get better data into .treeinfo
for https://fedorahosted.org/rel-eng/ticket/6008 refacter how we deal
with the data that feeds into .treeinfo
Deprecate --name for --family
Deprecate --flavor for --variant
rather than using --name as the iso base name use the value of
family if there is a variant add it with a - as the seperator
2015-03-12 10:15:55 -05:00
Daniel Mach
f116d9384f Initial code merge for Pungi 4.0. 2015-02-10 08:19:34 -05:00
Daniel Mach
f5c6d44000 Initial changes for Pungi 4.0. 2015-02-10 06:47:16 -05:00