almalinux/pungi
14
1
Fork 0
Code Issues Pull Requests Releases Activity

New upstream release 4.1.21

Browse Source 
Ported everything to python 3
This commit is contained in:
Lubomír Sedlář 2017-12-04 15:09:22 +01:00
parent a726c6a0b3
commit fa40100be6
6 changed files with 74 additions and 398 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
0001-gather-get_packages_to_gather-returns-a-tuple.patch
View File

@ -1,31 +0,0 @@
From 4282525f652e79f1d98e58f8ef108547b7256f45 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Lubom=C3=ADr=20Sedl=C3=A1=C5=99?= <lsedlar@redhat.com>
Date: Thu, 9 Nov 2017 14:11:41 +0100
Subject: [PATCH] gather: get_packages_to_gather returns a tuple
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
It always needs to return a tuple, otherwise there would be crashes.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
---
pungi/phases/gather/__init__.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pungi/phases/gather/__init__.py b/pungi/phases/gather/__init__.py
index 3b96abc..c4b27f0 100644
--- a/pungi/phases/gather/__init__.py
+++ b/pungi/phases/gather/__init__.py
@@ -517,7 +517,7 @@ def get_packages_to_gather(compose, arch=None, variant=None, include_arch=True,
be included in a list of packages.
"""
if compose.conf["gather_source"] == "module":
- return []
+ return ([], [])
arches = [arch] if arch else compose.get_arches()
--
2.13.6

120
790.patch
View File

@ -1,120 +0,0 @@
From 1a10a1fe8333e16fa448d96377baff0aab0ee780 Mon Sep 17 00:00:00 2001
From: Patrick Uiterwijk <patrick@puiterwijk.org>
Date: Nov 06 2017 14:17:33 +0000
Subject: Optionally do old_compose per release type
This would make sure that e.g. "updates" composes don't try to use "updates-testing" as an
old_compose_path, which would create practically useless deltarpms and for no repodata
reuse at all.
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
---
diff --git a/pungi/checks.py b/pungi/checks.py
index 1f31fd9..cd1940c 100644
--- a/pungi/checks.py
+++ b/pungi/checks.py
@@ -626,6 +626,10 @@ def make_schema():
"enum": ["yum", "dnf"],
},
+ "old_composes_per_release_type": {
+ "type": "boolean",
+ "default": False,
+ },
"hashed_directories": {
"type": "boolean",
"default": False,
diff --git a/pungi/phases/createrepo.py b/pungi/phases/createrepo.py
index 2860823..9b8984b 100644
--- a/pungi/phases/createrepo.py
+++ b/pungi/phases/createrepo.py
@@ -275,6 +275,7 @@ def _get_old_package_dirs(compose, repo_dir):
compose.old_composes,
compose.ci_base.release.short,
compose.ci_base.release.version,
+ compose.ci_base.release.type_suffix if compose.conf['old_composes_per_release_type'] else None,
compose.ci_base.base_product.short if compose.ci_base.release.is_layered else None,
compose.ci_base.base_product.version if compose.ci_base.release.is_layered else None,
allowed_statuses=['FINISHED', 'FINISHED_INCOMPLETE'],
diff --git a/pungi/phases/pkgset/common.py b/pungi/phases/pkgset/common.py
index c6c4f25..0ed76e8 100644
--- a/pungi/phases/pkgset/common.py
+++ b/pungi/phases/pkgset/common.py
@@ -55,7 +55,14 @@ def create_global_repo(compose, path_prefix):
old_compose_path = None
update_md_path = None
if compose.old_composes:
- old_compose_path = find_old_compose(compose.old_composes, compose.conf["release_short"], compose.conf["release_version"], compose.conf.get("base_product_short"), compose.conf.get("base_product_version"))
+ old_compose_path = find_old_compose(
+ compose.old_composes,
+ compose.ci_base.release.short,
+ compose.ci_base.release.version,
+ compose.ci_base.release.type_suffix if compose.conf['old_composes_per_release_type'] else None,
+ compose.ci_base.base_product.short if compose.ci_base.release.is_layered else None,
+ compose.ci_base.base_product.version if compose.ci_base.release.is_layered else None,
+ )
if old_compose_path is None:
compose.log_info("No suitable old compose found in: %s" % compose.old_composes)
else:
diff --git a/pungi/util.py b/pungi/util.py
index 678590a..0162e17 100644
--- a/pungi/util.py
+++ b/pungi/util.py
@@ -395,8 +395,8 @@ def get_file_size(path):
def find_old_compose(old_compose_dirs, release_short, release_version,
- base_product_short=None, base_product_version=None,
- allowed_statuses=None):
+ release_type_suffix=None, base_product_short=None,
+ base_product_version=None, allowed_statuses=None):
allowed_statuses = allowed_statuses or ("FINISHED", "FINISHED_INCOMPLETE", "DOOMED")
composes = []
@@ -417,6 +417,8 @@ def find_old_compose(old_compose_dirs, release_short, release_version,
# TODO: read .composeinfo
pattern = "%s-%s" % (release_short, release_version)
+ if release_type_suffix:
+ pattern += release_type_suffix
if base_product_short:
pattern += "-%s" % base_product_short
if base_product_version:
@@ -425,6 +427,12 @@ def find_old_compose(old_compose_dirs, release_short, release_version,
if not i.startswith(pattern):
continue
+ suffix = i[len(pattern):]
+ if release_type_suffix and (len(suffix) < 2 or not suffix[1].isdigit()):
+ # This covers the case where we are looking for -updates, but there
+ # is an updates-testing as well.
+ continue
+
path = os.path.join(compose_dir, i)
if not os.path.isdir(path):
continue
diff --git a/tests/test_util.py b/tests/test_util.py
index f88eafd..05804ac 100644
--- a/tests/test_util.py
+++ b/tests/test_util.py
@@ -253,6 +253,16 @@ class TestFindOldCompose(unittest.TestCase):
old = util.find_old_compose(self.tmp_dir, 'Fedora', 'Rawhide')
self.assertEqual(old, self.tmp_dir + '/Fedora-Rawhide-20160229.1')
+ def test_find_correct_type(self):
+ touch(self.tmp_dir + '/Fedora-26-updates-20160229.0/STATUS', 'FINISHED')
+ touch(self.tmp_dir + '/Fedora-26-updates-testing-20160229.0/STATUS', 'FINISHED')
+ old = util.find_old_compose(self.tmp_dir, 'Fedora', '26', '-updates')
+ self.assertEqual(old, self.tmp_dir + '/Fedora-26-updates-20160229.0')
+ old = util.find_old_compose(self.tmp_dir, 'Fedora', '26', '-updates-testing')
+ self.assertEqual(old, self.tmp_dir + '/Fedora-26-updates-testing-20160229.0')
+ old = util.find_old_compose(self.tmp_dir, 'Fedora', '26')
+ self.assertEqual(old, self.tmp_dir + '/Fedora-26-updates-testing-20160229.0')
+
def test_find_latest_with_two_digit_respin(self):
touch(self.tmp_dir + '/Fedora-Rawhide-20160228.n.9/STATUS', 'FINISHED')
touch(self.tmp_dir + '/Fedora-Rawhide-20160228.n.10/STATUS', 'FINISHED')

66
791.patch
View File

@ -1,66 +0,0 @@
From 1dbd0248d4ff9b8417b51d9692f8705022740e18 Mon Sep 17 00:00:00 2001
From: Patrick Uiterwijk <patrick@puiterwijk.org>
Date: Nov 06 2017 15:14:44 +0000
Subject: Implement version.compose_id version generator
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
---
diff --git a/pungi/util.py b/pungi/util.py
index 0162e17..0607b6c 100644
--- a/pungi/util.py
+++ b/pungi/util.py
@@ -782,9 +782,12 @@ def version_generator(compose, gen):
"""
if gen == '!OSTREE_VERSION_FROM_LABEL_DATE_TYPE_RESPIN':
return '%s.%s' % (compose.image_version, compose.image_release)
- if gen == '!RELEASE_FROM_LABEL_DATE_TYPE_RESPIN':
+ elif gen == '!RELEASE_FROM_LABEL_DATE_TYPE_RESPIN':
return compose.image_release
- if gen and gen[0] == '!':
+ elif gen == '!RELEASE_FROM_VERSION_COMPOSE_ID':
+ return '%s.%s' % (compose.ci_base.release.version,
+ compose.ci_base.id)
+ elif gen and gen[0] == '!':
raise RuntimeError("Unknown version generator '%s'" % gen)
return gen
diff --git a/tests/test_util.py b/tests/test_util.py
index 05804ac..3209676 100644
--- a/tests/test_util.py
+++ b/tests/test_util.py
@@ -601,6 +601,20 @@ class GetRepoFuncsTestCase(unittest.TestCase):
class TestVersionGenerator(unittest.TestCase):
+ def setUp(self):
+ ci = mock.MagicMock()
+ ci.respin = 0
+ ci.id = 'RHEL-8.0-20180101.0'
+ ci.release.version = '8'
+ ci.date = '20160101'
+ ci.type = 'nightly'
+ ci.type_suffix = ''
+ ci.label = 'RC-1.0'
+ ci.label_major_version = '1'
+
+ self.compose = mock.MagicMock()
+ self.compose.ci_base = ci
+
def test_unknown_generator(self):
compose = mock.Mock()
with self.assertRaises(RuntimeError) as ctx:
@@ -617,6 +631,10 @@ class TestVersionGenerator(unittest.TestCase):
compose = mock.Mock()
self.assertEqual(util.version_generator(compose, None), None)
+ def test_release_from_version_compose_id(self):
+ self.assertEqual(util.version_generator(self.compose, '!RELEASE_FROM_VERSION_COMPOSE_ID'),
+ '8.RHEL-8.0-20180101.0')
+
class TestTZOffset(unittest.TestCase):
@mock.patch('time.daylight', new=False)

96
796.patch
View File

@ -1,96 +0,0 @@
From 8181c5be48c736dadb1d2733306ab8edb8a2d05e Mon Sep 17 00:00:00 2001
From: Patrick Uiterwijk <puiterwijk@redhat.com>
Date: Nov 10 2017 10:14:11 +0000
Subject: Turn COMPOSE_ID version generator into DATE_RESPIN
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
---
diff --git a/doc/configuration.rst b/doc/configuration.rst
index fd5b67b..077457c 100644
--- a/doc/configuration.rst
+++ b/doc/configuration.rst
@@ -950,17 +950,17 @@ Version and release values for certain artifacts can be generated automatically
based on release version, compose label, date, type and respin. This can be
used to shorten the config and keep it the same for multiple uses.
-+----------------------------+-------------------+--------------+------------------+
-| Compose ID | Label | Version | Release |
-+============================+===================+==============+==================+
-| ``F-Rawhide-20170406.n.0`` | ``-`` | ``Rawhide`` | ``20170406.n.0`` |
-+----------------------------+-------------------+--------------+------------------+
-| ``F-26-20170329.1`` | ``Alpha-1.6`` | ``26_Alpha`` | ``1.6`` |
-+----------------------------+-------------------+--------------+------------------+
-| ``F-Atomic-25-20170407.0`` | ``RC-20170407.0`` | ``25`` | ``20170407.0`` |
-+----------------------------+-------------------+--------------+------------------+
-| ``F-Atomic-25-20170407.0`` | ``-`` | ``25`` | ``20170407.0`` |
-+----------------------------+-------------------+--------------+------------------+
++----------------------------+-------------------+--------------+--------------+--------+------------------+
+| Compose ID | Label | Version | Date | Respin | Release |
++============================+===================+==============+==============+========+==================+
+| ``F-Rawhide-20170406.n.0`` | ``-`` | ``Rawhide`` | ``20170406`` | ``0`` | ``20170406.n.0`` |
++----------------------------+-------------------+--------------+--------------+--------+------------------+
+| ``F-26-20170329.1`` | ``Alpha-1.6`` | ``26_Alpha`` | ``20170329`` | ``1`` | ``1.6`` |
++----------------------------+-------------------+--------------+--------------+--------+------------------+
+| ``F-Atomic-25-20170407.0`` | ``RC-20170407.0`` | ``25`` | ``20170407`` | ``0`` | ``20170407.0`` |
++----------------------------+-------------------+--------------+--------------+--------+------------------+
+| ``F-Atomic-25-20170407.0`` | ``-`` | ``25`` | ``20170407`` | ``0`` | ``20170407.0`` |
++----------------------------+-------------------+--------------+--------------+--------+------------------+
All non-``RC`` milestones from label get appended to the version. For release
either label is used or date, type and respin.
@@ -1230,6 +1230,8 @@ repository with a new commit.
* ``version`` -- (*str*) Version string to be added as versioning metadata.
If this option is set to ``!OSTREE_VERSION_FROM_LABEL_DATE_TYPE_RESPIN``,
a value will be generated automatically as ``$VERSION.$RELEASE``.
+ If this option is set to ``!VERSION_FROM_VERSION_DATE_RESPIN``,
+ a value will be generated automatically as ``$VERSION.$DATE.$RESPIN``.
:ref:`See how those values are created <auto-version>`.
* ``tag_ref`` -- (*bool*, default ``True``) If set to ``False``, a git
reference will not be created.
diff --git a/pungi/util.py b/pungi/util.py
index 0607b6c..2d2df82 100644
--- a/pungi/util.py
+++ b/pungi/util.py
@@ -784,9 +784,10 @@ def version_generator(compose, gen):
return '%s.%s' % (compose.image_version, compose.image_release)
elif gen == '!RELEASE_FROM_LABEL_DATE_TYPE_RESPIN':
return compose.image_release
- elif gen == '!RELEASE_FROM_VERSION_COMPOSE_ID':
- return '%s.%s' % (compose.ci_base.release.version,
- compose.ci_base.id)
+ elif gen == '!VERSION_FROM_VERSION_DATE_RESPIN':
+ return '%s.%s.%s' % (compose.ci_base.release.version,
+ compose.ci_base.date,
+ compose.compose_respin)
elif gen and gen[0] == '!':
raise RuntimeError("Unknown version generator '%s'" % gen)
return gen
diff --git a/tests/test_util.py b/tests/test_util.py
index 3209676..e0dd2b1 100644
--- a/tests/test_util.py
+++ b/tests/test_util.py
@@ -614,6 +614,7 @@ class TestVersionGenerator(unittest.TestCase):
self.compose = mock.MagicMock()
self.compose.ci_base = ci
+ self.compose.compose_respin = 0
def test_unknown_generator(self):
compose = mock.Mock()
@@ -631,9 +632,9 @@ class TestVersionGenerator(unittest.TestCase):
compose = mock.Mock()
self.assertEqual(util.version_generator(compose, None), None)
- def test_release_from_version_compose_id(self):
- self.assertEqual(util.version_generator(self.compose, '!RELEASE_FROM_VERSION_COMPOSE_ID'),
- '8.RHEL-8.0-20180101.0')
+ def test_release_from_version_date_respin(self):
+ self.assertEqual(util.version_generator(self.compose, '!VERSION_FROM_VERSION_DATE_RESPIN'),
+ '8.20160101.0')
class TestTZOffset(unittest.TestCase):

157
pungi.spec
View File

@ -1,45 +1,36 @@
Name: pungi
Version: 4.1.20
Release: 3%{?dist}
Version: 4.1.21
Release: 1%{?dist}
Summary: Distribution compose tool
Group: Development/Tools
License: GPLv2
URL: https://pagure.io/pungi
Source0: https://pagure.io/releases/%{name}/%{name}-%{version}.tar.bz2
Patch0: 0001-gather-get_packages_to_gather-returns-a-tuple.patch
Patch1: https://pagure.io/pungi/pull-request/790.patch
Patch2: https://pagure.io/pungi/pull-request/791.patch
Patch3: https://pagure.io/pungi/pull-request/796.patch
BuildRequires: python-nose, python-mock
BuildRequires: python-devel, python-setuptools, python2-productmd >= 1.3
BuildRequires: python-lockfile, kobo-rpmlib, createrepo_c
BuildRequires: python-lxml, libselinux-python, yum-utils, lorax
%if ! 0%{?epel}
BuildRequires: python-kickstart
BuildRequires: python-rpm
BuildRequires: python2-dnf
BuildRequires: python2-multilib
BuildRequires: python2-six
%else
BuildRequires: pykickstart
BuildRequires: rpm-python
BuildRequires: python-six
%endif
BuildRequires: yum => 3.4.3-28, createrepo >= 0.4.11
BuildRequires: gettext, git-core, cvs
BuildRequires: python-jsonschema
BuildRequires: python-enum34
BuildRequires: python2-libcomps
BuildRequires: kobo >= 0.6
%if 0%{?fedora} >= 27
BuildRequires: python2-koji
%endif
BuildRequires: python3-nose
BuildRequires: python3-mock
BuildRequires: python3-devel
BuildRequires: python3-setuptools
BuildRequires: python3-productmd >= 1.3
BuildRequires: python3-kobo-rpmlib
BuildRequires: createrepo_c
BuildRequires: python3-lxml
BuildRequires: python3-kickstart
BuildRequires: python3-rpm
BuildRequires: python3-dnf
BuildRequires: python3-multilib
BuildRequires: python3-six
BuildRequires: git-core
BuildRequires: python3-jsonschema
BuildRequires: python3-enum34
BuildRequires: python3-libcomps
BuildRequires: python3-kobo
BuildRequires: python3-koji
BuildRequires: python3-unittest2
BuildRequires: lorax
#deps for doc building
%if ! 0%{?epel}
BuildRequires: python-sphinx, texlive-collection-fontsrecommended
BuildRequires: python3-sphinx, texlive-collection-fontsrecommended
BuildRequires: texlive-cmap, texlive-babel-english, texlive-fancyhdr
BuildRequires: texlive-titlesec, texlive-framed, texlive-threeparttable
BuildRequires: texlive-mdwtools, texlive-wrapfig, texlive-parskip, texlive-upquote
@ -48,43 +39,26 @@ BuildRequires: tex(fncychap.sty)
BuildRequires: tex(tabulary.sty)
BuildRequires: tex(needspace.sty)
BuildRequires: latexmk
%endif
Requires: createrepo >= 0.4.11
Requires: yum => 3.4.3-28
Requires: lorax >= 22.1
Requires: repoview
Requires: python-lockfile
Requires: kobo >= 0.6
Requires: kobo-rpmlib
Requires: python-productmd >= 1.3
Requires: python-kickstart
Requires: libselinux-python
Requires: python3-kobo >= 0.6
Requires: python3-kobo-rpmlib
Requires: python3-productmd >= 1.3
Requires: python3-kickstart
Requires: createrepo_c
Requires: python-lxml
Requires: python3-lxml
Requires: koji >= 1.10.1-13
Requires: python3-koji-cli-plugins
# This is optional do not Require it
#eRquires: jigdo
Requires: cvs
Requires: yum-utils
Requires: isomd5sum
Requires: genisoimage
Requires: gettext
# this is x86 only
#Requires: syslinux
Requires: git
Requires: python-jsonschema
Requires: python3-jsonschema
Requires: libguestfs-tools-c
Requires: python-enum34
Requires: python2-dnf
Requires: python2-multilib
Requires: python2-libcomps
Requires: python2-six
%if 0%{?fedora} >= 27
Requires: python2-koji
%endif
Requires: python3-enum34
Requires: python3-dnf
Requires: python3-multilib
Requires: python3-libcomps
Requires: python3-six
Requires: python3-koji
BuildArch: noarch
@ -103,56 +77,45 @@ notification to Fedora Message Bus.
%prep
%setup -q
%patch0 -p1
%patch1 -p1
%patch2 -p1
%patch3 -p1
%build
%{__python} setup.py build
%if ! 0%{?epel}
%{__python3} setup.py build
cd doc
make latexpdf
make epub
make text
make man
make latexpdf SPHINXBUILD=/usr/bin/sphinx-build-3
make epub SPHINXBUILD=/usr/bin/sphinx-build-3
make text SPHINXBUILD=/usr/bin/sphinx-build-3
make man SPHINXBUILD=/usr/bin/sphinx-build-3
gzip _build/man/pungi.1
%endif
%install
%{__python} setup.py install -O1 --skip-build --root %{buildroot}
%{__python3} setup.py install -O1 --skip-build --root %{buildroot}
%{__install} -d %{buildroot}/var/cache/pungi
%if ! 0%{?epel}
%{__install} -d %{buildroot}%{_mandir}/man1
%{__install} -m 0644 doc/_build/man/pungi.1.gz %{buildroot}%{_mandir}/man1
%endif
%check
nosetests --exe
nosetests-3 --exe
./tests/data/specs/build.sh
ln -s $(which python3) bin/python
cd tests && ./test_compose.sh
%files
%license COPYING GPL
%doc AUTHORS
%if ! 0%{?epel}
%doc doc/_build/latex/Pungi.pdf doc/_build/epub/Pungi.epub doc/_build/text/*
%endif
%{python_sitelib}/%{name}
%{python_sitelib}/%{name}-%{version}-py?.?.egg-info
%{python3_sitelib}/%{name}
%{python3_sitelib}/%{name}-%{version}-py?.?.egg-info
%{_bindir}/%{name}
%{_bindir}/%{name}-koji
%{_bindir}/%{name}-gather
%{_bindir}/comps_filter
%{_bindir}/%{name}-make-ostree
%if ! 0%{?epel}
%{_mandir}/man1/pungi.1.gz
%endif
%{_datadir}/pungi
/var/cache/pungi
%files utils
%{python_sitelib}/%{name}_utils
%{python3_sitelib}/%{name}_utils
%{_bindir}/%{name}-create-unified-isos
%{_bindir}/%{name}-config-validate
%{_bindir}/%{name}-fedmsg-notification
@ -161,6 +124,32 @@ cd tests && ./test_compose.sh
%{_bindir}/%{name}-wait-for-signed-ostree-handler
%changelog
* Wed Dec 06 2017 Lubomír Sedlář <lsedlar@redhat.com> - 4.1.21-1
- tests: Use correct python version for config validation test (lsedlar)
- Use dnf backend for repoclosure on PY3 (lsedlar)
- Drop checks for git and cvs (lsedlar)
- Relax check for gettext (lsedlar)
- Drop check for repoquery command (lsedlar)
- Use modifyrepo_c if possible (lsedlar)
- pkgset: Add SRPMs to whitelist (lsedlar)
- modules: Allow multilib (lsedlar)
- add ability to specify ostree ref in OSTREE phase - update (onosek)
- add ability to specify ostree ref in OSTREE phase (onosek)
- buildinstall: Allow using external dire for runroot task (jkaluza)
- pkgset: Remove package skip optimization for bootable products (lsedlar)
- Add documentation for modular composes (lsedlar)
- osbs: Get correct path to repo for addons (lsedlar)
- Remove deprecated options (onosek)
- module-source: Log details about what packages are gathered (lsedlar)
- gather: Log details about nodeps method (lsedlar)
- gather: get_packages_to_gather returns a tuple (lsedlar)
- iso-wrapper: Fix calling wrong logger method (lsedlar)
- Turn COMPOSE_ID version generator into DATE_RESPIN (puiterwijk)
- iso-wrapper: Remove hacks for sorting (lsedlar)
- Report missing module dependencies earlier (lsedlar)
- Implement version.compose_id version generator (patrick)
- Optionally do old_compose per release type (patrick)
* Wed Nov 22 2017 Patrick Uiterwijk <puiterwijk@redhat.com> - 4.1.20-3
- Backport patch for PR#790 - old_composes per release type
- Backport patch for PR#791,796 - implement DATE_RESPIN version generator

2
sources
View File

@ -1 +1 @@
SHA512 (pungi-4.1.20.tar.bz2) = a34ad87da99a1b78531f6f31509fb5d08fd771fdef7517aaf9485dbce23b72fb6a03150c145ec1585d749456ec2d5aff15a79ca1918bd4cb9d6d51b08f2551f2
SHA512 (pungi-4.1.21.tar.bz2) = 439ddbc8755280e1a11c7dec077131b8d98281361a3eaa1c5ff4701dee6ab05240d7c1c2840a9a2697b16b91413d4ecf548ae28957d288ff1512161e86c3819f