almalinux/pungi
14
1
Fork 0
Code Issues Pull Requests Releases Activity

New upstream release 4.1.24

Browse Source
This commit is contained in:
Lubomír Sedlář 2018-05-02 14:39:25 +02:00
parent a31678c7a3
commit a4313835c2
13 changed files with 149 additions and 487 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
0001-Revert-Move-ostree-phase-and-pipelines-for-running-p.patch
View File

@ -1,7 +1,7 @@
From 3f60e62ea86a3180b45290102d3519347cf8788a Mon Sep 17 00:00:00 2001
From fa0ed11ee0c733e9c3e4785f6526ef889109111c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Lubom=C3=ADr=20Sedl=C3=A1=C5=99?= <lsedlar@redhat.com>
Date: Wed, 11 Apr 2018 09:18:59 +0200
Subject: [PATCH 1/9] Revert "Move ostree phase and pipelines for running
Subject: [PATCH 1/3] Revert "Move ostree phase and pipelines for running
phases"
This reverts commit 660c04368ba1abed310f121d01f0fa029eea5f11.
@ -15,7 +15,7 @@ This reverts commit 660c04368ba1abed310f121d01f0fa029eea5f11.
delete mode 100644 pungi/phases/weaver.py
diff --git a/bin/pungi-koji b/bin/pungi-koji
index 2a62a125..e5d31084 100755
index 9dd1cd65..0bb66d2c 100755
--- a/bin/pungi-koji
+++ b/bin/pungi-koji
@@ -358,27 +358,41 @@ def run_compose(compose, create_latest_link=True, latest_link_status=None):
@ -867,5 +867,5 @@ index ba80afe3..a8b9469a 100644
if __name__ == "__main__":
--
2.13.6
2.14.3

32
0001-tests-Use-dummy-modulesdir-for-DNF.patch
View File

@ -1,32 +0,0 @@
From 1436ea2b03a9b86a8706bd2e001550221914cb2e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Lubom=C3=ADr=20Sedl=C3=A1=C5=99?= <lsedlar@redhat.com>
Date: Wed, 4 Apr 2018 12:23:43 +0200
Subject: [PATCH] tests: Use dummy modulesdir for DNF
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Otherwise it tries to ensure it exists, and since the default is
/etc/dnf/modules.d, it's causing problems if the directory does not
exist and user does not have permissions to create it.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
---
tests/test_gather.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/tests/test_gather.py b/tests/test_gather.py
index 1daf3c60..25dfc399 100644
--- a/tests/test_gather.py
+++ b/tests/test_gather.py
@@ -1810,6 +1810,7 @@ class DNFDepsolvingTestCase(DepsolvingBase, unittest.TestCase):
conf = Conf(base_arch)
conf.persistdir = persistdir
conf.cachedir = self.cachedir
+ conf.modulesdir = os.path.join(persistdir, 'modules.d')
if exclude:
conf.exclude = exclude
dnf = DnfWrapper(conf)
--
2.13.6

82
0002-Revert-Other-repo-for-OstreeInstaller.patch
View File

@ -1,21 +1,21 @@
From fcc431622fc0305248bb966271c71c8d17fd5f69 Mon Sep 17 00:00:00 2001
From c62eb8dd27bc17406649e3560223f20b690de63f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Lubom=C3=ADr=20Sedl=C3=A1=C5=99?= <lsedlar@redhat.com>
Date: Wed, 11 Apr 2018 09:19:53 +0200
Subject: [PATCH 2/9] Revert "Other repo for OstreeInstaller"
Subject: [PATCH 2/3] Revert "Other repo for OstreeInstaller"
This reverts commit 5c081cb545715c2a912ff50fa57554e89d905868.
---
pungi/checks.py | 3 ++-
pungi/phases/ostree_installer.py | 8 ++-----
pungi/util.py | 21 +++++-------------
tests/test_ostree_installer_phase.py | 43 +++++++++++++++---------------------
4 files changed, 28 insertions(+), 47 deletions(-)
pungi/phases/ostree_installer.py | 11 ++------
pungi/util.py | 21 +++++----------
tests/test_ostree_installer_phase.py | 50 +++++++++++++-----------------------
4 files changed, 28 insertions(+), 57 deletions(-)
diff --git a/pungi/checks.py b/pungi/checks.py
index 6233d5d7..189ca948 100644
index d84f24ad..0c0623d6 100644
--- a/pungi/checks.py
+++ b/pungi/checks.py
@@ -938,7 +938,7 @@ def make_schema():
@@ -944,7 +944,7 @@ def make_schema():
"tag_ref": {"type": "boolean"},
"ostree_ref": {"type": "string"},
},
@ -24,7 +24,7 @@ index 6233d5d7..189ca948 100644
"additionalProperties": False,
}),
]
@@ -959,6 +959,7 @@ def make_schema():
@@ -965,6 +965,7 @@ def make_schema():
"template_repo": {"type": "string"},
"template_branch": {"type": "string"},
},
@ -33,7 +33,7 @@ index 6233d5d7..189ca948 100644
}),
diff --git a/pungi/phases/ostree_installer.py b/pungi/phases/ostree_installer.py
index b7843195..9bf5e255 100644
index a78fccad..9bf5e255 100644
--- a/pungi/phases/ostree_installer.py
+++ b/pungi/phases/ostree_installer.py
@@ -9,7 +9,7 @@ from kobo import shortcuts
@ -45,13 +45,16 @@ index b7843195..9bf5e255 100644
from ..wrappers import kojiwrapper, iso, lorax, scm
@@ -64,11 +64,7 @@ class OstreeInstallerThread(WorkerThread):
@@ -64,14 +64,7 @@ class OstreeInstallerThread(WorkerThread):
self.pool.log_info('[BEGIN] %s' % msg)
self.logdir = compose.paths.log.topdir('%s/%s/ostree_installer-%s' % (arch, variant, self.num))
- repo_baseurl = compose.paths.work.arch_repo('$basearch', create_dir=False)
- comps_baseurl = compose.paths.work.comps_repo('$basearch', variant=variant, create_dir=False)
- repos = get_repo_urls(None, # compose==None. Special value says that method should ignore deprecated variant-type repo
- shortcuts.force_list(config['repo']) + shortcuts.force_list(translate_path(compose, repo_baseurl)),
- shortcuts.force_list(config['repo'])
- + shortcuts.force_list(translate_path(compose, repo_baseurl))
- + shortcuts.force_list(translate_path(compose, comps_baseurl)),
- arch=arch,
- logger=self.pool)
+ repos = get_repo_urls(compose, shortcuts.force_list(config['repo']), arch=arch)
@ -59,10 +62,10 @@ index b7843195..9bf5e255 100644
output_dir = os.path.join(compose.paths.work.topdir(arch), variant.uid, 'ostree_installer')
util.makedirs(os.path.dirname(output_dir))
diff --git a/pungi/util.py b/pungi/util.py
index 261f3f25..8ea945da 100644
index 19df1f49..431c9978 100644
--- a/pungi/util.py
+++ b/pungi/util.py
@@ -678,8 +678,6 @@ def get_repo_url(compose, repo, arch='$basearch'):
@@ -683,8 +683,6 @@ def get_repo_url(compose, repo, arch='$basearch'):
Convert repo to repo URL.
@param compose - required for access to variants
@ -71,7 +74,7 @@ index 261f3f25..8ea945da 100644
@param repo - string or a dict which at least contains 'baseurl' key
@param arch - string to be used as arch in repo url
"""
@@ -690,17 +688,14 @@ def get_repo_url(compose, repo, arch='$basearch'):
@@ -695,17 +693,14 @@ def get_repo_url(compose, repo, arch='$basearch'):
raise RuntimeError('Baseurl is required in repo dict %s' % str(repo))
if '://' not in repo:
# this is a variant name
@ -93,7 +96,7 @@ index 261f3f25..8ea945da 100644
"""
Convert repos to a list of repo URLs.
@@ -711,11 +706,7 @@ def get_repo_urls(compose, repos, arch='$basearch', logger=None):
@@ -716,11 +711,7 @@ def get_repo_urls(compose, repos, arch='$basearch', logger=None):
urls = []
for repo in repos:
repo = get_repo_url(compose, repo, arch=arch)
@ -106,7 +109,7 @@ index 261f3f25..8ea945da 100644
return urls
@@ -773,7 +764,7 @@ def get_repo_dicts(repos, logger=None):
@@ -778,7 +769,7 @@ def get_repo_dicts(repos, logger=None):
repo_dict = get_repo_dict(repo)
if repo_dict == {}:
if logger:
@ -116,7 +119,7 @@ index 261f3f25..8ea945da 100644
repo_dicts.append(repo_dict)
return repo_dicts
diff --git a/tests/test_ostree_installer_phase.py b/tests/test_ostree_installer_phase.py
index 1c7af1ad..ba56d8c7 100644
index e3af4468..ba56d8c7 100644
--- a/tests/test_ostree_installer_phase.py
+++ b/tests/test_ostree_installer_phase.py
@@ -14,7 +14,6 @@ sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
@ -155,29 +158,31 @@ index 1c7af1ad..ba56d8c7 100644
'release': '20160321.n.0',
}
koji = KojiWrapper.return_value
@@ -159,7 +155,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -159,8 +155,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
t.process((self.compose, self.compose.variants['Everything'], 'x86_64', cfg), 1)
self.assertRunrootCall(koji,
- 'http://example.com/work/$basearch/repo',
- ['http://example.com/work/$basearch/repo',
- 'http://example.com/work/$basearch/comps_repo_Everything'],
+ 'file://%s/compose/Everything/x86_64/os' % self.topdir,
cfg['release'],
extra=['--logfile=%s/%s/lorax.log' % (self.topdir, LOG_PATH)])
self.assertIsoLinked(link, get_file_size, get_mtime, final_iso_path)
@@ -194,11 +190,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -195,12 +190,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
t.process((self.compose, self.compose.variants['Everything'], 'x86_64', cfg), 1)
- self.assertRunrootCall(koji,
- ('http://example.com/repo/x86_64/',
- 'http://example.com/work/$basearch/repo'),
- 'http://example.com/work/$basearch/repo',
- 'http://example.com/work/$basearch/comps_repo_Everything'),
- cfg['release'],
- isfinal=True,
+ self.assertRunrootCall(koji, 'http://example.com/repo/x86_64/', cfg['release'], isfinal=True,
extra=['--logfile=%s/%s/lorax.log' % (self.topdir, LOG_PATH)])
self.assertIsoLinked(link, get_file_size, get_mtime, final_iso_path)
self.assertImageAdded(self.compose, ImageCls, iso)
@@ -217,7 +209,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -219,7 +209,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
cfg = {
'release': '20160321.n.0',
'repo': [
@ -186,7 +191,7 @@ index 1c7af1ad..ba56d8c7 100644
'https://example.com/extra-repo1.repo',
'https://example.com/extra-repo2.repo',
],
@@ -234,9 +226,9 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -236,10 +226,9 @@ class OstreeThreadTest(helpers.PungiTestCase):
t.process((self.compose, self.compose.variants['Everything'], 'x86_64', cfg), 1)
sources = [
@ -194,11 +199,12 @@ index 1c7af1ad..ba56d8c7 100644
'https://example.com/extra-repo1.repo',
- 'https://example.com/extra-repo2.repo',
- 'http://example.com/work/$basearch/repo',
- 'http://example.com/work/$basearch/comps_repo_Everything',
+ 'https://example.com/extra-repo2.repo'
]
self.assertRunrootCall(koji, sources, cfg['release'], isfinal=True,
@@ -255,8 +247,8 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -258,8 +247,8 @@ class OstreeThreadTest(helpers.PungiTestCase):
cfg = {
'release': '20160321.n.0',
'repo': [
@ -209,7 +215,7 @@ index 1c7af1ad..ba56d8c7 100644
'https://example.com/extra-repo1.repo',
'https://example.com/extra-repo2.repo',
],
@@ -273,9 +265,10 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -276,10 +265,10 @@ class OstreeThreadTest(helpers.PungiTestCase):
t.process((self.compose, self.compose.variants['Everything'], 'x86_64', cfg), 1)
sources = [
@ -218,11 +224,12 @@ index 1c7af1ad..ba56d8c7 100644
'https://example.com/extra-repo1.repo',
- 'https://example.com/extra-repo2.repo',
- 'http://example.com/work/$basearch/repo',
- 'http://example.com/work/$basearch/comps_repo_Everything',
+ 'https://example.com/extra-repo2.repo'
]
self.assertRunrootCall(koji, sources, cfg['release'], isfinal=True,
@@ -326,7 +319,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -330,7 +319,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
get_dir_from_scm):
pool = mock.Mock()
cfg = {
@ -231,16 +238,17 @@ index 1c7af1ad..ba56d8c7 100644
'release': '20160321.n.0',
'add_template': ['some_file.txt'],
'add_arch_template': ['other_file.txt'],
@@ -353,7 +346,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -357,8 +346,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
'branch': 'f24', 'dir': '.'},
templ_dir, logger=pool._logger)])
self.assertRunrootCall(koji,
- 'http://example.com/work/$basearch/repo',
- ['http://example.com/work/$basearch/repo',
- 'http://example.com/work/$basearch/comps_repo_Everything'],
+ 'file://%s/compose/Everything/x86_64/os' % self.topdir,
cfg['release'],
isfinal=True,
extra=['--add-template=%s/some_file.txt' % templ_dir,
@@ -374,7 +367,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -379,7 +367,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
get_file_size, get_mtime, ImageCls, run):
pool = mock.Mock()
cfg = {
@ -249,16 +257,17 @@ index 1c7af1ad..ba56d8c7 100644
'release': '!RELEASE_FROM_LABEL_DATE_TYPE_RESPIN',
"installpkgs": ["fedora-productimg-atomic"],
"add_template": ["/spin-kickstarts/atomic-installer/lorax-configure-repo.tmpl"],
@@ -406,7 +399,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -411,8 +399,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
self.assertRunrootCall(
koji,
- 'http://example.com/work/$basearch/repo',
- ['http://example.com/work/$basearch/repo',
- 'http://example.com/work/$basearch/comps_repo_Everything'],
+ 'file://%s/compose/Everything/x86_64/os' % self.topdir,
'20151203.t.0',
isfinal=True,
extra=['--installpkgs=fedora-productimg-atomic',
@@ -435,7 +428,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -441,7 +428,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
get_file_size, get_mtime, ImageCls, run):
pool = mock.Mock()
cfg = {
@ -267,15 +276,16 @@ index 1c7af1ad..ba56d8c7 100644
'release': None,
"installpkgs": ["fedora-productimg-atomic"],
"add_template": ["/spin-kickstarts/atomic-installer/lorax-configure-repo.tmpl"],
@@ -467,7 +460,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
@@ -473,8 +460,7 @@ class OstreeThreadTest(helpers.PungiTestCase):
self.assertRunrootCall(
koji,
- 'http://example.com/work/$basearch/repo',
- ['http://example.com/work/$basearch/repo',
- 'http://example.com/work/$basearch/comps_repo_Everything'],
+ 'file://%s/compose/Everything/x86_64/os' % self.topdir,
'20151203.t.0',
isfinal=True,
extra=['--installpkgs=fedora-productimg-atomic',
--
2.13.6
2.14.3

75
0003-Revert-Ostree-can-use-pkgset-repos.patch
View File

@ -1,36 +1,40 @@
From 8a7bf97434cbbf2397d3209498eacc326fc130f2 Mon Sep 17 00:00:00 2001
From f51543a24a35145ecbc60e86951accf5369df364 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Lubom=C3=ADr=20Sedl=C3=A1=C5=99?= <lsedlar@redhat.com>
Date: Wed, 11 Apr 2018 09:20:51 +0200
Subject: [PATCH 3/9] Revert "Ostree can use pkgset repos"
Subject: [PATCH 3/3] Revert "Ostree can use pkgset repos"
This reverts commit c7cc200246300c6a3946b2e3a9f5f7693896a7d6.
---
pungi/phases/ostree.py | 3 +--
pungi/util.py | 35 +++++++++++++++++-------------
tests/test_config_validate_script.py | 4 +++-
tests/test_ostree_phase.py | 41 ++++++++----------------------------
tests/test_util.py | 26 +++++++++++++++--------
5 files changed, 50 insertions(+), 59 deletions(-)
pungi/phases/ostree.py | 7 +----
pungi/util.py | 35 +++++++++++++-----------
tests/test_config_validate_script.py | 4 ++-
tests/test_ostree_phase.py | 52 +++++++-----------------------------
tests/test_util.py | 26 +++++++++++-------
5 files changed, 50 insertions(+), 74 deletions(-)
diff --git a/pungi/phases/ostree.py b/pungi/phases/ostree.py
index 787bf07a..0550b18c 100644
index 2918fff8..0550b18c 100644
--- a/pungi/phases/ostree.py
+++ b/pungi/phases/ostree.py
@@ -58,8 +58,7 @@ class OSTreeThread(WorkerThread):
@@ -58,12 +58,7 @@ class OSTreeThread(WorkerThread):
repodir = os.path.join(workdir, 'config_repo')
self._clone_repo(repodir, config['config_url'], config.get('config_branch', 'master'))
- repo_baseurl = compose.paths.work.arch_repo('$basearch', create_dir=False)
- repos = get_repo_dicts(shortcuts.force_list(config['repo']) + shortcuts.force_list(translate_path(compose, repo_baseurl)), logger=self.pool)
- comps_repo = compose.paths.work.comps_repo('$basearch', variant=variant, create_dir=False)
- repos = shortcuts.force_list(config['repo']) + [translate_path(compose, repo_baseurl)]
- if compose.has_comps:
- repos.append(translate_path(compose, comps_repo))
- repos = get_repo_dicts(repos, logger=self.pool)
+ repos = get_repo_dicts(compose, shortcuts.force_list(config['repo']))
# copy the original config and update before save to a json file
new_config = copy.copy(config)
diff --git a/pungi/util.py b/pungi/util.py
index 8ea945da..b095598c 100644
index 431c9978..61a13244 100644
--- a/pungi/util.py
+++ b/pungi/util.py
@@ -718,16 +718,19 @@ def _translate_url_to_repo_id(url):
@@ -723,16 +723,19 @@ def _translate_url_to_repo_id(url):
return ''.join([s if s in list(_REPOID_CHARS) else '_' for s in url])
@ -55,7 +59,7 @@ index 8ea945da..b095598c 100644
"""
repo_dict = {}
if isinstance(repo, dict):
@@ -737,8 +740,10 @@ def get_repo_dict(repo):
@@ -742,8 +745,10 @@ def get_repo_dict(repo):
if name is None:
name = _translate_url_to_repo_id(url)
else:
@ -68,7 +72,7 @@ index 8ea945da..b095598c 100644
repo['name'] = name
repo['baseurl'] = url
return repo
@@ -749,24 +754,24 @@ def get_repo_dict(repo):
@@ -754,24 +759,24 @@ def get_repo_dict(repo):
repo_dict['name'] = _translate_url_to_repo_id(repo)
repo_dict['baseurl'] = repo
else:
@ -124,7 +128,7 @@ index 031a15a4..c73fe126 100644
interp = 'python2' if six.PY2 else 'python3'
p = subprocess.Popen([interp, PUNGI_CONFIG_VALIDATE, DUMMY_CONFIG],
diff --git a/tests/test_ostree_phase.py b/tests/test_ostree_phase.py
index d57dc682..3a9ebf32 100644
index ad14eca4..3a9ebf32 100644
--- a/tests/test_ostree_phase.py
+++ b/tests/test_ostree_phase.py
@@ -118,7 +118,7 @@ class OSTreeThreadTest(helpers.PungiTestCase):
@ -136,10 +140,12 @@ index d57dc682..3a9ebf32 100644
]
})
self.pool = mock.Mock()
@@ -150,28 +150,6 @@ class OSTreeThreadTest(helpers.PungiTestCase):
@@ -148,36 +148,6 @@ class OSTreeThreadTest(helpers.PungiTestCase):
return {'task_id': 1234, 'retcode': retcode, 'output': 'Foo bar\n'}
return fake_runroot
@mock.patch('pungi.wrappers.scm.get_dir_from_scm')
@mock.patch('pungi.wrappers.kojiwrapper.KojiWrapper')
- @mock.patch('pungi.wrappers.scm.get_dir_from_scm')
- @mock.patch('pungi.wrappers.kojiwrapper.KojiWrapper')
- def test_extra_config_content(self, KojiWrapper, get_dir_from_scm):
- get_dir_from_scm.side_effect = self._dummy_config_repo
- self.compose.conf['runroot_weights'] = {'ostree': 123}
@ -157,15 +163,21 @@ index d57dc682..3a9ebf32 100644
- self.assertTrue(os.path.isfile(extra_config_file))
- with open(extra_config_file, 'r') as f:
- extraconf_content = json.load(f)
- proper_extraconf_content = json.loads('{"repo": [{"name": "http:__example.com_work__basearch_repo", "baseurl": "http://example.com/work/$basearch/repo"}]}')
-
- proper_extraconf_content = {
- "repo": [
- {"name": "http:__example.com_work__basearch_repo",
- "baseurl": "http://example.com/work/$basearch/repo"},
- {"name": "http:__example.com_work__basearch_comps_repo_Everything",
- "baseurl": "http://example.com/work/$basearch/comps_repo_Everything"}
- ]
- }
- self.assertEqual(proper_extraconf_content, extraconf_content)
-
- @mock.patch('pungi.wrappers.scm.get_dir_from_scm')
- @mock.patch('pungi.wrappers.kojiwrapper.KojiWrapper')
@mock.patch('pungi.wrappers.scm.get_dir_from_scm')
@mock.patch('pungi.wrappers.kojiwrapper.KojiWrapper')
def test_run(self, KojiWrapper, get_dir_from_scm):
get_dir_from_scm.side_effect = self._dummy_config_repo
self.compose.conf['runroot_weights'] = {'ostree': 123}
@@ -291,7 +269,7 @@ class OSTreeThreadTest(helpers.PungiTestCase):
@@ -299,7 +269,7 @@ class OSTreeThreadTest(helpers.PungiTestCase):
arch='x86_64',
ref='fedora-atomic/25/x86_64',
commitid=None,
@ -174,7 +186,7 @@ index d57dc682..3a9ebf32 100644
local_repo_path=self.repo)])
@mock.patch('pungi.wrappers.scm.get_dir_from_scm')
@@ -424,14 +402,14 @@ class OSTreeThreadTest(helpers.PungiTestCase):
@@ -432,14 +402,14 @@ class OSTreeThreadTest(helpers.PungiTestCase):
koji.run_runroot_cmd.side_effect = self._mock_runroot(0)
cfg = {
@ -192,7 +204,7 @@ index d57dc682..3a9ebf32 100644
'name': 'Server',
'baseurl': 'Server',
'exclude': 'systemd-container'
@@ -450,13 +428,12 @@ class OSTreeThreadTest(helpers.PungiTestCase):
@@ -458,16 +428,12 @@ class OSTreeThreadTest(helpers.PungiTestCase):
extra_config_file = os.path.join(self.topdir, 'work/ostree-1/extra_config.json')
self.assertTrue(os.path.isfile(extra_config_file))
@ -200,7 +212,10 @@ index d57dc682..3a9ebf32 100644
- extra_config = json.load(extra_config_fd)
+ extra_config = json.load(open(extra_config_file, 'r'))
self.assertTrue(extra_config.get('keep_original_sources', False))
- self.assertEqual(len(extra_config.get('repo', [])), 2) # should equal to number of valid repositories in cfg['repo'] + default repository
- # should equal to number of valid repositories in cfg['repo'] + default repository + comps repository
- self.assertEqual(len(extra_config.get('repo', [])), 3)
- self.assertEqual(extra_config.get('repo').pop()['baseurl'],
- 'http://example.com/work/$basearch/comps_repo_Everything')
- self.assertEqual(extra_config.get('repo').pop()['baseurl'], 'http://example.com/work/$basearch/repo')
+ self.assertEqual(len(extra_config.get('repo', [])), len(cfg['repo']))
+ self.assertEqual(extra_config.get('repo').pop()['baseurl'], 'http://example.com/Server/$basearch/os')
@ -211,7 +226,7 @@ index d57dc682..3a9ebf32 100644
if __name__ == '__main__':
unittest.main()
diff --git a/tests/test_util.py b/tests/test_util.py
index 2b4c2d2c..b4c85a57 100644
index 1f22b63e..322301ff 100644
--- a/tests/test_util.py
+++ b/tests/test_util.py
@@ -588,35 +588,43 @@ class GetRepoFuncsTestCase(unittest.TestCase):
@ -268,5 +283,5 @@ index 2b4c2d2c..b4c85a57 100644
--
2.13.6
2.14.3

131
0004-Include-module-defaults-in-the-repodata.patch
View File

@ -1,131 +0,0 @@
From 7d3baa8531e11059866412ddc6ae1aae4e855fc2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Petr=20=C5=A0abata?= <contyk@redhat.com>
Date: Wed, 4 Apr 2018 12:59:52 +0200
Subject: [PATCH 4/9] Include module defaults in the repodata
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
If the compose configuration includes the module_defaults_dir (an
scm_dict), clone the directory, read the module defaults contained
therein and include relevant defaults in the combined modulemd file.
Only defaults for modules present in the variant are included.
This requires libmodulemd 1.2.0+.
Merges: https://pagure.io/pungi/pull-request/891
Signed-off-by: Petr Šabata <contyk@redhat.com>
---
pungi/compose.py | 4 ++++
pungi/phases/createrepo.py | 8 +++++++-
pungi/phases/init.py | 17 ++++++++++++++++-
tests/test_initphase.py | 3 +++
4 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/pungi/compose.py b/pungi/compose.py
index 4199d72a..07c67f4f 100644
--- a/pungi/compose.py
+++ b/pungi/compose.py
@@ -194,6 +194,10 @@ class Compose(kobo.log.LoggingBase):
return bool(self.conf.get("comps_file", False))
@property
+ def has_module_defaults(self):
+ return bool(self.conf.get("module_defaults_dir", False))
+
+ @property
def config_dir(self):
return os.path.dirname(self.conf._open_file or "")
diff --git a/pungi/phases/createrepo.py b/pungi/phases/createrepo.py
index 1ce2b171..49ff553a 100644
--- a/pungi/phases/createrepo.py
+++ b/pungi/phases/createrepo.py
@@ -218,9 +218,15 @@ def create_variant_repo(compose, arch, variant, pkg_type, modules_metadata=None)
raise AttributeError("module_metadata parameter was not passed and it is needed for module processing")
modules.append(repo_mmd)
+ module_names = set([x.get_name() for x in modules])
+ for mmddeffile in glob.glob(os.path.join(compose.config_dir, "module_defaults", "*.yaml")):
+ for mmddef in Modulemd.objects_from_file(mmddeffile):
+ if isinstance(mmddef, Modulemd.Defaults) and mmddef.peek_module_name() in module_names:
+ modules.append(mmddef)
+
with temp_dir() as tmp_dir:
modules_path = os.path.join(tmp_dir, "modules.yaml")
- Modulemd.Module.dump_all(modules, modules_path)
+ Modulemd.dump(modules, modules_path)
cmd = repo.get_modifyrepo_cmd(os.path.join(repo_dir, "repodata"),
modules_path, mdtype="modules",
diff --git a/pungi/phases/init.py b/pungi/phases/init.py
index a01168a9..81eec0f4 100644
--- a/pungi/phases/init.py
+++ b/pungi/phases/init.py
@@ -23,7 +23,8 @@ from pungi.phases.base import PhaseBase
from pungi.phases.gather import write_prepopulate_file
from pungi.wrappers.createrepo import CreaterepoWrapper
from pungi.wrappers.comps import CompsWrapper
-from pungi.wrappers.scm import get_file_from_scm
+from pungi.wrappers.scm import get_file_from_scm, get_dir_from_scm
+from pungi.util import temp_dir
class InitPhase(PhaseBase):
@@ -63,6 +64,10 @@ class InitPhase(PhaseBase):
# download variants.xml / product.xml?
+ # download module defaults
+ if self.compose.has_module_defaults:
+ write_module_defaults(self.compose)
+
# write prepopulate file
write_prepopulate_file(self.compose)
@@ -160,3 +165,13 @@ def create_comps_repo(compose, arch):
checksum=createrepo_checksum)
run(cmd, logfile=compose.paths.log.log_file(arch, "comps_repo"), show_cmd=True)
compose.log_info("[DONE ] %s" % msg)
+
+
+def write_module_defaults(compose):
+ scm_dict = compose.conf["module_defaults_dir"]
+
+ with temp_dir(prefix="moduledefaults_") as tmp_dir:
+ get_dir_from_scm(scm_dict, tmp_dir, logger=compose._logger)
+ compose.log_debug("Writing module defaults")
+ shutil.rmtree(os.path.join(compose.config_dir, "module_defaults"), ignore_errors=True)
+ shutil.copytree(tmp_dir, os.path.join(compose.config_dir, "module_defaults"))
diff --git a/tests/test_initphase.py b/tests/test_initphase.py
index ae6c4e52..8c3b0d9d 100644
--- a/tests/test_initphase.py
+++ b/tests/test_initphase.py
@@ -24,6 +24,7 @@ class TestInitPhase(PungiTestCase):
def test_run(self, write_prepopulate, write_variant, create_comps, write_arch, write_global):
compose = DummyCompose(self.topdir, {})
compose.has_comps = True
+ compose.has_module_defaults = False
compose.setup_optional()
phase = init.InitPhase(compose)
phase.run()
@@ -52,6 +53,7 @@ class TestInitPhase(PungiTestCase):
write_arch, write_global, copy_comps):
compose = DummyCompose(self.topdir, {})
compose.has_comps = True
+ compose.has_module_defaults = False
compose.variants['Everything'].groups = []
compose.variants['Everything'].modules = []
phase = init.InitPhase(compose)
@@ -81,6 +83,7 @@ class TestInitPhase(PungiTestCase):
write_arch, write_global, copy_comps):
compose = DummyCompose(self.topdir, {})
compose.has_comps = False
+ compose.has_module_defaults = False
phase = init.InitPhase(compose)
phase.run()
--
2.13.6

35
0005-Handle-relative-paths-in-module_defaults_dir.patch
View File

@ -1,35 +0,0 @@
From 91a03c693222a41a23d1a3250984e0d8c9372b75 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Petr=20=C5=A0abata?= <contyk@redhat.com>
Date: Wed, 4 Apr 2018 14:01:10 +0200
Subject: [PATCH 5/9] Handle relative paths in module_defaults_dir
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This now handles strings as well as sct_dicts with type=file that
include relative paths.
Signed-off-by: Petr Šabata <contyk@redhat.com>
---
pungi/phases/init.py | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/pungi/phases/init.py b/pungi/phases/init.py
index 81eec0f4..ef203568 100644
--- a/pungi/phases/init.py
+++ b/pungi/phases/init.py
@@ -169,6 +169,11 @@ def create_comps_repo(compose, arch):
def write_module_defaults(compose):
scm_dict = compose.conf["module_defaults_dir"]
+ if isinstance(scm_dict, dict):
+ if scm_dict["scm"] == "file":
+ scm_dict["dir"] = os.path.join(compose.config_dir, scm_dict["dir"])
+ else:
+ scm_dict = os.path.join(compose.config_dir, scm_dict)
with temp_dir(prefix="moduledefaults_") as tmp_dir:
get_dir_from_scm(scm_dict, tmp_dir, logger=compose._logger)
--
2.13.6

39
0006-Update-configuration-docs-with-module_defaults_dir.patch
View File

@ -1,39 +0,0 @@
From 587b4847843af878650d56e7c093cbef6b356801 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Petr=20=C5=A0abata?= <contyk@redhat.com>
Date: Wed, 4 Apr 2018 14:02:15 +0200
Subject: [PATCH 6/9] Update configuration docs with module_defaults_dir
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: Petr Šabata <contyk@redhat.com>
---
doc/configuration.rst | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/doc/configuration.rst b/doc/configuration.rst
index e5f1e9e0..9e6cf1a9 100644
--- a/doc/configuration.rst
+++ b/doc/configuration.rst
@@ -21,6 +21,7 @@ Minimal Config Example
# GENERAL SETTINGS
comps_file = "comps-f23.xml"
variants_file = "variants-f23.xml"
+ module_defaults_dir = "module_defaults"
# KOJI
koji_profile = "koji"
@@ -135,6 +136,10 @@ Options
(:ref:`scm_dict <scm_support>` or *str*) -- reference to variants XML file
that defines release variants and architectures
+**module_defaults_dir** [optional]
+ (:ref:`scm_dict <scm_support>` or *str*) -- reference the module defaults
+ directory containing modulemd-defaults YAML documents
+
**failable_deliverables** [optional]
(*list*) -- list which deliverables on which variant and architecture can
fail and not abort the whole compose. This only applies to ``buildinstall``
--
2.13.6

29
0007-Update-the-configuration-JSON-schema-for-module_defa.patch
View File

@ -1,29 +0,0 @@
From dc6fccd522efc0ae0c2072812b6fc0559cc74475 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Petr=20=C5=A0abata?= <contyk@redhat.com>
Date: Wed, 11 Apr 2018 10:33:42 +0200
Subject: [PATCH 7/9] Update the configuration JSON schema for
module_defaults_dir
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: Petr Šabata <contyk@redhat.com>
---
pungi/checks.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/pungi/checks.py b/pungi/checks.py
index 189ca948..1c2059b2 100644
--- a/pungi/checks.py
+++ b/pungi/checks.py
@@ -678,6 +678,7 @@ def make_schema():
"type": "boolean",
"default": True
},
+ "module_defaults_dir": {"$ref": "#/definitions/str_or_scm_dict"},
"pkgset_repos": {
"type": "object",
--
2.13.6

64
0008-Clone-module-defaults-into-work-directory.patch
View File

@ -1,64 +0,0 @@
From 9104444a7bc874c18de2a57851356e60a776341c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Lubom=C3=ADr=20Sedl=C3=A1=C5=99?= <lsedlar@redhat.com>
Date: Wed, 11 Apr 2018 16:05:08 +0200
Subject: [PATCH 8/9] Clone module defaults into work/ directory
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
We can not rely on config_dir being writable, and should not modify
anything in there anyway.
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
---
pungi/paths.py | 8 ++++++++
pungi/phases/createrepo.py | 2 +-
pungi/phases/init.py | 3 +--
3 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/pungi/paths.py b/pungi/paths.py
index 0e506795..37c6da13 100644
--- a/pungi/paths.py
+++ b/pungi/paths.py
@@ -340,6 +340,14 @@ class WorkPaths(object):
path = "%s.cfg" % path
return path
+ def module_defaults_dir(self, create_dir=True):
+ """
+ """
+ path = os.path.join(self.topdir(create_dir=create_dir), 'module_defaults')
+ if create_dir:
+ makedirs(path)
+ return path
+
class ComposePaths(object):
def __init__(self, compose):
diff --git a/pungi/phases/createrepo.py b/pungi/phases/createrepo.py
index 49ff553a..f8e61387 100644
--- a/pungi/phases/createrepo.py
+++ b/pungi/phases/createrepo.py
@@ -219,7 +219,7 @@ def create_variant_repo(compose, arch, variant, pkg_type, modules_metadata=None)
modules.append(repo_mmd)
module_names = set([x.get_name() for x in modules])
- for mmddeffile in glob.glob(os.path.join(compose.config_dir, "module_defaults", "*.yaml")):
+ for mmddeffile in glob.glob(os.path.join(compose.paths.work.module_defaults_dir(), "*.yaml")):
for mmddef in Modulemd.objects_from_file(mmddeffile):
if isinstance(mmddef, Modulemd.Defaults) and mmddef.peek_module_name() in module_names:
modules.append(mmddef)
diff --git a/pungi/phases/init.py b/pungi/phases/init.py
index ef203568..9d9212e7 100644
--- a/pungi/phases/init.py
+++ b/pungi/phases/init.py
@@ -178,5 +178,4 @@ def write_module_defaults(compose):
with temp_dir(prefix="moduledefaults_") as tmp_dir:
get_dir_from_scm(scm_dict, tmp_dir, logger=compose._logger)
compose.log_debug("Writing module defaults")
- shutil.rmtree(os.path.join(compose.config_dir, "module_defaults"), ignore_errors=True)
- shutil.copytree(tmp_dir, os.path.join(compose.config_dir, "module_defaults"))
+ shutil.copytree(tmp_dir, compose.paths.work.module_defaults_dir())
--
2.13.6

27
0009-init-Stop-creating-module-defaults-dir-twice.patch
View File

@ -1,27 +0,0 @@
From 57342f31e94118b172366b7a00b6b5fcfd62c28a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Lubom=C3=ADr=20Sedl=C3=A1=C5=99?= <lsedlar@redhat.com>
Date: Thu, 12 Apr 2018 13:37:30 +0200
Subject: [PATCH 9/9] init: Stop creating module defaults dir twice
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
JIRA: COMPOSE-2447
Signed-off-by: Lubomír Sedlář <lsedlar@redhat.com>
---
pungi/phases/init.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pungi/phases/init.py b/pungi/phases/init.py
index 9d9212e7..d6afcd92 100644
--- a/pungi/phases/init.py
+++ b/pungi/phases/init.py
@@ -178,4 +178,4 @@ def write_module_defaults(compose):
with temp_dir(prefix="moduledefaults_") as tmp_dir:
get_dir_from_scm(scm_dict, tmp_dir, logger=compose._logger)
compose.log_debug("Writing module defaults")
- shutil.copytree(tmp_dir, compose.paths.work.module_defaults_dir())
+ shutil.copytree(tmp_dir, compose.paths.work.module_defaults_dir(create_dir=False))
--
2.13.6

44
0010-Access-ci_base-date-via-compose.patch
View File

@ -1,44 +0,0 @@
From 527394707d118d36977b17091eb39212a1f1aae0 Mon Sep 17 00:00:00 2001
From: Patrick Uiterwijk <puiterwijk@redhat.com>
Date: Apr 15 2018 00:16:23 +0000
Subject: Access ci_base date via compose
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
---
diff --git a/pungi/util.py b/pungi/util.py
index 261f3f2..b8cf9ba 100644
--- a/pungi/util.py
+++ b/pungi/util.py
@@ -789,7 +789,7 @@ def version_generator(compose, gen):
return compose.image_release
elif gen == '!VERSION_FROM_VERSION_DATE_RESPIN':
return '%s.%s.%s' % (compose.ci_base.release.version,
- compose.ci_base.date,
+ compose.compose_date,
compose.compose_respin)
elif gen and gen[0] == '!':
raise RuntimeError("Unknown version generator '%s'" % gen)
diff --git a/tests/test_util.py b/tests/test_util.py
index 2b4c2d2..1f22b63 100644
--- a/tests/test_util.py
+++ b/tests/test_util.py
@@ -626,7 +626,6 @@ class TestVersionGenerator(unittest.TestCase):
ci.respin = 0
ci.id = 'RHEL-8.0-20180101.0'
ci.release.version = '8'
- ci.date = '20160101'
ci.type = 'nightly'
ci.type_suffix = ''
ci.label = 'RC-1.0'
@@ -635,6 +634,7 @@ class TestVersionGenerator(unittest.TestCase):
self.compose = mock.MagicMock()
self.compose.ci_base = ci
self.compose.compose_respin = 0
+ self.compose.compose_date = '20160101'
def test_unknown_generator(self):
compose = mock.Mock()

68
pungi.spec
View File

@ -1,23 +1,15 @@
Name: pungi
Version: 4.1.23
Release: 5%{?dist}
Version: 4.1.24
Release: 1%{?dist}
Summary: Distribution compose tool
Group: Development/Tools
License: GPLv2
URL: https://pagure.io/pungi
Source0: https://pagure.io/releases/%{name}/%{name}-%{version}.tar.bz2
Patch0: 0001-tests-Use-dummy-modulesdir-for-DNF.patch
Patch1: 0001-Revert-Move-ostree-phase-and-pipelines-for-running-p.patch
Patch2: 0002-Revert-Other-repo-for-OstreeInstaller.patch
Patch3: 0003-Revert-Ostree-can-use-pkgset-repos.patch
Patch4: 0004-Include-module-defaults-in-the-repodata.patch
Patch5: 0005-Handle-relative-paths-in-module_defaults_dir.patch
Patch6: 0006-Update-configuration-docs-with-module_defaults_dir.patch
Patch7: 0007-Update-the-configuration-JSON-schema-for-module_defa.patch
Patch8: 0008-Clone-module-defaults-into-work-directory.patch
Patch9: 0009-init-Stop-creating-module-defaults-dir-twice.patch
Patch10: 0010-Access-ci_base-date-via-compose.patch
Patch0: 0001-Revert-Move-ostree-phase-and-pipelines-for-running-p.patch
Patch1: 0002-Revert-Other-repo-for-OstreeInstaller.patch
Patch2: 0003-Revert-Ostree-can-use-pkgset-repos.patch
BuildRequires: python3-nose
BuildRequires: python3-mock
@ -42,7 +34,7 @@ BuildRequires: python3-koji
BuildRequires: python3-unittest2
BuildRequires: lorax
BuildRequires: python3-PyYAML
BuildRequires: libmodulemd
BuildRequires: libmodulemd >= 1.3.0
BuildRequires: python3-gobject
#deps for doc building
@ -75,7 +67,7 @@ Requires: python3-multilib
Requires: python3-libcomps
Requires: python3-six
Requires: python3-koji
Requires: libmodulemd
Requires: libmodulemd >= 1.3.0
Requires: python3-gobject
Requires: python3-pdc-client
@ -184,6 +176,52 @@ nosetests-3 --exe
%{_bindir}/%{name}-wait-for-signed-ostree-handler
%changelog
* Wed May 02 2018 Lubomír Sedlář <lsedlar@redhat.com> - 4.1.24-1
- koji-wrapper: Log failed subtasks (lsedlar)
- Update compose status when config validation fails (lsedlar)
- pkgset: Allow different inheritance for modules (lsedlar)
- ostree: Recognize force_new_commit option in old config (lsedlar)
- modules: Correctly report error for unexpected modules (lsedlar)
- modules: Allow context in variants XML (lsedlar)
- gather: Print profiling information to stderr (lsedlar)
- pkgset: Stop creating database for repodata (jkaluza)
- gather: Use another variant as lookaside (lsedlar)
- buildinstall: Use metadata if skipped (lsedlar)
- Allow reusing pkgset FileCache from old composes. (jkaluza)
- validation: Populate dict of all variants (lsedlar)
- gather: Stop pulling debuginfo and source for lookaside packages (lsedlar)
- Only use comps repo if we really have comps (lsedlar)
- pkgset: Use modules PDC API (lsedlar)
- Access ci_base date via compose (puiterwijk)
- Allow filtering comps for different variants (lsedlar)
- comps: Make filtering by attribute more generic (lsedlar)
- pkgset: Dump downloaded modulemd to logs (lsedlar)
- Fix PEP8 warning about if not x in y (lsedlar)
- Variant as a lookaside - configuration (onosek)
- Remove comps from arch repo (lsedlar)
- init: Stop creating module defaults dir twice (lsedlar)
- gather: Reduce logs from DNF gathering (lsedlar)
- Clone module defaults into work/ directory (lsedlar)
- Update the configuration JSON schema for module_defaults_dir (contyk)
- Update configuration docs with module_defaults_dir (contyk)
- Handle relative paths in module_defaults_dir (contyk)
- Include module defaults in the repodata (contyk)
- Add *.in fixtures to tarball (lsedlar)
- init: Always filter comps file (lsedlar)
- docs: Describe comps processing (lsedlar)
- gather: Use comps for given variant (lsedlar)
- docs: Fix typo (lsedlar)
- Add all packages to whitelist for hybrid variant (lsedlar)
- comps: Add tests for CompsFilter (lsedlar)
- comps: Move filtering into wrapper module (lsedlar)
- Tests fail if unittest2 library is missing (onosek)
- Add unittest2 and rpmdevools to contributing doc (rmarshall)
- pkgset: Construct UID for PDC modules (lsedlar)
- gather: Simplify creating temporary directory (lsedlar)
- buildinstall: Add extra repos (lsedlar)
- tests: Use dummy modulesdir for DNF (lsedlar)
- Update tests for Python 2.6 (onosek)
* Tue Apr 24 2018 Kevin Fenzi <kevin@scrye.com> - 4.1.23-5
- Backport fix for Accessing ci_base date via compose
- https://pagure.io/pungi/pull-request/910

2
sources
View File

@ -1 +1 @@
SHA512 (pungi-4.1.23.tar.bz2) = 0521e002f36d8effbdf4e412bb4161830b4b05f50ef3ec88af9bedc046246f7aa120f97274a4588f4469576ffd4f8c2195c4b3186e97738b6497e844a7f66aef
SHA512 (pungi-4.1.24.tar.bz2) = 91b192467fdd09d0f2ef99a1194f3b9a2fe7f316549c6ec844984f4eeefbf776e95f4f5cdbb9338c4dd32c9fd5a57262d9761cf1d14843f4c6b8775ea49698eb