Commit Graph

1814 Commits

Author SHA1 Message Date
Jan Stodola
73de195334 Use existing storage account
To avoid creating a new storage account for every new VM.

Related: rhbz#1673012
2019-04-22 15:57:07 +03:00
Jan Stodola
fabf428f00 Record date/time of VM creation
Related: rhbz#1673012
2019-04-22 15:57:07 +03:00
Chris Roberts
2ebf60f28d Update datastore for VMware testing
Cluster and Network changed due to new hardware in RDU2 DC

Related: rhbz#1656105
2019-04-18 10:35:53 +03:00
Jiri Kortus
6fb94326e7 Fixes for locked root account test
Fixes related to a few issues in the locked root account test I somehow
managed to overlook in the initial commit.

Related: rhbz#1687595
2019-04-09 18:54:03 +03:00
Jiri Kortus
277d391f01 Add checks for disabled root account
The root account checks are applied to generated and deployed images
to make sure that root account is locked, except for live ISO.

Related: rhbz#1687595
2019-03-29 15:22:57 +02:00
Jiri Kortus
5d6446a7bd 'compose info' is 'compose details' on RHEL-7
Add the correct composer-cli command 'compose details' in places
where 'compose info' was still used on RHEL-7
(see commit 808454b561)

Related: rhbz#1687595
2019-03-29 14:44:53 +02:00
Chris Roberts
c6e8befe21 Update some grammer issues in the test Bash scripts
Related: rhbz#1656105
2019-03-22 12:15:38 +02:00
Chris Roberts
48a28e0950 Update datastore for VMware testing
Datastore switched from NFS to iSCSI

Related: rhbz#1656105
2019-03-22 12:11:56 +02:00
Brian C. Lane
86718c517d Automatic commit of package [lorax-composer] release [19.7.29-1].
Created by command:

/bin/tito tag
2019-03-19 09:27:03 -07:00
Alexander Todorov
64baf7c147 Allow overriding $CLI outside test scripts
this will allow you to test against installed RPM like so:

    # export CLI="/usr/bin/composer-cli"
    # make test_images

If you already have lorax-composer running then you can directly
execute test scripts:

    # ./tests/cli/test_build_and_deploy_aws.sh

Related: rhbz#1687595
2019-03-15 11:02:00 +02:00
Alexander Todorov
e96a5917bb Use make ci inside test-in-copy target
this minimizes the possibility of these two to diverge over time.
make ci is the default for Jenkins and will also be used for
internal gating tests.

Related: rhbz#1687595
2019-03-15 11:02:00 +02:00
Alexander Todorov
4fb3e42a70 New test: Build live-iso and boot with KVM
explicitly enables sshd for live-iso during testing

Related: rhbz#1656105
2019-03-04 22:08:43 +02:00
Alexander Todorov
561302a30a New test: Build qcow2 compose and test it with QEMU-KVM
Related: rhbz#1656105
2019-02-27 21:46:44 +02:00
Alexander Todorov
c24ad2fed7 New test: Verify tar images with Docker and systemd-nspawn
- on some arches (also Fedora x86_64) systemd-nspawn may not be
  available
- delete composes from other tests in rlPhaseStartCleanup because
  we're seeing the tar compose kind of hanging in Jenkins and that
  test script is executed last so the slave may be running out of
  disk space. Be a good citizen and clean up after the previous
  tests.

Related: rhbz#1656105
2019-02-27 21:46:44 +02:00
Alexander Todorov
41e3f8fecb Update OpenStack flavor and network settings in tests
b/c we've migrated to Upshift we must use different instance type,
specify the desired network to connect to and update how we get
the ip address of the launched VM.

Related: rhbz#1656105
2019-02-27 21:46:44 +02:00
Alexander Todorov
44a79a8a6b Install ansible and openstacksdk inside virtualenv
otherwise we get a conflict with python-ipaddress which is a
dependency of python2-pip and can't really be removed!

Installing from RPM is also a no-go b/c openstacksdk is
not available in EPEL 7. OTOH the RDO repositories which
OpenStack recommends have older version (0.11) and ansible wants
0.12 or later!

Related: rhbz#1656105
2019-02-27 21:46:44 +02:00
Alexander Todorov
2f51fcfb8b Remove python-requests, python-dateutil and pyOpenSSL
b/c they conflict with the modules coming from pip and
prevent azure/ansible installation

Related: rhbz#1656105
2019-02-27 21:46:44 +02:00
Alexander Todorov
5adde2671e Add /usr/local/bin to PATH for tests
pip will install commands there and later scripts will not be able
to find them.

Related rhbz#1656105
2019-02-27 21:46:44 +02:00
Alexander Todorov
9c0f34c0ab Do not generate journal.xml from beakerlib
bacause this requires additional Python modules and we don't
really use it!

Related: rhbz#1656105
2019-02-27 21:46:44 +02:00
Jan Stodola
cd1f26d4e7 Expand parameters as separate words
Related rhbz#1656105
2019-02-27 21:46:44 +02:00
Brian C. Lane
619daa1307 Automatic commit of package [lorax-composer] release [19.7.28-1].
Created by command:

/bin/tito tag
2019-02-25 12:03:19 -08:00
Brian C. Lane
f9c1dad0ba lorax-composer: Check for STATUS before deleting
When there is no run or new symlink do one last check to make sure no
STATUS file was written. If it is missing, go ahead and remove the
results directory.

(cherry picked from commit a8f616c6da)

Related: rhbz#1659129
2019-02-21 16:47:07 -08:00
Brian C. Lane
0baa524d8d Check for existing CANCEL request, and exit on FINISHED
If another CANCEL request has already been made just exit from
uuid_cancel. If the build is FINISHED before it times out just exit,
don't remove the finished results.

(cherry picked from commit 962d10b3b7)

Related: rhbz#1659129
2019-02-21 09:37:24 -08:00
Brian C. Lane
48c8ae6819 Add cancel_func to virt and novirt_install functions
In addition to monitoring the logs for errors, call a function (or
functions) that tell it to cancel the anaconda process and cleanup.

Also check for a cancel after creating the squashfs image for live-iso
since that's a long running process.

This required adding a new argument to a number of existing functions,
passing it down to VirtualInstall and novirt_install where the function
is called.

(cherry picked from commit 4b84475612)

Resolves: rhbz#1659129
2019-02-21 09:37:24 -08:00
Brian C. Lane
ade25f34b3 Remove duplicate repositories from the sources list
In some cases when the host has, for whatever reason, multiple copies of
the same repo listed the build may fail with an error about running out
of space.

So this commit removes duplicate entries after the host's repos have been
loaded. It also adjusts some of the test repos to use different
temporary repo names for the tests.

Resolves: rhbz#1664128
2019-02-21 09:19:30 -08:00
Brian C. Lane
3e5e8b9f1d Remove unneeded else from for/else loop. It confuses pylint
(cherry picked from commit 2950f2641b)

Related: rhbz#1666517
2019-02-19 14:13:32 -08:00
David Shea
252c9f6226 Allow customizations to be specified as a toml list
Support both

  [customizations]
  hostname = "whatever"

and

  [[customizations]]
  hostname = "whatever"

in the blueprint data. The [[ syntax matches the other customization
directives (user, group, sshkey), and as such it's easy to accidentally
use it for the hostname without even realizing it's specifying something
different.

Add some tests for converting customizations to kickstarts.

(cherry picked from commit 35ab6a1336)

Resolves: rhbz#1666517
2019-02-19 14:04:43 -08:00
Jiri Kortus
56d0e704c0 Make sure compose build tests run with SELinux in enforcing mode
Related: rhbz#1654795
2019-02-06 13:59:17 +01:00
Brian C. Lane
47fd6e85b2 Add tests for metapackages and package name globs
This makes sure that depsolving shim installs the shim-* package, and
that depsolving grub2-efi-*-cdboot installs a specific -cdboot package.

Related: rhbz#1641601
2019-01-24 09:57:08 -08:00
Alexander Todorov
2ac027c78b Upgrade pip & setuptools b/c they are rather old
and sometimes Python dependencies fail to install with:

SetuptoolsOutdatedWarning: your setuptools is too old (<12)
2019-01-09 10:55:58 +02:00
Alexander Todorov
177e62a896 Workaround openstacksdk dependency issue
See:
fd61b54679 (diff-b4ef698db8ca845e5845c4618278f29a)

Note: may also affect master/rhel8-branch but haven't seen it
so far. For master we can do:

	dnf install ansible python3-openstacksdk

Not so easy on RHEL 7
2018-12-20 10:55:04 +02:00
Alexander Todorov
7143456043 On Python 2 Azure needs the futures module
this will prevent
ImportError: No module named concurrent.futures

see
https://github.com/Azure/azure-sdk-for-python/issues/2166
2018-12-20 10:55:04 +02:00
Alexander Todorov
d63de76cb3 On RHEL 7 we have Python 2, not Python 3 2018-12-20 10:55:04 +02:00
Alexander Todorov
5dc19500ad On RHEL 7 we have yum instead of dnf 2018-12-20 10:55:04 +02:00
Alexander Todorov
808454b561 On RHEL 7 compose info is compose details 2018-12-20 10:55:04 +02:00
Brian C. Lane
74792f6b6e Report an error if the blueprint doesn't exist
composer-cli uses TOML for 'blueprints save' which was returning an
empty 200 response if the blueprint didn't exist. Change this to return
a standard 400 error response if the blueprint doesn't exist.

composer-cli is already setup to handle receiving json when an error is
returned so just the toml API response for `blueprints/save` needed to
be changed.
2018-12-20 10:55:04 +02:00
Alexander Todorov
aebc3bf280 Build the HTML docs before running tests
on master docs/html/ is part of the git repository, here it is not
2018-12-20 10:55:04 +02:00
Brian C. Lane
dccd283dd2 Disable pylint errors with Flask and gevent
pylint has trouble with Flask response objects, thinking they are tuples
and returning no-member errors. It also doesn't recognize gevent.socket
members like AF_UNIX.
2018-12-20 10:55:04 +02:00
Alexander Todorov
f8137b94db Backport cloud image tests from master 2018-12-20 10:55:04 +02:00
Brian C. Lane
63306c1f64 Fix compose_args for openstack image
RHEL7 arguments are slightly different from the RHEL8 version this was
ported from.

Related: rhbz#1656105
2018-12-14 15:23:43 -08:00
Brian C. Lane
7291c8f8c3 Fix compose_args for vmdk image
RHEL7 arguments are slightly different from the RHEL8 version this was
ported from.

Related: rhbz#1656105
2018-12-14 15:23:43 -08:00
Brian C. Lane
f426a357b1 Fix compose_args for vhd image
RHEL7 arguments are slightly different from the RHEL8 version this was
ported from.

Related: rhbz#1656105
2018-12-14 15:23:43 -08:00
Brian C. Lane
ab17674f75 Fix compose_args for ami image
RHEL7 arguments are slightly different from the RHEL8 version this was
ported from.

Related: rhbz#1656105
2018-12-14 14:19:54 -08:00
Brian C. Lane
5797099552 Update projects list to return only the unique projects
The projects_list function uses a different yum request, and doesn't
return a full project dict. Updated it to also return only the unique
project names.

Related: rhbz#1657055
2018-12-07 15:07:36 -08:00
Brian C. Lane
b449d1cd81 Change yaps_to_module to proj_to_module
It now gets a project dict instead of a yum object, so change it to use
the dict's "name" and update the tests.

Related: rhbz#1657055
2018-12-07 14:41:23 -08:00
Brian C. Lane
0e54983be8 lorax-composer: Handle packages with multiple builds
When the repository has multiple arches, eg. i686 and x86_64, it should
add a new entry to the project's builds list, not create a new project
in the list.

This handles that by adding a modified insort_left function and
examining the packages returned from dnf to make sure they aren't
already listed in the results. It also handles adding them in sorted
order so that no further sorting needs to be done on the results.

Resolves: rhbz#1657055
(cherry picked from commit 663a0dcd73)
2018-12-07 14:10:54 -08:00
Brian C. Lane
72bd52e061 lorax-composer: Check the queue and results at startup
If the system ran out of space, or was rebooted unexpectedly, the state
of the queue symlinks, or the results STATUS files may be inconsistent.
This checks them and:
 * Removes broken symlinks from queue/new and queue/run
 * Removes symlinks from run and sets the build to FAILED
 * Sets builds w/o a STATUS to FAILED
 * Sets builds with STATUS of RUNNING to FAILED
 * Creates missing queue/new symlinks to results with STATUS of WAITING

So, any builds that were running during the reboot will be FAILED, and
any that were waiting to be started will be started upon rebooting.

Resolves: rhbz#1657054
(cherry picked from commit f0bac40d7f)
2018-12-07 13:48:10 -08:00
Brian C. Lane
e1d2e168ed Add an openstack image type
This is a qcow2 image with cloud-init in the template.

(cherry picked from commit 98f8b23129)

Resolves: rhbz#1656105
2018-12-07 13:59:45 -05:00
David Shea
5e64051665 Replace /etc/machine-id with an empty file
Since these images can be used to create multiple machines, they should
not have a unique machine-id attached to them. Replace /etc/machine-id
with an empty file so that it will be regenerated at boot time.

(cherry picked from commit 6fab72d894)

Related: rhbz#1656105
2018-12-07 13:59:45 -05:00
David Shea
4f28d87a6d Add virt guest agents to the qcow2 compose
(cherry picked from commit d5a1993640)

Resolves: rhbz#1656105
2018-12-07 13:59:45 -05:00