Commit Graph

2049 Commits

Author SHA1 Message Date
Alexander Todorov
ce361d6f48 New test: Verify tar images with Docker and systemd-nspawn
- on some arches (also Fedora x86_64) systemd-nspawn may not be
  available
- delete composes from other tests in rlPhaseStartCleanup because
  we're seeing the tar compose kind of hanging in Jenkins and that
  test script is executed last so the slave may be running out of
  disk space. Be a good citizen and clean up after the previous
  tests.

(cherry picked from commit ea78cce882)
2019-05-03 17:03:40 -07:00
Alexander Todorov
b7b8731c03 Update OpenStack flavor and network settings in tests
b/c we've migrated to Upshift we must use different instance type,
specify the desired network to connect to and update how we get
the ip address of the launched VM.

(cherry picked from commit c95d7084a6)
2019-05-03 17:03:40 -07:00
Brian C. Lane
a08dd1585c Remove 3G minimum from lorax-composer
The reason for the 3G minimum was because anaconda had a bug with how it
calculated minimum disk size when using kickstart. The gix for this has
been in Anaconda since 29.19-1, so we can now remove our limit and
create somewhat smaller disk images.

(cherry picked from commit 7e78dc368f)
2019-05-03 17:02:28 -07:00
Chris Lumens
1a76c4816f Run as root/weldr by default.
We need to be root to read the certificates that give access to the
package repos.  Right now, the alternative seems to be changing
permissions on the certs themselves, which seems less good.  We're
running anaconda as root anyway.

(cherry picked from commit 022e9eba3e)
2019-05-03 17:01:30 -07:00
Lars Karlitski
8814231a09 Pass ssl certificate options to anaconda
If a repository has `sslcacert`, `sslclientcert`, or `ssclientkey` set,
pass them to anaconda through the kickstart file. This is mostly the
case when using RHEL repositories that are accessed through a
subscription.

(cherry picked from commit e194b5926c)
2019-05-03 17:01:30 -07:00
Brian C. Lane
5ed5690fb6 Drop auth from the kickstart examples
System defaults should be sufficient these days.

(cherry picked from commit 2af0973155)
2019-05-03 17:01:30 -07:00
Alexander Todorov
ea63fbcc5b Keep OpenStack VMs with Tag keep_me
so we can have some systems for debugging

(cherry picked from commit 5dc895fbee)
2019-05-03 17:01:30 -07:00
Jiri Kortus
f7e71768fa Make sure compose build tests run with SELinux in enforcing mode
(cherry picked from commit 57be7313c4)
2019-05-03 17:01:30 -07:00
Alexander Todorov
2ad44f3a46 Update with instructions about commit log referencing Bugzilla
do not merge in rhel7 and rhel8 branches if commit log isn't
referencing an approved bug!

(cherry picked from commit 20ebfce164)
2019-05-03 17:01:30 -07:00
Jan Stodola
45710e435a Add script for removing old artifacts from Azure
(cherry picked from commit f26fcba902)
2019-05-03 17:01:30 -07:00
Jan Stodola
c6bff67549 Use existing storage account
To avoid creating a new storage account for every new VM.

(cherry picked from commit 8a8802aab7)
2019-05-03 17:01:30 -07:00
Jan Stodola
0d38224f12 Record date/time of VM creation
(cherry picked from commit e918ff807c)
2019-05-03 17:01:30 -07:00
Igor Gnatenko
f6bf2cf460 Remove obsolete Group tag
References: https://fedoraproject.org/wiki/Changes/Remove_Group_Tag
(cherry picked from commit 5217215c4b)
2019-05-03 17:00:24 -07:00
Brian C. Lane
6debc2252c fedora-livemedia.ks: Add packages needed to boot livecd on UEFI systems
(cherry picked from commit 9577f719ae)
2019-05-03 16:59:25 -07:00
Jan Stodola
43a79746fc Export OS_PROJECT_NAME variable in openstack scripts
The OS_PROJECT_NAME (or OS_TENANT_NAME) environment variable needs to be defined.
Use the OS_PROJECT_NAME, since it is recommended in the documentation instead of
the older OS_TENANT_NAME.

(cherry picked from commit cc6fdb2fac)
2019-05-03 16:57:37 -07:00
Jan Stodola
d23ec308d4 Collect results from all cleanup scripts
Correctly detect if any of the executed cleanup scripts didn't pass.

(cherry picked from commit 67dcff18d9)
2019-05-03 16:57:37 -07:00
Alexander Todorov
fe091060b8 Typo in PR template
(cherry picked from commit d8439bdc11)
2019-05-03 16:57:37 -07:00
Jan Stodola
12cd27375a Expand parameters as separate words
(cherry picked from commit 961ea18145)
2019-05-03 16:57:37 -07:00
Alexander Todorov
a839824f7a Add PR template with instructions for repo members
this will serve as a reminder that sometimes Jenkins jobs can be
missing or failing and also lists the comments which team members
can use to trigger Jenkins jobs, especially for PRs from
non-members.

(cherry picked from commit de6419f0d1)
2019-05-03 16:57:37 -07:00
Jan Stodola
34c24117fe Add script for removing old artifacts from OpenStack
(cherry picked from commit 6680e2ffec)
2019-05-03 16:57:37 -07:00
Jan Stodola
8885d7eaca Add script for removing old artifacts from AWS
(cherry picked from commit 677c858f60)
2019-05-03 16:57:37 -07:00
Brian C. Lane
2fd6c6b549 Add timestamps to program.log and dnf.log
In lorax, lorax-composer, and livemedia-creator.

(cherry picked from commit fd173f7265)
2019-05-03 16:54:37 -07:00
Alexander Todorov
a038f7d904 tests: use the first IP address if more than 1 retruned from OpenStack
(cherry picked from commit 883a959442)
2019-05-03 16:52:44 -07:00
Alexander Todorov
963c7d8f89 tests: remove a debugging command
(cherry picked from commit 6563a33f63)
2019-05-03 16:52:44 -07:00
David Shea
3daede741c Add openstack to the image type list in the docs
(cherry picked from commit d619430d6d)
2019-05-03 16:52:44 -07:00
Alexander Todorov
4f3b81e4e7 Teach OpenStack test to distinguish between RHEL and Fedora
(cherry picked from commit 642b909d24)
2019-05-03 16:50:33 -07:00
Alexander Todorov
7b306e2e84 Use full path for Azure playbook as well
(cherry picked from commit 543a4d88d9)
2019-05-03 16:47:52 -07:00
Alexander Todorov
9584cbd96f Use a temporary dir for ssh keys during testing
resolves path problems with sudo, we already have the same change
in the AWS test script.

(cherry picked from commit 16260c1569)
2019-05-03 16:47:46 -07:00
Alexander Todorov
c4f3ceb0ec Update V_DATASTORE b/c defaults appear to have been changed
(cherry picked from commit 59050df710)
2019-05-03 16:47:38 -07:00
Alexander Todorov
41aa84ba04 Clone pyvmomi samples in the correct directory
(cherry picked from commit e0c8cdc5a8)
2019-05-03 16:47:10 -07:00
Alexander Todorov
a8ff02113b Use full path when pushing toml files during testing
(cherry picked from commit 8f430be2a6)
2019-05-03 16:47:00 -07:00
Alexander Todorov
80c18a6e63 Add empty ci_after_success target for Jenkins
Jenkins uses templates to define all jobs which means they need to
have the same make targets even if the targets don't do anything.

(cherry picked from commit 57b4f2e8f3)
2019-05-03 16:46:54 -07:00
Alexander Todorov
efa9974609 Implicitly specify ssh key directory/files for testing
this is to work around identity/sudo problems

(cherry picked from commit 4b4f4bd0e0)
2019-05-03 16:46:48 -07:00
Alexander Todorov
a907a3757d [test] Clean up containers.json
(cherry picked from commit ed63611cb7)
2019-05-03 16:46:39 -07:00
Alexander Todorov
eda242623b Teach AWS test to distinguish between RHEL and Fedora
(cherry picked from commit ab048947cb)
2019-05-03 16:46:32 -07:00
Alexander Todorov
8c409c8263 Use a temporary shared dir when testing
otherwise composer-cli is unable to glob() the kickstart
files and we're left without supported compose types. Seen
during AWS testing for example.

Helps with running some of the tests via sudo b/c this is
what Jenkins requires.

(cherry picked from commit b88466fd74)
2019-05-03 16:43:41 -07:00
Alexander Todorov
9c17c0cc6c Copy blueprints used for testing to temporary directory
If trying to execute test_cli.sh inside a git checkout
we are going to get the following exception:

Traceback (most recent call last):
  File "./src/sbin/lorax-composer", line 251, in <module>
    repo = open_or_create_repo(server.config["REPO_DIR"])
  File "/home/jenkins/lorax/src/pylorax/api/recipes.py", line 306, in open_or_create_repo
gi.repository.GLib.Error: ggit-error: failed to stat '/home/jenkins/lorax/tests/pylorax/blueprints': Permission denied (-1)
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/popen_fork.py", line 54, in _send_signal
    os.kill(self.pid, sig)

From what I can tell open_or_create_repo() is trying to initialize
a git repository inside the blueprints directory which fails when
we have an active git checkout.

This doesn't happen when we run the tests in Travis CI because
rsync excludes .git/ inside the Docker container.

(cherry picked from commit c9d706a382)
2019-05-03 16:43:35 -07:00
Alexander Todorov
178eb927d6 Add make targets for Jenkins
these targets help hooking up things in Jenkins and enable us to
perform build & deploy tests for cloud images.

NOTE: use sudo -E to preserve the environment
(cherry picked from commit 366ae55abe)
2019-05-03 16:43:28 -07:00
Alexander Todorov
2184cff59c Teach test_cli.sh to execute test scripts via arguments
this will be used to invoke scripts that build/push cloud images
without having to duplicate the setup/teardown/report parts!

(cherry picked from commit af2ae790ce)
2019-05-03 16:41:00 -07:00
Alexander Todorov
63e35921fa new test: build an image and deploy it on Azure
(cherry picked from commit 13fbd0861b)
2019-05-03 16:40:54 -07:00
Alexander Todorov
9b31c3cad9 Fix typo in comment
(cherry picked from commit d6dcb38a78)
2019-05-03 16:40:48 -07:00
Brian C. Lane
a0a76a800a Fix reporting of coverage results to coverall.io
When I re-arranged the test-in-docker I didn't realize how .travis.yml
was extracting the results. This should fix it.

When running with test-in-docker we mount the source read-only on
/linux-ro/ inside the container and copy it over to /lorax/ for running
the tests.

The local directory ./.test-results/ is mounted on /test-results/ in the
container and the .coverage file is copied into there so that it is
available on the host.

(cherry picked from commit b61a91954a)
2019-05-03 16:40:42 -07:00
Alexander Todorov
3fd9699e6d For OpenStack build image with rng-tools installed
appears to be a decent workaround for slow boot likely caused
by lack of entropy for ssh-keygen, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1639326

(cherry picked from commit 6ae78fbd47)
2019-05-03 16:39:25 -07:00
Brian C. Lane
51a9c28bc6 Add tests for partitioned disk images
The setup for this is a bit complex, so it really ends up testing things
twice.

(cherry picked from commit fb4e6f2588)
2019-05-03 16:39:18 -07:00
Brian C. Lane
faad43ced8 Create a kpartx_disk_img function
So that it can be used in tests as well as in imgutils

(cherry picked from commit d69f01d4a8)
2019-05-03 16:39:12 -07:00
Brian C. Lane
e7310c7260 Add tests for pylorax.imgutils
Some of these can only run as root on a real system with access to loop
devices. They are skipped when running in a container.

(cherry picked from commit 063a1770e1)
2019-05-03 16:39:05 -07:00
Brian C. Lane
0aa206f661 Add tests to test_creator.py
Add a /.in-container file to the container root so that tests requiring root
and loop device support will be skipped when running in a container.

(cherry picked from commit bab4b20d0d)
2019-05-03 16:39:01 -07:00
Brian C. Lane
5fa5b8e142 Fix make_appliance and the libvirt.tmpl
Apparently nobody has used these since the switch to py3, xrange is now
range and it needs to read the file in binary mode when generating the
sha256.

(cherry picked from commit 8e749efbbf)
2019-05-03 16:38:54 -07:00
Brian C. Lane
ef45d169b0 Add some tests for creator.py
This is complicated by the fact that much of this module requires mount.
So for now just test the things that don't need mount.

(cherry picked from commit 134a333d92)
2019-05-03 16:38:47 -07:00
Brian C. Lane
c63b2e3881 tests: Add executils test
(cherry picked from commit fb87db0e1e)
2019-05-03 16:38:38 -07:00