1 Commits

Author	SHA1	Message	Date
Susant Sahani	151d610ce0	Add integration tests to iperf3 ... Justification Adds tests according to the CI wiki specifically the standard test interface in the spec. The playbook includes Tier1 level test cases that have been tested in the following contexts and is passing reliably: Classic. Test logs are stored in the artifacts directory. The following steps are used to execute the tests using the standard test interface: Test enveronment Make sure you have installed packages from the spec ``` ansible-2.4.1.0-2.fc27.noarch python2-dnf-2.7.5-1.fc27.noarch libselinux-python-2.7-2.fc27.x86_64 standard-test-roles-2.5-1.fc27.noarch Run tests for Classic Snip of the example test run for Classic tests: ``` ``` posix on :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: TEST PROTOCOL :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Package : iperf3 Installed : iperf3-3.5-1.fc29.x86_64 beakerlib RPM : beakerlib-1.17-13.fc29.noarch Test started : 2018-04-30 12:21:09 UTC Test finished : Test duration : Test name : miscellaneous-tests Distro : Fedora release 29 (Rawhide) Hostname : localhost Architecture : x86_64 CPUs : 1 x Westmere E56xx/L56xx/X56xx (Nehalem-C) RAM size : 5938 MB HDD size : 39.12 GB :: [ 12:21:09 ] :: [ WARNING ] :: POSIX mode detected and switched off :: [ 12:21:09 ] :: [ WARNING ] :: POSIX mode detected and switched off :: [ 12:21:09 ] :: [ WARNING ] :: Please fix your test to have /bin/bash shebang :: [ 12:21:09 ] :: [ WARNING ] :: Please fix your test to have /bin/bash shebang :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Setup :: Setup :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: iperf3-3.5-1.fc29.x86_64 :: [ 12:21:09 ] :: [ PASS ] :: Checking for the presence of iperf3 rpm :: [ 12:21:09 ] :: [ PASS ] :: Checking for the presence of iperf3 rpm :: [ 12:21:09 ] :: [ LOG ] :: Package versions: :: [ 12:21:09 ] :: [ LOG ] :: Package versions: :: [ 12:21:09 ] :: [ LOG ] :: iperf3-3.5-1.fc29.x86_64 :: [ 12:21:09 ] :: [ LOG ] :: iperf3-3.5-1.fc29.x86_64 :: [ 12:21:09 ] :: [ BEGIN ] :: Running 'cp iperf3-test.py /usr/bin/' :: [ 12:21:09 ] :: [ PASS ] :: Command 'cp iperf3-test.py /usr/bin/' (Expected 0, got 0) :: [ 12:21:09 ] :: [ PASS ] :: Command 'cp iperf3-test.py /usr/bin/' (Expected 0, got 0) :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 0s :: Duration: 0s :: Assertions: 2 good, 0 bad :: Assertions: 2 good, 0 bad :: RESULT: PASS :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Test :: Test :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:21:09 ] :: [ LOG ] :: iperf3 tests :: [ 12:21:09 ] :: [ LOG ] :: iperf3 tests :: [ 12:21:09 ] :: [ BEGIN ] :: Running '/usr/bin/python3 /usr/bin/iperf3-test.py' test_basic_tcp_client_default_port_5201 (__main__.Iperf3SanityTests) ... ok test_basic_udp_client_default_port_5201 (__main__.Iperf3SanityTests) ... ok test_server_daemon_pid_equal_to_pid_file (__main__.Iperf3SanityTests) ... ok test_server_log_file_exists (__main__.Iperf3SanityTests) ... ok test_server_pid_file_exists (__main__.Iperf3SanityTests) ... ok test_tcp_client_custom_port_5202 (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_custom_client_port (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_dscp (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_get_server_output (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_mss (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_parallel (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_reverse (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_time_15sec (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_title (__main__.Iperf3Tests) ... ok test_tcp_client_port_5202_ipv6 (__main__.Iperf3Tests) ... ok test_udp_client_custom_port_5202 (__main__.Iperf3Tests) ... ok test_udp_client_port_5202_ipv6 (__main__.Iperf3Tests) ... ok test_tcp_client_bind_to_interface_default_port (__main__.Iperf3TestsBindToInterface) ... ok ---------------------------------------------------------------------- Ran 18 tests in 266.543s OK :: [ 12:25:36 ] :: [ PASS ] :: Command '/usr/bin/python3 /usr/bin/iperf3-test.py' (Expected 0, got 0) :: [ 12:25:36 ] :: [ PASS ] :: Command '/usr/bin/python3 /usr/bin/iperf3-test.py' (Expected 0, got 0) :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 267s :: Duration: 267s :: Assertions: 1 good, 0 bad :: Assertions: 1 good, 0 bad :: RESULT: PASS :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Cleanup :: Cleanup :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:25:36 ] :: [ BEGIN ] :: Running 'rm /usr/bin/iperf3-test.py' :: [ 12:25:36 ] :: [ PASS ] :: Command 'rm /usr/bin/iperf3-test.py' (Expected 0, got 0) :: [ 12:25:36 ] :: [ PASS ] :: Command 'rm /usr/bin/iperf3-test.py' (Expected 0, got 0) :: [ 12:25:36 ] :: [ LOG ] :: iperf3 tests done :: [ 12:25:36 ] :: [ LOG ] :: iperf3 tests done :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 0s :: Duration: 0s :: Assertions: 1 good, 0 bad :: Assertions: 1 good, 0 bad :: RESULT: PASS :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: TEST PROTOCOL :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Package : iperf3 Installed : iperf3-3.5-1.fc29.x86_64 beakerlib RPM : beakerlib-1.17-13.fc29.noarch Test started : 2018-04-30 12:21:09 UTC Test finished : 2018-04-30 12:25:36 UTC (still running) Test duration : 267 seconds Test name : miscellaneous-tests Distro : Fedora release 29 (Rawhide) Hostname : localhost Architecture : x86_64 CPUs : 1 x Westmere E56xx/L56xx/X56xx (Nehalem-C) RAM size : 5938 MB HDD size : 39.12 GB :: [ 12:21:09 ] :: [ WARNING ] :: POSIX mode detected and switched off :: [ 12:21:09 ] :: [ WARNING ] :: Please fix your test to have /bin/bash shebang :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Setup :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:21:09 ] :: [ PASS ] :: Checking for the presence of iperf3 rpm :: [ 12:21:09 ] :: [ LOG ] :: Package versions: :: [ 12:21:09 ] :: [ LOG ] :: iperf3-3.5-1.fc29.x86_64 :: [ 12:21:09 ] :: [ PASS ] :: Command 'cp iperf3-test.py /usr/bin/' (Expected 0, got 0) :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 0s :: Assertions: 2 good, 0 bad :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Test :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:21:09 ] :: [ LOG ] :: iperf3 tests :: [ 12:25:36 ] :: [ PASS ] :: Command '/usr/bin/python3 /usr/bin/iperf3-test.py' (Expected 0, got 0) :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 267s :: Assertions: 1 good, 0 bad :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Cleanup :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:25:36 ] :: [ PASS ] :: Command 'rm /usr/bin/iperf3-test.py' (Expected 0, got 0) :: [ 12:25:36 ] :: [ LOG ] :: iperf3 tests done :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 0s :: Assertions: 1 good, 0 bad :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: miscellaneous-tests :: miscellaneous-tests :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:25:36 ] :: [ LOG ] :: JOURNAL XML: /var/tmp/beakerlib-oMWxgE1/journal.xml :: [ 12:25:36 ] :: [ LOG ] :: JOURNAL XML: /var/tmp/beakerlib-oMWxgE1/journal.xml :: [ 12:25:36 ] :: [ LOG ] :: JOURNAL TXT: /var/tmp/beakerlib-oMWxgE1/journal.txt :: [ 12:25:36 ] :: [ LOG ] :: JOURNAL TXT: /var/tmp/beakerlib-oMWxgE1/journal.txt :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 267s :: Duration: 267s :: Phases: 3 good, 0 bad :: Phases: 3 good, 0 bad :: OVERALL RESULT: PASS :: OVERALL RESULT: PASS posix on :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: TEST PROTOCOL :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Package : iperf3 Installed : iperf3-3.5-1.fc29.x86_64 beakerlib RPM : beakerlib-1.17-13.fc29.noarch Test started : 2018-04-30 12:21:09 UTC Test finished : Test duration : Test name : miscellaneous-tests Distro : Fedora release 29 (Rawhide) Hostname : localhost Architecture : x86_64 CPUs : 1 x Westmere E56xx/L56xx/X56xx (Nehalem-C) RAM size : 5938 MB HDD size : 39.12 GB :: [ 12:21:09 ] :: [ WARNING ] :: POSIX mode detected and switched off :: [ 12:21:09 ] :: [ WARNING ] :: POSIX mode detected and switched off :: [ 12:21:09 ] :: [ WARNING ] :: Please fix your test to have /bin/bash shebang :: [ 12:21:09 ] :: [ WARNING ] :: Please fix your test to have /bin/bash shebang :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Setup :: Setup :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: iperf3-3.5-1.fc29.x86_64 :: [ 12:21:09 ] :: [ PASS ] :: Checking for the presence of iperf3 rpm :: [ 12:21:09 ] :: [ PASS ] :: Checking for the presence of iperf3 rpm :: [ 12:21:09 ] :: [ LOG ] :: Package versions: :: [ 12:21:09 ] :: [ LOG ] :: Package versions: :: [ 12:21:09 ] :: [ LOG ] :: iperf3-3.5-1.fc29.x86_64 :: [ 12:21:09 ] :: [ LOG ] :: iperf3-3.5-1.fc29.x86_64 :: [ 12:21:09 ] :: [ BEGIN ] :: Running 'cp iperf3-test.py /usr/bin/' :: [ 12:21:09 ] :: [ PASS ] :: Command 'cp iperf3-test.py /usr/bin/' (Expected 0, got 0) :: [ 12:21:09 ] :: [ PASS ] :: Command 'cp iperf3-test.py /usr/bin/' (Expected 0, got 0) :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 0s :: Duration: 0s :: Assertions: 2 good, 0 bad :: Assertions: 2 good, 0 bad :: RESULT: PASS :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Test :: Test :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:21:09 ] :: [ LOG ] :: iperf3 tests :: [ 12:21:09 ] :: [ LOG ] :: iperf3 tests :: [ 12:21:09 ] :: [ BEGIN ] :: Running '/usr/bin/python3 /usr/bin/iperf3-test.py' test_basic_tcp_client_default_port_5201 (__main__.Iperf3SanityTests) ... ok test_basic_udp_client_default_port_5201 (__main__.Iperf3SanityTests) ... ok test_server_daemon_pid_equal_to_pid_file (__main__.Iperf3SanityTests) ... ok test_server_log_file_exists (__main__.Iperf3SanityTests) ... ok test_server_pid_file_exists (__main__.Iperf3SanityTests) ... ok test_tcp_client_custom_port_5202 (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_custom_client_port (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_dscp (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_get_server_output (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_mss (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_parallel (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_reverse (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_time_15sec (__main__.Iperf3Tests) ... ok test_tcp_client_custom_port_5202_title (__main__.Iperf3Tests) ... ok test_tcp_client_port_5202_ipv6 (__main__.Iperf3Tests) ... ok test_udp_client_custom_port_5202 (__main__.Iperf3Tests) ... ok test_udp_client_port_5202_ipv6 (__main__.Iperf3Tests) ... ok test_tcp_client_bind_to_interface_default_port (__main__.Iperf3TestsBindToInterface) ... ok ---------------------------------------------------------------------- Ran 18 tests in 266.543s OK :: [ 12:25:36 ] :: [ PASS ] :: Command '/usr/bin/python3 /usr/bin/iperf3-test.py' (Expected 0, got 0) :: [ 12:25:36 ] :: [ PASS ] :: Command '/usr/bin/python3 /usr/bin/iperf3-test.py' (Expected 0, got 0) :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 267s :: Duration: 267s :: Assertions: 1 good, 0 bad :: Assertions: 1 good, 0 bad :: RESULT: PASS :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Cleanup :: Cleanup :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:25:36 ] :: [ BEGIN ] :: Running 'rm /usr/bin/iperf3-test.py' :: [ 12:25:36 ] :: [ PASS ] :: Command 'rm /usr/bin/iperf3-test.py' (Expected 0, got 0) :: [ 12:25:36 ] :: [ PASS ] :: Command 'rm /usr/bin/iperf3-test.py' (Expected 0, got 0) :: [ 12:25:36 ] :: [ LOG ] :: iperf3 tests done :: [ 12:25:36 ] :: [ LOG ] :: iperf3 tests done :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 0s :: Duration: 0s :: Assertions: 1 good, 0 bad :: Assertions: 1 good, 0 bad :: RESULT: PASS :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: TEST PROTOCOL :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Package : iperf3 Installed : iperf3-3.5-1.fc29.x86_64 beakerlib RPM : beakerlib-1.17-13.fc29.noarch Test started : 2018-04-30 12:21:09 UTC Test finished : 2018-04-30 12:25:36 UTC (still running) Test duration : 267 seconds Test name : miscellaneous-tests Distro : Fedora release 29 (Rawhide) Hostname : localhost Architecture : x86_64 CPUs : 1 x Westmere E56xx/L56xx/X56xx (Nehalem-C) RAM size : 5938 MB HDD size : 39.12 GB :: [ 12:21:09 ] :: [ WARNING ] :: POSIX mode detected and switched off :: [ 12:21:09 ] :: [ WARNING ] :: Please fix your test to have /bin/bash shebang :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Setup :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:21:09 ] :: [ PASS ] :: Checking for the presence of iperf3 rpm :: [ 12:21:09 ] :: [ LOG ] :: Package versions: :: [ 12:21:09 ] :: [ LOG ] :: iperf3-3.5-1.fc29.x86_64 :: [ 12:21:09 ] :: [ PASS ] :: Command 'cp iperf3-test.py /usr/bin/' (Expected 0, got 0) :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 0s :: Assertions: 2 good, 0 bad :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Test :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:21:09 ] :: [ LOG ] :: iperf3 tests :: [ 12:25:36 ] :: [ PASS ] :: Command '/usr/bin/python3 /usr/bin/iperf3-test.py' (Expected 0, got 0) :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 267s :: Assertions: 1 good, 0 bad :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Cleanup :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:25:36 ] :: [ PASS ] :: Command 'rm /usr/bin/iperf3-test.py' (Expected 0, got 0) :: [ 12:25:36 ] :: [ LOG ] :: iperf3 tests done :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 0s :: Assertions: 1 good, 0 bad :: RESULT: PASS :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: miscellaneous-tests :: miscellaneous-tests :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: [ 12:25:36 ] :: [ LOG ] :: JOURNAL XML: /var/tmp/beakerlib-oMWxgE1/journal.xml :: [ 12:25:36 ] :: [ LOG ] :: JOURNAL XML: /var/tmp/beakerlib-oMWxgE1/journal.xml :: [ 12:25:36 ] :: [ LOG ] :: JOURNAL TXT: /var/tmp/beakerlib-oMWxgE1/journal.txt :: [ 12:25:36 ] :: [ LOG ] :: JOURNAL TXT: /var/tmp/beakerlib-oMWxgE1/journal.txt :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :: Duration: 267s :: Duration: 267s :: Phases: 3 good, 0 bad :: Phases: 3 good, 0 bad :: OVERALL RESULT: PASS :: OVERALL RESULT: PASS ``` Tested by : Bruno Goncalves <bgoncalv@redhat.com>	2018-04-30 19:04:07 +05:30