Add tests according to the CI

justification
Adds tests according to the CI wiki specifically the standard test interface in the spec.
The playbook includes Tier1 level test cases that have been tested in the following contexts and
is passing reliably: Classic. Test logs are stored in the artifacts directory.
The following steps are used to execute the tests using the standard test interface:

Test enveronment
Make sure you have installed packages from the spec
```
ansible-2.4.1.0-2.fc28.noarch
python2-dnf-2.7.5-1.fc28.noarch
libselinux-python-2.7-2.fc28.x86_64
standard-test-roles-2.5-1.fc28.noarch
Run tests for Classic
Snip of the example test run for Classic tests:
```

```
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Setup
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

lldpad-1.0.1-9.git036e314.fc28.x86_64
:: [ 09:17:11 ] :: [   PASS   ] :: Checking for the presence of lldpad rpm
:: [ 09:17:11 ] :: [   LOG    ] :: Package versions:
:: [ 09:17:11 ] :: [   LOG    ] ::   lldpad-1.0.1-9.git036e314.fc28.x86_64
:: [ 09:17:11 ] :: [  BEGIN   ] :: Running 'systemctl stop firewalld'
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'systemctl stop firewalld' (Expected 0,5, got 0)
:: [ 09:17:11 ] :: [  BEGIN   ] :: Running 'setenforce 0'
setenforce: SELinux is disabled
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'setenforce 0' (Expected 0,1, got 1)
:: [ 09:17:11 ] :: [  BEGIN   ] :: Running 'mkdir -p /run/systemd/network'
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'mkdir -p /run/systemd/network' (Expected 0, got 0)
:: [ 09:17:11 ] :: [  BEGIN   ] :: Running 'cp lldp.network /run/systemd/network/'
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'cp lldp.network /run/systemd/network/' (Expected 0, got 0)
:: [ 09:17:11 ] :: [  BEGIN   ] :: Running 'cp tcpdumpd.service /run/systemd/system'
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'cp tcpdumpd.service /run/systemd/system' (Expected 0, got 0)
:: [ 09:17:11 ] :: [  BEGIN   ] :: Running 'systemctl daemon-reload'
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'systemctl daemon-reload' (Expected 0, got 0)
:: [ 09:17:11 ] :: [  BEGIN   ] :: Running 'cp lldpad-test.py /usr/bin/'
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'cp lldpad-test.py /usr/bin/' (Expected 0, got 0)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 0s
::   Assertions: 8 good, 0 bad
::   RESULT: PASS

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Test
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 09:17:11 ] :: [   LOG    ] :: lldpad tests
:: [ 09:17:11 ] :: [  BEGIN   ] :: Running '/usr/bin/python3 /usr/bin/lldpad-test.py'
test_lldpad_configured (__main__.lldpadTests)
Verify lldpad-peer interface configured ... ok
test_lldpad_trasmitted_packets (__main__.lldpadTests)
verify at the other end of veth(lldpad) lldpad has trasmitted packets. Extract from tcpdump ... reading from file /tmp/lldpad-tcp-dump.pcap, link-type EN10MB (Ethernet)
ok
test_lldptool_get_tlv (__main__.lldpadTests)
Verify lldpad got the packet transmitted from lldpad-peer interface ... ok
test_management_address (__main__.lldpadTests)
Test Get/Set a Management Address ... ok
test_systemd_networkd_lldp (__main__.lldpadTestsViaNetworkd)
Receive LLDP packets via networkd ... ok

----------------------------------------------------------------------
Ran 5 tests in 99.310s

OK
:: [ 09:18:50 ] :: [   PASS   ] :: Command '/usr/bin/python3 /usr/bin/lldpad-test.py' (Expected 0, got 0)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 99s
::   Assertions: 1 good, 0 bad
::   RESULT: PASS

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Cleanup
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 09:18:51 ] :: [  BEGIN   ] :: Running 'rm /usr/bin/lldpad-test.py /run/systemd/network/lldp.network /run/systemd/system/tcpdumpd.service'
:: [ 09:18:51 ] :: [   PASS   ] :: Command 'rm /usr/bin/lldpad-test.py /run/systemd/network/lldp.network /run/systemd/system/tcpdumpd.service' (Expected 0, got 0)
:: [ 09:18:51 ] :: [  BEGIN   ] :: Running 'systemctl daemon-reload'
:: [ 09:18:51 ] :: [   PASS   ] :: Command 'systemctl daemon-reload' (Expected 0, got 0)
:: [ 09:18:51 ] :: [  BEGIN   ] :: Running 'setenforce 1'
setenforce: SELinux is disabled
:: [ 09:18:51 ] :: [   PASS   ] :: Command 'setenforce 1' (Expected 0,1, got 1)
:: [ 09:18:51 ] :: [   LOG    ] :: lldpad tests done
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 0s
::   Assertions: 3 good, 0 bad
::   RESULT: PASS

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   TEST PROTOCOL
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

    Package       : lldpad
    Installed     : lldpad-1.0.1-9.git036e314.fc28.x86_64
    beakerlib RPM : beakerlib-1.17-13.fc28.noarch
    Test started  : 2018-06-10 09:17:10 IST
    Test finished : 2018-06-10 09:18:51 IST (still running)
    Test duration : 101 seconds
    Test name     : unknown
    Distro        : Fedora release 28 (Twenty Eight)
    Hostname      : Zeus
    Architecture  : x86_64
    CPUs          : 8 x Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz
    RAM size      : 15731 MB
    HDD size      : 226.83 GB

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Setup
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 09:17:11 ] :: [   PASS   ] :: Checking for the presence of lldpad rpm
:: [ 09:17:11 ] :: [   LOG    ] :: Package versions:
:: [ 09:17:11 ] :: [   LOG    ] ::   lldpad-1.0.1-9.git036e314.fc28.x86_64
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'systemctl stop firewalld' (Expected 0,5, got 0)
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'setenforce 0' (Expected 0,1, got 1)
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'mkdir -p /run/systemd/network' (Expected 0, got 0)
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'cp lldp.network /run/systemd/network/' (Expected 0, got 0)
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'cp tcpdumpd.service /run/systemd/system' (Expected 0, got 0)
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'systemctl daemon-reload' (Expected 0, got 0)
:: [ 09:17:11 ] :: [   PASS   ] :: Command 'cp lldpad-test.py /usr/bin/' (Expected 0, got 0)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 0s
::   Assertions: 8 good, 0 bad
::   RESULT: PASS

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Test
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 09:17:11 ] :: [   LOG    ] :: lldpad tests
:: [ 09:18:50 ] :: [   PASS   ] :: Command '/usr/bin/python3 /usr/bin/lldpad-test.py' (Expected 0, got 0)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 99s
::   Assertions: 1 good, 0 bad
::   RESULT: PASS

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Cleanup
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 09:18:51 ] :: [   PASS   ] :: Command 'rm /usr/bin/lldpad-test.py /run/systemd/network/lldp.network /run/systemd/system/tcpdumpd.service' (Expected 0, got 0)
:: [ 09:18:51 ] :: [   PASS   ] :: Command 'systemctl daemon-reload' (Expected 0, got 0)
:: [ 09:18:51 ] :: [   PASS   ] :: Command 'setenforce 1' (Expected 0,1, got 1)
:: [ 09:18:51 ] :: [   LOG    ] :: lldpad tests done
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 0s
::   Assertions: 3 good, 0 bad
::   RESULT: PASS

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   unknown
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 09:18:51 ] :: [   LOG    ] :: JOURNAL XML: /var/tmp/beakerlib-RRyct40/journal.xml
:: [ 09:18:51 ] :: [   LOG    ] :: JOURNAL TXT: /var/tmp/beakerlib-RRyct40/journal.txt
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Duration: 101s
::   Phases: 3 good, 0 bad
::   OVERALL RESULT: PASS
```
This commit is contained in:
Susant Sahani 2018-05-19 20:02:24 +05:30
parent 14833b505a
commit 451044af05
7 changed files with 317 additions and 0 deletions

View File

@ -0,0 +1,36 @@
# SPDX-License-Identifier: LGPL-2.1+
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#
# Makefile of /CoreOS/lldpad
# Description: Test lldpad working ok
# Author: Susant Sahani<susant@redhat.com>
#
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
export TEST=/CoreOS/lldpad
export TESTVERSION=1.0
BUILT_FILES=
FILES=$(METADATA) runtest.sh Makefile PURPOSE
.PHONY: all install download clean
run: $(FILES) build
./runtest.sh
build: $(BUILT_FILES)
test -x runtest.sh || chmod a+x runtest.sh
clean:
rm -f *~ $(BUILT_FILES)
include /usr/share/rhts/lib/rhts-make.include
$(METADATA): Makefile
@echo "Owner: Susant Sahani<susant@redhat.com>" > $(METADATA)
@echo "Name: $(TEST)" >> $(METADATA)
@echo "TestVersion: $(TESTVERSION)" >> $(METADATA)
@echo "Path: $(TEST_DIR)" >> $(METADATA)
@echo "Description: Test if the ABI hasn't changed" >> $(METADATA)
@echo "Type: Sanity" >> $(METADATA)
@echo "TestTime: 5m" >> $(METADATA)
@echo "RunFor: lldpad" >> $(METADATA)
@echo "Requires: lldpad" >> $(METADATA)
@echo "Priority: Normal" >> $(METADATA)
@echo "License: GPLv2" >> $(METADATA)
@echo "Confidential: no" >> $(METADATA)
@echo "Destructive: no" >> $(METADATA)
@echo "Releases: -Fedora 28" >> $(METADATA)
rhts-lint $(METADATA)

View File

@ -0,0 +1,3 @@
PURPOSE of /CoreOS/lldpad
Description: tests for lldpad
Author: Susant Sahani<susant@redhat.com>

View File

@ -0,0 +1,8 @@
[Match]
Name=lldpad
[Network]
DHCP=no
IPv6AcceptRA=false
LLDP=yes
EmitLLDP=yes

View File

@ -0,0 +1,197 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: LGPL-2.1+
# ~~~
# lldpad-test.py integration test
# Description: Test for lldpad: Link Layer Discovery Protocol (LLDP) agent daemon
#
# Author: Susant Sahani <susant@redhat.com>
# Copyright (c) 2018 Red Hat, Inc.
# ~~~
import errno
import os
import sys
import time
import unittest
import subprocess
import signal
import shutil
import psutil
from pyroute2 import IPRoute
LLDPAD_TCP_DUMP_FILE='/tmp/lldpad-tcp-dump.pcap'
def setUpModule():
"""Initialize the environment, and perform sanity checks on it."""
if shutil.which('lldpad') is None:
raise OSError(errno.ENOENT, 'lldpad not found')
class lldpadUtilities():
"""Provide a set of utility functions start stop lldpad ."""
def Startlldpad(self):
"""Start lldpad"""
subprocess.check_output(['systemctl', 'start', 'lldpad'])
self.addCleanup(subprocess.call, ['systemctl', 'stop', 'lldpad'])
def ConfigureLldpad(self):
subprocess.check_output(['lldptool', '-L', '-i', 'lldpad-peer', 'adminStatus=rxtx'])
subprocess.check_output(['lldptool', '-L', '-i', 'lldpad', 'adminStatus=rx'])
def CaptureLLDPPackets(self):
"""Start tcpdump to capture packets"""
subprocess.check_output(['systemctl','start', 'tcpdumpd.service'])
self.addCleanup(subprocess.call, ['rm', LLDPAD_TCP_DUMP_FILE])
def StopCaptureLLDPPackets(self):
subprocess.check_output(['systemctl', 'stop', 'tcpdumpd.service'])
def SetupVethInterface(self):
"""Setup veth interface"""
ip = IPRoute()
ip.link('add', ifname='lldpad', peer='lldpad-peer', kind='veth')
idx_ladvd= ip.link_lookup(ifname='lldpad')[0]
idx_ladvd_peer = ip.link_lookup(ifname='lldpad-peer')[0]
ip.link('set', index=idx_ladvd, address='02:01:02:03:04:08')
ip.link('set', index=idx_ladvd_peer, address='02:01:02:03:04:09')
ip.link('set', index=idx_ladvd, state='up')
ip.link('set', index=idx_ladvd_peer, state='up')
ip.close()
time.sleep(4)
def TearDownVethInterface(self):
ip = IPRoute()
ip.link('del', index=ip.link_lookup(ifname='lldpad')[0])
ip.close()
def FindLLDPFieldsinTCPDump(self, **kwargs):
"""Look attributes in lldpad logs."""
contents = subprocess.check_output(['tcpdump', '-v', '-r', LLDPAD_TCP_DUMP_FILE]).rstrip().decode('utf-8')
if kwargs is not None:
for key in kwargs:
self.assertRegex(contents, kwargs[key])
class lldpadTestsViaNetworkd(unittest.TestCase, lldpadUtilities):
def setUp(self):
""" Setup """
self.SetupVethInterface()
self.Startlldpad()
self.ConfigureLldpad()
def tearDown(self):
self.TearDownVethInterface()
def test_systemd_networkd_lldp(self):
""" Receive LLDP packets via networkd """
subprocess.check_output(['systemctl', 'restart', 'systemd-networkd'])
time.sleep(30)
output=subprocess.check_output(['networkctl','lldp', 'lldpad']).rstrip().decode('utf-8')
self.assertRegex(output, "lldpad")
self.assertRegex(output, "02:01:02:03:04:09")
""" Verify LLDP Packets received by lldpad transmitted from networkd """
output = subprocess.check_output(['lldptool', 'get-tlv', '-n', '-i', 'lldpad-peer']).rstrip().decode('utf-8')
self.assertRegex(output, "Ifname: lldpad")
self.assertRegex(output, "120")
class lldpadTests(unittest.TestCase, lldpadUtilities):
def setUp(self):
""" Setup """
self.SetupVethInterface()
self.Startlldpad()
self.ConfigureLldpad()
""" TTL is 120 """
self.CaptureLLDPPackets()
time.sleep(10)
self.StopCaptureLLDPPackets()
time.sleep(1)
def tearDown(self):
self.TearDownVethInterface()
def test_lldpad_configured(self):
""" Verify lldpad-peer interface configured """
# Chassis ID TLV
# MAC: 02:01:02:03:04:09
# Port ID TLV
# MAC: 02:01:02:03:04:09
# Time to Live TLV
# 120
# End of LLDPDU TLV
time.sleep(2)
output = subprocess.check_output(['lldptool', '-t', '-i', 'lldpad-peer']).rstrip().decode('utf-8')
self.assertRegex(output, "Chassis ID TLV")
self.assertRegex(output, "MAC: 02:01:02:03:04:09")
self.assertRegex(output, "Port ID TLV")
self.assertRegex(output, "MAC: 02:01:02:03:04:09")
self.assertRegex(output, "Time to Live TLV")
self.assertRegex(output, "120")
self.assertRegex(output, "End of LLDPDU TLV")
def test_lldptool_get_tlv(self):
""" Verify lldpad got the packet transmitted from lldpad-peer interface """
# Chassis ID TLV
# MAC: 02:01:02:03:04:09
# Port ID TLV
# MAC: 02:01:02:03:04:09
# Time to Live TLV
# 120
# End of LLDPDU TLV
time.sleep(2)
output = subprocess.check_output(['lldptool', 'get-tlv', '-n', '-i', 'lldpad']).rstrip().decode('utf-8')
self.assertRegex(output, "Chassis ID TLV")
self.assertRegex(output, "MAC: 02:01:02:03:04:09")
self.assertRegex(output, "Port ID TLV")
self.assertRegex(output, "MAC: 02:01:02:03:04:09")
self.assertRegex(output, "Time to Live TLV")
self.assertRegex(output, "120")
self.assertRegex(output, "End of LLDPDU TLV")
def test_management_address(self):
""" Test Get/Set a Management Address"""
subprocess.check_output(['lldptool', '-T', '-i', 'lldpad', '-V', 'mngAddr', 'ipv4=192.168.10.10'])
subprocess.check_output(['lldptool', '-T', '-i', 'lldpad', '-V', 'mngAddr', 'ipv6=::1'])
output=subprocess.check_output(['lldptool', '-t', '-i', 'lldpad', '-V', 'mngAddr', '-c', 'ipv4']).rstrip().decode('utf-8')
self.assertRegex(output, "ipv4=192.168.10.10");
output=subprocess.check_output(['lldptool', '-t', '-i', 'lldpad', '-V', 'mngAddr', '-c', 'ipv6']).rstrip().decode('utf-8')
self.assertRegex(output, "ipv6=::1");
def test_lldpad_trasmitted_packets(self):
""" verify at the other end of veth(lldpad) lldpad has trasmitted packets. Extract from tcpdump"""
# 16:46:06.007162 LLDP, length 46
# Chassis ID TLV (1), length 7
# Subtype MAC address (4): 02:01:02:03:04:09 (oui Unknown)
# Port ID TLV (2), length 7
# Subtype MAC address (3): 02:01:02:03:04:09 (oui Unknown)
# Time to Live TLV (3), length 2: TTL 120s
# End TLV (0), length 0
self.FindLLDPFieldsinTCPDump(test2='02:01:02:03:04:09',
test3='TTL.*120s')
if __name__ == '__main__':
unittest.main(testRunner=unittest.TextTestRunner(stream=sys.stdout,
verbosity=2))

View File

@ -0,0 +1,49 @@
#!/bin/bash
# SPDX-License-Identifier: LGPL-2.1+
# ~~~
# runtest.sh of lldpad
# Description: Tests for The lldpad package provides the Linux user space
# daemon and configuration tool for Intel's Link Layer Discovery Protocol (LLDP)
# agent with Enhanced Ethernet support.
# Author: Susant Sahani <susant@redhat.com>
# Copyright (c) 2018 Red Hat, Inc.
# ~~~~
# Include Beaker environment
. /usr/share/beakerlib/beakerlib.sh || exit 1
PACKAGE="lldpad"
SERVICE_UNITDIR="/run/systemd/system"
NETWORK_UNITDIR="/run/systemd/network"
rlJournalStart
rlPhaseStartSetup
rlAssertRpm $PACKAGE
rlRun "systemctl stop firewalld" 0,5
rlRun "setenforce 0" 0,1
rlRun "mkdir -p $NETWORK_UNITDIR"
rlRun "cp lldp.network $NETWORK_UNITDIR/"
rlRun "cp tcpdumpd.service $SERVICE_UNITDIR"
rlRun "systemctl daemon-reload"
rlRun "cp lldpad-test.py /usr/bin/"
rlPhaseEnd
rlPhaseStartTest
rlLog "lldpad tests"
rlRun "/usr/bin/python3 /usr/bin/lldpad-test.py"
rlPhaseEnd
rlPhaseStartCleanup
rlRun "rm /usr/bin/lldpad-test.py $NETWORK_UNITDIR/lldp.network $SERVICE_UNITDIR/tcpdumpd.service"
rlRun "systemctl daemon-reload"
rlRun "setenforce 1" 0,1
rlLog "lldpad tests done"
rlPhaseEnd
rlJournalPrintText
rlJournalEnd
rlGetTestState

View File

@ -0,0 +1,10 @@
[Unit]
Description=TCPDumpd
After=multi-user.target network.target
[Service]
Type=simple
ExecStart=/usr/sbin/tcpdump -pnnli lldpad ether proto 0x88cc -vvv -w "/tmp/lldpad-tcp-dump.pcap"
[Install]
WantedBy=multi-user.target

14
tests/tests.yml Normal file
View File

@ -0,0 +1,14 @@
- hosts: localhost
roles:
- role: standard-test-beakerlib
tags:
- classic
tests:
- miscellaneous-tests
required_packages:
- lldpad
- python3
- tcpdump
- systemd
- iproute
- python3-pyroute2