ksc/tests/sanity/README.md

README

The following test cases will be considered as a part of ksc gating.

Installation issues

1. Check whether Module.symvers has been installed
2. Check whether kabi-whitelists package has been installed
3. Python Bugzilla interface has been installed
4. Expect error when nm is not found in $PATH.

Valid uses of ksc

List of tests:

1. No external symbol required
2. Whitelisted symbol use only
3. Non-whitelisted symbol use only
4. Mixed whitelisted and non-whitelisted symbol use
5. Mixed whitelisted and non-whitelisted symbol use with multiple -k arguments
6. ksc called on 8139, crc32_generic, xor

Testing will be done as follows:

1. If applicable, try and build a kernel module usecase.
2. Pass kernel module(s) to ksc, record output.
3. Using the kabi-tools ksc parser [1], parse ksc output.
4. Using the nm tool, uname -i and kabi-whitelist, indepedently compose the dictionary that ksc parser produces on ksc output.
5. Check for match.

Note that this series of tests checks both ksc valid functionality as well as syntax requirements (indeed, should syntax of a ksc report be significantly changed, ksc parser would produce different results).

[1] http://git.engineering.redhat.com/git/users/ckalina/kabi-greylists-devel.git/tree/src/greylists/ksc.py

Submission tests

Partner RHBZ is used for this purpose.

1. All of the 'Valid uses of ksc' test cases (called w/ -k) will be re-tested here as well to test immediate submission after generating.
2. Use any ksc-reports.txt to test submission onto partner RHBZ; in particular test:
   • failure on invalid product
   • failure on invalid RHBZ credentials (username, password)
   • failure on invalid RHBZ API key
   • correct submission (using the gating-embedded bugzilla-cli tool to download and check details of bug and attachments).

Further tests include:

1. Check whether justification carry-over works within the same ko file.
2. Check whether justification carry-over does not happen when different ko file is used.

Invalid uses of ksc

1. Expect error when argument arity is not expected (e.g., $ ksc -k, w/o a ko file provided)
2. Expect error when passing non-kernel module as a kernel module (-k)
3. Expect error when passing a file w/o read permissions (e.g., kernel module, Module.symvers, ...)
4. Expect error when passing a non-existent file (e.g., kernel module, ...)
5. Expect error when passing a malformed kernel module.
6. Expect error when passing a malformed Module.symvers.
7. Expect error when passing a ksc-report.txt as a source of justifications.

Compatibility Tests

1. Verify that no options were removed from -h/--help.
2. Verify that no options were removed from manpage.

Test known bugs

1. EOFError when piping through ksc