79 lines
2.7 KiB
Markdown
79 lines
2.7 KiB
Markdown
# README
|
|
|
|
The following test cases will be considered as a part of `ksc` gating.
|
|
|
|
## Installation issues
|
|
|
|
1. Check whether Module.symvers has been installed
|
|
1. Check whether kabi-stablelists package has been installed
|
|
1. Python Bugzilla interface has been installed
|
|
1. Expect error when nm is not found in $PATH.
|
|
|
|
## Valid uses of `ksc`
|
|
|
|
List of tests:
|
|
|
|
1. No external symbol required
|
|
1. Whitelisted symbol use only
|
|
1. Non-whitelisted symbol use only
|
|
1. Mixed whitelisted and non-whitelisted symbol use
|
|
1. Mixed whitelisted and non-whitelisted symbol use with multiple -k arguments
|
|
1. `ksc` called on 8139, crc32_generic, xor
|
|
|
|
Testing will be done as follows:
|
|
|
|
1. If applicable, try and build a kernel module usecase.
|
|
1. Pass kernel module(s) to ksc, record output.
|
|
1. Using the kabi-tools ksc parser [1], parse ksc output.
|
|
1. Using the nm tool, uname -i and kabi-whitelist, indepedently compose
|
|
the dictionary that ksc parser produces on ksc output.
|
|
1. Check for match.
|
|
|
|
Note that this series of tests checks both ksc valid functionality as well as
|
|
syntax requirements (indeed, should syntax of a ksc report be significantly
|
|
changed, ksc parser would produce different results).
|
|
|
|
[1] http://git.engineering.redhat.com/git/users/ckalina/kabi-greylists-devel.git/tree/src/greylists/ksc.py
|
|
|
|
## Submission tests
|
|
|
|
Partner RHBZ is used for this purpose.
|
|
|
|
1. All of the 'Valid uses of `ksc`' test cases (called w/ -k) will be re-tested
|
|
here as well to test immediate submission after generating.
|
|
1. Use any ksc-reports.txt to test submission onto partner RHBZ; in particular
|
|
test:
|
|
- failure on invalid product
|
|
- failure on invalid RHBZ credentials (username, password)
|
|
- failure on invalid RHBZ API key
|
|
- correct submission (using the gating-embedded bugzilla-cli tool to
|
|
download and check details of bug and attachments).
|
|
|
|
## Further tests include:
|
|
|
|
1. Check whether justification carry-over works within the same ko file.
|
|
1. Check whether justification carry-over does not happen when different ko
|
|
file is used.
|
|
|
|
## Invalid uses of `ksc`
|
|
|
|
1. Expect error when argument arity is not expected (e.g., $ ksc -k, w/o a ko
|
|
file provided)
|
|
1. Expect error when passing non-kernel module as a kernel module (-k)
|
|
1. Expect error when passing a file w/o read permissions (e.g., kernel module,
|
|
Module.symvers, ...)
|
|
1. Expect error when passing a non-existent file (e.g., kernel module, ...)
|
|
1. Expect error when passing a malformed kernel module.
|
|
1. Expect error when passing a malformed Module.symvers.
|
|
1. Expect error when passing a ksc-report.txt as a source of justifications.
|
|
|
|
## Compatibility Tests
|
|
|
|
1. Verify that no options were removed from -h/--help.
|
|
1. Verify that no options were removed from manpage.
|
|
|
|
## Test known bugs
|
|
|
|
1. EOFError when piping through ksc
|
|
|