The previous state of the patch did not work correctly when used with
negative salt lengths, which OpenSSL uses a magic values. Setting the
saltlength to max would yield an approved state in the indicator, while
it is not approved.
Additionally, update the patch to change the default PSS salt length
with the current state of discussion upstream (see
https://github.com/openssl/openssl/pull/19724).
Resolves: rhbz#2142087
Signed-off-by: Clemens Lang <cllang@redhat.com>
FIPS 186-4 section 5 "The RSA Digital Signature Algorithm", subsection
5.5 "PKCS #1" says: "For RSASSA-PSS […] the length (in bytes) of the
salt (sLen) shall satisfy 0 ≤ sLen ≤ hLen, where hLen is the length of
the hash function output block (in bytes)."
It is not exactly clear from this text whether hLen refers to the
message digest or the hash function used for the mask generation
function MGF1. PKCS#1 v2.1 suggests it is the former:
| Typical salt lengths in octets are hLen (the length of the output of
| the hash function Hash) and 0. In both cases the security of
| RSASSA-PSS can be closely related to the hardness of inverting RSAVP1.
| Bellare and Rogaway [4] give a tight lower bound for the security of
| the original RSA-PSS scheme, which corresponds roughly to the former
| case, while Coron [12] gives a lower bound for the related Full Domain
| Hashing scheme, which corresponds roughly to the latter case. In [13]
| Coron provides a general treatment with various salt lengths ranging
| from 0 to hLen; see [27] for discussion. See also [31], which adapts
| the security proofs in [4][13] to address the differences between the
| original and the present version of RSA-PSS as listed in Note 1 above.
Since OpenSSL defaults to creating signatures with the maximum salt
length, blocking the use of longer salts would probably lead to
significant problems in practice. Instead, introduce an explicit
indicator that can be obtained from the EVP_PKEY_CTX object using
EVP_PKEY_CTX_get_params() with the
OSSL_SIGNATURE_PARAM_REDHAT_FIPS_INDICATOR
parameter.
Change the default automatic behavior when signing to use at most the
digest size as salt length.
Signed-off-by: Clemens Lang <cllang@redhat.com>
Resolves: rhbz#2144012
