RUSTSEC-2021-0129

Invalid handling of `X509_verify_cert()` internal errors in libssl

Internally libssl in OpenSSL calls `X509_verify_cert()` on the client side to verify a certificate supplied by a server. That function may return a negative return value to indicate an internal error (for example out of memory). Such a negative return value is mishandled by OpenSSL and will cause an IO function (such as `SSL_connect()` or `SSL_do_handshake()`) to not indicate success and a subsequent call to `SSL_get_error()` to return the value `SSL_ERROR_WANT_RETRY_VERIFY`. This return value is only supposed to be returned by OpenSSL if the application has previously called `SSL_CTX_set_cert_verify_callback()`. Since most applications do not do this the `SSL_ERROR_WANT_RETRY_VERIFY` return value from `SSL_get_error()` will be totally unexpected and applications may not behave correctly as a result. The exact behaviour will depend on the application but it could result in crashes, infinite loops or other similar incorrect responses. This issue is made more serious in combination with a separate bug in OpenSSL 3.0 that will cause `X509_verify_cert()` to indicate an internal error when processing a certificate chain. This will occur where a certificate does not include the Subject Alternative Name extension but where a Certificate Authority has enforced name constraints. This issue can occur even with valid chains.