Compared to the previous implementation based on OpenSSL, the new implementation
has these advantages:
1. Strict and reliable detection of invalid strings, including strings with
less than 4 bytes of garbage at the end;
2. Allows to use Base64 strings without '=' padding.
When a client sends no SNI is a common situation. But currently the server
processes it as an error and returns SSL_TLSEXT_ERR_ALERT_FATAL causing
termination of a current TLS session. The problem occurs if configuration has
more than one certificate bundle in a listener.
This fix changes the return code to SSL_TLSEXT_ERR_OK and the log level of a
message.
To support TLS sessions, Unit uses the OpenSSL built-in session cache; the
cache_size option defines the number sessions to store. To disable the feather,
the option must be zero.
To perform various configuration operations on SSL_CTX, OpenSSL provides
SSL_CONF_cmd(). Specifically, to configure ciphers for a listener,
"CipherString" and "Ciphersuites" file commands are used:
https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html
This feature can be configured in the "tls/conf_commands" section.
A crash was caused by an incorrect timer handler nxt_h1p_idle_timeout() if
SSL_shutdown() returned SSL_ERROR_WANT_READ/SSL_ERROR_WANT_WRITE.
The flag SSL_RECEIVED_SHUTDOWN is used to avoid getting SSL_ERROR_WANT_READ, so
the server won't wait for a close notification from a client.
For SSL_ERROR_WANT_WRITE, a correct timer handler is set up.
The certificate is selected by matching the arriving SNI to the common name and
the alternatives names. If no certificate matches the name, the first bundle in
the array is chosen.
This is a workaround for an issue in OpenSSL 1.1.1, where the /dev/random and
/dev/urandom files remain open after all listening sockets were removed:
- https://github.com/openssl/openssl/issues/7419
An immediate return statement on connection errors was mistakenly added to the
beginning of nxt_openssl_conn_io_shutdown() in ecd3c5bbf7d8, breaking the TLS
connection finalization procedure. As a result, a TLS connection was left
unfinalized if it had been closed prematurely or a fatal protocol error had
occurred, which caused memory and socket descriptor leakage.
Moreover, in some cases (notably, on handshake errors in tests with kqueue on
macOS) the read event was triggered later and nxt_h1p_conn_error() was called
the second time; after the change in af93c866b4f0, the latter call crashed the
router process in an attempt to remove a connection from the idle queue twice.
After event is delivered from the kernel its further processing is blocked.
Non-ready TSL I/O operation should mark connection I/O state as not ready
to unblock events and to allow their further processing. Otherwise
the connection hangs.
