This is the first patch in adding WebAssembly language module support.
This just adds a new NXT_APP_WASM type, required by subsequent commits.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This makes it much more clear what's what.
This is in preparation for adding WebAssembly language module support.
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
Normally Unit responds to HTTP requests by including a header like
Server: Unit/1.30.0
however it can sometimes be beneficial to withhold the version
information and in this case just respond with
Server: Unit
This patch adds a new "settings.http" boolean option called
server_version, which defaults to true, in which case the full version
information is sent. However this can be set to false, e.g
"settings": {
"http": {
"server_version": false
}
},
in which case Unit responds without the version information as the
latter example above shows.
Link: <https://www.ietf.org/rfc/rfc9110.html#section-10.2.4>
Closes: <https://github.com/nginx/unit/issues/158>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
- Configuration: added "/config/settings/http/log_route".
Type: bool
Default: false
This adds configurability to the error log. It allows enabling and
disabling logs related to how the router performs selection of the
routes.
- HTTP: logging request line.
Log level: [notice]
The request line is essential to understand which logs correspond to
which request when reading the logs.
- HTTP: logging route that's been discarded.
Log level: [info]
- HTTP: logging route whose action is selected.
Log level: [notice]
- HTTP: logging when "fallback" action is taken.
Log level: [notice]
Closes: <https://github.com/nginx/unit/issues/758>
Link: <https://github.com/nginx/unit/pull/824>
Link: <https://github.com/nginx/unit/pull/839>
Suggested-by: Timo Stark <t.stark@nginx.com>
Suggested-by: Mark L Wood-Patrick <mwoodpatrick@gmail.com>
Suggested-by: Liam Crilly <liam@nginx.com>
Tested-by: Liam Crilly <liam@nginx.com>
Acked-by: Artem Konev <a.konev@f5.com>
Cc: Andrew Clayton <a.clayton@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Reviewed-by: Zhidao Hong <z.hong@f5.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Currently when using Unix domain sockets for requests, if unit is
reconfigured then it will fail if it tries to bind(2) again to a Unix
domain socket with something like
2023/02/25 19:15:50 [alert] 35274#35274 bind(\"unix:/tmp/unit.sock\") failed (98: Address already in use)
When closing such a socket we really need to unlink(2) it. However that
presents a problem in that when running as root, while the main process
runs as root and creates the socket, it's the router process, that runs
as an unprivileged user, e.g nobody, that closes the socket and would
thus remove it, but couldn't due to not having permission, even if the
socket is mode 0666, you need write permissions on the containing
directory to remove a file.
There are several options to solve this, all with varying degrees of
complexity and utility.
1) Give the user who the router process runs as write permission to
the directory containing the listen sockets. These can then be
unlink(2)'d from the router process.
Simple and would work, but perhaps not the most elegant.
2) Using capabilities(7). The router process could temporarily attain
the CAP_DAC_OVERRIDE capability, unlink(7) the socket, then
relinquish the capability until required again.
These are Linux specific (other systems may have similar mechanisms
which would be extra work to support). There is also a, albeit
small, window where the router process is running with elevated
privileges.
3) Have the main process do the unlink(2), it is after all the process
that created the socket.
This is what this commit implements.
We create a new port IPC message type of NXT_PORT_MSG_SOCKET_UNLINK,
that is used by the router process to notify the main process about a
Unix domain socket to unlink(2).
Upon doing a reconfigure the router process will call
nxt_router_listen_socket_release() which will close the socket, we
extend this function in the case of non-abstract Unix domain sockets, so
that it will send a message to the main process containing a copy of the
nxt_sockaddr_t structure that will contain the filename of the socket.
In the main process the handler that we have defined,
nxt_main_port_socket_unlink_handler(), for this message type will run
and allow us to look for the socket in question in the listen_sockets
array and remove it and unlink(2) the socket.
This then allows the reconfigure to work if it tries to bind(2) again to
a socket that previously existed.
Link: <https://github.com/nginx/unit/issues/669>
Link: <https://github.com/nginx/unit/pull/735>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
In nxt_router_prepare_msg() we create a buffer (nxt_unit_request_t *req)
that gets sent to an application process that contains details about a
client request.
This buffer was always a little larger than needed due to allocating space
for the remote address _and_ port and the local address _and_ port. We
also allocate space for the local port separately.
->{local,remote}->length includes the port number and ':' and also the
'[]' for IPv6. E.g [2001:db8::1]:8080
->{local,remote}->address_length represents the length of the unadorned
IP address. E.g 2001:db8::1
Update the buffer size so that we only allocate what is actually needed.
Suggested-by: Zhidao HONG <z.hong@f5.com>
Cc: Zhidao HONG <z.hong@f5.com>
Reviewed-by: Zhidao HONG <z.hong@f5.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This fixes an issue reported by @Peter2121 on GitHub.
In nxt_router_prepare_msg() we create a buffer (nxt_unit_request_t *req)
that gets sent to an application process that contains details about a
client request.
The req structure comprises various members with the final member being
an array (specified as a flexible array member, with its actual length
denoted by the req->fields_count member) of nxt_unit_field_t's. These
structures specify the length and offset for the various request headers
name/value pairs which are stored after some request metadata that is
stored immediately after this array of structs as individual nul
terminated strings.
After this we have the body content data (if any). So it looks a little
like
(gdb) x /64bs 0x7f38c976e060
0x7f38c976e060: "\353\346\244\t\006" <-- First nxt_unit_field_t
0x7f38c976e066: ""
0x7f38c976e067: ""
0x7f38c976e068: "T\001"
0x7f38c976e06b: ""
0x7f38c976e06c: "Z\001"
0x7f38c976e06f: ""
...
0x7f38c976e170: "\362#\244\v$" <-- Last nxt_unit_field_t
0x7f38c976e176: ""
0x7f38c976e177: ""
0x7f38c976e178: "\342\002"
0x7f38c976e17b: ""
0x7f38c976e17c: "\352\002"
0x7f38c976e17f: ""
0x7f38c976e180: "POST" <-- Start of request metadata
0x7f38c976e185: "HTTP/1.1"
0x7f38c976e18e: "unix:"
0x7f38c976e194: "unix:/dev/shm/842.sock"
0x7f38c976e1ab: ""
0x7f38c976e1ac: "fedora"
0x7f38c976e1b3: "/842.php"
0x7f38c976e1bc: "HTTP_HOST" <-- Start of header fields
0x7f38c976e1c6: "fedora"
0x7f38c976e1cd: "HTTP_X_FORWARDED_PROTO"
0x7f38c976e1e4: "https"
...
0x7f38c976e45a: "HTTP_COOKIE"
0x7f38c976e466: "PHPSESSID=8apkg25r9s9vju3pi085i21eh4"
0x7f38c976e48b: "public_form=sended" <-- Body content
Well that's how things are supposed to look! When using Unix domain
sockets what we actually got looked like
...
0x7f6141f3445a: "HTTP_COOKIE"
0x7f6141f34466: "PHPSESSID=uo5b2nu9buijkc89jotbgmd60vpublic_form=sended"
Here, the body content (from a POST for example) has been appended
straight onto the end of the last header field value. In this case
corrupting the PHP session cookie. The body content would still be
found by the application as its offset into this buffer is correct.
This problem was actually caused by a0327445 ("PHP: allowed to specify
URLs without a trailing '/'.") which added an extra item into this
request buffer specifying the port number that unit is listening on that
handled this request.
Unfortunately when I wrote that patch I didn't increase the size of this
request buffer to accommodate it.
When using normal TCP sockets we actually end up allocating more space
than required for this buffer, we track the end of this buffer up to
where the body content would go and so we have a few spare bytes between
the nul byte of the last field header value and the start of the body
content.
When using Unix domain sockets, they have no associated port number and
thus the port number has a length of 0 bytes, but we still write a '\0'
in there using up a byte that we didn't account for, this causes us to
loose the nul byte of the last header fields value causing the body data
to be appended to the last header field value.
The fix is simple, account for the local port length, we also add 1 to
it, this covers the nul byte, even if there is no port as with Unix
domain sockets.
Closes: <https://github.com/nginx/unit/issues/842>
Fixes: a0327445 ("PHP: allowed to specify URLs without a trailing '/'.")
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
It's for the introduction of njs support.
For each option that supports native variable and JS template literals introduced next,
it's unified as template string.
No functional changes.
Both @lucatacconi & @mwoodpatrick reported what appears to be the same
issue on GitHub. Namely that when using the PHP language module and
trying to access a URL that is a directory but without specifying the
trailing '/', they were getting a '503 Service Unavailable' error.
Note: This is when _not_ using the 'script' option.
E.g with the following config
{
"listeners": {
"[::1]:8080": {
"pass": "applications/php"
}
},
"applications": {
"php": {
"type": "php",
"root": "/var/tmp/unit-php"
}
}
}
and with a directory path of /var/tmp/unit-php/foo containing an
index.php, you would see the following
$ curl http://localhost/foo
<title>Error 503</title>
Error 503
However
$ curl http://localhost/foo/
would work and serve up the index.php
This commit fixes the above so you get the desired behaviour without
specifying the trailing '/' by doing the following
1] If the URL doesn't end in .php and doesn't have a trailing '/'
then check if the requested path is a directory.
2) If it is a directory then create a 301 re-direct pointing to it.
This matches the behaviour of the likes of nginx, Apache and
lighttpd.
This also matches the behaviour of the "share" action in Unit.
This doesn't effect the behaviour of the 'script' option which bypasses
the nxt_php_dynamic_request() function.
This also adds a couple of tests to test/test_php_application.py to
ensure this continues to work.
Closes: <https://github.com/nginx/unit/issues/717>
Closes: <https://github.com/nginx/unit/issues/753>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This is a preparatory patch that renames the 'local' and 'local_length'
members of the nxt_unit_request_t structure to 'local_addr' and
'local_addr_length' in preparation for the adding of 'local_port' and
'local_port_length' members.
Suggested-by: Zhidao HONG <z.hong@f5.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This makes the replacement of forwarded request header
like client_ip and protocol more generic.
It's a prerequirement for protocol replacement.
No functional changes.
The previous commit added more generic APIs for handling
NXT_CONF_VALUE_ARRAY and non-NXT_CONF_VALUE_ARRAY together.
Modify calling code to remove special cases for arrays and
non-arrays, taking special care that the path for non arrays is
logically equivalent to the previous special cased code.
Use the now-generic array code only.
The reference to the access_log structure is stored in the current
nxt_router_conf_t and the global nxt_router_t. When the reference is copied,
the reference counter should be adjusted accordingly.
This closes#593 issue on GitHub.
Application process started with shared port (and queue) already configured.
But still waits for PORT_ACK message from router to start request processing
(so-called "ready state").
Waiting for router confirmation is necessary. Otherwise, the application may
produce response and send it to router before the router have the information
about the application process. This is a subject of further optimizations.
port->app field is not thread safe and should be used in main thread only.
To release port after request processing, application reference should be
obtained from corresponding request descriptor.
This commit introduces the replacement of the client address based on the value
of a specified HTTP header. This is intended for use when Unit is placed
behind a reverse proxy like nginx or a CDN.
You must specify the source addresses of the trusted proxies. This can be
accomplished with any valid IP pattern supported by Unit's match block:
["10.0.0.1", "10.4.0.0/16", "!192.168.1.1"]
The feature is configured per listener.
The client address replacement functionality only operates when there is a
source IP match and the specified header is present. Typically this would be
an 'X-Forwarded-For' header.
{
"listeners": {
"127.0.0.1:8080": {
"client_ip": {
"header": "X-Forwarded-For",
"source": [
"10.0.0.0/8"
]
},
"pass": "applications/my_app"
},
}
}
If a request occurs and Unit receives a header like below:
"X-Forwarded-For: 84.123.23.23"
By default, Unit trusts the last rightmost IP in the header, so REMOTE_ADDR
will be set to 84.123.23.23 if the connection originated from 10.0.0.0/8.
If Unit runs behind consecutive reverse proxies and receives a header similar
to the following:
"X-Forwarded-For: 84.123.23.23, 10.0.0.254"
You will need to enable "recursive" checking, which walks the header from
last address to first and chooses the first non-trusted address it finds.
{
"listeners": {
"127.0.0.1:8080": {
"client_ip": {
"header": "X-Forwarded-For",
"source": [
"10.0.0.0/8"
]
"recursive": true,
},
"pass": "applications/my_app"
},
}
}
If a connection from 10.0.0.0/8 occurs, the chain is walked. Here, 10.0.0.254
is also a trusted address so the client address will be replaced with
84.123.23.23.
If all IP addresses in the header are trusted, the client address is set to
the first address in the header:
If 10.0.0.0/8 is trusted and "X-Forwarded-For: 10.0.0.3, 10.0.0.2, 10.0.0.1",
the client address will be replaced with 10.0.0.3.
When processing a restart request, the router sends a QUIT message to all
existing processes of the application. Then, a new shared application port is
created to ensure that new requests won't be handled by the old processes of
the application.
To support TLS sessions, Unit uses the OpenSSL built-in session cache; the
cache_size option defines the number sessions to store. To disable the feather,
the option must be zero.
A new application thread port message can be processed in the router after the
application is removed from the router. Assertion for this case is replaced by
a condition to store the new thread port until receiving the stop notification
from the application process.
Because of the incorrect 'last' field assignment, multiple listeners with
a TLS certificate did not initialize properly, which caused a router crash
while establishing a connection.
Test with multiple TLS listeners added.
The issue was introduced in the c548e46fe516 commit.
This closes#561 issue on GitHub.
To perform various configuration operations on SSL_CTX, OpenSSL provides
SSL_CONF_cmd(). Specifically, to configure ciphers for a listener,
"CipherString" and "Ciphersuites" file commands are used:
https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html
This feature can be configured in the "tls/conf_commands" section.
A crash was caused by an incorrect timer handler nxt_h1p_idle_timeout() if
SSL_shutdown() returned SSL_ERROR_WANT_READ/SSL_ERROR_WANT_WRITE.
The flag SSL_RECEIVED_SHUTDOWN is used to avoid getting SSL_ERROR_WANT_READ, so
the server won't wait for a close notification from a client.
For SSL_ERROR_WANT_WRITE, a correct timer handler is set up.
This patch fixes a possible race between the nxt_router_conf_wait() and
nxt_router_listen_socket_release() function calls and improves the 7f1b2eaa2d58
commit fix.
When an invalid TLS configuration is applied (such as the conf_commands
feature), nxt_cert_store_get() creates a buffer to send a certificate request
to the main process and adds its default completion handler to an asynchronous
queue to free the allocated buffer. However, if configuration fails,
nxt_router_conf_error() removes the memory pool used to allocate the buffer,
causing a crash when the completion handler is dispatched.
Assertion "src/nxt_buf.c:208 assertion failed: data == b->parent" is triggered
when is NXT_DEBUG enabled in the configure script.
This patch uses a reference counter to retain the memory pool and redefines the
completion handler to free the buffer before releasing the memory pool.
Listen socket is actually closed in the instant timer handler. This patch moves
the "configuration has been applied" notification to the timer handler to avoid
a situation when the user gets the response from the controller, but the listen
socket is still open in the router.
This fixes memory and shm file descriptor leakage that occurred when a large
request body was passed via shared memory. The leakage was caught with the
"test_settings_body_buffer_size" test. The main condition is the
"body_buffer_size" value exceeding 10 Mb (a shm segment). Thus, the router was
forced to split the body into several shm segments, but these buffers were not
freed because of dummy completion handlers.
The certificate is selected by matching the arriving SNI to the common name and
the alternatives names. If no certificate matches the name, the first bundle in
the array is chosen.
pthread_t on Solaris is an integer type with size not equal to pointer size.
To avoid warnings, type casts to and from pointer needs to be done via
uintptr_t type.
This change originally proposed by Juraj Lutter <juraj@lutter.sk>.
After WebSocket processing, the application port was released with incorrect
reason ("got request"), unnecessarily decrementing the active request counter.
The assertion was triggered only on application removal; a test was added
for this case.
The controller process awaits the response from the router for every
configration change request. This patch adds error reporting for various error
conditions which may happen because of file descriptors or memory shortage.
Lack of a response lead to the controller awaiting the response, thus being
unable to process other client reconfiguration requests that also became stuck.
