Andrei reported an issue with building unit when using '-O0' with GCC
producing the following compiler errors
cc -c -pipe -fPIC -fvisibility=hidden -O -W -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wmissing-prototypes -Werror -g -O0 -I src -I build \
\
\
-o build/src/nxt_unit.o \
-MMD -MF build/src/nxt_unit.dep -MT build/src/nxt_unit.o \
src/nxt_unit.c
src/nxt_unit.c: In function ‘nxt_unit_log’:
src/nxt_unit.c:6601:9: error: ‘msg’ may be used uninitialized [-Werror=maybe-uninitialized]
6601 | p = nxt_unit_snprint_prefix(p, end, pid, level);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6682:1: note: by argument 2 of type ‘const char *’ to ‘nxt_unit_snprint_prefix’ declared here
6682 | nxt_unit_snprint_prefix(char *p, const char *end, pid_t pid, int level)
| ^~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6582:22: note: ‘msg’ declared here
6582 | char msg[NXT_MAX_ERROR_STR], *p, *end;
| ^~~
src/nxt_unit.c: In function ‘nxt_unit_req_log’:
src/nxt_unit.c:6645:9: error: ‘msg’ may be used uninitialized [-Werror=maybe-uninitialized]
6645 | p = nxt_unit_snprint_prefix(p, end, pid, level);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6682:1: note: by argument 2 of type ‘const char *’ to ‘nxt_unit_snprint_prefix’ declared here
6682 | nxt_unit_snprint_prefix(char *p, const char *end, pid_t pid, int level)
| ^~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6625:35: note: ‘msg’ declared here
6625 | char msg[NXT_MAX_ERROR_STR], *p, *end;
| ^~~
cc1: all warnings being treated as errors
The above was reproduced with
$ ./configure --cc-opt=-O0 && ./configure python && make -j4
This warning doesn't happen on clang (15.0.4) or GCC (8.3) and seems to
have been introduced in GCC 11. The above is from GCC (12.2.1, Fedora
37).
The trigger of this GCC issue is actually part of a commit I introduced
a few months back to constify some function parameters and it seems the
consensus for how to resolve this problem is to simply remove the const
qualifier from the *end parameter to nxt_unit_snprint_prefix().
Reported-by: Andrei Zeliankou <zelenkov@nginx.com>
Link: <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100417>
Link: <https://github.com/samtools/htslib/pull/1285>
Link: <https://gcc.gnu.org/gcc-11/changes.html>
Fixes: 4418f99 ("Constified numerous function parameters.")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
pthread_mutex_init(3) may fail for several reasons, and failing to
check will cause Undefined Behavior when those errors happen. Add
missing checks, and correctly deinitialize previously created
stuff before exiting from the API.
Signed-off-by: Alejandro Colomar <alx@nginx.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Reviewed-by: Zhidao HONG <z.hong@f5.com>
In nxt_unit_create() we could leak a mutex created in
nxt_unit_ctx_init().
This could happen if nxt_unit_ctx_init() succeeded but later on we
bailed out of nxt_unit_create(), we would destroy the mutex created in
nxt_unit_create() but not the one created in nxt_unit_ctx_init().
Reorder things so that we do the call to nxt_unit_create() after all the
other checks so if it fails we don't leak the mutex it created.
Co-developed-by: Andrew Clayton <a.clayton@f5.com>
Signed-off-by: Andrew Clayton <a.clayton@f5.com>
Signed-off-by: Alex Colomar <a.colomar@f5.com>
As was pointed out by the cppcheck[0] static code analysis utility there
was a useless assignment in nxt_unit_request_read(). The size parameter
is passed in by value and was being modified without being used again.
[0]: https://cppcheck.sourceforge.io/
As was pointed out by the cppcheck[0] static code analysis utility, lib
was being set in nxt_unit_shm_open() regardless of platform when in fact
it's only used when (NXT_HAVE_MEMFD_CREATE || NXT_HAVE_SHM_OPEN).
Move the variable declaration & definition to be within the
#if (NXT_HAVE_MEMFD_CREATE || NXT_HAVE_SHM_OPEN)
block.
[0]: https://cppcheck.sourceforge.io/
As was pointed out by the cppcheck[0] static code analysis utility we
can mark numerous function parameters as 'const'. This acts as a hint to
the compiler about our intentions and the compiler will tell us when we
deviate from them.
[0]: https://cppcheck.sourceforge.io/
Port's "data" field may be used by application and thus need to be set to NULL.
The issue was introduced in the f8a0992944df commit.
Found by Coverity (CID 374352).
Application process started with shared port (and queue) already configured.
But still waits for PORT_ACK message from router to start request processing
(so-called "ready state").
Waiting for router confirmation is necessary. Otherwise, the application may
produce response and send it to router before the router have the information
about the application process. This is a subject of further optimizations.
The "recv_msg.incoming_buf" is checked after jumping to the "done" label
if nxt_socket_msg_oob_get_fds() returns an error.
Also moved initialization of "port_msg" near to its first usage.
Found by Coverity (CID 373899).
pthread_t on Solaris is an integer type with size not equal to pointer size.
To avoid warnings, type casts to and from pointer needs to be done via
uintptr_t type.
This change originally proposed by Juraj Lutter <juraj@lutter.sk>.
This partially reverts the optimisation introduced in 1d84b9e4b459 to avoid an
unpredictable block in nxt_unit_process_port_msg(). Under high load, this
function may never return control to its caller, and the external event loop
(in Node.js and Python asyncio) won't be able to process other scheduled
events.
To reproduce the issue, two request processing types are needed: 'fast' and
'furious'. The 'fast' one simply returns a small response, while the 'furious'
schedules asynchronous calls to external resources. Thus, if Unit is subjected
to a large amount of 'fast' requests, the 'furious' request processing freezes
until the high load ends.
The issue was found by Wu Jian Ping (@wujjpp) during Node.js stream
implementation discussion and relates to PR #502 on GitHub.
The nxt_unit_ctx_port_recv() function may return the NXT_UNIT_AGAIN code, in
which case an attempt to reread the message should be made.
The issue was reproduced in load testing with response sizes 16k and up.
In the rare case of a NXT_UNIT_AGAIN result, a buffer of size -1 was processed,
which triggered a 'message too small' alert; after that, the app process was
terminated.
Under high load, a queue synchonization issue may occur, starting from the
steady state when an app queue message is dequeued immediately after it has been
enqueued. In this state, the router always puts the first message in the queue
and is forced to notify the app about a new message in an empty queue using a
socket pair. On the other hand, the application dequeues and processes the
message without reading the notification from the socket, so the socket buffer
overflows with notifications.
The issue was reproduced during Unit load tests. After a socket buffer
overflow, the router is unable to notify the app about a new first message.
When another message is enqueued, a notification is not required, so the queue
grows without being read by the app. As a result, request processing stops.
This patch changes the notification algorithm by counting the notifications in
the pipe instead of getting the number of messages in the queue.
If shared queue is empty, allocated read buffer should be explicitly
released.
Found by Coverity (CID 363943).
The issue was introduced in f5ba5973a0a3.
The issue occurred under highly concurrent request load in Go applications.
Such applications are multi-threaded but use a single libunit context; any
thread-safe code in the libunit context is only required for Go applications.
As a result of improper request state reset, the recycled request structure was
recovered in the released state, so further operations with this request
resulted in 'response already sent' warnings. However, the actual response was
never delivered to the router and the client.
The issue only occurred in Go applications because "port_send" is overloaded
only in Go. To reproduce it, send multiple concurrent requests to the
application after it has initialised. The warning message "[unit] [go] port
NNN:dd not found" is the first visible aspect of the issue; the second and more
valuable one is a closed connection, an error response, or a hanging response to
some requests.
When the application starts, it is unaware of the router's worker thread ports,
so it requests the ports from the router after receiving requests from the
corresponding router worker threads. When multiple requests are processed
simultaneously, the router port may be required by several requests, so request
processing starts only after the application receives the required port
information. The port should be added to the Go port repository after its
'ready' flag is updated. Otherwise, Unit may start processing some requests and
use the port before it is in the repository.
The issue was introduced in changeset 78836321a126.
Application shared queue only capable to pass one shared memory buffer.
The rest buffers in chain needs to be send directly to application in response
to REQ_HEADERS_AC message.
The issue can be reproduced for configurations where 'body_buffer_size' is
greater than memory segment size (10 Mb). Requests with body size greater
than 10 Mb are just `stuck` e.g. not passed to application awaiting for more
data from router.
The bug was introduced in 1d84b9e4b459 (v1.19.0).
The PORT_ACK message is the router's response to the application's NEW_PORT
message. After receiving PORT_ACK, the application is safe to process requests
using this port.
This message avoids a racing condition when the application starts processing a
request from the shared queue and sends REQ_HEADERS_ACK. The REQ_HEADERS_ACK
message contains the application port ID as reply_port, which the router uses
to send request data. When the application creates a new port, it
immediately sends it to the main router thread. Because the request is
processed outside the main thread, a racing condition can occur between the
receipt of the new port in the main thread and the receipt of REQ_HEADERS_ACK
in the worker router thread where the same port is specified as reply_port.
Data in the queue and the socket are transmitted independently; special
READ_QUEUE and READ_SOCKET message types are used for synchronization.
The warning was accidentally committed with changeset 1d84b9e4b459.
Large-bodied requests are added to the request hash to be found when the body
arrives. However, changeset 1d84b9e4b459 introduced a bug: the 'in_hash' flag,
used to remove the request from the hash at request release, was cleared after
the first successful request lookup. As a result, the entry was never removed.
This change aids heap usage analysis in applications.
The alloc and free functions are also required for lvlhash due to the upcoming
threading support, because using main nxt_memalign() and nxt_free() isn't safe
in a multithreaded app environment. The reason is that these functions may use
thread-local structures which aren't initialized properly in applications.
The default libunit behavior relies on blocking the recv() call for port file
descriptors, which an application may override if needed. For external
applications, port file descriptors were toggled to blocking mode before the
exec() call. If the exec() call failed, descriptor remained blocked, so the
process hanged while trying to read from it.
This patch moves file descriptor mode switch inside libunit.
Generic process-to-process shared memory exchange is no more required. Here,
it is transformed into a router-to-application pattern. The outgoing shared
memory segments collection is now the property of the application structure.
The applications connect to the router only, and the process only needs to group
the ports.
This is the port shared between all application processes which use it to pass
requests for processing. Using it significantly simplifies the request
processing code in the router. The drawback is 2 more file descriptors per each
configured application and more complex libunit message wait/read code.
The application process needs to request the shared memory segment from the
router instead of the latter pushing the segment before sending a request to
the application. This is required to simplify the communication between the
router and the application and to prepare the router for using the application
shared port and then the queue.
