Shared app queue takes more memory than port memory. To unmap all memory pages
correct size need to be specified for munmap() call. Otherwise 4 Mb memory
leaked on each configured application removal.
The issue was introduced in 1d84b9e4b459.
Language versions are bumped to:
- python 3.9
- ruby 2.7
- go 1.15
- node 15
- perl 5.32
Removed images:
- python 2.7 due to EOL
- full/latest due to incompatibility with the new scheme
After WebSocket processing, the application port was released with incorrect
reason ("got request"), unnecessarily decrementing the active request counter.
The assertion was triggered only on application removal; a test was added
for this case.
The controller process awaits the response from the router for every
configration change request. This patch adds error reporting for various error
conditions which may happen because of file descriptors or memory shortage.
Lack of a response lead to the controller awaiting the response, thus being
unable to process other client reconfiguration requests that also became stuck.
Each application in router process required fd for a request queue shared
memory. When the number of file descripts close to the limit, and port sockets
successfully opened, router needs to properly handle the errors.
This patch closes port sockets before destroying port structure to avoid
file descriptors leakage and assertion in debug build.
Running `test_respawn_` test cases on a machine with Unit
daemon in background would fail tests because `ps ax` was
used without filtering out other unit instances.
This patch also prevents from tests killing other Unit
processes not related to tests.
The mount points are unmounted in the main process after it
detects the app process died. By testing the `tmpfs: true`
first, it happens that main could start the `tmpfs: false`
test case before main cleans the old process mount points.
This partially reverts the optimisation introduced in 1d84b9e4b459 to avoid an
unpredictable block in nxt_unit_process_port_msg(). Under high load, this
function may never return control to its caller, and the external event loop
(in Node.js and Python asyncio) won't be able to process other scheduled
events.
To reproduce the issue, two request processing types are needed: 'fast' and
'furious'. The 'fast' one simply returns a small response, while the 'furious'
schedules asynchronous calls to external resources. Thus, if Unit is subjected
to a large amount of 'fast' requests, the 'furious' request processing freezes
until the high load ends.
The issue was found by Wu Jian Ping (@wujjpp) during Node.js stream
implementation discussion and relates to PR #502 on GitHub.
ServerRequest now inherit stream Readable object. ServerResponse
provides 'writable' property.
Thanks to Wu Jian Ping (@wujjpp).
This closes#274, closes#317 issues and closes#502 PR on GitHub.
Changes introduced in a27532e3a17b effectively broke building of the unit
package due to missed dh_installsystemd script in older debhelper 9.x.
Once Ubuntu 16.04 reach EOL, the following actions should be made:
- this commit should be reverted;
- minimal debhelper version should be increased to 11.
This is required in order to build Debian packages from current tip
without making manual interventions.
Moving forward, this should be a part of every version bump commit.
When a static file larger than NXT_HTTP_STATIC_BUF_SIZE (128K) is served, two
buffers are allocated and chained; each retains the whole request memory pool.
Starting from 41331471eee7, the completion handler was called once for a linked
buffer chain, but the second buffer got lost.
This patch improves the completion handler's treatment of static buffers to
handle all linked buffers.
The Go compiler can't detect changes to C header files when
compiling CGO applications, and then this leads to Go test
samples being linked with wrong libunit.
This patch creates a new cache directory reused throughout
the test suite.
The nxt_unit_ctx_port_recv() function may return the NXT_UNIT_AGAIN code, in
which case an attempt to reread the message should be made.
The issue was reproduced in load testing with response sizes 16k and up.
In the rare case of a NXT_UNIT_AGAIN result, a buffer of size -1 was processed,
which triggered a 'message too small' alert; after that, the app process was
terminated.
Under high load, a queue synchonization issue may occur, starting from the
steady state when an app queue message is dequeued immediately after it has been
enqueued. In this state, the router always puts the first message in the queue
and is forced to notify the app about a new message in an empty queue using a
socket pair. On the other hand, the application dequeues and processes the
message without reading the notification from the socket, so the socket buffer
overflows with notifications.
The issue was reproduced during Unit load tests. After a socket buffer
overflow, the router is unable to notify the app about a new first message.
When another message is enqueued, a notification is not required, so the queue
grows without being read by the app. As a result, request processing stops.
This patch changes the notification algorithm by counting the notifications in
the pipe instead of getting the number of messages in the queue.
Multithreaded application may create different shared memory segments in
different threads. The segments then passed to different router threads.
Because of this multithreading, the order of adding incoming segments is
not determined and there can be situation when some of the incoming segments
are not initialized yet.
This patch simply adds check for NULL to skip non-initialized segments.
Crash reproduced during load tests with high number of simultaneous
connections (1024 and more).
The WSGI environment dictionary contains a number of static items, that are
pre-initialized on application start. Then it's copied for each request to be
filled with request-related data.
Now this dictionary copy operation will be done between processing of requests,
which should save some CPU cycles during request processing and thus reduce
response latency for non-peak load periods.
The code had a wrong assumption that "mount namespaces" automatically
unmounts process mounts when exits but this happens only with
unprivileged mounts.
