The reason for the change is that the req_app_link reference count
was incorrect if the application crashed at start; in this case,
the nxt_request_app_link_update_peer() function was never called.
This closes#332 issue on GitHub.
There was a change (ruby/ruby@6c70fed) in Ruby 2.6 that moved
RUBY_DESCRIPTION global constant definition out of Init_version().
Unit initialized Ruby incorrectly, so the constant was not defined.
This closes#330 issue on GitHub.
Name and value in each header are 0-terminated, so additional 2 bytes
should be allocated for them. There were several attempts to add these
2 bytes to headers in language modules, but some modules weren't updated.
Also, adding these 2 bytes is specific to the implementation which may be
changed later, so extending this mechanics to modules may cause errors.
Python 3.8 has 'tp_print' field in PyTypeObject struct. This field is
attributed as deprecated. So, clang generates warning (which is turned to
error) as a result of initializing this field. From the other hand, it is
impossible to omit this field in positional initialization. The solution
is to use designated initializer.
Silencing usage message during configure python.
This is related to #331 issue on GitHub.
There was a typo: nxt_queue_head() used instead of nxt_queue_first() in
connection iteration loop. This prevents idle connection close on quit.
This closes#334 issue on GitHub.
Thanks to 洪志道 (Hong Zhi Dao).
Currently almost all Unit object files depends on generated nxt_version.h.
This patch adds missing dependence and fixes running make with multiple
jobs.
This closes#318 issue on GitHub.
Header names and values are stored 0-terminated for ease of use in different
languages, so magic number 2 should be added to each name-value pair size.
When Unit starts, the main process waits for module discovery message for a
while. If a QUIT signal arrives at this time, the router and controller
processes created by main and Unit stay running. Also, the main process
doesn't stop them after the second QUIT signal is received in this case.
Each request references the router process structure that owns all memory
maps. The process structure has a reference counter; each request increases
the counter to lock the structure in memory until request processing ends.
Incoming and outgoing buffers reference memory maps that the process owns,
so the process structure should be released only when all buffers are
released to avoid invalid memory access and a crash.
This describes the libunit library mechanism used for application processes.
The background of this issue is as follows:
The issue was found on buildbot when the router crashed during Java
websocket tests. The Java application receives a notification from the
master process; when the notification is processed, libunit deletes the
process structure from its process hash and decrements the use counter;
however, active websocket connections maintain their use counts on the
process structure. After that, when the master process is stopping the
application, libunit releases active websocket connections. At this point,
it's important to release the connections' memory buffers before the
corresponding process structure and all shared memory segments are released.
To pass Go object references to C and back we use hack with casting to
unsafe and then to uintptr. However, we should not store such references
because Go not guaratnee it will be available by the same address.
Introducing map with integer key helps to avoid dereference stored address.
This closes#253 and #309 issues on GitHub.
By design, Unit context is created for the thread which reads messages from
the router. However, Go request handlers are called in a separate goroutine
that may be executed in a different thread. To avoid a racing condition,
access to lists of free structures in the context should be serialized. This
patch should fix random crashes in Go applications under high load.
This is related to #253 and #309 issues on GitHub.
The "nxt_http_websocket" request state, defined in "nxt_http_websocket.c",
is used in "nxt_router.c" and must be linked with external symbol declared
in "nxt_router.c".
Due to the missing "extern" keyword, building Unit with some linkers
(notably gold and LLD) caused WebSocket connections to get stuck or even
crash the router process.
All allocated blocks for lvlhash required to be aligned because lower
address bits used for various extra information. Using unaligned blocks
may cause invalid memory aceess.
This was issue found on buildbot running large configuration tests.
Master port stores two file descriptors and works as a read port on the master
process side. After a fork, the port switches into write mode and the read
socket closes, but the same event structure is used for the write socket.
However, the inherited structure remained in read state, telling the epoll
engine to use MOD operation instead of ADD. The patch resets read event
state, so the engine may write using proper ADD operation.
This is required to assemble fragmented messages correctly. Stream
identifier is unique only for messages generated within a process, but
the (stream, pid) pair should be enough to avoid collisions. Adding
reply_port seems redundant because it's enough to add stream to a pid.
This closes#199 issue on GitHub.
Thanks to 洪志道 (Hong Zhi Dao).
Sending large plain (exceeding port's max_size, not in shared memory) messages
causes message fragmentation. First message fragment is sent successfully,
but the next fragment may fail with the EAGAIN error. In this case, the
message has to be pushed back to queue head for additional processing.
Related to #167 issue on GitHub.
Unlike EAGAIN, ENOBUFS returned on OSX when trying to send many relatively
small (64 bytes) fragments.
Found during investigation of #167 issue on GitHub.
Before this fix, large plain message (i.e. configuration) send may fail
with the 'Message too big' error, because internal fragmentation
implementation does not account for 16 byte message header.
This closes#167 issue on GitHub.
As far as I understand, this field is important to control the number of
buffers send in a single write attempt. Furthermore, having uninitialized
field is always bad.
This closes#204 issue on GitHub.
Thanks to 洪志道 (Hong Zhi Dao).
This issue was introduced in libunit commit (e0f0cd7d244a). All port
sockets in application should be in blocking mode whereas Unit itself
operates non-blocking sockets.
Having non-blocking sockets in application may cause send error during
intensive response packets generation.
See https://mailman.nginx.org/pipermail/unit/2018-October/000080.html.
Router requests application start by sending port message with application
parameters to master. In response router gets application port. Stream id
used to link request and response. If application process terminates
unexpectedly (e.g. before ready state), it is required to notify router about
application start failure. For this purpose stream id is stored in process
initialization parameters.
Stream id sequentially incremented and starts from 1 in case of router
resurrection. New router application start stream id may coincide with
stream id of currently running/terminating application processes. In such
cases router may receive REMOVE_PID message with same stream id as used in
application start request and mistakenly reports application start failure.
This commit tries to avoid such errors by resetting stream for processes in
ready state.
Router reconfiguration may lead to starting new application processes, opening
listen ports etc. These actions are asynchronous and require message
exchange with master process. Router stores reference for controller
(response) port in structure associated with reconfiguration process (tmcf).
The port used to provide reconfiguration response (either success or failed)
to controller.
This commit increases response port use counter to make sure port structure
not freed while router waits for result of asynchronous operations.
Attribute 'application' is optional in listener object. This commit adds
missing code to reset 'lscf' to avoid garbage and crash when the attribute
is absent.
Unit master process restarts the router if the router accidentally dies.
New router process receives the configuration from controller and starts
configured applications. The information of running applications cannot
be transferred to router because currently there is no persistent application
identifier. To avoid orphan application processes started by died router,
master process stops all currently running applications once it receives
SIGCHLD for router process.
For accurate app descriptor release, it is required to count the number of
use counts. Use count increased when:
- app linked to configuration app queue;
- socket conf stores pointer to app;
- request for start app process posted to router service thread;
Application port has pointer to app, but it does not increase use count
to avoid use count loop.
Timer needs a pointer to nxt_timer_t which is stored in engine timers tree.
nxt_timer_t now resides in nxt_app_joint_t and does not lock the application.
Start process port RPC handlers is also linked to nxt_app_joint_t.
App joint (nxt_app_joint_t) is a 'weak pointer':
- single threaded;
- use countable;
- store pointer to nxt_app_t (which can be NULL);
nxt_app_t has pointer to nxt_app_joint_t and update its pointer to app.
Unit router process may send mmap file decritptor to the application process
for further information exchange. During this process there may be various
errors, which should be described in application error log. If file descriptor
cannot be properly transferred with 'new mmap' message, fd variable will
be assigned to -1 and further syscalls using this file descriptor will fail.
For 'new port' message fd is checked in the same way.
This commit adds early 'invalid file descriptor' diagnostic and write
corresponding message to error log.
Found by Coverity (CID 308515).
For the optimization purpose, function nxt_unit_remove_process() expects
lib->mutex to be locked. The function then moves ports queue into temporary
queue and releases mutex. In nxt_unit_done() there were two errors: mutex was
not locked before nxt_unit_remove_process() call and mutex was not destroyed.
It is hard to tell what was possible negative impact of this errors.
Found by Coverity (CID 308517).
Changeset #699 fixes shared memory allocation: continous buffer with
requested size should be allocated or function failed. For body longer
than 10 Mb, this allocation will definitely fails.
For body buffer it is not required to send it in a single continous buffer,
so, need to request minimum reasonable amount of shared memory and try to
extend it, if possible or allocate next buffer.
Previously, one shared memory chunk was allocated under mutex and other
chunks (if required) were allocated using atomic operations. So such
allocation is not guaranteed and the result buffer can be less than
requested.
This commit moves multiple chunks allocation under mutex and guarantees
the result buffer is large enough.
This retains application during reconfiguration if an incoming connection
was already established before the reconfiguration, but no request was yet
created for the connection and thus no application was yet assigned to
the request.
If the first buffer contained only header and there were other buffers in chain,
those buffers were not processed. In particular, this broke Perl application
responses with empty body.
Max Romanov