After a process exits, all ports linked to it from other processes
should be closed. All unsent file descriptors in port queue, marked as
"close after send", should be closed to avoid resource leakage.
All allocated blocks for lvlhash required to be aligned because lower
address bits used for various extra information. Using unaligned blocks
may cause invalid memory aceess.
This was issue found on buildbot running large configuration tests.
Master port stores two file descriptors and works as a read port on the master
process side. After a fork, the port switches into write mode and the read
socket closes, but the same event structure is used for the write socket.
However, the inherited structure remained in read state, telling the epoll
engine to use MOD operation instead of ADD. The patch resets read event
state, so the engine may write using proper ADD operation.
This is required to assemble fragmented messages correctly. Stream
identifier is unique only for messages generated within a process, but
the (stream, pid) pair should be enough to avoid collisions. Adding
reply_port seems redundant because it's enough to add stream to a pid.
This closes#199 issue on GitHub.
Thanks to 洪志道 (Hong Zhi Dao).
Sending large plain (exceeding port's max_size, not in shared memory) messages
causes message fragmentation. First message fragment is sent successfully,
but the next fragment may fail with the EAGAIN error. In this case, the
message has to be pushed back to queue head for additional processing.
Related to #167 issue on GitHub.
Before this fix, large plain message (i.e. configuration) send may fail
with the 'Message too big' error, because internal fragmentation
implementation does not account for 16 byte message header.
This closes#167 issue on GitHub.
As far as I understand, this field is important to control the number of
buffers send in a single write attempt. Furthermore, having uninitialized
field is always bad.
This closes#204 issue on GitHub.
Thanks to 洪志道 (Hong Zhi Dao).
nxt_sendbuf_completion() has been renamed to nxt_port_buf_completion()
and moved to src/nxt_port_socket.c. nxt_sendbuf_completion0() has been
renamed to nxt_sendbuf_completion().
- Each sendmsg() transmits no more than port->max_size payload data.
- Longer buffers are fragmented and send using multiple sendmsg() calls.
- On receive side, buffers are connected in chain.
- Number of handler calls is the same as number of nxt_port_socket_write()
calls.
- nxt_buf_make_plain() function introduced to make single plain buffer from
the chain.
Port message handler may perform fork() and then close port read file
descriptor and enable write on same event fd. Next read attempt in this case
may cause different errors in log file.
For empty write queue cases, it is possible to avoid allocation and enqueue
send message structures. Send message initialized on stack and passed to
write handler. If immediate write fails, send message allocated from engine
pool and enqueued.
Use counter helps to simplify logic around port and application free.
Port 'post' function introduced to simplify post execution of particular
function to original port engine's thread.
Write message queue is protected by mutex which makes port write operation
thread safe.
To allow use port from different threads, the first step is to avoid using
port's memory pool for temporary allocations required to send data through
the port. Including but not limited by:
- buffers for data;
- send message structures;
- new mmap fd notifications;
It is still safe to use port memory pool for incoming buffers allocations
because recieve operation bound to single thread.
NXT_PORT_MSG_LAST - mark message as last;
NXT_PORT_MSG_CLOSE_FD - close fd right after send;
Type constants altered to include last flag for single buffer messages.
Last sign is critical for coming port RPC layer. Handlers unregistered on last
message. Create sync buffer is not convenient, extra parameter is better.
New port message type introduced NXT_PORT_MSG_REMOVE_PID. Default handler
removes process description from nxt_runtime_t with all ports, incoming and
outgoing mmaps etc.
There is a case in router where we use port in router connection thread.
Buffers are allocated within connection memory pool which can be used only in
this router thread. sendmsg() can be postponed into main router thread and
completion handler will compare current engine and post itself to correct
engine.
