Changeset #699 fixes shared memory allocation: continous buffer with
requested size should be allocated or function failed. For body longer
than 10 Mb, this allocation will definitely fails.
For body buffer it is not required to send it in a single continous buffer,
so, need to request minimum reasonable amount of shared memory and try to
extend it, if possible or allocate next buffer.
Previously, one shared memory chunk was allocated under mutex and other
chunks (if required) were allocated using atomic operations. So such
allocation is not guaranteed and the result buffer can be less than
requested.
This commit moves multiple chunks allocation under mutex and guarantees
the result buffer is large enough.
"All problems in computer science can be
solved by another level of indirection"
Butler Lampson
Completion handlers for application response buffers executed after
sending the data to client. Application worker can be stopped right
after send response buffers to router. Worker stop causes removal
of all data structures for the worker.
To prevent shared memory segment unmap, need to count the number of
buffers which uses it. So instead of direct reference to shared
memory, need to reference to intermediate 'handler' structure with
use counter and pointer to shared memory.
Two different router threads may send different requests to single
application worker. In this case shared memory fds from worker
to router will be send over 2 different router ports. These fds
will be received and processed by different threads in any order.
This patch made possible to add incoming shared memory segments in
arbitrary order. Additionally, array and memory pool are no longer
used to store segments because of pool's single threaded nature.
Custom array-like structure nxt_port_mmaps_t introduced.
This allows to use shared memory to communicate with main process.
This patch changes shared memory segment format and breaks compatibility
with older modules.
To allow use port from different threads, the first step is to avoid using
port's memory pool for temporary allocations required to send data through
the port. Including but not limited by:
- buffers for data;
- send message structures;
- new mmap fd notifications;
It is still safe to use port memory pool for incoming buffers allocations
because recieve operation bound to single thread.
Do not reuse shared memory segment with different port until this segment
successfully received and indexed on other side. However, segment can be used
to transfer data via the port it was sent at any time.
NXT_PORT_MSG_LAST - mark message as last;
NXT_PORT_MSG_CLOSE_FD - close fd right after send;
Type constants altered to include last flag for single buffer messages.
Last sign is critical for coming port RPC layer. Handlers unregistered on last
message. Create sync buffer is not convenient, extra parameter is better.
There is a case in router where we use port in router connection thread.
Buffers are allocated within connection memory pool which can be used only in
this router thread. sendmsg() can be postponed into main router thread and
completion handler will compare current engine and post itself to correct
engine.
nxt_port_mmap_t stored in arrays and it is unsafe to store
pointer to array element.
Shared memory structures and macros moved to separate header
file to be used by GO package.
