Commit Graph

1642 Commits

Author SHA1 Message Date
Max Romanov
f147943f63 Style fixes for 2 file descriptors transfer over port.
Two consecutive fd and fd2 fields replaced with array.
2020-08-11 21:48:27 +03:00
Max Romanov
acb0cca49d Moving file descriptor blocking to libunit.
The default libunit behavior relies on blocking the recv() call for port file
descriptors, which an application may override if needed.  For external
applications, port file descriptors were toggled to blocking mode before the
exec() call.  If the exec() call failed, descriptor remained blocked, so the
process hanged while trying to read from it.

This patch moves file descriptor mode switch inside libunit.
2020-08-11 21:48:16 +03:00
Max Romanov
8cf522bf2d Wrapping close() call in libunit for logging. 2020-08-11 19:20:36 +03:00
Max Romanov
e227fc9e62 Introducing application and port shared memory queues.
The goal is to minimize the number of syscalls needed to deliver a message.
2020-08-11 19:20:34 +03:00
Max Romanov
a82cf4ffb6 Circular queues implementations and a test.
- naive circular queue, described in the article "A Scalable, Portable, and
 Memory-Efficient Lock-Free FIFO Queue" by Ruslan Nikolaev:
https://drops.dagstuhl.de/opus/volltexte/2019/11335/pdf/LIPIcs-DISC-2019-28.pdf
- circular queue, proposed by Valentin Bartenev in the "Unit router application
IPC" design draft
2020-08-11 19:20:32 +03:00
Max Romanov
a1e9df2aef Port message extended to transfer 2 file descriptors. 2020-08-11 19:20:30 +03:00
Max Romanov
72475ee11c Made router port message handlers into static functions.
Mostly harmless.
2020-08-11 19:20:28 +03:00
Max Romanov
496f41c134 Tests: reducing the number of generated applications.
Each application initializes a shared port with 2 file descriptors, so the test
fails because the router reaches the open files limit.
2020-08-11 19:20:25 +03:00
Max Romanov
bab4a9e9f2 Tests: skipping idle zero timeout.
This is a temporary solution after the 'shared port' patch.  The application
process becomes idle immediately after creation.  Even if it starts processing
a request (without acknowledging it yet), it is stopped by the router because
an 'out-of-idle-time' event occurs.
2020-08-11 19:20:23 +03:00
Max Romanov
f4a118f84a Adding debug messages to catch process management issues. 2020-08-11 19:20:20 +03:00
Max Romanov
2f3d27fa22 Process structures refactoring in runtime and libunit.
Generic process-to-process shared memory exchange is no more required.  Here,
it is transformed into a router-to-application pattern.  The outgoing shared
memory segments collection is now the property of the application structure.
The applications connect to the router only, and the process only needs to group
the ports.
2020-08-11 19:20:17 +03:00
Max Romanov
8359560612 Introducing the shared application port.
This is the port shared between all application processes which use it to pass
requests for processing.  Using it significantly simplifies the request
processing code in the router.  The drawback is 2 more file descriptors per each
configured application and more complex libunit message wait/read code.
2020-08-11 19:20:15 +03:00
Max Romanov
6e31d6cd39 Changing router to application shared memory exchange protocol.
The application process needs to request the shared memory segment from the
router instead of the latter pushing the segment before sending a request to
the application.  This is required to simplify the communication between the
router and the application and to prepare the router for using the application
shared port and then the queue.
2020-08-11 19:20:13 +03:00
Max Romanov
3cbc22a6dc Changing router to application port exchange protocol.
The application process needs to request the port from the router instead of the
latter pushing the port before sending a request to the application.  This is
required to simplify the communication between the router and the application
and to prepare the router to use the application shared port and then the queue.
2020-08-11 19:20:10 +03:00
Max Romanov
bf647588ff Adding a reference counter to the libunit port structure.
The goal is to minimize the number of (pid, id) to port hash lookups which
require a library mutex lock.  The response port is found once per request,
while the read port is initialized at startup.
2020-08-11 19:20:06 +03:00
Max Romanov
ec3389b63b Libunit refactoring: port management.
- Changed the port management callbacks to notifications, which e. g. avoids
the need to call the libunit function
- Added context and library instance reference counts for a safer resource
release
- Added the router main port initialization
2020-08-11 19:19:55 +03:00
Max Romanov
3a721e1d96 Fixing leaked configuration objects.
If there are no listen sockets, the router configuration usage counter
remains 0 and never decreases.  The only moment to release a configuration is
right after a configuration update.
2020-08-09 10:26:19 +03:00
Max Romanov
0f3abebd01 Fixing connection remote sockaddr leakage.
Earlier patch 1bf971f83571 fixes connection leakage.  But connection
free requires separate remote sockaddr release.
2020-08-09 10:22:05 +03:00
Valentin Bartenev
91280b4c0b Ruby: fixed gem mount paths.
The gem paths must depend on the specified interpreter.
Also, gemdir looks redundant as it's already included in Gem.default_path().
2020-08-09 01:00:44 +03:00
Valentin Bartenev
317fabc83e Ruby: simplified commands in ./configure script.
There is no reason to use printf instead of just print.

No functional changes.
2020-08-09 01:00:44 +03:00
Valentin Bartenev
7e1d800f08 Ruby: removed unused variable from ./configure script. 2020-08-09 01:00:44 +03:00
Valentin Bartenev
0d7a6885a3 Configure: verifying the Ruby library path.
An attempt to build a Ruby module for a custom Ruby installation that has the
same major version as the system Ruby may unexpectedly cause the use of the
system Ruby library.

This closes #449 issue on GitHub.
2020-08-08 23:52:31 +03:00
Max Romanov
375cbc2cc4 Node.js: correct port data memory release.
According to libuv documentation, uv_poll_t memory should be released
in a callback function passed to uv_close().  Otherwise, the Node.js application
process may crash at exit.
2020-08-07 15:06:24 +03:00
Max Romanov
78fd04adcf Fixing listen event connection leakage.
A connection object is allocated in advance for each listen event object to be
used for the established connection.  This connection needs to be freed when the
listen event is destroyed.
2020-08-07 15:06:18 +03:00
Valentin Bartenev
b0ff245ca8 Improved mkstemp() error reporting.
The invocation parameters should be logged as well, notably the path of the file
that is failed to be created.

Also, log level changed to ALERT as it's quite critical error.
2020-08-05 16:11:20 +03:00
Valentin Bartenev
2b53c7bbbd Fixed nxt_conn_accept_alloc() behavior in low memory conditions.
Earlier, if nxt_mp_create() failed to allocate memory while accepting a new
connection, the resulting NULL was subsequently passed to nxt_mp_destroy(),
crashing the process.

More, if nxt_mp_create() was successful but nxt_sockaddr_cache_alloc() failed,
the connection object wasn't destroyed properly, leaving the connection counter
in an inconsistent state.  Repeated, this condition lowered the connection
capacity of the process and could eventually prevent it from accepting
connections altogether.
2020-08-05 14:55:34 +03:00
Tiago Natel de Moura
b28b4459b0 Isolation: fixed the generation of mounts table.
Since the introduction of rootfs feature, some language modules
can't be configured multiple times.

Now the configure generates a separate nxt_<module>_mounts.h for
each module compiled.
2020-07-31 12:21:21 +01:00
Tiago Natel de Moura
f3471c29c0 PHP: fixed version comparison in configure script.
Some PPAs for Ubuntu package PHP with versions like:
 7.2.28-3+ubuntu18.04.1+deb.sury.org+1

But the script expected only "X.Y.Z".

The issue was introduced in:
 http://hg.nginx.org/unit/rev/2ecb15904ba5
2020-07-28 16:17:18 +01:00
Axel Duch
c3e6901f53 Configuration: fixed buffer over-read in pattern validation.
There was an undefined behavior in the validation function, caused by testing
one character after the string if a wildcard was at the end.
2020-07-28 14:51:33 +01:00
Andrei Zeliankou
f1e445bdef Tests: added PHP test with time check in error log messages. 2020-07-28 04:53:40 +01:00
Andrei Zeliankou
355ed9697d Tests: fixed double stop() call for some tests. 2020-07-28 04:53:32 +01:00
Andrei Zeliankou
dc1377dc48 Tests: style. 2020-07-28 03:09:50 +01:00
Max Romanov
c617480eef Using plain shared memory for configuration pass.
There is no restrictions on configration size and using segmented shared memory
only doubles memory usage because to parse configration on router side,
it needs to be 'plain' e. g. located in single continous memory buffer.
2020-07-25 11:06:32 +03:00
Valentin Bartenev
10f90f0d48 Configuration: added checking for presence of mandatory fields. 2020-07-24 20:25:20 +03:00
Valentin Bartenev
2a71a8a9f4 Added missing ending indicator in object members validation lists.
This fixes undefined behaviour due to array over-read if an unknown parameter
is specified in an uidmap, a gidmap, or a php target object.
2020-07-24 20:25:15 +03:00
Axel Duch
29cf3cc6c1 Configuration: removing redundant check.
Thanks to 洪志道 (Hong Zhi Dao).
2020-07-24 17:10:26 +01:00
Axel Duch
85a1e083af Minor changes and renaming an NJS artifact to NXT.
This is partially related to #434 issue on Github.
Thanks to 洪志道 (Hong Zhi Dao).
2020-07-24 13:10:24 +01:00
Max Romanov
762511c510 Fixing request_app_link reference counting.
Racing conditions reproduced periodically on test_python_process_switch.
2020-07-23 14:25:46 +03:00
Max Romanov
9641fb0ef1 Fixing various router crashes on exit caused by runtime pool free.
Currently, the router exits without waiting for the worker threads to stop.
There is a short gap between the runtime memory pool's free and the exit, during
which a worker thread may try to access a runtime structure.  In turn, this may
cause a crash.  For now, it is better to keep this memory allocated.
2020-07-23 14:25:21 +03:00
Max Romanov
d3c8d62280 PHP: using nxt_unit_default_init() for module structure init.
Using this function in all language modules helps to avoid code duplication
and reduce the size of future patches.
2020-07-23 14:25:12 +03:00
Max Romanov
137c1e736f Fixing main and application port structs file descriptor init.
Correct value for non-initialized file descriptor is -1, because most of the
checks in libunit compares file descriptor with -1 before performing an
action.  Using 0 as default value, may cause to close file descriptor #0, this
may affect application logic.

It is not required to list this patch in changelog because impact is not seen
by end users.
2020-07-23 14:24:55 +03:00
Max Romanov
fa696569f9 PHP: removing assertion to fix build on macOS.
The nxt_assert macro uses nxt_thread_context, which caused the following linker
error when using it in the library:

ld: illegal thread local variable reference to regular symbol
_nxt_thread_context for architecture x86_64
2020-07-23 14:24:16 +03:00
Max Romanov
661c223eda Tests: switching stdout to blocking before printing log.
This is another attempt to fix the following error message:
BlockingIOError: [Errno 11] write could not complete without blocking
2020-07-22 10:05:10 +03:00
Max Romanov
ef71948196 Fixing buffer overflow check in discovery.
Incorrect check prevents Unit to start without modules.

This issue was introduced in 4a3ec07f4b19.
2020-07-22 10:04:57 +03:00
Valentin Bartenev
d86e0a7aec PHP: logging in request context when possible. 2020-07-21 20:27:37 +03:00
Valentin Bartenev
f46ef1b121 PHP: fixed incorrect time in interpreter error log messages.
Previously, the log message callback used a generic log function, that relied on the process time cache.
Since there were no time update calls in the application processes, all log lines were printed with the
same time, usually correlated with the process start.

Now, a non-cached logging function from libunit is used.
2020-07-21 20:27:37 +03:00
Valentin Bartenev
f69d470752 Fixed non-debug log time format in libunit.
This makes log format used in libunit consistent with the daemon, where milliseconds are printed only in the
debug log level.

Currently a compile time switch is used, since there's no support for runtime changing of a log level for now.
But in the future this should be a runtime condition, similar to nxt_log_time_handler().
2020-07-21 20:27:37 +03:00
Konstantin Pavlov
55bac9153b Docker: run entrypoint scripts for unitd-debug as well. 2020-07-13 15:06:26 +03:00
Axel Duch
b6792b00ae Router: route patterns multi wildcards fix.
Matching 'start' and 'end' position now adjusted to avoid false matching.

This is related to #434 issue on Github.
Thanks to 洪志道 (Hong Zhi Dao).
2020-07-10 10:28:53 +01:00
Igor Sysoev
18fbfc3d50 Destroying temporary router configuration.
The lifespan of a listening socket is longer than both router
configuration's and temporary router configuration's lifespan,
so the sockets should be stored in persistent queues. Safety
is ensured by the fact that the router processes only one new
configuration at any time.
2020-07-06 15:32:20 +03:00