Commit Graph

1749 Commits

Author SHA1 Message Date
Andrei Zeliankou
6dc9c47ccd Tests: style. 2021-01-13 06:22:43 +00:00
Andrei Zeliankou
f5ac143246 Tests: unit_stop() removed where possible.
Since wait_for_record() was introduced there is no need
to stop Unit before parsing unit.log.
2021-01-12 06:20:23 +00:00
Tiago Natel de Moura
a0bc946db3 Tests: fixed test_respawn.py to act upon test processes.
Running `test_respawn_` test cases on a machine with Unit
daemon in background would fail tests because `ps ax` was
used without filtering out other unit instances.

This patch also prevents from tests killing other Unit
processes not related to tests.
2021-01-08 10:38:46 +00:00
Tiago Natel de Moura
13a06497de Tests: reordered asserts to avoid a test race.
The mount points are unmounted in the main process after it
detects the app process died.  By testing the `tmpfs: true`
first, it happens that main could start the `tmpfs: false`
test case before main cleans the old process mount points.
2021-01-07 16:51:27 +00:00
Max Romanov
d65a66f9d8 Libunit: processing single port message.
This partially reverts the optimisation introduced in 1d84b9e4b459 to avoid an
unpredictable block in nxt_unit_process_port_msg().  Under high load, this
function may never return control to its caller, and the external event loop
(in Node.js and Python asyncio) won't be able to process other scheduled
events.

To reproduce the issue, two request processing types are needed: 'fast' and
'furious'.  The 'fast' one simply returns a small response, while the 'furious'
schedules asynchronous calls to external resources.  Thus, if Unit is subjected
to a large amount of 'fast' requests, the 'furious' request processing freezes
until the high load ends.

The issue was found by Wu Jian Ping (@wujjpp) during Node.js stream
implementation discussion and relates to PR #502 on GitHub.
2020-12-29 19:01:24 +03:00
Max Romanov
d3d6864bdc Node.js: ServerRequest and ServerResponse compliance to Stream API.
ServerRequest now inherit stream Readable object.  ServerResponse
provides 'writable' property.

Thanks to Wu Jian Ping (@wujjpp).

This closes #274, closes #317 issues and closes #502 PR on GitHub.
2020-12-29 19:00:54 +03:00
Andrei Belov
3abca42caf Packages: fixed building for Ubuntu 16.04 "xenial".
Changes introduced in a27532e3a17b effectively broke building of the unit
package due to missed dh_installsystemd script in older debhelper 9.x.

Once Ubuntu 16.04 reach EOL, the following actions should be made:
 - this commit should be reverted;
 - minimal debhelper version should be increased to 11.
2020-12-28 12:51:30 +03:00
Andrei Belov
c981ac6558 Packages: fixed an ability to override package version.
This was broken since 00d8049418cf.
2020-12-24 16:17:27 +03:00
Andrei Belov
c369e25ae7 Version bump for unit modules.
This is required in order to build Debian packages from current tip
without making manual interventions.

Moving forward, this should be a part of every version bump commit.
2020-12-24 15:18:49 +03:00
Andrei Belov
c99b175988 Packages: set the "--user" value to "nobody" for build-stage tests.
This allows tests to pass well in cases when the default user specified in
the ./configure parameters does not exist in the building environment.
2020-12-23 18:31:18 +03:00
Andrei Belov
53af12def0 Tests: ability to run unitd with specified "--user" option. 2020-12-23 18:30:17 +03:00
Max Romanov
b7dba90062 Static: fixing request memory pool leakage in router.
When a static file larger than NXT_HTTP_STATIC_BUF_SIZE (128K) is served, two
buffers are allocated and chained; each retains the whole request memory pool.

Starting from 41331471eee7, the completion handler was called once for a linked
buffer chain, but the second buffer got lost.

This patch improves the completion handler's treatment of static buffers to
handle all linked buffers.
2020-12-23 11:01:36 +03:00
Valentin Bartenev
cac762ab7e Python: multiple values in the "path" option. 2020-12-22 17:53:41 +03:00
Konstantin Pavlov
65295c8141 Packages: check and create unit user on each post-script invocation. 2020-12-22 12:57:24 +03:00
Tiago Natel de Moura
03436d2ec2 Tests: introduced a separate cache directory for Go builds.
The Go compiler can't detect changes to C header files when
compiling CGO applications, and then this leads to Go test
samples being linked with wrong libunit.

This patch creates a new cache directory reused throughout
the test suite.
2020-12-21 11:00:46 +00:00
Max Romanov
7b669ed866 Libunit: fixed shared memory waiting.
The nxt_unit_ctx_port_recv() function may return the NXT_UNIT_AGAIN code, in
which case an attempt to reread the message should be made.

The issue was reproduced in load testing with response sizes 16k and up.
In the rare case of a NXT_UNIT_AGAIN result, a buffer of size -1 was processed,
which triggered a 'message too small' alert; after that, the app process was
terminated.
2020-12-18 00:25:28 +03:00
Max Romanov
7389a50835 Limiting app queue notifications count in socket.
Under high load, a queue synchonization issue may occur, starting from the
steady state when an app queue message is dequeued immediately after it has been
enqueued.  In this state, the router always puts the first message in the queue
and is forced to notify the app about a new message in an empty queue using a
socket pair.  On the other hand, the application dequeues and processes the
message without reading the notification from the socket, so the socket buffer
overflows with notifications.

The issue was reproduced during Unit load tests.  After a socket buffer
overflow, the router is unable to notify the app about a new first message.
When another message is enqueued, a notification is not required, so the queue
grows without being read by the app.  As a result, request processing stops.

This patch changes the notification algorithm by counting the notifications in
the pipe instead of getting the number of messages in the queue.
2020-12-18 00:25:27 +03:00
Max Romanov
c0449e13f8 Router: fixed crash in OOSM processing.
Multithreaded application may create different shared memory segments in
different threads.  The segments then passed to different router threads.
Because of this multithreading, the order of adding incoming segments is
not determined and there can be situation when some of the incoming segments
are not initialized yet.

This patch simply adds check for NULL to skip non-initialized segments.

Crash reproduced during load tests with high number of simultaneous
connections (1024 and more).
2020-12-17 19:27:44 +03:00
Andrei Belov
53d847615b Packages: run non-privileged processes under "unit" user. 2020-12-17 14:30:18 +03:00
Konstantin Pavlov
a5fa9673d9 Packages: added pcre2 to build depends.
While at it, propagate unit build depends to modules.
2020-11-24 14:27:06 +03:00
Konstantin Pavlov
9c09bc1772 Packages: dropped support for non-systemd distributions. 2020-11-24 13:25:09 +03:00
Andrei Belov
c33b10a41f Packages: eliminated debuild warning about debian/rules. 2020-12-16 09:56:04 +03:00
Andrei Belov
0c1abda391 Packages: the NOTICE file added to debian binaries.
This resolves the following lintian error:
https://lintian.debian.org/tags/missing-notice-file-for-apache-license.html

While here, changed upstream changelog name to conform with the policy:
https://lintian.debian.org/tags/wrong-name-for-upstream-changelog.html
2020-12-16 09:56:01 +03:00
Andrei Belov
d15acabfc5 Packages: introduced PYTEST_ARGS env variable for test targets.
This is useful for running particular tests, e.g.:
PYTEST_ARGS='test/test_respawn.py::TestRespawn::test_respawn_router' make test
2020-12-16 09:55:59 +03:00
Tiago Natel de Moura
ea5844ae7b Added a changelog for 5e6c2b8fb3fe. 2020-12-15 13:01:44 +00:00
Tiago Natel de Moura
66ac35e9b9 Tests: fixed bug that disabled isolation tests. 2020-12-15 11:06:49 +00:00
Valentin Bartenev
8d65a3303b Python: WSGI environment copying moved out of request processing.
The WSGI environment dictionary contains a number of static items, that are
pre-initialized on application start.  Then it's copied for each request to be
filled with request-related data.

Now this dictionary copy operation will be done between processing of requests,
which should save some CPU cycles during request processing and thus reduce
response latency for non-peak load periods.
2020-12-14 17:15:49 +03:00
Tiago Natel de Moura
1e9def50c8 Isolation: fixed unmounting when mnt namespace is in place.
The code had a wrong assumption that "mount namespaces" automatically
unmounts process mounts when exits but this happens only with
unprivileged mounts.
2020-12-14 12:00:28 +00:00
Andrei Zeliankou
425ffc9416 Tests: hyphen sign used unstead of underscore as more common. 2020-12-12 20:08:03 +00:00
Andrei Zeliankou
16ac829c8f Tests: added tests for PHP_AUTH_* variables. 2020-12-10 19:28:41 +00:00
Andrei Zeliankou
7be62c3c21 Tests: TestUnit class removed.
Prerequisite checks moved to the fixture in conftest.py.
2020-12-09 16:17:46 +00:00
Andrei Zeliankou
4c846ae693 Tests: isolation check moved to the pytest_sessionstart().
This change eliminates the need for some classes
to run Unit one more time before running tests.
2020-12-09 16:15:50 +00:00
Konstantin Pavlov
783cdc2a3d Docker: creating tags and pushing to AWS ECR as well. 2020-12-02 17:33:09 +03:00
Max Romanov
d3796d1fb7 Ruby: fixed crash on thread start.
Ruby threads need to be created with GVL; otherwise, an attempt to access
locked resources may occur, causing a crash.

The issue was occasionally reproduced on Ubuntu 18.04 with Ruby 2.5.1
while running test_ruby_application_threads.
2020-12-07 18:17:25 +00:00
Tiago Natel de Moura
c7bd96b476 Tests: fixed ruby isolation.
While alternating between running priv and unpriv tests
locally, it happens that unpriv tests can't bind mount
or create sub directories inside directories created
by root.  This patch fixes this by pointing "rootfs"
to temporary directory.

Now the priv and unpriv test uses the same test function.
2020-12-08 16:38:41 +00:00
Andrei Zeliankou
b2e767819f Tests: skip_alert() converted to the fixture. 2020-12-08 14:37:33 +00:00
Andrei Zeliankou
8f91628563 Tests: utils module introduced. 2020-12-08 14:37:25 +00:00
Andrei Belov
12a0d259a5 Docs: special handling for empty "date" and "time" XML attributes. 2020-12-08 13:17:21 +03:00
Valentin Bartenev
2348229dc7 PHP: populating PHP_AUTH_* server variables.
This closes #498 issue on GitHub.
2020-12-08 01:59:46 +03:00
Valentin Bartenev
bda76b04e9 HTTP: fixed status line format for unknown status codes.
According to Section #3.1.2 of RFC 7230, after the status code
there must be a space even if the reason phrase is empty.

Also, only 3 digits allowed.

This closes #507 issue on GitHub.
2020-12-07 18:50:56 +03:00
Max Romanov
db42527b1b Node.js: avoided use of request struct for debug logging.
This fixes a crash on exit of Node.js application.  The crash reproduced
on Ubuntu 20.10 with Node.js v15.1.0.  Tests 'test_node_websockets_two_clients'
and 'test_node_websockets_7_13_1__7_13_2'.

The reason of the crash is using request struct which was already freed.

The issue was introduced in 5be509fda29e.
2020-12-07 17:56:18 +03:00
Andrei Zeliankou
07789a23e9 Tests: options moved to the separate class.
This change is necessary to separate the logic
and prevent possible circular dependency.
2020-12-06 16:01:59 +00:00
Max Romanov
55296e6ff2 Node.js: removing unnecessary warnings.
Warnings changed for debug messages.
2020-11-30 23:30:20 +03:00
Valentin Bartenev
956fce6614 Libunit: improved error logging around initialization env variable. 2020-11-24 16:40:35 +03:00
Valentin Bartenev
35178dbb06 Version bump. 2020-11-24 16:40:28 +03:00
Valentin Bartenev
806a9b2515 Added tag 1.21.0 for changeset f804aaf7eee1 2020-11-19 21:13:19 +03:00
Valentin Bartenev
9e85732e9a Generated Dockerfiles for Unit 1.21.0. 2020-11-19 21:12:30 +03:00
Valentin Bartenev
fa1a3298ef Added version 1.21.0 CHANGES. 2020-11-19 19:59:18 +03:00
Axel Duch
e3af18834d Router: matching regular expressions support. 2020-11-17 15:03:30 +00:00
Max Romanov
2a381a82a6 Libunit: fixing read buffer leakage.
If shared queue is empty, allocated read buffer should be explicitly
released.

Found by Coverity (CID 363943).
The issue was introduced in f5ba5973a0a3.
2020-11-19 13:49:12 +03:00