Commit Graph

1089 Commits

Author SHA1 Message Date
Max Romanov
567545213d Python: fixing ASGI receive() issues.
The receive() call never blocks for a GET request and always returns the same
empty body message.  The Starlette framework creates a separate task when
receive() is called in a loop until an 'http.disconnect' message is received.

The 'http.disconnect' message was previously issued after the response header
had been sent.  However, the correct behavior is to respond with
'http.disconnect' after sending the response is complete.

This closes #564 issue on GitHub.
2021-07-20 10:37:50 +03:00
Max Romanov
daa051e7e7 Router: fixing assertion on app thread port handle.
A new application thread port message can be processed in the router after the
application is removed from the router.  Assertion for this case is replaced by
a condition to store the new thread port until receiving the stop notification
from the application process.
2021-07-19 16:23:13 +03:00
Oisin Canty
655e321075 Ruby: process and thread lifecycle hooks.
This feature allows one to specify blocks of code that are called when certain
lifecycle events occur.  A user configures a "hooks" property on the app
configuration that points to a script.  This script will be evaluated on boot
and should contain blocks of code that will be called on specific events.

An example of configuration:

{
    "type": "ruby",
    "processes": 2,
    "threads": 2,
    "user": "vagrant",
    "group": "vagrant",
    "script": "config.ru",
    "hooks": "hooks.rb",
    "working_directory": "/home/vagrant/unit/rbhooks",
    "environment": {
        "GEM_HOME": "/home/vagrant/.ruby"
    }
}

An example of a valid "hooks.rb" file follows:

File.write("./hooks.#{Process.pid}", "hooks evaluated")

on_worker_boot do
    File.write("./worker_boot.#{Process.pid}", "worker booted")
end

on_thread_boot do
    File.write("./thread_boot.#{Process.pid}.#{Thread.current.object_id}",
               "thread booted")
end

on_thread_shutdown do
    File.write("./thread_shutdown.#{Process.pid}.#{Thread.current.object_id}",
               "thread shutdown")
end

on_worker_shutdown do
    File.write("./worker_shutdown.#{Process.pid}", "worker shutdown")
end

This closes issue #535 on GitHub.
2021-07-02 12:57:55 +00:00
Oisin Canty
7d2bc04e39 Fixing crash during IPv6 text address generation.
When the textual representation of an IPv6 nxt_sockaddr_t was being
generated, a crash would occur if the address had a full IPv6 form:

f607:7403:1e4b:6c66:33b2:843f:2517:da27

This was caused by a variable that tracks the location of a
collapsed group ("::") that was not set to a sane default.  When
the address was generated, a group would be inserted when
it was not necessary, thus causing an overflow.

This closes #481 issue on GitHub.
2021-07-02 10:55:13 +00:00
Max Romanov
54bf3e1912 Deduplicating code for closing fds in nxt_port_send_msg_t. 2021-07-01 16:23:56 +03:00
Max Romanov
2ac9c627aa Fixing memory and descriptor leakage in case of port send failure.
In rare cases, when the destination process had finished running but no
notification of this was received yet, send could fail with an error, and the
send message structure with file descriptors could leak.

The leakage was periodically reproduced by respawn tests on FreeBSD 12.
2021-07-01 16:23:51 +03:00
Oisin Canty
830729a6c5 Ruby: improved logging of exceptions without backtraces.
If an exception was raised with a backtrace of zero length, the
nxt_ruby_exception_log() routine would return without logging the
exception class and message.  This commit fixes the issue.
2021-07-01 11:16:43 +00:00
Max Romanov
cfba69781a Fixing multiple TLS-enabled listeners initialization.
Because of the incorrect 'last' field assignment, multiple listeners with
a TLS certificate did not initialize properly, which caused a router crash
while establishing a connection.

Test with multiple TLS listeners added.

The issue was introduced in the c548e46fe516 commit.

This closes #561 issue on GitHub.
2021-07-01 13:56:40 +03:00
Zhidao HONG
c16123e749 Router: split nxt_http_return_conf_t from nxt_http_action_t.
No functional changes.
2021-05-24 16:15:42 +08:00
Max Romanov
1e3f7808b1 Node.js: improving and test packaging.
The patch removes the "files" section from package.json to avoid future issues
with missing files.  For package testing purposes, 'npm pack' is used instead
of plain 'tar' to simulate packaging more accurately.
2021-06-15 10:35:15 +03:00
Max Romanov
fd3558456e Node.js: packaging new loader.js and loader.mjs.
The files loader.js and loader.mjs (introduced in f85b85094541 and 3c551b9721df)
were added to the packaged files list.
2021-06-02 16:14:22 +03:00
Andrey Suvorov
3f7ccf142f Enabling SSL_CTX configuration by using SSL_CONF_cmd().
To perform various configuration operations on SSL_CTX, OpenSSL provides
SSL_CONF_cmd().  Specifically, to configure ciphers for a listener,
"CipherString" and "Ciphersuites" file commands are used:
https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html


This feature can be configured in the "tls/conf_commands" section.
2021-05-26 11:19:47 -07:00
Andrey Suvorov
3efffddd95 Fixing crash during TLS connection shutdown.
A crash was caused by an incorrect timer handler nxt_h1p_idle_timeout() if
SSL_shutdown() returned SSL_ERROR_WANT_READ/SSL_ERROR_WANT_WRITE.

The flag SSL_RECEIVED_SHUTDOWN is used to avoid getting SSL_ERROR_WANT_READ, so
the server won't wait for a close notification from a client.

For SSL_ERROR_WANT_WRITE, a correct timer handler is set up.
2021-05-26 11:11:58 -07:00
Oisin Canty
d67a0c8711 Static: handled unknown MIME types when MIME-filtering active. 2021-05-26 16:48:11 +00:00
Oisin Canty
81e31872e3 MIME: added PHP. 2021-05-26 16:48:05 +00:00
Max Romanov
24905c1a00 Fixing racing condition on listen socket close in router (v2).
This patch fixes a possible race between the nxt_router_conf_wait() and
nxt_router_listen_socket_release() function calls and improves the 7f1b2eaa2d58
commit fix.
2021-05-25 18:01:00 +03:00
Oisin Canty
2fe76afaa6 Configuration: generalized application "targets" validation. 2021-05-25 13:21:29 +00:00
Oisin Canty
c160ea11e4 Node.js: renamed "require_shim" to "loader". 2021-05-24 09:01:42 +00:00
Valentin Bartenev
539551c89f PHP: adopted "file_handle" to Zend API changes in 8.1.0-dev.
This fixes building module with the development version of PHP after the change:
c732ab400a
2021-05-21 14:41:35 +03:00
Oisin Canty
f60389a782 Python: support for multiple targets. 2021-05-20 13:02:45 +00:00
Oisin Canty
ead6ed999a Ruby: changing deprecated rb_cData to rb_cObject.
Ruby 3.0 deprecated rb_cData with the intention to remove it in release 3.1.
This commit changes references of rb_cData to rb_cObject.  This was done so we
can support distributions that package Ruby 3.0, such as Fedora 34.

We also need to call rb_undef_alloc_func because we're no longer deriving from
rb_cData.  This prevents unnecessary allocations.

See:
https://docs.ruby-lang.org/en/3.0.0/doc/extension_rdoc.html

"It is recommended that klass derives from a special class called Data
(rb_cData) but not from Object or other ordinal classes.  If it doesn't,
you have to call rb_undef_alloc_func(klass)."
2021-05-18 10:14:43 +00:00
Andrey Suvorov
19dfeba86b Fixing a crash after applying the wrong TLS configuration.
When an invalid TLS configuration is applied (such as the conf_commands
feature), nxt_cert_store_get() creates a buffer to send a certificate request
to the main process and adds its default completion handler to an asynchronous
queue to free the allocated buffer.  However, if configuration fails,
nxt_router_conf_error() removes the memory pool used to allocate the buffer,
causing a crash when the completion handler is dispatched.


Assertion "src/nxt_buf.c:208 assertion failed: data == b->parent" is triggered
when is NXT_DEBUG enabled in the configure script.


This patch uses a reference counter to retain the memory pool and redefines the
completion handler to free the buffer before releasing the memory pool.
2021-05-17 14:28:38 -07:00
Max Romanov
c216f26d30 Fixing racing condition on listen socket close in router.
Listen socket is actually closed in the instant timer handler.  This patch moves
the "configuration has been applied" notification to the timer handler to avoid
a situation when the user gets the response from the controller, but the listen
socket is still open in the router.
2021-05-17 17:34:15 +03:00
Oisin Canty
a0c083af20 Node.js: a shim for overriding "http" and "websocket" modules.
Also added stubs for Server.address()
This was done to prevent crashes in some popular frameworks like express

Supports both CommonJS and the new ES Modules system syntax e.g:

app.js:
const http = require('http')

app.mjs:
import http from "http"

Usage on Node 14.16.x and higher:
{
    "type": "external",
    "processes": {"spare": 0},
    "working_directory": '/project',
    "executable": "/usr/bin/env",
    "arguments": [
        "node",
        "--loader",
        "unit-http/require_shim.mjs"
        "--require",
        "unit-http/require_shim",
        "app.js"
    ]
}

Usage on Node 14.15.x and lower:
{
    "type": "external",
    "processes": {"spare": 0},
    "working_directory": '/project',
    "executable": "/usr/bin/env",
    "arguments": [
        "node",
        "--require",
        "unit-http/require_shim",
        "app.js"
    ]
}
2021-05-12 09:26:55 +00:00
Valentin Bartenev
b0e32bc015 PHP: forced initialization of $_SERVER in fastcgi_finish_request().
The "auto_globals_jit" PHP option postponed the initialization of the $_SERVER
global variable until the script using it had been loaded (e. g. via the
"include" expression).  As a result, nxt_php_register_variables() could be
called after fastcgi_finish_request() had finished the request and nulled
ctx->req, which thus caused a segmentation fault.
2021-05-07 07:46:25 +03:00
Oisin Canty
b9d5eb285a Static: implemented MIME filtering 2021-05-06 14:22:21 +00:00
Zhidao HONG
de631d8c36 Fixed format and arguments mismatches in error log messages. 2021-05-05 17:23:33 +08:00
Zhidao HONG
8bea2977bc Fixed building without openat2(). 2021-05-05 16:30:26 +08:00
Zhidao HONG
53279af5d4 Static: support for openat2() features.
Support for chrooting, rejecting symlinks, and rejecting crossing mounting
points on a per-request basis during static file serving.
2021-04-29 22:04:34 +08:00
Zhidao HONG
113afb09ea Router: grouped app and share fields in nxt_http_action_t.
This is a prerequisite for further introduction of openat2() features.
No functional changes.
2021-04-22 13:13:06 +08:00
Max Romanov
b8052b050e Fixing shm buffer leakage when sending over the port queue.
When the shm buffer is sent over the port queue, it needs to be completed
because it's sent over the port socket.
2021-03-25 16:55:16 +03:00
Valentin Bartenev
067c6096e2 Node.js: used distinct placeholder for version in "package.json".
This makes the "sed" instruction simpler and more portable, as the previous
variant didn't work well on BSD systems due to the "\s" metacharacter.

Thanks to Sergey A. Osokin <osa@FreeBSD.org.ru> for spotting this issue.

Also, this should prevent accidentally creating a version 1.0.0 package.
2021-03-25 16:15:03 +03:00
Max Romanov
9957a959df Releasing shm buffers for large body requests.
This fixes memory and shm file descriptor leakage that occurred when a large
request body was passed via shared memory.  The leakage was caught with the
"test_settings_body_buffer_size" test.  The main condition is the
"body_buffer_size" value exceeding 10 Mb (a shm segment).  Thus, the router was
forced to split the body into several shm segments, but these buffers were not
freed because of dummy completion handlers.
2021-03-25 14:16:30 +03:00
Andrey Suvorov
d2b0882d89 Added ability to configure multiple certificates on a listener.
The certificate is selected by matching the arriving SNI to the common name and
the alternatives names.  If no certificate matches the name, the first bundle in
the array is chosen.
2021-03-24 13:19:36 -07:00
Valentin Bartenev
699a3ea2eb Certificates: fixed in name attributes processing.
The idea is to put SAN after CN, but the previous version of the code
incorrectly assumed that CN was always present, which caused writes
outside the allocated object if there were no standard name attributes.
2021-03-24 16:55:47 +03:00
Valentin Bartenev
a6c6dcf5f7 Certificates: moved SAN processing to a separate function.
No functional changes.
2021-03-24 16:38:05 +03:00
Valentin Bartenev
f18a41c84b Certficates: fixed counting DNS SAN entries.
Previously, entries of any type were counted during object allocation
but only DNS type entries were actually processed.  As a result,
if some certificate entries had another type, returning information
about the certificate caused uninitialized memory access.
2021-03-24 16:38:05 +03:00
Max Romanov
f267dd0a8d Workaround for an OpenSSL bug about not closing /dev/*random.
This is a workaround for an issue in OpenSSL 1.1.1, where the /dev/random and
/dev/urandom files remain open after all listening sockets were removed:

 - https://github.com/openssl/openssl/issues/7419
2021-03-24 11:43:31 +03:00
Valentin Bartenev
b04832da84 Disabled logging alerts to syslog.
It feels to be causing more harm than good, because syslog() can be blocking,
which is even more critical under resource exhaustion conditions when some
alerts are expected.
2021-03-24 08:05:07 +03:00
Valentin Bartenev
99337728ed Fixed building the PHP 5 module with ZTS, broken by dab8544b5440.
This closes #525 issue on GitHub.
2021-03-15 15:03:32 +03:00
Valentin Bartenev
4c261a7ff8 Ruby: fixed encodings initialization.
The Ruby interpreter expects an explicit setlocale() call before initialization
to pick up character encodings in the "Encoding" class from the environment.

This closes #531 issue on GitHub.
2021-03-15 15:03:32 +03:00
Valentin Bartenev
b0a1266835 Fixed certificates loading on startup with some filesystems.
It appears that readdir() on Linux detects file types unreliably, always setting
the "d_type" field to DT_UNKNOWN for some less common filesystems.  As a result,
all files were skipped and no certificate bundles were found when the state
directory was located on such filesystems.

Skipping "." and ".." instead of any non-regular files should be enough, as no
other non-regular files normally appear in this directory.

This closes #368 issue on GitHub.
2021-03-15 14:57:31 +03:00
Valentin Bartenev
a1107e859b Fixed TLS connection shutdown on errors.
An immediate return statement on connection errors was mistakenly added to the
beginning of nxt_openssl_conn_io_shutdown() in ecd3c5bbf7d8, breaking the TLS
connection finalization procedure.  As a result, a TLS connection was left
unfinalized if it had been closed prematurely or a fatal protocol error had
occurred, which caused memory and socket descriptor leakage.

Moreover, in some cases (notably, on handshake errors in tests with kqueue on
macOS) the read event was triggered later and nxt_h1p_conn_error() was called
the second time; after the change in af93c866b4f0, the latter call crashed the
router process in an attempt to remove a connection from the idle queue twice.
2021-03-15 14:57:01 +03:00
Max Romanov
1950d0aee2 Closing app outgoing shared memory file descriptor.
This fixes file descriptor leakage in router.  Shared memory file used to
send data from router to application.  These files are shared among all
processes of same application and router keeps the opened file descriptor since
06017e6e3a5f commit.
2021-03-02 19:33:37 +03:00
Max Romanov
73ac0496fe Fixing warnings on Solaris.
pthread_t on Solaris is an integer type with size not equal to pointer size.
To avoid warnings, type casts to and from pointer needs to be done via
uintptr_t type.

This change originally proposed by Juraj Lutter <juraj@lutter.sk>.
2021-03-02 18:31:03 +03:00
Max Romanov
fddde539c9 Fixing NetBSD compatibility.
Instead of PTHREAD_STACK_MIN define, NetBSD requires to get minimum stack
size using sysctl(_SC_THREAD_STACK_MIN).

This change originally proposed by Juraj Lutter <juraj@lutter.sk>.
2021-03-02 18:30:34 +03:00
Max Romanov
75a5dcfc4e Fixing shared app queue unmap size.
Shared app queue takes more memory than port memory.  To unmap all memory pages
correct size need to be specified for munmap() call.  Otherwise 4 Mb memory
leaked on each configured application removal.

The issue was introduced in 1d84b9e4b459.
2021-02-03 23:23:28 +03:00
Max Romanov
b1685dbc76 Fixing possible NULL dereference.
For listen socket request reply port can be NULL if Router crashes immediately
after issuing the request.

Found by Coverity (CID 366310).
2021-02-03 23:23:17 +03:00
Max Romanov
8c88537e6e Using shared memory to pass configuration to main process.
This patch is required to remove fragmented messages functionality.
2021-02-03 23:23:06 +03:00
Valentin Bartenev
93ac087e96 Fixed building by GCC 10 with -flto and -O2.
This closes #467 issue on GitHub.
2021-02-01 18:55:49 +03:00