Commit Graph

913 Commits

Author SHA1 Message Date
Max Romanov
c5cb2432c4 Fixing router connection pool leakage.
The connection's local socket address is allocated from the connection
pool before the request is passed to the application; however, with keep-alive
connections, this field was unconditionally reset by a socket configuration
value that could be NULL.  For the next request, the address was allocated
again from the same connection pool.  Nonetheless, all leaked addresses
were released when the connection was closed.

The issue introduced in changeset 5c7dd85fabd5.
2020-09-30 16:36:57 +03:00
Max Romanov
153e8a8779 Fixing leakage caused by incorrect in_hash flag cleanup.
Large-bodied requests are added to the request hash to be found when the body
arrives.  However, changeset 1d84b9e4b459 introduced a bug: the 'in_hash' flag,
used to remove the request from the hash at request release, was cleared after
the first successful request lookup.  As a result, the entry was never removed.
2020-09-30 01:17:09 +03:00
Valentin Bartenev
67d33fac66 MIME: added AVIF and APNG image formats.
AVIF is a modern image format based on the AV1 video codec.  It generally has
better compression than other widely used formats (WebP, JPEG, PNG, and GIF)
and is designed to supersede them.  Support was already added to the latest
version of Chrome.

APNG extends PNG to permit animated images that work similarly to animated GIF.
It's supported by most modern browsers.

Also removed duplicated ".svg" entry.
2020-09-29 23:23:32 +03:00
Max Romanov
61eba6eef1 Wrapping libunit's malloc() and free() calls for logging purposes.
This change aids heap usage analysis in applications.
The alloc and free functions are also required for lvlhash due to the upcoming
threading support, because using main nxt_memalign() and nxt_free() isn't safe
in a multithreaded app environment.  The reason is that these functions may use
thread-local structures which aren't initialized properly in applications.
2020-09-29 22:58:04 +03:00
Max Romanov
c721a5378d Fixing request buffer memory leakage in router.
The issue was introduced in changeset 1d84b9e4b459.  The request buffer was
transferred via the shared application queue, but the buffer position and the
'sent' flag were not updated after the buffer had been sent.
2020-09-29 22:57:56 +03:00
Max Romanov
c4b000f9cc Supporting HTTP/1.0 keep-alive.
The Apache HTTP server benchmarking tool, ab, issues HTTP/1.0 requests with
the 'Connection: Keep-Alive' header and expects a 'Connection: Keep-Alive'
header in the response.
2020-09-29 22:57:46 +03:00
Artem Konev
8ee96c224a Updated racially charged language in messages and comments. 2020-09-18 11:46:15 +01:00
Max Romanov
f16ae01b12 Python: app module callable name configuration.
Now it is possible to specify the name of the application callable using
optional parameter 'callable'.  Default value is 'application'.

This closes #290 issue on GitHub.
2020-09-18 13:41:58 +03:00
Igor Sysoev
6b9882fc14 Fixed segmentation fault during reconfiguration.
If idle connection was closed before h1proto had been allocated
then c->socket.data is NULL.  This happens if nxt_h1p_idle_response()
is called by nxt_h1p_idle_close().  However, h1p->conn_write_tail
is used only in nxt_h1p_request_send() that would not be called
after nxt_h1p_idle_response().

The bug was introduced in f237e8c553fd.
2020-09-18 13:20:06 +03:00
Igor Sysoev
6cfbf4ba79 Fixed segmentation fault during reconfiguration. 2020-09-18 13:20:05 +03:00
Igor Sysoev
09f4db1fc0 Fixed use-after-free error during reconfiguration.
An idle connection was not removed from idle connection list
if the connections detected that listening socket had been closed.
2020-09-18 13:20:02 +03:00
hongzhidao
806135f1c9 Router: fixed "pass" to upstreams.
Messed up return values in nxt_upstream_find() caused error in applying any
configuration with a valid "pass" value in router configuration pointing to
upstream.  That wasn't the case in "listeners" objects, where the return value
wasn't checked.

Also, it caused segfault in cases where the "pass" option was configured with
variables and resulting value was pointing to a non-existent upstream.

Added missing return checks as well to catch possible memory allocation errors.

The bug was introduced in d32bc428f46b.

This closes #472 issue on GitHub.
2020-08-28 00:53:36 -04:00
Tiago Natel de Moura
77ecb6ab49 Isolation: remove redundant macro. 2020-09-16 17:52:53 +01:00
Max Romanov
d5973fb557 Python: changed request headers format in router protocol.
The coming ASGI support requires raw HTTP headers format.  Headers grouping
and upcase code were moved to WSGI module.
2020-09-15 20:12:25 +03:00
Max Romanov
5163551ffe Hardening header names comparation for grouping. 2020-09-15 20:11:48 +03:00
Max Romanov
d94dac091f Python: split module initialization from WSGI implementation.
This is required for futher ASGI implementation.
2020-09-14 13:27:02 +03:00
Max Romanov
d483aa74e6 Python: source file moved to 'python' sub-directory.
No functional changes.  Get ready for an increase in file number.
2020-09-14 12:07:30 +03:00
Max Romanov
bd4ca6a057 Fixing WebSocket frame retain function.
Some of the pointers were not adjusted after frame's memory re-allocation.
Fortunately, this function was not used and the bug has no effect.
2020-09-10 12:16:32 +03:00
Igor Sysoev
22c88f0253 Upstream connection was not closed for short chunked response. 2020-09-07 15:21:14 +03:00
Valentin Bartenev
70c2a4645e Vars: added $host.
This closes #407 issue on GitHub.
2020-08-28 19:34:49 +03:00
Tiago Natel de Moura
b65a8636bb Isolation: added "automount" option.
Now it's possible to disable default bind mounts of
languages by setting:

  {
      "isolation": {
           "automount": {
               "language_deps": false
           }
     }
  }

In this case, the user is responsible to provide a "rootfs"
containing the language libraries and required files for
the application.
2020-08-25 15:25:51 +01:00
Tiago Natel de Moura
30a242aa3c PHP: added bind mounts for extensions directory. 2020-08-25 13:28:14 +01:00
Valentin Bartenev
dcfa92c161 Configuration: removed "reschedule_timeout" option.
It's not used since cbcd76704c90.

This option is a leftover from previous IPC between router and applications
processes.  It was never documented, though.

Thanks to 洪志道 (Hong Zhi Dao).
2020-08-21 20:50:04 +03:00
Tiago Natel de Moura
4ad8c5b898 Isolation: mount tmpfs by default. 2020-08-20 15:44:29 +01:00
Tiago Natel de Moura
a8a7eeb1fc Moved isolation related code to "nxt_isolation.c". 2020-08-20 15:22:58 +01:00
Valentin Bartenev
9bf6efc55a Configuration: improved error message of invalid listener address.
Thanks to 洪志道 (Hong Zhi Dao).
This closes #466 issue on GitHub.
2020-08-19 15:36:57 +03:00
Max Romanov
7ffc617ae8 Supporting platforms without sendfile() implementation.
This is a quick and dirty sendfile() replacement.

This closes #452 PR on GitHub.
2020-08-17 12:28:48 +03:00
Max Romanov
4ac7a6f55f Style: changing preprocessor directives.
Using #if directives instead of #ifdef the same way as in other places.
2020-08-17 12:28:40 +03:00
Max Romanov
b04b5ce430 Fixing router assertion in result of application prefork error.
Buffer for application prefork request allocated from temp conf mem_pool.
If error response from main process received before buffer completion handler,
temp conf mem_pool destroyed and router may crash in completion handler.

Assertion "src/nxt_buf.c:208 assertion failed: data == b->parent" triggered
when NXT_DEBUG_ALLOC enabled in configure.

This patch disables completion handler and memory allocated for buffer
released with memory pool.
2020-08-13 16:08:38 +03:00
Tiago Natel de Moura
b9ed3384cb Fixed error handling of prefork callback.
Previously, an error during the prefork phase triggered assert:

 src/nxt_port.c:27 assertion failed: port->pair[0] == -1

and resulted in exiting of the main process.

This could be easily reproduced by pushing a configuration with "rootfs",
when daemon is running without required permissions.
2020-08-13 12:25:52 +01:00
Valentin Bartenev
a58f224e26 Fixed typo in return value check.
Found by Coverity (CID 361277).
2020-08-13 03:45:54 +03:00
Valentin Bartenev
93146616cf Basic variables support. 2020-08-13 02:46:54 +03:00
Remi Collet
21ac95f17e PHP: compatibility with 8.0.0 Beta 1.
This closes #441 PR on GitHub.
2020-08-12 22:55:01 +03:00
Max Romanov
09685e2b41 Responding with error in case of first process start failure.
After shared application port introducing, request queue in router was
removed and requests may stuck forever waiting for another process start.
2020-08-12 15:25:29 +03:00
Max Romanov
2136eb411c Fixing issues found by static analyzer. 2020-08-12 13:37:49 +03:00
Max Romanov
fd2c01c58f Fixing return value initialization. 2020-08-11 21:48:46 +03:00
Max Romanov
f147943f63 Style fixes for 2 file descriptors transfer over port.
Two consecutive fd and fd2 fields replaced with array.
2020-08-11 21:48:27 +03:00
Max Romanov
acb0cca49d Moving file descriptor blocking to libunit.
The default libunit behavior relies on blocking the recv() call for port file
descriptors, which an application may override if needed.  For external
applications, port file descriptors were toggled to blocking mode before the
exec() call.  If the exec() call failed, descriptor remained blocked, so the
process hanged while trying to read from it.

This patch moves file descriptor mode switch inside libunit.
2020-08-11 21:48:16 +03:00
Max Romanov
8cf522bf2d Wrapping close() call in libunit for logging. 2020-08-11 19:20:36 +03:00
Max Romanov
e227fc9e62 Introducing application and port shared memory queues.
The goal is to minimize the number of syscalls needed to deliver a message.
2020-08-11 19:20:34 +03:00
Max Romanov
a82cf4ffb6 Circular queues implementations and a test.
- naive circular queue, described in the article "A Scalable, Portable, and
 Memory-Efficient Lock-Free FIFO Queue" by Ruslan Nikolaev:
https://drops.dagstuhl.de/opus/volltexte/2019/11335/pdf/LIPIcs-DISC-2019-28.pdf
- circular queue, proposed by Valentin Bartenev in the "Unit router application
IPC" design draft
2020-08-11 19:20:32 +03:00
Max Romanov
a1e9df2aef Port message extended to transfer 2 file descriptors. 2020-08-11 19:20:30 +03:00
Max Romanov
72475ee11c Made router port message handlers into static functions.
Mostly harmless.
2020-08-11 19:20:28 +03:00
Max Romanov
f4a118f84a Adding debug messages to catch process management issues. 2020-08-11 19:20:20 +03:00
Max Romanov
2f3d27fa22 Process structures refactoring in runtime and libunit.
Generic process-to-process shared memory exchange is no more required.  Here,
it is transformed into a router-to-application pattern.  The outgoing shared
memory segments collection is now the property of the application structure.
The applications connect to the router only, and the process only needs to group
the ports.
2020-08-11 19:20:17 +03:00
Max Romanov
8359560612 Introducing the shared application port.
This is the port shared between all application processes which use it to pass
requests for processing.  Using it significantly simplifies the request
processing code in the router.  The drawback is 2 more file descriptors per each
configured application and more complex libunit message wait/read code.
2020-08-11 19:20:15 +03:00
Max Romanov
6e31d6cd39 Changing router to application shared memory exchange protocol.
The application process needs to request the shared memory segment from the
router instead of the latter pushing the segment before sending a request to
the application.  This is required to simplify the communication between the
router and the application and to prepare the router for using the application
shared port and then the queue.
2020-08-11 19:20:13 +03:00
Max Romanov
3cbc22a6dc Changing router to application port exchange protocol.
The application process needs to request the port from the router instead of the
latter pushing the port before sending a request to the application.  This is
required to simplify the communication between the router and the application
and to prepare the router to use the application shared port and then the queue.
2020-08-11 19:20:10 +03:00
Max Romanov
bf647588ff Adding a reference counter to the libunit port structure.
The goal is to minimize the number of (pid, id) to port hash lookups which
require a library mutex lock.  The response port is found once per request,
while the read port is initialized at startup.
2020-08-11 19:20:06 +03:00
Max Romanov
ec3389b63b Libunit refactoring: port management.
- Changed the port management callbacks to notifications, which e. g. avoids
the need to call the libunit function
- Added context and library instance reference counts for a safer resource
release
- Added the router main port initialization
2020-08-11 19:19:55 +03:00