Commit Graph

1233 Commits

Author SHA1 Message Date
Tiago Natel de Moura
e207415a78 Introducing application prototype processes. 2021-11-09 15:48:44 +03:00
Tiago Natel de Moura
1de660b6df Changed nxt_process_* for reuse.
This enables the reuse of process creation functions.
2021-11-09 15:48:44 +03:00
Tiago Natel de Moura
ff6a7053f5 Introduced SCM_CREDENTIALS / SCM_CREDS in the socket control msgs. 2021-11-09 15:48:44 +03:00
Zhidao HONG
aee908bcbd Router: matching query string support.
The "query" option matches decoded arguments, including plus ('+') to
space (' ').  Like "uri", it can be a string or an array of strings.
2021-11-05 22:56:34 +08:00
Zhidao HONG
1260add0f5 HTTP: removed surplus check for r->args is not NULL. 2021-11-05 11:19:15 +08:00
Zhidao HONG
40ad333a9c Router: fixed nxt_http_route_arguments_parse().
A valid query string argument is a string of "key=value\[&key=value ...\]"
pairs with non-empty keys.  The fix removes invalid empty arguments.
2021-11-05 11:10:03 +08:00
Zhidao HONG
85ab3f5ab5 Configuration: improved matching pattern error messages. 2021-11-05 10:51:41 +08:00
Valentin Bartenev
b6f4b3afb1 Improved logging of app module load errors. 2021-11-02 17:34:23 +03:00
Max Romanov
bba97134e9 Moving request limit control to libunit.
Introducting application graceful stop.  For now only used when application
process reach request limit value.

This closes #585 issue on GitHub.
2021-10-28 17:46:54 +03:00
Max Romanov
803e037302 Python: creating and reusing asgi_add_reader() wrapper. 2021-10-28 17:46:52 +03:00
Max Romanov
86138113eb Adding explicit app reference to nxt_router_app_port_release().
port->app field is not thread safe and should be used in main thread only.
To release port after request processing, application reference should be
obtained from corresponding request descriptor.
2021-10-28 17:46:50 +03:00
Valentin Bartenev
1441f42b5d Fixed memleaks if PID checks fail in nxt_port_incoming_port_mmap().
Memory allocated for "mem" and "mmap_handler" leaked in that case.
Also removed one dead assigment of "hdr" pointer.
2021-10-27 20:37:34 +03:00
Valentin Bartenev
561dbeb98d Fixed a potential descriptor leak if mmap() failed. 2021-10-26 19:31:43 +03:00
Valentin Bartenev
7bf6253941 Custom implementation of Base64 decoding function.
Compared to the previous implementation based on OpenSSL, the new implementation
has these advantages:

 1. Strict and reliable detection of invalid strings, including strings with
    less than 4 bytes of garbage at the end;

 2. Allows to use Base64 strings without '=' padding.
2021-10-26 15:43:44 +03:00
Zhidao HONG
78a4063063 Removed unused declarations.
Declarations became unused after 6976d36be926.

No functional changes.
2021-10-12 10:32:17 +08:00
Zhidao HONG
5fa5b1464f Configuration: automatic migration to the new "share" behavior. 2021-10-09 10:44:31 +08:00
Artem Konev
cdaa8e2523 Fixed invalid call sequence in nxt_tls_ticket_key_callback().
The bug has been introduced in 0bca988e9541.
2021-10-08 13:44:14 +01:00
Valentin Bartenev
8db8330f84 Static: removed surplus assignment.
It's not needed after 69d823e5710a.
Found by Clang Static Analyzer.
2021-10-04 15:33:56 +03:00
Valentin Bartenev
9baf22e7bf Static: fixed possible descriptor leak introduced in a946d8cd7f8c. 2021-10-04 15:33:04 +03:00
Zhidao HONG
95e6535909 Static: multiple paths in the "share" option. 2021-10-01 10:03:55 +08:00
Zhidao HONG
c5220944d2 Static: variables in the "share" option.
This commit supports variable in the "share" option, the finding path to
file serve is the value from "share". An example:
{
    "share": "/www/data/static$uri"
}
2021-09-30 22:17:28 +08:00
Zhidao HONG
37144d6849 Static: variables in the "chroot" option. 2021-09-28 23:08:26 +08:00
Max Romanov
c07f3d3ff6 Fixed WebSocket connection hang issue after listener reconfigure.
Because the configuration values were read from the listener's configuration,
an established WebSocket connection was unable to work properly (i. e. stuck)
if the listener was removed.  The correct source of configuration values is the
request config joint.

This is related to issue #581 on GitHub.
2021-09-20 09:01:08 +03:00
Max Romanov
d21ebcce83 Fixing build with glibc 2.34.
Explicitly using the sysconf() call to obtain the minimum thread stack size
instead of the PTHREAD_STACK_MIN macro.

This closes #576 PR on GitHub.
2021-09-14 19:35:49 +03:00
Zhidao HONG
a336928e10 Router: refactored variable pass.
Since the "pass" option supports both strings and variables, a generic
nxt_var_t structure can be used in the configuration phase, and the "name"
field in actions is redundant.

No functional changes.
2021-09-07 21:13:44 +08:00
Zhidao HONG
9c1894bf77 Var: keeping raw variable string for debug. 2021-09-06 19:27:45 +08:00
Valentin Bartenev
dbc5a742fd TLS: refactored nxt_tls_ticket_key_callback().
Deduplicated code and improved style.
No functional changes.
2021-08-25 10:33:32 +03:00
Andrey Suvorov
e0aa132172 Added TLS session tickets support. 2021-08-17 16:52:32 -07:00
Zhidao HONG
48a9399f23 Introduced the generic API nxt_buf_dummy_completion().
No functional changes.
2021-08-12 17:39:00 +08:00
Zhidao HONG
598f1493f6 Log: renamed related variables "log" as "_log" to prevent conflicts. 2021-08-12 17:41:21 +08:00
Oisin Canty
ca373aaccd Router: client IP address replacement.
This commit introduces the replacement of the client address based on the value
of a specified HTTP header.  This is intended for use when Unit is placed
behind a reverse proxy like nginx or a CDN.

You must specify the source addresses of the trusted proxies.  This can be
accomplished with any valid IP pattern supported by Unit's match block:

["10.0.0.1", "10.4.0.0/16", "!192.168.1.1"]

The feature is configured per listener.

The client address replacement functionality only operates when there is a
source IP match and the specified header is present.  Typically this would be
an 'X-Forwarded-For' header.

{
    "listeners": {
        "127.0.0.1:8080": {
            "client_ip": {
                "header": "X-Forwarded-For",
                "source": [
                    "10.0.0.0/8"
                ]
            },
            "pass": "applications/my_app"
        },
    }
}

If a request occurs and Unit receives a header like below:

"X-Forwarded-For: 84.123.23.23"

By default, Unit trusts the last rightmost IP in the header, so REMOTE_ADDR
will be set to 84.123.23.23 if the connection originated from 10.0.0.0/8.

If Unit runs behind consecutive reverse proxies and receives a header similar
to the following:

"X-Forwarded-For: 84.123.23.23, 10.0.0.254"

You will need to enable "recursive" checking, which walks the header from
last address to first and chooses the first non-trusted address it finds.

{
    "listeners": {
        "127.0.0.1:8080": {
            "client_ip": {
                "header": "X-Forwarded-For",
                "source": [
                    "10.0.0.0/8"
                ]
                "recursive": true,
            },
            "pass": "applications/my_app"
        },
    }
}

If a connection from 10.0.0.0/8 occurs, the chain is walked.  Here, 10.0.0.254
is also a trusted address so the client address will be replaced with
84.123.23.23.

If all IP addresses in the header are trusted, the client address is set to
the first address in the header:

If 10.0.0.0/8 is trusted and "X-Forwarded-For: 10.0.0.3, 10.0.0.2, 10.0.0.1",
the client address will be replaced with 10.0.0.3.
2021-08-12 08:23:16 +00:00
Oisin Canty
73ea6a1c3a Introduced nxt_sockaddr_parse_optport() for addresses w/o ports. 2021-08-12 08:23:09 +00:00
Max Romanov
3580842d34 Python: fixing misprint in error message. 2021-08-09 10:15:00 +03:00
Oisin Canty
60cf139961 Router: fixed crash when matching an empty address pattern array.
A crash would occur when the router tried to match an
against an empty address pattern array.

The following configuration was used to reproduce the
issue:

{
    "listeners": {
        "127.0.0.1:8082": {
            "pass": "routes"
        }
    },
    "routes": [
        {
            "match": {
                "source": []
            },
            "action": {
                "return": 200
            }
        }
    ]
}
2021-08-05 16:00:01 +00:00
Zhidao HONG
d16cf04167 Router: fixed segmentation fault.
In the case that routes or upstreams is empty and the pass option is a variable.
If the resolved pass is routes or upstreams, a segment error occurred.
2021-08-02 12:30:38 +08:00
Max Romanov
db03dfad67 Fixed dead assignments.
Found by Clang Static Analyzer.
2021-08-03 13:59:27 +03:00
Max Romanov
fa9fb29be2 Application restart introduced.
When processing a restart request, the router sends a QUIT message to all
existing processes of the application.  Then, a new shared application port is
created to ensure that new requests won't be handled by the old processes of
the application.
2021-07-29 19:50:39 +03:00
Zhidao HONG
f3a1c1deb5 Router: split nxt_http_app_conf_t from nxt_http_action_t.
No functional changes.
2021-07-24 11:44:52 +08:00
Zhidao HONG
b47f1ac7ea Router: renamed nxt_http_proxy_create() as nxt_http_proxy_init().
No functional changes.
2021-07-26 15:00:46 +08:00
Zhidao HONG
a3df6efc8d Router: split nxt_http_static_conf_t from nxt_http_action_t.
No functional changes.
2021-07-23 09:14:43 +08:00
Andrey Suvorov
f965e358b6 Changing SNI callback return code if a client sends no SNI.
When a client sends no SNI is a common situation.  But currently the server
processes it as an error and returns SSL_TLSEXT_ERR_ALERT_FATAL causing
termination of a current TLS session.  The problem occurs if configuration has
more than one certificate bundle in a listener.

This fix changes the return code to SSL_TLSEXT_ERR_OK and the log level of a
message.
2021-07-22 11:23:48 -07:00
Andrey Suvorov
c37ff7ed0e Enabling configure TLS sessions.
To support TLS sessions, Unit uses the OpenSSL built-in session cache; the
cache_size option defines the number sessions to store.  To disable the feather,
the option must be zero.
2021-07-21 15:22:52 -07:00
Max Romanov
f27fbd9b4d Python: using default event_loop for main thread for ASGI.
Unit's ASGI implementation creates a new event loop to run an application for
each thread since 542b5b8c0647.  This may cause unexpected exceptions or
strange bugs if asyncio synchronisation primitives are initialised before the
application starts (e.g. globally).

Although the approach with a new event loop for the main thread is consistent
and helps to prepare the application to run in multiple threads, it can be a
source of pain for people who just want to run single-threaded ASGI
applications in Unit.

This is related to #560 issue on GitHub.
2021-07-20 10:37:54 +03:00
Max Romanov
dfbdc1c11a Python: fixing exceptions in Future.set_result for ASGI implementation.
An ASGI application can cancel the Future object returned by the receive()
call.  In this case, Unit's ASGI implementation should not call set_result()
because the Future is already handled.  In particular, the Starlette framework
was noted to cancel the received Future.

This patch adds a done() check for the Future before attempting a set_result().

This is related to #564 issue on GitHub.
2021-07-20 10:37:53 +03:00
Max Romanov
567545213d Python: fixing ASGI receive() issues.
The receive() call never blocks for a GET request and always returns the same
empty body message.  The Starlette framework creates a separate task when
receive() is called in a loop until an 'http.disconnect' message is received.

The 'http.disconnect' message was previously issued after the response header
had been sent.  However, the correct behavior is to respond with
'http.disconnect' after sending the response is complete.

This closes #564 issue on GitHub.
2021-07-20 10:37:50 +03:00
Max Romanov
daa051e7e7 Router: fixing assertion on app thread port handle.
A new application thread port message can be processed in the router after the
application is removed from the router.  Assertion for this case is replaced by
a condition to store the new thread port until receiving the stop notification
from the application process.
2021-07-19 16:23:13 +03:00
Oisin Canty
655e321075 Ruby: process and thread lifecycle hooks.
This feature allows one to specify blocks of code that are called when certain
lifecycle events occur.  A user configures a "hooks" property on the app
configuration that points to a script.  This script will be evaluated on boot
and should contain blocks of code that will be called on specific events.

An example of configuration:

{
    "type": "ruby",
    "processes": 2,
    "threads": 2,
    "user": "vagrant",
    "group": "vagrant",
    "script": "config.ru",
    "hooks": "hooks.rb",
    "working_directory": "/home/vagrant/unit/rbhooks",
    "environment": {
        "GEM_HOME": "/home/vagrant/.ruby"
    }
}

An example of a valid "hooks.rb" file follows:

File.write("./hooks.#{Process.pid}", "hooks evaluated")

on_worker_boot do
    File.write("./worker_boot.#{Process.pid}", "worker booted")
end

on_thread_boot do
    File.write("./thread_boot.#{Process.pid}.#{Thread.current.object_id}",
               "thread booted")
end

on_thread_shutdown do
    File.write("./thread_shutdown.#{Process.pid}.#{Thread.current.object_id}",
               "thread shutdown")
end

on_worker_shutdown do
    File.write("./worker_shutdown.#{Process.pid}", "worker shutdown")
end

This closes issue #535 on GitHub.
2021-07-02 12:57:55 +00:00
Oisin Canty
7d2bc04e39 Fixing crash during IPv6 text address generation.
When the textual representation of an IPv6 nxt_sockaddr_t was being
generated, a crash would occur if the address had a full IPv6 form:

f607:7403:1e4b:6c66:33b2:843f:2517:da27

This was caused by a variable that tracks the location of a
collapsed group ("::") that was not set to a sane default.  When
the address was generated, a group would be inserted when
it was not necessary, thus causing an overflow.

This closes #481 issue on GitHub.
2021-07-02 10:55:13 +00:00
Max Romanov
54bf3e1912 Deduplicating code for closing fds in nxt_port_send_msg_t. 2021-07-01 16:23:56 +03:00
Max Romanov
2ac9c627aa Fixing memory and descriptor leakage in case of port send failure.
In rare cases, when the destination process had finished running but no
notification of this was received yet, send could fail with an error, and the
send message structure with file descriptors could leak.

The leakage was periodically reproduced by respawn tests on FreeBSD 12.
2021-07-01 16:23:51 +03:00