02d1984c91
On GH, @tonychuuy reported an issue when using Units 'share' action they
would get the following error in the unit log
2024/01/15 17:53:41 [error] 49#52 *103 file "/var/www/html/public/vendor/telescope/app.css" has changed while sending response to a client
This would happen when trying to serve files over a certain size and the
requested file would not be sent.
This is due to a somewhat bogus check in
nxt_http_static_buf_completion()
I say bogus because it's not clear what the check is trying to
accomplish and the error message is not entirely accurate either.
The check in question goes like
n = pread(file->fd, buf, size, offset);
return n;
...
if (n != size) {
if (n >= 0) {
/* log file changed error and finish */
/* >> Problem is here << */
}
/* log general error and finish */
}
If the number of bytes read is not what we asked for and is > -1 (i.e
not an error) then it says the file has changed, but really it only
checks if the file has _shrunk_ (we can't get back _more_ bytes than we
asked for) since it was stat'd.
This is what happens
recvfrom(22, "GET /tfile HTTP/1.1\r\nHost: local"..., 2048, 0, NULL, NULL) = 82
openat(AT_FDCWD, "/mnt/9p/tfile", O_RDONLY|O_NONBLOCK) = 23
newfstatat(23, "", {st_mode=S_IFREG|0644, st_size=149922, ...}, AT_EMPTY_PATH) = 0
We get a request from a client, open the requested file and stat(2) it to
get the file size.
We would then go into a pread/writev loop reading the file data and
sending it to the client until it's all been sent.
However what was happening in this case was this (showing a dummy file
of 149922 bytes)
pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072, 0) = 61440
write(2, "2024/01/17 15:30:50 [error] 1849"..., 109) = 109
We wanted to read 131072 bytes but only read 61440 bytes, the above
check triggered and the file transfer was aborted and the above error
message logged.
Normally for a regular file you will only get less bytes than asked for
if the read call is interrupted by a signal or you're near the end of
file.
There is however at least another situation where this may happen, if
the file in question is being served from a network filesystem.
It turns out that was indeed the case here, the files where being served
over the 9P filesystem protocol. Unit was running in a docker container
in an Ubuntu VM under Windows/WSL2 and the files where being passed
through to the VM from Windows over 9P.
Whatever the intention of this check, it is clearly causing issues in
real world scenarios.
If it was really desired to check if the had changed since it was
opened/stat'd then it would require a different methodology and be a
patch for another day. But as it stands this current check does more
harm than good, so lets just remove it.
With it removed we now get for the above test file
recvfrom(22, "GET /tfile HTTP/1.1\r\nHost: local"..., 2048, 0, NULL, NULL) = 82
openat(AT_FDCWD, "/mnt/9p/tfile", O_RDONLY|O_NONBLOCK) = 23
newfstatat(23, "", {st_mode=S_IFREG|0644, st_size=149922, ...}, AT_EMPTY_PATH) = 0
mmap(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f367817b000
pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072, 0) = 61440
pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 18850, 61440) = 18850
writev(22, [{iov_base="HTTP/1.1 200 OK\r\nLast-Modified: "..., iov_len=171}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=61440}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=18850}], 3) = 80461
pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 69632, 80290) = 61440
pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192, 141730) = 8192
close(23) = 0
writev(22, [{iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=61440}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=8192}], 2) = 69632
So we can see we do two pread(2)s's and a writev(2), then another two
pread(2)s and another writev(2) and all the file data has been read and
sent to the client.
Reported-by: tonychuuy <https://github.com/tonychuuy>
Link: <https://en.wikipedia.org/wiki/9P_(protocol)>
Fixes: 08a8d1510 ("Basic support for serving static files.")
Closes: https://github.com/nginx/unit/issues/1064
Reviewed-by: Zhidao Hong <z.hong@f5.com>
Reviewed-by: Andrei Zeliankou <zelenkov@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
NGINX Unit
Universal Web App Server
NGINX Unit is a lightweight and versatile open-source server that has two primary capabilities:
- serves static media assets,
- runs application code in seven languages.
Unit compresses several layers of the modern application stack into a potent, coherent solution with a focus on performance, low latency, and scalability. It is intended as a universal building block for any web architecture regardless of its complexity, from enterprise-scale deployments to your pet's homepage.
Its native RESTful JSON API enables dynamic updates with zero interruptions and flexible configuration, while its out-of-the-box productivity reliably scales to production-grade workloads. We achieve that with a complex, asynchronous, multithreading architecture comprising multiple processes to ensure security and robustness while getting the most out of today's computing platforms.
Quick Installation
macOS
$ brew install nginx/unit/unit
For details and available language packages, see the docs.
Docker
$ docker pull unit
For a description of image tags, see the docs.
Amazon Linux, Fedora, Red Hat
$ wget https://raw.githubusercontent.com/nginx/unit/master/tools/setup-unit && chmod +x setup-unit
# ./setup-unit repo-config && yum install unit
# ./setup-unit welcome
For details and available language packages, see the docs.
Debian, Ubuntu
$ wget https://raw.githubusercontent.com/nginx/unit/master/tools/setup-unit && chmod +x setup-unit
# ./setup-unit repo-config && apt install unit
# ./setup-unit welcome
For details and available language packages, see the docs.
Running a Hello World App
Unit runs apps in a variety of languages. Let's consider a basic example, choosing PHP for no particular reason.
Suppose you saved a PHP script as /www/helloworld/index.php:
<?php echo "Hello, PHP on Unit!"; ?>
To run it on Unit with the unit-php module installed, first set up an
application object. Let's store our first config snippet in a file called
config.json:
{
"helloworld": {
"type": "php",
"root": "/www/helloworld/"
}
}
Saving it as a file isn't necessary, but can come in handy with larger objects.
Now, PUT it into the /config/applications section of Unit's control API,
usually available by default via a Unix domain socket:
# curl -X PUT --data-binary @config.json --unix-socket \
/path/to/control.unit.sock http://localhost/config/applications
{
"success": "Reconfiguration done."
}
Next, reference the app from a listener object in the /config/listeners
section of the API. This time, we pass the config snippet straight from the
command line:
# curl -X PUT -d '{"127.0.0.1:8080": {"pass": "applications/helloworld"}}' \
--unix-socket /path/to/control.unit.sock http://localhost/config/listeners
{
"success": "Reconfiguration done."
}
Now Unit accepts requests at the specified IP and port, passing them to the application process. Your app works!
$ curl 127.0.0.1:8080
Hello, PHP on Unit!
Finally, query the entire /config section of the control API:
# curl --unix-socket /path/to/control.unit.sock http://localhost/config/
Unit's output should contain both snippets, neatly organized:
{
"listeners": {
"127.0.0.1:8080": {
"pass": "applications/helloworld"
}
},
"applications": {
"helloworld": {
"type": "php",
"root": "/www/helloworld/"
}
}
}
For full details of configuration management, see the docs.
OpenAPI Specification
Our OpenAPI specification aims to simplify configuring and integrating NGINX Unit deployments and provide an authoritative source of knowledge about the control API.
Although the specification is still in the early beta stage, it is a promising step forward for the NGINX Unit community. While working on it, we kindly ask you to experiment and provide feedback to help improve its functionality and usability.
Community
-
The go-to place to start asking questions and share your thoughts is GitHub Discussions.
-
Our GitHub issues page offers space for a more technical discussion at your own pace.
-
The project map on GitHub sheds some light on our current work and plans for the future.
-
Our official website may provide answers not easily found otherwise.
-
Get involved with the project by contributing! See the contributing guide for details.
-
To reach the team directly, subscribe to the mailing list.
-
For security issues, email us, mentioning NGINX Unit in the subject and following the CVSS v3.1 spec.