Wasm-wc: Improve request buffer handling

When Unit receives a request, if the body of that request is greater
than a certain amount (16KiB by default) then it is written to a
temporary file.

When a language module goes to read the request body in such situations
it will end up using read(2).

The wasm-wasi-component language module was failing to properly read
request bodies of around 2GiB or more.

This is because (on Linux at least) read(2) (and other related system
calls) will only read (or write) at most 0x7ffff000 (2,147,479,552)
bytes, this is the case for both 32 and 64-bit systems.

Regardless, it's probably not a good idea doing IO in such large chunks
anyway.

This patch changes the wasm-wasi-component language module to read the
request buffer in 32MiB chunks (this matches the original 'wasm'
language module).

We are still limited to a 4GiB address space and can only upload files a
little under 4GiB.

Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This commit is contained in:
Andrew Clayton
2024-01-29 11:35:13 +00:00
parent 79c8177247
commit ac3a54d671

View File

@@ -495,14 +495,21 @@ impl NxtRequestInfo {
fn request_read(&mut self, dst: &mut BytesMut) { fn request_read(&mut self, dst: &mut BytesMut) {
unsafe { unsafe {
let rest = dst.spare_capacity_mut(); let rest = dst.spare_capacity_mut();
let amt = bindings::nxt_unit_request_read( let mut total_bytes_read = 0;
self.info, loop {
rest.as_mut_ptr().cast(), let amt = bindings::nxt_unit_request_read(
rest.len(), self.info,
); rest.as_mut_ptr().wrapping_add(total_bytes_read).cast(),
32 * 1024 * 1024,
);
total_bytes_read += amt as usize;
if total_bytes_read >= rest.len() {
break;
}
}
// TODO: handle failure when `amt` is negative // TODO: handle failure when `amt` is negative
let amt: usize = amt.try_into().unwrap(); let total_bytes_read: usize = total_bytes_read.try_into().unwrap();
dst.set_len(dst.len() + amt); dst.set_len(dst.len() + total_bytes_read);
} }
} }