deflateReset starts a new member. Bytes already written out cannot be
taken back, so gunzip/inflate see a truncated member followed by a new
header. Refuse reset in that case. Raw deflate has no wrapper header
and is unchanged.
Signed-off-by: Xia Chao <shapirolutts@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/66179
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Resetting a ZstdCompress stream while a frame is still in
progress dropped the frame state, but any bytes already
written out stayed at the start of the output stream. The
next frame was then appended to that fragment, so the
resulting stream could not be decompressed. The failure was
silent: the compressor reported no error at all.
ZSTD_reset_session_only cancels unflushed internal data, so
refuse reset only when the incomplete frame has already
emitted output.
Signed-off-by: Xia Chao <shapirolutts@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/66088
Reviewed-By: James M Snell <jasnell@gmail.com>
Discovered when testing some other changes that zstd was not
handling certain chunk boundaries very well. This fixes it.
- Decodes concatenated and skippable zstd frames across all
chunk boundaries.
- Preserves rejectGarbageAfterEnd semantics.
- Reports corruption in subsequent frames.
- Handles trailing partial frame identifiers consistently.
- Covers classic sync/async and iterable APIs.
- Updated documentation and internal typings.
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
PR-URL: https://github.com/nodejs/node/pull/65865
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
This function call can fail with `Z_VERSION_ERROR` if the compiled
library vs loaded library mismatched in version number or in
stream structure size.
In those cases, zlib doesn't initialize the `strm_.msg` field to
null. Therefore, when a `CompressionError` object is created via
`ErrorForMessage()`, it can read a stale or uninitialized `strm_.msg`
pointer that will cause a crash.
Example ASAN report:
```
AddressSanitizer: SEGV on unknown address
#0 __strlen_avx2
string/../sysdeps/x86_64/multiarch/strlen-avx2.S:76
#1 strlen (/work/node/out/Debug/node+0x1a42ab7)
#2 v8::(anonymous namespace)::StringLength(char const*)
/work/node/out/../deps/v8/src/api/api.cc:7581:16
#3 v8::(anonymous namespace)::StringLength(unsigned char const*)
/work/node/out/../deps/v8/src/api/api.cc:7587:10
#4 v8::String::NewFromOneByte(v8::Isolate*,
unsigned char const*, v8::NewStringType, int)
/work/node/out/../deps/v8/src/api/api.cc:7677:3
#5 node::OneByteString(v8::Isolate*,
char const*, int, v8::NewStringType)
/work/node/out/../src/util-inl.h:166:10
#6 node::(anonymous namespace)::CompressionStream<
node::(anonymous namespace)::ZlibContext>
::EmitError(node::(anonymous namespace)
::CompressionError const&)
/work/node/out/../src/node_zlib.cc:565:7
#7 node::(anonymous namespace)::CompressionStream<
node::(anonymous namespace)::ZlibContext>
::CheckError()
/work/node/out/../src/node_zlib.cc:519:5
#8 node::(anonymous namespace)::CompressionStream<
node::(anonymous namespace)::ZlibContext>
::AfterThreadPoolWork(int)
/work/node/out/../src/node_zlib.cc:543:10
#9 node::ThreadPoolWork::ScheduleWork()
::'lambda'(uv_work_s*, int)
::operator()(uv_work_s*, int) const
/work/node/out/../src/threadpoolwork-inl.h:57:15
#10 node::ThreadPoolWork::ScheduleWork()
::'lambda'(uv_work_s*, int)
::__invoke(uv_work_s*, int)
/work/node/out/../src/threadpoolwork-inl.h:48:7
#11 uv__work_done /work/libuv-1.51.0/src/threadpool.c:330:5
#12 uv__async_io.part.0
/work/libuv-1.51.0/src/unix/async.c:208:5
```
Signed-off-by: ndossche <nora.dossche@ugent.be>
PR-URL: https://github.com/nodejs/node/pull/63476
Reviewed-By: Anna Henningsen <anna@addaleax.net>
The Reset() method did not check the write_in_progress_ flag before
resetting the compression stream. This allowed reset() to free the
compression library's internal state while a worker thread was still
using it during an async write, causing a use-after-free.
Add a write_in_progress_ guard to Reset() that throws an error if a
write is in progress, matching the existing pattern used by Close()
and Write().
PR-URL: TODO
Refs: https://hackerone.com/reports/3609132
PR-URL: https://github.com/nodejs/node/pull/62325
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
This is both valuable as a diagnostic tool and as a way to inform
the JS runtime about external allocations.
Currently, this is a feature only enabled in the statically linked
builds of zstd, so with `--shared-zstd`, we fall back to the
non-tracking variant.
PR-URL: https://github.com/nodejs/node/pull/61717
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
This aligns zstd streams with other compression libraries in this
regard, and enables releasing memory early when the stream ends in
JS instead of waiting for GC to clean up the wrapper object (which
is a problem that is exacerbated in the zstd context because we
do not track memory and report memory pressure to V8 yet).
PR-URL: https://github.com/nodejs/node/pull/61717
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Call `V8::ExternalMemoryAccounter::Update` instead of
`V8::ExternalMemoryAccounter::Increase` to report memory difference to
V8
Calling `V8::ExternalMemoryAccounter::Increase` with a signed integer on
32-bit platforms causes instances where GC inside GC takes place leading
to a crash in certain cases.
During GC, native objects are destructed. In destructor for
`CompressionStream` class used by zlib, memory release information is
passed onto `V8::ExternalMemoryAccounter::Increase()` instead of
`V8::ExternalMemoryAccounter::Decrease()` which triggers V8's memory
limits, thus triggering GC inside GC which leads to crash.
Bug initially introduced in commit
1d5d7b6eed
For full report see https://hackerone.com/reports/3302484
PR-URL: https://github.com/nodejs/node/pull/59623
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Adds optional dictionary support to zlib’s zstdCompress and
zstdDecompress APIs. This enables better compression ratios when the
dictionary matches expected input structure or content patterns.
The implementation allows passing a `dictionary` buffer through the
options object. Support was added to both streaming and convenience
methods. Tests and documentation were also updated to reflect this new
capability.
Fixes: https://github.com/nodejs/node/issues/59105
PR-URL: https://github.com/nodejs/node/pull/59240
Reviewed-By: Anna Henningsen <anna@addaleax.net>
The function AllocForBrotli prefixes the allocated memory with its
size, and returns a pointer to the region after it. This pointer can
however no longer be suitably aligned. Correct this by allocating
the maximum of the the size of the size_t and the max alignment.
On Arm 32bits the size_t is 4 bytes long, but the alignment is 8 for
some NEON instructions. When Brotli is compiled with optimizations
enabled newer GCC versions will use the NEON instructions and trigger
a bus error killing node.
see https://github.com/google/brotli/issues/1159
PR-URL: https://github.com/nodejs/node/pull/57727
Reviewed-By: Shelley Vohr <shelley.vohr@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
This patch exposes the crc32() function from zlib to user-land.
It computes a 32-bit Cyclic Redundancy Check checksum of `data`. If
`value` is specified, it is used as the starting value of the checksum,
otherwise, 0 is used as the starting value.
```js
const zlib = require('node:zlib');
const { Buffer } = require('node:buffer');
let crc = zlib.crc32('hello'); // 907060870
crc = zlib.crc32('world', crc); // 4192936109
crc = zlib.crc32(Buffer.from('hello')); // 907060870
crc = zlib.crc32(Buffer.from('world'), crc); // 4192936109
```
PR-URL: https://github.com/nodejs/node/pull/52692
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
- Use `final` to indicate the classes that we actually
instantiate
- Properly use `const` (and the necessary associated `const_cast`
for zlib because we don’t define `ZLIB_CONST` and allow shared
builds)
- Store the JS callback in an internal field rather than a `Global`
(which improves memory leak debugging capabilities, removes
a potential future memory leak footgun, and aligns the code
with the rest of the codebase more closely)
- Other minor C++ cleanup
PR-URL: https://github.com/nodejs/node/pull/42247
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Xia Chao