The GN build always compiles the bundled copies of brotli, c-ares,
HdrHistogram, llhttp, libuv, nghttp2, SQLite and zstd, so anyone
packaging a GN-built Node.js for a Linux distribution has no
counterpart to configure's --shared-* options.
Add node_shared_* args named after the GYP variables. When one is set,
that dependency's GN template defines its target as a group carrying a
pkg-config config instead of the bundled sources, so every existing
"deps/<name>" reference picks up the system library without further
changes, and config.gypi reports the choice like a GYP build does.
Refs: https://github.com/nodejs/node/pull/55903
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65797
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
fs.cp() and fs.promises.cp() walked the tree in JavaScript with several
thread pool round trips per entry (opendir batches, two stat()s, the
copyFile(), a chmod()), all awaited in sequence: a 2 100-file tree took
~215 ms with ~110 ms of that on the main thread, against ~36 ms for
fs.cpSync(), which copies the tree in C++ when no filter is given.
Factor that C++ walk into CopyDirRecursive(), which records the error
instead of throwing so that it can run on any thread, and run it as one
ThreadPoolWork request (CpDirJob) for fs.cp()/fs.promises.cp() when
the destination directory does not exist yet and nothing has to run per
entry (no filter, no dereference, permission model off). Copying into
an existing tree keeps the JavaScript walk and its rules for what may
already be there. The same tree now takes ~30 ms with under 1 ms on the
main thread.
For that job the walk follows the JavaScript walk's rules rather than
cpSync's: it creates every directory with mkdir() and every file with
an exclusive uv_fs_copyfile() (honouring the copyFile() mode flags) and
fails with EEXIST if anything has appeared in their place since the
JavaScript check, so it never opens or follows something it did not
create; sockets, FIFOs and unknown entries are reported back to
JavaScript, which rejects them with the same SystemErrors as before;
relative link targets are made absolute lexically as path.resolve()
does. cpSync keeps merging into existing directories, skipping special
files and canonicalizing link targets.
The walk now uses the error_code overloads of std::filesystem throughout
(directory iteration included), so an unreadable directory inside the
tree is reported as EACCES by both cp() and cpSync() instead of
terminating the process, which cpSync() has done since the walk moved
to C++. Filesystem errors raised inside the walk keep their codes, with
'cp' or 'copyfile'/'mkdir' as the syscall. With preserveTimestamps the
walk stamps each directory it filled, root included, so cp() keeps
preserving directory times and cpSync() without a filter now does too.
Both file copies set the destination's mode before writing the data,
which clears setuid and setgid, so the walk puts the source mode back
after each file it copied; the JavaScript walk already chmod()s there.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65488
Refs: https://github.com/nodejs/node/pull/58461
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Jake Yuesong Li <jake.yuesong@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
The C++ fast path that fs.cpSync() takes when no filter is given created
the destination directories with default permissions, so a 0700
directory came out of the copy as 0755 (with the default umask). The
JavaScript implementation, which fs.cp(), fs.promises.cp() and
fs.cpSync() with a filter still use, chmod()s every directory it
creates to the mode of its source, and so did cpSync before the port.
Set the source directory's permissions on each directory the copy
creates (the destination root included); directories that already
exist keep theirs, as before.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65488
Refs: https://github.com/nodejs/node/pull/58461
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Jake Yuesong Li <jake.yuesong@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
V8 defaults `v8_enable_sandbox` to on whenever the shared pointer
compression cage and the external code space are enabled, and that is
the configuration embedders that use the sandbox build with. Now that
the sandbox builds and passes the tests, follow that default for
`--experimental-pointer-compression-shared-cage` so the configuration
is reachable from `configure`. Multi-cage pointer compression builds
stay without it: there every IsolateGroup gets its own sandbox, and
`NodeArrayBufferAllocator` always allocates from the default one.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/62237
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
With V8_ENABLE_SANDBOX every ArrayBuffer backing store has to be
allocated inside the sandbox, so memory that Node.js or a library
allocated itself cannot be wrapped and has to be copied in. Several
places already special-cased this, each with its own #ifdef, but a
build with the sandbox enabled still failed to compile
(`kMaxSafeBufferSizeForSandbox`), aborted in `crypto.randomUUID()`
(`SecureBuffer` was UNREACHABLE) and threw from every caller of the
malloc-owning `Buffer::New()` (v8.serialize, string transcoding, IPC,
the public `Buffer::New(isolate, data, length)`), and trace_events,
SEA assets and FFI still wrapped outside memory unconditionally.
Add `AdoptIntoBackingStore()`, which wraps the memory as before in
regular builds and copies it into an isolate-allocated backing store,
running the deleter right away, when the sandbox is enabled, and use it
at all of those sites so the #ifdef lives in one place. The trace
category flag cannot be copied since JS polls it, so under the sandbox
`getCategoryEnabledBuffer()` returns nothing and lib falls back to
`isTraceCategoryEnabled()`; FFI zero-copy views throw
ERR_OPERATION_FAILED there for the same reason.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/62237
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
ffi.getInt8() through ffi.getFloat64(), ffi.setInt8() through
ffi.setFloat64(), ffi.toBuffer() and ffi.toArrayBuffer() return
undefined instead of throwing when a required argument is omitted, so a
call that read or wrote nothing cannot be told apart from one that read
a zero byte. All 22 helpers behave this way.
GetValidatedPointerAddress() and GetValidatedSize() already reject the
same argument when it is passed explicitly as undefined. The
args.Length() test in front of them short-circuits the call and returns
Nothing without scheduling an exception. These six are the only tests in
src/ where args.Length() can skip a call that throws; the only other
Length() tests that guard a call at all guard Buffer::HasInstance(),
which cannot throw. The remaining tests in this file guard an inline
predicate and throw in the branch, which is why setUint8(ptr) reports
"Expected an offset argument" while setUint8() reports nothing at all.
Drop those tests. FunctionCallbackInfo::operator[] returns Undefined for
an out-of-range index, which is exactly the value these validators
reject, so each missing argument now produces the error its explicit
undefined counterpart produces. The documentation already describes this
behavior: the signatures are ffi.getInt8(pointer[, offset]),
ffi.setInt8(pointer, offset, value) and
ffi.toBuffer(pointer, length[, copy]), and the getters are documented to
return a number or a bigint.
ExportBytes() carried the same two tests. They are unreachable through
the public API because exportBytes is not exported and its three callers
all validate len in JavaScript first, but they are the same shape.
Signed-off-by: Soul Lee <alus20x@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65500
Fixes: https://github.com/nodejs/node/issues/65499
Refs: https://github.com/nodejs/node/pull/62072
Refs: https://github.com/nodejs/node/pull/62858
Refs: https://github.com/nodejs/node/pull/65842
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Most node:bench tests record a sample with `b.start();
process.hrtime.bigint(); b.end(1)`, counting on the extra clock read to
make end() see a later timestamp than start(). On a debian12-x64 CI
host the monotonic clock is coarse enough that all three reads returned
the same value, so end() threw ERR_INVALID_STATE ("insufficient clock
precision for benchmark sample") and test-bench-harness-errors failed;
every test using the idiom can fail the same way there.
Add test/common/bench.js with completeSample(), which spins until
process.hrtime.bigint() moves between start() and end(), and use it at
all of those sites, bench-runner fixtures included.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65780
Fixes: https://github.com/nodejs/node/issues/64226
Refs: https://github.com/nodejs/node/issues/59146
Refs: https://github.com/nodejs/node/issues/54534
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
test-wasi-pthread fails now and then on every platform with
"Assertion failed: r == 0 (c/pthread.c: main: 17)", i.e.
pthread_create() itself reporting an error. The fixture implements
`thread-spawn` by starting a Worker and blocking in
Atomics.wait(result, 0, 0, 1000) until the worker signals that it has
instantiated the module. Two things go wrong there: the worker signals
success by storing 0, the value the main thread is already waiting on,
so when the worker is quicker than the main thread its notify is lost
and the wait runs into the timeout; and one second is not always enough
for a Worker to start and instantiate a threads build on the slower CI
hosts (arm debug, Windows, macOS). Either way spawn() returns -6 and
wasi-libc turns that into a pthread_create() failure.
Wait on a sentinel value that neither outcome writes, and give the
worker a platform-scaled 30 seconds. Drop the flaky markers.
Fixes: https://github.com/nodejs/node/issues/64226
Refs: https://github.com/nodejs/node/issues/59146
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65780
Refs: https://github.com/nodejs/node/issues/54534
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
test-run-watch-emit-restarted expected exactly one test:watch:restarted
event, but it starts run({ watch: true }) right after writing the
fixtures into the watched directory. Watch backends that deliver events
with some latency, FSEvents on macOS most visibly, can still report
those setup writes once the first run is under way, which restarts it
and makes the later, intentional write the second restart. The test has
been marked flaky on macOS x64 for that reason.
Wait for the first drain, then require that the write is followed by a
restart and a drain, ignoring whatever the setup produced before it, and
drop the flaky marker.
Refs: https://github.com/nodejs/node/issues/54534
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65780
Fixes: https://github.com/nodejs/node/issues/64226
Refs: https://github.com/nodejs/node/issues/59146
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Sending a dgram socket over IPC duplicates the handle instead of moving
it: `dgram.Socket` has no `postSend` in the child_process conversion
table, and `bind()` calls `recvStart()` unconditionally, so the parent
copy keeps consuming datagrams after the child adopts the handle. Both
sockets then compete for every packet, and the child receives nothing
until it wins that race. Locally the parent took up to 111 consecutive
packets first, stretching the run to about 1.2s; the retry interval is
the only thing keeping the test alive, and how long the parent can hold
the socket is a property of the scheduler, not of the test.
Close the parent copy as soon as the child reports it is receiving, so
the child is the only reader by the time the first datagram is sent.
Signed-off-by: Christian Aurich <christian.aurichzm@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65767
Refs: https://github.com/nodejs/reliability/issues/1642
Refs: https://github.com/nodejs/reliability/issues/1649
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
The test allocates a buffer one byte longer than MAX_STRING_LENGTH and
expects toString('utf8') to throw ERR_STRING_TOO_LONG. MAX_STRING_LENGTH
counts UTF-16 units rather than bytes, so that only holds if every byte
decodes to a single unit. For the two Buffer.allocUnsafe variants the
contents are whatever the allocator hands back, and a single multi-byte
UTF-8 sequence anywhere in the buffer decodes to fewer units than it
occupies bytes, bringing the result down to MAX_STRING_LENGTH or less
so that nothing is thrown.
This goes unnoticed when the allocator returns zeroed pages, which is
why the test passes elsewhere. The AIX builders expose non-zero
contents and fail with "Missing expected exception (Error)".
Refs: https://github.com/nodejs/reliability/issues/1649
Signed-off-by: Christian Aurich <christian.aurichzm@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65755
Refs: https://github.com/nodejs/reliability/issues/1648
Refs: https://github.com/nodejs/reliability/issues/1649
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Chengzhong Wu