Read buffers for streams that emit their data to JS were allocated per
read: a 64KB backing store, tracked in a map, and then - since reads
rarely fill the whole buffer - reallocated to the right size and copied.
Allocate read buffers from a 64KB slab instead. Reads reserve the
suggested size from the slab and JS receives a view over the slab's
ArrayBuffer at the read's offset, using the offset mechanism that
onStreamRead already supports. Unused reservation space is rewound when
a read returns less than was reserved, so small reads (e.g. TLS records)
share a slab.
This removes the per-read allocations, the map bookkeeping and the
resize copy.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/64455
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
fs.cp() and fs.promises.cp() walked the tree in JavaScript with several
thread pool round trips per entry (opendir batches, two stat()s, the
copyFile(), a chmod()), all awaited in sequence: a 2 100-file tree took
~215 ms with ~110 ms of that on the main thread, against ~36 ms for
fs.cpSync(), which copies the tree in C++ when no filter is given.
Factor that C++ walk into CopyDirRecursive(), which records the error
instead of throwing so that it can run on any thread, and run it as one
ThreadPoolWork request (CpDirJob) for fs.cp()/fs.promises.cp() when
the destination directory does not exist yet and nothing has to run per
entry (no filter, no dereference, permission model off). Copying into
an existing tree keeps the JavaScript walk and its rules for what may
already be there. The same tree now takes ~30 ms with under 1 ms on the
main thread.
For that job the walk follows the JavaScript walk's rules rather than
cpSync's: it creates every directory with mkdir() and every file with
an exclusive uv_fs_copyfile() (honouring the copyFile() mode flags) and
fails with EEXIST if anything has appeared in their place since the
JavaScript check, so it never opens or follows something it did not
create; sockets, FIFOs and unknown entries are reported back to
JavaScript, which rejects them with the same SystemErrors as before;
relative link targets are made absolute lexically as path.resolve()
does. cpSync keeps merging into existing directories, skipping special
files and canonicalizing link targets.
The walk now uses the error_code overloads of std::filesystem throughout
(directory iteration included), so an unreadable directory inside the
tree is reported as EACCES by both cp() and cpSync() instead of
terminating the process, which cpSync() has done since the walk moved
to C++. Filesystem errors raised inside the walk keep their codes, with
'cp' or 'copyfile'/'mkdir' as the syscall. With preserveTimestamps the
walk stamps each directory it filled, root included, so cp() keeps
preserving directory times and cpSync() without a filter now does too.
Both file copies set the destination's mode before writing the data,
which clears setuid and setgid, so the walk puts the source mode back
after each file it copied; the JavaScript walk already chmod()s there.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65488
Refs: https://github.com/nodejs/node/pull/58461
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Jake Yuesong Li <jake.yuesong@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Avoid repeating EC key-pair validation after a pair has already been
established or validated. Invalidate the positive-only cache whenever
public-key mutation can make the pair inconsistent.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65615
Reviewed-By: James M Snell <jasnell@gmail.com>
Readable.fromWeb() and the read side of Duplex.fromWeb() allocated a
promise, a read-result object, and two reaction closures for every
chunk through reader.read(); a single reused read request now delivers
chunks through readableStreamDefaultReaderRead() instead, forwarding
each chunk in a microtask to preserve the delivery order relative to
errors and destroy.
Writable.fromWeb() and the write side of Duplex.fromWeb() paid two
derived promises off writer.ready plus the writer.write() promise and a
fresh closure pair per chunk; a single shared write request now
dispatches chunks directly and settles the node callback, with failures
delivered in a microtask.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/65548
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: James M Snell <jasnell@gmail.com>
With source maps enabled, every module that carries a sourceMappingURL
had its map decoded (or read from disk), JSON-parsed and its sources
resolved to absolute URLs while the module was being loaded, and the
per-line length table used for coverage was built with a per-code-point
loop. None of that is needed unless a stack trace is later mapped.
Keep the URL on the cache entry and resolve the payload on the first
findSourceMap() for that file. Under NODE_V8_COVERAGE the payload is
still resolved at load time, since the cache is serialized during
shutdown. lineLengths() now splits on '\n' with indexOf and only falls
back to the code point walk when the source contains U+2028/U+2029.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65490
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Default parameters are evaluated before the function body, so the
tracing context is allocated before the no-subscriber fast path.
Initialize it only for subscribed traces. Preserve method arity with an
undefined default and preserve explicit null behavior.
Add focused regression coverage and reusable TracingChannel benchmarks.
Signed-off-by: Romain Lanz <romain.lanz@pm.me>
PR-URL: https://github.com/nodejs/node/pull/65513
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
With this change, the TLS pause mechanisms used in node:tls are now made
generic, and used in QUIC to pause the handshake entirely for the
session event, thereby stopping almost all events being emitted
(everything except qlog) so that all the previously existing deferral
machinery is unnecessary.
This is useful immediately to clean up & simplify things, but also
medium term because this makes the 0RTT flow significantly clearer,
which will help with the upcoming dynamic attach for QUIC (which
requires deferring ticket processing & 0RTT delivery until after
user code).
As part of this, this changes the internal flow of how ALPN & SNI are
resolved. Instead of calculating them synchronously within the OpenSSL
callbacks, we precalculate them at the pause, and then the OpenSSL
callbacks just return the precalculated values.
This is intended to only create visible changes to node:quic. Though it
touches node:tls, it preserves all existing behaviour (including
callback order and other details).
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65522
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
The JavaScript recursive watcher used on platforms without a native one
(notably Linux) armed an fs.watch() handle and a stat() for every file
in the tree, and answered every event by stat()ing and re-reading the
whole directory it happened in. A 13k-entry tree cost 13.5k inotify
watches, ~150 ms and ~50 MB to set up, and appending to one file in a
3900-entry directory cost ~9 ms of CPU per event. A file replaced by
rename() (the usual editor save) also stopped being reported, since its
watch stayed on the old inode.
inotify reports changes to the entries of a watched directory, with
their names, so on Linux watch each directory once (symbolic links keep
their own watcher, as before), keep the set of known paths, and resolve
an event with a single stat() of the named entry: unknown names are
added and reported as 'rename', vanished ones are dropped and reported
as 'rename', file changes are reported as 'change'. kqueue and event
ports only report that a directory changed, so on the other platforms
served by this fallback every file keeps its own watcher and a
directory event rescans that directory, as before. unref() and ref()
now reach the underlying handles.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65486
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Enumerate usable ciphers and aliases from activated OpenSSL 3 providers
instead of maintaining lists of provider-only algorithms. Skip numeric
OID aliases and filter NULL, TLS composite, multiblock, and
encrypt-then-MAC implementations that the Cipher APIs cannot use.
Preserve the OpenSSL 1.1.1 and BoringSSL paths.
Expose CBC-CTS, SM4-GCM, SM4-CCM, SM4-XTS, and additional AES key wrap
implementations. Add `ctsMode` (CS1/CS2/CS3) and `xtsStandard`
(GB/IEEE) options for selecting provider CTS and SM4-XTS variants.
Keep ordinary cipher construction on the original binding and legacy
lookup paths.
Lazily cache successful provider fetches per Environment for
string initialization and `getCipherInfo()`.
Index entries by case-insensitive query, canonical, and alias names.
Deduplicate owners by provider and canonical identity. Return borrowed
pointers on warm hits.
Use the shared process-wide FIPS-state generation to invalidate
per-Environment cipher caches and refresh `getCiphers()` snapshots in
the main thread and workers. Existing cipher contexts retain their
implementation and can finish across a transition. Release provider
owners before unloading worker addon DSOs.
Enforce one-shot updates for CBC-CTS, AES key wrap, SIV/GCM-SIV, and
CCM decryption. Reject finalization without required input or CCM tags,
and defer authentication failures to `final()`. Document streaming and
XTS data-unit constraints.
Add known-answer vectors, option validation, provider round trips,
cache, worker, snapshot, FIPS transition, and construction benchmark
coverage.
Fixes: https://github.com/nodejs/node/issues/43040
Fixes: https://github.com/nodejs/node/issues/64866
Refs: https://github.com/nodejs/node/issues/62982
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65484
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
readdir({ recursive: true }) asked the binding for names only and then
called internalModuleStat() on every entry to find the directories to
descend into; with withFileTypes it built the Dirents and still stat()ed
every entry that was not already a directory. Both variants also ran
path.join() and path.relative() per entry to build the relative result.
Ask the binding for file types in all cases, descend into directories
directly, and only stat() symbolic links and entries of unknown type
(which is what could point to a directory). The relative name is the
parent's prefix plus the entry name. Results, their order and the
symlink-following behavior are unchanged for fs.readdirSync, fs.readdir
and fs.promises.readdir.
The known_issues test for Buffer paths (#58892) called back without
checking the error; the error now reaches the callback instead of being
thrown from the completion handler, so the test asserts success to keep
expressing the issue.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65487
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
The encode-and-enqueue transform walked the chunk code unit by code
unit, materializing a single-character string per index and building
the output with string concatenation. The only state that crosses
chunks is a trailing high (leading) surrogate, and TextEncoder.encode's
USVString conversion already replaces every interior lone surrogate
with U+FFFD, which is exactly what the spec loop produces. Join a
pending high surrogate with the incoming chunk, hold back a new
trailing high surrogate, and encode the rest in a single native call.
The streaming decode path also reuses a single options object instead
of allocating { stream: true } per chunk.
An encoding-streams benchmark is added since the suite had no
TextEncoderStream/TextDecoderStream row. Encoding improves by ~546%
with 1KB string chunks and ~20% with 16-character chunks; decode is
unchanged.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/65414
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Mattias Buelens <mattias@buelens.com>
Add an --analyze flag that performs statistical analysis directly
after benchmarks complete, eliminating the need for R and compare.R.
When --analyze is specified, compare.js collects the rate data during
the run and prints a statistical summary table instead of CSV output.
The table matches the format of compare.R: improvement percentage,
significance stars (* p<0.05, ** p<0.01, *** p<0.001), and confidence
intervals at three risk levels.
Also adds a --max-regression N option that causes the compare.js to
exit with 1 (error) when the `--new` is N% slower. Useful for CI
use to detect regressions.
Uses the histogram API's welchTest() and cohensD() methods introduced
in the previous commit. Benchmark rates are scaled to integers for
HdrHistogram recording; the --scale option (default 1000) controls
the multiplier for precision.
Usage:
node benchmark/compare.js --old ./node-old --new ./node-new \
--analyze url
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode/Opus
PR-URL: https://github.com/nodejs/node/pull/65416
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
The MIME regex used for data: URLs could backtrack super-linearly on a
malformed URL lacking a ',' separator. Make the optional parameter
group anchored on ';' so it cannot overlap with the media-type group.
Adds a benchmark (esm/get-data-protocol-format) parameterized over
path length, so a backtracking regression shows up as an ops/sec
cliff instead of a wall-clock assertion in a test, per review.
Fixes: https://github.com/nodejs/node/issues/61904
Signed-off-by: skdas20 <skdas5405@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/61951
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
The spec's [[backpressureChangePromise]] is only ever observed by the
transform source pull algorithm (settles when backpressure next becomes
true) and by a sink write arriving under backpressure (settles when it
next becomes false), both internal. Replace the promise record with
direct continuation delivery: a parked pull is completed by enqueueing
the readable controller's pull-fulfilled step on the shared resolved
promise, and a parked write by a cached per-stream continuation that
resolves the sink promise with the perform-transform promise, so
adoption reproduces the previous derived-chain settle depth exactly.
Each delivery lands at the same microtask position as the old record's
reaction.
transformStreamDefaultControllerPerformTransform now mirrors the
reference implementation's promiseCall().then(undefined, rejection
steps) directly instead of running an async wrapper pair per chunk: the
transformer.transform callback is wrapped raw, a non-thenable result
reuses the shared resolved promise, and a synchronous throw is
delivered through a rejected promise, keeping the error timing inside a
reaction.
A pipe-through benchmark is added since the suite had no
transform-throughput row.
pipeThrough passthrough improves by ~8% and a writer-driven transform
loop by ~14%; the pipe-to and read families are unchanged.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/65143
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
StringBytes::Write() already used simdutf to encode one-byte strings
as UTF-8 but sent every two-byte (UTF-16) string through
v8::String::WriteUtf8V2(), which is several times slower. That path
is behind Buffer.from(string), buf.write(), fs.write*() with string
data and every string written to a libuv stream, and JSON.stringify()
output is a two-byte string as soon as any value in the payload is
outside Latin-1.
Encode two-byte strings with simdutf as well whenever their UTF-8 form
is guaranteed to fit in the target: well-formed input is converted
directly, and input with unpaired surrogates is converted from a copy
passed through simdutf::to_well_formed_utf16(), which replaces each
unpaired surrogate with U+FFFD exactly like kReplaceInvalidUtf8 (this
mirrors what TextEncoder already does). Writes that have to truncate
at a character boundary keep using WriteUtf8V2(), so their output is
byte-for-byte unchanged, and so do strings of up to 32 code units, for
which V8 is already as fast (the same threshold TextEncoder uses).
buf.write() of a 2 KiB two-byte string improves ~5x (astral-heavy and
lone-surrogate strings ~3.5x and ~5x), Buffer.from() of a 64 KiB JSON
string ~2.7x; one-byte strings are unaffected.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65324
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
The highWaterMark values were passed as properties of the underlying
source and sink dictionaries, where they are ignored: a queuing
strategy's highWaterMark is read from the constructors' second argument.
Every configuration therefore measured the identical workload at the
default highWaterMark of 1, which also explains the historically high
run-to-run variance of this benchmark family.
Pass the strategies as the constructors' second argument and cover the
default (1) alongside buffered (1024, 4096) configurations.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/65138
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
AllocateCodeNear() searched 1024 pages in each direction for a free
page next to the native target, one MAP_FIXED_NOREPLACE mmap per
candidate. Every probe is expected to fail, so an exhausted window cost
up to 2048 failing syscalls before taking the plain-mmap fallback that
was already there, all to enable a jmp rel32 instead of movabs+jmp.
Resolving a fast-eligible signature plateaued at ~790us, against ~28us
for one that allocates no trampoline.
Reduce the window to 16 pages per direction, bounding the exhausted
case to 32 probes while still covering typical per-library symbol
counts. Placement stays opportunistic: the caller already emits the
absolute form when EmitJmpRel32() reports the target is out of range.
Signed-off-by: Trivikram Kamat <16024985+trivikr@users.noreply.github.com>
Assisted-by: claude:opus-5
PR-URL: https://github.com/nodejs/node/pull/64969
Fixes: https://github.com/nodejs/node/issues/64968
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
* fix duplicate address insertion in SocketAddressBlockList
* fix BlockList rule listing order to match apply
* add minor bound check in BlockList
* improve performance of BlockList apply
* eliminating shared_ptr
* check fast api path
* add clear method to BlockList
* general storage improvements to BlockList
* use shared locks for BlockList reads
* add bulk address adding to BlockList
* add BlockList benchmark
* add remove range/subnet to BlockList
* add cidr notation parsing to BlockList
* add additional apis to BlockList
* add private subnet presets to BlockList
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: OpenCode/Opus
PR-URL: https://github.com/nodejs/node/pull/64974
Reviewed-By: Tim Perry <pimterry@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
The byte controller's [[pendingPullIntos]] list was still a plain array
consumed with ArrayPrototypeShift, while every other per-chunk queue in
the WHATWG streams implementation has moved to the Queue ring buffer.
BYOB reads push and shift one descriptor per read, and Array.prototype
shift has real per-call cost even at length 1.
Back the descriptor list with the same lazily materialized Queue used
for the request queues, so constructing a byte stream still allocates
no descriptor storage.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/64818
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Fix broken parsing for --cv-threshold and --max-increases so the
value after the equals sign is used.
Use the configured CV threshold when checking overall stability and
report the configured thresholds in calibrate-n output. Also handle
invalid numeric values for --runs and --cv-threshold consistently
with the other numeric options.
Signed-off-by: Luan Muniz <luan@luanmuniz.com.br>
PR-URL: https://github.com/nodejs/node/pull/64146
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Speed up Interface construction:
- Hoist the history accessor property descriptors to module scope and
define them with a single ObjectDefineProperties call, instead of
allocating six closures and four descriptor objects per instance.
- Stop assigning the history options onto the input stream. This avoids
hidden class transitions on the user provided stream and no longer
mutates it observably.
- Only check process.env.TERM for a dumb terminal when the interface is
in terminal mode. Reading process.env goes through the environment
interceptor and is comparatively expensive, and _ttyWrite is never
called when terminal is false.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/64585
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Matteo Collina