Cloning a histogram with `structuredClone()` or `postMessage()` shares
the native histogram instead of copying it. Capturing the state of a
histogram at a point in time, while other code keeps recording into
it, requires serializing it with `export()` and parsing the result
with `importHistogram()`.
Add `histogram.snapshot()`, which returns a new, independent
`Histogram` containing a copy of the histogram's configuration,
recorded values, `exceeds` count, and EWMA state, without the
serialization round trip. Values cannot be recorded into the returned
histogram. The method is available on all histograms, including
`RecordableHistogram` and `ELDHistogram` instances.
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: OpenCode
PR-URL: https://github.com/nodejs/node/pull/66099
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Query salt length and digest restrictions instead of serializing the key
to SPKI and parsing its algorithm identifier. A readable salt length
distinguishes restricted keys, including empty parameter sequences, from
unrestricted keys.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
Assisted-by: Codex
PR-URL: https://github.com/nodejs/node/pull/66108
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Retrieving asymmetricKeyDetails only needs the modulus, public exponent,
and RSA-PSS restrictions. Add a public-only Rsa view so provider-backed
keys do not also extract private components or probe additional primes.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
Assisted-by: Codex
PR-URL: https://github.com/nodejs/node/pull/66108
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
`performance.nodeTiming.uvMetricsInfo` returns the libuv event loop
metrics as numbers, which are only exact up to
`Number.MAX_SAFE_INTEGER`. Add `uvMetricsInfoBigInt`, which returns
the same metrics as bigints backed by `uint64_t` storage, carrying the
full 64-bit range reported by libuv.
A single native call fills both a `Float64Array` and a
`BigUint64Array`, so `uvMetricsInfo` does not pay for bigint
allocation and conversion. The new property is omitted from
`toJSON()`, as `JSON.stringify()` cannot serialize bigints.
Assisted-by: OpenCode
Signed-off-by: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/66094
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Add `Buffer.stringLength(input[, encoding])`, the counterpart of
`Buffer.byteLength()`: it returns the number of UTF-16 code units that
`buf.toString(encoding)` would produce, without decoding.
For UTF-8 the count is computed with simdutf. Invalid input is counted
with the same maximal-subpart replacement that the decoder applies, so
the result always matches `toString().length`. The other encodings are
computed from `byteLength` alone.
This lets code that accumulates streamed input check the result against
`buffer.constants.MAX_STRING_LENGTH` and size its memory budget before
decoding.
Refs: https://github.com/nodejs/node/issues/66062
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/66064
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Provider-backed keys can have no numeric OpenSSL ID. Identify and
construct asymmetric keys by algorithm name, replacing the custom PQC
name-to-NID substitution with provider-aware matching in ncrypto.
Centralize known algorithm names, public key-type names, capabilities,
and backend compatibility in ncrypto. Use named key generation jobs and
remove asymmetric EVP_PKEY constants from the internal JavaScript
binding.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
Assisted-by: Codex
PR-URL: https://github.com/nodejs/node/pull/65966
Reviewed-By: James M Snell <jasnell@gmail.com>
The current recordable `Histogram` does not maintain any sense
of time. Samples accumulate indefinitely as they are collected.
This implements a relatively simple sliding-window mechanism
that can be count or time based. The key benefit is that the
window remains fixed/bounded while samples are recorded.
The sliding window is chunk based. Internally, it maintains
a ring buffer of a fixed number of individual histograms.
When snapshot() is called, those are materialized into a
single combined histogram. As the window slides, older
chunks (and all of the samples they hold) are dropped from
the window so the window drops however many samples happened
to be in that chunk. This does mean that precision of the
window is determined by the chunk size.
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
PR-URL: https://github.com/nodejs/node/pull/65825
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Implements quantile-respectful density estimate calcuation
on Histogram. Helps with tail-focused latency analysis
without retaining raw samples.
Baking this directly into Node.js, based on a 1 million
sample, 1k bin workload, this impl is roughly 150-325x
faster than performing the equivalent in Rscript.
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
PR-URL: https://github.com/nodejs/node/pull/65806
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
The per-stream state records were built as object literals with
__proto__: null, which V8 creates as dictionary-mode objects: roughly
50x slower to allocate, and every subsequent property load on them is a
dictionary lookup. These records back every hot path, so both stream
construction and per-chunk field accesses were paying for it.
Replace the literals with classes whose prototype has a null prototype,
so instances stay in fast mode while Object.prototype remains excluded
from the lookup chain. Every field ever assigned is declared up front
so the shape never transitions.
Also add benchmark/webstreams/lifecycle.js covering the short-lived
stream pattern (create, few chunks, close) that first exposed this.
Assisted-by: ChatGPT
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/65625
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Read buffers for streams that emit their data to JS were allocated per
read: a 64KB backing store, tracked in a map, and then - since reads
rarely fill the whole buffer - reallocated to the right size and copied.
Allocate read buffers from a 64KB slab instead. Reads reserve the
suggested size from the slab and JS receives a view over the slab's
ArrayBuffer at the read's offset, using the offset mechanism that
onStreamRead already supports. Unused reservation space is rewound when
a read returns less than was reserved, so small reads (e.g. TLS records)
share a slab.
This removes the per-read allocations, the map bookkeeping and the
resize copy.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/64455
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
fs.cp() and fs.promises.cp() walked the tree in JavaScript with several
thread pool round trips per entry (opendir batches, two stat()s, the
copyFile(), a chmod()), all awaited in sequence: a 2 100-file tree took
~215 ms with ~110 ms of that on the main thread, against ~36 ms for
fs.cpSync(), which copies the tree in C++ when no filter is given.
Factor that C++ walk into CopyDirRecursive(), which records the error
instead of throwing so that it can run on any thread, and run it as one
ThreadPoolWork request (CpDirJob) for fs.cp()/fs.promises.cp() when
the destination directory does not exist yet and nothing has to run per
entry (no filter, no dereference, permission model off). Copying into
an existing tree keeps the JavaScript walk and its rules for what may
already be there. The same tree now takes ~30 ms with under 1 ms on the
main thread.
For that job the walk follows the JavaScript walk's rules rather than
cpSync's: it creates every directory with mkdir() and every file with
an exclusive uv_fs_copyfile() (honouring the copyFile() mode flags) and
fails with EEXIST if anything has appeared in their place since the
JavaScript check, so it never opens or follows something it did not
create; sockets, FIFOs and unknown entries are reported back to
JavaScript, which rejects them with the same SystemErrors as before;
relative link targets are made absolute lexically as path.resolve()
does. cpSync keeps merging into existing directories, skipping special
files and canonicalizing link targets.
The walk now uses the error_code overloads of std::filesystem throughout
(directory iteration included), so an unreadable directory inside the
tree is reported as EACCES by both cp() and cpSync() instead of
terminating the process, which cpSync() has done since the walk moved
to C++. Filesystem errors raised inside the walk keep their codes, with
'cp' or 'copyfile'/'mkdir' as the syscall. With preserveTimestamps the
walk stamps each directory it filled, root included, so cp() keeps
preserving directory times and cpSync() without a filter now does too.
Both file copies set the destination's mode before writing the data,
which clears setuid and setgid, so the walk puts the source mode back
after each file it copied; the JavaScript walk already chmod()s there.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65488
Refs: https://github.com/nodejs/node/pull/58461
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Jake Yuesong Li <jake.yuesong@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Avoid repeating EC key-pair validation after a pair has already been
established or validated. Invalidate the positive-only cache whenever
public-key mutation can make the pair inconsistent.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65615
Reviewed-By: James M Snell <jasnell@gmail.com>
Readable.fromWeb() and the read side of Duplex.fromWeb() allocated a
promise, a read-result object, and two reaction closures for every
chunk through reader.read(); a single reused read request now delivers
chunks through readableStreamDefaultReaderRead() instead, forwarding
each chunk in a microtask to preserve the delivery order relative to
errors and destroy.
Writable.fromWeb() and the write side of Duplex.fromWeb() paid two
derived promises off writer.ready plus the writer.write() promise and a
fresh closure pair per chunk; a single shared write request now
dispatches chunks directly and settles the node callback, with failures
delivered in a microtask.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/65548
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: James M Snell <jasnell@gmail.com>
With source maps enabled, every module that carries a sourceMappingURL
had its map decoded (or read from disk), JSON-parsed and its sources
resolved to absolute URLs while the module was being loaded, and the
per-line length table used for coverage was built with a per-code-point
loop. None of that is needed unless a stack trace is later mapped.
Keep the URL on the cache entry and resolve the payload on the first
findSourceMap() for that file. Under NODE_V8_COVERAGE the payload is
still resolved at load time, since the cache is serialized during
shutdown. lineLengths() now splits on '\n' with indexOf and only falls
back to the code point walk when the source contains U+2028/U+2029.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65490
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Default parameters are evaluated before the function body, so the
tracing context is allocated before the no-subscriber fast path.
Initialize it only for subscribed traces. Preserve method arity with an
undefined default and preserve explicit null behavior.
Add focused regression coverage and reusable TracingChannel benchmarks.
Signed-off-by: Romain Lanz <romain.lanz@pm.me>
PR-URL: https://github.com/nodejs/node/pull/65513
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
With this change, the TLS pause mechanisms used in node:tls are now made
generic, and used in QUIC to pause the handshake entirely for the
session event, thereby stopping almost all events being emitted
(everything except qlog) so that all the previously existing deferral
machinery is unnecessary.
This is useful immediately to clean up & simplify things, but also
medium term because this makes the 0RTT flow significantly clearer,
which will help with the upcoming dynamic attach for QUIC (which
requires deferring ticket processing & 0RTT delivery until after
user code).
As part of this, this changes the internal flow of how ALPN & SNI are
resolved. Instead of calculating them synchronously within the OpenSSL
callbacks, we precalculate them at the pause, and then the OpenSSL
callbacks just return the precalculated values.
This is intended to only create visible changes to node:quic. Though it
touches node:tls, it preserves all existing behaviour (including
callback order and other details).
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65522
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
The JavaScript recursive watcher used on platforms without a native one
(notably Linux) armed an fs.watch() handle and a stat() for every file
in the tree, and answered every event by stat()ing and re-reading the
whole directory it happened in. A 13k-entry tree cost 13.5k inotify
watches, ~150 ms and ~50 MB to set up, and appending to one file in a
3900-entry directory cost ~9 ms of CPU per event. A file replaced by
rename() (the usual editor save) also stopped being reported, since its
watch stayed on the old inode.
inotify reports changes to the entries of a watched directory, with
their names, so on Linux watch each directory once (symbolic links keep
their own watcher, as before), keep the set of known paths, and resolve
an event with a single stat() of the named entry: unknown names are
added and reported as 'rename', vanished ones are dropped and reported
as 'rename', file changes are reported as 'change'. kqueue and event
ports only report that a directory changed, so on the other platforms
served by this fallback every file keeps its own watcher and a
directory event rescans that directory, as before. unref() and ref()
now reach the underlying handles.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65486
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Enumerate usable ciphers and aliases from activated OpenSSL 3 providers
instead of maintaining lists of provider-only algorithms. Skip numeric
OID aliases and filter NULL, TLS composite, multiblock, and
encrypt-then-MAC implementations that the Cipher APIs cannot use.
Preserve the OpenSSL 1.1.1 and BoringSSL paths.
Expose CBC-CTS, SM4-GCM, SM4-CCM, SM4-XTS, and additional AES key wrap
implementations. Add `ctsMode` (CS1/CS2/CS3) and `xtsStandard`
(GB/IEEE) options for selecting provider CTS and SM4-XTS variants.
Keep ordinary cipher construction on the original binding and legacy
lookup paths.
Lazily cache successful provider fetches per Environment for
string initialization and `getCipherInfo()`.
Index entries by case-insensitive query, canonical, and alias names.
Deduplicate owners by provider and canonical identity. Return borrowed
pointers on warm hits.
Use the shared process-wide FIPS-state generation to invalidate
per-Environment cipher caches and refresh `getCiphers()` snapshots in
the main thread and workers. Existing cipher contexts retain their
implementation and can finish across a transition. Release provider
owners before unloading worker addon DSOs.
Enforce one-shot updates for CBC-CTS, AES key wrap, SIV/GCM-SIV, and
CCM decryption. Reject finalization without required input or CCM tags,
and defer authentication failures to `final()`. Document streaming and
XTS data-unit constraints.
Add known-answer vectors, option validation, provider round trips,
cache, worker, snapshot, FIPS transition, and construction benchmark
coverage.
Fixes: https://github.com/nodejs/node/issues/43040
Fixes: https://github.com/nodejs/node/issues/64866
Refs: https://github.com/nodejs/node/issues/62982
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65484
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
readdir({ recursive: true }) asked the binding for names only and then
called internalModuleStat() on every entry to find the directories to
descend into; with withFileTypes it built the Dirents and still stat()ed
every entry that was not already a directory. Both variants also ran
path.join() and path.relative() per entry to build the relative result.
Ask the binding for file types in all cases, descend into directories
directly, and only stat() symbolic links and entries of unknown type
(which is what could point to a directory). The relative name is the
parent's prefix plus the entry name. Results, their order and the
symlink-following behavior are unchanged for fs.readdirSync, fs.readdir
and fs.promises.readdir.
The known_issues test for Buffer paths (#58892) called back without
checking the error; the error now reaches the callback instead of being
thrown from the completion handler, so the test asserts success to keep
expressing the issue.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65487
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
The encode-and-enqueue transform walked the chunk code unit by code
unit, materializing a single-character string per index and building
the output with string concatenation. The only state that crosses
chunks is a trailing high (leading) surrogate, and TextEncoder.encode's
USVString conversion already replaces every interior lone surrogate
with U+FFFD, which is exactly what the spec loop produces. Join a
pending high surrogate with the incoming chunk, hold back a new
trailing high surrogate, and encode the rest in a single native call.
The streaming decode path also reuses a single options object instead
of allocating { stream: true } per chunk.
An encoding-streams benchmark is added since the suite had no
TextEncoderStream/TextDecoderStream row. Encoding improves by ~546%
with 1KB string chunks and ~20% with 16-character chunks; decode is
unchanged.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/65414
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Mattias Buelens <mattias@buelens.com>
Add an --analyze flag that performs statistical analysis directly
after benchmarks complete, eliminating the need for R and compare.R.
When --analyze is specified, compare.js collects the rate data during
the run and prints a statistical summary table instead of CSV output.
The table matches the format of compare.R: improvement percentage,
significance stars (* p<0.05, ** p<0.01, *** p<0.001), and confidence
intervals at three risk levels.
Also adds a --max-regression N option that causes the compare.js to
exit with 1 (error) when the `--new` is N% slower. Useful for CI
use to detect regressions.
Uses the histogram API's welchTest() and cohensD() methods introduced
in the previous commit. Benchmark rates are scaled to integers for
HdrHistogram recording; the --scale option (default 1000) controls
the multiplier for precision.
Usage:
node benchmark/compare.js --old ./node-old --new ./node-new \
--analyze url
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode/Opus
PR-URL: https://github.com/nodejs/node/pull/65416
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
The MIME regex used for data: URLs could backtrack super-linearly on a
malformed URL lacking a ',' separator. Make the optional parameter
group anchored on ';' so it cannot overlap with the media-type group.
Adds a benchmark (esm/get-data-protocol-format) parameterized over
path length, so a backtracking regression shows up as an ops/sec
cliff instead of a wall-clock assertion in a test, per review.
Fixes: https://github.com/nodejs/node/issues/61904
Signed-off-by: skdas20 <skdas5405@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/61951
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
James M Snell