This adds support for transferring net.BoundSocket instances to other
threads via the worker_threads postMessage() transfer list, and for
sending them to child processes as the sendHandle argument of
subprocess.send(), following on from the BoundSocket introduction.
A BoundSocket reserves a port synchronously at construction time.
Making it transferable means a port can be reserved on one thread or
process and the bound (but not yet listening or connected) TCP handle
handed off to another to listen or connect on, without racing on the
bind.
For threads, BoundSocket implements kTransfer/kTransferList/
kDeserialize, moving the underlying TCP handle with the same mechanism
used for net.Socket and net.Server transfer. For child processes, the
handleConversion entry reuses the same transfer protocol on the
sending side and the same _TransferredBoundSocket deserialization path
on the receiving side; the underlying transport is that of cluster's
shared-handle scheduling: SCM_RIGHTS on Unix and WSADuplicateSocket on
Windows, both of which carry bind state.
In both cases the source instance is left in the adopted state:
address(), fd() and close() throw ERR_SOCKET_HANDLE_ADOPTED. Transfer
requires an un-adopted, open TCP handle, otherwise
ERR_WORKER_HANDLE_NOT_TRANSFERABLE is thrown; pipe (path) binds are
not transferable and throw ERR_INVALID_HANDLE_TYPE when sent over IPC.
On the receiving side the local address is re-derived from the handle
rather than trusted from serialized state.
Signed-off-by: Guy Bedford <guybedford@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64725
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
The embedder docs say each Environment has exactly one `uv_loop_t` and
that an `IsolateData` can be shared between Environments, and
`CreateIsolateData()` takes the loop, so several same-thread
Environments naturally end up on one loop. Nothing mentions that
`FreeEnvironment()` then runs that loop, with JavaScript disallowed on
the isolate, until the freed Environment's handles are gone, so the
other Environments' callbacks can fire inside it. Document that, and
point at it from `FreeEnvironment()` in node.h.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65691
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
SocketAddressLRU::Upsert always inserts an entry before evicting down
to max_size_. With max_size_ == 0, it evicts the entry it just
inserted and then accesses the now-missing key via
map_[address]->second. operator[] recreates the key with a
default-constructed std::list iterator, which is then dereferenced.
This is undefined behavior, observed as a SIGSEGV in Endpoint::Receive
on the first UDP packet accepted by a QuicEndpoint constructed with
{ addressLRUSize: 0 }.
SocketAddressLRU has no useful semantics for a zero-capacity cache,
and Upsert's callers rely on it returning a valid pointer. Reject 0
(and 0n) at the options-parsing boundary instead of changing Upsert's
contract.
Signed-off-by: Christian Aurich <christian.aurichzm@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65827
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Tim Perry <pimterry@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
A native addon inside a virtual file system could not be require()d:
dlopen() and LoadLibrary() open a shared object by path, and a VFS path
has no inode for them to open.
Read the addon's bytes from the VFS and load them from a private,
self-cleaning image instead, using the smallest on-disk footprint each
platform allows: on Linux an anonymous memfd loaded through
/proc/self/fd, so the bytes never reach the file system; on other POSIX
platforms a file in a 0700 mkdtemp() directory, unlinked right after
loading, with the mapping keeping it alive; on Windows a temp file
opened FILE_FLAG_DELETE_ON_CLOSE whose handle is held until exit.
This is internal. process.dlopen() keeps its documented
(module, filename[, flags]) signature and ignores anything further; the
bytes are passed to a dlopenBinary() reachable only through the
process_methods binding, which the VFS hook calls for VFS paths. Addons
on the real file system are untouched and load directly.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/65680
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: James M Snell <jasnell@gmail.com>
Every `new Worker()` runs the whole internal bootstrap (realm, node, web
exposure, thread and process-state switches) in its fresh isolate,
compiling ~80 builtins with the code cache; only the main thread
deserializes its principal context from the built-in snapshot. That
bootstrap is about half of a worker's cold start.
The bootstrapped principal context in the snapshot is nearly
thread-neutral: the worker-side switch scripts (is_not_main_thread,
does_not_own_process_state) are written as overrides of the main-thread
ones, and the per-thread values of the `worker` binding were the only
thread-specific data baked into the context. Let a worker deserialize
that same context and EnvSerializeInfo and apply the two worker-side
switches on top:
- worker binding: threadId, threadName, isMainThread, isInternalThread,
ownsProcessState and resourceLimits become lazy properties of the
per-isolate template, computed from the Environment on first read.
- CreateEnvironment(): when a worker (its IsolateData has a Worker)
passes an empty context, deserialize kNodeMainContextIndex and run
internal/bootstrap/switches/is_not_main_thread and, unless the worker
owns process state, does_not_own_process_state after
InitializeMainContext().
- Worker::Run(): take that path when the embedded built-in snapshot is
in use (not an embedder's or a --snapshot-blob one, which has run
application code), browser globals are not disabled and
--no-worker-snapshot was not given; otherwise bootstrap as before.
- is_not_main_thread.js: also delete _debugPause and the profiler idle
notifier helpers that is_main_thread.js installs.
- pre_execution: run the snapshot's deserialize callbacks (Buffer pool,
default resolver, cached cwd) on worker threads too, now that a worker
can come from a snapshot.
- --[no-]worker-snapshot per-isolate option, documented.
Sequential new Worker() -> 'online' -> terminate goes from ~20.9 ms to
~10.3 ms per worker on x64 Linux; --no-worker-snapshot restores the old
number.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65336
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
prepare(), function(), aggregate(), deserialize(), applyChangeset() and
backup() validated the connection, then read their options bag with
Object::Get(). A property getter runs arbitrary JavaScript at that
point, so a getter calling close() invalidates what was just checked.
Five of the six then passed a null sqlite3* to SQLite and crashed;
prepare() reported a spurious "out of memory".
Re-check IsOpen() after option parsing, immediately before the SQLite
call, keeping the early check so invalid calls still fail before any
user code runs. IsOpen() is the only condition a getter can change:
authorizer and callback depths are RAII-managed. createSession()
already parsed options first, so it only gains the early check.
deserialize() also latched the buffer length before reading
options.dbName. A getter that shrank the backing store left the length
too large; CopyContents() then handed the uninitialized remainder to
SQLite, from where serialize() returned it to JavaScript. Check the
CopyContents() result instead of discarding it.
function() and aggregate() cast the callback's length property with
As<Int32>() and no IsInt32() guard. length is configurable, so any type
reached the cast and produced a silently wrong arity.
Fixes: https://github.com/nodejs/node/issues/65586
Signed-off-by: Trevor Burnham <trevorburnham@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65595
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
With this change, the TLS pause mechanisms used in node:tls are now made
generic, and used in QUIC to pause the handshake entirely for the
session event, thereby stopping almost all events being emitted
(everything except qlog) so that all the previously existing deferral
machinery is unnecessary.
This is useful immediately to clean up & simplify things, but also
medium term because this makes the 0RTT flow significantly clearer,
which will help with the upcoming dynamic attach for QUIC (which
requires deferring ticket processing & 0RTT delivery until after
user code).
As part of this, this changes the internal flow of how ALPN & SNI are
resolved. Instead of calculating them synchronously within the OpenSSL
callbacks, we precalculate them at the pause, and then the OpenSSL
callbacks just return the precalculated values.
This is intended to only create visible changes to node:quic. Though it
touches node:tls, it preserves all existing behaviour (including
callback order and other details).
Signed-off-by: Tim Perry <pimterry@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65522
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Allow --force-fips to take an optional provider or strict mode. Keep
provider as the current default and explicit compatibility mode.
In strict mode, return failure from the OpenSSL 3.4+ FIPS indicator
callback when it reports a non-approved operation. Install the callback
for enforcement without implicitly enabling diagnostics;
--enable-fips-indicator-events remains the independent observation
opt-in.
Keep the bare form mapped to provider for compatibility, allowing the
default to change to strict in a future major release. Preserve the mode
when forwarding flags to child test processes, and keep the parser's
internal mode storage out of process.allowedNodeEnvironmentFlags.
Document the callback scope limitations and cover provider
compatibility, validation, synchronous crypto, WebCrypto, Workers, and
opt-in event publication.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65645
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
OpenSSL 3.4 added an indicator callback for operations that its FIPS
provider allows after an approved-mode check has been relaxed.
Publish these indicators on the crypto.fips.indicator diagnostics
channel when Node.js is started with --enable-fips-indicator-events.
Leave the OpenSSL callback unset otherwise so ordinary cryptographic
operations have no indicator callback overhead.
Keep the channel observation-only so enforcement remains the
responsibility of the provider configuration, and independent diagnostic
subscribers can coexist.
Install the native callback during process initialization so it is not
changed while Workers or thread-pool jobs can be using OpenSSL. Preserve
callbacks installed by embedders, and queue bounded, coalesced messages
for asynchronous delivery on the main environment.
Document the channel as best-effort telemetry that cannot identify the
originating call or establish FIPS validation.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65645
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Add a node:vfs provider backed by a node:zlib ZIP archive - a ZipBuffer
held in memory or a ZipFile on disk - that exposes the archive's members
as a virtual filesystem tree. Directories are recognized both explicitly
(a "name/" entry) and implicitly (any entry under "name/"), and a file
opened for writing commits its content as a new archive entry when its
handle is closed. The provider is read-only unless the backing archive
is writable, and offers both asynchronous and synchronous operations.
Available as vfs.ZipProvider.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/64915
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
The `#sec-using-declarations` fragment does not exist in the explicit
resource management proposal, so the link silently resolves to the top
of the page. The proposal defines `using` under
`#sec-let-and-const-declarations`, and the only `using`-related section
ids are static semantics entries.
Point the reference at the MDN page for `using` instead, matching the
`[`using`]` definition already used in fs.md.
Signed-off-by: Soul Lee <alus20x@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65632
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
