Mounting a virtual file system requires calling vfs.mount() from inside
the program, so a program cannot itself be served from one: something
already running has to mount the VFS first.
Add two startup flags. --vfs-mount=<source> mounts a directory or an
archive as a virtual file system, and may be repeated.
--vfs-load=<source> mounts that source the same way and additionally
runs the entry point and all subsequent require()/import resolution
against it rather than against the real file system, so an application
can be run straight out of a directory or a ZIP archive:
node --experimental-vfs --vfs-load=my-app.zip
Both options append to one list, so mounts happen in the order written:
node --experimental-vfs --vfs-mount=a --vfs-load=b --vfs-mount=c
mounts a, b and c in that order and runs b. Mounting the same source
twice mounts it twice, at two mount points, and the entry point comes
from the mount --vfs-load contributed rather than the earlier one.
The provider backing a source is chosen from the source itself rather
than from its name: a directory is served by RealFSProvider and a file
whose bytes are a ZIP archive by ZipProvider, so an archive can carry
any extension. vfs.registerProvider() registers a provider for formats
there is no built-in for; selection is deferred until -r and --import
preloads have run, so a preloaded module can register one before its
source is claimed.
The entry point comes from the mount the way `node <directory>` takes
one, from package.json "main" or index.js. Nothing is consumed as an
entry point argument, so every positional reaches the program unchanged
from argv[2] on, and argv[1] reports the named source rather than the
reserved mount point, which is an opaque implementation detail.
--vfs-load may be given at most once, and is not permitted in
NODE_OPTIONS: which entry point runs is the command line's decision, and
NODE_OPTIONS is parsed first, so an environment variable could otherwise
redirect any invocation. --vfs-mount is permitted there. Workers inherit
the same mounts in the same order but not the loading: a worker mounts
what the parent mounted and runs its own entry point.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/65748
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
A native addon inside a virtual file system could not be require()d:
dlopen() and LoadLibrary() open a shared object by path, and a VFS path
has no inode for them to open.
Read the addon's bytes from the VFS and load them from a private,
self-cleaning image instead, using the smallest on-disk footprint each
platform allows: on Linux an anonymous memfd loaded through
/proc/self/fd, so the bytes never reach the file system; on other POSIX
platforms a file in a 0700 mkdtemp() directory, unlinked right after
loading, with the mapping keeping it alive; on Windows a temp file
opened FILE_FLAG_DELETE_ON_CLOSE whose handle is held until exit.
This is internal. process.dlopen() keeps its documented
(module, filename[, flags]) signature and ignores anything further; the
bytes are passed to a dlopenBinary() reachable only through the
process_methods binding, which the VFS hook calls for VFS paths. Addons
on the real file system are untouched and load directly.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/65680
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: James M Snell <jasnell@gmail.com>
Every `new Worker()` runs the whole internal bootstrap (realm, node, web
exposure, thread and process-state switches) in its fresh isolate,
compiling ~80 builtins with the code cache; only the main thread
deserializes its principal context from the built-in snapshot. That
bootstrap is about half of a worker's cold start.
The bootstrapped principal context in the snapshot is nearly
thread-neutral: the worker-side switch scripts (is_not_main_thread,
does_not_own_process_state) are written as overrides of the main-thread
ones, and the per-thread values of the `worker` binding were the only
thread-specific data baked into the context. Let a worker deserialize
that same context and EnvSerializeInfo and apply the two worker-side
switches on top:
- worker binding: threadId, threadName, isMainThread, isInternalThread,
ownsProcessState and resourceLimits become lazy properties of the
per-isolate template, computed from the Environment on first read.
- CreateEnvironment(): when a worker (its IsolateData has a Worker)
passes an empty context, deserialize kNodeMainContextIndex and run
internal/bootstrap/switches/is_not_main_thread and, unless the worker
owns process state, does_not_own_process_state after
InitializeMainContext().
- Worker::Run(): take that path when the embedded built-in snapshot is
in use (not an embedder's or a --snapshot-blob one, which has run
application code), browser globals are not disabled and
--no-worker-snapshot was not given; otherwise bootstrap as before.
- is_not_main_thread.js: also delete _debugPause and the profiler idle
notifier helpers that is_main_thread.js installs.
- pre_execution: run the snapshot's deserialize callbacks (Buffer pool,
default resolver, cached cwd) on worker threads too, now that a worker
can come from a snapshot.
- --[no-]worker-snapshot per-isolate option, documented.
Sequential new Worker() -> 'online' -> terminate goes from ~20.9 ms to
~10.3 ms per worker on x64 Linux; --no-worker-snapshot restores the old
number.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65336
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Allow --force-fips to take an optional provider or strict mode. Keep
provider as the current default and explicit compatibility mode.
In strict mode, return failure from the OpenSSL 3.4+ FIPS indicator
callback when it reports a non-approved operation. Install the callback
for enforcement without implicitly enabling diagnostics;
--enable-fips-indicator-events remains the independent observation
opt-in.
Keep the bare form mapped to provider for compatibility, allowing the
default to change to strict in a future major release. Preserve the mode
when forwarding flags to child test processes, and keep the parser's
internal mode storage out of process.allowedNodeEnvironmentFlags.
Document the callback scope limitations and cover provider
compatibility, validation, synchronous crypto, WebCrypto, Workers, and
opt-in event publication.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65645
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
OpenSSL 3.4 added an indicator callback for operations that its FIPS
provider allows after an approved-mode check has been relaxed.
Publish these indicators on the crypto.fips.indicator diagnostics
channel when Node.js is started with --enable-fips-indicator-events.
Leave the OpenSSL callback unset otherwise so ordinary cryptographic
operations have no indicator callback overhead.
Keep the channel observation-only so enforcement remains the
responsibility of the provider configuration, and independent diagnostic
subscribers can coexist.
Install the native callback during process initialization so it is not
changed while Workers or thread-pool jobs can be using OpenSSL. Preserve
callbacks installed by embedders, and queue bounded, coalesced messages
for asynchronous delivery on the main environment.
Document the channel as best-effort telemetry that cannot identify the
originating call or establish FIPS validation.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65645
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
While the Options objects are not a significant overhead,
they still waste a few hundred bytes due to sloppy definition
and alignment issues. Tighten it up and shave off a couple
hundred wasted bytes.
Signed-off-by: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65145
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Accept WHATWG URL objects in private-key inputs and load referenced keys
through OpenSSL STORE loaders. Pass optional property queries and
passphrases while preserving provider-owned EVP_PKEY objects for
ordinary KeyObject and CryptoKey operations.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/63949
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Decided to take a short break from the work on QUIC
to implement a DTLS API. Very experimental at this
point but the basic API is there (inspired by the
QUIC API work).
The implementation is based on OpenSSL's built-in
DTLS support and no other dependencies are required.
DTLS is a datagram-based version of TLS that is used
for things like WebRTC and CoAP. It provides similar
security guarantees as TLS but is designed to work over
UDP instead of TCP.
This shouldn't be considered ready for production
but it is a good starting point for experimentation
and feedback.
```bash
./configure --experimental-dtls
make -j{nproc}
./node --experimental-dtls my-dtls-app.js
```
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
PR-URL: https://github.com/nodejs/node/pull/63182
Fixes: https://github.com/nodejs/node/issues/61630
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Adds a `tags` option to test(), it(), suite(), and describe() that
accepts an array of string labels. Tags are canonicalized to lowercase
and inherited from suites to nested tests by union. Reporter events
expose the tag set on every test, and TestContext exposes the test's
tags via `context.tags`.
Filtering is done by literal tag name through the new
`--experimental-test-tag-filter=<tag>` flag (or `testTagFilters` on
run()). The flag may be specified more than once; tests must contain
every filter to run. Untagged tests are excluded under any positive
filter.
The tagging mechanism is gated behind a one-shot ExperimentalWarning.
Signed-off-by: atlowChemi <chemi@atlow.co.il>
PR-URL: https://github.com/nodejs/node/pull/63221
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Pietro Marchini <pietro.marchini94@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
Add --permission-audit flag that enables the permission model in
warning-only mode. Instead of throwing ERR_ACCESS_DENIED, it emits
a message via diagnostics channel and allows the operation to
continue.
Publish permission check results to per-scope diagnostics channels
(e.g., node:permission-model:fs) so users can observe permission
decisions at runtime via diagnostics_channel.
Refs: https://github.com/nodejs/node/issues/59935
PR-URL: https://github.com/nodejs/node/pull/61869
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Instead of relying on a WASM build of postject to perform the
injection, add LIEF as dependency and generate the SEA directly
from core via a new CLI option --build-sea which takes the SEA
config. This simplifies SEA generation for users and makes it
easier to debug/maintain the SEA building process.
PR-URL: https://github.com/nodejs/node/pull/61167
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
This lays the initial groundwork for enabling the
use of IsolateGroups.
Every isolate in V8 is created within a group. When pointer
compression is enabled, all isolates within a single group
are limited to a 4 GB shared pointer cage. By default, all
isolates in the process share the same group, which means
that when running with pointer compression, the entire
process would be limited to a single 4 GB shared pointer
cage. But, we can create as many IsolateGroups as we want,
limited only by the amount of virtual memory available on
the machine.
PR-URL: https://github.com/nodejs/node/pull/60254
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
While I get that macros aren't the most loved thing in
the world, they do help reduce boilerplate, and there's
a lot of boilerplate in the QUIC code. This commit cleans
up some of that boilerplate, particularly around the
use of v8 APIs.
PR-URL: https://github.com/nodejs/node/pull/59342
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Marco