--vfs-mount mounted a source without running it, and shared one ordered
list of sources with --vfs-load, so neither option could say which entry
it had contributed: the entry point was recovered from the position of
--vfs-load among the mounts, in a list NODE_OPTIONS could prepend to.
Nothing needs more than one mount from the command line: a program that
wants more can mount them itself through node:vfs, where it also gets
the instance.
Remove --vfs-mount, leaving --vfs-load with the single source it mounts
and runs, and reserve layer 0 for that source, numbering the file
systems a program mounts itself from 1. The source is then at the same
mount point in every thread, whatever else that thread mounts -
including a thread where a --require preload mounted a file system of
its own first - so a path into it stays valid in a worker.
A worker still does not run that entry point: it inherits the source but
not the decision to load from it. A worker created with its own execArgv
inherits neither, so the documentation now says that such a worker must
be given --experimental-vfs and --vfs-load again to run a script from
the mount, and that --experimental-vfs is also what makes node:vfs
available to the worker's own code.
ERR_VFS_INVALID_TARGET now names --vfs-load as the source's origin, and
the startup test moves to test-vfs-load.js, with the cases that covered
mounting without loading removed and cases for the reserved mount point
added.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/66162
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Mounting a virtual file system requires calling vfs.mount() from inside
the program, so a program cannot itself be served from one: something
already running has to mount the VFS first.
Add two startup flags. --vfs-mount=<source> mounts a directory or an
archive as a virtual file system, and may be repeated.
--vfs-load=<source> mounts that source the same way and additionally
runs the entry point and all subsequent require()/import resolution
against it rather than against the real file system, so an application
can be run straight out of a directory or a ZIP archive:
node --experimental-vfs --vfs-load=my-app.zip
Both options append to one list, so mounts happen in the order written:
node --experimental-vfs --vfs-mount=a --vfs-load=b --vfs-mount=c
mounts a, b and c in that order and runs b. Mounting the same source
twice mounts it twice, at two mount points, and the entry point comes
from the mount --vfs-load contributed rather than the earlier one.
The provider backing a source is chosen from the source itself rather
than from its name: a directory is served by RealFSProvider and a file
whose bytes are a ZIP archive by ZipProvider, so an archive can carry
any extension. vfs.registerProvider() registers a provider for formats
there is no built-in for; selection is deferred until -r and --import
preloads have run, so a preloaded module can register one before its
source is claimed.
The entry point comes from the mount the way `node <directory>` takes
one, from package.json "main" or index.js. Nothing is consumed as an
entry point argument, so every positional reaches the program unchanged
from argv[2] on, and argv[1] reports the named source rather than the
reserved mount point, which is an opaque implementation detail.
--vfs-load may be given at most once, and is not permitted in
NODE_OPTIONS: which entry point runs is the command line's decision, and
NODE_OPTIONS is parsed first, so an environment variable could otherwise
redirect any invocation. --vfs-mount is permitted there. Workers inherit
the same mounts in the same order but not the loading: a worker mounts
what the parent mounted and runs its own entry point.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/65748
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
A native addon inside a virtual file system could not be require()d:
dlopen() and LoadLibrary() open a shared object by path, and a VFS path
has no inode for them to open.
Read the addon's bytes from the VFS and load them from a private,
self-cleaning image instead, using the smallest on-disk footprint each
platform allows: on Linux an anonymous memfd loaded through
/proc/self/fd, so the bytes never reach the file system; on other POSIX
platforms a file in a 0700 mkdtemp() directory, unlinked right after
loading, with the mapping keeping it alive; on Windows a temp file
opened FILE_FLAG_DELETE_ON_CLOSE whose handle is held until exit.
This is internal. process.dlopen() keeps its documented
(module, filename[, flags]) signature and ignores anything further; the
bytes are passed to a dlopenBinary() reachable only through the
process_methods binding, which the VFS hook calls for VFS paths. Addons
on the real file system are untouched and load directly.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/65680
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: James M Snell <jasnell@gmail.com>
Every `new Worker()` runs the whole internal bootstrap (realm, node, web
exposure, thread and process-state switches) in its fresh isolate,
compiling ~80 builtins with the code cache; only the main thread
deserializes its principal context from the built-in snapshot. That
bootstrap is about half of a worker's cold start.
The bootstrapped principal context in the snapshot is nearly
thread-neutral: the worker-side switch scripts (is_not_main_thread,
does_not_own_process_state) are written as overrides of the main-thread
ones, and the per-thread values of the `worker` binding were the only
thread-specific data baked into the context. Let a worker deserialize
that same context and EnvSerializeInfo and apply the two worker-side
switches on top:
- worker binding: threadId, threadName, isMainThread, isInternalThread,
ownsProcessState and resourceLimits become lazy properties of the
per-isolate template, computed from the Environment on first read.
- CreateEnvironment(): when a worker (its IsolateData has a Worker)
passes an empty context, deserialize kNodeMainContextIndex and run
internal/bootstrap/switches/is_not_main_thread and, unless the worker
owns process state, does_not_own_process_state after
InitializeMainContext().
- Worker::Run(): take that path when the embedded built-in snapshot is
in use (not an embedder's or a --snapshot-blob one, which has run
application code), browser globals are not disabled and
--no-worker-snapshot was not given; otherwise bootstrap as before.
- is_not_main_thread.js: also delete _debugPause and the profiler idle
notifier helpers that is_main_thread.js installs.
- pre_execution: run the snapshot's deserialize callbacks (Buffer pool,
default resolver, cached cwd) on worker threads too, now that a worker
can come from a snapshot.
- --[no-]worker-snapshot per-isolate option, documented.
Sequential new Worker() -> 'online' -> terminate goes from ~20.9 ms to
~10.3 ms per worker on x64 Linux; --no-worker-snapshot restores the old
number.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65336
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Allow --force-fips to take an optional provider or strict mode. Keep
provider as the current default and explicit compatibility mode.
In strict mode, return failure from the OpenSSL 3.4+ FIPS indicator
callback when it reports a non-approved operation. Install the callback
for enforcement without implicitly enabling diagnostics;
--enable-fips-indicator-events remains the independent observation
opt-in.
Keep the bare form mapped to provider for compatibility, allowing the
default to change to strict in a future major release. Preserve the mode
when forwarding flags to child test processes, and keep the parser's
internal mode storage out of process.allowedNodeEnvironmentFlags.
Document the callback scope limitations and cover provider
compatibility, validation, synchronous crypto, WebCrypto, Workers, and
opt-in event publication.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65645
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
OpenSSL 3.4 added an indicator callback for operations that its FIPS
provider allows after an approved-mode check has been relaxed.
Publish these indicators on the crypto.fips.indicator diagnostics
channel when Node.js is started with --enable-fips-indicator-events.
Leave the OpenSSL callback unset otherwise so ordinary cryptographic
operations have no indicator callback overhead.
Keep the channel observation-only so enforcement remains the
responsibility of the provider configuration, and independent diagnostic
subscribers can coexist.
Install the native callback during process initialization so it is not
changed while Workers or thread-pool jobs can be using OpenSSL. Preserve
callbacks installed by embedders, and queue bounded, coalesced messages
for asynchronous delivery on the main environment.
Document the channel as best-effort telemetry that cannot identify the
originating call or establish FIPS validation.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65645
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
A compile cache generated ahead of time and shipped inside an
application package should only ever be read: the package may be
immutable or covered by an integrity check, and a cache directory that
appears at run time would be a surprise. Add readOnly to
module.enableCompileCache() and NODE_COMPILE_CACHE_READONLY=1: existing
entries are loaded as before, nothing is serialized or persisted,
flushCompileCache() is a no-op, and the cache directory is used as found
rather than created, so enabling against a missing directory fails
instead of making one.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65302
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: René <contact.9a5d6388@renegade334.me.uk>
A portable compile cache is meant to be reused wherever the same module
layout is found, which includes being generated once (at build time,
say) and shipped read-only with an application for whoever runs it. On
platforms with uids the cache subdirectory is suffixed with the uid of
the user who created it, so such a cache is only ever found by that one
user. Omit the suffix in portable mode: a user who cannot write to the
directory still reads it, and a failed write is only a cache miss.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65293
Reviewed-By: James M Snell <jasnell@gmail.com>
Upgrades the experimental tag filter introduced by stage 1 to accept a
boolean expression instead of a literal tag name.
Grammar: `and`/`&&`, `or`/`||`, `not`/`!`, parentheses for grouping,
and `*` wildcards inside identifiers. Standard precedence
(`not > and > or`); binary operators are left-associative. Word forms
require whitespace separation; punctuation forms do not. Untagged
tests evaluate `false` for any include expression and `true` for
`not X`, so excluding tags does not accidentally remove untagged
tests.
The flag and `testTagFilters` option are still repeatable; multiple
expressions still AND together. Malformed expressions fail fast at
the parent process at startup.
Tag value validation tightens to reject whitespace, operator
characters (`& | ! ( ) *`), and the reserved words `and`/`or`/`not`
in any casing - a breaking change relative to the stage 1 ship,
acceptable at Stability 1.0 (Early development).
Signed-off-by: atlowChemi <chemi@atlow.co.il>
PR-URL: https://github.com/nodejs/node/pull/63054
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Distinguish OpenSSL 3 provider setup from FIPS/property-query state,
document configuration precedence and provider limitations, and retain
the OpenSSL 1.1 and runtime control guidance.
Regenerate the CLI manpage.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64982
Reviewed-By: Richard Lau <richard.lau@ibm.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
Accept WHATWG URL objects in private-key inputs and load referenced keys
through OpenSSL STORE loaders. Pass optional property queries and
passphrases while preserving provider-owned EVP_PKEY objects for
ordinary KeyObject and CryptoKey operations.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/63949
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Expand the documentation for the --permission-audit flag, which was
fixed in 51c09eae6c to no longer throw ERR_ACCESS_DENIED on denied
operations. The previous docs only had a two-sentence description in
cli.md and no mention in the permissions guide or process.permission
API docs.
- permissions.md: add enforce vs audit mode overview, a new
"Audit Mode" subsection listing the diagnostics channel names
(node:permission-model:*) and the { permission, resource } message
shape, and a usage example. Update the Runtime API section to
mention both --permission and --permission-audit.
- cli.md: expand the --permission-audit section to clarify that
--permission is not required, --allow-* flags are not needed,
errors are not thrown, and --permission takes precedence when both
are set. Add a cross-reference from --permission to
--permission-audit.
- process.md: note that process.permission is available under both
flags, and clarify permission.has() and permission.drop() behavior
in audit mode.
- node.1: regenerated via `make node.1`.
Refs: https://github.com/nodejs/node/pull/64426
Signed-off-by: Adrian Estrada <edsadr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64791
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Previously in order to collect the locations of the TLA, we wait
until right before evalutation to ensure instantiation is
completed so that we can use
v8::Module::GetStalledTopLevelAwaitMessages(). Now we try to
add an additioanl shortcut to the source code in the module wraps
instead during compilation for modules that contain TLAs and
use acron to locate the TLAs when we need to throw
ERR_REQUIRE_AYNSC_MODULE, so we can do this as early as before
instantiation and do not need to run the module again to collect
the locations.
In addition, we now collect the require stack for
ERR_REQUIRE_ASYNC_MODULE too for better metadata in the errors.
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64154
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Philipp Dunkel