A compile cache generated ahead of time and shipped inside an
application package should only ever be read: the package may be
immutable or covered by an integrity check, and a cache directory that
appears at run time would be a surprise. Add readOnly to
module.enableCompileCache() and NODE_COMPILE_CACHE_READONLY=1: existing
entries are loaded as before, nothing is serialized or persisted,
flushCompileCache() is a no-op, and the cache directory is used as found
rather than created, so enabling against a missing directory fails
instead of making one.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65302
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: René <contact.9a5d6388@renegade334.me.uk>
The MIME regex used for data: URLs could backtrack super-linearly on a
malformed URL lacking a ',' separator. Make the optional parameter
group anchored on ';' so it cannot overlap with the media-type group.
Adds a benchmark (esm/get-data-protocol-format) parameterized over
path length, so a backtracking regression shows up as an ops/sec
cliff instead of a wall-clock assertion in a test, per review.
Fixes: https://github.com/nodejs/node/issues/61904
Signed-off-by: skdas20 <skdas5405@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/61951
Reviewed-By: Aviv Keller <me@aviv.sh>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Contexts that are not deserialized from the built-in snapshot -- worker
threads, and the main context of embedders that create their own
isolate or of `node --no-node-snapshot` -- compile (with the code cache
at best) every builtin the bootstrap touches, so each eagerly required
builtin is startup time (~0.15-0.4 ms apiece). A number of them are only
required eagerly so that they end up in the snapshot, or for features
the bootstrap path never uses.
Load lazily what those paths do not need:
- is_main_thread.js: preload util, url, the ESM loader (translators,
resolver, module_job/map, source maps, node:module, vm modules, mime,
data_url, the TypeScript stripper), internal/blob and
internal/dns/utils only while building a snapshot; they load on first
use otherwise.
- fs: internal/blob (+ internal/encoding and its tables) is only used
by fs.openAsBlob().
- internal/url: internal/data_url (+ internal/mime) is only used by the
Buffer-returning file URL helpers.
- internal/process/execution, the CommonJS loader, esm/translators and
esm/load: the TypeScript stripper and data: URL helpers are only
needed for TypeScript sources / data: URLs.
- pre_execution: internal/dns/utils (+ internal/net) is only needed up
front to validate an explicit --dns-result-order or to register the
resolver's snapshot serializer; the default order becomes the
variable's initializer.
- internal/worker: event_loop_utilization and error_serdes are only
needed once a sub-worker's ELU is read or it reports an error.
- worker_threads: `locks` is defined lazily, like util's lazy exports.
Main-thread startup with the snapshot is unchanged (the same modules
are preloaded into it; the bootstrap-modules test lists are adjusted).
A bare worker compiles 95 -> 83 builtins (cold start -5%); without the
snapshot an empty CommonJS entry point compiles 76 -> 59 builtins and an
empty ES module entry point 76 -> 69.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65329
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
getNearestParentPackageJSON() memoized its answer per file, so every
module loaded still made one native call, and TraverseParent() builds
several std::filesystem::path temporaries per directory level and
serializes the whole package.json, which the JS side then usually
discarded because it already had that package.json deserialized.
The native traversal starts at the directory of the given path, so the
answer only depends on that directory: key the memo by it (following
NormalizePath()'s trailing-separator rule), so that all modules in a
directory share one native call. When the permission model is enabled
the traversal also depends on the read permissions in effect at call
time, so that configuration keeps the per-file cache.
Loading a 1000-module tree spread over ~240 directories goes from 1000
to 236 native calls.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65326
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jordan Harband <ljharb@gmail.com>
This brings back several improvements that were reverted by mistake when
landing https://redirect.github.com/nodejs/node/pull/64154
- Update the documentation about how the removal of side effects of
source collection
- Add non-enumerable `requireStack` and `topLevelAwaitLocations`
properties to `ERR_REQUIRE_ASYNC_MODULE`, the latter is only
populated when --experimental-print-required-tla is enabled
- Add "Required module: <url>" to the error message to identify the
required ESM entry point regardless of whether the flag is enabled
- Fix TLA caret column from 0-based to 1-based
- Store module source via a private symbol instead of a public property
- Use `hasAsyncGraph` (post-instantiation) in `throwIfAsyncGraph`
instead of walking the graph before instantiation
- Merge the require stack checking into the
`common.expectRequiredTLAError` helper.
- Removed tests that are made redundant by the snapshot tests
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64260
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Previously in order to collect the locations of the TLA, we wait
until right before evalutation to ensure instantiation is
completed so that we can use
v8::Module::GetStalledTopLevelAwaitMessages(). Now we try to
add an additioanl shortcut to the source code in the module wraps
instead during compilation for modules that contain TLAs and
use acron to locate the TLAs when we need to throw
ERR_REQUIRE_AYNSC_MODULE, so we can do this as early as before
instantiation and do not need to run the module again to collect
the locations.
In addition, we now collect the require stack for
ERR_REQUIRE_ASYNC_MODULE too for better metadata in the errors.
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64154
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
The CommonJS loader was passing the bare filesystem path as the
`//# sourceURL` comment of type-stripped TypeScript, this leads to
two problems:
1. It reports hasSourceURL = false if the path contains any
whitespaces as this breaks V8's magic comment parser.
2. The inspector would incorrectly report a file path verbatim as
the script's URL.
Pass the module's file: URL as the sourceURL so the reported URL
is consistent across loaders and won't lead to an issue in
inspector clients that actually except a real URL for the
scripts.
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/63705
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Since the ESM loader is captured in the snapshot now, there's
no need to lazy load the helpers. Load them eagerly to capture
them into the snapshot. This also reduces the noise coming out
of --print-bytecode since we no longer compile the helper
functions at run time.
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/63550
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Decided to take a short break from the work on QUIC
to implement a DTLS API. Very experimental at this
point but the basic API is there (inspired by the
QUIC API work).
The implementation is based on OpenSSL's built-in
DTLS support and no other dependencies are required.
DTLS is a datagram-based version of TLS that is used
for things like WebRTC and CoAP. It provides similar
security guarantees as TLS but is designed to work over
UDP instead of TCP.
This shouldn't be considered ready for production
but it is a good starting point for experimentation
and feedback.
```bash
./configure --experimental-dtls
make -j{nproc}
./node --experimental-dtls my-dtls-app.js
```
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
PR-URL: https://github.com/nodejs/node/pull/63182
Fixes: https://github.com/nodejs/node/issues/61630
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
- For imported CJS, if it's not customized by asynchronous hooks,
make sure it won't use the quirky re-invented require in all
cases.
- When the imported CJS module is customized by synchronous hooks,
in the synthetic module evalutation step, avoid calling the
respective default step again.
- Make the branching of loadCJSModuleWithModuleLoad() and
loadCJSModuleWithSpecialRequire() more explicit, and fold
the tentative fs read in the 'commonjs' translator into the
share createCJSModuleWrap() helper instead of checking it
twice in the same path.
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/62920
Fixes: https://github.com/nodejs/node/issues/63060
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Currently, --watch mode only tracks dependencies from the main
module graph (require/import). Worker thread entry points created
via new Worker() are not included, so changes to worker files do
not trigger restarts.
This change hooks into Worker initialization and registers the
worker entry file with watch mode, ensuring restarts when worker
files change.
Fixes: https://github.com/nodejs/node/issues/62275
Signed-off-by: SudhansuBandha <sudhansu9.vssut@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/62368
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Temporarily restore fs patchability in ESM loader as a workaround
for helping downstream projects that depend on this undocumented
hidden contract transition into using hook proper APIs. This patch
intentionally avoids adding a test and instead adds warning
comments to hopefully steer new code away from depending on it.
PR-URL: https://github.com/nodejs/node/pull/62835
Refs: https://github.com/nodejs/node/issues/62012
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Extends the Wasm ESM Integration for importing WebAssembly
modules in either the source phase or instance phase to support
importing static JS string constants from the special
import name `wasm:js/string-constants`.
PR-URL: https://github.com/nodejs/node/pull/62198
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
This adds a embedder_module_hdo for identifying embedder-run
modules in the dynamic import handler and import.meta initializer,
and a SourceTextModuleTypes for customizing source text module
compilation in the JS land via compileSourceTextModule().
Also, refactors the existing embedder module compilation code
to reuse the builtin resolution logic.
PR-URL: https://github.com/nodejs/node/pull/61654
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Aditi Singh <aditisingh1400@gmail.com>
