This brings back several improvements that were reverted by mistake when
landing https://redirect.github.com/nodejs/node/pull/64154
- Update the documentation about how the removal of side effects of
source collection
- Add non-enumerable `requireStack` and `topLevelAwaitLocations`
properties to `ERR_REQUIRE_ASYNC_MODULE`, the latter is only
populated when --experimental-print-required-tla is enabled
- Add "Required module: <url>" to the error message to identify the
required ESM entry point regardless of whether the flag is enabled
- Fix TLA caret column from 0-based to 1-based
- Store module source via a private symbol instead of a public property
- Use `hasAsyncGraph` (post-instantiation) in `throwIfAsyncGraph`
instead of walking the graph before instantiation
- Merge the require stack checking into the
`common.expectRequiredTLAError` helper.
- Removed tests that are made redundant by the snapshot tests
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64260
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Previously in order to collect the locations of the TLA, we wait
until right before evalutation to ensure instantiation is
completed so that we can use
v8::Module::GetStalledTopLevelAwaitMessages(). Now we try to
add an additioanl shortcut to the source code in the module wraps
instead during compilation for modules that contain TLAs and
use acron to locate the TLAs when we need to throw
ERR_REQUIRE_AYNSC_MODULE, so we can do this as early as before
instantiation and do not need to run the module again to collect
the locations.
In addition, we now collect the require stack for
ERR_REQUIRE_ASYNC_MODULE too for better metadata in the errors.
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64154
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Since the ESM loader is captured in the snapshot now, there's
no need to lazy load the helpers. Load them eagerly to capture
them into the snapshot. This also reduces the noise coming out
of --print-bytecode since we no longer compile the helper
functions at run time.
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/63550
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
- For imported CJS, if it's not customized by asynchronous hooks,
make sure it won't use the quirky re-invented require in all
cases.
- When the imported CJS module is customized by synchronous hooks,
in the synthetic module evalutation step, avoid calling the
respective default step again.
- Make the branching of loadCJSModuleWithModuleLoad() and
loadCJSModuleWithSpecialRequire() more explicit, and fold
the tentative fs read in the 'commonjs' translator into the
share createCJSModuleWrap() helper instead of checking it
twice in the same path.
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/62920
Fixes: https://github.com/nodejs/node/issues/63060
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Currently, --watch mode only tracks dependencies from the main
module graph (require/import). Worker thread entry points created
via new Worker() are not included, so changes to worker files do
not trigger restarts.
This change hooks into Worker initialization and registers the
worker entry file with watch mode, ensuring restarts when worker
files change.
Fixes: https://github.com/nodejs/node/issues/62275
Signed-off-by: SudhansuBandha <sudhansu9.vssut@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/62368
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
This adds a embedder_module_hdo for identifying embedder-run
modules in the dynamic import handler and import.meta initializer,
and a SourceTextModuleTypes for customizing source text module
compilation in the JS land via compileSourceTextModule().
Also, refactors the existing embedder module compilation code
to reuse the builtin resolution logic.
PR-URL: https://github.com/nodejs/node/pull/61654
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Aditi Singh <aditisingh1400@gmail.com>
When the source text module is compiled without custom callbacks,
instead of calling into JS land from the per-isolate import.meta
initializer and then back to C++ land to set up lazy data
properties, just do the initialization all in C++ land.
Only import.meta.resolve initialization will call back
into JS land to generate a closure that call the cascaded loader
for resolution.
In addition, simplify the loader structure by merging
allowImportMetaResolve into isForAsyncLoaderHookWorker - the two
are essentially equivalent, as import.meta.resolve is only
allowed in a non-loader-hook worker thread's loader.
PR-URL: https://github.com/nodejs/node/pull/60603
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
ESM resolution and loading is now always synchronous from a
non-loader-hook thread. If no asynchrnous loader hooks are
registered, the resolution/loading is entirely synchronous.
If asynchronous loader hooks are registered, these would be
synchronous on the non-loader-hook thread, and asynchronous
on the loader hook thread.
This avoids several races caused by async/sync loading sharing
the same cache. In particular, asynchronous loader hooks
now works with `require(esm)` - previously it tends to break
due to races.
In addition, when an asynchronous loader hook
returns a promise that never settles, the main thread no longer
silently exits with exit code 13, leaving the code below
any module loading calls silently ignored without being executed.
Instead, it now throws ERR_ASYNC_LOADER_REQUEST_NEVER_SETTLED
which can be caught and handled by the main thread. If the module
request comes from `import()`, the never-settling promise is
now relayed to the result returned by `import()`.
Drive-by: when annotating the error about importing undetectable
named exports from CommonJS, it now no longer reload the source
code of the CommonJS module, and instead reuses format information
cached when the module was loaded for linking.
PR-URL: https://github.com/nodejs/node/pull/60380
Fixes: https://github.com/nodejs/node/issues/59666
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
- This updates the comments that assume loader hooks must be async
- Differentiate the sync/async loader hook paths in naming
`#customizations` is now `#asyncLoaderHooks` to make it clear
it's from the async APIs.
- Differentiate the paths running on the loader hook thread
(affects the loading of async other loader hooks and are async)
v.s. paths on the main thread calling out to code on the loader
hook thread (do not handle loading of other async loader hooks, and
can be sync by blocking).
- `Hooks` is now `AsyncLoaderHooksOnLoaderHookWorker`
- `CustomizedModuleLoader` is now
`AsyncLoaderHooksProxiedToLoaderHookWorker` and moved into
`lib/internal/modules/esm/hooks.js` as it implements the same
interface as `AsyncLoaderHooksOnLoaderHookWorker`
- `HooksProxy` is now `AsyncLoaderHookWorker`
- Adjust the JSDoc accordingly
- Clarify the "loader worker" as the "async loader hook worker"
i.e. when there's no _async_ loader hook registered, there won't
be this worker, to avoid the misconception that this worker
is spawned unconditionally.
- The code run on the loader hook worker to process
`--experimental-loader` is moved into
`lib/internal/modules/esm/worker.js` for clarity.
- The initialization configuration `forceDefaultLoader` is split
into `shouldSpawnLoaderHookWorker` and `shouldPreloadModules`
as those can be separate.
- `--experimental-vm-modules` is now processed during pre-execution
and no longer part of the initialization of the built-in ESM
loader, as it only exposes the vm APIs of ESM, and is unrelated
to built-in ESM loading.
PR-URL: https://github.com/nodejs/node/pull/60278
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
`v8::Module::IsGraphAsync()` traverses the dependencies to find if they
contain TLA each time. `ModuleWrap.hasAsyncGraph` caches the result
and exposes the property to JS land so that the presence of the property
`module.hasAsyncGraph` can be consistent.
This also allows C++ access of cached `hasAsyncGraph`.
This merges the `intantiateSync`/`instantiate` and
`getNamespaceSync`/`getNamespace` as they are always sync.
PR-URL: https://github.com/nodejs/node/pull/59703
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Clarify the names - hasAsyncGraph means either the module or its
dependencies contains top-level await; hasTopLevelAwait means the
module itself contains top-level await. Theoratically the former
can be inferred from iterating over the dependencies but for the
built-in loader it's currently not fully reliable until we eliminate
async linking.
Also remove the hasTopLevelAwait method - we can simply put it
on the module wrap for source text modules right after compilation.
PR-URL: https://github.com/nodejs/node/pull/59675
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
This tracks the asynchronicity in the ModuleWraps when they turn out to
contain TLA after instantiation, and throw the right error
(ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes
the freezing of ModuleWraps since it's not meaningful to freeze
this when the rest of the module loader is mutable, and we
can record the asynchronicity in the ModuleWrap right after
compilation after we get a V8 upgrade that contains
v8::Module::HasTopLevelAwait() instead of searching through
the module graph repeatedly which can be slow.
PR-URL: https://github.com/nodejs/node/pull/55520
Fixes: https://github.com/nodejs/node/issues/55516
Refs: https://github.com/nodejs/node/issues/52697
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
This lays the foundation for supporting synchronous hooks proposed
in https://github.com/nodejs/loaders/pull/198 for ESM.
- Corrects and adds several JSDoc comments for internal functions
of the ESM loader, as well as explaining how require() for
import CJS work in the special resolve/load paths. This doesn't
consolidate it with import in require(esm) yet due to caching
differences, which is left as a TODO.
- The moduleProvider passed into ModuleJob is replaced as
moduleOrModulePromise, we call the translators directly in the
ESM loader and verify it right after loading for clarity.
- Reuse a few refactored out helpers for require(esm) in
getModuleJobForRequire().
PR-URL: https://github.com/nodejs/node/pull/54769
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Joyee Cheung