A compile cache generated ahead of time and shipped inside an
application package should only ever be read: the package may be
immutable or covered by an integrity check, and a cache directory that
appears at run time would be a surprise. Add readOnly to
module.enableCompileCache() and NODE_COMPILE_CACHE_READONLY=1: existing
entries are loaded as before, nothing is serialized or persisted,
flushCompileCache() is a no-op, and the cache directory is used as found
rather than created, so enabling against a missing directory fails
instead of making one.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65302
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: René <contact.9a5d6388@renegade334.me.uk>
A package.json that exists but cannot be read was treated the same as
one that is not there: the read failure returned no config and
resolution continued as if the package had none. Fields such as
"exports" and "type" silently disappear, so a specifier can resolve to
a different file than the package declares, while an unparsable
package.json already throws ERR_INVALID_PACKAGE_CONFIG.
Keep treating ENOENT and ENOTDIR as "no package config here", and
report any other read failure with the underlying error.
Fixes: https://github.com/nodejs/node/issues/65220
Signed-off-by: Paul Bouchon <mail@bitpshr.net>
PR-URL: https://github.com/nodejs/node/pull/65223
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
This patch extends `LoadEnvironment` to support loading ES modules,
and adds the following new types:
```cpp
enum class ModuleFormat : uint8_t {
kCommonJS,
kModule,
};
// Data for specifying an entry point script for LoadEnvironment().
// This class uses an opaque layout to allow future additions without
// breaking ABI. Use the setter methods to configure the entry point.
class ModuleData {
void set_source(std::string_view source);
void set_format(ModuleFormat format);
void set_resource_name(std::string_view name);
std::string_view source() const;
ModuleFormat format() const;
std::string_view resource_name() const;
};
class StartExecutionCallbackInfoWithModule {
void set_env(Environment* env);
void set_process_object(v8::Local<v8::Object> process_object);
void set_native_require(v8::Local<v8::Function> native_require);
void set_run_module(v8::Local<v8::Function> run_module);
void set_data(void* data);
Environment* env();
v8::Local<v8::Object> process();
v8::Local<v8::Function> native_require();
v8::Local<v8::Function> run_module();
void* data();
};
```
And two new `LoadEnvironment()` overloads:
```cpp
// Run entry point with ModuleData configuration
MaybeLocal<Value> LoadEnvironment(
Environment* env,
const ModuleData* entry_point,
EmbedderPreloadCallback preload = nullptr);
// Callback-based with new StartExecutionCallbackInfoWithModule
MaybeLocal<Value> LoadEnvironment(
Environment* env,
StartExecutionCallbackWithModule cb,
EmbedderPreloadCallback preload = nullptr,
void* callback_data = nullptr);
```
PR-URL: https://github.com/nodejs/node/pull/61548
Refs: https://github.com/nodejs/node/issues/53565
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Aditi Singh <aditisingh1400@gmail.com>
When the source text module is compiled without custom callbacks,
instead of calling into JS land from the per-isolate import.meta
initializer and then back to C++ land to set up lazy data
properties, just do the initialization all in C++ land.
Only import.meta.resolve initialization will call back
into JS land to generate a closure that call the cascaded loader
for resolution.
In addition, simplify the loader structure by merging
allowImportMetaResolve into isForAsyncLoaderHookWorker - the two
are essentially equivalent, as import.meta.resolve is only
allowed in a non-loader-hook worker thread's loader.
PR-URL: https://github.com/nodejs/node/pull/60603
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
`src/node_modules.cc` needs to be consistent with `src/node_file.cc` in
how it translates the utf8 strings to `std::wstring` otherwise we might
end up in situation where we can read the source code of imported
package from disk, but fail to recognize that it is an ESM (or CJS) and
cause runtime errors. This type of error is possible on Windows when the
path contains unicode characters and "Language for non-Unicode programs"
is set to "Chinese (Traditional, Taiwan)".
See: #58768
PR-URL: https://github.com/nodejs/node/pull/60575
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Stefan Stojanovic <stefan.stojanovic@janeasystems.com>
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
This integrates TypeScript into the compile cache by caching
the transpilation (either type-stripping or transforming) output
in addition to the V8 code cache that's generated from the
transpilation output.
Locally this speeds up loading with type stripping of
`benchmark/fixtures/strip-types-benchmark.ts` by ~65% and
loading with type transforms of
`fixtures/transform-types-benchmark.ts` by ~128%.
When comparing loading .ts and loading pre-transpiled .js on-disk
with the compile cache enabled, previously .ts loaded 46% slower
with type-stripping and 66% slower with transforms compared to
loading .js files directly.
After this patch, .ts loads 12% slower with type-stripping and
22% slower with transforms compared to .js.
(Note that the numbers are based on microbenchmark fixtures and
do not necessarily represent real-world workloads, though with
bigger real-world files, the speed up should be more significant).
PR-URL: https://github.com/nodejs/node/pull/56629
Fixes: https://github.com/nodejs/node/issues/54741
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
V8 introduced `v8::LocalVector` somewhat recently as an alternative
to using `std::vector<v8::Local<T>>` to help ensure that Local handles
are handled correctly. This updates most (but not all) of our uses
of `std::vector<v8::Local<T>>` to use `v8::LocalVector<T>` with a few
other minor cleanups encountered along the way.
PR-URL: https://github.com/nodejs/node/pull/56417
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
This implements an API for users to intentionally flush the
accumulated compile cache instead of waiting until process
shutdown. It may be useful for application that loads dependencies
first and then either reload itself in other instances, or spawning
other instances that load an overlapping set of its dependencies -
in this case its useful to flush the cache early instead of waiting
until the shutdown of itself.
Currently flushing is triggered by either process
shutdown or user requests. In the future we should simply start the
writes right after module loading on a separate thread, and this method
only blocks until all the pending writes (if any) on the other thread
are finished. In that case, the off-thread writes should finish long
before any attempt of flushing is made so the method would then only
incur a negligible overhead from thread synchronization.
PR-URL: https://github.com/nodejs/node/pull/54971
Fixes: https://github.com/nodejs/node/issues/54770
Fixes: https://github.com/nodejs/node/issues/54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
This patch adds the following API for tools to enable compile
cache dynamically and query its status.
- module.enableCompileCache(cacheDir)
- module.getCompileCacheDir()
In addition this adds a NODE_DISABLE_COMPILE_CACHE environment
variable to disable the code cache enabled by the APIs as
an escape hatch to avoid unexpected/undesired effects of
the compile cache (e.g. less precise test coverage).
When the module.enableCompileCache() method is invoked without
a specified directory, Node.js will use the value of
the NODE_COMPILE_CACHE environment variable if it's set, or
defaults to `path.join(os.tmpdir(), 'node-compile-cache')`
otherwise. Therefore it's recommended for tools to call this
method without specifying the directory to allow overrides.
PR-URL: https://github.com/nodejs/node/pull/54501
Fixes: https://github.com/nodejs/node/issues/53639
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
