By default, Node.js enables trap-handler-based WebAssembly bound
checks. As a result, V8 does not need to insert inline bound checks
int the code compiled from WebAssembly which may speedup WebAssembly
execution significantly, but this optimization requires allocating
a big virtual memory cage (currently 10GB). If the Node.js process
does not have access to a large enough virtual memory address space
due to system configurations or hardware limitations, users won't
be able to run any WebAssembly that involves allocation in this
virtual memory cage and will see an out-of-memory error.
```console
$ ulimit -v 5000000
$ node -p "new WebAssembly.Memory({ initial: 10, maximum: 100 });"
[eval]:1
new WebAssembly.Memory({ initial: 10, maximum: 100 });
^
RangeError: WebAssembly.Memory(): could not allocate memory
at [eval]:1:1
at runScriptInThisContext (node:internal/vm:209:10)
at node:internal/process/execution:118:14
at [eval]-wrapper:6:24
at runScript (node:internal/process/execution:101:62)
at evalScript (node:internal/process/execution:136:3)
at node:internal/main/eval_string:49:3
```
`--disable-wasm-trap-handler` disables this optimization so that
users can at least run WebAssembly (with a less optimial performance)
when the virtual memory address space available to their Node.js
process is lower than what the V8 WebAssembly memory cage needs.
PR-URL: https://github.com/nodejs/node/pull/52766
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
The output of `node -p process.config.variables.asan` includes
a newline character so it's never exactly "1", which means
asan is always "off" for the status files. This fixes the
detection by stripping whitespaces from the output.
PR-URL: https://github.com/nodejs/node/pull/52766
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
So that if there are circular dependencies in the synchronous
module graph, they could be resolved using the cached jobs.
In case linking fails and the error gets caught, reset the
cache right after linking. If it succeeds, the caller will
cache it again. Otherwise the error bubbles up to users,
and since we unset the cache for the unlinkable module
the next attempt would still fail.
PR-URL: https://github.com/nodejs/node/pull/52868
Fixes: https://github.com/nodejs/node/issues/52864
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
In this patch we split the serialization routine into two different
routines: `getCLIOptionsValues()` for only serializing the key-value
pairs and `getCLIOptionsInfo()` for getting additional information such
as help text etc. The former is used a lot more frequently than the
latter, which is only used for generating `--help` and building
`process.allowedNodeEnvironmentFlags`.
`getCLIOptionsValues()` also adds `--no-` entries for boolean options so
there is no need to special case in the JS land.
This patch also refactors the option serialization routines to
use v8::Object constructor that takes key-value pairs in one go
to avoid calling Map::Set or Object::Set repeatedly which can go
up to a patched prototype.
PR-URL: https://github.com/nodejs/node/pull/52451
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
This patch exposes the crc32() function from zlib to user-land.
It computes a 32-bit Cyclic Redundancy Check checksum of `data`. If
`value` is specified, it is used as the starting value of the checksum,
otherwise, 0 is used as the starting value.
```js
const zlib = require('node:zlib');
const { Buffer } = require('node:buffer');
let crc = zlib.crc32('hello'); // 907060870
crc = zlib.crc32('world', crc); // 4192936109
crc = zlib.crc32(Buffer.from('hello')); // 907060870
crc = zlib.crc32(Buffer.from('world'), crc); // 4192936109
```
PR-URL: https://github.com/nodejs/node/pull/52692
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
This patch:
1. Adds ESM syntax detection to compileFunctionForCJSLoader()
for --experimental-detect-module and allow it to emit the
warning for how to load ESM when it's used to parse ESM as
CJS but detection is not enabled.
2. Moves the ESM detection of --experimental-detect-module for
the entrypoint from executeUserEntryPoint() into
Module.prototype._compile() and handle it directly in the
CJS loader so that the errors thrown during compilation *and
execution* during the loading of the entrypoint does not
need to be bubbled all the way up. If the entrypoint doesn't
parse as CJS, and detection is enabled, the CJS loader will
re-load the entrypoint as ESM on the spot asynchronously using
runEntryPointWithESMLoader() and cascadedLoader.import(). This
is fine for the entrypoint because unlike require(ESM) we don't
the namespace of the entrypoint synchronously, and can just
ignore the returned value. In this case process.mainModule is
reset to undefined as they are not available for ESM entrypoints.
3. Supports --experimental-detect-module for require(esm).
PR-URL: https://github.com/nodejs/node/pull/52047
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Instead of using an async function wrapper, just try compiling code with
unknown module format as SourceTextModule when it cannot be compiled
as CJS and the error message indicates that it's worth a retry. If
it can be parsed as SourceTextModule then it's considered ESM.
Also, move shouldRetryAsESM() to C++ completely so that
we can reuse it in the CJS module loader for require(esm).
Drive-by: move methods that don't belong to ContextifyContext
out as static methods and move GetHostDefinedOptions to
ModuleWrap.
PR-URL: https://github.com/nodejs/node/pull/52413
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
This patch implements automatic on-disk code caching that can be enabled
via an environment variable NODE_COMPILE_CACHE.
When set, whenever Node.js compiles a CommonJS or a ECMAScript Module,
it will use on-disk [V8 code cache][] persisted in the specified
directory to speed up the compilation. This may slow down the first
load of a module graph, but subsequent loads of the same module graph
may get a significant speedup if the contents of the modules do not
change. Locally, this speeds up loading of
test/fixtures/snapshot/typescript.js from ~130ms to ~80ms.
To clean up the generated code cache, simply remove the directory.
It will be recreated the next time the same directory is used for
`NODE_COMPILE_CACHE`.
Compilation cache generated by one version of Node.js may not be used
by a different version of Node.js. Cache generated by different versions
of Node.js will be stored separately if the same directory is used
to persist the cache, so they can co-exist.
Caveat: currently when using this with V8 JavaScript code coverage, the
coverage being collected by V8 may be less precise in functions that are
deserialized from the code cache. It's recommended to turn this off when
running tests to generate precise coverage.
Implementation details:
There is one cache file per module on disk. The directory layout
is:
- Compile cache directory (from NODE_COMPILE_CACHE)
- 8b23c8fe: CRC32 hash of CachedDataVersionTag + NODE_VERESION
- 2ea3424d:
- 10860e5a: CRC32 hash of filename + module type
- 431e9adc: ...
- ...
Inside the cache file, there is a header followed by the actual
cache content:
```
[uint32_t] code size
[uint32_t] code hash
[uint32_t] cache size
[uint32_t] cache hash
... compile cache content ...
```
When reading the cache file, we'll also check if the code size
and code hash match the code that the module loader is loading
and whether the cache size and cache hash match the file content
read. If they don't match, or if V8 rejects the cache passed,
we'll ignore the mismatch cache, and regenerate the cache after
compilation succeeds and rewrite it to disk.
PR-URL: https://github.com/nodejs/node/pull/52535
Refs: https://github.com/nodejs/node/issues/47472
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Previously there is an edge case where submodules loaded by require()
may not be loaded by import() again from different intermediate
edges in the graph. This patch fixes that, added tests, and added
debug logs.
Drive-by: make loader a private field so it doesn't show up in logs.
PR-URL: https://github.com/nodejs/node/pull/52487
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
This refactors the code that compiles SourceTextModule for the
built-in ESM loader to use a common routine so that it's easier
to customize cache handling for the ESM loader. In addition
this introduces a common symbol for import.meta and import()
so that we don't need to create additional closures as handlers,
since we can get all the information we need from the V8 callback
already. This should reduce the memory footprint of ESM as well.
PR-URL: https://github.com/nodejs/node/pull/52291
Refs: https://github.com/nodejs/node/issues/47472
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
This patch adds `require()` support for synchronous ESM graphs under
the flag `--experimental-require-module`
This is based on the the following design aspect of ESM:
- The resolution can be synchronous (up to the host)
- The evaluation of a synchronous graph (without top-level await) is
also synchronous, and, by the time the module graph is instantiated
(before evaluation starts), this is is already known.
If `--experimental-require-module` is enabled, and the ECMAScript
module being loaded by `require()` meets the following requirements:
- Explicitly marked as an ES module with a `"type": "module"` field in
the closest package.json or a `.mjs` extension.
- Fully synchronous (contains no top-level `await`).
`require()` will load the requested module as an ES Module, and return
the module name space object. In this case it is similar to dynamic
`import()` but is run synchronously and returns the name space object
directly.
```mjs
// point.mjs
export function distance(a, b) {
return (b.x - a.x) ** 2 + (b.y - a.y) ** 2;
}
class Point {
constructor(x, y) { this.x = x; this.y = y; }
}
export default Point;
```
```cjs
const required = require('./point.mjs');
// [Module: null prototype] {
// default: [class Point],
// distance: [Function: distance]
// }
console.log(required);
(async () => {
const imported = await import('./point.mjs');
console.log(imported === required); // true
})();
```
If the module being `require()`'d contains top-level `await`, or the
module graph it `import`s contains top-level `await`,
[`ERR_REQUIRE_ASYNC_MODULE`][] will be thrown. In this case, users
should load the asynchronous module using `import()`.
If `--experimental-print-required-tla` is enabled, instead of throwing
`ERR_REQUIRE_ASYNC_MODULE` before evaluation, Node.js will evaluate the
module, try to locate the top-level awaits, and print their location to
help users fix them.
PR-URL: https://github.com/nodejs/node/pull/51977
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
The string writing/reading was intended for debugging info
in snapshot, which had a CHECK_GT(length, 0) check, it then
got repurposed for SEA resource writing/reading and turned
into a helper for string views, but was not updated to handle
empty views, causing occasional crash in the CI when the
read is protected. This patch fixes it.
PR-URL: https://github.com/nodejs/node/pull/52000
Fixes: https://github.com/nodejs/node/issues/50740
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
When the entry point is a module and the graph it imports still
contains unsettled top-level await when the Node.js instance
finishes the event loop, search from the entry point module
for unsettled top-level await and print their location.
To avoid unnecessary overhead, we register a promise that only
gets settled when the entry point graph evaluation returns
from await, and only search the module graph if it's still
unsettled by the time the instance is exiting.
This patch only handles this for entry point modules. Other kinds of
modules are more complicated so will be left for the future.
Drive-by: update the terminology "unfinished promise" to the
more correct one "unsettled promise" in the codebase.
PR-URL: https://github.com/nodejs/node/pull/51999
Fixes: https://github.com/nodejs/node/issues/42868
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Split the `internal/process/esm_loader` file which contains the
singleton cascaded loader:
- The the singleton cascaded loader now directly resides in
`internal/modules/esm/loader`, where the constructor also lives.
This file is the root of most circular dependency of ESM code,
(because components of the loader need the singleton itself),
so this makes the dependency more obvious. Added comments about
loading it lazily to avoid circular dependency.
- The getter to the cascaded loader is also turned into a method
to make the side effect explicit.
- The sequence of `loadESM()` and `handleMainPromise` is now merged
together into `runEntryPointWithESMLoader()` in
`internal/modules/run_main` because this is intended to run entry
points with the ESM loader and not just any module.
- Documents how top-level await is handled.
PR-URL: https://github.com/nodejs/node/pull/51999
Fixes: https://github.com/nodejs/node/issues/42868
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
This is similar to the `queryObjects()` console API provided by the
Chromium DevTools console. It can be used to search for objects that
have the matching constructor on its prototype chain in the entire
heap, which can be useful for memory leak regression tests. To avoid
surprising results, users should avoid using this API on constructors
whose implementation they don't control, or on constructors that can
be invoked by other parties in the application.
To avoid accidental leaks, this API does not return raw references to
the objects found. By default, it returns the count of the objects
found. If `options.format` is `'summary'`, it returns an array
containing brief string representations for each object. The visibility
provided in this API is similar to what the heap snapshot provides,
while users can save the cost of serialization and parsing and directly
filer the target objects during the search.
We have been using this API internally for the test suite, which
has been more stable than any other leak regression testing
strategies in the CI. With a public implementation we can now
use the public API instead.
PR-URL: https://github.com/nodejs/node/pull/51927
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
In the SEA tests, if any of these steps fail:
1. Copy the executable
2. Inject the SEA blob
3. Signing the SEA
We skip the test because the error likely comes from the system or
postject and is not something the Node.js core can fix. We only leave
an exception for a basic test that test injecting empty files as
SEA to ensure the workflow is working (but we still skip if copying
fails or signing fails on Windows).
PR-URL: https://github.com/nodejs/node/pull/51887
Refs: https://github.com/nodejs/node/issues/49630
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Previously it would not compile if the build is configured with
--disable-single-executable-application because we use directives
to exclude the definition of SEA-related code completely. This patch
changes them so that the SEA code are still compiled and internals
can still check whether the executable is an SEA. The executable would
not try to load the SEA blob at all if SEA is disabled. If future
modifications to the C++ code attempt to load the SEA blob when SEA
is disabled, UNREACHABLE() would be raised. If user attempt to
generate the SEA blob with --experimental-sea-config with an executable
that disables SEA, they would get an error.
PR-URL: https://github.com/nodejs/node/pull/51808
Fixes: https://github.com/nodejs/node/issues/51730
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Previously the test assumes that when the queued finalizer is run,
it must be run at a point where env->can_call_into_js() is false
(typically, during Environment shutdown), which is not certain.
If GC kicks in early and the second pass finalizer is queued before
the event loop runs the check callbacks, the finalizer would then
be called in check callbacks (via native immediates), where
the finalizer can still call into JS. Essentially, addons can't
make assumptions about where the queued finalizer would be called.
This patch updates the assertions in the test to account for that.
PR-URL: https://github.com/nodejs/node/pull/51898
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
In many cases we can query the environment variable directly instead
of encoding it into UTF8 first and then decoding it again from
UTF8 via an intermediate V8 string.
Drive-by: pass per_process::system_environment explicitly when
the real environment variables are supposed to be used instead of
relying on fallback with defaults.
PR-URL: https://github.com/nodejs/node/pull/51829
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
If the server is not referenced, it might go away too soon
and the client may not get enough ends for it to close
itself, resulting a timeout.
This patch updates the test to simply close the server when
enough requests have been processed, and keep the server
referenced while the test is ongoing.
Drive-by: add more logs to facilitate debugging.
PR-URL: https://github.com/nodejs/node/pull/51863
Refs: https://github.com/nodejs/reliability/issues/791
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
The child processes are supposed to get 4 messages (2 ends, 2 writes).
Previously the mustCall() wrapping the message listener attempts
to match exactly 1 invocation which is bound to fail but could be
swallowed if the child happens to be killed before the exit event
is fired for the mustCall() check to work. In the CI, on some
machines the kill() could happen after the child process finishes
with the mustCall() check, resulting in EPERM errors in kill().
This patch fixes the mustCall() checks (updating the expected
invocation count to 4) and swallow the errors when kill() fails,
which should be fine because they are only there for cleanup.
PR-URL: https://github.com/nodejs/node/pull/51841
Refs: https://github.com/nodejs/node/issues/51813
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Move the child process code into a fixture and split the test
so that it can be run in parallel and it's easier to identify
where the failure is coming from. Also use the
spawnSyncAndExitWithoutError() utility so that the test shows
complete information on failure.
Instead of marking all the wasi tests as flaky, only mark the
wasi-poll one which is flaking in the CI now.
PR-URL: https://github.com/nodejs/node/pull/51836
Refs: https://github.com/nodejs/node/issues/51822
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Previously we wrapped the embedder entry point callback into a
binding and then invoke the binding from JS land which was a bit
convoluted. Now we just call it directly from C++.
The main scripts that needed to tail call the embedder callback now
return the arguments in an array so that the C++ land can extract
the arguments and pass them to the callback. We also set
`PauseOnNextJavascriptStatement()` for --inspect-brk and mark
the bootstrap complete milestone directly in C++ for these
execution modes.
PR-URL: https://github.com/nodejs/node/pull/51557
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Previously we only had an internal assertion to ensure certain
code is executed before any user-provided CJS is run. This patch
adds another assertion for ESM.
Note that this internal state is not updated during source text
module execution via vm because to run any code via vm, some
user JS code must have already been executed anyway.
In addition this patch moves the states into internal/modules/helpers
to avoid circular dependencies. Also moves toggling the states to
true *right before* user code execution instead of after in case
we are half-way in the execution when internals try to check them.
PR-URL: https://github.com/nodejs/node/pull/51748
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
We previously build the ESM facade (synthetic modules re-exporting
builtin's exports) for builtins even when they are not directly
import'ed (which rarely happens for internal builtins as that
requires --expose-internals). This patch removes
the eager generation to avoid the overhead and the extra
promises created in facade building when it's not reqested by the user.
When the facade is needed the ESM loader that can be requested
it in the translator on-demand.
Drive-by: set the ModuleWrap prototype to null in the built-in
snapshot.
PR-URL: https://github.com/nodejs/node/pull/51669
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
By default V8 only compiles the top-level function and
skips code generation for inner functions - that would
only be done when those inner functions are invoked.
Since builtins are compiled as wrapped functions, most
functions that look visually top-level are not actually
included in the built-in code cache. For most of the
builtins this is not too bad because usually only a subset of
all builtin functions are needed by a particular
application and including all their code in the binary
would incur an unnecessary size overhead. But there is also
a subset of more commonly used builtins and it would be
better to include the inner functions in the built-in
code cache because they are more universally used by
most applications.
This patch changes the compilation strategy to eager compilation
(including inner functions) for the following scripts:
1. Primordials (internal/per_context/*), in all situations.
2. Bootstrap scripts (internal/bootstrap/*) and main scripts
(internal/main/*), when being compiled for built-in code
cache.
3. Any scripts loaded during built-in snapshot generation.
We can't compile the code eagerly during snapshot generation
and include them into the V8 snapshot itself just now because
we need to start the inspector before context deserialization
for coverage collection to work. So leave that as a TODO.
With this patch the binary size increases by about 666KB
(~0.6% increase) in return the worker startup can be 18-19% faster.
PR-URL: https://github.com/nodejs/node/pull/51672
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
It is safe to create the console properties for the global
console at snapshot build time. Streams must still be created
lazily however because they need special synchronization for
the handles.
PR-URL: https://github.com/nodejs/node/pull/51700
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Previously we had two encodings for JS files:
1. If a file contains only ASCII characters, encode it as a one-byte
string (interpreted as uint8_t array during loading).
2. If a file contains any characters with code point above 127,
encode it as a two-byte string (interpreted as uint16_t array
during loading).
This was done because V8 only supports Latin-1 and UTF16 encoding
as underlying representation for strings. To store the JS code
as external strings to save encoding cost and memory overhead
we need to follow the representations supported by V8.
Notice that there is a gap in the Latin1 range (128-255) that we
encoded as two-byte, which was an undocumented TODO for a long
time. That was fine previously because then files that contained
code points beyond the 0-127 range contained code points >255.
Now we have undici which contains code points in the range 0-255
(minus a replaceable code point >255). So this patch adds handling
for the 128-255 range to reduce the size overhead caused by encoding
them as two-byte. This could reduce the size of the binary by
~500KB and helps future files with this kind of code points.
Drive-by: replace `’` with `'` in undici.js to make it a Latin-1
only string. That could be removed if undici updates itself to
replace this character in the comment.
PR-URL: https://github.com/nodejs/node/pull/51605
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
With this patch:
Users can now include assets by adding a key-path dictionary
to the configuration as the `assets` field. At build time, Node.js
would read the assets from the specified paths and bundle them into
the preparation blob. In the generated executable, users can retrieve
the assets using the `sea.getAsset()` and `sea.getAssetAsBlob()` API.
```json
{
"main": "/path/to/bundled/script.js",
"output": "/path/to/write/the/generated/blob.blob",
"assets": {
"a.jpg": "/path/to/a.jpg",
"b.txt": "/path/to/b.txt"
}
}
```
The single-executable application can access the assets as follows:
```cjs
const { getAsset } = require('node:sea');
// Returns a copy of the data in an ArrayBuffer
const image = getAsset('a.jpg');
// Returns a string decoded from the asset as UTF8.
const text = getAsset('b.txt', 'utf8');
// Returns a Blob containing the asset.
const blob = getAssetAsBlob('a.jpg');
```
Drive-by: update the documentation to include a section dedicated
to the injected main script and refer to it as "injected main
script" instead of "injected module" because it's a script, not
a module.
PR-URL: https://github.com/nodejs/node/pull/50960
Refs: https://github.com/nodejs/single-executable/issues/68
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
This patch adds support for using
`vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER` as
`importModuleDynamically` in all APIs that take the option
except `vm.SourceTextModule`. This allows users to have a shortcut
to support dynamic import() in the compiled code without missing
the compilation cache if they don't need customization of the
loading process. We emit an experimental warning when the
`import()` is actually handled by the default loader through
this option instead of requiring `--experimental-vm-modules`.
In addition this refactors the documentation for
`importModuleDynamically` and adds a dedicated section for it
with examples.
`vm.SourceTextModule` is not supported in this patch because
it needs additional refactoring to handle `initializeImportMeta`,
which can be done in a follow-up.
PR-URL: https://github.com/nodejs/node/pull/51244
Fixes: https://github.com/nodejs/node/issues/51154
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
This puts it together with the cjsParseCache and reduces the
circular dependency on the singleton loader, which is the
only place where this cache is stored.
Drive-by: remove always-false module status check because there's
no longer a local module variable after
https://github.com/nodejs/node/pull/34605 which is now invalid
leftover code at this point and only doesn't throw because
we happen to have a top-level variable called module.
PR-URL: https://github.com/nodejs/node/pull/51157
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
- Add support for --build-snapshot-config which allows passing
snapshot configurations via a JSON configuration file.
- Add support for node::SnapshotConfig in the embedder API
The initial configurable options are:
- "builder" (SnapshotConfig::builder_script_path): path to the
builder script.
- "withoutCodeCache" (SnapshotFlags::kWithoutCodeCache): disable
code cache generation.
PR-URL: https://github.com/nodejs/node/pull/50453
Refs: https://github.com/nodejs/node/issues/42566
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
This patch implements a debugging flag that dumps the current
stack trace when a promise is created or resolved. To reduce
noise we ignore before and after events (as the execution is
less interesting) and use the per-isolate hook to avoid extra
JS frames. This flag can assist work in reducing unnecessary
promise overhead.
PR-URL: https://github.com/nodejs/node/pull/50899
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
- Make processBindingAllowList a separate list from
runtimeDeprecatedList and legacyWrapperList instead of being
an umbrella one, so it's easier to see the stages the bindings
are in.
- Cache process.binding() results so we don't need to mutate
runtimeDeprecatedList.
PR-URL: https://github.com/nodejs/node/pull/50773
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: James M Snell <jasnell@gmail.com>
We previously used a text that appears to be an excerpt of
https://zh.wikipedia.org/wiki/%E5%8D%97%E8%B6%8A%E5%9B%BD
and can have copyright/license complications. It may
also include some geopolitical nuances. The text has been
repeated through out the code base without much reuse.
This patch consolidates the fixtures by adding a common helper
string as `fixtures.utf8TestText` which is identical to a copy
in test/fixtures/utf8_test_text.txt. It also updates the text
to a copy of 蘭亭集序, It was chosen because:
1. It's a well-known Chinese classical piece written in 353 CE
and therefore in public domain. The string is copied from
https://zh.wikisource.org/zh-hant/%E8%98%AD%E4%BA%AD%E9%9B%86%E5%BA%8F
which contains a disclaimer of copyright for this reason.
2. The text is in suitable length for general UTF8 string
read/write tests (including punctuations, 389 code points and
1167 bytes).
3. This is also commonly used as reference text for Chinese text
layout tests.
4. It's a timeless and harmless preface for a collection of poems,
written by a uncontroversial figure who passed away >1600 years
ago and contains no geopolitical nuances. Background and an
English translation of this text can be found at
https://en.wikipedia.org/wiki/Lantingji_Xu
PR-URL: https://github.com/nodejs/node/pull/50732
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
This is a ESM benchmark, rewrite it so that we are directly
benchmarking the ESM import.meta paths and using number of
loads for op/s calculation, instead of doing it in startup
benchmarks and nesting number of process/workers spawn
for op/s calculation.
PR-URL: https://github.com/nodejs/node/pull/50683
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
When --node-builtin-modules-path is used, we read and create
new strings for builtins in each realm which increases the memory
usage. As a result GC may not be able to keep up with the
allocation done in the loop in the test.
As a workaround, give GC a bit more time by waiting for a timer
in the loop.
PR-URL: https://github.com/nodejs/node/pull/50735
Refs: https://github.com/nodejs/node/issues/50726
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
This benchmarks the startup of various CLI tools that are already
checked into the source code. We use --version because the output
tends to be minimal and fewer operations are done to generate
these so that the startup cost is still dominated by a more
indispensible part of the CLI.
PR-URL: https://github.com/nodejs/node/pull/50684
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
There is currently no reliable way to detect this leak because:
1. We cannot reliably get a reference to the channel from the
API to detect finalization without creating another strong reference.
2. This test does gc() and then checks memory usage - however the
use of gc() disables code aging which can actually lead to increased
memory usage overall, as it is not intended to be used to lower
memory usage in the first place.
3. The implementation of diagnostics channels relies on ephemeron gc
which is inefficient, it's not reliable to use the typical "create
a lot of objects and see if it crashes" trick to check leaks.
Skip the test for now until we find a way to test it reliably.
To avoid flakiness in the CI, it's better to remove an unreliable
test altogether.
PR-URL: https://github.com/nodejs/node/pull/50327
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Joyee Cheung