As discussed in https://github.com/nodejs/node/pull/45888, using a
global `BuiltinLoader` instance is probably undesirable in a world
in which embedders are able to create Node.js Environments with
different sources and therefore mutually incompatible code
caching properties.
This PR makes it so that `BuiltinLoader` is no longer a global
singleton and instead only shared between `Environment`s that
have a direct relation to each other, and addresses a few
thread safety issues along with that.
PR-URL: https://github.com/nodejs/node/pull/45942
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
The term `Environment` in `kAllowedInEnvironment` and
`kDisallowedInEnvironment` has nothing to do with the commonly used
term `node::Environment`. Rename the member to `kAllowedInEnvvar`
and `kDisallowedInEnvvar` respectively to reflect they are options
of `OptionEnvvarSettings`.
As `OptionEnvvarSettings` is part of the public embedder APIs, the
legacy names are still preserved.
PR-URL: https://github.com/nodejs/node/pull/45057
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
We have been ignoring inspector port binding errors during startup.
Handling this error would be a breaking change and it's probably
surprising to refuse to launch the Node.js instance simply because
the inspector cannot listen to the port anyway. So just turn the
return value of InitializeInspector() and remove the TODOs for
handling the error.
PR-URL: https://github.com/nodejs/node/pull/44903
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Node.js still changes flags after initializationg; either because
tests need to set their own flags (which V8 tests also still allow),
or because it's explicitly requested via the "v8.setFlagsFromString"
method that Node.js provides.
PR-URL: https://github.com/nodejs/node/pull/44741
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Add an ExitCode enum class and use it throughout the code base
instead of hard-coding the exit codes everywhere. At the moment,
the exit codes used in many places do not actually conform to
what the documentation describes. With the new enums (which
are also available to the JS land as constants in an internal
binding) we could migrate to a more consistent usage of the
codes, and eventually expose the constants to the user land
when they are stable enough.
PR-URL: https://github.com/nodejs/node/pull/44746
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Commit dae283d96f from August 2020 introduced a call to EntropySource()
in SecretKeyGenTraits::DoKeyGen() in src/crypto/crypto_keygen.cc. There
are two problems with that:
1. It does not check the return value, it assumes EntropySource() always
succeeds, but it can (and sometimes will) fail.
2. The random data returned byEntropySource() may not be
cryptographically strong and therefore not suitable as keying
material.
An example is a freshly booted system or a system without /dev/random or
getrandom(2).
EntropySource() calls out to openssl's RAND_poll() and RAND_bytes() in a
best-effort attempt to obtain random data. OpenSSL has a built-in CSPRNG
but that can fail to initialize, in which case it's possible either:
1. No random data gets written to the output buffer, i.e., the output is
unmodified, or
2. Weak random data is written. It's theoretically possible for the
output to be fully predictable because the CSPRNG starts from a
predictable state.
Replace EntropySource() and CheckEntropy() with new function CSPRNG()
that enforces checking of the return value. Abort on startup when the
entropy pool fails to initialize because that makes it too easy to
compromise the security of the process.
Refs: https://hackerone.com/bugs?report_id=1690000
Refs: https://github.com/nodejs/node/pull/35093
RAND_bytes() may return 0 to indicate an error, in which case the buffer
might not have been filled with random data at all. Instead of ignoring
this case, let V8 use its own entropy source. Historically, this used to
be a weak source of entropy, but V8 now implements a proper source even
on Windows. And even if V8's own entropy source turns out to be weak, it
does not matter much: V8's PRNG itself is not cryptographically secure,
so even if it is seeded from a cryptographically secure entropy source,
it does not produce cryptographically secure random numbers.
PR-URL: https://github.com/nodejs/node/pull/44493
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
This patch stores the metadata about the Node.js binary
into the SnapshotData and adds fields denoting how the
snapshot was generated, on what platform it was
generated as well as the V8 cached data version flag.
Instead of simply crashing when the metadata doesn't
match, Node.js now prints an error message and exit with
1 for the customized snapshot, or ignore the snapshot
and start from scratch if it's the default one.
PR-URL: https://github.com/nodejs/node/pull/44132
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
The term "native module" dates back to some of the oldest code
in the code base. Within the context of Node.js core it usually
refers to modules that are native to Node.js (e.g. fs, http),
but it can cause confusion for people who don't work on this
part of the code base, as "native module" can also refer to
native addons - which is even the case in some of the API
docs and error messages.
This patch tries to make the usage of these terms more consistent.
Now within the context of Node.js core:
- JavaScript scripts that are built-in to Node.js are now referred
to as "built-in(s)". If they are available as modules,
they can also be referred to as "built-in module(s)".
- Dynamically-linked shared objects that are loaded into
the Node.js processes are referred to as "addons".
We will try to avoid using the term "native modules" because it could
be ambiguous.
Changes in this patch:
File names:
- node_native_module.h -> node_builtins.h,
- node_native_module.cc -> node_builtins.cc
C++ binding names:
- `native_module` -> `builtins`
`node::Environment`:
- `native_modules_without_cache` -> `builtins_without_cache`
- `native_modules_with_cache` -> `builtins_with_cache`
- `native_modules_in_snapshot` -> `builtins_in_cache`
- `native_module_require` -> `builtin_module_require`
`node::EnvSerializeInfo`:
- `native_modules` -> `builtins
`node::native_module::NativeModuleLoader`:
- `native_module` namespace -> `builtins` namespace
- `NativeModuleLoader` -> `BuiltinLoader`
- `NativeModuleRecordMap` -> `BuiltinSourceMap`
- `NativeModuleCacheMap` -> `BuiltinCodeCacheMap`
- `ModuleIds` -> `BuiltinIds`
- `ModuleCategories` -> `BuiltinCategories`
- `LoadBuiltinModuleSource` -> `LoadBuiltinSource`
`loader.js`:
- `NativeModule` -> `BuiltinModule` (the `NativeModule` name used in
`process.moduleLoadList` is kept for compatibility)
And other clarifications in the documentation and comments.
PR-URL: https://github.com/nodejs/node/pull/44135
Fixes: https://github.com/nodejs/node/issues/44036
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
So far, process initialization has been a bit all over the place
in Node.js. `InitializeNodeWithArgs()` is our main public API
for this, but inclusion of items in it vs. `InitializeOncePerProcess()`
and `PlatformInit()` has been random at best. Likewise,
some pieces of initialization have been guarded by
`NODE_SHARED_MODE`, but also fairly randomly and without
any meaningful connection to shared library usage.
This leaves embedders in a position to cherry-pick some of
the initialization code into their own code to make their
application behave like typical Node.js applications to the
degree to which they desire it.
Electron takes an alternative route and makes direct use of
`InitializeOncePerProcess()` already while it is a private
API, with a `TODO` to add it to the public API in Node.js.
This commit addresses that `TODO`, and `TODO`s around the
`NODE_SHARED_MODE` usage. Specifically:
- `InitializeOncePerProcess()` and `TearDownOncePerProcess()`
are added to the public API.
- The `flags` option of these functions are merged with the
`flags` option for `InitializeNodeWithArgs()`, since they
essentially share the same semantics.
- The return value of the function is made an abstract class,
rather than a struct, for easier API/ABI stability.
- Initialization code from `main()` is brought into these
functions (since that makes sense in general).
- Add a `TODO` for turning `InitializeNodeWithArgs()` into
a small wrapper around `InitializeOncePerProcess()` and
eventually removing it (at least one major release cycle
each, presumably).
- Remove `NODE_SHARED_MODE` guards and replace them with
runtime options.
PR-URL: https://github.com/nodejs/node/pull/44121
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
This patch:
- Make NativeModuleLoader::LookupAndCompile() detect parameters based
on module IDs. This allows us to compile more builtins when
generating the embedded bootstrap, including
- internal/per_context/*
- internal/bootstrap/*
- internal/main/*
- Move pre_execution.js to lib/internal/process as it needs to be
compiled as a regular built-in module, unlike other scripts
in lib/internal/bootstrap
- Move markBootstrapComplete() to the performance binding instead of
making it a function-wrapper-based global to reduce number of
special cases.
PR-URL: https://github.com/nodejs/node/pull/44018
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
This patch introduces `--build-snapshot` and `--snapshot-blob` options
for creating and using user land snapshots.
For the initial iteration, user land CJS modules and ESM are not yet
supported in the snapshot, so only one single file can be snapshotted
(users can bundle their applications into a single script with their
bundler of choice to build a snapshot though).
A subset of builtins should already work, and support for more builtins
are being added. This PR includes tests checking that the TypeScript
compiler and the marked markdown renderer (and the builtins they use)
can be snapshotted and deserialized.
To generate a snapshot using `snapshot.js` as entry point and write the
snapshot blob to `snapshot.blob`:
```
$ echo "globalThis.foo = 'I am from the snapshot'" > snapshot.js
$ node --snapshot-blob snapshot.blob --build-snapshot snapshot.js
```
To restore application state from `snapshot.blob`, with `index.js` as
the entry point script for the deserialized application:
```
$ echo "console.log(globalThis.foo)" > index.js
$ node --snapshot-blob snapshot.blob index.js
I am from the snapshot
```
Users can also use the `v8.startupSnapshot` API to specify an entry
point at snapshot building time, thus avoiding the need of an additional
entry script at deserialization time:
```
$ echo "require('v8').startupSnapshot.setDeserializeMainFunction(() => console.log('I am from the snapshot'))" > snapshot.js
$ node --snapshot-blob snapshot.blob --build-snapshot snapshot.js
$ node --snapshot-blob snapshot.blob
I am from the snapshot
```
Note that this patch only adds functionality to the `node` executable
for building run-time user-land snapshots, the generated snapshot is
stored into a separate file on disk. Building a single binary with both
Node.js and an embedded snapshot has already been possible with the
`--node-snapshot-main` option to the `configure` script if the user
compiles Node.js from source. It would be a different task to enable the
`node` executable to produce a single binary that contains both Node.js
and an embedded snapshot without building Node.js from source, which
should be layered on top of the SEA (Single Executable Apps) initiative.
Known limitations/bugs that are being fixed in the upstream:
- V8 hits a DCHECK when deserializing certain mutated globals, e.g.
`Error.stackTraceLimit` (it should work fine in the release build,
however): https://chromium-review.googlesource.com/c/v8/v8/+/3319481
- Layout of V8's read-only heap can be inconsistent after
deserialization, resulting in memory corruption:
https://bugs.chromium.org/p/v8/issues/detail?id=12921
PR-URL: https://github.com/nodejs/node/pull/38905
Refs: https://github.com/nodejs/node/issues/35711
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
This commit adds the setting of an appname (configuration section
name), 'nodejs_conf', to be used when reading OpenSSL configuration
files.
The motivation for this is that currently the default OpenSSL
configuration, 'openssl_conf', element will be used which may be
undesirable as it might configure OpenSSL in unwanted ways. With this
commit it is still possible to use a default openssl.cnf file but the
only section that Node.js will read from is a section named
'nodejs_conf'.
PR-URL: https://github.com/nodejs/node/pull/43124
Refs: https://github.com/nodejs/node/issues/40366
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Beth Griggs <bgriggs@redhat.com>
This adds a --build-snapshot runtime option which is currently only
supported by the node_mksnapshot binary, and a --node-snapshot-main
configure option that makes use it to run a custom script when
building the embedded snapshot. The idea is to have this experimental
feature in core as a configure-time feature for now, and investigate
the renaming V8 bugs before we make it available to more users via
making it a runtime option.
PR-URL: https://github.com/nodejs/node/pull/42466
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Instead of passing the snapshot blob, the per-isolate data
indices and the EnvSerializeInfo separately, use the aggregate
type Snapshot to carry these around, and refactor
NodeMainInstance so that it owns the v8::Isolate::CreateParams
when it owns its isolate. This also gets rid of the owns_isolate_
and deserialize_mode_ booleans in NodeMainInstance since
NodeMainInstance can compute these by just checking if it has
pointers to the CreateParams or the SnapshotData.
PR-URL: https://github.com/nodejs/node/pull/42360
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
This change allows all boolean flags to be negated using the `--no-`
prefix.
Flags that are `true` by default (for example `--deprecation`) are
still documented as negations.
With this change, it becomes possible to easily flip the default
value of a boolean flag and to override the value of a flag passed
in the NODE_OPTIONS environment variable.
`process.allowedNodeEnvironmentFlags` contains both the negated and
non-negated versions of boolean flags.
Co-authored-by: Anna Henningsen <anna@addaleax.net>
PR-URL: https://github.com/nodejs/node/pull/39023
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
This patch:
- Moves the snapshot building code to src/ so that we can reuse it
later when generating custom snapshots from an entry point accepted
by the node binary.
- Create a SnapshotData struct that incorporates all the data useful
for a snapshot blob, including both the V8 data and the Node.js
data.
PR-URL: https://github.com/nodejs/node/pull/38902
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
This commit moves the check for FIPS from the crypto module
initialization to process startup.
The motivation for this is that when OpenSSL is not FIPS enabled and the
command line options --enable-fips, or --force-fips are used, there will
only be an error raised if the crypto module is used. This can be
surprising and we have gotten feedback that users assumed that there
would be an error if these options were specified and FIPS is not
available.
PR-URL: https://github.com/nodejs/node/pull/38859
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Richard Lau <rlau@redhat.com>
Anna Henningsen