This is a major refactor of our Node’s parser. See `node_options.cc`
for how it is used, and `node_options-inl.h` for the bulk
of its implementation.
Unfortunately, the implementation has come to have some
complexity, in order to meet the following goals:
- Make it easy to *use* for defining or changing options.
- Keep it (mostly) backwards-compatible.
- No tests were harmed as part of this commit.
- Be as consistent as possible.
- In particular, options can now generally accept arguments
through both `--foo=bar` notation and `--foo bar` notation.
We were previously very inconsistent on this point.
- Separate into different levels of scope, namely
per-process (global), per-Isolate and per-Environment
(+ debug options).
- Allow programmatic accessibility in the future.
- This includes a possible expansion for `--help` output.
This commit also leaves a number of `TODO` comments, mostly for
improving consistency even more (possibly with having to modify
tests), improving embedder support, as well as removing pieces of
exposed configuration variables that should never have become
part of the public API but unfortunately are at this point.
PR-URL: https://github.com/nodejs/node/pull/22392
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Currently, node.js depends on inspector_protocol indirectly through the
dependency on v8.
This is a dependency violation that will make it hard to roll V8 into
Node if V8 gets a newer inspector protocol version with incompatible
API. In fact, this surfaced on one of our bots when we tried to roll new
inspector_protocol into V8.
This patch adds inspector protocol and its required dependencies to node
deps:
- jinja2
- markupsafe
PR-URL: https://github.com/nodejs/node/pull/21975
Reviewed-By: Eugene Ostroukhov <eostroukhov@google.com>
Reviewed-By: Aleksei Koziatinskii <ak239spb@gmail.com>
Add methods that allow inspection of heap snapshots and a JS
version of our own embedder graph.
These can be used in tests and might also prove useful for
ad-hoc debugging. Usage requires `--expose-internals` and
prints a warning similar to our other modules whose primary
purpose is test support.
PR-URL: https://github.com/nodejs/node/pull/21741
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
This will enable more detailed heap snapshots based on
a newer V8 API.
This commit itself is not tied to that API and could
be backported.
PR-URL: https://github.com/nodejs/node/pull/21742
Reviewed-By: James M Snell <jasnell@gmail.com>
Port of the V8 internal v8::tracing::TracedValue that allows
structured data to be included in the trace event. The v8 class
is not exported in the public API so we cannot use it directly.
This is a simplified and slightly modified port. This commit only
adds the class, it does not add uses of it. Those will come in
separate PRs/commits.
PR-URL: https://github.com/nodejs/node/pull/21475
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
This patch speeds up the startup time and reduce the startup memory
footprint by using V8 code cache when comiling builtin modules.
The current approach is demonstrated in the `with-code-cache`
Makefile target (no corresponding Windows target at the moment).
1. Build the binary normally (`src/node_code_cache_stub.cc` is used),
by now `internalBinding('code_cache')` is an empty object
2. Run `tools/generate_code_cache.js` with the binary, which generates
the code caches by reading source code of builtin modules off source
code exposed by `require('internal/bootstrap/cache').builtinSource`
and then generate a C++ file containing static char arrays of the
code cache, using a format similar to `node_javascript.cc`
3. Run `configure` with the `--code-cache-path` option so that
the newly generated C++ file will be used when compiling the
new binary. The generated C++ file will put the cache into
the `internalBinding('code_cache')` object with the module
ids as keys
4. The new binary tries to read the code cache from
`internalBinding('code_cache')` and use it to compile
builtin modules. If the cache is used, it will put the id
into `require('internal/bootstrap/cache').compiledWithCache`
for bookkeeping, otherwise the id will be pushed into
`require('internal/bootstrap/cache').compiledWithoutCache`
This patch also added tests that verify the code cache is
generated and used when compiling builtin modules.
The binary with code cache:
- Is ~1MB bigger than the binary without code cahe
- Consumes ~1MB less memory during start up
- Starts up about 60% faster
PR-URL: https://github.com/nodejs/node/pull/21405
Reviewed-By: John-David Dalton <john.david.dalton@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Gus Caplan <me@gus.host>
This patch split part of the bootstrappers into three files:
- `lib/internal/process/main_thread_only.js`: contains bootstrappers
that can only be run in the main thread, including
- `setupStdio` for the main thread that sets up `process.stdin`,
`process.stdout`, `process.error` that may interact with external
resources, e.g. TTY/File/Pipe/TCP sockets
- `setupProcessMethods` that setup methods changing process-global
states, e.g. `process.chdir`, `process.umask`, `process.setuid`
- `setupSignalHandlers`
- `setupChildProcessIpcChannel` that setup `process.send` for
child processes.
- `lib/internal/process/worker_thread_only.js`: contains bootstrappers
that can only be run in the worker threads, including
- `setupStdio` for the worker thread that are streams to be
manipulated or piped to the parent thread
- `lib/internal/process/per_thread.js`: contains bootstrappers
that can be run in all threads, including:
- `setupAssert` for `process.assert`
- `setupCpuUsage` for `process.cpuUsage`
- `setupHrtime` for `process.hrtime` and `process.hrtime.bigint`
- `setupMemoryUsage` for `process.memoryUsage`
- `setupConfig` for `process.config`
- `setupKillAndExit` for `process.kill` and `process.exit`
- `setupRawDebug` for `process._rawDebug`
- `setupUncaughtExceptionCapture` for
`process.setUncaughtExceptionCaptureCallback` and
`process.hasUncaughtExceptionCaptureCallback`
Hopefully in the future we can sort more bootstrappers in
`boostrap/node.js` into these three files and further group
them into functions that can be run before creating the
snapshot / after loading the snapshot.
This patch also moves most of the `isMainThread` conditionals
into the main bootstrapper instead of letting them scattered around
special-casing different implementations.
PR-URL: https://github.com/nodejs/node/pull/21378
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Scrypt is a password-based key derivation function that is designed to
be expensive both computationally and memory-wise in order to make
brute-force attacks unrewarding.
OpenSSL has had support for the scrypt algorithm since v1.1.0. Add a
Node.js API modeled after `crypto.pbkdf2()` and `crypto.pbkdf2Sync()`.
Changes:
* Introduce helpers for copying buffers, collecting openssl errors, etc.
* Add new infrastructure for offloading crypto to a worker thread.
* Add a `AsyncWrap` JS class to simplify pbkdf2(), randomBytes() and
scrypt().
Fixes: https://github.com/nodejs/node/issues/8417
PR-URL: https://github.com/nodejs/node/pull/20816
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Implement utilities for easier debugging of Node.js core code,
inspired by the HTTP/2 debugging code. Debugging is, however,
implemented at runtime rather than at compile time, controlled
through a new `NODE_DEBUG_NATIVE=categories` environment variable.
The runtime overhead in the debugging-disabled case amounts to
1 well-cachable one-byte read per debug call.
PR-URL: https://github.com/nodejs/node/pull/20987
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
This change adds a new inspector domain for receiving Node tracing
data.
1. Node.js now can extend Inspector protocol with new domains with
the API defined in the src/inspector/node_protocol.pdl.
2. Plumbing code will be generated at the build time. /json/protocol
HTTP endpoint returns both V8 and Node.js inspector protocol.
3. "NodeTracing" domain was introduced. It is based on the Chrome
"Tracing" domain.
PR-URL: https://github.com/nodejs/node/pull/20608
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ali Ijaz Sheikh <ofrobots@google.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
This moves the `assert` parts from `internal/errors` into an own
file. `internal/errors` got bigger and bigger and it was difficult
to keep a good overview of what was going on. While doing so it
also removes the `internalAssert` function and just lazy loads
`assert`.
PR-URL: https://github.com/nodejs/node/pull/20486
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
A bug was introduced together with the FixedQueue implementation for
process.nextTick which meant that the queue wouldn't necessarily
fully clear on each run through. Fix it and abstract the data
structure into an internal module that can later be used elsewhere.
PR-URL: https://github.com/nodejs/node/pull/20468
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
On inspector side we are going to deprecate .json protocol
definition formation and replace it with new .pdl format.
New format allows us to use multiline commends and contains less
noise in comparision then .json.
PR-URL: https://github.com/nodejs/node/pull/20141
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@keybase.io>
Removes the requirement to use `--trace-events-enabled` to enable
trace events. Tracing is enabled automatically if there are any
enabled categories.
Adds a new `trace_events` module with an API for enabling/disabling
trace events at runtime without a command line flag.
```js
const trace_events = require('trace_events');
const categories = [ 'node.perf', 'node.async_hooks' ];
const tracing = trace_events.createTracing({ categories });
tracing.enable();
// do stuff
tracing.disable();
```
Multiple `Tracing` objects may exist and be enabled at any point
in time. The enabled trace event categories is the union of all
enabled `Tracing` objects and the `--trace-event-categories`
flag.
PR-URL: https://github.com/nodejs/node/pull/19803
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Ali Ijaz Sheikh <ofrobots@google.com>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
- For Windows, nasm is new build requirements and openssl_no_asm is
set to 1 with warning if it is not installed.
- For use of openssl assemble codes, either gas_version >= 2.23,
xcode_version >= 5.0 ,llvm_version >= 3.3 or nasm_version >= 2.10 is
needed. Otherwise, openssl_no_asm is set to 1 with warning.
- FIPS is not supported in OpenSSL-1.1.0 so that it leads an error
when openssl_fips options is enabled in configure.
Fixes: https://github.com/nodejs/node/issues/4270
PR-URL: https://github.com/nodejs/node/pull/19794
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Anna Henningsen