Add ZIP archive support to the node:zlib module through three classes
and a set of helpers:
- ZipEntry: a single archive member, with buffered reads (content()),
bounded-memory streaming reads (contentIterator()), and
create()/createStream() for building members.
- ZipFile: random access to an archive backed by a file descriptor,
reading members lazily without retaining their content and writing
new members in place; opened with open()/openSync().
- ZipBuffer: a zero-copy, in-memory view over an archive already held
in a Buffer.
createZipArchive() serializes a sequence of entries into an archive
byte stream, and setMaxZipContentSize() bounds the default in-memory
decompression size. Every operation has both an asynchronous and a
synchronous form.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/64339
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Aviv Keller <me@aviv.sh>
This brings back several improvements that were reverted by mistake when
landing https://redirect.github.com/nodejs/node/pull/64154
- Update the documentation about how the removal of side effects of
source collection
- Add non-enumerable `requireStack` and `topLevelAwaitLocations`
properties to `ERR_REQUIRE_ASYNC_MODULE`, the latter is only
populated when --experimental-print-required-tla is enabled
- Add "Required module: <url>" to the error message to identify the
required ESM entry point regardless of whether the flag is enabled
- Fix TLA caret column from 0-based to 1-based
- Store module source via a private symbol instead of a public property
- Use `hasAsyncGraph` (post-instantiation) in `throwIfAsyncGraph`
instead of walking the graph before instantiation
- Merge the require stack checking into the
`common.expectRequiredTLAError` helper.
- Removed tests that are made redundant by the snapshot tests
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64260
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Previously in order to collect the locations of the TLA, we wait
until right before evalutation to ensure instantiation is
completed so that we can use
v8::Module::GetStalledTopLevelAwaitMessages(). Now we try to
add an additioanl shortcut to the source code in the module wraps
instead during compilation for modules that contain TLAs and
use acron to locate the TLAs when we need to throw
ERR_REQUIRE_AYNSC_MODULE, so we can do this as early as before
instantiation and do not need to run the module again to collect
the locations.
In addition, we now collect the require stack for
ERR_REQUIRE_ASYNC_MODULE too for better metadata in the errors.
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/64154
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Port the domain module from createHook (async_hooks) to
AsyncLocalStorage using the AsyncContextFrame-based implementation.
Key changes:
- Use AsyncLocalStorage for domain context propagation instead of
async_hooks.createHook()
- Lazy initialization that triggers AsyncContextFrame prototype swap
on first domain use
- Use enterWith instead of ALS.run() so domain context is NOT
automatically restored on exception - this matches the original
domain.run() behavior where exit() only runs on success
- Add ERR_ASYNC_RESOURCE_DOMAIN_REMOVED error for AsyncResource.domain
- Update DEP0097 to End-of-Life status
- Remove tests that relied on the removed MakeCallback domain property
The domain module now uses the AsyncContextFrame version of
AsyncLocalStorage directly for proper context propagation across
async boundaries.
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/61095
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Map BoringSSL's native renegotiation failure to
ERR_TLS_RENEGOTIATION_UNSUPPORTED when TLSSocket#renegotiate() is
called. This avoids exposing an implementation-specific OpenSSL error
when the TLS backend does not support caller-initiated renegotiation.
Signed-off-by: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/63161
Reviewed-By: Tim Perry <pimterry@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Add a non-interactive probe mode to `node inspect` for inspecting
runtime values in a run-to-completion application. This allows
users to perform printf-style debugging without having to modify
the application code and clean up afterwards, it also supports
structured output to be consumed by tools.
Probe mode launches the application, sets one or more source
breakpoints, evaluates one expression at each hit, and prints a
single text or JSON report when execution ends.
Interface:
node inspect [--json] [--preview] [--timeout=<ms>] [--port=<port>]
--probe <file>:<line>[:<col>] --expr <expr> ...
[--] <script> [args...]
Example:
```js
// cli.js
let maxRSS = 0;
for (let i = 0; i < 2; i++) {
const { rss } = process.memoryUsage();
maxRSS = Math.max(maxRSS, rss);
}
```
```
$ node inspect --probe cli.js:5 --expr 'rss' cli.js
Hit 1 at cli.js:5
rss = 54935552
Hit 2 at cli.js:5
rss = 55083008
Completed
```
(Prettified JSON to fit in commit message restrictions).
```js
$ node inspect --json --probe cli.js:5 --expr 'rss' cli.js
{"v":1,"probes":[{"expr":"rss","target":["cli.js",5]}],
"results":[
{"probe":0,"event":"hit","hit":1,
"result":{"type":"number","value":55443456,
"description":"55443456"}},
{"probe":0,"event":"hit","hit":2,
"result":{"type":"number","value":55574528,
"description":"55574528"}},
{"event":"completed"}]}
```
Signed-off-by: Joyee Cheung <joyeec9h3@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/62713
Reviewed-By: Jan Martin <jan.krems@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
WebAssembly is disabled when Node.js is run with --jitless. The
internal TypeScript stripper relies on WebAssembly, and previously
failed obscurely with a ReferenceError in this mode.
This commit adds an explicit check for WebAssembly support when
loading the TypeScript parser. If WebAssembly is unavailable,
it now throws a descriptive ERR_WEBASSEMBLY_NOT_SUPPORTED error.
Fixes: https://github.com/nodejs/node/issues/61353
PR-URL: https://github.com/nodejs/node/pull/61382
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
ESM resolution and loading is now always synchronous from a
non-loader-hook thread. If no asynchrnous loader hooks are
registered, the resolution/loading is entirely synchronous.
If asynchronous loader hooks are registered, these would be
synchronous on the non-loader-hook thread, and asynchronous
on the loader hook thread.
This avoids several races caused by async/sync loading sharing
the same cache. In particular, asynchronous loader hooks
now works with `require(esm)` - previously it tends to break
due to races.
In addition, when an asynchronous loader hook
returns a promise that never settles, the main thread no longer
silently exits with exit code 13, leaving the code below
any module loading calls silently ignored without being executed.
Instead, it now throws ERR_ASYNC_LOADER_REQUEST_NEVER_SETTLED
which can be caught and handled by the main thread. If the module
request comes from `import()`, the never-settling promise is
now relayed to the result returned by `import()`.
Drive-by: when annotating the error about importing undetectable
named exports from CommonJS, it now no longer reload the source
code of the CommonJS module, and instead reuses format information
cached when the module was loaded for linking.
PR-URL: https://github.com/nodejs/node/pull/60380
Fixes: https://github.com/nodejs/node/issues/59666
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Split `module.link(linker)` into two synchronous step
`sourceTextModule.linkRequests()` and
`sourceTextModule.instantiate()`. This allows creating vm modules and
resolving the dependencies in a complete synchronous procedure.
This also makes `syntheticModule.link()` redundant. The link step for a
SyntheticModule is no-op and is already taken care in the constructor
by initializing the binding slots with the given export names.
PR-URL: https://github.com/nodejs/node/pull/59000
Refs: https://github.com/nodejs/node/issues/37648
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
This patch implements proxy support for HTTP and HTTPS clients and
agents in the `http` and `https` built-ins`. When NODE_USE_ENV_PROXY
is set to 1, the default global agent would parse the
HTTP_PROXY/http_proxy, HTTPS_PROXY/https_proxy, NO_PROXY/no_proxy
settings from the environment variables, and proxy the requests
sent through the built-in http/https client accordingly.
To support this, `http.Agent` and `https.Agent` now accept a few new
options:
- `proxyEnv`: when it's an object, the agent would read and parse
the HTTP_PROXY/http_proxy, HTTPS_PROXY/https_proxy, NO_PROXY/no_proxy
properties from it, and apply them based on the protocol it uses
to send requests. This option allows custom agents to
reuse built-in proxy support by composing options. Global agents
set this to `process.env` when NODE_USE_ENV_PROXY is 1.
- `defaultPort` and `protocol`: these allow setting of the default port
and protocol of the agents. We also need these when configuring
proxy settings and deciding whether a request should be proxied.
Implementation-wise, this adds a `ProxyConfig` internal class to handle
parsing and application of proxy configurations. The configuration
is parsed during agent construction. When requests are made,
the `createConnection()` methods on the agents would check whether
the request should be proxied. If yes, they either connect to the
proxy server (in the case of HTTP reqeusts) or establish a tunnel
(in the case of HTTPS requests) through either a TCP socket (if the
proxy uses HTTP) or a TLS socket (if the proxy uses HTTPS).
When proxying HTTPS requests through a tunnel, the connection listener
is invoked after the tunnel is established. Tunnel establishment uses
the timeout of the request options, if there is one. Otherwise it uses
the timeout of the agent.
If an error is encountered during tunnel establishment, an
ERR_PROXY_TUNNEL would be emitted on the returned socket. If the proxy
server sends a errored status code, the error would contain an
`statusCode` property. If the error is caused by timeout, the error
would contain a `proxyTunnelTimeout` property.
This implementation honors the built-in socket pool and socket limits.
Pooled sockets are still keyed by request endpoints, they are just
connected to the proxy server instead, and the persistence of the
connection can be maintained as long as the proxy server respects
connection/proxy-connection or persist by default (HTTP/1.1)
PR-URL: https://github.com/nodejs/node/pull/58980
Refs: https://github.com/nodejs/node/issues/57872
Refs: https://github.com/nodejs/node/issues/8381
Refs: https://github.com/nodejs/node/issues/15620
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
This tracks the asynchronicity in the ModuleWraps when they turn out to
contain TLA after instantiation, and throw the right error
(ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes
the freezing of ModuleWraps since it's not meaningful to freeze
this when the rest of the module loader is mutable, and we
can record the asynchronicity in the ModuleWrap right after
compilation after we get a V8 upgrade that contains
v8::Module::HasTopLevelAwait() instead of searching through
the module graph repeatedly which can be slow.
PR-URL: https://github.com/nodejs/node/pull/55520
Fixes: https://github.com/nodejs/node/issues/55516
Refs: https://github.com/nodejs/node/issues/52697
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
It is confusing to have both ERR_CRYPTO_SCRYPT_INVALID_PARAMETER and
ERR_CRYPTO_INVALID_SCRYPT_PARAMS. The former was the original error
code, added in 371103dae8, but parameter
validation gradually changed and now produces
ERR_CRYPTO_INVALID_SCRYPT_PARAMS for all parameter validation errors
coming from OpenSSL, as well as different error codes for validation
errors coming from JavaScript. The only remaining use of
ERR_CRYPTO_SCRYPT_INVALID_PARAMETER is in the validation logic that
ensures that no two synonymous options were passed. We already have an
error code for that particular case, ERR_INCOMPATIBLE_OPTION_PAIR, so
replace these last instances of ERR_CRYPTO_SCRYPT_INVALID_PARAMETER with
that error code and remove ERR_CRYPTO_SCRYPT_INVALID_PARAMETER. If there
ever is need again for such an error code, we can just use
ERR_CRYPTO_INVALID_SCRYPT_PARAMS.
Refs: https://github.com/nodejs/node/pull/35093
Refs: https://github.com/nodejs/node/pull/21525
Refs: https://github.com/nodejs/node/pull/20816
PR-URL: https://github.com/nodejs/node/pull/53305
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Philipp Dunkel