Several Environments can share a thread, so state in src/ that belongs
to one of them cannot be kept in a thread_local. Add a cpplint rule
that rejects thread_local in src/ unless the declaration is marked with
NOLINTNEXTLINE(runtime/thread_local), and mark the existing uses, which
are per-thread by design.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/66411
Refs: https://github.com/nodejs/node/pull/66239
Reviewed-By: Anna Henningsen <anna@addaleax.net>
process._linkedBinding() is subject to the permission model's addon
scope since the check was added to GetLinkedBinding(). For an embedder
that implements part of its runtime as linked bindings this means its
own bootstrap cannot reach them under --permission unless the user
also passes --allow-addons, which allows loading addons from the file
system as well.
Add EnvironmentFlags::kNoAddonPermissionForLinkedBindings. When set,
GetLinkedBinding() skips the addon permission check for that
Environment and the worker threads it creates; process.dlopen() stays
gated and the default behavior is unchanged.
Refs: https://github.com/nodejs/node/pull/65432
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/66067
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Xuguang Mei <meixuguang@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
The operating system's dynamic loader cannot open a library that lives
in a mounted virtual file system: the reserved mount path has no real
inode. Native addons already handle this in require(): the loader
hands their bytes to process.dlopen(), which loads them from a
private, self-cleaning image - an anonymous in-memory memfd on Linux.
Make ffi.dlopen() and new DynamicLibrary() do the same transparently.
Mirroring the fs handler integration, the VFS hook installer sets a
library reader into node:ffi while at least one VFS is mounted and
clears it when the last one unmounts; DynamicLibrary consults it
before every load, so the dependency points from the VFS into ffi and
ffi never loads any VFS code. The reader hands the library's bytes to
the native constructor, which loads them from the same kind of image,
released right after the load, while library.path keeps reporting the
virtual path. Libraries on the real file system are unaffected and
load directly, and pay only a null check while no VFS is mounted.
The AddonImage materializer moves from an anonymous namespace in
node_binding.cc to node_binding.h so that node_ffi.cc can reuse it.
On Windows the image is now written and closed before the load,
because the loader shares read alone and a retained writable
delete-on-close handle failed the load with ERROR_SHARING_VIOLATION;
since a mapped image cannot be unlinked there, it is kept with the
module it loaded as and both are released at process exit. On POSIX
the image still never outlives the constructor call, so nothing is
left for dlclose() to clean up.
Also fix the VFS dlopen hook forwarding a missing flags argument as
undefined, which process.dlopen() coerces to 0 - not a valid
dlopen(2) mode - so loading any addon from the real file system
failed with EINVAL while a VFS was mounted.
Co-authored-by: Philipp Dunkel <pipobscure@users.noreply.github.com>
Signed-off-by: Matteo Collina <hello@matteocollina.com>
PR-URL: https://github.com/nodejs/node/pull/65909
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: James M Snell <jasnell@gmail.com>
A native addon inside a virtual file system could not be require()d:
dlopen() and LoadLibrary() open a shared object by path, and a VFS path
has no inode for them to open.
Read the addon's bytes from the VFS and load them from a private,
self-cleaning image instead, using the smallest on-disk footprint each
platform allows: on Linux an anonymous memfd loaded through
/proc/self/fd, so the bytes never reach the file system; on other POSIX
platforms a file in a 0700 mkdtemp() directory, unlinked right after
loading, with the mapping keeping it alive; on Windows a temp file
opened FILE_FLAG_DELETE_ON_CLOSE whose handle is held until exit.
This is internal. process.dlopen() keeps its documented
(module, filename[, flags]) signature and ignores anything further; the
bytes are passed to a dlopenBinary() reachable only through the
process_methods binding, which the VFS hook calls for VFS paths. Addons
on the real file system are untouched and load directly.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
PR-URL: https://github.com/nodejs/node/pull/65680
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: James M Snell <jasnell@gmail.com>
The `advanced` IPC serialization codec was implemented in JavaScript
(ChildProcessSerializer / ChildProcessDeserializer in
lib/internal/child_process/serialization.js). It allocated a wrapper
serializer/deserializer per message and crossed the JS/C++ boundary
several times for every message (writeHeader, writeValue, releaseBuffer,
readHeader, readValue and friends).
Move the codec into a native `ipc_serdes` binding that drives the V8
ValueSerializer/ValueDeserializer with a C++ delegate. The wire format
is preserved byte-for-byte: a big-endian uint32 length prefix followed
by the V8 payload, with ArrayBufferViews tagged as host objects so that
Node Buffers round-trip as Buffers rather than plain Uint8Arrays. The
JSON codec is left unchanged.
A cctest (test/cctest/test_node_ipc_serdes.cc) exercises the binding
directly, covering round-trips of primitives, objects, typed arrays and
Buffers (including the Buffer-vs-Uint8Array distinction) and asserting
the big-endian length-prefix framing.
Round-trip throughput
(benchmark/child_process/child-process-ipc-roundtrip):
payload before after change
64 B ~300k/s ~800k/s +166%
1 KiB ~272k/s ~616k/s +126%
16 KiB ~91k/s ~120k/s +32%
64 KiB ~30k/s ~35k/s +16%
The gain is largest for small messages, where per-message JavaScript
overhead dominated, and tapers for large messages, where the actual
serialization (already native) dominates.
Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
PR-URL: https://github.com/nodejs/node/pull/63933
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Decided to take a short break from the work on QUIC
to implement a DTLS API. Very experimental at this
point but the basic API is there (inspired by the
QUIC API work).
The implementation is based on OpenSSL's built-in
DTLS support and no other dependencies are required.
DTLS is a datagram-based version of TLS that is used
for things like WebRTC and CoAP. It provides similar
security guarantees as TLS but is designed to work over
UDP instead of TCP.
This shouldn't be considered ready for production
but it is a good starting point for experimentation
and feedback.
```bash
./configure --experimental-dtls
make -j{nproc}
./node --experimental-dtls my-dtls-app.js
```
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
PR-URL: https://github.com/nodejs/node/pull/63182
Fixes: https://github.com/nodejs/node/issues/61630
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Add a C++ API for diagnostics channels that allows native code to check
for subscribers and publish messages without unnecessary JS boundary
crossings. Uses a shared AliasedUint32Array buffer between C++ and JS
to track subscriber counts per channel, enabling a fast inline check
(HasSubscribers) that reads the buffer directly.
PR-URL: https://github.com/nodejs/node/pull/61869
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Adds a debug-only macro that can be used to track when a V8 fast API is
called. A map of counters is maintained in in thread-local storage and
an internal API can be called to get the total count associated with
a call id.
Specific tests are added and `crypto.timingSafeEqual` as well as
internal documentation are updated to show how to use the macro
and test fast API calls without running long loops.
PR-URL: https://github.com/nodejs/node/pull/54317
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
This is similar to the `queryObjects()` console API provided by the
Chromium DevTools console. It can be used to search for objects that
have the matching constructor on its prototype chain in the entire
heap, which can be useful for memory leak regression tests. To avoid
surprising results, users should avoid using this API on constructors
whose implementation they don't control, or on constructors that can
be invoked by other parties in the application.
To avoid accidental leaks, this API does not return raw references to
the objects found. By default, it returns the count of the objects
found. If `options.format` is `'summary'`, it returns an array
containing brief string representations for each object. The visibility
provided in this API is similar to what the heap snapshot provides,
while users can save the cost of serialization and parsing and directly
filer the target objects during the search.
We have been using this API internally for the test suite, which
has been more stable than any other leak regression testing
strategies in the CI. With a public implementation we can now
use the public API instead.
PR-URL: https://github.com/nodejs/node/pull/51927
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
This patch removes special case in the internal binding loader
for natives, and implements it using the builtins internal
binding. Internally we do not actually need the natives binding,
so implement it as a legacy wrapper instead.
PR-URL: https://github.com/nodejs/node/pull/48186
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
This patch makes the binding templates ObjectTemplates, since we
don't actually need the constructor function. This also avoids
setting the properties on prototype, and instead initializes them
directly on the object template.
Previously the initialization was similar to:
```
function Binding() {}
Binding.prototype.property = ...;
module.exports = new Binding;
```
Now it's similar to:
```
module.exports = { property: ... };
```
PR-URL: https://github.com/nodejs/node/pull/47913
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Instead of storing the function template of BaseObject for checking
if an object is BaseObject by calling HasInstance, simply checks
the first internal field of the object, which is always set in the
BaseObject constructor. This is simpler and faster
(there is now no need to iterate over the inheritance for the check).
PR-URL: https://github.com/nodejs/node/pull/47217
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
This is the initial work to bootstrap Web interfaces that are defined
with extended attributes `[Exposed=*]`.
The ShadowRealm instances are garbage-collected once it is
unreachable. However, V8 can not infer the reference cycles between
the per-realm strong persistent function handles and the realm's
context handle. To allow the context to be gc-ed once it is not
reachable, the per-realm persistent handles are attached to the
context's global object and the persistent handles are set as weak.
PR-URL: https://github.com/nodejs/node/pull/46809
Refs: https://github.com/nodejs/node/issues/42528
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
As discussed in https://github.com/nodejs/node/pull/45888, using a
global `BuiltinLoader` instance is probably undesirable in a world
in which embedders are able to create Node.js Environments with
different sources and therefore mutually incompatible code
caching properties.
This PR makes it so that `BuiltinLoader` is no longer a global
singleton and instead only shared between `Environment`s that
have a direct relation to each other, and addresses a few
thread safety issues along with that.
PR-URL: https://github.com/nodejs/node/pull/45942
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Bindings that need to be loaded in distinct contexts of node::Realm in
the same node::Environment instance needs to share the per-isolate
templates.
Add a new binding registration callback, which is called with
per-IsolateData. This allows bindings to define templates and
share them across realms eagerly, and avoid accidental modification on
the templates when the per-context callback is called multiple times.
This also tracks loaded bindings and built-in modules with node::Realm.
PR-URL: https://github.com/nodejs/node/pull/45547
Refs: https://github.com/nodejs/node/issues/42528
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
The THROW_ERR_* functions interpret the first argument as a printf-like
format string, which is problematic when it contains unsanitized user
input. This typically happens when a printf-like function is used to
produce the error message, which is then passed to a THROW_ERR_*
function, which again interprets the error message as a format string.
Fix such occurrences by properly formatting error messages using static
format strings only, and in a single step.
PR-URL: https://github.com/nodejs/node/pull/44314
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
The term "native module" dates back to some of the oldest code
in the code base. Within the context of Node.js core it usually
refers to modules that are native to Node.js (e.g. fs, http),
but it can cause confusion for people who don't work on this
part of the code base, as "native module" can also refer to
native addons - which is even the case in some of the API
docs and error messages.
This patch tries to make the usage of these terms more consistent.
Now within the context of Node.js core:
- JavaScript scripts that are built-in to Node.js are now referred
to as "built-in(s)". If they are available as modules,
they can also be referred to as "built-in module(s)".
- Dynamically-linked shared objects that are loaded into
the Node.js processes are referred to as "addons".
We will try to avoid using the term "native modules" because it could
be ambiguous.
Changes in this patch:
File names:
- node_native_module.h -> node_builtins.h,
- node_native_module.cc -> node_builtins.cc
C++ binding names:
- `native_module` -> `builtins`
`node::Environment`:
- `native_modules_without_cache` -> `builtins_without_cache`
- `native_modules_with_cache` -> `builtins_with_cache`
- `native_modules_in_snapshot` -> `builtins_in_cache`
- `native_module_require` -> `builtin_module_require`
`node::EnvSerializeInfo`:
- `native_modules` -> `builtins
`node::native_module::NativeModuleLoader`:
- `native_module` namespace -> `builtins` namespace
- `NativeModuleLoader` -> `BuiltinLoader`
- `NativeModuleRecordMap` -> `BuiltinSourceMap`
- `NativeModuleCacheMap` -> `BuiltinCodeCacheMap`
- `ModuleIds` -> `BuiltinIds`
- `ModuleCategories` -> `BuiltinCategories`
- `LoadBuiltinModuleSource` -> `LoadBuiltinSource`
`loader.js`:
- `NativeModule` -> `BuiltinModule` (the `NativeModule` name used in
`process.moduleLoadList` is kept for compatibility)
And other clarifications in the documentation and comments.
PR-URL: https://github.com/nodejs/node/pull/44135
Fixes: https://github.com/nodejs/node/issues/44036
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
This adds a --build-snapshot runtime option which is currently only
supported by the node_mksnapshot binary, and a --node-snapshot-main
configure option that makes use it to run a custom script when
building the embedded snapshot. The idea is to have this experimental
feature in core as a configure-time feature for now, and investigate
the renaming V8 bugs before we make it available to more users via
making it a runtime option.
PR-URL: https://github.com/nodejs/node/pull/42466
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Shelley Vohr