StringBytes::Write() already used simdutf to encode one-byte strings
as UTF-8 but sent every two-byte (UTF-16) string through
v8::String::WriteUtf8V2(), which is several times slower. That path
is behind Buffer.from(string), buf.write(), fs.write*() with string
data and every string written to a libuv stream, and JSON.stringify()
output is a two-byte string as soon as any value in the payload is
outside Latin-1.
Encode two-byte strings with simdutf as well whenever their UTF-8 form
is guaranteed to fit in the target: well-formed input is converted
directly, and input with unpaired surrogates is converted from a copy
passed through simdutf::to_well_formed_utf16(), which replaces each
unpaired surrogate with U+FFFD exactly like kReplaceInvalidUtf8 (this
mirrors what TextEncoder already does). Writes that have to truncate
at a character boundary keep using WriteUtf8V2(), so their output is
byte-for-byte unchanged, and so do strings of up to 32 code units, for
which V8 is already as fast (the same threshold TextEncoder uses).
buf.write() of a 2 KiB two-byte string improves ~5x (astral-heavy and
lone-surrogate strings ~3.5x and ~5x), Buffer.from() of a 64 KiB JSON
string ~2.7x; one-byte strings are unaffected.
Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/65324
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
StringBytes::StorageSize had a CHECK that fatal-asserted when a
hex-encoded string with an odd number of characters was written
through Writev (e.g. via HTTP requests which are automatically
corked). Writing the same string via a single Write did not crash
because StringBytes::Write delegates to HexDecode, which silently
drops the trailing incomplete nibble.
Remove the CHECK and let integer division handle odd lengths, which
is consistent with StringBytes::Size and HexDecode.
Fixes: https://github.com/nodejs/node/issues/45150
Signed-off-by: RajeshKumar11 <kakumanurajeshkumar@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/63658
Reviewed-By: Edy Silva <edigleyssonsilva@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
This change refactors `StringBytes::WriteUCS2()` in multiple ways.
The `flags` argument being passed to `WriteUCS2()` is not useful: the
only really relevant flag is `NO_NULL_TERMINATION` since V8 ignores
`REPLACE_INVALID_UTF8`, `HINT_MANY_WRITES_EXPECTED`, and
`PRESERVE_ONE_BYTE_NULL` for UTF-16 strings. However, `WriteUCS2()`
might not null-terminate the result correctly regardless of whether
`NO_NULL_TERMINATION` is set because it makes multiple calls to
`String::Write()` internally. For these reasons, this patch removes the
`flags` argument entirely and always assumes `NO_NULL_TERMINATION`.
Next, this patch replaces the calls to the deprecated function
`String::Write()` with calls to the new function `String::WriteV2()`,
which always succeeds and always writes a predictable number of
characters, removing the need to deal with a return value here.
Lastly, this patch simplifies the implementation of `WriteUCS2()` and
computes the exact number of characters `nchars` from the beginning,
removing the need to later check again if the number of characters is
zero.
PR-URL: https://github.com/nodejs/node/pull/58163
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Projects that seek to implement Node.js compatible APIs end up
needed to reproduce various bits of functionality internally in
order to faithfully replicate the Node.js behaviors. This is
particularly true for things like byte manipulation, base64 and
hex encoding, and other low-level operations. This change
proposes moving much of this low-level byte manipulation code
out of nodejs/src and into a new `nbytes` library. Initially this
new library will exist in the `deps` directory but the intent is
to spin out a new separate repository to be its home in the future.
Doing so will allow other projects to use the nbytes library with
exactly the same implementation as Node.js.
This commit moves only the byte swapping and legacy base64 handling
code. Additional commits will move additional byte manipulation
logic into the library.
PR-URL: https://github.com/nodejs/node/pull/53507
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
The pattern of getting the address of the element at index 0 of a
container is generally used to materialize a pointer to the backing
data of a container, however `std::string` and `std::vector`
provide a `data()` accessor to retrieve the data pointer which
should be preferred.
This also ensures that in the case that the container is empty, the
data pointer access does not perform an errant memory access.
PR-URL: https://github.com/nodejs/node/pull/47750
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Assuming that UncheckedMalloc(0) returns a non-nullptr is non-standard
and we use other allocators as well (e.g., OPENSSL_malloc) that do not
guarantee this behavior. It is the caller's responsibility to check that
size != 0 implies UncheckedMalloc(size) != nullptr, and that's exactly
what the checked variants (Malloc etc.) already do.
The current behavior is also inconsistent with UncheckedRealloc(), which
always returns a nullptr when the size is 0, and with the documentation
in src/README.md as well as with multiple comments in the source code.
This changes UncheckedMalloc(), UncheckedCalloc(), and
UncheckedRealloc() to always return a nullptr when the size is 0 instead
of doing fake allocations in UncheckedMalloc() and UncheckedCalloc()
while returning a nullptr from UncheckedRealloc(). This is consistent
with existing documentation and comments.
Refs: https://github.com/nodejs/node/issues/8571
Refs: https://github.com/nodejs/node/pull/8572
PR-URL: https://github.com/nodejs/node/pull/44543
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
This parameter was always being set to `nullptr` by its callers, either
explicitly or implicitly via the default argument. It was also buggy, as
in some cases it wouldn't be written to, potentially leaking stack
memory (see the early returns in `StringBytes::WriteUCS2`). Remove it
entirely.
PR-URL: https://github.com/nodejs/node/pull/44092
Reviewed-By: Feng Yu <F3n67u@outlook.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Two things in one on this commit:
(a) For the QUIC implementation, we need to separate out various bits
from node_crypto.cc to allow them to be reused. That's where this
commit starts.
(b) Quite a bit of the node_crypto.cc code was just messy in terms of
it's organization and lack of error handling and use of Local vs.
MaybeLocal. This cleans that up a bit and hopefully makes certain
parts a bit more manageable also.
Signed-off-by: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/32016
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Matteo Collina