mirror of
https://github.com/nodejs/node.git
synced 2026-11-09 15:11:11 -05:00
Shelley Vohr
55e4ca33f3
StringBytes::Write() already used simdutf to encode one-byte strings as UTF-8 but sent every two-byte (UTF-16) string through v8::String::WriteUtf8V2(), which is several times slower. That path is behind Buffer.from(string), buf.write(), fs.write*() with string data and every string written to a libuv stream, and JSON.stringify() output is a two-byte string as soon as any value in the payload is outside Latin-1. Encode two-byte strings with simdutf as well whenever their UTF-8 form is guaranteed to fit in the target: well-formed input is converted directly, and input with unpaired surrogates is converted from a copy passed through simdutf::to_well_formed_utf16(), which replaces each unpaired surrogate with U+FFFD exactly like kReplaceInvalidUtf8 (this mirrors what TextEncoder already does). Writes that have to truncate at a character boundary keep using WriteUtf8V2(), so their output is byte-for-byte unchanged, and so do strings of up to 32 code units, for which V8 is already as fast (the same threshold TextEncoder uses). buf.write() of a 2 KiB two-byte string improves ~5x (astral-heavy and lone-surrogate strings ~3.5x and ~5x), Buffer.from() of a 64 KiB JSON string ~2.7x; one-byte strings are unaffected. Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com> PR-URL: https://github.com/nodejs/node/pull/65324 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day> Reviewed-By: Daniel Lemire <daniel@lemire.me> Reviewed-By: James M Snell <jasnell@gmail.com>