Commit Graph
29 Commits
Author SHA1 Message Date
Colin Ihrig 6067bea027 sqlite: restore changes from #55373
PR #55373 introduced a performance improvement for the all()
method of prepared statements. Those changes appear to have been
accidentally overwritten in #54181. This change restores the
overwritten code.

Refs: https://github.com/nodejs/node/pull/55373
Refs: https://github.com/nodejs/node/pull/54181
PR-URL: https://github.com/nodejs/node/pull/56908
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2025-04-16 03:29:12 -03:00
James M Snell f41bc4cfd7 src: improve error handling in node_sqlite
PR-URL: https://github.com/nodejs/node/pull/56891
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2025-04-16 03:29:11 -03:00
René bc672fcfdd sqlite: allow returning ArrayBufferViews from user-defined functions
PR-URL: https://github.com/nodejs/node/pull/56790
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2025-04-16 03:26:27 -03:00
Yagiz Nizipli a68f127a30 src: reduce string allocations on sqlite
PR-URL: https://github.com/nodejs/node/pull/57227
Backport-PR-URL: https://github.com/nodejs/node/pull/57533
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Edy Silva <edigleyssonsilva@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com>
2025-03-31 17:57:10 -03:00
Edy Silva 227603dc30 sqlite,test,doc: allow Buffer and URL as database location
PR-URL: https://github.com/nodejs/node/pull/56991
Backport-PR-URL: https://github.com/nodejs/node/pull/57533
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2025-03-31 17:56:50 -03:00
James M Snell 76921b822b src: minor cleanups on OneByteString usage
* Provide a OneByteString variant that accepts std::string_view
* Use FIXED_ONE_BYTE_STRING in appropriate places

Signed-off-by: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/56482
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
2025-02-03 20:09:56 +01:00
Alex YangandAntoine du Hamel 07ff3ddcb5 sqlite: support TypedArray and DataView in StatementSync
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/56385
Fixes: https://github.com/nodejs/node/issues/56384
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2025-02-03 20:09:54 +01:00
Bart Louwers 3462263e8b sqlite: pass conflict type to conflict resolution handler
PR-URL: https://github.com/nodejs/node/pull/56352
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2025-01-31 01:16:46 +01:00
Edigleysson Silva (Edy) 97c38352d0 sqlite: aggregate constants in a single property
PR-URL: https://github.com/nodejs/node/pull/56213
Fixes: https://github.com/nodejs/node/issues/56193
Refs: https://github.com/nodejs/node/issues/56193
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yongsheng Zhang <zyszys98@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
2025-01-05 13:05:33 -05:00
Colin Ihrig 2268c1ea8b sqlite: add support for custom functions
This commit adds support to node:sqlite for defining custom
functions that can be invoked from SQL.

Fixes: https://github.com/nodejs/node/issues/54349
PR-URL: https://github.com/nodejs/node/pull/55985
Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2025-01-05 13:05:28 -05:00
Alex Yang f5c6955722 sqlite: support db.loadExtension
PR-URL: https://github.com/nodejs/node/pull/53900
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2025-01-05 13:05:27 -05:00
tpoisseau b4041e554a sqlite: add StatementSync.prototype.iterate method
PR-URL: https://github.com/nodejs/node/pull/54213
Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2025-01-05 12:57:23 -05:00
Bart Louwers 457e73f4c9 sqlite: add support for SQLite Session Extension
PR-URL: https://github.com/nodejs/node/pull/54181
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2024-11-27 11:03:37 -05:00
Michaël Zasso 6858f7a4d3 src: use env strings to create sqlite results
PR-URL: https://github.com/nodejs/node/pull/55785
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2024-11-27 11:03:37 -05:00
Tobias Nießen 428701a6d8 sqlite: improve error handling using MaybeLocal
As per James' suggestion, consistently use MaybeLocal and avoid Check()
and ToLocalChecked() entirely.

Refs: https://github.com/nodejs/node/pull/54687
PR-URL: https://github.com/nodejs/node/pull/55571
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2024-11-27 11:03:31 -05:00
Tobias Nießen 4e5878536a sqlite: add readOnly option
Allow opening existing SQLite databases with SQLITE_OPEN_READONLY set.

PR-URL: https://github.com/nodejs/node/pull/55567
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
2024-11-27 11:03:27 -05:00
Tobias Nießen 8c35ad12de sqlite: refactor open options
Move options that are only relevant for opening the database into a
self-contained class.

PR-URL: https://github.com/nodejs/node/pull/55442
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2024-11-27 11:03:25 -05:00
Fedor Indutny c3c403040a sqlite: cache column names in stmt.all()
While the statement is running, it is impossible to modify the column
names and thus it is beneficial to create the host-language (JS) keys
once per all rows and reuse them for all results.

With this change the performance of `.all()` improves by around 25%
depending on the number of columns and rows in the result.

PR-URL: https://github.com/nodejs/node/pull/55373
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2024-11-27 01:23:22 -05:00
Tobias Nießen 57a9d3f15e sqlite: disable DQS misfeature by default
Double-quoted string (DQS) literals are not allowed by the SQL standard,
which defines that text enclosed in double quotes is to be interpreted
as an identifier only and never as a string literal. Nevertheless, for
historical reasons, SQLite allows double-quoted string literals in some
cases, which leads to inconsistent behavior and subtle bugs.

This commit changes the behavior of the built-in Node.js API for SQLite
such that the DQS misfeature is disabled by default. This is recommended
by the developers of SQLite. Users can explicitly enable DQS for
compatibility with legacy database schemas if necessary.

PR-URL: https://github.com/nodejs/node/pull/55297
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Jake Yuesong Li <jake.yuesong@gmail.com>
2024-10-10 01:42:13 +02:00
Tobias Nießen c126543374 sqlite: make sourceSQL and expandedSQL string-valued properties
Change sourceSQL and expandedSQL from being methods to being
string-valued properties. These fields

- are conceptually properties (and not actions),
- are derived deterministically from the current state of the object,
- require no parameters, and
- are inexpensive to compute.

Also, following the naming conventions of ECMAScript for new features,
most function names should usually contain a verb, whereas names of
(dynamically computed) properties generally should not, so the current
names also seem more appropriate for properties than for functions.

PR-URL: https://github.com/nodejs/node/pull/54721
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2024-10-09 22:57:21 +02:00
Tobias Nießen 67f5f46c56 sqlite: enable foreign key constraints by default
For historical reasons and to maintain compatibibility with legacy
database schemas, SQLite does not enable foreign key constraints by
default. For new applications, however, this behavior is undesirable.
Currently, any application that wishes to use foreign keys must use

    PRAGMA foreign_keys = ON;

to explicitly enable enforcement of such constraints.

This commit changes the behavior of the SQLite API built into Node.js
to enable foreign key constraints by default. This behavior can be
overridden by users to maintain compatibility with legacy database
schemas.

PR-URL: https://github.com/nodejs/node/pull/54777
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2024-10-09 22:57:21 +02:00
Tobias Nießen 7766349dd0 sqlite: fix segfault in expandedSQL
The call to sqlite3_expanded_sql() may return NULL depending on various
factors. Handle this case instead of running into a segmentation fault.

PR-URL: https://github.com/nodejs/node/pull/54687
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
2024-09-12 11:06:47 +02:00
Tobias Nießen 4c1b98ba2b sqlite: remove unnecessary auto assignment
Assignment is not necessary here at all.

PR-URL: https://github.com/nodejs/node/pull/54686
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
2024-09-12 11:06:42 +02:00
Michaël Zasso 72c554abab sqlite: return results with null prototype
These objects are dictionaries, and a query can return columns with
special names like `__proto__` (which would be ignored without this
change).

Also construct the object by passing vectors of properties for better
performance and improve error handling by using `MaybeLocal`.

PR-URL: https://github.com/nodejs/node/pull/54350
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2024-08-30 15:52:13 -03:00
cjihrig 94e2ea6f5c sqlite: ensure statement finalization on db close
This commit adds statement tracking to the DatabaseSync class.
When a database is closed manually or via garbage collection, it
will force all associated prepared statements to be finalized.
This should mitigate "zombie" connections which can introduce
test flakiness in the CI on Windows.

PR-URL: https://github.com/nodejs/node/pull/54014
Fixes: https://github.com/nodejs/node/issues/54006
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
2024-08-14 08:37:02 +02:00
Deokjin Kim 3fdcf7a47d src: return undefined if no rows are returned in SQLite
For now, { key: null, value: null} is returned even though
no rows are returned from database when `statement.get()`
is called. So return empty value if return value of
`sqlite3_step` is `SQLITE_DONE`.

PR-URL: https://github.com/nodejs/node/pull/53981
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2024-08-05 11:15:43 -03:00
Tobias Nießen 8664b9ad60 src,test: disallow unsafe integer coercion in SQLite
Currently, by default (i.e., when use_big_ints_ has not explicitly been
set to true), reading a SQLite integer value that is not a safe integer
in JavaScript is likely to yield an incorrect number.

Instead, err on the side of caution and throw if the stored integer is
not a safe integer in JavaScript and if use_big_ints_ has not been set
to true.

PR-URL: https://github.com/nodejs/node/pull/53851
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2024-08-05 11:15:42 -03:00
Tobias Nießen bac3a485f6 src: fix potential segmentation fault in SQLite
The Local<Value> returned from ColumnToValue() and ColumnNameToValue()
may be empty (if a JavaScript exception is pending), in which case a
segmentation fault may occur at the call sites, which do not check if
the Local<Value> is empty. Fix this bug returning early if an exception
is pending (as indicated by the Local being empty).

In the long term, these functions should return MaybeLocal instead of
Local, but this patch is supposed to be a minimal bug fix only.

PR-URL: https://github.com/nodejs/node/pull/53850
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2024-07-28 08:53:26 +02:00
Colin Ihrig b31394920d lib,src,test,doc: add node:sqlite module
PR-URL: https://github.com/nodejs/node/pull/53752
Fixes: https://github.com/nodejs/node/issues/53264
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-07-16 12:01:19 +02:00