Commit Graph
46 Commits
Author SHA1 Message Date
Pulkit Gupta dad666ad06 test_runner: fixed test object is incorrectly passed to setup()
PR-URL: https://github.com/nodejs/node/pull/50982
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-12-24 08:23:54 +00:00
Pulkit Gupta f801b58e77 test_runner: fix infinite loop when files are undefined in test runner
PR-URL: https://github.com/nodejs/node/pull/51047
Fixes: https://github.com/nodejs/node/issues/48823
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-12-10 09:35:22 +00:00
Moshe Atlow 3cce03a03f test_runner: pass abortSignal to test files
PR-URL: https://github.com/nodejs/node/pull/50630
Fixes: https://github.com/nodejs/node/issues/50583
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-11-10 22:09:15 +00:00
Raz Luvaton 527589b755 test_runner: disallow array in run options
PR-URL: https://github.com/nodejs/node/pull/49935
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-10-06 20:21:15 +00:00
Colin Ihrig 4b35a9cfda test_runner: add test location for FileTests
This commit adds the previously missing test location for
FileTest tests.

Fixes: https://github.com/nodejs/node/issues/49926
Fixes: https://github.com/nodejs/node/issues/49927
PR-URL: https://github.com/nodejs/node/pull/49999
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2023-10-03 18:19:51 +00:00
Moshe Atlow c9e72e34ab test_runner: accept testOnly in run
PR-URL: https://github.com/nodejs/node/pull/49753
Fixes: https://github.com/nodejs/node/issues/49733
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
2023-09-24 10:46:13 +00:00
Raz Luvaton ee391f3781 test_runner: add __proto__ null
PR-URL: https://github.com/nodejs/node/pull/48663
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2023-07-29 13:22:16 +00:00
Moshe Atlow 48345d0f62 watch: decrease debounce rate
PR-URL: https://github.com/nodejs/node/pull/48926
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-07-27 20:39:14 +00:00
Moshe Atlow b814de6d77 watch: use debounce instead of throttle
PR-URL: https://github.com/nodejs/node/pull/48926
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-07-27 20:39:13 +00:00
Raz Luvaton 99881304c4 test_runner: add shards support
PR-URL: https://github.com/nodejs/node/pull/48639
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-07-05 03:57:28 +00:00
Moshe Atlow 7cd4e70948 test_runner: support passing globs
PR-URL: https://github.com/nodejs/node/pull/47653
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-06-25 14:40:34 +00:00
Chemi Atlow 78f6952751 fs: use kResistStopPropagation
PR-URL: https://github.com/nodejs/node/pull/48521
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-06-24 15:52:38 +00:00
Moshe Atlow 5d685e4198 test_runner: emit test:watch:drained event
PR-URL: https://github.com/nodejs/node/pull/48259
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-06-02 00:28:09 +00:00
Moshe Atlow ca71ae6549 test_runner: stop watch mode when abortSignal aborted
PR-URL: https://github.com/nodejs/node/pull/48259
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-06-02 00:28:08 +00:00
Moshe Atlow e74465d678 test_runner: pass FORCE_COLOR to child process
PR-URL: https://github.com/nodejs/node/pull/48057
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-05-25 10:49:39 +00:00
Moshe Atlow 40a7a055c6 test_runner: dont split lines on test:stdout
PR-URL: https://github.com/nodejs/node/pull/48057
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-05-25 10:49:38 +00:00
Moshe Atlow f0e70eede2 test_runner: fix test deserialize edge cases
PR-URL: https://github.com/nodejs/node/pull/48106
Fixes: https://github.com/nodejs/node/issues/48103
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-05-23 21:03:19 +00:00
Shiba 1229a22e51 test_runner: delegate stderr and stdout formatting to reporter
Introduce new `TestsStream` events `test:stderr` and `test:stdout`
to delegate `stderr` and `stdout` (e.g. `console.log()`) formatting
to the reporter. And patch existing reporters to:
- Spec: output the message as it is
- TAP: stay the same with existing `test:diagnostic`

PR-URL: https://github.com/nodejs/node/pull/48045
Fixes: https://github.com/nodejs/node/issues/48011
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-05-23 04:44:48 +00:00
Moshe Atlow 7116bc08d7 test_runner: use v8.serialize instead of TAP
PR-URL: https://github.com/nodejs/node/pull/47867
Fixes: https://github.com/nodejs/node/issues/44656
Fixes: https://github.com/nodejs/node/issues/47955
Fixes: https://github.com/nodejs/node/issues/47481
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-05-15 14:27:45 +00:00
cjihrig ab3a54f2ee test_runner: remove no-op validation
The code already checks if testNamePatterns is an
array, and converts it to an array if it is not.

PR-URL: https://github.com/nodejs/node/pull/47687
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Deokjin Kim <deokjin81.kim@gmail.com>
2023-04-25 16:03:51 +00:00
Moshe Atlow dccd25e5bb test_runner: fix test runner concurrency
PR-URL: https://github.com/nodejs/node/pull/47675
Fixes: https://github.com/nodejs/node/issues/47365
Fixes: https://github.com/nodejs/node/issues/47696
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-04-24 22:32:01 +00:00
Moshe Atlow dac02632a6 test_runner: fix test counting
PR-URL: https://github.com/nodejs/node/pull/47675
Fixes: https://github.com/nodejs/node/issues/47365
Fixes: https://github.com/nodejs/node/issues/47696
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-04-24 22:32:00 +00:00
atlowChemi c6279690be test_runner: add testNamePatterns to run api
Accept a `testNamePatterns` value in the `run` fn, and drill those
patterns to the spawned processes.

PR-URL: https://github.com/nodejs/node/pull/47648
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-04-23 15:30:23 +03:00
Moshe Atlow 46a3cff698 test_runner: fix spec skip detection
PR-URL: https://github.com/nodejs/node/pull/47537
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-04-16 07:26:37 +00:00
Moshe Atlow d1eaded0d1 test_runner: count nested tests
PR-URL: https://github.com/nodejs/node/pull/47094
Fixes: https://github.com/nodejs/node/issues/46762
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-03-21 08:16:38 +00:00
Steve Herzog 334bb179df test_runner: support defining test reporter in NODE_OPTIONS
Adds --test-reporter and --test-reporter-destination as
allowable options in NODE_OPTIONS. Also adds the CLI flag
--test-child-process to allow forcing the default
test-reporter for inter-process communication.

Fixes: https://github.com/nodejs/node/issues/46484
PR-URL: https://github.com/nodejs/node/pull/46688
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-03-14 16:59:35 +00:00
cjihrig 4e3bc06e65 test_runner: track bootstrapping process
This commit updates the test harness and root test to track
when bootstrapping has completed.

PR-URL: https://github.com/nodejs/node/pull/46962
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2023-03-07 13:59:37 -05:00
Moshe Atlow 9c61d414c4 test_runner: reset count on watch mode
PR-URL: https://github.com/nodejs/node/pull/46577
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-02-26 16:28:30 +02:00
Moshe Atlow ce49f79ed5 test_runner: bootstrap reporters before running tests
PR-URL: https://github.com/nodejs/node/pull/46737
Fixes: https://github.com/nodejs/node/issues/46747
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-02-21 23:06:13 +02:00
Moshe Atlow 3354f89dcf test_runner: flatten TAP output when running using --test
PR-URL: https://github.com/nodejs/node/pull/46440
Fixes: https://github.com/nodejs/node/issues/45833
Refs: https://github.com/nodejs/node/issues/45833
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-02-18 16:19:24 +00:00
Antoine du Hamel fe514bf960 lib: enforce use of trailing commas for functions
PR-URL: https://github.com/nodejs/node/pull/46629
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
2023-02-14 18:45:16 +01:00
Moshe Atlow 1118db718c test_runner: fix missing test diagnostics
PR-URL: https://github.com/nodejs/node/pull/46450
Fixes: https://github.com/nodejs/node/issues/45911
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-02-02 20:00:06 +00:00
Pulkit Gupta 61c65b066b test_runner: top-level diagnostics not ommited when running with --test
PR-URL: https://github.com/nodejs/node/pull/46441
Fixes: https://github.com/nodejs/node/issues/45910
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-02-02 10:42:43 +00:00
cjihrig 1ecc03eccd doc,lib,src,test: rename --test-coverage
Add experimental to the name as requested during review.

PR-URL: https://github.com/nodejs/node/pull/46017
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
2023-01-08 16:10:00 -05:00
cjihrig 8c95cb012b test_runner: add initial code coverage support
This commit adds code coverage functionality to the node:test
module. When node:test is used in conjunction with the new
--test-coverage CLI flag, a coverage report is created when
the test runner finishes. The coverage summary is forwarded to
any test runner reporters so that the display can be customized
as desired. This new functionality is compatible with the
existing NODE_V8_COVERAGE environment variable as well.

There are still several limitations, which will be addressed in
subsequent pull requests:

- Coverage is only reported for a single process. It is possible
  to merge coverage reports together. Once this is done, the
  --test flag will be supported as well.
- Source maps are not currently supported.
- Excluding specific files or directories from the coverage
  report is not currently supported. Node core modules and
  node_modules/ are excluded though.

PR-URL: https://github.com/nodejs/node/pull/46017
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
2023-01-08 16:09:41 -05:00
Moshe Atlow 9eb363a3e0 test_runner: report file in test runner events
PR-URL: https://github.com/nodejs/node/pull/46030
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-01-02 21:22:54 +00:00
Moshe Atlow a1b27b25bb test_runner: add reporters
PR-URL: https://github.com/nodejs/node/pull/45712
Fixes: https://github.com/nodejs/node/issues/45648
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2022-12-19 17:35:57 +00:00
Moshe Atlow 232efb06fe test_runner: parse yaml
PR-URL: https://github.com/nodejs/node/pull/45815
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-12-13 14:51:35 +00:00
Colin Ihrig cc2732d764 test_runner: don't parse TAP from stderr
This commit stops the test runner CLI from parsing child
process stderr as TAP. Per the TAP spec, TAP can only come from
stdout. To avoid losing stderr data, those logs are injected
into the parser as unknown tokens so that they are output as
comments.

PR-URL: https://github.com/nodejs/node/pull/45618
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2022-12-02 03:12:07 +00:00
Colin Ihrig fec0fbc333 test_runner: remove stdout and stderr from error
The CLI test runner parses the TAP output from child processes
now and displays it. This commit removes a previous workaround
for displaying child process stdout and stderr when tests
failures occurred.

PR-URL: https://github.com/nodejs/node/pull/45592
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2022-11-25 02:48:21 +00:00
Wassim Chegham f8ce9117b1 test_runner: add initial TAP parser
Work in progress

PR-URL: https://github.com/nodejs/node/pull/43525
Refs: https://github.com/nodejs/node/issues/43344
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2022-11-21 22:50:21 +00:00
Moshe Atlow 97355c74e8 test_runner: support watch mode
PR-URL: https://github.com/nodejs/node/pull/45214
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-11-13 21:40:26 +00:00
Joyee Cheung be525d7d04 src: consolidate exit codes in the code base
Add an ExitCode enum class and use it throughout the code base
instead of hard-coding the exit codes everywhere. At the moment,
the exit codes used in many places do not actually conform to
what the documentation describes. With the new enums (which
are also available to the JS land as constants in an internal
binding) we could migrate to a more consistent usage of the
codes, and eventually expose the constants to the user land
when they are stable enough.

PR-URL: https://github.com/nodejs/node/pull/44746
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
2022-10-06 12:41:16 +00:00
Moshe Atlow 9825a7e01d test: fix test-runner-inspect
PR-URL: https://github.com/nodejs/node/pull/44620
Fixes: https://github.com/nodejs/node/issues/44600
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-09-14 20:24:20 +00:00
Moshe Atlow a165193c5c test_runner: support using --inspect with --test
PR-URL: https://github.com/nodejs/node/pull/44520
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-09-10 17:01:42 +00:00
Moshe Atlow 59527de13d test_runner: support programmatically running --test
PR-URL: https://github.com/nodejs/node/pull/44241
Fixes: https://github.com/nodejs/node/issues/44023
Fixes: https://github.com/nodejs/node/issues/43675
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-08-24 15:54:44 +03:00