This adds a new `test` option. Using it automatically uses a single
minimal option matrix to verify the benchmark works as expected.
Using the new `all` option makes sure all test suites are run.
On top of that the benchmarks will from now on report properly
what category might have a typo, if any.
The http duration was also refactored to use a option instead of
relying on a configuration setting.
The fixture folder is ignored as test suite from now on.
PR-URL: https://github.com/nodejs/node/pull/31396
Fixes: https://github.com/nodejs/node/issues/31083
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
* Let users provide more than one pattern by repeating the flag
* Add new flag --exclude to exclude patterns
* Add tests for --filter
* Document --filter
This commit also fixes a bug where things like
`compare.js --new --old binary --new binary` was acceptable (now the
script will exit and print the usage message).
PR-URL: https://github.com/nodejs/node/pull/29987
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Code in benchmark directory sometimes uses `function () {}` for
anonymous callbacks and sometimes uses `() => {}`. Multi-line arrays
sometimes have a trailing comma and sometimes do not. Update to always
use arrow functions for anonymous callbacks and trailing commas for
multiline arrays.
PR-URL: https://github.com/nodejs/node/pull/25944
Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
This changes the execution order from "iter, file, binary" to "file,
iter, binary". This means the csv no longer has to buffered completely.
This also has the added effect that stopping compare.js early or
interfering with performance only affects a single benchmark, instead of
all of them.
Refs: https://github.com/nodejs/node/issues/8659
PR-URL: https://github.com/nodejs/node/pull/9064
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
In order to comply with linting rules used in the rest of the code base,
eliminate redeclared variables. A conservative approach is used so as to
avoid unintentional performance issues (for example, as might be seen in
some situations when using `let` instead of `var`).
PR-URL: https://github.com/nodejs/node/pull/5468
Reviewed-By: Brian White <mscdex@mscdex.net>
Before this commit, only benchmark targets defined in Makefile could
be used. This commit allows execution of common.js directly and
passing of filter arguments directly, allowing you to run either a
subset of benchmarks or a single specific benchmark for comparison.
PR-URL: https://github.com/iojs/io.js/pull/711
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
The benchmark compare would drop the last run of the binary pairs. So
when they were only run once an error would arise because no data was
generated for the second binary.
This will run the benchmarks the number of times specified by NODE_BENCH_RUNS,
to attempt to reduce variability.
If the number of runs is high enough, it'll also throw out the top and bottom
quartiles, since that's where the outliers will be.
It's not very fancy statistics-fu, but it's better than nothing.
Also, linted this file. It had tabs in it. TABS!
Antoine du Hamel