Benchmarking¶
The galois
package comes with benchmarking tests. They are contained in the benchmarks/
folder. They are pytest
tests using the pytest-benchmark
extension.
Running a benchmark test¶
To run a benchmark, invoke pytest
on the benchmarks/
folder or a specific test set (e.g., benchmarks/test_field_arithmetic.py
). It is also
advised to pass extra arguments to format the display --benchmark-columns=min,max,mean,stddev,median
and --benchmark-sort=name
.
$ pytest benchmarks/test_field_arithmetic.py --benchmark-columns=min,max,mean,stddev,median --benchmark-sort=name
======================================================================================= test session starts ========================================================================================
platform linux -- Python 3.8.10, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/matt/repos/galois, configfile: setup.cfg
plugins: benchmark-3.4.1, extra-durations-0.1.3, anyio-2.1.0
collected 56 items
benchmarks/test_field_arithmetic.py ........................................................ [100%]
==================================================================================== sum of all tests durations ====================================================================================
25.80s
----------------------- benchmark "GF(2) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 8 tests ------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------
test_add 54.9031 (1.04) 238.1941 (1.19) 62.2950 (1.06) 14.3986 (1.30) 58.4391 (1.03)
test_additive_inverse 52.8959 (1.0) 237.0442 (1.18) 58.6939 (1.0) 11.1021 (1.0) 56.5751 (1.0)
test_divide 207.9280 (3.93) 636.3150 (3.17) 234.0875 (3.99) 38.3417 (3.45) 220.6110 (3.90)
test_multiplicative_inverse 188.9290 (3.57) 652.8331 (3.25) 229.7088 (3.91) 57.9788 (5.22) 201.3633 (3.56)
test_multiply 54.6200 (1.03) 228.7410 (1.14) 61.9206 (1.05) 12.5660 (1.13) 58.8121 (1.04)
test_power 229.3210 (4.34) 284.5279 (1.42) 246.6029 (4.20) 22.5143 (2.03) 238.7560 (4.22)
test_scalar_multiply 1,561.2941 (29.52) 3,148.2361 (15.67) 2,058.2764 (35.07) 361.3528 (32.55) 1,969.9985 (34.82)
test_subtract 54.5362 (1.03) 200.8791 (1.0) 62.6200 (1.07) 13.5863 (1.22) 59.0470 (1.04)
------------------------------------------------------------------------------------------------------------------------------------------
---------------------- benchmark "GF(257) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 8 tests -----------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------
test_add 161.1579 (1.22) 583.2699 (1.20) 197.0225 (1.18) 52.4775 (1.15) 175.6141 (1.21)
test_additive_inverse 132.4308 (1.0) 546.5120 (1.12) 166.7050 (1.0) 46.9263 (1.03) 144.6060 (1.0)
test_divide 6,306.7721 (47.62) 7,277.9991 (14.94) 6,658.3270 (39.94) 374.7916 (8.21) 6,597.4400 (45.62)
test_multiplicative_inverse 6,299.4179 (47.57) 6,477.2971 (13.29) 6,392.0670 (38.34) 75.0566 (1.64) 6,367.5269 (44.03)
test_multiply 337.8210 (2.55) 487.2240 (1.0) 362.2038 (2.17) 45.6575 (1.0) 344.7749 (2.38)
test_power 5,026.9060 (37.96) 5,523.7100 (11.34) 5,273.6145 (31.63) 204.7190 (4.48) 5,208.3279 (36.02)
test_scalar_multiply 1,033.5341 (7.80) 2,472.3031 (5.07) 1,223.1746 (7.34) 205.6682 (4.50) 1,157.2575 (8.00)
test_subtract 172.0500 (1.30) 655.0739 (1.34) 211.9266 (1.27) 59.9908 (1.31) 188.1695 (1.30)
------------------------------------------------------------------------------------------------------------------------------------------
------------------------ benchmark "GF(257) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-lookup'": 8 tests ------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------
test_add 175.6609 (1.23) 217.3430 (1.22) 182.2442 (1.20) 12.6704 (1.09) 177.3606 (1.23)
test_additive_inverse 142.2670 (1.0) 178.2598 (1.0) 152.1027 (1.0) 13.3374 (1.15) 144.3694 (1.0)
test_divide 414.2381 (2.91) 543.3671 (3.05) 473.2308 (3.11) 50.1936 (4.32) 475.6054 (3.29)
test_multiplicative_inverse 336.2088 (2.36) 459.0931 (2.58) 363.2939 (2.39) 40.4252 (3.48) 342.7650 (2.37)
test_multiply 319.1240 (2.24) 414.7580 (2.33) 342.5294 (2.25) 33.0754 (2.85) 322.0430 (2.23)
test_power 446.4111 (3.14) 505.2469 (2.83) 461.2746 (3.03) 24.9575 (2.15) 449.4530 (3.11)
test_scalar_multiply 1,037.2710 (7.29) 2,068.3082 (11.60) 1,197.2376 (7.87) 185.2372 (15.94) 1,137.1800 (7.88)
test_subtract 176.0179 (1.24) 212.1262 (1.19) 181.7260 (1.19) 11.6174 (1.0) 177.3450 (1.23)
------------------------------------------------------------------------------------------------------------------------------------------
------------------------- benchmark "GF(2^8) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 8 tests --------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------------
test_add 62.4850 (1.03) 232.4961 (1.0) 73.3393 (1.0) 19.2285 (1.0) 67.0920 (1.01)
test_additive_inverse 60.8349 (1.0) 303.9362 (1.31) 75.7256 (1.03) 23.9701 (1.25) 66.5609 (1.0)
test_divide 31,357.7619 (515.46) 33,928.1799 (145.93) 32,024.4656 (436.66) 1,071.7328 (55.74) 31,594.5051 (474.67)
test_multiplicative_inverse 16,130.9310 (265.16) 22,589.3550 (97.16) 18,422.6860 (251.20) 2,467.6099 (128.33) 17,968.8521 (269.96)
test_multiply 1,726.0939 (28.37) 2,139.3239 (9.20) 1,832.1673 (24.98) 172.9170 (8.99) 1,769.2349 (26.58)
test_power 15,418.5530 (253.45) 19,151.2981 (82.37) 17,459.3566 (238.06) 1,456.6692 (75.76) 17,972.3101 (270.01)
test_scalar_multiply 1,424.5091 (23.42) 3,249.2320 (13.98) 1,838.0753 (25.06) 336.6038 (17.51) 1,740.8130 (26.15)
test_subtract 66.2010 (1.09) 324.3710 (1.40) 85.7436 (1.17) 29.3386 (1.53) 74.9400 (1.13)
------------------------------------------------------------------------------------------------------------------------------------------------
------------------------ benchmark "GF(2^8) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-lookup'": 8 tests ------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------
test_add 66.6410 (1.09) 307.5062 (1.25) 92.5952 (1.25) 31.7356 (1.53) 78.6111 (1.19)
test_additive_inverse 61.1460 (1.0) 288.4439 (1.17) 73.9261 (1.0) 21.2221 (1.03) 66.1870 (1.0)
test_divide 381.8041 (6.24) 642.9779 (2.61) 486.1582 (6.58) 102.6931 (4.96) 477.1240 (7.21)
test_multiplicative_inverse 308.8908 (5.05) 450.9010 (1.83) 343.5373 (4.65) 58.9238 (2.85) 309.4219 (4.67)
test_multiply 310.6489 (5.08) 510.5541 (2.07) 392.5511 (5.31) 74.9355 (3.62) 372.2322 (5.62)
test_power 454.8710 (7.44) 614.1132 (2.49) 502.7098 (6.80) 63.6973 (3.08) 480.4349 (7.26)
test_scalar_multiply 1,337.2621 (21.87) 3,347.7580 (13.58) 1,646.4154 (22.27) 351.6510 (17.00) 1,511.9230 (22.84)
test_subtract 63.4668 (1.04) 246.5090 (1.0) 76.8375 (1.04) 20.6870 (1.0) 69.5318 (1.05)
------------------------------------------------------------------------------------------------------------------------------------------
-------------------------- benchmark "GF(3^5) Array Arithmetic: shape=(1_000,), ufunc_mode='jit-calculate'": 8 tests ---------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------------
test_add 600.3019 (1.34) 875.1580 (1.42) 669.5663 (1.40) 80.0338 (2.23) 650.1831 (1.43)
test_additive_inverse 447.7361 (1.0) 618.4638 (1.0) 478.0377 (1.0) 51.4350 (1.43) 453.8295 (1.0)
test_divide 20,522.7621 (45.84) 26,730.4152 (43.22) 23,591.0309 (49.35) 2,306.3176 (64.17) 23,143.9110 (51.00)
test_multiplicative_inverse 18,970.5859 (42.37) 29,393.6098 (47.53) 22,934.1439 (47.98) 4,165.3422 (115.89) 23,078.4880 (50.85)
test_multiply 1,429.8242 (3.19) 1,609.7031 (2.60) 1,518.5410 (3.18) 83.9082 (2.33) 1,542.0800 (3.40)
test_power 17,494.5770 (39.07) 20,862.5218 (33.73) 19,221.0962 (40.21) 1,306.0528 (36.34) 19,433.4229 (42.82)
test_scalar_multiply 1,319.3421 (2.95) 3,319.4488 (5.37) 1,828.1196 (3.82) 359.4200 (10.00) 1,704.8109 (3.76)
test_subtract 594.8299 (1.33) 726.0030 (1.17) 614.7950 (1.29) 35.9430 (1.0) 602.5580 (1.33)
------------------------------------------------------------------------------------------------------------------------------------------------
-------------------- benchmark "GF(3^5) Array Arithmetic: shape=(1_000,), ufunc_mode='jit-lookup'": 8 tests --------------------
Name (time in us) Min Max Mean StdDev Median
--------------------------------------------------------------------------------------------------------------------------------
test_add 48.6970 (1.09) 75.5680 (1.07) 57.5135 (1.18) 10.6115 (1.33) 56.1608 (1.23)
test_additive_inverse 45.3601 (1.02) 72.2490 (1.03) 52.5008 (1.07) 11.3766 (1.42) 47.2669 (1.04)
test_divide 46.1771 (1.04) 70.9470 (1.01) 51.1555 (1.05) 9.0236 (1.13) 47.0870 (1.03)
test_multiplicative_inverse 49.7589 (1.12) 78.2839 (1.11) 54.3773 (1.11) 9.2918 (1.16) 50.3049 (1.10)
test_multiply 44.4769 (1.0) 70.4550 (1.0) 48.9275 (1.0) 7.9862 (1.0) 45.5696 (1.0)
test_power 74.0550 (1.67) 148.8561 (2.11) 102.2906 (2.09) 26.8338 (3.36) 100.3731 (2.20)
test_scalar_multiply 68.6261 (1.54) 294.9270 (4.19) 82.6070 (1.69) 23.0046 (2.88) 74.2599 (1.63)
test_subtract 53.4819 (1.20) 106.8390 (1.52) 79.2563 (1.62) 24.2412 (3.04) 83.1240 (1.82)
--------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
======================================================================================= 56 passed in 26.60s ========================================================================================
Comparing with previous benchmarks¶
If you’d like to compare the performance impacts of a branch, for instance, check out master
and run pytest
with the --benchmark-save
option.
This will save a file in .benchmarks/0001_master.json
.
$ git checkout master
$ pytest benchmarks/test_field_arithmetic.py --benchmark-columns=min,max,mean,stddev,median --benchmark-sort=name --benchmark-save=master
$ git checkout branch
$ pytest benchmarks/test_field_arithmetic.py --benchmark-columns=min,max,mean,stddev,median --benchmark-sort=name --benchmark-compare=0001_master
Comparing against benchmarks from: Linux-CPython-3.8-64bit/0001_master.on
======================================================================================= test session starts ========================================================================================
platform linux -- Python 3.8.10, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/matt/repos/galois, configfile: setup.cfg
plugins: benchmark-3.4.1, extra-durations-0.1.3, anyio-2.1.0
collected 56 items
benchmarks/test_field_arithmetic.py ........................................................ [100%]
==================================================================================== sum of all tests durations ====================================================================================
23.72s
----------------------------- benchmark "GF(2) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 16 tests -----------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master) 53.6300 (1.04) 570.4910 (3.85) 74.1800 (1.31) 29.7015 (4.18) 66.1639 (1.23)
test_add (NOW) 53.8658 (1.05) 148.1280 (1.0) 56.6440 (1.0) 7.1116 (1.0) 55.1331 (1.03)
test_additive_inverse (0001_master) 51.5240 (1.0) 200.4388 (1.35) 62.2888 (1.10) 15.7254 (2.21) 55.8060 (1.04)
test_additive_inverse (NOW) 51.7170 (1.00) 176.7289 (1.19) 57.3724 (1.01) 11.7825 (1.66) 53.6300 (1.0)
test_divide (0001_master) 206.0279 (4.00) 823.5171 (5.56) 265.5769 (4.69) 68.8627 (9.68) 239.2294 (4.46)
test_divide (NOW) 205.6009 (3.99) 698.4309 (4.72) 225.6514 (3.98) 35.2674 (4.96) 212.8950 (3.97)
test_multiplicative_inverse (0001_master) 186.9558 (3.63) 566.1491 (3.82) 220.2184 (3.89) 49.3615 (6.94) 198.8420 (3.71)
test_multiplicative_inverse (NOW) 187.0971 (3.63) 464.8890 (3.14) 203.0407 (3.58) 27.2545 (3.83) 193.4364 (3.61)
test_multiply (0001_master) 56.0421 (1.09) 248.2592 (1.68) 81.5853 (1.44) 31.2515 (4.39) 68.5630 (1.28)
test_multiply (NOW) 53.5559 (1.04) 283.5400 (1.91) 59.3049 (1.05) 13.7564 (1.93) 54.8789 (1.02)
test_power (0001_master) 234.1510 (4.54) 330.9071 (2.23) 272.5068 (4.81) 40.0108 (5.63) 274.3008 (5.11)
test_power (NOW) 230.1259 (4.47) 405.7041 (2.74) 277.7140 (4.90) 74.4280 (10.47) 236.8400 (4.42)
test_scalar_multiply (0001_master) 1,345.2091 (26.11) 2,872.6910 (19.39) 1,627.9802 (28.74) 253.5061 (35.65) 1,575.4123 (29.38)
test_scalar_multiply (NOW) 1,277.7401 (24.80) 2,155.3310 (14.55) 1,377.6186 (24.32) 142.8746 (20.09) 1,313.8375 (24.50)
test_subtract (0001_master) 60.0331 (1.17) 300.8121 (2.03) 83.1536 (1.47) 28.1426 (3.96) 69.4490 (1.29)
test_subtract (NOW) 53.4630 (1.04) 185.5700 (1.25) 59.3924 (1.05) 12.0366 (1.69) 54.9241 (1.02)
------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------- benchmark "GF(257) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 16 tests ----------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master) 175.3040 (1.37) 635.8509 (1.89) 197.5639 (1.37) 40.8794 (3.55) 186.2240 (1.37)
test_add (NOW) 158.3470 (1.23) 636.1029 (1.89) 176.8827 (1.22) 32.4656 (2.82) 167.7470 (1.23)
test_additive_inverse (0001_master) 139.4011 (1.09) 504.7920 (1.50) 170.0383 (1.18) 38.9894 (3.38) 153.7615 (1.13)
test_additive_inverse (NOW) 128.2750 (1.0) 481.9639 (1.43) 144.6905 (1.0) 27.1122 (2.35) 136.4225 (1.0)
test_divide (0001_master) 6,233.1830 (48.59) 6,527.6541 (19.42) 6,394.2735 (44.19) 119.4256 (10.36) 6,390.6419 (46.84)
test_divide (NOW) 5,915.1719 (46.11) 6,276.5831 (18.68) 6,073.0814 (41.97) 174.3328 (15.13) 6,044.0000 (44.30)
test_multiplicative_inverse (0001_master) 6,131.1359 (47.80) 6,474.0989 (19.26) 6,265.0689 (43.30) 157.5507 (13.67) 6,174.8459 (45.26)
test_multiplicative_inverse (NOW) 5,846.0971 (45.57) 7,729.8910 (23.00) 6,541.4660 (45.21) 748.3626 (64.94) 6,168.4800 (45.22)
test_multiply (0001_master) 294.6239 (2.30) 336.0880 (1.0) 300.5223 (2.08) 11.5243 (1.0) 296.6555 (2.17)
test_multiply (NOW) 313.1870 (2.44) 502.4392 (1.49) 340.8181 (2.36) 54.2113 (4.70) 315.9265 (2.32)
test_power (0001_master) 5,487.4821 (42.78) 5,796.6399 (17.25) 5,624.7412 (38.87) 126.0851 (10.94) 5,565.2100 (40.79)
test_power (NOW) 4,765.4652 (37.15) 5,108.4792 (15.20) 4,907.8213 (33.92) 126.2470 (10.95) 4,901.8471 (35.93)
test_scalar_multiply (0001_master) 1,030.8721 (8.04) 2,567.3539 (7.64) 1,164.7107 (8.05) 198.8079 (17.25) 1,100.3308 (8.07)
test_scalar_multiply (NOW) 1,032.1550 (8.05) 2,005.4830 (5.97) 1,144.9685 (7.91) 132.6795 (11.51) 1,092.9224 (8.01)
test_subtract (0001_master) 160.0061 (1.25) 377.1251 (1.12) 177.7399 (1.23) 31.2034 (2.71) 165.8800 (1.22)
test_subtract (NOW) 164.6511 (1.28) 705.7940 (2.10) 177.2582 (1.23) 31.8202 (2.76) 170.3722 (1.25)
------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------- benchmark "GF(257) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-lookup'": 16 tests ------------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master) 177.8340 (1.28) 227.8131 (1.34) 184.8953 (1.29) 15.3610 (1.59) 179.1745 (1.28)
test_add (NOW) 188.0738 (1.36) 321.0511 (1.88) 226.4540 (1.58) 50.5466 (5.24) 195.8210 (1.40)
test_additive_inverse (0001_master) 139.9061 (1.01) 170.4299 (1.0) 149.0880 (1.04) 10.9319 (1.13) 143.6260 (1.03)
test_additive_inverse (NOW) 138.5838 (1.0) 171.5040 (1.01) 143.1082 (1.0) 9.6383 (1.0) 139.7599 (1.0)
test_divide (0001_master) 384.4770 (2.77) 472.9840 (2.78) 407.4521 (2.85) 31.3674 (3.25) 393.6000 (2.82)
test_divide (NOW) 368.6091 (2.66) 469.5239 (2.75) 397.4139 (2.78) 33.2011 (3.44) 392.4301 (2.81)
test_multiplicative_inverse (0001_master) 335.2840 (2.42) 371.9509 (2.18) 345.5276 (2.41) 13.9113 (1.44) 337.5154 (2.41)
test_multiplicative_inverse (NOW) 338.9800 (2.45) 375.3309 (2.20) 344.6998 (2.41) 11.1648 (1.16) 339.7759 (2.43)
test_multiply (0001_master) 314.1060 (2.27) 373.7470 (2.19) 322.5519 (2.25) 18.2331 (1.89) 315.9530 (2.26)
test_multiply (NOW) 283.0350 (2.04) 342.2331 (2.01) 294.4223 (2.06) 18.5191 (1.92) 284.2465 (2.03)
test_power (0001_master) 461.0980 (3.33) 516.3020 (3.03) 492.7848 (3.44) 28.5527 (2.96) 509.1929 (3.64)
test_power (NOW) 446.6961 (3.22) 513.4281 (3.01) 471.9436 (3.30) 24.9776 (2.59) 468.0229 (3.35)
test_scalar_multiply (0001_master) 1,034.2200 (7.46) 2,384.6650 (13.99) 1,175.8929 (8.22) 188.4351 (19.55) 1,120.0898 (8.01)
test_scalar_multiply (NOW) 1,047.1060 (7.56) 2,285.6970 (13.41) 1,148.9137 (8.03) 135.4746 (14.06) 1,106.0899 (7.91)
test_subtract (0001_master) 175.6351 (1.27) 217.9069 (1.28) 186.5705 (1.30) 16.5222 (1.71) 178.6321 (1.28)
test_subtract (NOW) 177.5022 (1.28) 222.6690 (1.31) 188.9046 (1.32) 17.0226 (1.77) 179.9939 (1.29)
------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------- benchmark "GF(2^8) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 16 tests -------------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master) 65.4031 (1.10) 303.5041 (1.36) 85.0358 (1.27) 26.2409 (2.35) 76.7412 (1.23)
test_add (NOW) 63.1949 (1.06) 376.9179 (1.69) 73.6567 (1.10) 18.6649 (1.67) 67.6350 (1.08)
test_additive_inverse (0001_master) 59.5278 (1.0) 223.1461 (1.0) 68.2673 (1.02) 17.9042 (1.60) 62.5600 (1.0)
test_additive_inverse (NOW) 60.6440 (1.02) 261.6630 (1.17) 67.1257 (1.0) 11.1651 (1.0) 64.8301 (1.04)
test_divide (0001_master) 20,575.8919 (345.65) 22,262.2380 (99.77) 21,375.1204 (318.43) 682.3102 (61.11) 21,379.7691 (341.75)
test_divide (NOW) 20,864.8040 (350.51) 21,292.5780 (95.42) 21,095.4332 (314.27) 182.5212 (16.35) 21,152.4100 (338.11)
test_multiplicative_inverse (0001_master) 14,893.7618 (250.20) 18,845.2560 (84.45) 16,354.3480 (243.64) 1,507.6655 (135.03) 16,245.4580 (259.68)
test_multiplicative_inverse (NOW) 15,077.0759 (253.28) 15,525.0910 (69.57) 15,313.5644 (228.13) 166.0180 (14.87) 15,339.7880 (245.20)
test_multiply (0001_master) 1,781.0490 (29.92) 1,911.8502 (8.57) 1,850.5634 (27.57) 48.3193 (4.33) 1,857.6779 (29.69)
test_multiply (NOW) 1,514.2660 (25.44) 1,614.7611 (7.24) 1,565.8663 (23.33) 36.6022 (3.28) 1,559.7660 (24.93)
test_power (0001_master) 17,853.8628 (299.92) 20,269.1101 (90.83) 19,207.0828 (286.14) 893.4667 (80.02) 19,430.7449 (310.59)
test_power (NOW) 14,477.4460 (243.20) 16,110.0640 (72.20) 14,925.3466 (222.35) 685.6721 (61.41) 14,588.3670 (233.19)
test_scalar_multiply (0001_master) 1,414.1060 (23.76) 3,601.1150 (16.14) 1,645.1560 (24.51) 297.7188 (26.67) 1,549.7539 (24.77)
test_scalar_multiply (NOW) 1,418.7060 (23.83) 3,286.4269 (14.73) 1,593.0689 (23.73) 246.1296 (22.04) 1,503.2839 (24.03)
test_subtract (0001_master) 61.8761 (1.04) 226.9482 (1.02) 78.5680 (1.17) 20.9293 (1.87) 72.5901 (1.16)
test_subtract (NOW) 62.4610 (1.05) 247.6051 (1.11) 72.8939 (1.09) 18.7591 (1.68) 66.7050 (1.07)
------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------- benchmark "GF(2^8) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-lookup'": 16 tests ------------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master) 62.1150 (1.04) 242.7939 (1.33) 73.7056 (1.08) 19.0953 (1.73) 67.1479 (1.05)
test_add (NOW) 62.2938 (1.04) 183.1381 (1.0) 68.0219 (1.0) 11.0402 (1.0) 65.1181 (1.02)
test_additive_inverse (0001_master) 63.4750 (1.06) 306.6689 (1.67) 83.7360 (1.23) 27.5077 (2.49) 74.2881 (1.16)
test_additive_inverse (NOW) 59.8810 (1.0) 240.5860 (1.31) 68.7484 (1.01) 17.0596 (1.55) 63.7915 (1.0)
test_divide (0001_master) 376.1090 (6.28) 427.3648 (2.33) 404.0765 (5.94) 20.4260 (1.85) 410.8525 (6.44)
test_divide (NOW) 368.1141 (6.15) 473.6630 (2.59) 423.8058 (6.23) 39.0911 (3.54) 431.1745 (6.76)
test_multiplicative_inverse (0001_master) 370.6738 (6.19) 658.4080 (3.60) 465.4766 (6.84) 108.3985 (9.82) 448.2989 (7.03)
test_multiplicative_inverse (NOW) 309.5521 (5.17) 342.8990 (1.87) 317.9003 (4.67) 12.0781 (1.09) 311.4310 (4.88)
test_multiply (0001_master) 344.9500 (5.76) 527.0049 (2.88) 376.0530 (5.53) 67.0493 (6.07) 348.1212 (5.46)
test_multiply (NOW) 293.8330 (4.91) 365.5059 (2.00) 313.4176 (4.61) 25.9667 (2.35) 303.4060 (4.76)
test_power (0001_master) 512.7750 (8.56) 614.1500 (3.35) 546.0020 (8.03) 42.9520 (3.89) 523.0750 (8.20)
test_power (NOW) 480.9520 (8.03) 541.6819 (2.96) 496.3040 (7.30) 25.6325 (2.32) 484.7571 (7.60)
test_scalar_multiply (0001_master) 1,343.6000 (22.44) 3,204.0619 (17.50) 1,808.8479 (26.59) 342.6193 (31.03) 1,742.1505 (27.31)
test_scalar_multiply (NOW) 1,273.2670 (21.26) 2,594.8270 (14.17) 1,382.8612 (20.33) 174.7877 (15.83) 1,314.1120 (20.60)
test_subtract (0001_master) 62.6270 (1.05) 264.0169 (1.44) 77.2011 (1.13) 20.9450 (1.90) 69.2611 (1.09)
test_subtract (NOW) 62.4580 (1.04) 225.9151 (1.23) 69.4670 (1.02) 12.8329 (1.16) 65.2981 (1.02)
------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------- benchmark "GF(3^5) Array Arithmetic: shape=(1_000,), ufunc_mode='jit-calculate'": 16 tests --------------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master) 577.8901 (1.29) 669.9329 (1.15) 596.5873 (1.22) 29.3569 (1.20) 581.5226 (1.28)
test_add (NOW) 565.7980 (1.27) 654.5840 (1.12) 580.1125 (1.19) 26.6488 (1.09) 569.4189 (1.25)
test_additive_inverse (0001_master) 528.4690 (1.18) 1,180.2840 (2.02) 679.2012 (1.39) 198.8098 (8.13) 596.6179 (1.31)
test_additive_inverse (NOW) 447.2181 (1.0) 583.9660 (1.0) 488.9684 (1.0) 50.0549 (2.05) 454.5329 (1.0)
test_divide (0001_master) 23,370.1861 (52.26) 27,859.3549 (47.71) 25,382.3458 (51.91) 1,936.6428 (79.20) 25,000.1471 (55.00)
test_divide (NOW) 26,724.1660 (59.76) 29,631.6862 (50.74) 28,053.3344 (57.37) 1,366.7746 (55.90) 27,384.5338 (60.25)
test_multiplicative_inverse (0001_master) 19,703.0620 (44.06) 24,010.1409 (41.12) 21,643.3670 (44.26) 1,981.0911 (81.02) 21,131.4450 (46.49)
test_multiplicative_inverse (NOW) 18,754.1139 (41.94) 24,443.9621 (41.86) 21,822.6110 (44.63) 2,057.5147 (84.15) 21,855.1469 (48.08)
test_multiply (0001_master) 1,425.0348 (3.19) 1,521.4889 (2.61) 1,475.4896 (3.02) 45.4442 (1.86) 1,496.0892 (3.29)
test_multiply (NOW) 1,683.6938 (3.76) 2,381.5830 (4.08) 2,060.1566 (4.21) 281.4423 (11.51) 2,146.3181 (4.72)
test_power (0001_master) 23,206.8000 (51.89) 27,687.5449 (47.41) 25,244.8478 (51.63) 2,196.7235 (89.84) 24,164.1460 (53.16)
test_power (NOW) 17,747.6569 (39.68) 22,459.7279 (38.46) 19,107.0794 (39.08) 1,926.5370 (78.79) 18,604.5689 (40.93)
test_scalar_multiply (0001_master) 1,225.6859 (2.74) 4,148.2730 (7.10) 1,776.8746 (3.63) 573.3293 (23.45) 1,577.6126 (3.47)
test_scalar_multiply (NOW) 1,220.4021 (2.73) 2,658.5320 (4.55) 1,471.6397 (3.01) 248.3855 (10.16) 1,377.4155 (3.03)
test_subtract (0001_master) 584.9910 (1.31) 672.4251 (1.15) 595.7514 (1.22) 24.4517 (1.0) 587.2779 (1.29)
test_subtract (NOW) 571.2900 (1.28) 636.8279 (1.09) 591.2687 (1.21) 26.5326 (1.09) 575.3681 (1.27)
------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------ benchmark "GF(3^5) Array Arithmetic: shape=(1_000,), ufunc_mode='jit-lookup'": 16 tests -------------------------
Name (time in us) Min Max Mean StdDev Median
------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master) 49.7401 (1.17) 76.4418 (1.15) 57.4466 (1.24) 10.9692 (1.50) 53.2519 (1.22)
test_add (NOW) 47.1489 (1.11) 72.2532 (1.09) 54.1565 (1.16) 10.4557 (1.43) 49.3161 (1.13)
test_additive_inverse (0001_master) 43.5489 (1.03) 108.0472 (1.62) 66.0670 (1.42) 29.6528 (4.04) 46.5869 (1.06)
test_additive_inverse (NOW) 42.7819 (1.01) 69.3691 (1.04) 48.8280 (1.05) 10.3019 (1.40) 44.3211 (1.01)
test_divide (0001_master) 45.0709 (1.06) 86.1811 (1.30) 51.6534 (1.11) 14.1129 (1.92) 45.8754 (1.05)
test_divide (NOW) 44.6450 (1.05) 68.3721 (1.03) 48.9857 (1.05) 8.0531 (1.10) 45.6730 (1.04)
test_multiplicative_inverse (0001_master) 42.4671 (1.0) 66.5220 (1.0) 46.4964 (1.0) 7.3332 (1.0) 43.8021 (1.0)
test_multiplicative_inverse (NOW) 44.0120 (1.04) 68.5172 (1.03) 48.4346 (1.04) 7.5601 (1.03) 44.8425 (1.02)
test_multiply (0001_master) 44.4860 (1.05) 67.9540 (1.02) 48.5520 (1.04) 7.6291 (1.04) 45.4651 (1.04)
test_multiply (NOW) 43.8211 (1.03) 67.5640 (1.02) 48.4292 (1.04) 7.3999 (1.01) 44.8920 (1.02)
test_power (0001_master) 65.6650 (1.55) 97.5579 (1.47) 73.2096 (1.57) 12.2555 (1.67) 68.3501 (1.56)
test_power (NOW) 65.0741 (1.53) 95.6031 (1.44) 71.8517 (1.55) 11.9027 (1.62) 66.6430 (1.52)
test_scalar_multiply (0001_master) 68.4580 (1.61) 300.1180 (4.51) 78.9913 (1.70) 19.5450 (2.67) 73.0332 (1.67)
test_scalar_multiply (NOW) 69.1570 (1.63) 256.4839 (3.86) 77.5875 (1.67) 13.4846 (1.84) 74.0800 (1.69)
test_subtract (0001_master) 49.5000 (1.17) 76.2481 (1.15) 61.0748 (1.31) 11.9175 (1.63) 56.3171 (1.29)
test_subtract (NOW) 47.5000 (1.12) 88.6670 (1.33) 65.7632 (1.41) 16.4101 (2.24) 64.0899 (1.46)
------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
======================================================================================= 56 passed in 24.48s ========================================================================================
Or if you have two saved benchmarks, you can simply compare them without re-running the tests.
$ pytest-benchmark compare 0001_master 0002_master