Benchmarking

The galois package comes with benchmarking tests. They are contained in the benchmarks/ folder. They are pytest tests using the pytest-benchmark extension.

Running a benchmark test

To run a benchmark, invoke pytest on the benchmarks/ folder or a specific test set (e.g., benchmarks/test_field_arithmetic.py). It is also advised to pass extra arguments to format the display --benchmark-columns=min,max,mean,stddev,median and --benchmark-sort=name.

$ pytest benchmarks/test_field_arithmetic.py --benchmark-columns=min,max,mean,stddev,median --benchmark-sort=name
======================================================================================= test session starts ========================================================================================
platform linux -- Python 3.8.10, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/matt/repos/galois, configfile: setup.cfg
plugins: benchmark-3.4.1, extra-durations-0.1.3, anyio-2.1.0
collected 56 items

benchmarks/test_field_arithmetic.py ........................................................                                                                                                 [100%]

==================================================================================== sum of all tests durations ====================================================================================
25.80s

----------------------- benchmark "GF(2) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 8 tests ------------------------
Name (time in us)                      Min                   Max                  Mean              StdDev                Median
------------------------------------------------------------------------------------------------------------------------------------------
test_add                           54.9031 (1.04)       238.1941 (1.19)        62.2950 (1.06)      14.3986 (1.30)        58.4391 (1.03)
test_additive_inverse              52.8959 (1.0)        237.0442 (1.18)        58.6939 (1.0)       11.1021 (1.0)         56.5751 (1.0)
test_divide                       207.9280 (3.93)       636.3150 (3.17)       234.0875 (3.99)      38.3417 (3.45)       220.6110 (3.90)
test_multiplicative_inverse       188.9290 (3.57)       652.8331 (3.25)       229.7088 (3.91)      57.9788 (5.22)       201.3633 (3.56)
test_multiply                      54.6200 (1.03)       228.7410 (1.14)        61.9206 (1.05)      12.5660 (1.13)        58.8121 (1.04)
test_power                        229.3210 (4.34)       284.5279 (1.42)       246.6029 (4.20)      22.5143 (2.03)       238.7560 (4.22)
test_scalar_multiply            1,561.2941 (29.52)    3,148.2361 (15.67)    2,058.2764 (35.07)    361.3528 (32.55)    1,969.9985 (34.82)
test_subtract                      54.5362 (1.03)       200.8791 (1.0)         62.6200 (1.07)      13.5863 (1.22)        59.0470 (1.04)
------------------------------------------------------------------------------------------------------------------------------------------

---------------------- benchmark "GF(257) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 8 tests -----------------------
Name (time in us)                      Min                   Max                  Mean              StdDev                Median
------------------------------------------------------------------------------------------------------------------------------------------
test_add                          161.1579 (1.22)       583.2699 (1.20)       197.0225 (1.18)      52.4775 (1.15)       175.6141 (1.21)
test_additive_inverse             132.4308 (1.0)        546.5120 (1.12)       166.7050 (1.0)       46.9263 (1.03)       144.6060 (1.0)
test_divide                     6,306.7721 (47.62)    7,277.9991 (14.94)    6,658.3270 (39.94)    374.7916 (8.21)     6,597.4400 (45.62)
test_multiplicative_inverse     6,299.4179 (47.57)    6,477.2971 (13.29)    6,392.0670 (38.34)     75.0566 (1.64)     6,367.5269 (44.03)
test_multiply                     337.8210 (2.55)       487.2240 (1.0)        362.2038 (2.17)      45.6575 (1.0)        344.7749 (2.38)
test_power                      5,026.9060 (37.96)    5,523.7100 (11.34)    5,273.6145 (31.63)    204.7190 (4.48)     5,208.3279 (36.02)
test_scalar_multiply            1,033.5341 (7.80)     2,472.3031 (5.07)     1,223.1746 (7.34)     205.6682 (4.50)     1,157.2575 (8.00)
test_subtract                     172.0500 (1.30)       655.0739 (1.34)       211.9266 (1.27)      59.9908 (1.31)       188.1695 (1.30)
------------------------------------------------------------------------------------------------------------------------------------------

------------------------ benchmark "GF(257) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-lookup'": 8 tests ------------------------
Name (time in us)                      Min                   Max                  Mean              StdDev                Median
------------------------------------------------------------------------------------------------------------------------------------------
test_add                          175.6609 (1.23)       217.3430 (1.22)       182.2442 (1.20)      12.6704 (1.09)       177.3606 (1.23)
test_additive_inverse             142.2670 (1.0)        178.2598 (1.0)        152.1027 (1.0)       13.3374 (1.15)       144.3694 (1.0)
test_divide                       414.2381 (2.91)       543.3671 (3.05)       473.2308 (3.11)      50.1936 (4.32)       475.6054 (3.29)
test_multiplicative_inverse       336.2088 (2.36)       459.0931 (2.58)       363.2939 (2.39)      40.4252 (3.48)       342.7650 (2.37)
test_multiply                     319.1240 (2.24)       414.7580 (2.33)       342.5294 (2.25)      33.0754 (2.85)       322.0430 (2.23)
test_power                        446.4111 (3.14)       505.2469 (2.83)       461.2746 (3.03)      24.9575 (2.15)       449.4530 (3.11)
test_scalar_multiply            1,037.2710 (7.29)     2,068.3082 (11.60)    1,197.2376 (7.87)     185.2372 (15.94)    1,137.1800 (7.88)
test_subtract                     176.0179 (1.24)       212.1262 (1.19)       181.7260 (1.19)      11.6174 (1.0)        177.3450 (1.23)
------------------------------------------------------------------------------------------------------------------------------------------

------------------------- benchmark "GF(2^8) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 8 tests --------------------------
Name (time in us)                       Min                    Max                   Mean                StdDev                 Median
------------------------------------------------------------------------------------------------------------------------------------------------
test_add                            62.4850 (1.03)        232.4961 (1.0)          73.3393 (1.0)         19.2285 (1.0)          67.0920 (1.01)
test_additive_inverse               60.8349 (1.0)         303.9362 (1.31)         75.7256 (1.03)        23.9701 (1.25)         66.5609 (1.0)
test_divide                     31,357.7619 (515.46)   33,928.1799 (145.93)   32,024.4656 (436.66)   1,071.7328 (55.74)    31,594.5051 (474.67)
test_multiplicative_inverse     16,130.9310 (265.16)   22,589.3550 (97.16)    18,422.6860 (251.20)   2,467.6099 (128.33)   17,968.8521 (269.96)
test_multiply                    1,726.0939 (28.37)     2,139.3239 (9.20)      1,832.1673 (24.98)      172.9170 (8.99)      1,769.2349 (26.58)
test_power                      15,418.5530 (253.45)   19,151.2981 (82.37)    17,459.3566 (238.06)   1,456.6692 (75.76)    17,972.3101 (270.01)
test_scalar_multiply             1,424.5091 (23.42)     3,249.2320 (13.98)     1,838.0753 (25.06)      336.6038 (17.51)     1,740.8130 (26.15)
test_subtract                       66.2010 (1.09)        324.3710 (1.40)         85.7436 (1.17)        29.3386 (1.53)         74.9400 (1.13)
------------------------------------------------------------------------------------------------------------------------------------------------

------------------------ benchmark "GF(2^8) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-lookup'": 8 tests ------------------------
Name (time in us)                      Min                   Max                  Mean              StdDev                Median
------------------------------------------------------------------------------------------------------------------------------------------
test_add                           66.6410 (1.09)       307.5062 (1.25)        92.5952 (1.25)      31.7356 (1.53)        78.6111 (1.19)
test_additive_inverse              61.1460 (1.0)        288.4439 (1.17)        73.9261 (1.0)       21.2221 (1.03)        66.1870 (1.0)
test_divide                       381.8041 (6.24)       642.9779 (2.61)       486.1582 (6.58)     102.6931 (4.96)       477.1240 (7.21)
test_multiplicative_inverse       308.8908 (5.05)       450.9010 (1.83)       343.5373 (4.65)      58.9238 (2.85)       309.4219 (4.67)
test_multiply                     310.6489 (5.08)       510.5541 (2.07)       392.5511 (5.31)      74.9355 (3.62)       372.2322 (5.62)
test_power                        454.8710 (7.44)       614.1132 (2.49)       502.7098 (6.80)      63.6973 (3.08)       480.4349 (7.26)
test_scalar_multiply            1,337.2621 (21.87)    3,347.7580 (13.58)    1,646.4154 (22.27)    351.6510 (17.00)    1,511.9230 (22.84)
test_subtract                      63.4668 (1.04)       246.5090 (1.0)         76.8375 (1.04)      20.6870 (1.0)         69.5318 (1.05)
------------------------------------------------------------------------------------------------------------------------------------------

-------------------------- benchmark "GF(3^5) Array Arithmetic: shape=(1_000,), ufunc_mode='jit-calculate'": 8 tests ---------------------------
Name (time in us)                       Min                    Max                   Mean                StdDev                 Median
------------------------------------------------------------------------------------------------------------------------------------------------
test_add                           600.3019 (1.34)        875.1580 (1.42)        669.5663 (1.40)        80.0338 (2.23)        650.1831 (1.43)
test_additive_inverse              447.7361 (1.0)         618.4638 (1.0)         478.0377 (1.0)         51.4350 (1.43)        453.8295 (1.0)
test_divide                     20,522.7621 (45.84)    26,730.4152 (43.22)    23,591.0309 (49.35)    2,306.3176 (64.17)    23,143.9110 (51.00)
test_multiplicative_inverse     18,970.5859 (42.37)    29,393.6098 (47.53)    22,934.1439 (47.98)    4,165.3422 (115.89)   23,078.4880 (50.85)
test_multiply                    1,429.8242 (3.19)      1,609.7031 (2.60)      1,518.5410 (3.18)        83.9082 (2.33)      1,542.0800 (3.40)
test_power                      17,494.5770 (39.07)    20,862.5218 (33.73)    19,221.0962 (40.21)    1,306.0528 (36.34)    19,433.4229 (42.82)
test_scalar_multiply             1,319.3421 (2.95)      3,319.4488 (5.37)      1,828.1196 (3.82)       359.4200 (10.00)     1,704.8109 (3.76)
test_subtract                      594.8299 (1.33)        726.0030 (1.17)        614.7950 (1.29)        35.9430 (1.0)         602.5580 (1.33)
------------------------------------------------------------------------------------------------------------------------------------------------

-------------------- benchmark "GF(3^5) Array Arithmetic: shape=(1_000,), ufunc_mode='jit-lookup'": 8 tests --------------------
Name (time in us)                   Min                 Max                Mean             StdDev              Median
--------------------------------------------------------------------------------------------------------------------------------
test_add                        48.6970 (1.09)      75.5680 (1.07)      57.5135 (1.18)     10.6115 (1.33)      56.1608 (1.23)
test_additive_inverse           45.3601 (1.02)      72.2490 (1.03)      52.5008 (1.07)     11.3766 (1.42)      47.2669 (1.04)
test_divide                     46.1771 (1.04)      70.9470 (1.01)      51.1555 (1.05)      9.0236 (1.13)      47.0870 (1.03)
test_multiplicative_inverse     49.7589 (1.12)      78.2839 (1.11)      54.3773 (1.11)      9.2918 (1.16)      50.3049 (1.10)
test_multiply                   44.4769 (1.0)       70.4550 (1.0)       48.9275 (1.0)       7.9862 (1.0)       45.5696 (1.0)
test_power                      74.0550 (1.67)     148.8561 (2.11)     102.2906 (2.09)     26.8338 (3.36)     100.3731 (2.20)
test_scalar_multiply            68.6261 (1.54)     294.9270 (4.19)      82.6070 (1.69)     23.0046 (2.88)      74.2599 (1.63)
test_subtract                   53.4819 (1.20)     106.8390 (1.52)      79.2563 (1.62)     24.2412 (3.04)      83.1240 (1.82)
--------------------------------------------------------------------------------------------------------------------------------

Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
======================================================================================= 56 passed in 26.60s ========================================================================================

Comparing with previous benchmarks

If you’d like to compare the performance impacts of a branch, for instance, check out master and run pytest with the --benchmark-save option. This will save a file in .benchmarks/0001_master.json.

$ git checkout master
$ pytest benchmarks/test_field_arithmetic.py --benchmark-columns=min,max,mean,stddev,median --benchmark-sort=name --benchmark-save=master
$ git checkout branch
$ pytest benchmarks/test_field_arithmetic.py --benchmark-columns=min,max,mean,stddev,median --benchmark-sort=name --benchmark-compare=0001_master
Comparing against benchmarks from: Linux-CPython-3.8-64bit/0001_master.on
======================================================================================= test session starts ========================================================================================
platform linux -- Python 3.8.10, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/matt/repos/galois, configfile: setup.cfg
plugins: benchmark-3.4.1, extra-durations-0.1.3, anyio-2.1.0
collected 56 items

benchmarks/test_field_arithmetic.py ........................................................                                                                                                 [100%]

==================================================================================== sum of all tests durations ====================================================================================
23.72s

----------------------------- benchmark "GF(2) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 16 tests -----------------------------
Name (time in us)                                  Min                   Max                  Mean              StdDev                Median
------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master)                         53.6300 (1.04)       570.4910 (3.85)        74.1800 (1.31)      29.7015 (4.18)        66.1639 (1.23)
test_add (NOW)                                 53.8658 (1.05)       148.1280 (1.0)         56.6440 (1.0)        7.1116 (1.0)         55.1331 (1.03)
test_additive_inverse (0001_master)            51.5240 (1.0)        200.4388 (1.35)        62.2888 (1.10)      15.7254 (2.21)        55.8060 (1.04)
test_additive_inverse (NOW)                    51.7170 (1.00)       176.7289 (1.19)        57.3724 (1.01)      11.7825 (1.66)        53.6300 (1.0)
test_divide (0001_master)                     206.0279 (4.00)       823.5171 (5.56)       265.5769 (4.69)      68.8627 (9.68)       239.2294 (4.46)
test_divide (NOW)                             205.6009 (3.99)       698.4309 (4.72)       225.6514 (3.98)      35.2674 (4.96)       212.8950 (3.97)
test_multiplicative_inverse (0001_master)     186.9558 (3.63)       566.1491 (3.82)       220.2184 (3.89)      49.3615 (6.94)       198.8420 (3.71)
test_multiplicative_inverse (NOW)             187.0971 (3.63)       464.8890 (3.14)       203.0407 (3.58)      27.2545 (3.83)       193.4364 (3.61)
test_multiply (0001_master)                    56.0421 (1.09)       248.2592 (1.68)        81.5853 (1.44)      31.2515 (4.39)        68.5630 (1.28)
test_multiply (NOW)                            53.5559 (1.04)       283.5400 (1.91)        59.3049 (1.05)      13.7564 (1.93)        54.8789 (1.02)
test_power (0001_master)                      234.1510 (4.54)       330.9071 (2.23)       272.5068 (4.81)      40.0108 (5.63)       274.3008 (5.11)
test_power (NOW)                              230.1259 (4.47)       405.7041 (2.74)       277.7140 (4.90)      74.4280 (10.47)      236.8400 (4.42)
test_scalar_multiply (0001_master)          1,345.2091 (26.11)    2,872.6910 (19.39)    1,627.9802 (28.74)    253.5061 (35.65)    1,575.4123 (29.38)
test_scalar_multiply (NOW)                  1,277.7401 (24.80)    2,155.3310 (14.55)    1,377.6186 (24.32)    142.8746 (20.09)    1,313.8375 (24.50)
test_subtract (0001_master)                    60.0331 (1.17)       300.8121 (2.03)        83.1536 (1.47)      28.1426 (3.96)        69.4490 (1.29)
test_subtract (NOW)                            53.4630 (1.04)       185.5700 (1.25)        59.3924 (1.05)      12.0366 (1.69)        54.9241 (1.02)
------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------- benchmark "GF(257) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 16 tests ----------------------------
Name (time in us)                                  Min                   Max                  Mean              StdDev                Median
------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master)                        175.3040 (1.37)       635.8509 (1.89)       197.5639 (1.37)      40.8794 (3.55)       186.2240 (1.37)
test_add (NOW)                                158.3470 (1.23)       636.1029 (1.89)       176.8827 (1.22)      32.4656 (2.82)       167.7470 (1.23)
test_additive_inverse (0001_master)           139.4011 (1.09)       504.7920 (1.50)       170.0383 (1.18)      38.9894 (3.38)       153.7615 (1.13)
test_additive_inverse (NOW)                   128.2750 (1.0)        481.9639 (1.43)       144.6905 (1.0)       27.1122 (2.35)       136.4225 (1.0)
test_divide (0001_master)                   6,233.1830 (48.59)    6,527.6541 (19.42)    6,394.2735 (44.19)    119.4256 (10.36)    6,390.6419 (46.84)
test_divide (NOW)                           5,915.1719 (46.11)    6,276.5831 (18.68)    6,073.0814 (41.97)    174.3328 (15.13)    6,044.0000 (44.30)
test_multiplicative_inverse (0001_master)   6,131.1359 (47.80)    6,474.0989 (19.26)    6,265.0689 (43.30)    157.5507 (13.67)    6,174.8459 (45.26)
test_multiplicative_inverse (NOW)           5,846.0971 (45.57)    7,729.8910 (23.00)    6,541.4660 (45.21)    748.3626 (64.94)    6,168.4800 (45.22)
test_multiply (0001_master)                   294.6239 (2.30)       336.0880 (1.0)        300.5223 (2.08)      11.5243 (1.0)        296.6555 (2.17)
test_multiply (NOW)                           313.1870 (2.44)       502.4392 (1.49)       340.8181 (2.36)      54.2113 (4.70)       315.9265 (2.32)
test_power (0001_master)                    5,487.4821 (42.78)    5,796.6399 (17.25)    5,624.7412 (38.87)    126.0851 (10.94)    5,565.2100 (40.79)
test_power (NOW)                            4,765.4652 (37.15)    5,108.4792 (15.20)    4,907.8213 (33.92)    126.2470 (10.95)    4,901.8471 (35.93)
test_scalar_multiply (0001_master)          1,030.8721 (8.04)     2,567.3539 (7.64)     1,164.7107 (8.05)     198.8079 (17.25)    1,100.3308 (8.07)
test_scalar_multiply (NOW)                  1,032.1550 (8.05)     2,005.4830 (5.97)     1,144.9685 (7.91)     132.6795 (11.51)    1,092.9224 (8.01)
test_subtract (0001_master)                   160.0061 (1.25)       377.1251 (1.12)       177.7399 (1.23)      31.2034 (2.71)       165.8800 (1.22)
test_subtract (NOW)                           164.6511 (1.28)       705.7940 (2.10)       177.2582 (1.23)      31.8202 (2.76)       170.3722 (1.25)
------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------- benchmark "GF(257) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-lookup'": 16 tests ------------------------------
Name (time in us)                                  Min                   Max                  Mean              StdDev                Median
------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master)                        177.8340 (1.28)       227.8131 (1.34)       184.8953 (1.29)      15.3610 (1.59)       179.1745 (1.28)
test_add (NOW)                                188.0738 (1.36)       321.0511 (1.88)       226.4540 (1.58)      50.5466 (5.24)       195.8210 (1.40)
test_additive_inverse (0001_master)           139.9061 (1.01)       170.4299 (1.0)        149.0880 (1.04)      10.9319 (1.13)       143.6260 (1.03)
test_additive_inverse (NOW)                   138.5838 (1.0)        171.5040 (1.01)       143.1082 (1.0)        9.6383 (1.0)        139.7599 (1.0)
test_divide (0001_master)                     384.4770 (2.77)       472.9840 (2.78)       407.4521 (2.85)      31.3674 (3.25)       393.6000 (2.82)
test_divide (NOW)                             368.6091 (2.66)       469.5239 (2.75)       397.4139 (2.78)      33.2011 (3.44)       392.4301 (2.81)
test_multiplicative_inverse (0001_master)     335.2840 (2.42)       371.9509 (2.18)       345.5276 (2.41)      13.9113 (1.44)       337.5154 (2.41)
test_multiplicative_inverse (NOW)             338.9800 (2.45)       375.3309 (2.20)       344.6998 (2.41)      11.1648 (1.16)       339.7759 (2.43)
test_multiply (0001_master)                   314.1060 (2.27)       373.7470 (2.19)       322.5519 (2.25)      18.2331 (1.89)       315.9530 (2.26)
test_multiply (NOW)                           283.0350 (2.04)       342.2331 (2.01)       294.4223 (2.06)      18.5191 (1.92)       284.2465 (2.03)
test_power (0001_master)                      461.0980 (3.33)       516.3020 (3.03)       492.7848 (3.44)      28.5527 (2.96)       509.1929 (3.64)
test_power (NOW)                              446.6961 (3.22)       513.4281 (3.01)       471.9436 (3.30)      24.9776 (2.59)       468.0229 (3.35)
test_scalar_multiply (0001_master)          1,034.2200 (7.46)     2,384.6650 (13.99)    1,175.8929 (8.22)     188.4351 (19.55)    1,120.0898 (8.01)
test_scalar_multiply (NOW)                  1,047.1060 (7.56)     2,285.6970 (13.41)    1,148.9137 (8.03)     135.4746 (14.06)    1,106.0899 (7.91)
test_subtract (0001_master)                   175.6351 (1.27)       217.9069 (1.28)       186.5705 (1.30)      16.5222 (1.71)       178.6321 (1.28)
test_subtract (NOW)                           177.5022 (1.28)       222.6690 (1.31)       188.9046 (1.32)      17.0226 (1.77)       179.9939 (1.29)
------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------- benchmark "GF(2^8) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-calculate'": 16 tests -------------------------------
Name (time in us)                                   Min                    Max                   Mean                StdDev                 Median
------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master)                          65.4031 (1.10)        303.5041 (1.36)         85.0358 (1.27)        26.2409 (2.35)         76.7412 (1.23)
test_add (NOW)                                  63.1949 (1.06)        376.9179 (1.69)         73.6567 (1.10)        18.6649 (1.67)         67.6350 (1.08)
test_additive_inverse (0001_master)             59.5278 (1.0)         223.1461 (1.0)          68.2673 (1.02)        17.9042 (1.60)         62.5600 (1.0)
test_additive_inverse (NOW)                     60.6440 (1.02)        261.6630 (1.17)         67.1257 (1.0)         11.1651 (1.0)          64.8301 (1.04)
test_divide (0001_master)                   20,575.8919 (345.65)   22,262.2380 (99.77)    21,375.1204 (318.43)     682.3102 (61.11)    21,379.7691 (341.75)
test_divide (NOW)                           20,864.8040 (350.51)   21,292.5780 (95.42)    21,095.4332 (314.27)     182.5212 (16.35)    21,152.4100 (338.11)
test_multiplicative_inverse (0001_master)   14,893.7618 (250.20)   18,845.2560 (84.45)    16,354.3480 (243.64)   1,507.6655 (135.03)   16,245.4580 (259.68)
test_multiplicative_inverse (NOW)           15,077.0759 (253.28)   15,525.0910 (69.57)    15,313.5644 (228.13)     166.0180 (14.87)    15,339.7880 (245.20)
test_multiply (0001_master)                  1,781.0490 (29.92)     1,911.8502 (8.57)      1,850.5634 (27.57)       48.3193 (4.33)      1,857.6779 (29.69)
test_multiply (NOW)                          1,514.2660 (25.44)     1,614.7611 (7.24)      1,565.8663 (23.33)       36.6022 (3.28)      1,559.7660 (24.93)
test_power (0001_master)                    17,853.8628 (299.92)   20,269.1101 (90.83)    19,207.0828 (286.14)     893.4667 (80.02)    19,430.7449 (310.59)
test_power (NOW)                            14,477.4460 (243.20)   16,110.0640 (72.20)    14,925.3466 (222.35)     685.6721 (61.41)    14,588.3670 (233.19)
test_scalar_multiply (0001_master)           1,414.1060 (23.76)     3,601.1150 (16.14)     1,645.1560 (24.51)      297.7188 (26.67)     1,549.7539 (24.77)
test_scalar_multiply (NOW)                   1,418.7060 (23.83)     3,286.4269 (14.73)     1,593.0689 (23.73)      246.1296 (22.04)     1,503.2839 (24.03)
test_subtract (0001_master)                     61.8761 (1.04)        226.9482 (1.02)         78.5680 (1.17)        20.9293 (1.87)         72.5901 (1.16)
test_subtract (NOW)                             62.4610 (1.05)        247.6051 (1.11)         72.8939 (1.09)        18.7591 (1.68)         66.7050 (1.07)
------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------- benchmark "GF(2^8) Array Arithmetic: shape=(100_000,), ufunc_mode='jit-lookup'": 16 tests ------------------------------
Name (time in us)                                  Min                   Max                  Mean              StdDev                Median
------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master)                         62.1150 (1.04)       242.7939 (1.33)        73.7056 (1.08)      19.0953 (1.73)        67.1479 (1.05)
test_add (NOW)                                 62.2938 (1.04)       183.1381 (1.0)         68.0219 (1.0)       11.0402 (1.0)         65.1181 (1.02)
test_additive_inverse (0001_master)            63.4750 (1.06)       306.6689 (1.67)        83.7360 (1.23)      27.5077 (2.49)        74.2881 (1.16)
test_additive_inverse (NOW)                    59.8810 (1.0)        240.5860 (1.31)        68.7484 (1.01)      17.0596 (1.55)        63.7915 (1.0)
test_divide (0001_master)                     376.1090 (6.28)       427.3648 (2.33)       404.0765 (5.94)      20.4260 (1.85)       410.8525 (6.44)
test_divide (NOW)                             368.1141 (6.15)       473.6630 (2.59)       423.8058 (6.23)      39.0911 (3.54)       431.1745 (6.76)
test_multiplicative_inverse (0001_master)     370.6738 (6.19)       658.4080 (3.60)       465.4766 (6.84)     108.3985 (9.82)       448.2989 (7.03)
test_multiplicative_inverse (NOW)             309.5521 (5.17)       342.8990 (1.87)       317.9003 (4.67)      12.0781 (1.09)       311.4310 (4.88)
test_multiply (0001_master)                   344.9500 (5.76)       527.0049 (2.88)       376.0530 (5.53)      67.0493 (6.07)       348.1212 (5.46)
test_multiply (NOW)                           293.8330 (4.91)       365.5059 (2.00)       313.4176 (4.61)      25.9667 (2.35)       303.4060 (4.76)
test_power (0001_master)                      512.7750 (8.56)       614.1500 (3.35)       546.0020 (8.03)      42.9520 (3.89)       523.0750 (8.20)
test_power (NOW)                              480.9520 (8.03)       541.6819 (2.96)       496.3040 (7.30)      25.6325 (2.32)       484.7571 (7.60)
test_scalar_multiply (0001_master)          1,343.6000 (22.44)    3,204.0619 (17.50)    1,808.8479 (26.59)    342.6193 (31.03)    1,742.1505 (27.31)
test_scalar_multiply (NOW)                  1,273.2670 (21.26)    2,594.8270 (14.17)    1,382.8612 (20.33)    174.7877 (15.83)    1,314.1120 (20.60)
test_subtract (0001_master)                    62.6270 (1.05)       264.0169 (1.44)        77.2011 (1.13)      20.9450 (1.90)        69.2611 (1.09)
test_subtract (NOW)                            62.4580 (1.04)       225.9151 (1.23)        69.4670 (1.02)      12.8329 (1.16)        65.2981 (1.02)
------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------- benchmark "GF(3^5) Array Arithmetic: shape=(1_000,), ufunc_mode='jit-calculate'": 16 tests --------------------------------
Name (time in us)                                   Min                    Max                   Mean                StdDev                 Median
------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master)                         577.8901 (1.29)        669.9329 (1.15)        596.5873 (1.22)        29.3569 (1.20)        581.5226 (1.28)
test_add (NOW)                                 565.7980 (1.27)        654.5840 (1.12)        580.1125 (1.19)        26.6488 (1.09)        569.4189 (1.25)
test_additive_inverse (0001_master)            528.4690 (1.18)      1,180.2840 (2.02)        679.2012 (1.39)       198.8098 (8.13)        596.6179 (1.31)
test_additive_inverse (NOW)                    447.2181 (1.0)         583.9660 (1.0)         488.9684 (1.0)         50.0549 (2.05)        454.5329 (1.0)
test_divide (0001_master)                   23,370.1861 (52.26)    27,859.3549 (47.71)    25,382.3458 (51.91)    1,936.6428 (79.20)    25,000.1471 (55.00)
test_divide (NOW)                           26,724.1660 (59.76)    29,631.6862 (50.74)    28,053.3344 (57.37)    1,366.7746 (55.90)    27,384.5338 (60.25)
test_multiplicative_inverse (0001_master)   19,703.0620 (44.06)    24,010.1409 (41.12)    21,643.3670 (44.26)    1,981.0911 (81.02)    21,131.4450 (46.49)
test_multiplicative_inverse (NOW)           18,754.1139 (41.94)    24,443.9621 (41.86)    21,822.6110 (44.63)    2,057.5147 (84.15)    21,855.1469 (48.08)
test_multiply (0001_master)                  1,425.0348 (3.19)      1,521.4889 (2.61)      1,475.4896 (3.02)        45.4442 (1.86)      1,496.0892 (3.29)
test_multiply (NOW)                          1,683.6938 (3.76)      2,381.5830 (4.08)      2,060.1566 (4.21)       281.4423 (11.51)     2,146.3181 (4.72)
test_power (0001_master)                    23,206.8000 (51.89)    27,687.5449 (47.41)    25,244.8478 (51.63)    2,196.7235 (89.84)    24,164.1460 (53.16)
test_power (NOW)                            17,747.6569 (39.68)    22,459.7279 (38.46)    19,107.0794 (39.08)    1,926.5370 (78.79)    18,604.5689 (40.93)
test_scalar_multiply (0001_master)           1,225.6859 (2.74)      4,148.2730 (7.10)      1,776.8746 (3.63)       573.3293 (23.45)     1,577.6126 (3.47)
test_scalar_multiply (NOW)                   1,220.4021 (2.73)      2,658.5320 (4.55)      1,471.6397 (3.01)       248.3855 (10.16)     1,377.4155 (3.03)
test_subtract (0001_master)                    584.9910 (1.31)        672.4251 (1.15)        595.7514 (1.22)        24.4517 (1.0)         587.2779 (1.29)
test_subtract (NOW)                            571.2900 (1.28)        636.8279 (1.09)        591.2687 (1.21)        26.5326 (1.09)        575.3681 (1.27)
------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------ benchmark "GF(3^5) Array Arithmetic: shape=(1_000,), ufunc_mode='jit-lookup'": 16 tests -------------------------
Name (time in us)                               Min                 Max               Mean             StdDev             Median
------------------------------------------------------------------------------------------------------------------------------------------
test_add (0001_master)                      49.7401 (1.17)      76.4418 (1.15)     57.4466 (1.24)     10.9692 (1.50)     53.2519 (1.22)
test_add (NOW)                              47.1489 (1.11)      72.2532 (1.09)     54.1565 (1.16)     10.4557 (1.43)     49.3161 (1.13)
test_additive_inverse (0001_master)         43.5489 (1.03)     108.0472 (1.62)     66.0670 (1.42)     29.6528 (4.04)     46.5869 (1.06)
test_additive_inverse (NOW)                 42.7819 (1.01)      69.3691 (1.04)     48.8280 (1.05)     10.3019 (1.40)     44.3211 (1.01)
test_divide (0001_master)                   45.0709 (1.06)      86.1811 (1.30)     51.6534 (1.11)     14.1129 (1.92)     45.8754 (1.05)
test_divide (NOW)                           44.6450 (1.05)      68.3721 (1.03)     48.9857 (1.05)      8.0531 (1.10)     45.6730 (1.04)
test_multiplicative_inverse (0001_master)   42.4671 (1.0)       66.5220 (1.0)      46.4964 (1.0)       7.3332 (1.0)      43.8021 (1.0)
test_multiplicative_inverse (NOW)           44.0120 (1.04)      68.5172 (1.03)     48.4346 (1.04)      7.5601 (1.03)     44.8425 (1.02)
test_multiply (0001_master)                 44.4860 (1.05)      67.9540 (1.02)     48.5520 (1.04)      7.6291 (1.04)     45.4651 (1.04)
test_multiply (NOW)                         43.8211 (1.03)      67.5640 (1.02)     48.4292 (1.04)      7.3999 (1.01)     44.8920 (1.02)
test_power (0001_master)                    65.6650 (1.55)      97.5579 (1.47)     73.2096 (1.57)     12.2555 (1.67)     68.3501 (1.56)
test_power (NOW)                            65.0741 (1.53)      95.6031 (1.44)     71.8517 (1.55)     11.9027 (1.62)     66.6430 (1.52)
test_scalar_multiply (0001_master)          68.4580 (1.61)     300.1180 (4.51)     78.9913 (1.70)     19.5450 (2.67)     73.0332 (1.67)
test_scalar_multiply (NOW)                  69.1570 (1.63)     256.4839 (3.86)     77.5875 (1.67)     13.4846 (1.84)     74.0800 (1.69)
test_subtract (0001_master)                 49.5000 (1.17)      76.2481 (1.15)     61.0748 (1.31)     11.9175 (1.63)     56.3171 (1.29)
test_subtract (NOW)                         47.5000 (1.12)      88.6670 (1.33)     65.7632 (1.41)     16.4101 (2.24)     64.0899 (1.46)
------------------------------------------------------------------------------------------------------------------------------------------

Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
======================================================================================= 56 passed in 24.48s ========================================================================================

Or if you have two saved benchmarks, you can simply compare them without re-running the tests.

$ pytest-benchmark compare 0001_master 0002_master