redo benchmarks

after things are somewhat finalized, redo benchmarks. the numbers will have changed.

I spent some time playing with pytest-benchmark, refactoring the benchmark tests over it (see lelit/python-rapidjson@cd2944c).

The new report looks like this:

----------------------------------------------------------------------- benchmark '100 arrays dict: deserialize': 5 tests -----------------------------------------------------------------------
Name (time in ms)                                Min                 Max                Mean             StdDev              Median                IQR            Outliers(*)  Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_loads[100 arrays dict-ujson]           427.4862 (1.0)      455.3450 (1.01)     437.0200 (1.0)      10.7544 (1.23)     434.4067 (1.0)      10.3217 (1.59)             1;0       5           1
test_loads[100 arrays dict-rapidjson]       430.0161 (1.01)     452.6185 (1.0)      438.6803 (1.00)      8.7120 (1.0)      436.6605 (1.01)     10.9410 (1.69)             1;0       5           1
test_loads[100 arrays dict-stdlib json]     499.2808 (1.17)     533.8289 (1.18)     510.2276 (1.17)     13.7887 (1.58)     504.5833 (1.16)     14.1718 (2.18)             1;0       5           1
test_loads[100 arrays dict-yajl]            527.6370 (1.23)     549.6103 (1.21)     534.2751 (1.22)      8.7378 (1.00)     531.5936 (1.22)      6.4887 (1.0)              1;1       5           1
test_loads[100 arrays dict-simplejson]      599.5375 (1.40)     634.8729 (1.40)     611.3847 (1.40)     14.8568 (1.71)     603.5450 (1.39)     20.5571 (3.17)             1;0       5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------- benchmark '100 arrays dict: serialize': 5 tests ------------------------------------------------------------------------
Name (time in ms)                                Min                 Max                Mean            StdDev              Median                IQR            Outliers(*)  Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dumps[100 arrays dict-rapidjson]       140.1384 (1.0)      141.1041 (1.0)      140.4387 (1.0)      0.3216 (1.0)      140.3395 (1.0)       0.3880 (1.0)              1;0       8           1
test_dumps[100 arrays dict-ujson]           231.3472 (1.65)     234.2445 (1.66)     232.4973 (1.66)     1.1323 (3.52)     232.1457 (1.65)      1.5452 (3.98)             2;0       5           1
test_dumps[100 arrays dict-yajl]            311.4754 (2.22)     317.5603 (2.25)     312.8790 (2.23)     2.6282 (8.17)     311.6470 (2.22)      1.9111 (4.93)             1;1       5           1
test_dumps[100 arrays dict-stdlib json]     491.5805 (3.51)     492.7548 (3.49)     491.9703 (3.50)     0.4803 (1.49)     491.8106 (3.50)      0.6330 (1.63)             1;0       5           1
test_dumps[100 arrays dict-simplejson]      917.6096 (6.55)     933.6302 (6.62)     927.5285 (6.60)     6.8315 (21.24)    929.8802 (6.63)     11.0774 (28.55)            1;0       5           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------- benchmark '100 dicts array: deserialize': 5 tests --------------------------------------------------------------------
Name (time in us)                               Min                Max               Mean            StdDev             Median               IQR            Outliers(*)  Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_loads[100 dicts array-ujson]           26.4340 (1.0)      54.3560 (1.0)      27.0991 (1.0)      1.4436 (1.0)      26.7870 (1.0)      0.1520 (1.03)       1372;1837   23903           1
test_loads[100 dicts array-rapidjson]       27.4190 (1.04)     76.0240 (1.40)     28.1014 (1.04)     1.5027 (1.04)     27.7880 (1.04)     0.1480 (1.0)        1454;2103   25243           1
test_loads[100 dicts array-yajl]            36.1980 (1.37)     76.9890 (1.42)     37.2179 (1.37)     1.8790 (1.30)     36.7600 (1.37)     0.2150 (1.45)       1272;1982   18073           1
test_loads[100 dicts array-stdlib json]     39.1660 (1.48)     78.0220 (1.44)     40.0418 (1.48)     1.8287 (1.27)     39.7040 (1.48)     0.2170 (1.47)        673;1065   14820           1
test_loads[100 dicts array-simplejson]      48.2300 (1.82)     89.6710 (1.65)     49.3048 (1.82)     2.7131 (1.88)     48.7010 (1.82)     0.2220 (1.50)        736;1194   13700           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------- benchmark '100 dicts array: serialize': 5 tests -----------------------------------------------------------------------
Name (time in us)                                Min                 Max                Mean            StdDev              Median               IQR            Outliers(*)  Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dumps[100 dicts array-rapidjson]        15.2200 (1.0)       88.4860 (1.81)      15.8214 (1.0)      0.9064 (1.02)      15.7440 (1.0)      0.2370 (3.29)        452;1123   34563           1
test_dumps[100 dicts array-ujson]            23.0000 (1.51)      48.9660 (1.0)       23.2667 (1.47)     0.8881 (1.0)       23.1940 (1.47)     0.0720 (1.0)         304;1252   32160           1
test_dumps[100 dicts array-yajl]             29.9910 (1.97)      72.7160 (1.49)      33.3768 (2.11)     2.1636 (2.44)      32.8180 (2.08)     0.7665 (10.65)        506;767   10703           1
test_dumps[100 dicts array-stdlib json]      50.7780 (3.34)      95.7930 (1.96)      52.0233 (3.29)     1.8536 (2.09)      51.5830 (3.28)     0.6200 (8.61)        733;1011   14137           1
test_dumps[100 dicts array-simplejson]      111.4200 (7.32)     185.0700 (3.78)     114.0086 (7.21)     3.9207 (4.41)     113.0280 (7.18)     1.1460 (15.92)        353;716    5322           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------ benchmark '256 Trues array: deserialize': 5 tests -------------------------------------------------------------------
Name (time in us)                              Min                Max              Mean            StdDev            Median               IQR            Outliers(*)  Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_loads[256 Trues array-ujson]           5.2450 (1.0)      30.2280 (1.38)     5.5433 (1.0)      0.4123 (1.05)     5.5230 (1.0)      0.0450 (1.50)        337;3045  105631           1
test_loads[256 Trues array-rapidjson]       5.6670 (1.08)     22.0830 (1.01)     5.8852 (1.06)     0.3945 (1.0)      5.8630 (1.06)     0.0410 (1.37)        372;3534   88881           1
test_loads[256 Trues array-stdlib json]     8.3860 (1.60)     21.8760 (1.0)      8.6123 (1.55)     0.4157 (1.05)     8.5840 (1.55)     0.0610 (2.03)        295;2032   58973           1
test_loads[256 Trues array-yajl]            8.9690 (1.71)     38.4910 (1.76)     9.2590 (1.67)     0.4870 (1.23)     9.2360 (1.67)     0.0300 (1.0)         403;5875   75398           1
test_loads[256 Trues array-simplejson]      9.6890 (1.85)     36.2400 (1.66)     9.9498 (1.79)     0.6301 (1.60)     9.8990 (1.79)     0.0810 (2.70)        284;1809   45307           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------- benchmark '256 Trues array: serialize': 5 tests ---------------------------------------------------------------------
Name (time in us)                               Min                Max               Mean            StdDev             Median               IQR            Outliers(*)  Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dumps[256 Trues array-rapidjson]        6.0380 (1.0)      23.7480 (1.10)      6.5454 (1.0)      0.4143 (1.0)       6.5100 (1.0)      0.1060 (2.79)        441;2222   59085           1
test_dumps[256 Trues array-yajl]             7.2160 (1.20)     21.5560 (1.0)       7.3183 (1.12)     0.4200 (1.01)      7.2860 (1.12)     0.0380 (1.0)         316;3827   73212           1
test_dumps[256 Trues array-ujson]            7.3660 (1.22)     29.5610 (1.37)      7.7217 (1.18)     0.5130 (1.24)      7.5230 (1.16)     0.5570 (14.66)        823;303   80283           1
test_dumps[256 Trues array-stdlib json]     13.0770 (2.17)     39.4690 (1.83)     13.5478 (2.07)     0.6782 (1.64)     13.4890 (2.07)     0.1920 (5.05)        283;1107   38038           1
test_dumps[256 Trues array-simplejson]      16.5890 (2.75)     36.1220 (1.68)     16.9698 (2.59)     0.6383 (1.54)     16.9050 (2.60)     0.1240 (3.26)        231;1721   27630           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------- benchmark '256 ascii array: deserialize': 5 tests --------------------------------------------------------------------
Name (time in us)                               Min                Max               Mean            StdDev             Median               IQR            Outliers(*)  Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_loads[256 ascii array-rapidjson]       29.4540 (1.0)      72.2840 (1.20)     30.3372 (1.0)      2.1236 (1.08)     29.6100 (1.0)      0.1010 (1.17)       3172;4092   28086           1
test_loads[256 ascii array-stdlib json]     30.4970 (1.04)     73.7630 (1.23)     31.3026 (1.03)     2.0936 (1.07)     30.6830 (1.04)     0.0880 (1.02)       2235;2828   24295           1
test_loads[256 ascii array-simplejson]      32.2530 (1.10)     60.1630 (1.0)      33.3338 (1.10)     2.3188 (1.18)     32.5700 (1.10)     0.2080 (2.42)       2306;2863   22567           1
test_loads[256 ascii array-ujson]           32.8890 (1.12)     73.0150 (1.21)     33.6786 (1.11)     1.9596 (1.0)      33.1480 (1.12)     0.1880 (2.19)       2222;3197   26529           1
test_loads[256 ascii array-yajl]            42.5690 (1.45)     89.3190 (1.48)     43.5191 (1.43)     2.4497 (1.25)     42.7060 (1.44)     0.0860 (1.0)        2259;3457   20656           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------- benchmark '256 ascii array: serialize': 5 tests ----------------------------------------------------------------------
Name (time in us)                               Min                 Max               Mean            StdDev             Median               IQR            Outliers(*)  Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dumps[256 ascii array-ujson]           21.2760 (1.0)       58.8020 (1.0)      21.4620 (1.0)      0.9262 (1.0)      21.3540 (1.0)      0.0340 (1.0)         297;2376   27769           1
test_dumps[256 ascii array-rapidjson]       30.8300 (1.45)      61.9820 (1.05)     32.3609 (1.51)     1.1453 (1.24)     32.1200 (1.50)     0.7270 (21.38)        669;474   22071           1
test_dumps[256 ascii array-stdlib json]     42.7630 (2.01)     141.3480 (2.40)     44.3877 (2.07)     2.7277 (2.95)     43.5200 (2.04)     0.4170 (12.26)      2044;2483   17323           1
test_dumps[256 ascii array-yajl]            42.8660 (2.01)      88.3900 (1.50)     54.4870 (2.54)     1.8311 (1.98)     54.3080 (2.54)     1.4082 (41.42)        663;356   14285           1
test_dumps[256 ascii array-simplejson]      53.8720 (2.53)      93.8890 (1.60)     55.1597 (2.57)     2.6277 (2.84)     54.3860 (2.55)     0.3720 (10.94)      1153;1498   12637           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------- benchmark '256 doubles array rapidjson (not precise): deserialize': 1 tests -----------------------------------
Name (time in us)                                                   Min      Max     Mean  StdDev   Median     IQR  Outliers(*)  Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------
test_loads_float[256 doubles array rapidjson (not precise)]     16.4660  35.2370  16.9308  0.8218  16.8720  0.1000     443;3583   43910           1
---------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------- benchmark '256 doubles array ujson (not precise): deserialize': 1 tests -----------------------------------
Name (time in us)                                               Min      Max     Mean  StdDev   Median     IQR  Outliers(*)  Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------
test_loads_float[256 doubles array ujson (not precise)]     29.8210  67.2390  30.2854  0.6311  30.2530  0.1490     340;1089   26853           1
-----------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------- benchmark '256 doubles array: deserialize': 5 tests ---------------------------------------------------------------------
Name (time in us)                                 Min                 Max               Mean            StdDev             Median               IQR            Outliers(*)  Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_loads[256 doubles array-yajl]            34.1460 (1.0)       86.0950 (1.33)     34.6925 (1.0)      1.3992 (1.34)     34.5910 (1.0)      0.1420 (1.0)         371;1041   24243           1
test_loads[256 doubles array-ujson]           34.8070 (1.02)      71.2940 (1.11)     35.2943 (1.02)     1.1262 (1.08)     35.1650 (1.02)     0.1820 (1.28)         473;832   21939           1
test_loads[256 doubles array-rapidjson]       36.4650 (1.07)      64.4950 (1.0)      37.3429 (1.08)     1.0452 (1.0)      37.2520 (1.08)     0.2570 (1.81)         439;661   21679           1
test_loads[256 doubles array-stdlib json]     71.4140 (2.09)     109.5760 (1.70)     72.1894 (2.08)     1.7565 (1.68)     71.9750 (2.08)     0.2300 (1.62)         324;498   11231           1
test_loads[256 doubles array-simplejson]      72.9480 (2.14)     118.5380 (1.84)     73.7427 (2.13)     1.7546 (1.68)     73.5270 (2.13)     0.2160 (1.52)         351;586   11262           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------- benchmark '256 doubles array: serialize': 5 tests -----------------------------------------------------------------------
Name (time in us)                                  Min                 Max                Mean            StdDev              Median               IQR            Outliers(*)  Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dumps[256 doubles array-rapidjson]        27.9710 (1.0)       55.6260 (1.0)       28.7755 (1.0)      0.8986 (1.0)       28.6820 (1.0)      0.1020 (1.0)         408;1085   23206           1
test_dumps[256 doubles array-yajl]            121.3160 (4.34)     302.5800 (5.44)     122.1088 (4.24)     3.5924 (4.00)     121.6450 (4.24)     0.1030 (1.01)         133;567    7316           1
test_dumps[256 doubles array-stdlib json]     175.4180 (6.27)     259.2370 (4.66)     177.4971 (6.17)     3.5329 (3.93)     176.8750 (6.17)     0.2835 (2.78)         144;455    5125           1
test_dumps[256 doubles array-simplejson]      176.6470 (6.32)     234.0620 (4.21)     179.3320 (6.23)     4.4912 (5.00)     177.3890 (6.18)     0.6140 (6.02)        790;1101    4761           1
test_dumps[256 doubles array-ujson]           204.1630 (7.30)     254.7190 (4.58)     205.2278 (7.13)     3.2855 (3.66)     204.6420 (7.13)     0.1810 (1.77)         110;374    4548           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------- benchmark '256 unicode array: deserialize': 5 tests ---------------------------------------------------------------------------
Name (time in us)                                    Min                   Max                  Mean             StdDev                Median                IQR            Outliers(*)  Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_loads[256 unicode array-rapidjson]         375.7880 (1.0)        416.2200 (1.0)        378.2719 (1.0)       3.9810 (1.0)        377.3680 (1.0)       1.6412 (1.0)           89;166    2523           1
test_loads[256 unicode array-ujson]             467.3050 (1.24)       665.4830 (1.60)       470.0586 (1.24)      7.4369 (1.87)       468.2010 (1.24)      1.7273 (1.05)          96;147    1933           1
test_loads[256 unicode array-yajl]            1,131.6610 (3.01)     1,307.2400 (3.14)     1,141.3178 (3.02)     13.1664 (3.31)     1,137.3130 (3.01)      4.3810 (2.67)         104;124     866           1
test_loads[256 unicode array-simplejson]      1,917.9710 (5.10)     2,047.1910 (4.92)     1,930.5017 (5.10)     12.4906 (3.14)     1,926.9750 (5.11)      7.2150 (4.40)           64;68     500           1
test_loads[256 unicode array-stdlib json]     2,392.3390 (6.37)     2,619.9160 (6.29)     2,411.7167 (6.38)     16.7867 (4.22)     2,406.5865 (6.38)     11.7520 (7.16)           50;27     412           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------- benchmark '256 unicode array: serialize': 5 tests ------------------------------------------------------------------------
Name (time in us)                                  Min                 Max                Mean            StdDev              Median                IQR            Outliers(*)  Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dumps[256 unicode array-rapidjson]       190.9280 (1.0)      335.0220 (1.19)     198.0676 (1.0)      7.1008 (2.05)     195.0570 (1.0)      11.8950 (51.27)         120;53    4622           1
test_dumps[256 unicode array-yajl]            223.3130 (1.17)     280.9310 (1.0)      227.3972 (1.15)     4.2025 (1.21)     226.5220 (1.16)      1.6395 (7.07)         169;275    3157           1
test_dumps[256 unicode array-ujson]           259.3370 (1.36)     298.0110 (1.06)     260.5556 (1.32)     3.4709 (1.0)      259.7400 (1.33)      0.2320 (1.0)          143;385    3610           1
test_dumps[256 unicode array-stdlib json]     331.0830 (1.73)     412.4440 (1.47)     332.9698 (1.68)     5.0309 (1.45)     331.8970 (1.70)      0.4360 (1.88)         109;338    2886           1
test_dumps[256 unicode array-simplejson]      409.5850 (2.15)     708.4140 (2.52)     413.4949 (2.09)     8.7299 (2.52)     412.2115 (2.11)      2.0360 (8.78)         123;179    2266           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------- benchmark 'composite object: deserialize': 5 tests --------------------------------------------------------------------
Name (time in us)                                Min                 Max               Mean            StdDev             Median               IQR            Outliers(*)  Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_loads[composite object-ujson]           30.3210 (1.0)       58.4970 (1.0)      30.8569 (1.0)      1.1142 (1.0)      30.6790 (1.0)      0.1450 (1.0)         726;1076   19856           1
test_loads[composite object-rapidjson]       33.8570 (1.12)     568.6920 (9.72)     34.8940 (1.13)     4.0707 (3.65)     34.6540 (1.13)     0.3430 (2.37)        381;1038   20754           1
test_loads[composite object-yajl]            49.9580 (1.65)      96.9150 (1.66)     50.8299 (1.65)     1.7447 (1.57)     50.5330 (1.65)     0.2740 (1.89)         642;911   14167           1
test_loads[composite object-stdlib json]     53.6270 (1.77)      93.7570 (1.60)     54.8664 (1.78)     2.1583 (1.94)     54.3420 (1.77)     0.3190 (2.20)        750;1048   11026           1
test_loads[composite object-simplejson]      72.8670 (2.40)     111.0160 (1.90)     73.8968 (2.39)     2.0169 (1.81)     73.4675 (2.39)     0.2730 (1.88)         618;764   10050           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------- benchmark 'composite object: serialize': 5 tests ---------------------------------------------------------------------
Name (time in us)                                Min                 Max               Mean            StdDev             Median               IQR            Outliers(*)  Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dumps[composite object-rapidjson]       17.1860 (1.0)       36.0860 (1.0)      17.5599 (1.0)      0.7882 (1.0)      17.4780 (1.0)      0.1260 (1.38)        274;1347   26887           1
test_dumps[composite object-ujson]           25.3010 (1.47)      63.3830 (1.76)     25.6196 (1.46)     1.0005 (1.27)     25.5360 (1.46)     0.0910 (1.0)          291;857   26746           1
test_dumps[composite object-stdlib json]     48.6350 (2.83)      97.2620 (2.70)     50.5648 (2.88)     3.2087 (4.07)     49.3130 (2.82)     0.3350 (3.68)       2096;2357   14314           1
test_dumps[composite object-yajl]            53.0930 (3.09)      94.4090 (2.62)     56.4876 (3.22)     1.9511 (2.48)     56.1460 (3.21)     1.0500 (11.54)        600;790   10054           1
test_dumps[composite object-simplejson]      66.7210 (3.88)     141.4720 (3.92)     67.9917 (3.87)     1.9708 (2.50)     67.7680 (3.88)     0.2583 (2.84)         219;535    8153           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------- benchmark 'medium complex objects: deserialize': 5 tests ----------------------------------------------------------------------
Name (time in us)                                       Min                 Max                Mean            StdDev              Median               IQR            Outliers(*)  Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_loads[medium complex objects-rapidjson]        95.6300 (1.0)      155.3130 (1.0)       98.0602 (1.0)      2.8672 (1.0)       98.4100 (1.0)      2.9560 (10.75)        266;126    8524           1
test_loads[medium complex objects-ujson]           106.0490 (1.11)     230.1830 (1.48)     107.4590 (1.10)     3.1763 (1.11)     107.0810 (1.09)     0.2750 (1.0)          202;510    8279           1
test_loads[medium complex objects-stdlib json]     130.6580 (1.37)     172.9230 (1.11)     133.4339 (1.36)     3.1841 (1.11)     132.9150 (1.35)     0.9030 (3.28)         261;430    6373           1
test_loads[medium complex objects-yajl]            142.8160 (1.49)     235.7950 (1.52)     145.6921 (1.49)     3.3020 (1.15)     145.1540 (1.47)     0.9560 (3.48)         182;540    5806           1
test_loads[medium complex objects-simplejson]      151.9160 (1.59)     202.6790 (1.30)     153.8091 (1.57)     3.3073 (1.15)     153.2525 (1.56)     0.5790 (2.11)         235;398    5804           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------- benchmark 'medium complex objects: serialize': 5 tests -----------------------------------------------------------------------
Name (time in us)                                       Min                 Max                Mean            StdDev              Median               IQR            Outliers(*)  Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dumps[medium complex objects-rapidjson]        43.4730 (1.0)       75.6350 (1.0)       45.2557 (1.0)      1.4572 (1.0)       45.1870 (1.0)      0.3040 (3.01)        476;2988   17152           1
test_dumps[medium complex objects-ujson]            64.2520 (1.48)     149.5130 (1.98)      64.7693 (1.43)     2.0479 (1.41)      64.4880 (1.43)     0.1010 (1.0)          361;979   13701           1
test_dumps[medium complex objects-yajl]            132.3970 (3.05)     205.1840 (2.71)     142.7499 (3.15)     7.3296 (5.03)     141.7120 (3.14)     2.0665 (20.46)       732;1028    4644           1
test_dumps[medium complex objects-stdlib json]     147.9030 (3.40)     262.4490 (3.47)     151.9092 (3.36)     4.3509 (2.99)     152.2410 (3.37)     3.5945 (35.59)        196;150    4769           1
test_dumps[medium complex objects-simplejson]      204.6790 (4.71)     375.5430 (4.97)     207.1550 (4.58)     3.9516 (2.71)     206.5620 (4.57)     0.8412 (8.33)         143;312    4077           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

(*) Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.

I do not understand the meaning of the rightmost four columns, but it seems pretty and informative, what do you think?

I added a new option (see lelit/python-rapidjson@701c29f) that allows focusing on RapidJSON alone, to better coupling with the compare past runs functionality: there's an example command and its output in the README.

I will try to propose a simple command line tool that reads from one archived benchmark results (pytest-benchmark saves a JSON file under ./benchmarks at each run, when given the benchmark-autosave option: I wonder if we should make that the default, adding the option to [pytest] in setup.py...) and emits some kind of reST table to make it easier the update of ./docs/benchmarks.rst.

This can be closed together with #42, now that #46 has been merged in.

thanks

Navigation Menu

Repository navigation

redo benchmarks #24

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

redo benchmarks #24

Description

Activity

lelit commented on Mar 24, 2016

lelit commented on Mar 25, 2016

lelit commented on Aug 31, 2016

lelit commented on Dec 9, 2016

kenrobbins commented on Dec 9, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions