nostrdb

an unfairly fast embedded nostr database backed by lmdb
git clone git://jb55.com/nostrdb
Log | Files | Refs | Submodules | README | LICENSE

benchmarks.md (4551B)

      1 # Benchmarks
      2 
      3 Benchmarks are defined for raw C structs, Googles `flatc` generated C++
      4 and the `flatcc` compilers C ouput.
      5 
      6 These can be run with:
      7 
      8     scripts/benchmark.sh
      9 
     10 and requires a C++ compiler installed - the benchmark for flatcc alone can be
     11 run with:
     12 
     13     test/benchmark/benchflatcc/run.sh
     14 
     15 this only requires a system C compiler (cc) to be installed (and
     16 flatcc's build environment).
     17 
     18 A summary for OS-X 2.2 GHz Haswell core-i7 is found below. Generated
     19 files for OS-X and Ubuntu are found in the benchmark folder.
     20 
     21 The benchmarks use the same schema and dataset as Google FPL's
     22 FlatBuffers benchmark.
     23 
     24 In summary, 1 million iterations runs at about 500-540MB/s at 620-700
     25 ns/op encoding buffers and 29-34ns/op traversing buffers. `flatc` and
     26 `flatcc` are close enough in performance for it not to matter much.
     27 `flatcc` is a bit faster encoding but it is likely due to less memory
     28 allocation. Throughput and time per operatin are of course very specific
     29 to this test case.
     30 
     31 Generated JSON parser/printer shown below, for flatcc only but for OS-X
     32 and Linux.
     33 
     34 
     35 ## operation: flatbench for raw C structs encode (optimized)
     36     elapsed time: 0.055 (s)
     37     iterations: 1000000
     38     size: 312 (bytes)
     39     bandwidth: 5665.517 (MB/s)
     40     throughput in ops per sec: 18158707.100
     41     throughput in 1M ops per sec: 18.159
     42     time per op: 55.070 (ns)
     43 
     44 ## operation: flatbench for raw C structs decode/traverse (optimized)
     45     elapsed time: 0.012 (s)
     46     iterations: 1000000
     47     size: 312 (bytes)
     48     bandwidth: 25978.351 (MB/s)
     49     throughput in ops per sec: 83263946.711
     50     throughput in 1M ops per sec: 83.264
     51     time per op: 12.010 (ns)
     52 
     53 ## operation: flatc for C++ encode (optimized)
     54     elapsed time: 0.702 (s)
     55     iterations: 1000000
     56     size: 344 (bytes)
     57     bandwidth: 490.304 (MB/s)
     58     throughput in ops per sec: 1425301.380
     59     throughput in 1M ops per sec: 1.425
     60     time per op: 701.606 (ns)
     61 
     62 ## operation: flatc for C++ decode/traverse (optimized)
     63     elapsed time: 0.029 (s)
     64     iterations: 1000000
     65     size: 344 (bytes)
     66     bandwidth: 11917.134 (MB/s)
     67     throughput in ops per sec: 34642832.398
     68     throughput in 1M ops per sec: 34.643
     69     time per op: 28.866 (ns)
     70 
     71 
     72 ## operation: flatcc for C encode (optimized)
     73     elapsed time: 0.626 (s)
     74     iterations: 1000000
     75     size: 336 (bytes)
     76     bandwidth: 536.678 (MB/s)
     77     throughput in ops per sec: 1597255.277
     78     throughput in 1M ops per sec: 1.597
     79     time per op: 626.074 (ns)
     80 
     81 ## operation: flatcc for C decode/traverse (optimized)
     82     elapsed time: 0.029 (s)
     83     iterations: 1000000
     84     size: 336 (bytes)
     85     bandwidth: 11726.930 (MB/s)
     86     throughput in ops per sec: 34901577.551
     87     throughput in 1M ops per sec: 34.902
     88     time per op: 28.652 (ns)
     89 
     90 ## JSON benchmark
     91 
     92 *Note: this benchmark is only available for `flatcc`. It uses the exact
     93 same data set as above.*
     94 
     95 The benchmark uses Grisu3 floating point parsing and printing algorithm
     96 with exact fallback to strtod/sprintf when the algorithm fails to be
     97 exact. Better performance can be gained by enabling inexact Grisu3 and
     98 SSE 4.2 in build options, but likely not worthwhile in praxis.
     99 
    100 ## operation: flatcc json parser and printer for C encode (optimized)
    101 
    102 (encode means printing from existing binary buffer to JSON)
    103 
    104     elapsed time: 1.407 (s)
    105     iterations: 1000000
    106     size: 722 (bytes)
    107     bandwidth: 513.068 (MB/s)
    108     throughput in ops per sec: 710619.931
    109     throughput in 1M ops per sec: 0.711
    110     time per op: 1.407 (us)
    111 
    112 ## operation: flatcc json parser and printer for C decode/traverse (optimized)
    113 
    114 (decode/traverse means parsing json to flatbuffer binary and calculating checksum)
    115 
    116     elapsed time: 2.218 (s)
    117     iterations: 1000000
    118     size: 722 (bytes)
    119     bandwidth: 325.448 (MB/s)
    120     throughput in ops per sec: 450758.672
    121     throughput in 1M ops per sec: 0.451
    122     time per op: 2.218 (us)
    123 
    124 ## JSON parsing and printing on same hardware in Virtual Box Ubuntu
    125 
    126 Numbers for Linux included because parsing is significantly faster.
    127 
    128 ## operation: flatcc json parser and printer for C encode (optimized)
    129 
    130     elapsed time: 1.210 (s)
    131     iterations: 1000000
    132     size: 722 (bytes)
    133     bandwidth: 596.609 (MB/s)
    134     throughput in ops per sec: 826328.137
    135     throughput in 1M ops per sec: 0.826
    136     time per op: 1.210 (us)
    137 
    138 ## operation: flatcc json parser and printer for C decode/traverse
    139 
    140     elapsed time: 1.772 (s)
    141     iterations: 1000000
    142     size: 722 (bytes)
    143     bandwidth: 407.372 (MB/s)
    144     throughput in ops per sec: 564227.736
    145     throughput in 1M ops per sec: 0.564
    146     time per op: 1.772 (us)
    147