JMH Gradle Plugin is used to build and run benchmarks. Benchmark results are obtained on a PC running under Windows 7 with Intel(R) Core(TM) i7-3770 3.4 GHz CPU and 64-bit OpenJDK build 1.8.0_131-1-ojdkbuild-b11 with the following parameters: -Xms1g -Xmx1g. To get results in your environment, run in the project root directory:
./gradlew clean jar jmh
Tokyo Cabinet Benchmark is useful for comparing the performance of key/value storages. For one million 8-char string keys ('00000000', '00000001', etc.) and values equal to the keys, four operations are measured:
- writing all key/value pairs in ascending order;
- writing all key/value pairs in random order;
- reading all key/value pairs in ascending order;
- reading all key/value pairs in random order.
Currently, benchmark results are available for Xodus stores with key prefixing (Patricia tree) and without (BTree), MapDb tree map, Chronicle Map, H2 MVStore Map, LMDB JNI and Akiban PersistIt. All the scores are in seconds per single benchmark run for one million keys. Excerpt of the output of the build running benchmarks is as follows:
Benchmark Mode Cnt Score Error Units
chronicle.JMHChronicleMapTokyoCabinetReadBenchmark.randomRead ss 10 0.553 ± 0.020 s/op
chronicle.JMHChronicleMapTokyoCabinetReadBenchmark.successiveRead ss 10 0.180 ± 0.028 s/op
chronicle.JMHChronicleMapTokyoCabinetWriteBenchmark.randomWrite ss 10 0.726 ± 0.037 s/op
chronicle.JMHChronicleMapTokyoCabinetWriteBenchmark.successiveWrite ss 10 0.675 ± 0.008 s/op
env.JMHEnvTokyoCabinetReadBenchmark.randomRead ss 10 1.880 ± 0.016 s/op
env.JMHEnvTokyoCabinetReadBenchmark.successiveRead ss 10 0.156 ± 0.013 s/op
env.JMHEnvTokyoCabinetWriteBenchmark.randomWrite ss 10 1.888 ± 0.059 s/op
env.JMHEnvTokyoCabinetWriteBenchmark.successiveWrite ss 10 0.462 ± 0.022 s/op
env.JMHEnvWithPrefixingTokyoCabinetReadBenchmark.randomRead ss 10 0.994 ± 0.004 s/op
env.JMHEnvWithPrefixingTokyoCabinetReadBenchmark.successiveRead ss 10 0.240 ± 0.008 s/op
env.JMHEnvWithPrefixingTokyoCabinetWriteBenchmark.randomWrite ss 10 1.486 ± 0.366 s/op
env.JMHEnvWithPrefixingTokyoCabinetWriteBenchmark.successiveWrite ss 10 0.461 ± 0.050 s/op
h2.JMH_MVStoreTokyoCabinetReadBenchmark.randomRead ss 10 13.092 ± 0.029 s/op
h2.JMH_MVStoreTokyoCabinetReadBenchmark.successiveRead ss 10 0.109 ± 0.003 s/op
h2.JMH_MVStoreTokyoCabinetWriteBenchmark.randomWrite ss 10 1.860 ± 0.006 s/op
h2.JMH_MVStoreTokyoCabinetWriteBenchmark.successiveWrite ss 10 0.517 ± 0.097 s/op
lmdb.JMH_LMDBTokyoCabinetReadBenchmark.randomRead ss 10 0.825 ± 0.004 s/op
lmdb.JMH_LMDBTokyoCabinetReadBenchmark.successiveRead ss 10 0.098 ± 0.001 s/op
lmdb.JMH_LMDBTokyoCabinetWriteBenchmark.randomWrite ss 10 0.831 ± 0.003 s/op
lmdb.JMH_LMDBTokyoCabinetWriteBenchmark.successiveWrite ss 10 0.178 ± 0.001 s/op
mapdb.JMHMapDbTokyoCabinetReadBenchmark.randomRead ss 10 7.154 ± 0.036 s/op
mapdb.JMHMapDbTokyoCabinetReadBenchmark.successiveRead ss 10 0.126 ± 0.004 s/op
mapdb.JMHMapDbTokyoCabinetWriteBenchmark.randomWrite ss 10 9.075 ± 0.030 s/op
mapdb.JMHMapDbTokyoCabinetWriteBenchmark.successiveWrite ss 10 11.982 ± 0.132 s/op
persistit.JMHPersistItTokyoCabinetReadBenchmark.randomRead ss 10 1.316 ± 0.020 s/op
persistit.JMHPersistItTokyoCabinetReadBenchmark.successiveRead ss 10 0.857 ± 0.301 s/op
persistit.JMHPersistItTokyoCabinetWriteBenchmark.randomWrite ss 10 1.763 ± 0.019 s/op
persistit.JMHPersistItTokyoCabinetWriteBenchmark.successiveWrite ss 10 0.632 ± 0.059 s/op
The same results in table form:
| Random Read | Successive Read | Random Write | Successive Write | |
|---|---|---|---|---|
| Xodus store with key prefixing (Patricia) | 0.994 | 0.240 | 1.486 | 0.461 |
| Xodus store without key prefixing (BTree) | 1.880 | 0.156 | 1.888 | 0.462 |
| BDB JE database with key prefixing | 3.493 | 1.307 | 5.937 | 3.416 |
| BDB JE database without key prefixing | 3.288 | 1.251 | 5.831 | 3.458 |
| MapDB tree map | 7.154 | 0.126 | 9.075 | 11.982 |
| Chronicle Map | 0.553 | 0.180 | 0.726 | 0.675 |
| H2 MVStore Map | 13.092 | 0.109 | 1.860 | 0.517 |
| LMDB JNI | 0.825 | 0.098 | 0.831 | 0.178 |
| Akiban PersistIt | 1.316 | 0.857 | 1.763 | 0.632 |
Results for BerkeleyDb JE are obtained in a similar environment as above (another JRE used), but the code of benchmark cannot be distributed under Apache 2.0 License.