Performances
RESTHeart has been designed and developed with lightness and performances as fundamental parameters. On this regards, also thanks to its caching capabilities, RESTHeart often overcomes the results that can be achieved accessing MongoDB directly via its Java driver.
This section includes the performance test results gathered by the SoftInstigate’s development team and includes all the information needed to autonomously reproduce the tests.
Results
Test case 1
Measure the execution time to create 1 million documents with random data, using 200 test threads.
In this scenario, RESTHeart introduces just a 2,41% overhead over the total execution time:
| Execution Time | TPS | |
|---|---|---|
| RESTHeart | 250s | 3990 tps |
| Direct | 244s | 4086 tps |
Test case 2
Measure the execution time to query a collection 100.000 times, getting 5 documents each time (limit 5) and skipping just 25 documents, under different concurrency levels.
RESTHeart delivers better performances under any concurrency level over direct access via MongoDB driver:
| Threads | 50 | 100 | 200 | 250 | 400 | 500 |
|---|---|---|---|---|---|---|
| RESTHeart | 78s | 82s | 78s | 76s | 76s | 76s |
| Direct | 97s | 95s | 96s | 96s | 109s | 112s |
Test case 3
Measure the execution time to query a collection 2.000 times, getting 5 documents each time (limit 5) and skipping just 250.000 documents, under different concurrency levels.
Thanks to the eager pre-allocation DBCursor engine, queries with significant skip parameter executes much faster (50 times in this case) with RESTHeart:
| Threads | 1 | 2 | 4 | 5 | 8 | 10 | 20 | 40 | 50 | 80 | 100 | 200 | 400 | 500 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RESTHeart | 16,28s | 6,22s | 5,05s | 2,53s | 3,76s | 3,6s | 2,98s | 5,65s | 9,04s | 10,74s | 6,76s | 9,24s | 6,76s | 12,71s |
| Direct | 1091s | 627s | 324s | 328s | 329s | 325s | 324s | 321s | 321s | 304s | 302s | 305s | 327s | 327s |
Test case 4
Measure the execution time to query a collection 500 times, getting 5 documents each time (limit 5) skipping more and more documents each time, with a concurrency level of 4.
Thanks to the eager pre-allocation DBCursor engine, queries with significant skip parameter executes much faster (up to 470 times in this scenario) with RESTHeart.
(Kskips = 1000 skip)
| Kskips | 250 | 500 | 750 | 1.000 | 1.250 | 1.500 | 1.750 | 2.000 | 2.250 |
|---|---|---|---|---|---|---|---|---|---|
| RESTHeart | 2,6s | 1,91s | 1,9s | 1,88s | 1,58s | 2,05s | 1,51s | 1,52s | 1,51s |
| Direct | 79s | 156s | 242s | 317s | 375s | 453s | 558s | 601s | 713s |
How we tested
Hardware
MongoDB and RESTHeart running on Sunfire X2200 M2 with 2 CPU with 16 Gbyte of RAM. See full specification here. This is an old server with 2 dual core 2,2GHz AMD Opteron CPUs.
Test cases run by MacBook Pro client with 2,66 GHz Intel Core i7 and 8 GB 1067 MHz DDR3
The client and the server on the same local network linked by a 10/100 Ethernet switch.
Software
- Server OS: Ubuntu server 64bit 14.04.1 LTS
- Client OS: Mac OS X Yosemite 10.10.1
- MongoDB: 2.6.7
- RESTHeart: commit 01d403a5db8b765ad5b0a8eec1fda420c392ab58
Configurations
MongoDB: run (without authentication enabled) with the following command
$numactl --interleave=all /opt/mongodb/bin/mongod --fork --syslog
RESTHeart: run with default parameters with the following exception:
- logging to file off
- eager-cursor-allocation-linear-slice-heights set to [50]
- io-threads: 8
- worker-threads: 64
Test cases
We used the brilliant load test tool.
The test case code is available on Github as part of the RESTHeart source code baseline. You can find it here.