RESTHeart has been designed and developed with lightness and performances as fundamental parameters. On this regards, also thanks to its caching capabilities, RESTHeart often overcomes the results that can be achieved accessing MongoDB directly via its Java driver.

This section includes the performance test results gathered by the SoftInstigate’s development team and includes all the information needed to autonomously reproduce the tests.

Results

Test case 1

Measure the execution time to create 1 million documents with random data, using 200 test threads.

In this scenario, RESTHeart introduces just a 2,41% overhead over the total execution time:

	Execution Time	TPS
RESTHeart	250s	3990 tps
Direct	244s	4086 tps

Test case 2

Measure the execution time to query a collection 100.000 times, getting 5 documents each time (limit 5) and skipping just 25 documents, under different concurrency levels. 

RESTHeart delivers better performances under any concurrency level over direct access via MongoDB driver:

Threads	50	100	200	250	400	500
RESTHeart	78s	82s	78s	76s	76s	76s
Direct	97s	95s	96s	96s	109s	112s

Test case 3

Measure the execution time to query a collection 2.000 times, getting 5 documents each time (limit 5) and skipping just 250.000 documents, under different concurrency levels.

Thanks to the eager pre-allocation DBCursor engine, queries with significant skip parameter executes much faster (50 times in this case) with RESTHeart:

Threads	1	2	4	5	8	10	20	40	50	80	100	200	400	500
RESTHeart	16,28s	6,22s	5,05s	2,53s	3,76s	3,6s	2,98s	5,65s	9,04s	10,74s	6,76s	9,24s	6,76s	12,71s
Direct	1091s	627s	324s	328s	329s	325s	324s	321s	321s	304s	302s	305s	327s	327s

Test case 4

Measure the execution time to query a collection 500 times, getting 5 documents each time (limit 5) skipping more and more documents each time, with a concurrency level of 4.

Thanks to the eager pre-allocation DBCursor engine, queries with significant skip parameter executes much faster (up to 470 times in this scenario) with RESTHeart.

(Kskips = 1000 skip)

Kskips	250	500	750	1.000	1.250	1.500	1.750	2.000	2.250
RESTHeart	2,6s	1,91s	1,9s	1,88s	1,58s	2,05s	1,51s	1,52s	1,51s
Direct	79s	156s	242s	317s	375s	453s	558s	601s	713s

How we tested

Hardware

MongoDB and RESTHeart running on Sunfire X2200 M2 with 2 CPU with 16 Gbyte of RAM. See full specification here. This is an old server with 2 dual core 2,2GHz AMD Opteron CPUs.

Test cases run by MacBook Pro client with 2,66 GHz Intel Core i7 and 8 GB 1067 MHz DDR3

The client and the server on the same local network linked by a 10/100 Ethernet switch.

Software

• Server OS: Ubuntu server 64bit 14.04.1 LTS
• Client OS: Mac OS X Yosemite 10.10.1
• MongoDB: 2.6.7
• RESTHeart: commit 01d403a5db8b765ad5b0a8eec1fda420c392ab58

Configurations

MongoDB: run (without authentication enabled) with the following command

$numactl --interleave=all /opt/mongodb/bin/mongod --fork --syslog

RESTHeart: run with default parameters with the following exception:

• logging to file off
• eager-cursor-allocation-linear-slice-heights set to [50]
• io-threads: 8 
• worker-threads: 64 

Test cases

We used the brilliant load test tool.

The test case code is available on Github as part of the RESTHeart source code baseline. You can find it here.