MemcacheDB, Tokyo Tyrant, Redis performance test

luobo2ni

　　I had tested the following key-value store for set() and get()

• MemcacheDB, use memcached client protocol.
  
• Tokyo Tyrant (Tokyo Cabinet), use memcached client protocol
  
• Redis, use JRedis Java client
  

1. Test environment
1.1 Hardware/OS
　　2 Linux boxes in a LAN, 1 server and 1 test client
　　Linux Centos 5.2 64bit
　　Intel(R) Xeon(R) CPU E5410  @ 2.33GHz (L2 cache: 6M), Quad-Core * 2
　　8G memory
　　SCSI disk (standalone disk, no other access)
1.2 Software version
　　db-4.7.25.tar.gz
　　libevent-1.4.11-stable.tar.gz
　　memcached-1.2.8.tar.gz
　　memcachedb-1.2.1-beta.tar.gz
　　redis-0.900_2.tar.gz
　　tokyocabinet-1.4.9.tar.gz
　　tokyotyrant-1.1.9.tar.gz
1.3 Configuration
　　Memcachedb startup parameter
　　Test 100 bytes
　　./memcachedb -H /data5/kvtest/bdb/data -d -p 11212 -m 2048 -N -L 8192
　　(Update: As mentioned by Steve, the 100-byte-test missed the -N paramter, so I added it and updated the data)
　　Test 20k bytes
　　./memcachedb -H /data5/kvtest/mcdb/data -d -p 11212 -b 21000 -N -m 2048
　　Tokyo Tyrant (Tokyo Cabinet) configuration
　　Use default Tokyo Tyrant sbin/ttservctl
　　use .tch database, hashtable database
　　ulimsiz=”256m”
　　sid=1
　　dbname=”$basedir/casket.tch#bnum=50000000″ # default 1M is not enough!
　　maxcon=”65536″
　　retval=0
　　Redis configuration
　　timeout 300
　　save 900 1
　　save 300 10
　　save 60 10000
　　# no maxmemory settings
1.4 Test client
　　Client in Java, JDK1.6.0, 16 threads
　　Use Memcached client java_memcached-release_2.0.1.jar
　　JRedis client for Redis test, another JDBC-Redis has poor performance.
2. Small data>　　Test 1, 1-5,000,000 as key, 100 bytes string value, do set, then get test, all get test has result.
　　Request per second(mean)

StoreWriteReadMemcached55,989 50,974 Memcachedb25,583 35,260 Tokyo Tyrant42,988 46,238 Redis85,765 71,708　　Server Load Average
StoreWriteReadMemcached1.80, 1.53, 0.871.17, 1.16, 0.83MemcacheDB1.44, 0.93, 0.644.35, 1.94, 1.05Tokyo Tyrant3.70, 1.71, 1.142.98, 1.81, 1.26Redis1.06, 0.32, 0.181.56, 1.00, 0.543. Larger data>　　Test 2, 1-500,000 as key, 20k bytes string value, do set, then get test, all get test has result.
　　Request per second(mean)
　　(Aug 13 Update: fixed a bug on get() that read non-exist key)

StoreWriteReadMemcachedb357 327 Tokyo Tyrant3,501 257 Redis1,542 957 4. Some notes about the test
　　When test Redis server, the memory goes up steadily, consumed all 8G and then use swap(and write speed slow down), after all memory and swap space is used, the client will get exceptions. So use Redis in a productive environment should limit to a small data>So compare Redis together with MemcacheDB/TC may not fair because Redis actually does not save data to disk during the test.
　　Tokyo cabinet and memcachedb are very stable during heavy load, use very little memory in set test and less than physical memory in get test.

　　MemcacheDB peformance is poor for write large data>　　The call response time was not monitored in this test.