links for 2009-04-21

  • * Built on Tokyo Tyrant. One of the fastest key-value databases [benchmark]. Tokyo Tyrant has been in development for many years and is used in production by Plurk.com, mixi.jp and scribd.com (to name a few)…
    * Great performance (comparable to memcached!)
    * Can store millions of keys on very few servers – tested in production
    * Scale out by just adding nodes
    * Nodes are replicated via master-master replication. Automatic failover and load balancing is supported from the start
    * Ability to script and extend using Lua. Included extensions are incr and a fixed list
    * Hot backups and restore: Take backups and restore servers without shutting them down
    * LightCloud manager can control nodes, take backups and give you a status on how your nodes are doing
    * Very small foot print (lightcloud client is around ~500 lines and manager about ~400)
    * Python only, but LightCloud should be easy to port to other languages.
    * Ruby port under development!
  • Voldemort is a distributed key-value storage system

    * Data is automatically replicated over multiple servers.
    * Data is automatically partitioned so each server contains only a subset of the total data
    * Server failure is handled transparently
    * Pluggable serialization is supported to allow rich keys and values including lists and tuples with named fields, as well as to integrate with common serialization frameworks like Protocol Buffers, Thrift, and Java Serialization
    * Data items are versioned to maximize data integrity in failure scenarios without compromising availability of the system
    * Each node is independent of other nodes with no central point of failure or coordination
    * Good single node performance: you can expect 10-20k operations per second depending on the machines, the network, and the replication factor
    * Support for pluggable data placement strategies to support things like distribution across data centers that are geographical far apar

  • Tokyo Cabinet is a library of routines for managing a database. The database is a simple data file containing records, each is a pair of a key and a value. Every key and value is serial bytes with variable length. Both binary data and character string can be used as a key and a value. There is neither concept of data tables nor data types. Records are organized in hash table, B+ tree, or fixed-length array.

    Tokyo Cabinet is developed as the successor of GDBM and QDBM on the following purposes. They are achieved and Tokyo Cabinet replaces conventional DBM products.

  • You can think of Neo4j as a high-performance graph engine with all the features of a mature and robust database. The programmer works with an object-oriented, flexible network structure rather than with strict and static tables — yet enjoys all the benefits of a fully transactional, enterprise-strength database.