Wednesday, June 4, 2014

In-memory database sizing - throw out conventional wisdom

Sizing an in-memory database does not follow conventional database sizing rules.

For traditional databases, you buy a decent server machine, likely one with many CPU cores and reasonable memory, and then focus on application IOPS (I/O Operations per Second). If you are really going to stress the database, you must choose disks that can support the I/O needs of your application, today and in the future. Because these systems often use many disks to achieve high I/O performance, capacity is usually an afterthought.

With in-memory databases, throw out everything you know about sizing databases. Performance is now the afterthought, but capacity must be carefully considered.

To size an in-memory clustered database application you must consider the following factors:

  • First, compute the size of the data your application expects to host. This means computing the size of all rows in all tables. And also computing the size of all table indexes. VoltDB provides an easy-to-use interactive sizing tool in our application catalog report, described later in this post. 
  • If you want a highly available database cluster, all data must be stored redundantly. You’ll need to multiply your logical data size by the replication factor to understand your memory requirements. For example, if you have 100GB of table and index data, and you want your database to be able to withstand losing one of the nodes in the cluster, you will want each datum to be stored twice, and on two different physical machines. This means that you need to size the data requirement of your database to be 200GB (100GB of data X 2 copies of the data). 
  • Allow for some extra memory, perhaps an additional 25%, for database and operating system overhead. VoltDB uses extra memory for storing intermediate results, maintaining an undo log, caching SQL plans and buffering network data. Note that on a machine with 4GB, you’ll want to leave a larger fraction of memory free than on a machine with 256GB.
Read more here

Leave a Reply

All Tech News IN © 2011 & Main Blogger .