Friday, October 25, 2013

How Dropbox Stores Stuff for 200 Million Users


How does Dropbox store files for those 200 million users, who store 1 billion files every 24 hours? The company uses more than uses more than 10,000 physical servers to manage user content, along with Amazon Web Services. User metadata is stored in the company’s data centers, while the actual files reside on Amazon’s S3 storage service. Dropbox also uses Amazon EC2 instances to help the data centers “talk” to its cloud storage.

What are the tough challenges for Dropbox? Ziga Mahkovec, a member of the Dropbox engineering team, identified several areas:

  • Serving Photo Thumbnails – Dropbox needed a more efficient “pipeline” to convert newly uploaded photos into thumbnails. By batching http requests, Mahkovec said, the Dropbox team was able to improve performance by 40 percent. 
  • Desktop Client Throughput – This problem was addressed by uploading file chunks in parallel using multiple connections, leading to a 2.5x speedup across Dropbox, and a 4X improvement in Japan. 
  • Storage Latency on Amazon S3 – “The problem with using a service like S3 is that we don’t have much control over performance,” said Mahkovec. “It’s a black box to us. So we have to do good monitoring.” In this case, Dropbox pooled its requests to avoid making new connections.
Read the complete article here

Leave a Reply

All Tech News IN © 2011 & Main Blogger .