I have started reading the Amazon DynamoDB research paper, which is one of the most influential works in key-value stores.
In this blog post, let’s focus on Partitioning.
Partitioning
Partitioning involves dividing the data and storing it across multiple nodes (computers).
The operations of Amazon DynamoDB key-value store are:
put(key, value)get(key)
If Amazon DynamoDB were designed to store the entire organization’s data on a single node, it could result in key collisions and create performance bottlenecks during key updates and retrievals. There should be a mechanism to partition the data and distribute it across multiple nodes. This way, as the dataset grows incrementally, new nodes can be seamlessly added to handle the increasing volume of key-value storage requests.
In the paper, they employed a technique known as consistent hashing.