Sharding is a process of distributing large amounts of data across multiple MongoDB servers. It is used to improve database performance, storage capacity, and scalability.

Sharding is also known as Horizontal Scaling.

In MongoDB, each server that stores a portion of the data is called a Shard.

Example

Suppose a college database contains 1,00,000 student records. Instead of storing all records on one server, MongoDB can distribute them across multiple shards.

This is a simplified example of range-based distribution. Actual distribution depends on the shard key and balancing.

Architecture of Sharding

Simplified MongoDB sharded cluster architecture

Main Components

1. Shard: Stores a portion of the database records. Each shard is normally deployed as a replica set.
2. Config Server: Stores metadata and configuration information about the sharded cluster.
3. Query Router (mongos): Receives requests from applications and routes queries to the appropriate shards.
4. Shard Key: A field used to determine how data is distributed among shards.

How to Perform Sharding in MongoDB (Step-by-Step)

The following example uses MongoDB on Windows with three shard servers, one config server replica set, and one mongos router.

For simplicity, all instances run on the same computer using different ports. This is suitable for learning, not production.

Step 1: Create Directories

Open Command Prompt as Administrator and create the following directories:

mkdir C:\data\shard1
mkdir C:\data\shard2
mkdir C:\data\shard3
mkdir C:\data\config

Step 2: Start Config Server

Open a new Command Prompt window and run:

mongod --configsvr --replSet configRS --port 27019 --dbpath C:\data\config --bind_ip localhost

Open another Command Prompt and connect:

mongosh --port 27019

Initialize the config server replica set:

rs.initiate({
  _id: "configRS",
  configsvr: true,
  members: [
    { _id: 0, host: "localhost:27019" }
  ]
})

Step 3: Start Shard Servers

Open three separate Command Prompt windows.

Shard 1:

mongod --shardsvr --replSet shardRS1 --port 27018 --dbpath C:\data\shard1 --bind_ip localhost

Shard 2:

mongod --shardsvr --replSet shardRS2 --port 27020 --dbpath C:\data\shard2 --bind_ip localhost

Shard 3:

mongod --shardsvr --replSet shardRS3 --port 27021 --dbpath C:\data\shard3 --bind_ip localhost

Keep all three windows running.

Step 4: Initialize Replica Sets

Open separate mongosh connections.

Shard 1:

mongosh --port 27018

rs.initiate({
  _id: "shardRS1",
  members: [
    { _id: 0, host: "localhost:27018" }
  ]
})

Shard 2:

mongosh --port 27020

rs.initiate({
  _id: "shardRS2",
  members: [
    { _id: 0, host: "localhost:27020" }
  ]
})

Shard 3:

mongosh --port 27021

rs.initiate({
  _id: "shardRS3",
  members: [
    { _id: 0, host: "localhost:27021" }
  ]
})

Wait until each replica set has elected its PRIMARY.

Step 5: Start MongoDB Query Router (mongos)

Open another Command Prompt:

mongos --configdb configRS/localhost:27019 --port 27017 --bind_ip localhost

The mongos process connects the application to the sharded cluster.

Step 6: Connect to mongos

Open another Command Prompt:

mongosh --port 27017

Step 7: Add Shards to the Cluster

Run the following commands in the mongos shell:

sh.addShard("shardRS1/localhost:27018")

sh.addShard("shardRS2/localhost:27020")

sh.addShard("shardRS3/localhost:27021")

Check the sharding status:

sh.status()

Step 8: Create Database and Collection

use college

db.createCollection("students")

Step 9: Enable Sharding on Collection

Use student_id as the shard key.

sh.shardCollection(
  "college.students",
  { student_id: "hashed" }
)

The hashed shard key helps distribute records based on the hash of student_id.

Note: In current MongoDB versions, you can shard a collection directly without first running sh.enableSharding("college").

Step 10: Insert Records

use college

db.students.insertMany([
  { student_id: 101, name: "Amit", course: "BCA" },
  { student_id: 102, name: "Rahul", course: "BSC IT" },
  { student_id: 103, name: "Priya", course: "BCA" },
  { student_id: 104, name: "Neha", course: "BBA" },
  { student_id: 105, name: "Ravi", course: "BCA" }
])

For a larger example, insert 10,000 records:

for (let i = 1; i <= 10000; i++) {
  db.students.insertOne({
    student_id: i,
    name: "Student" + i,
    course: "BCA"
  })
}

MongoDB distributes the data according to the shard key and cluster balancing rules. Distribution may not be equal immediately.

Step 11: Verify Sharding

Check cluster status:

sh.status()

Check collection distribution:

db.students.getShardDistribution()

You can also inspect how a query is routed:

db.students.find({
  student_id: 101
}).explain("executionStats")

Advantages of Sharding

1. Horizontal Scaling: Distributes data across multiple servers.
2. Improved Performance: Can distribute query workloads across shards.
3. Large Data Storage: Supports databases that exceed the capacity of a single server.
4. Load Distribution: Reduces workload on individual servers.
5. Scalability: Allows adding shards as data increases.

Difference Between Sharding and Replication

Important: This demonstration uses single-member replica sets without authentication, appropriate only for local practice. Production deployments should use multiple replica set members, authentication, and suitable network security.

Conclusion: Sharding in MongoDB is a technique used to distribute large datasets across multiple servers. It improves scalability and helps manage large amounts of data efficiently.