Kbaran at 22:37, 16 April 2018

2018-04-16T22:37:41Z

New page

[[Category:MongoDB]]

{{Copyright Notice}}

== Author ==
* [http://www.kamilbaran.pl Kamil Baran]
 

== What is Sharding? ==
* process of splitting data across multiple machines
* manual sharding (any database engine)
* MongoDB supports '''autosharding'''
* sharding is the most difficult and complex configuration of MongoDB
* familiarise yourself with standalone servers and replica sets before sharding
 

== When to use it? ==
* not too early because it adds complexity to the deployment
* not too late because it's difficult to shard an overloaded system without downtime
* to increase available RAM and disk space
* to reduce the load on a single server
* to read or write data with greater throughput than a single server can handle
 

== How to deploy MongoDB Sharded Cluster? ==
=== MongoDB Config Servers ===
* config servers store the metadata for a sharded cluster
* from version 3.4 it has to be deployed as a replica set using WiredTiger storage engine
* the admin database and the config database exist on the config servers
* deployed on separate physical machines, geographically distributed
* it doesn't need many resources (1MB / 200GB)
* it can be deployed on machines with other services
* backups should be done often and especially before any maintenance
<syntaxhighlight lang="bash">
$ mongod --configsvr --replSet sh_config --dbpath c:\data\sh_config_1 --port 27019
</syntaxhighlight>
 

=== MongoDB Router (the mongos process) ===
* '''mongos''' is the process for client applications to connect to
* it replaces mongod used for standalone server or primary in replica sets
* the number of instances is not limited
* common setup is one mongos per application server (on the same machine)
* must know where the config servers are
* it does not need data directory
* set up clients to send all requests to the mongos!

<syntaxhighlight lang="bash">
$ mongos --configdb sh_config/localhost:27019 --port 27017
</syntaxhighlight>
 

=== Adding a new shard into the cluster ===
* a shard should be a replica set
* replica set name will be used as a name of a shard

<syntaxhighlight lang="JavaScript">
$ mongod --shardsvr --replSet shard_a --dbpath c:\data\shard_a_1 --port 27018
$ mongo localhost:27017
> sh.addShard("shard_a/localhost:27018")
> sh.status()
</syntaxhighlight>
 

=== Test deployment of MongoDB Sharded Cluster ===
* commands below will start sharded cluster with
** 3 shards on ports 30000, 30001, 30002 (MongoDB 3.2: 20000, 20001, 20002)
** one mongos process on port 30999 (MongoDB 3.2: 20006)
 

<syntaxhighlight lang="JavaScript">
$ mkdir c:\data\db\test0
$ mongo --nodb
> cluster = new ShardingTest({shards : 3, config : 3, other : { rs : true }})
</syntaxhighlight>
 

== How Sharding Works? ==
* databases and collections are not sharded by default
* you must enable sharding for each database and each collection separately
* shard key, chunks and chunks ranges, splitting chunks, split threshold
* $minKey <= '''chunk1''' < split point 1 <= '''chunk2''' < $maxKey
* balancer and moving chunks
 

=== Range Based Sharding ===
<syntaxhighlight lang="JavaScript">
> sh.status()
> sh.enableSharding("test")
> use test
> db.visitors.createIndex({"visitor" : 1})
> sh.shardCollection("test.visitors", {"visitor" : 1})
</syntaxhighlight>
<syntaxhighlight lang="JavaScript">
> db.visitors.explain(1).find({visitor:"visitor1234"})
> db.visitors.explain(1).find({i:1234})
</syntaxhighlight>
=== Hash Based Sharding ===
=== Zone Based Sharding ===

== Sharded Cluster Administration ==
=== Balancer ===
<syntaxhighlight lang="JavaScript">
> sh.getBalancerState()
> sh.stopBalancer()
> sh.isBalancerRunning()
> sh.setBalancerState(true)
</syntaxhighlight>
 

=== Operations on chunks ===
<syntaxhighlight lang="JavaScript">
use config
db.settings.save({ _id:"chunksize", value:1 })
</syntaxhighlight>
<syntaxhighlight lang="JavaScript">
sh.splitFind("test.visitors", {visitor:"visitor_10"})
sh.splitAt("test.visitors", {visitor:"visitor_10"})
use admin
db.runCommand({mergeChunks:"test.visitors", bounds:[{visitor:"visitor_10"}, {visitor:"visitor_20"}]})
db.adminCommand({moveChunk:"test.visitors", find:{visitor:"visitor_99999"}, to:"shard_a"})</syntaxhighlight>
 

=== Refreshing mongos cache ===
<syntaxhighlight lang="JavaScript">
> use admin
> db.runCommand({flushRouterConfig : 1})
</syntaxhighlight>

Sharding in MongoDB - Revision history

Kbaran at 22:37, 16 April 2018