1/ Sharding
Before talking about Adaptive State Sharding, we must first know what sharding is. This technology was already used in the traditional world before being extended to blockchains.
It is a system that allows to “partition” data coming from the same place.
The goal is to separate different types of information from the same database on several servers.
This makes it possible to process large volumes of data quickly and easily. Most large companies use this system to manage their data.
2/ Scalability
It is also important to talk about the “scalability” of a blockchain. The latter has indeed limits of use at the level of transactions per second (TPS).
It must be able to support a workload during a period of moderate activity (cruising speed), but during a large peak of use, thus increasing TPS on the network, the blockchain must be able to respond to everyone without being congested, at the risk of seeing transaction fees increase or even transactions cancelled. This is where “scalability” comes into play.
The goal of this process is to give a blockchain the ability to scale up, to be able to process more transactions per second, to be “scalable”.
This is simply explained in this diagram: if the number of TPS increases and approaches the maximum capacity of the blockchain, the latter adapts to accept this growing number of requests.
3/ The different shards on the MultiversX blockchain
A shard can be translated as “éclat” or “fragment” in French. We have 3 shards + 1 Metachain.
- Shards 0, 1 and 2 process all the transactions that users make. Each one works in parallel on a part of the blockchain.
- The Metachain retrieves all the data sent by shards 0,1 and 2 to process them definitively in order to register them in the blockchain. It is a bit like the clerk of the network.
I invite you to read the thread of CaptainRoXtar which summarizes this structure perfectly.
https://twitter.com/CaptainRoXtar/status/1614244292301668356
4/ The different types of sharding
There are three main types of sharding:
- Network Sharding
Network sharding manages the way nodes are grouped into shards. The goal is to increase the speed of execution by communicating directly through several shards rather than to the entire network at once.
- Transaction sharding
Transaction sharding manages the distribution of transactions in the different shards: sending tokens, creation of smart contracts, creation of tokens/NFT… everything is taken into account.
- State sharding
State sharding is a system for sharing the “state” of the blockchain. Each node keeps only a part of the state of the blockchain at the moment T.
If the two wallets involved in a transaction are located on a different shard, the network will have to perform an “update” of the state in both shards.
These nodes must be reshuffled, this is where the “node shuffling” comes in:
Node reshuffling is applied at the end of each epoch:
- Some validators are extracted from each shard and leave.
- Those wishing to return to a shard are queued to synchronize during an epoch.
- Those who have already spent an epoch in the queue are integrated.
The latter are then randomly integrated on each shard and will spend an epoch to synchronize while securing the network. Then the process is repeated 24 hours later.
5/ Adaptive State Sharding
MultiversX therefore undertook to combine these three types of sharding in order to create the Adaptive State Sharding, the goal is to assemble the advantages of the three types of sharding seen above, in order to obtain the best possible performances.
The blockchain thus becomes scalable, allowing transactions to be dynamically distributed over different shards according to the processing load. It is therefore able to increase or decrease the number of shards.
To avoid any problems, the shards must be balanced in order to distribute the workload correctly. A shard must not be overloaded.
We therefore have a blockchain made up of an evolving sharding where the possibility of increasing or decreasing the number of shards to function correctly exists.
Sometimes it happens that the blockchain gets congested due to a huge demand. When transactions take a long time to go through the fault lies with the Metachain. Indeed, since the shards 0,1 and 2 send the information to the Metachain, the Metachain becomes a bottleneck and struggles to process and register the tx in the blockchain.
The solution in addition to increasing the number of shards, and why not add a second Metachain.
It is important to know that for the moment, the Adaptive State Sharding is not in operation, because the blockchain does not have the use of it yet. The day it will need it, the number of shards will be increased.
