Sharding
Last updated
Sharding is a database partitioning technique used by companies with the purpose of scalability, enabling them to process more transactions per second. Sharding splits a blockchain's entire network into smaller partitions, known as 'shards'. Each shard is comprised of its own data, making it distinctive and independent when compared to other shards.
Sharding can help reduce the latency or slowness of a network since it splits a blockchain network into separate shards. However, there are some security concerns surrounding sharding in which shards can be attacked.
Blockchain networks and their respective are gaining in popularity due to the widespread application of the technology, which includes and financial transactions. As the popularity of blockchain grows, so too does the workload and transactional volume that is handled by the network. If we think of a blockchain as a shared database, as more and more data is added, the network needs to find new ways to be able to process all of that data efficiently and quickly, which is where sharding can help.
The of blockchain technology makes it attractive since it allows transactions to be consensually shared across multiple sites and geographies. As transactions are recorded, copies are sent to the shared network within seconds, creating public "witnesses". If one portion of the network falls victim to fraud or a malicious attack, the shared network's participants can identify what was changed by the fraudsters since they all maintain a copy of the ledger's transactions. As a result, blockchain technology and its distributed ledger system can help reduce fraud and limit the damage from .
However, one of the major challenges with blockchain technology is that as additional computers are added to the network and more transactions are processed, the network can become bogged down, slowing the process — called 'latency'. Latency is a hurdle to blockchain being adopted for widespread use, particularly when compared to the current electronic payment systems that work quickly and efficiently. In other words, is a challenge for blockchain since the networks may not be able to handle the increased amounts of data and transaction flow as more and more industries adopt the technology.
One of the solutions being considered for creating latency-free scalability is the process of sharding. Sharding is designed to spread out the workload of a network into partitions, which may help reduce latency and allow more transactions to be processed by the blockchain.
Before exploring how sharding is accomplished within a blockchain network, it's important to review how data is currently stored and processed.
Currently, in blockchain, each node in a network must process or handle all of the transaction volumes within the network. Nodes in a blockchain are independent and are responsible for maintaining and storing all of the data within a decentralized network. In other words, each node must store critical information, such as account balances and transaction history. Blockchain networks were established so that every node must process all of the operations, data and transactions on the network.
While it ensures a blockchain’s security by storing every transaction in all of the nodes, this model slows transaction processing considerably. Slow speeds for processing transactions do not bode well for a future in which blockchain becomes responsible for millions of transactions.
Sharding can help since it partitions or spreads out the transactional workload from a blockchain network so that every node doesn't need to handle or process all of the blockchain's workload. In a way, sharding compartmentalizes the workload into partitions or shards.
Sharding can be accomplished through the horizontal partitioning of databases through division into rows. Shards are conceptualized based on characteristics.
For example, one shard might be responsible for storing the state and transaction history for a specific type of address. Also, it might be possible to divide shards based on the type of digital asset stored in them. Transactions involving that digital asset might be made possible through a combination of shards.
Each shard is still able to be shared amongst the other shards, which maintains a key aspect of blockchain technology — the decentralized ledger, enabling accessibility of ledger transactions to every user.
One of the main issues in the practice that has arisen is security. Though each shard is separate and only processes its own data, there is a security concern regarding the corruption of the shards, where one shard takes over another shard, resulting in a loss of information or data.
If we think of each shard as its own blockchain network with its authenticated users and data, a hacker or through a cyber attack could take over a shard. The attacker could then introduce false transactions or a malicious program.
Also, it's important to note that sharding is still in the early testing phase of being used for blockchain networks. As a result, all of the potential issues and challenges have yet to be worked out.
One of blockchain’s biggest challenges has been scalability: how does a network grow to handle increased activity without compromising on security? Our solution to that question is Nightshade Sharding.
While there are many new blockchains claiming to have thousands of transactions per second (TPS) and many layer-2 scaling solutions, due to their non-sharded designs, they would all hit an upper limit of what the network could handle eventually.
NEAR’s Nightshade Sharding, on the other hand, allows NEAR Protocol to scale linearly with the number of shards, thereby having the ability to satisfy the demand for transactions as more and more users start to use NEAR. Ultimately it prepares NEAR for mass consumer adoption of billions of users.
Consider a rental transaction in which multiple shards are involved. These shards correspond to different entities involved in the transaction, from customer names to configured into a smart lock that is made available to the renter upon rent payment.
, one of the most prominent blockchain companies, is on the front line of testing sharding as a possible solution to latency and scalability issues. Ethereum plans to roll out 64 new shard chains after what it calls 'The Merge', takes place, where the Ethereum Mainnet will "merge" with the Beacon Chain proof-of-stake system. Ethereum has combated the potential of a shard attack by randomly assigning nodes to certain shards and constantly reassigning them at random intervals. This random sampling would make it difficult for hackers to know when and where to corrupt a shard.
Another Protocol that is known for its attempt at sharding is Near Protocol. Since NEAR released the design for the , it was always meant to be a sharded blockchain. However, when preparing for the launch in early 2020, we realized that there was no need to launch a completely sharded network.
1) . By . Updated November 12, 2021. Reviewed by Fact checked by 2) Near.