In order to work towards full and secure sharding, we’ve been diving deeper into RapidChain.. The paper presents a path to full sharding of compute, storage, and communication. In contrast to Omniledger, which uses only compute and storage, RapidChain introduces sharding of communication via Kademlia routing. Rapidchain also uses information dispersal algorithm (IDA) to improve block gossip time by more than 5.3x.
Among other contributions, RapidChain presents a new approach to decentralized bootstrapping to start the first epoch. Using a multi-stage (finite steps) election process, nodes select a leader/root group. This root group selects the first shard, reference shard, which generates randomness used for partitioning all nodes at random into specific shards.
The consensus process then starts with a reconfiguration process at the end of each epoch. RapidChain uses a synchronous BFT consensus, which we will dive into in future posts. The paper assumes a slowly adaptive adversary that can attack at the end of each epoch or while joining/leaving during reconfiguration. We are discussing more internally on the reasoning for only assuming attackers can compromising nodes at the end of each epoch. Using peer discovery, nodes can find their neighbors and know the constituents of their shards.
Any transaction sent to any node is then routed to a target shard via Kademlia. This shard then processes the transaction.
The paper demonstrated its architecture using 2MB block size and showed that with 4,000 nodes, it achieved 7,380 TPS with 8.7 sec latency. The architecture used only 1/16x storage per node (with 250 nodes per shard) and had a time to failure of 4,580 years (see Table 1 for comparison to OmniLedger).
There’s quite a bit more to be covered and this is just a quick introduction. We will be exploring each component of the paper in more detail, but please feel free to ask us questions and share your own thoughts on full and secure sharding.!