Harmony is a sharded blockchain with four shards running each running consensus separately. Increasing usage and adoption of all of the shards is crucial to the Harmony ecosystem. This project will enable smart contracts running on OpenHarmony to pass messages asynchronously, while protecting the network from spam and other attacks.
The design enables smart contract developers to send and receive messages across shards using Solidity (or the EVM-targeting language of their choice) to call a new special “precompiled” contract, the “(message) router”. This explicit message passing design is safer than using cross-shard contract calls. To avoid spam attacks, sending messages must both pay for gas on the sending shard, and reserve tokens to pay for gas on the receiving shard. To prevent messages from being stuck with too little gas (possibly due to malicious attack), additional tokens can be contributed to this reserve until the message is received.
The final implementation will be built by MuKn and overseen by the Harmony team.
We intend the work to proceed as follows:
Implementing the safe cross-shard communication design as described in the design document. This includes:
Modifying Harmony core code to reflect the protocol and messaging changes described there;
Implementing the message router contract, and making corresponding changes to the SDK that allow clients to send messages across shards via the message router contract.
The implementation will be reviewed by the Harmony core team before being deployed and tested on testnet.
Once deployed to testnet, the implementation will be hardened and improved until it receives final approval by the Harmony core team to launch on mainnet.
We see the project having two milestones:
Milestone 1) build the prototype and deploy to testnet, and
Milestone 2) harden and test the design, then deploy to mainnet.
Proposal ask
$216k (50% upfront, 25% upon testnet launch, 25% upon mainnet launch)
Justification
Harmony is a sharded blockchain where there are 4 shards each running consensus separately. Increasing usage and adoption of all of the shards is crucial to the Harmony ecosystem. This project will enable smart contracts running on OpenHarmony to pass messages asynchronously, while protecting the network from spam and other attacks.
MuKn has successfully completed the design phase of the cross-shard messaging design ( Design cross-shard communication and smart contract functionality). The collaboration between MuKn and Harmony was smooth and pleasant. We would like to continue work with MuKn on the implementation of cross-shard messaging protocol.
Hi
Cross-shard communication is a must and will make Harmony an even better network.
In simple English and with a few diagrams could you please explain to the Community how this will work?
I want to know how you envisage this will be happening a more detailed document for the most Advanced readers would be be good as well.
So Cross-shard communication does it means in simple terms the I send a TX payment say of 100 $ONE I press sent and the "packet " is split in 4 then send via the most efficient fastest nodes then reassembled prior to arriving to the recipients wallet?
this split can act as double security I think if i am on the right track with nodes etc
Hi, I’m Ian, I’m the one working on this on behalf of MuKn. Per @rongjian’s request, I’ll be doing weekly status updates here; the previous two weeks I’d sent them via email, so if the below doesn’t make sense due to lack of context, feel free to ask questions. I’d originally wrote this with him in mind as the reader, so it assumes having read the proposal closely and also a familiarity with the internals of Harmony One’s codebase.
I started a branch on the original proposal where I am updating it
re: any deviations from the original design, so it can continue to
serve as documentation. So far the changes are very minor (e.g.
using the existing nonce, as we discussed via discord). See main...impl · MuKn / Harmony1 Proposal · GitLab
The router contract itself is written, and reasonably well tested.
vm.Context has a new EmitCXMessage callback used by the router
contract.
I’ve wired said callback up in core.ApplyMessage, and added a field
to ExecutionResult for any messages sent during a transaction.
I am currently exploring one stack frame above that in
ApplyTransaction.
It seems like it would almost work to just attach the extra message
information to the CXReceipt we are already returning, but this doesn’t quite work because a transaction could send more than one message. I
see two obvious avenues to explore:
Return more than one CXReceipt.
Instead of having a CXReceipt per-message, pack the data for all
messages into one receipt.
My instinct is to go for (1); this is more in-line with the original
proposal, and we want to be able to process message sends on the
receiver separately. (2) would be less invasive code-wise at least
as far as the sender is concerned, but it feels wrong to me since
the existing receipt has To/From etc. fields which would either have to
be replaced or would feel redundant.
My worry with (1) is that having the same TxHash for multiple sends
might causing problems on the receiving end that require some effort
to work around, but this is just a hunch; I haven’t gotten to digging
into that bit of the code yet so maybe it will be fine. I’ll do that
next.
If you want to follow my progress in more detail, the branch I’m working
on is here:
I went with option (1) for what to do with multiple message sends. The logic for emitting cross shard messages is now wired up. This ended up being easier than I expected.
I spent some time working on the logic for processing incoming cross-shard messages. This is WIP, but I have a clear enough picture of what the rest of it will look like that I decided to shifted gears to work on something with more unknown-unknowns to me, so I’m not hitting all the surprises towards the end of the project.
Namely: incoming cross-shard messages need to compete with other transactions for gas, so I’ve been digging into the details of how new blocks are proposed, and building a picture of what needs to change to facilitate cross shard messages in the pool. It’s coming together.
Wrapped my head around the block proposal logic well enough to get started.
After getting over that hump, I modified Worker.CommitTransaction and some related functions so that it will interleave normal transactions with incoming messages when choosing what to add to the block. This needs integration both with the WIP I mentioned last time (which needs to be completed) for the logic of actually updating state in response to a message, and some smaller changes are still needed in the code that calls this (in node_newblock.go). I also want to try to clean it up a bit; it feels like I’m checking “is this receipt a message or a balance transfer?” in a few too many places. And then all of this needs more testing, especially end-to-end/integration testing.
5/26/22: editing this reply since apparently discourse has a limit of 3 consecutive replies from the same person:
This week I did more work towards handling the receiving side of cross-shard transactions; the call sites of Worker.CommitTransaction now actually pass in a list of CXReceipts to consider, whereas last week this code was passing in an empty slice. I’ve also been working more on the logic to actually commit the effects of a message receipt.
All of this needs more testing, and I suspect that will shake out some more loose ends.
We’ve assigned a new developer, Emmanuel Angel, to the cross-shard messaging project. He’s been getting up to speed under the close supervision of our architect Fare and zenhack. While this may have caused a short-term delay to the current deliverable (building and deploying cross-shard messaging to testnet), we think it will overall speed up the project. Starting next week, please look forward to our usual weekly project updates!
Not directly, but this project will of course allow messaging with the proposed Game Shard. I couldn’t tell you when the Game Shard might be available though.
We’ve been asked by the Harmony One team to pause our work on cross-shard messaging for now. We’re excited to start back up once we get the green light to do so. In the meantime, come check us out over at MuKn.io or join our discord at MuKn
