Resources

People often ask me "How did you learn how to hack?" The answer: by reading. This page is a collection of the blog posts and other articles that I have accumulated over the years of my journey. Enjoy!

Different nodes in blockchain need to always come to the same state for a network to work. If the network is split in some way, then the network will not be able to come to consensus, taking the entire chain down. On the Provenance testnet, they noticed that the perceived state (app has) was different. So, what happened?

Using their block explorer, the final call was to MarkerTransferAuthorization with an authz wrapped call. So, the modules bank, authz, auth and marker were the only possible culprits here.

The Cosmos SDK has a tool just for this type of issue: iaviewer for reviewing the AVL tree for state changes. From this tool, they wrote a diff for the two different chain states. This came with some results.

The IAVL tree is just a rebalancing binary tree. The shape of it should change for each new node that is added. The good node wrote out an authz grant to the state store but the bad node did not. So, the bug must be in the authz grant side of it.

When setting up the grant, it was using the time.Now() value in Go. Yikes! A big source of non-determinism. When performing an upgrade, nodes that came on sooner were fine on these grants. However, some nodes that came on later after the upgrade would have been this grant fail! In the end of the day, this was actually an issue with the Cosmos SDK itself.

An interesting post on a debugging a Cosmos blockchain app hash issue. I bet I'll see this in the future so it was a super helpful post for me!