We have made sure to stick to the framework proposed dydx.forum/t/validator-introduction-framework/248
Validators are located in India with sentries geographically distributed all over the globe (Frankfurt, London, Sydney, and others).
We use a hybrid strategy where we usually host validators on bare metal with sentries hosted on clouds. Validators are usually based in Bangalore, India, whereas sentries are distributed geographically for better P2P connectivity and fault tolerance.
We started validating since mid-2020, so that adds up to more than 3 years of experience.
In total, we operate validators for 28 networks. Some of the big names include Sui, Axelar, Osmosis, The Graph Protocol, Mina, Umee, Agoric, and many other Cosmos-based SDK chains. The full list can be found here.
Based on your participation in any previous testnets, mainnets, are there any best practices to be aware of? What are some things that made previous testnets, mainnet launches successful and/or things to avoid that have gone poorly?
We would like to categorize our learnings into two categories:
- Validators should be run in a sentry architecture wherever possible.
- Remote signers should be used for better security.
- In some networks, remote signers can cause missed blocks, so be careful of that too.
- Testnets are for practice and experiments. We make sure we have performed each possible validator operation in a testnet at least once.
There should be sufficient time between gentx collections and genesis release. Sometimes teams do not realize that creating genesis from all gentxs is a tedious task. If any of the gentxs is wrong, you have to exclude it from the genesis to start the network. However, knowing if a gentx is correct or not depends upon its signature, and the only way to know that is by starting the network each time with some set of gentxs. It’s sort of like trial and error. The best way is to have a CI/CD pipeline built on GitHub that checks the PR, tries to start the network with that PR, and shows the results as soon as possible. I have seen some testnets and even mainnet launches being postponed because there wasn’t sufficient time between gentx collection and genesis release, and the debugging process took longer than expected. Also, validator would need to resubmit their gentx if they have submitted a wrong one.
Never start the network with inflation. We have seen it happen many times. It is always a good practice to start the network with no inflation and then enable inflation with an on-chain governance proposal. Some validators are ready with their auto-compounding scripts, while others are not, and it can create a skewed voting power distribution very early on.
Clear announcements should be made using either Discord relative time or always sticking to the standard UTC timezone. Announcements like “we will start collecting gentx tomorrow morning” are bad on so many levels that I can’t even write them all. TLDR: we all live in different time zones, and it might already be morning for me, plus I have no clue which timezone you are in, so there is no way I can deduce what time gentx collection starts.
An Emergency Email SOP should be built. Validators should be provided with instructions on what an emergency email will look like, including the title and the email address from which it will be sent. Validators can then set up a PagerDuty incident to receive immediate call alerts and take action promptly. Axelar has implemented this approach, and we find it highly valuable.
Yes, we indeed have a very detailed SOP that guides our voting. The four founding pillars of our voting mechanism are:
Understand the Proposal: We commit to thoroughly understanding each proposal, including its potential immediate and long-term impacts.
Consider the Implications: We carefully evaluate the potential risks and benefits of each proposal and assess its alignment with the project’s mission and values.
Listen to the Community: We actively seek diverse perspectives from our community while staying vigilant against potential biases and misinformation.
Reflect on Personal Interests: We pledge to balance our personal interests with the overall health and success of our project.
In the spirit of transparency, our voting history is publicly available for review on our website along with our voting mechanisms on the king.super.site/governance-record
Are you planning to play any additional roles in the dYdX ecosystem (e.g., market maker, trader, indexer, front-end, other)?
We would love to operate a decentralized frontend. I think there was a blog post on it as well that there are plans to decentralize the frontend too. Also, we would be happy to operate an indexer if needed. And of course, we will be a user of the dYdX exchange.
Yes, we have a discord tool that can pull proposals directly from the chain and post them into Discord, well-formatted and tagged with a specific role to inform about the new proposal. This reduces the headache for the team to post every proposal manually.
Not only that, it has all the necessary links:
- Link to the proposal on the explorer
- Link to Keplr where one can connect their wallet and vote
Currently, we are hosting it for around 14 Cosmos chains like Osmosis, Axelar, Gravity Bridge, Evmos, Juno, etc.
We believe this could be of great use for dYdX in keeping the community updated on new proposals and potentially increasing on-chain governance participation. We have also made a bunch of improvements to it recently
Apart from this, we can code any custom tools, or anything that the team believes could be of use to the community. We are always open to build services for the projects we validate on. You can take a look at our services page to view the full list of services we host for other networks: king.super.site/services
- We have closely worked with the Axelar team and helped them organize their testnets and improve their documentation site. We received a grant for our work from Axelar.
- We also helped the Ume team with the early stage of their testnet and building tools that supported early incentivized testnets. They decided to give us a grant for our work and help during the early phases.
- We have also received grants from Aleo for working with them on the Leo programming contest.
- We receive a monthly grant from Osmosis for hosting a customized version of our governance proposal discord tool for them.