So, your team has finished the alpha version of your blockchain, and it's time to launch the testnet, followed by the mainnet. You have a real blockchain with independent participants, a strong economic model, security, and you've designed governance; now it's time to put it all to the test. In an ideal crypto-anarchic world, you would release the genesis block online, along with the final node code, and the validators would take care of everything themselves, setting up all the auxiliary services while everything falls into place. But that's a fictional scenario. In reality, the team must prepare a fair amount of auxiliary software and perform various tasks to help validators launch a stable network. This is the subject of this article.
Launching networks based on consensus mechanisms like 'proof-of-stake,' where validators are determined by votes from the system's token holders, is quite a specific endeavor. Even launching traditional, centrally managed systems with dozens or hundreds of participants is a challenging task. servers Starting a blockchain requires the efforts of loyal yet independent participants. In a corporation, administrators have full access to all machines, logs, and overall monitoring during the launch. However, validators will not allow anyone near their servers and are likely to prefer building their infrastructure independently, as it controls access to the primary assets of the validatorsâvoting stakes. This very behavior enables the construction of distributed secure networksâindependence from the cloud providers used, virtual and bare-metal servers, diverse operating systems; all of this makes attacks on such a network extremely inefficientâtoo many different software variations are in use. For instance, Ethereum has two main node implementations, one in Go and one in Rust, and an attack effective against one implementation will not work against the other.
Therefore, all processes for launching and operating blockchains must be organized so that any validator, or even a small group of validators, could throw their computers out of the window at any moment, and nothing should break. The remaining validators must continue to effectively support the network and onboard new validators. When launching a network, with one validator in Europe, another in South America, and a third in Asia, achieving coordinated work among several dozen independent groups and engaging them in the outcome is quite challenging.
Validators
Letâs imagine the launch of a hypothetical modern blockchain (most of the described applies to blockchains based on any modern family of blockchains: Ethereum, EOS, Polkadot, Cosmos, and others with proof-of-stake consensus). The main actors in such blockchains are validator teams that set up their own independent servers, validating and producing new blocks, and receiving rewards provided by the network for those participating in the consensus. Launching new networks requires several dozen validators (that is how many can relatively efficiently achieve consensus in seconds), so the project announces registration, during which validators share public information about themselves with users, convincing them that they will provide quality service for the launched network.
Validation is a business that allows for a very accurate assessment of a validator's potential income, to quickly shift resources between projects, and in the event of success of the chosen network, the validator can develop the project as a full participant in the DAO and as a responsible party, or simply provide excellent technical service for fully transparent, fairly earned money. In calculating rewards for validators, projects strive to consider the expenses of the validators, making the reward for blocks such that this business is profitable, while also preventing validators from crashing the economy by flooding it with money and depriving other users of the network.
Validator business requires ensuring high service fault tolerance, which means a high level of preparation for DevOps and developers, as well as expensive computing resources. Even without the need to mine hashes in proof-of-work networks, a blockchain node is a large service that occupies a lot of memory, consumes significant computations, validates, writes to disk, and transmits large volumes of data over the network. Storing the transaction log and chains of blocks for a blockchain with several thousand small transactions per block now requires storage of 50 Gb or more, and for blocks, it must be an SSD. The state database of blockchains supporting smart contracts can already exceed 64Gb of RAM. Servers with the required specifications are quite expensive; an Ethereum or EOS node can cost between $100 and $200/month. Additionally, there is an increased labor cost for round-the-clock work from developers and DevOps who address issues even at night during a launch, as some validators may easily be located in another hemisphere. Nevertheless, in favorable moments, owning a validator node can generate significant income (in the case of EOS, up to $10,000 per day).
Validation is just one of the new potential IT roles for entrepreneurs and companies. As programmers devise increasingly sophisticated algorithms to reward honesty and penalize deceit and theft, services are emerging that perform functions such as publishing important data (oracles), overseeing (slashing deposits and punishing deceivers by publishing proof of fraud), dispute resolution services, insurance, and options. Even garbage collection presents a potentially large market in smart contract systems where payment for data storage is necessary.
Blockchain launch issues
The openness of blockchain, allowing free participation in the network's operations from any country and the ease of connection for any script kiddie following instructions on GitHub, is not always an advantage. The pursuit of a new token often pushes validators to 'mine a new coin at launch,' hoping for price growth and the opportunity to quickly cash out their earnings. Additionally, this means that your validator could be anyone, even anonymous; votes can be cast for them just as for other validators (though it will be challenging for the anonymous one to gather votes from stakeholders, so weâll leave the scary tales about anonymous cryptocurrencies to politicians). However,
The project team has the task of somehow bringing those into their network who can ensure stable node operation in the future, understand security, quickly solve problems, cooperate with other validators, and act collectively â the quality of that token, into which the network participants plan to invest their time and resources, heavily relies on these qualities. Adequate founders understand the risks well and know that when launching software of this scale, they will inevitably face coding and configuration errors, and that the stability of the network depends on how well developers and validators will collaboratively address such issues.
The team is ready to vote in the mainnet for any validators, but they would like to know which ones are good. The ones with the largest portfolio? Almost no one has that now. Based on team profiles on Linkedin? Experienced DevOps or security professionals won't provide you with any Linkedin profiles. Based on statements in chat, posts, and help provided to others during the preparation phase? Sure, but thatâs subjective and inaccurate.
In such conditions, there is only one solution â to create a game that effectively addresses everyoneâs issues, where participants can select the best validators, but most importantly â to test the blockchain's resilience and conduct a full-scale battle test of the blockchain under active use, changes in consensus, and the emergence and resolution of errors. For the first time, this procedure was introduced as a game by the team from Cosmos, and this idea is undoubtedly an excellent way to prepare the network for launching a reliable and fault-tolerant mainnet.
Game of Validators
I will describe the validator game as we designed it for the DAO.Casino (DAOBet) blockchain, which is based on a fork of EOS called Haya and has a governance mechanism that is quite similar â validators are elected through votes from any account, during which a portion of the balance used to vote for the validator is frozen. Any account holding the main BET token can vote for a selected validator with any part of its balance. Votes are tallied, and a top list of validators is established. Different blockchains organize this process in various ways, and usually, this is the part where the new blockchain differs from the parent one. I must say, in our case, EOS fully justifies its âOSâ designation, as we genuinely use EOS as the foundational operating system to deploy a modified version of the blockchain for the tasks of DAOBet.
I will outline specific issues and how they can be addressed within the game. Imagine a network where your server can be openly attacked, where continuous interaction with the network is needed to maintain your validator position, promoting your validator and ensuring that it produces blocks that are delivered to other validators on time; otherwise, the validator will be removed from the list.
How to select the top winners?
The main technical requirement for the game is that its results must be publicly verifiable. This means that the results of the game: the top winners, must be formed strictly based on data that any participant can verify. In a centralized system, we could measure the uptime of each validator and reward those who were online the most or processed the maximum amount of network traffic. We could gather data on CPU load and memory usage and reward those who worked diligently. However, any such metric collection implies the existence of a central collection point, and the nodes are all independent and can behave as they wish, sending any data.
Therefore, a natural solution is that winners should be determined based on data from the blockchain, as it allows us to see which validator produced which block and what transactions were included in it. We have named this number Validator Points (VP), and earning them is the primary goal of validators in the game. In our case, the simplest, easily publicly verifiable, and effective metric of a validator's 'utility' is VP = number_of_blocks_produced_by_validator over a given time period.
This straightforward choice is due to the fact that governance in EOS already addresses many emerging issues, as EOS is the successor of three generations of functioning blockchains with extensive experience in complex network management. Practically any issues a validator encounters with the network, processor, or disk lead to a single problem â they sign fewer blocks and receive lower payment for their work, which again brings us back to the number of blocks signed â for EOS, this is an excellent and simple option.
For other blockchains, the method of calculating Validator Points may vary. For example, in pBFT-based consensus mechanisms (Tendermint/Cosmos, the Aura consensus from Parity Substrate), where each block must be signed by multiple validators, it makes sense to count individual validator signatures rather than blocks. It may also make sense to consider incomplete consensus rounds that use the resources of other validators; this largely depends on the type of consensus.
How to model real-world operating conditions
The founders' task is to test validators in conditions that are close to reality, all while having no centralized control. This problem can be addressed with a faucet contract that distributes equal amounts of the main token to validators and anyone else interested. To receive tokens in their balance, a transaction must be initiated, and they must ensure that the network includes it in a block. Therefore, for a validator to succeed, they need to continually top up their balance with new tokens and vote for themselves, pushing their way to the top. This activity puts a constant load on the network, and parameters can be adjusted to ensure that the flow of requests is substantial enough for a thorough network test. Thus, plan the faucet contract in advance as an essential tool for launching the network, and start selecting its parameters early.
Requesting tokens from the faucet and validators voting still does not completely and fairly emulate the operation of the blockchain, especially in highly loaded modes. Therefore, the blockchain team will still have to create additional benchmarks to stress the network. Specially designed smart contracts play a crucial role in this, allowing individual subsystems to be tested. For storage testing, the contract saves random data to the blockchain, while for checking network resources, the test contract requires a large volume of input data, thereby inflating the transaction volume â by triggering a flow of such transactions at random times, the team simultaneously tests the stability of the code and the resilience of the validators.
An important issue is updating node code and conducting hard forks. It is necessary that, in the event of a bug, vulnerability, or collusion among malicious validators, the validators have a plan of action already developed during validator scenarios. One could devise schemes for awarding VP for the rapid implementation of hard forks, for example by penalizing all validators who have not updated their node code to the new version, but this is complex to implement and complicates calculations. Simulating an emergency hard fork situation can be artificially achieved by 'breaking' the blockchain at a designated block. Block production halts, and ultimately, those who start signing blocks earlier will benefit, so VP based on the number of signed blocks fits well here.
How to inform participants about the network status and fix errors
Despite the mistrust among validators, timely access to current network status information is beneficial for everyone to make faster decisions. Therefore, the project team is establishing a service for collecting and visualizing multiple metrics from validator servers, which allows one to see the situation simultaneously across the entire network, enabling quick identification of what is happening. Additionally, both the validators and the project benefit from the prompt correction of identified errors by the project team. Thus, aside from collecting metrics, it makes sense to immediately start gathering logs and error data from validator machines to a machine accessible to blockchain developers. No one benefits from distorting information, so these services are established by the project team and can be trusted. It is sensible to gather system metrics from validators, and importantly, the most critical metrics from the blockchain itselfâfor DAOBet, this includes the finalization time and the lag of the last finalized block. This way, the team can observe increased memory consumption on nodes when running benchmarks and address issues from specific validators.
Key points regarding the validator game
As it turns out, if you want to officially allow validators to attack each other's machines (unofficially they can already do this) â you need to legally formulate this separately as a security test, since in the legislation of some countries, DDoS or network attacks can lead to penalties. Another important question is how to reward validators. Natural prizes include project tokens that will be transferred to the mainnet, but a massive distribution of tokens to anyone who can launch a node is also not the best option. Most likely, you will have to balance between two extremes:
Distribute the entire prize fund according to earned VP
this is very democratic and allows everyone who invested time and resources in the validator game to earn
but it attracts random people to the game without prepared infrastructure
Distribute the prize fund to the top-N validators based on the results of the game
the winners are likely to be the validators who consistently held out during the game, very strictly focused on winning
some validators may not want to participate, undervaluing their chances of winning, especially if there are renowned validators among the participants
Which option to prefer is up to you
There is one more point â it is by no means guaranteed that dozens of validators will rush to participate in the game at your invitation, and among those who decide to try, not all will even install and launch a node â usually, at this stage, projects have quite sparse documentation, errors occur, and developers working under time pressure do not respond to questions very promptly. Therefore, before launching the game, it is also necessary to foresee actions in case the needed number of validators is not met. In this case, at the start of the game, the missing validators are launched by the project team, participate in the consensus, but cannot be winners.
Conclusion
In conclusion, I tried to compile from the above a list of what needs to be devised, done, and launched for an effective conduct of the validator game
What needs to be done to launch a real validator game:
develop your blockchain đ
- create and set up a web interface and provide CLI for voting for validators
- Ensure that metrics from the running validator node can be sent to a centralized service (for example, Prometheus)
- Set up a metrics collection server (Prometheus + Grafana) for the validator game
- Devise a method for calculating Validator Points (VP)
- Develop a public script that calculates a validator's VP based on blockchain data
- Create a web interface to display the top validators and the state of the validator game (how much time is left, how many VP each has, etc.)
- Develop and automate the deployment of an arbitrary number of your own nodes, design the process for connecting validators to the game (when and how to disconnect their nodes, cast and withdraw votes for them)
- Calculate how many tokens need to be distributed and develop a contract-faucet
- Create a benchmarking script (token transfers, heavy storage usage, massive network utilization)
- Gather all participants in one chat for quick communication
- Launch the blockchain slightly before the start of the game
- Wait for the starting block, begin the game
- Test the network with several types of transactions
- Implement a hard fork
- Change the list of validators
- Repeat items 13, 14, and 15 in a different order while maintaining network stability
- Wait for the final block, end the game, calculate VP
It should be noted that the validator game is a new concept, having been conducted only a couple of times, so this text should not be taken as a ready-made guide. No analogs exist in modern IT business â imagine banks competing with each other before launching a payment system on who will better handle client transactions. Traditional approaches are unlikely to help you create large decentralized networks, so embrace new business models, conduct your games, identify the worthy, reward them, and let your distributed systems operate quickly and reliably.
Source: habr.com
