{"id":54229,"date":"2019-12-21T00:00:00","date_gmt":"2019-12-20T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/kratkij-obzor-instrumentov-dlya-testirovaniya-i-benchmarkinga-blokchejnov"},"modified":"2020-02-18T14:02:13","modified_gmt":"2020-02-18T11:02:13","slug":"kratkij-obzor-instrumentov-dlya-testirovaniya-i-benchmarkinga-blokchejnov","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/kratkij-obzor-instrumentov-dlya-testirovaniya-i-benchmarkinga-blokchejnov","title":{"rendered":"A brief overview of blockchain testing and benchmarking tools","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"A<\/p>\n

<\/p>\n

Today, solutions for testing and benchmarking blockchains are tailored to specific blockchains or their forks. However, there are several more general solutions that vary in functionality: some are open-source projects, others are offered as SaaS, but the majority are internal solutions developed by the blockchain team. Nevertheless, they all tackle similar tasks. In this article, I will briefly review several products specifically designed for blockchain testing.<\/p>\n

<\/a><\/noindex><\/p>\n

The operation of a blockchain network resembles that of a distributed database, so similar tools and methods can be used for testing. To better understand how distributed databases are tested, take a look at this good collection of resources and articles. from here<\/a><\/noindex>. For example, latency is broken down in detail in this article<\/a><\/noindex>, and to understand how bugs are found in replication algorithms, I recommend checking out this the article<\/a><\/noindex>.<\/p>\n

<\/p>\n

I will describe several popular solutions for testing and benchmarking blockchains. I would appreciate it if you could share other helpful software products for addressing the same tasks in the comments.<\/p>\n

<\/p>\n

\"A<\/p>\n

<\/p>\n

I will start with a tool that, although not specifically designed for blockchains, allows for effective testing of their operation provided that there is an already deployed network to experiment on. The critical factor for the reliability of a distributed system is the ability to continue operating in the event of server and network problems. These can include network lags, disk overflows, unavailability of external services (DNS), hardware failures, and hundreds of other reasons. To test the resilience of any systems working consistently across a large number of machines, you can use Gremlin<\/a><\/noindex>. It employs a highly effective approach known as Chaos Engineering. <\/p>\n

<\/p>\n

With its own network agent, Gremlin creates a variety of problems on the necessary number of machines: network lags, resource overloads (CPU, disk, memory, network), disconnects certain protocols, etc. For blockchains, Gremlin can be used on test network servers, emulating real issues and observing network behavior. Developers and administrators can see what happens in the system during failures or code updates in a controlled environment. The network must be pre-configured and deployed, and necessary metrics must be set up.<\/p>\n

<\/p>\n

Gremlin is a convenient tool for architects, DevOps, and security professionals, providing a universal solution for testing any ready and deployed distributed systems, including blockchains.<\/p>\n

<\/p>\n

\"A<\/p>\n

<\/p>\n

Hyperledger Caliper is a much more specialized solution Hyperledger Caliper<\/a><\/noindex>. Currently, Caliper supports several blockchains from the Hyperledger family (Fabric, Sawtooth, Iroha, Burrow, Besu), as well as Ethereum and the FISCO BCOS network. <\/p>\n

<\/p>\n

With Caliper, you can define the topology of a blockchain network and the contracts for testing, as well as describe the configuration of the node. Blockchain nodes are deployed in Docker containers on a single machine. You can then select the required test configurations<\/a><\/noindex> and receive a report file with test results after the run. You can learn about the full list of Caliper metrics and the benchmarking approach here Hyperledger Blockchain Performance Metrics<\/a><\/noindex>, this is a great article if you're interested in the topic of blockchain benchmarking. Additionally, metric collection can be configured to a separate Prometheus\/Grafana setup. <\/p>\n

<\/p>\n

Hyperledger Caliper is a tool aimed at developers and system architects, as it provides repeatability of tests and automation of testing and benchmarking. It is used when developing the core of blockchains: consensus algorithms, virtual machines for processing smart contracts, peer-to-peer layers, and other system mechanisms.<\/p>\n

<\/p>\n

\"A<\/p>\n

<\/p>\n

MixBytes Tank<\/a><\/noindex> \u2014 a tool that emerged during the development of consensus algorithms and finality for EOS-based networks and testing parachains based on Parity Substrate (Polkadot). Functionally, it is similar to Hyperledger Caliper, as it allows collecting important metrics from nodes of any distributed system and client machines running test scripts. <\/p>\n

<\/p>\n

MixBytes Tank utilizes several cloud services (Digital Ocean, Google Cloud Engine, etc.) that can launch multiple nodes, carry out preliminary configuration procedures, simultaneously execute several benchmarks on different machines, collect necessary metrics, and automatically shut down the network. <\/p>\n

<\/p>\n

MixBytes Tank allows for economical resource usage on cloud servers by automatically shutting down unnecessary resources after tests are conducted. Another distinguishing feature is the use of the Molecule package, which enables developers to locally test the deployment of the required blockchain.<\/p>\n

<\/p>\n

MixBytes Tank helps identify bottlenecks and errors in algorithms at early stages that arise in real networks with a large number of geographically distributed servers and clients. The tank will help you understand what will happen on the nodes if clients send transactions with a specified tps under highly repeatable conditions and with a real number of nodes spread across different continents, if required.<\/p>\n

<\/p>\n

\"A<\/p>\n

<\/p>\n

Whiteblock Genesis is a platform for testing Ethereum-based blockchains. This tool has a sufficiently broad functionality: it allows you to launch a network, create the required number of accounts, deploy the required number of clients, configure network topology, specify bandwidth and packet loss parameters, and run the test. <\/p>\n

<\/p>\n

Whiteblock Genesis provides its own resources for conducting tests. Developers just need to specify the test parameters, execute them using the ready-made API, and receive results using a convenient dashboard.<\/p>\n

<\/p>\n

Whiteblock Genesis allows for configuring a detailed test that the platform will automatically conduct upon every significant code change. This will enable early detection of errors and immediate assessment of the impact of changes on key network parameters, such as transaction speed and resources consumed by nodes. <\/p>\n

<\/p>\n

Madt<\/h1>\n

<\/p>\n

Another interesting young product for testing distributed systems \u2014 madt<\/a><\/noindex>. It is written in Python and allows you to create the necessary network topology and the required number of servers and clients using a simple configuration script (an example<\/a><\/noindex>). After that, the service deploys the network in several docker containers and raises a web interface where you can observe messages from the servers and clients of the network. Madt can be used for blockchain testing \u2014 in the project repository, there is a test for a p2p network based on the Kademlia protocol, where delays in data delivery to nodes are gradually increased and the status of this data is checked.<\/p>\n

<\/p>\n

Madt has appeared quite recently, but considering its very flexible architecture, it can evolve into a functional product. <\/p>\n

<\/p>\n

Other solutions<\/h1>\n

<\/p>\n

Almost any testing of the systemic part of blockchains requires running preliminary scenarios, preparing accounts and conditions for the test (this may involve testing consensus errors that can generate numerous chain forks, handling hard fork scenarios, changing system parameters, etc.). All these manipulations are performed differently in various blockchains, so teams find it easier to gradually adapt testing and benchmarking of the product to internal CI\/CD and use their own developments, which gradually become more complex as the functionality of the blockchain develops.<\/p>\n

<\/p>\n

However, using ready-made solutions can significantly reduce the time these teams spend on testing, so I believe this software will actively develop in the coming years.<\/p>\n

<\/p>\n

Conclusion<\/h1>\n

<\/p>\n

In concluding this brief overview, I will list several important characteristics of tools for testing blockchains:<\/p>\n

<\/p>\n