{"id":38856,"date":"2019-10-31T22:26:22","date_gmt":"2019-10-31T19:26:22","guid":{"rendered":"https:\/\/prohoster.info\/blog\/publichnyj-test-reshenie-dlya-privatnosti-i-masshtabiruemosti-v-efiriume\/"},"modified":"2019-10-31T22:26:22","modified_gmt":"2019-10-31T19:26:22","slug":"publichnyj-test-reshenie-dlya-privatnosti-i-masshtabiruemosti-v-efiriume","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/publichnyj-test-reshenie-dlya-privatnosti-i-masshtabiruemosti-v-efiriume","title":{"rendered":"Public Test: A Solution for Privacy and Scalability on Ethereum","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><b>Blockchain<\/b> \u2014 an innovative technology promising to enhance many areas of human life. It transfers real processes and products into the digital space, ensures speed and reliability of financial operations, reduces their cost, and also allows for the creation of modern DAPP applications using smart contracts in decentralized networks.<\/p>\n<p>Given the numerous advantages and diverse applications of blockchain, it may seem strange that this promising technology has not yet penetrated all industries. The problem is that modern decentralized blockchains lack scalability. Ethereum processes about 20 transactions per second, which is not sufficient to meet the needs of today's dynamic businesses. At the same time, companies using blockchain technology hesitate to abandon Ethereum due to its high level of protection against hacks and network failures.<\/p>\n<p>To ensure decentralization, security, and scalability in blockchain, thereby addressing the Scalability Trilemma, the development team <noindex><a rel=\"nofollow\" href=\"https:\/\/opporty.com\/\">Opporty<\/a><\/noindex> created Plasma Cash \u2014 a sidechain consisting of a smart contract and a private network based on Node.js, periodically transmitting its state to the root chain (Ethereum).<\/p>\n<p><img decoding=\"async\" alt=\"Public Test: A Solution for Privacy and Scalability on Ethereum\" src=\"\/wp-content\/uploads\/2019\/10\/02cc45df3474179d0936c2a86fb7dee3.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2>Key processes in Plasma Cash<\/h2>\n<p>\n<b>1. <\/b>A user calls the smart contract function `deposit`, passing the amount in ETH that they wish to place in the Plasma Cash token. The smart contract function creates the token and generates an event regarding this.<\/p>\n<p><b>2. <\/b>Plasma Cash nodes, subscribed to events from the smart contract, receive the event of the deposit creation and add the token creation transaction to the pool.<\/p>\n<p><b>3. <\/b>Periodically, special Plasma Cash nodes take all transactions from the pool (up to 1 million) and form a block from them, calculating the Merkle tree and, accordingly, the hash. This block is sent to other nodes for verification. The nodes check whether the Merkle hash is valid, whether the transactions are valid (for example, whether the sender of the token is its owner). After block verification, the node calls the `submitBlock` function of the smart contract, which saves the number and Merkle hash of the block to the root chain. The smart contract generates an event about the successful addition of the block. Transactions are removed from the pool. <\/p>\n<p><b>4. <\/b>Nodes that received the block submission event begin to apply the transactions that were added to the block.<\/p>\n<p><b>5. <\/b>At some point, the owner (or non-owner) of the token wants to withdraw it from Plasma Cash. To do this, they call the `startExit` function, passing it the information about the last 2 transactions for the token, which confirm that they are indeed the owner of the token. The smart contract, using the Merkle hash, checks for the presence of the transactions in the blocks and sends the token for withdrawal, which will occur in two weeks.<\/p>\n<p><b>6. <\/b>If the token withdrawal operation occurred with violations (the token was spent after the withdrawal procedure began or the token was already someone else's before withdrawal), the owner of the token can dispute the withdrawal within two weeks.<\/p>\n<p><img decoding=\"async\" alt=\"Public Test: A Solution for Privacy and Scalability on Ethereum\" src=\"\/wp-content\/uploads\/2019\/10\/0626ca6b010eb847175dbfb0b6d7b357.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h2>Privacy is achieved in two ways<\/h2>\n<p>\n<b>1. <\/b>The root chain has no knowledge of the transactions that are formed and transmitted within the child chain. Public information remains about who deposited and withdrew ETH to\/from Plasma Cash.<\/p>\n<p><b>2. <\/b>The child chain allows for anonymous transactions using zk-SNARKs.<\/p>\n<h2>Technology stack<\/h2>\n<p><\/p>\n<ul>\n<li>NodeJS<\/li>\n<li>Redis<\/li>\n<li>Ethereum<\/li>\n<li>Solid<\/li>\n<\/ul>\n<p><\/p>\n<h2>Testing <\/h2>\n<p>\nWhile developing Plasma Cash, we tested the system's speed and obtained the following results:<\/p>\n<ul>\n<li>up to 35,000 transactions per second are added to the pool;<\/li>\n<li>up to 1,000,000 transactions can be stored in a block.<\/li>\n<\/ul>\n<p>\nThe tests were conducted on the following 3 servers:<\/p>\n<p><i>1. Intel Core i7-6700 Quad-Core Skylake incl. NVMe SSD \u2014 512 GB, 64 GB DDR4 RAM<\/i><br \/>\n 3 validating Plasma Cash nodes were launched.<\/p>\n<p><i>2. AMD Ryzen 7 1700X Octa-Core \"Summit Ridge\" (Zen), SATA SSD \u2014 500 GB, 64 GB DDR4 RAM<\/i><br \/>\n A Ropsten testnet ETH node was launched.<br \/>\n 3 validating Plasma Cash nodes were launched.<\/p>\n<p><i>3. Intel Core i9-9900K Octa-Core incl. NVMe SSD \u2014 1 TB, 64 GB DDR4 RAM<\/i><br \/>\n 1 submission Plasma Cash node was launched.<br \/>\n 3 validating Plasma Cash nodes were launched.<br \/>\n A test was initiated to add transactions to the Plasma Cash network.<\/p>\n<p><b>Total: <\/b>10 Plasma Cash nodes in a private network.<\/p>\n<h3>Test 1<\/h3>\n<p>\nThere is a limit of 1 million transactions per block. Therefore, 1 million transactions are spread across 2 blocks (since the system manages to take a portion of the transactions and submit them while they are being sent).<\/p>\n<p><center><div class=\"youtube-placeholder\" data-id=\"pKwqyGkEgdQ\" onclick=\"loadVideo(this)\">\r\n        <img decoding=\"async\" src=\"https:\/\/img.youtube.com\/vi\/pKwqyGkEgdQ\/hqdefault.jpg\" alt=\"Play video\" loading=\"lazy\" width=\"480\" height=\"360\" style=\"width:100%;height:auto;\">\r\n        <div class=\"play-button\"><\/div>\r\n    <\/div><\/center><br \/>\nInitial state: last block #7; 1 million transactions and tokens are stored in the database.<\/p>\n<p>00:00 \u2014 script for transaction generation started<br \/>\n01:37 \u2014 1 million transactions created and sending to node began<br \/>\n01:46 \u2014 the submitting node took 240k transactions from the pool and is forming block #8. We also see that 320k transactions are added to the pool in 10 seconds<br \/>\n01:58 \u2014 block #8 signed and sent for validation<br \/>\n02:03 \u2014 Block #8 has been validated, and the `submitBlock` function of the smart contract has been called with the Merkle hash and block number.<br \/>\n02:10 \u2014 The demo script has finished running, which sent 1 million transactions in 32 seconds.<br \/>\n02:33 \u2014 Nodes began receiving information that block #8 was added to the root chain and started processing 240k transactions.<br \/>\n02:40 \u2014 240k transactions that were already in block #8 have been removed from the pool.<br \/>\n02:56 \u2014 The submit node took the remaining 760k transactions from the pool and started calculating the Merkle hash and signing block #9.<br \/>\n03:20 \u2014 All nodes contain 1 million 240k transactions and tokens.<br \/>\n03:35 \u2014 Block #9 has been signed and is being sent for validation to other nodes. <br \/>\n03:41 \u2014 A network error occurred.<br \/>\n04:40 \u2014 Validation of block #9 has timed out.<br \/>\n04:54 \u2014 The submit node took the remaining 760k transactions from the pool and started calculating the Merkle hash and signing block #9.<br \/>\n05:32 \u2014 Block #9 has been signed and is being sent for validation to other nodes.<br \/>\n05:53 \u2014 Block #9 has been validated and sent to the root chain.<br \/>\n06:17 \u2014 Nodes began receiving information that block #9 was added to the root chain and started processing 760k transactions.<br \/>\n06:47 \u2014 The pool has been cleared of the transactions that were included in block #9.<br \/>\n09:06 \u2014 All nodes contain 2 million transactions and tokens.<\/p>\n<h3>Test 2<\/h3>\n<p>\nThere is a limit of 350k on the block. As a result, we have 3 blocks.<\/p>\n<p><center><div class=\"youtube-placeholder\" data-id=\"mpTfTPKYRIc\" onclick=\"loadVideo(this)\">\r\n        <img decoding=\"async\" src=\"https:\/\/img.youtube.com\/vi\/mpTfTPKYRIc\/hqdefault.jpg\" alt=\"Play video\" loading=\"lazy\" width=\"480\" height=\"360\" style=\"width:100%;height:auto;\">\r\n        <div class=\"play-button\"><\/div>\r\n    <\/div><\/center><br \/>\nInitial state: the last block is #9; 2 million transactions and tokens are saved in the database.<\/p>\n<p>00:00 \u2014 The transaction generation script is already running.<br \/>\n00:44 \u2014 1 million transactions have been created and are being sent to the node.<br \/>\n00:56 \u2014 The submit node took 320k transactions from the pool and is forming block #10. We also see that 320k transactions are being added to the pool in 10 seconds.<br \/>\n01:12 \u2014 Block #10 has been signed and is being sent to other nodes for validation.<br \/>\n01:18 \u2014 The demo script has finished, which sent 1 million transactions in 34 seconds.<br \/>\n01:20 \u2014 Block #10 has been validated and sent to the root chain. <br \/>\n01:51 \u2014 All nodes received information from the root chain that block #10 has been added and begin applying 320k transactions.<br \/>\n02:01 \u2014 The pool has cleared 320k transactions that were added to block #10.<br \/>\n02:15 \u2014 The submit node took 350k transactions from the pool and is forming block #11.<br \/>\n02:34 \u2014 Block #11 has been signed and is being sent to other nodes for validation.<br \/>\n02:51 \u2014 Block #11 has been validated and sent to the root chain. <br \/>\n02:55 \u2014 The last node has executed the transactions from block #10.<br \/>\n10:59 \u2014 the transaction with submission of block #9 was executed very slowly in the root chain, but it has been completed, and all nodes received the information and began processing 350k transactions<br \/>\n11:05 \u2014 the pool was cleared of 320k transactions that were added to block #11<br \/>\n12:10 \u2014 all nodes contain 1 million 670k transactions and tokens<br \/>\n12:17 \u2014 the submission node took 330k transactions from the pool and is forming block #12<br \/>\n12:32 \u2014 block #12 has been signed and sent to other nodes for validation<br \/>\n12:39 \u2014 block #12 has been validated and sent to the root chain <br \/>\n13:44 \u2014 all nodes have received information from the root chain that block #12 has been added and begin to apply 330k transactions<br \/>\n14:50 \u2014 all nodes contain 2 million transactions and tokens<\/p>\n<h3>Test 3<\/h3>\n<p>\nOn the first and second servers, one validating node was replaced with a submission node. <\/p>\n<p><center><div class=\"youtube-placeholder\" data-id=\"w5QHab3heIc\" onclick=\"loadVideo(this)\">\r\n        <img decoding=\"async\" src=\"https:\/\/img.youtube.com\/vi\/w5QHab3heIc\/hqdefault.jpg\" alt=\"Play video\" loading=\"lazy\" width=\"480\" height=\"360\" style=\"width:100%;height:auto;\">\r\n        <div class=\"play-button\"><\/div>\r\n    <\/div><\/center><br \/>\nInitial state: last block #84; no transactions and tokens are stored in the database<\/p>\n<p>00:00 \u2014 3 scripts have been launched that generate and send 1 million transactions each<br \/>\n01:38 \u2014 1 million transactions have been created and the sending to submission node #3 has started<br \/>\n01:50 \u2014 submission node #3 took 330k transactions from the pool and is forming block #85 (f21). We also see that 350k transactions are being added to the pool in 10 seconds<br \/>\n01:53 \u2014 1 million transactions have been created and the sending to submission node #1 has started<br \/>\n01:50 \u2014 submission node #3 took 330k transactions from the pool and is forming block #85 (f21). We also see that 350k transactions are being added to the pool in 10 seconds<br \/>\n02:01 \u2014 submission node #1 took 250k transactions from the pool and is forming block #85 (65e)<br \/>\n02:06 \u2014 block #85 (f21) has been signed and sent to other nodes for validation<br \/>\n02:08 \u2014 the demo script of server #3 has finished working, which sent 1 million transactions in 30 seconds<br \/>\n02:14 \u2014 block #85 (f21) has been validated and sent to the root chain <br \/>\n02:19 \u2014 block #85 (65e) has been signed and sent to other nodes for validation<br \/>\n02:22 \u2014 1 million transactions have been created and the sending to submission node #2 has started<br \/>\n02:27 \u2014 block #85 (65e) has been validated and sent to the root chain <br \/>\n02:29 \u2014 submission node #2 took 111855 transactions from the pool and is forming block #85 (256).<br \/>\n02:36 \u2014 block #85 (256) has been signed and sent to other nodes for validation<br \/>\n02:36 \u2014 the demo script of server #1 has finished working, which sent 1 million transactions in 42.5 seconds<br \/>\n02:38 \u2014 block #85 (256) has been validated and sent to the root chain<br \/>\n03:08 \u2014 the demo script of server #2 has finished working, which sent 1 million transactions in 47 seconds <br \/>\n03:38 \u2014 all nodes received information from the root chain that blocks #85 (f21), #86 (65e), #87 (256) have been added and begin to apply 330k, 250k, 111855 transactions<br \/>\n03:49 \u2014 the pool cleared 330k, 250k, 111855 transactions that were added to blocks #85 (f21), #86(65e), #87(256)<br \/>\n03:59 \u2014 submit node #1 took 888145 transactions from the pool and is forming block #88 (214), submit node #2 took 750k transactions from the pool and is forming block #88 (50a), submit node #3 took 670k transactions from the pool and is forming block #88 (d3b)<br \/>\n04:44 \u2014 block #88 (d3b) has been signed and sent to other nodes for validation<br \/>\n04:58 \u2014 block #88 (214) has been signed and sent to other nodes for validation<br \/>\n05:11 \u2014 block #88 (50a) has been signed and sent to other nodes for validation<br \/>\n05:11 \u2014 block #85 (d3b) has been validated and sent to the root chain <br \/>\n05:36 \u2014 block #85 (214) has been validated and sent to the root chain <br \/>\n05:43 \u2014 all nodes received information from the root chain that blocks #88 (d3b), #89(214) have been added, and they start applying 670k, 750k transactions<br \/>\n06:50 \u2014 due to a connection failure, block #85 (50a) was not validated<br \/>\n06:55 \u2014 submit node #2 took 888145 transactions from the pool and is forming block #90 (50a)<br \/>\n08:14 \u2014 block #90 (50a) has been signed and sent to other nodes for validation<br \/>\n09:04 \u2014 block #90 (50a) has been validated and sent to the root chain <br \/>\n11:23 \u2014 all nodes received information from the root chain that block #90 (50a) has been added, and they start applying 888145 transactions. Meanwhile, server #3 has long applied transactions from blocks #88 (d3b), #89(214)<br \/>\n12:11 \u2014 all pools are empty<br \/>\n13:41 \u2014 all nodes of server #3 contain 3 million transactions and tokens<br \/>\n14:35 \u2014 all nodes of server #1 contain 3 million transactions and tokens<br \/>\n19:24 \u2014 all nodes of server #2 contain 3 million transactions and tokens <\/p>\n<h2>Obstacles<\/h2>\n<p>\nDuring the development of Plasma Cash, we encountered the following problems, which we gradually resolved and are still resolving:<\/p>\n<p><b>1.<\/b> Conflict of interaction between various functions of the system. For example, the function of adding transactions to the pool blocked the work of submitting and validating blocks, and vice versa, which led to a drop in speed.<\/p>\n<p><b>2. <\/b>It was not immediately clear how to send a huge number of transactions while minimizing data transfer costs.<\/p>\n<p><b>3. <\/b>It was unclear how and where to store data to achieve high results.<\/p>\n<p><b>4. <\/b>It was unclear how to organize the network among nodes, as the size of a block with 1 million transactions occupies about 100 MB.<\/p>\n<p><b>5.<\/b> Working in single-thread mode breaks the connection between nodes during long computations (for example, constructing the Merkle tree and calculating its hash).<\/p>\n<h2>How did we handle all of this?<\/h2>\n<p>\nThe first version of the Plasma Cash node was a sort of hybrid that could do everything at once: accept transactions, submit and validate blocks, and provide an API for data access. Since NodeJS is originally single-threaded, the heavy function of calculating the Merkle tree blocked the transaction addition function. We saw two possible solutions to this problem:<\/p>\n<p><b>1. <\/b>Run multiple NodeJS processes, each performing specific functions.<\/p>\n<p><b>2. <\/b>Use worker_threads and move part of the code execution to threads.<\/p>\n<p>In the end, we utilized both options simultaneously: we logically divided a single node into three parts that can operate separately but at the same time synchronously.<\/p>\n<p><b>1.<\/b> A submit node that accepts transactions into the pool and handles block creation.<\/p>\n<p><b>2.<\/b> A validating node that checks the validity of nodes.<\/p>\n<p><b>3. <\/b>API node \u2014 provides an API for data access.<\/p>\n<p>At the same time, each node can be connected to via a unix socket using the CLI.<\/p>\n<p>Heavy operations, such as calculating the Merkle tree, were moved to a separate thread.<\/p>\n<p>Thus, we achieved normal operation of all Plasma Cash functions simultaneously and without failures.<\/p>\n<p>As soon as the system was functionally operational, we began testing the speed, and unfortunately, we received unsatisfactory results: 5,000 transactions per second and up to 50,000 transactions per block. We had to investigate what was incorrectly implemented.<\/p>\n<p>To start, we began testing the communication mechanism with Plasma Cash to find out the system's peak capability. Previously, we noted that the Plasma Cash node provides a unix socket interface. Initially, it was text-based. JSON objects were transmitted using `JSON.parse()` and `JSON.stringify()`. <\/p>\n<pre><code class=\"plaintext\">{\"action\": \"sendTransaction\",\"payload\":{\"prevHash\": \"0x8a88cc4217745fd0b4eb161f6923235da10593be66b841d47da86b9cd95d93e0\",\"prevBlock\": 41,\"tokenId\": \"57570139642005649136210751546585740989890521125187435281313126554130572876445\",\"newOwner\": \"0x200eabe5b26e547446ae5821622892291632d4f4\",\"type\": \"pay\",\"data\": \"\",\"signature\": \"0xd1107d0c6df15e01e168e631a386363c72206cb75b233f8f3cf883134854967e1cd9b3306cc5c0ce58f0a7397ae9b2487501b56695fe3a3c90ec0f61c7ea4a721c\"}}\n<\/code><\/pre>\n<p>\nWe measured the speed of sending such objects and obtained ~130k per second. We tried to replace the standard JSON handling functions, but performance did not improve. It seems that the V8 engine is well optimized for these operations.<\/p>\n<p>Working with transactions, tokens, and blocks was done through classes. When creating such classes, the performance dropped by half, indicating that OOP is not suitable for us. We had to rewrite everything with a purely functional approach.<\/p>\n<h2>Database entry<\/h2>\n<p>\nInitially, Redis was chosen for data storage as one of the most efficient solutions that meets our requirements: a key-value store, working with hash tables and sets. We ran redis-benchmark and achieved ~80k operations per second in 1 pipelining mode.<\/p>\n<p>For high performance, we configured Redis more finely: <\/p>\n<ul>\n<li>We set up a UNIX socket connection.<\/li>\n<li>We disabled state saving to disk (for reliability, a replica can be configured and saving can be done in a separate Redis).<\/li>\n<\/ul>\n<p>\nIn Redis, a pool is a hash table, as we need the ability to fetch all transactions in one request and delete transactions individually. We tried using a regular list, but it worked slower when unloading the entire list. <\/p>\n<p>With the standard NodeJS Redis library, we achieved a performance of 18k transactions per second. The speed dropped by 9 times. <\/p>\n<p>Since the benchmark showed us capabilities clearly 5 times greater, we began optimizing. We switched the library to ioredis and achieved a performance of 25k per second. We added transactions individually using the `hset` command. Thus, we generated many requests to Redis. The idea arose to combine transactions into batches and send them with one command, `hmset`. The result was 32k per second. <\/p>\n<p>For several reasons, which we will describe below, we work with data using `Buffer`, and as it turned out, converting it to text (`buffer.toString('hex')`) before writing can provide additional performance. Thus, the speed has been increased to 35k per second. For now, we have decided to pause further optimization.<\/p>\n<p>We had to switch to a binary protocol because:<\/p>\n<p><b>1. <\/b>The system often computes hashes, signatures, etc., and it needs data in `Buffer.<\/p>\n<p><b>2.<\/b> When transmitting between services, binary data weighs less than text. For example, sending a block with 1 million transactions, the data in text can take up more than 300 megabytes.<\/p>\n<p><b>3.<\/b> Constant data conversion affects performance.<\/p>\n<p>Therefore, we based our work on our own binary protocol for storing and transmitting data, developed using the remarkable `binary-data` library.<\/p>\n<p>As a result, we created the following data structures:<\/p>\n<h3> \u2014 Transaction<\/h3>\n<p><\/p>\n<pre><code class=\"plaintext\">  ```json\n  {\n    prevHash: BD.types.buffer(20),\n    prevBlock: BD.types.uint24le,\n    tokenId: BD.types.string(null),\n    type: BD.types.uint8,\n    newOwner: BD.types.buffer(20),\n    dataLength: BD.types.uint24le,\n    data: BD.types.buffer(({current}) =&gt; current.dataLength),\n    signature: BD.types.buffer(65),\n    hash: BD.types.buffer(32),\n    blockNumber: BD.types.uint24le,\n    timestamp: BD.types.uint48le,\n  }\n  ```\n<\/code><\/pre>\n<p><\/p>\n<h3> \u2014 Token<\/h3>\n<p><\/p>\n<pre><code class=\"plaintext\">  ```json\n  {\n    id: BD.types.string(null),\n    owner: BD.types.buffer(20),\n    block: BD.types.uint24le,\n    amount: BD.types.string(null),\n  }\n  ```\n<\/code><\/pre>\n<p><\/p>\n<h3> \u2014 Block<\/h3>\n<p><\/p>\n<pre><code class=\"plaintext\">  ```json\n  {\n    number: BD.types.uint24le,\n    merkleRootHash: BD.types.buffer(32),\n    signature: BD.types.buffer(65),\n    countTx: BD.types.uint24le,\n    transactions: BD.types.array(Transaction.Protocol, ({current}) =&gt; current.countTx),\n    timestamp: BD.types.uint48le,\n  }\n  ```\n<\/code><\/pre>\n<p>\nWith the ordinary commands `BD.encode(block, Protocol).slice();` and ` BD.decode(buffer, Protocol)`, we transform the data into `Buffer` for storage in Redis or sending to another node and extract the data back.<\/p>\n<p>We also have 2 binary protocols for data transmission between services:<\/p>\n<p><i> \u2014 Protocol for interacting with Plasma Node via unix socket<\/i><\/p>\n<pre><code class=\"plaintext\">  ```json\n  {\n    type: BD.types.uint8,\n    messageId: BD.types.uint24le,\n    error: BD.types.uint8,\n    length: BD.types.uint24le,\n    payload: BD.types.buffer(({node}) =&gt; node.length)\n  }\n  ```\n<\/code><\/pre>\n<p>\n where:<\/p>\n<ul>\n<li> <b>`type`<\/b> \u2014 the action to be performed, for example, 1 \u2014 sendTransaction, 2 \u2014 getTransaction;<\/li>\n<li> <b>`payload`<\/b> \u2014 data to be passed to the corresponding function;<\/li>\n<li> <b>`messageId`<\/b> \u2014 message id for identifying the response. <\/li>\n<\/ul>\n<p>\n<i> \u2014 Protocol for interaction between nodes<\/i><\/p>\n<pre><code class=\"plaintext\">  ```json\n  {\n    code: BD.types.uint8,\n    versionProtocol: BD.types.uint24le,\n    seq: BD.types.uint8,\n    countChunk: BD.types.uint24le,\n    chunkNumber: BD.types.uint24le,\n    length: BD.types.uint24le,\n    payload: BD.types.buffer(({node}) =&gt; node.length)\n  }\n  ```\n<\/code><\/pre>\n<p>\n where:<\/p>\n<ul>\n<li> <b>`code`<\/b> \u2014 message code, for example 6 \u2014 PREPARE_NEW_BLOCK, 7 \u2014 BLOCK_VALID, 8 \u2014 BLOCK_COMMIT;<\/li>\n<li> <b>`versionProtocol`<\/b> \u2014 protocol version, since nodes with different versions can be running in the network and behave differently;<\/li>\n<li> <b>`seq`<\/b> \u2014 message identifier;<\/li>\n<li> <b>`countChunk`<\/b> and <b>`chunkNumber`<\/b> necessary for splitting large messages;<\/li>\n<li> <b>`length`<\/b> and <b>`payload`<\/b> length and the data itself.<\/li>\n<\/ul>\n<p>\nSince we pre-typed the data, the final system works much faster than the `rlp` library from Ethereum. Unfortunately, we have not yet been able to discard it, as we need to finalize the smart contract, which we plan to do in the future.<\/p>\n<p>If we manage to achieve speed <b>35 000<\/b> transactions per second, we also need to process them in an optimal time. Since the approximate time to form a block takes 30 seconds, we need to include in the block <b>1 000 000<\/b> transactions, which means transferring more than <b>100<\/b> MB of data. <\/p>\n<p>Initially, we used the `ethereumjs-devp2p` library for node communication, but it could not handle such a volume of data. As a result, we utilized the `ws` library and configured the transfer of binary data over websockets. Of course, we also faced issues when transferring large data packets, but we split them into chunks and now these problems no longer exist.<\/p>\n<p>The construction of the Merkle tree and calculating the hash <b>1 000 000<\/b> of transactions takes about<b> 10<\/b> seconds of continuous computation. During this time, the connection to all nodes tends to drop. It was decided to move this computation to a separate thread.<\/p>\n<h2>Conclusions:<\/h2>\n<p>\nIn fact, our conclusions are not new, but for some reason, many specialists forget about them during development. <\/p>\n<ul>\n<li>Using Functional Programming instead of Object-Oriented Programming increases performance.<\/li>\n<li>Monolith is worse than a service-oriented architecture for a productive system on NodeJS.<\/li>\n<li>Using `worker_threads` for heavy computations improves system responsiveness, especially when dealing with I\/O operations.<\/li>\n<li>Unix sockets are more stable and faster than HTTP requests.<\/li>\n<li>If you need to quickly transfer large amounts of data over the network, it's better to use websockets and send binary data split into chunks, which can be retransmitted if they do not reach their destination, and then reassembled into a single message.<\/li>\n<\/ul>\n<p>\nWe invite you to visit <b>GitHub<\/b> of the project: <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/opporty-com\/Plasma-Cash\/tree\/new-version\">https:\/\/github.com\/opporty-com\/Plasma-Cash\/tree\/new-version<\/a><\/noindex><\/p>\n<p>The article was co-authored by <i>Alexander Nashiwan<\/i>, senior developer at <noindex><a rel=\"nofollow\" href=\"https:\/\/clever-solution.com\/\">Clever Solution Inc<\/a><\/noindex>.<br \/>\n<br \/>Source: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/471096\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0411\u043b\u043e\u043a\u0447\u0435\u0439\u043d \u2014 \u0438\u043d\u043d\u043e\u0432\u0430\u0446\u0438\u043e\u043d\u043d\u0430\u044f \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u044f, \u043e\u0431\u0435\u0449\u0430\u044e\u0449\u0430\u044f \u0443\u043b\u0443\u0447\u0448\u0438\u0442\u044c \u043c\u043d\u043e\u0433\u0438\u0435 \u0441\u0444\u0435\u0440\u044b \u0447\u0435\u043b\u043e\u0432\u0435\u0447\u0435\u0441\u043a\u043e\u0439 \u0436\u0438\u0437\u043d\u0438. \u041e\u043d\u0430 \u043f\u0435\u0440\u0435\u043d\u043e\u0441\u0438\u0442 \u0440\u0435\u0430\u043b\u044c\u043d\u044b\u0435 \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u044b \u0438 \u043f\u0440\u043e\u0434\u0443\u043a\u0442\u044b \u0432 \u0446\u0438\u0444\u0440\u043e\u0432\u043e\u0435 \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u043e, \u043e\u0431\u0435\u0441\u043f\u0435\u0447\u0438\u0432\u0430\u0435\u0442 \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u0438 \u043d\u0430\u0434\u0435\u0436\u043d\u043e\u0441\u0442\u044c \u0444\u0438\u043d\u0430\u043d\u0441\u043e\u0432\u044b\u0445 \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0439, \u0441\u043d\u0438\u0436\u0430\u0435\u0442 \u0438\u0445 \u0441\u0442\u043e\u0438\u043c\u043e\u0441\u0442\u044c, \u0430 \u0442\u0430\u043a\u0436\u0435 \u043f\u043e\u0437\u0432\u043e\u043b\u044f\u0435\u0442 \u0441\u043e\u0437\u0434\u0430\u0432\u0430\u0442\u044c \u0441\u043e\u0432\u0440\u0435\u043c\u0435\u043d\u043d\u044b\u0435 DAPP \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u044f \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u0438\u043d\u0442\u0435\u043b\u043b\u0435\u043a\u0442\u0443\u0430\u043b\u044c\u043d\u044b\u0445 \u043a\u043e\u043d\u0442\u0440\u0430\u043a\u0442\u043e\u0432 \u0432 \u0434\u0435\u0446\u0435\u043d\u0442\u0440\u0430\u043b\u0438\u0437\u043e\u0432\u0430\u043d\u043d\u044b\u0445 \u0441\u0435\u0442\u044f\u0445. \u0423\u0447\u0438\u0442\u044b\u0432\u0430\u044f \u043c\u043d\u043e\u0433\u043e\u0447\u0438\u0441\u043b\u0435\u043d\u043d\u044b\u0435 \u043f\u0440\u0435\u0438\u043c\u0443\u0449\u0435\u0441\u0442\u0432\u0430 \u0438 \u0440\u0430\u0437\u043d\u043e\u043e\u0431\u0440\u0430\u0437\u043d\u044b\u0435 \u0441\u0444\u0435\u0440\u044b \u043f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0438\u044f \u0431\u043b\u043e\u043a\u0447\u0435\u0439\u043d, \u043c\u043e\u0436\u0435\u0442 \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u044c\u0441\u044f \u0441\u0442\u0440\u0430\u043d\u043d\u044b\u043c, \u0447\u0442\u043e \u044d\u0442\u0430 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":29146,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-38856","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0411\u043b\u043e\u043a\u0447\u0435\u0439\u043d \u2014 \u0438\u043d\u043d\u043e\u0432\u0430\u0446\u0438\u043e\u043d\u043d\u0430\u044f \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u044f, \u043e\u0431\u0435\u0449\u0430\u044e\u0449\u0430\u044f \u0443\u043b\u0443\u0447\u0448\u0438\u0442\u044c \u043c\u043d\u043e\u0433\u0438\u0435 \u0441\u0444\u0435\u0440\u044b \u0447\u0435\u043b\u043e\u0432\u0435\u0447\u0435\u0441\u043a\u043e\u0439 \u0436\u0438\u0437\u043d\u0438.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/publichnyj-test-reshenie-dlya-privatnosti-i-masshtabiruemosti-v-efiriume\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041f\u0443\u0431\u043b\u0438\u0447\u043d\u044b\u0439 \u0442\u0435\u0441\u0442: \u0440\u0435\u0448\u0435\u043d\u0438\u0435 \u0434\u043b\u044f \u043f\u0440\u0438\u0432\u0430\u0442\u043d\u043e\u0441\u0442\u0438 \u0438 \u043c\u0430\u0441\u0448\u0442\u0430\u0431\u0438\u0440\u0443\u0435\u043c\u043e\u0441\u0442\u0438 \u0432 \u042d\u0444\u0438\u0440\u0438\u0443\u043c\u0435 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0411\u043b\u043e\u043a\u0447\u0435\u0439\u043d \u2014 \u0438\u043d\u043d\u043e\u0432\u0430\u0446\u0438\u043e\u043d\u043d\u0430\u044f \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u044f, \u043e\u0431\u0435\u0449\u0430\u044e\u0449\u0430\u044f \u0443\u043b\u0443\u0447\u0448\u0438\u0442\u044c \u043c\u043d\u043e\u0433\u0438\u0435 \u0441\u0444\u0435\u0440\u044b \u0447\u0435\u043b\u043e\u0432\u0435\u0447\u0435\u0441\u043a\u043e\u0439 \u0436\u0438\u0437\u043d\u0438.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/publichnyj-test-reshenie-dlya-privatnosti-i-masshtabiruemosti-v-efiriume\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-10-31T19:26:22+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2019-10-31T19:26:22+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Public testing: a solution for privacy and scalability in Ethereum | ProHoster","description":"Blockchain is an innovative technology that promises to improve many areas of human life.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/publichnyj-test-reshenie-dlya-privatnosti-i-masshtabiruemosti-v-efiriume","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041f\u0443\u0431\u043b\u0438\u0447\u043d\u044b\u0439 \u0442\u0435\u0441\u0442: \u0440\u0435\u0448\u0435\u043d\u0438\u0435 \u0434\u043b\u044f \u043f\u0440\u0438\u0432\u0430\u0442\u043d\u043e\u0441\u0442\u0438 \u0438 \u043c\u0430\u0441\u0448\u0442\u0430\u0431\u0438\u0440\u0443\u0435\u043c\u043e\u0441\u0442\u0438 \u0432 \u042d\u0444\u0438\u0440\u0438\u0443\u043c\u0435 | ProHoster","og:description":"\u0411\u043b\u043e\u043a\u0447\u0435\u0439\u043d \u2014 \u0438\u043d\u043d\u043e\u0432\u0430\u0446\u0438\u043e\u043d\u043d\u0430\u044f \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u044f, \u043e\u0431\u0435\u0449\u0430\u044e\u0449\u0430\u044f \u0443\u043b\u0443\u0447\u0448\u0438\u0442\u044c \u043c\u043d\u043e\u0433\u0438\u0435 \u0441\u0444\u0435\u0440\u044b \u0447\u0435\u043b\u043e\u0432\u0435\u0447\u0435\u0441\u043a\u043e\u0439 \u0436\u0438\u0437\u043d\u0438.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/publichnyj-test-reshenie-dlya-privatnosti-i-masshtabiruemosti-v-efiriume","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:26:22+00:00","article:modified_time":"2019-10-31T19:26:22+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"38856","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-23 23:42:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 12:15:36","updated":"2026-01-23 23:42:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/38856","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=38856"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/38856\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/29146"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=38856"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=38856"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=38856"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}