Blockchain is an innovative technology that promises to improve many areas of human life. It brings real processes and products to the digital space, ensures the speed and reliability of financial transactions, reduces their cost, and also allows you to create modern DAPP applications using smart contracts in decentralized networks.
Given the many benefits and diverse applications of blockchain, it may seem strange that this promising technology has not yet made its way into every industry. The problem is that modern decentralized blockchains lack scalability. Ethereum processes about 20 transactions per second, which is not enough to meet the needs of today's dynamic business. At the same time, companies using blockchain technology are hesitant to abandon Ethereum due to its high degree of protection against hacking and network failures.
To ensure decentralization, security, and scalability on the blockchain, thus solving the Scalability Trilemma, the development team created Plasma Cash, a child chain consisting of a smart contract and a private network based on Node.js, periodically transferring its state to the root chain (Ethereum).
Key processes in Plasma Cash
1. The user calls the `deposit` function of the smart contract, passing in the amount in ETH that he wants to place in the Plasma Cash token. The smart contract function creates a token and generates an event about it.
2. Plasma Cash nodes subscribed to smart contract events receive a deposit creation event and add a token creation transaction to the pool.
3. Periodically, special Plasma Cash nodes take all transactions from the pool (up to 1 million) and form a block from them, calculate the Merkle tree and, accordingly, the hash. This block is sent to other nodes for verification. The nodes check whether the Merkle hash is valid, whether the transactions are valid (for example, whether the sender of the token is its owner). After verifying the block, the node calls the `submitBlock` function of the smart contract, which saves the number and Merkle hash of the block to the chain chain. The smart contract generates an event about the successful addition of a block. Transactions are removed from the pool.
4. Nodes that receive a block submit event start applying the transactions that were added to the block.
5. At some point, the owner (or non-owner) of the token wants to withdraw it from Plasma Cash. To do this, he calls the `startExit` function, passing to it information about the last 2 transactions on the token, which confirm that he is the owner of the token. The smart contract, using the Merkle hash, checks the presence of transactions in blocks and sends the token for withdrawal, which will occur in two weeks.
6. If the token withdrawal operation occurred with violations (the token was spent after the start of the withdrawal procedure or the token was already someone else's before the withdrawal), the owner of the token can refute the withdrawal within two weeks.
Privacy is achieved in two ways
1. The root chain knows nothing about the transactions that are formed and sent within the child chain. The information about who deposited and withdrawn ETH to/from Plasma Cash remains public.
2. The child chain allows anonymous transactions using zk-SNARKs.
Technology stack
- NodeJS
- Redis
- Etherium
- soild
The test is
While developing Plasma Cash, we tested the speed of the system and got the following results:
- up to 35 transactions per second are added to the pool;
- up to 1 transactions can be stored in a block.
The tests were carried out on the following 3 servers:
1. Intel Core i7-6700 Quad-Core Skylake incl. NVMe SSD - 512 GB, 64 GB DDR4 RAM
3 validating Plasma Cash nodes have been raised.
2. AMD Ryzen 7 1700X Octa-Core "Summit Ridge" (Zen), SATA SSD - 500 GB, 64 GB DDR4 RAM
A Ropsten testnet ETH node has been raised.
3 validating Plasma Cash nodes were raised.
3. Intel Core i9-9900K Octa-Core incl. NVMe SSD - 1 TB, 64 GB DDR4 RAM
1 Plasma Cash node submission was raised.
3 validating Plasma Cash nodes were raised.
A test was launched to add transactions to the Plasma Cash network.
Total: 10 Plasma Cash nodes in a private network.
Test 1
There is a limit of 1 million transactions in a block. Therefore, 1 million transactions fall into 2 blocks (since the system manages to take part of the transactions and submit them while they are being sent).
Initial state: last block #7; 1 million transactions and tokens are stored in the database.
00:00 — launch of the transaction generation script
01:37 — 1 million transactions have been created and sending to the node has begun
01:46 — the submit node took 240k transactions from the pool and forms block #8. We also see that 320k transactions are added to the pool in 10 seconds
01:58 — block #8 signed and sent for validation
02:03 — Block #8 is validated and the `submitBlock` function of the smart contract is called with the Merkle hash and block number
02:10 - demo script finished working, which sent 1 million transactions in 32 seconds
02:33 — nodes started receiving information that block #8 was added to the root chain, and began to execute 240k transactions
02:40 - 240k transactions were deleted from the pool, which are already in block # 8
02:56 — the submit node took the remaining 760k transactions from the pool and started calculating the Merkle hash and signing block #9
03:20 - all nodes contain 1 million 240k transactions and tokens
03:35 — block #9 is signed and sent for validation to other nodes
03:41 - A network error has occurred
04:40 — Waiting for block #9 validation stopped due to timeout
04:54 — the submit node took the remaining 760k transactions from the pool and started calculating the Merkle hash and signing block #9
05:32 — block #9 is signed and sent for validation to other nodes
05:53 - block #9 is validated and sent to the root chain
06:17 — nodes began to receive information that block #9 was added to the root chain and began to execute 760k transactions
06:47 - the pool was cleared of transactions that are in block # 9
09:06 — all nodes contain 2 million transactions and tokens
Test 2
There is a limit of 350k per block. As a result, we have 3 blocks.
Initial state: last block #9; 2 million transactions and tokens are stored in the database
00:00 — transaction generation script is already running
00:44 — 1 million transactions have been created and sending to the node has begun
00:56 — the submit node took 320k transactions from the pool and forms block #10. We also see that 320k transactions are added to the pool in 10 seconds
01:12 — block #10 is signed and sent to other nodes for validation
01:18 - demo script finished working, which sent 1 million transactions in 34 seconds
01:20 — block #10 validated and sent to the root chain
01:51 - all nodes received information from the root chain that block #10 has been added, and start applying 320k transactions
02:01 - the pool was cleared for 320k transactions that were added to block # 10
02:15 — the submit node took 350k transactions from the pool and forms block #11
02:34 — block #11 is signed and sent to other nodes for validation
02:51 — block #11 validated and sent to the root chain
02:55 — the last node completed transactions from block #10
10:59 - a transaction with the submission of block # 9 was performed in the root chain for a very long time, but it was completed and all nodes received information about this and began to execute 350k transactions
11:05 - the pool was cleared for 320k transactions that were added to block # 11
12:10 - all nodes contain 1 million 670k transactions and tokens
12:17 — the submit node took 330k transactions from the pool and forms block #12
12:32 — block #12 is signed and sent to other nodes for validation
12:39 — block #12 validated and sent to the root chain
13:44 - all nodes received information from the root chain that block #12 has been added and start applying 330k transactions
14:50 — all nodes contain 2 million transactions and tokens
Test 3
In the first and second servers, one validating node was replaced by a submit node.
Initial state: last block #84; 0 transactions and tokens are stored in the database
00:00 - 3 scripts are launched that generate and send 1 million transactions each
01:38 — 1 million transactions have been created and sending to submit node #3 has begun
01:50 — submit node #3 took 330k transactions from the pool and forms block #85 (f21). We also see that 350k transactions are added to the pool in 10 seconds
01:53 — 1 million transactions have been created and sending to submit node #1 has begun
01:50 — submit node #3 took 330k transactions from the pool and forms block #85 (f21). We also see that 350k transactions are added to the pool in 10 seconds
02:01 — submit node #1 took 250k transactions from the pool and forms block #85 (65e)
02:06 — block #85 (f21) is signed and sent to other nodes for validation
02:08 — server demo script #3 finished working, which sent 1 million transactions in 30 seconds
02:14 — block #85 (f21) validated and sent to the root chain
02:19 — block #85 (65e) signed and sent to other nodes for validation
02:22 — 1 million transactions have been created and sending to submit node #2 has begun
02:27 — block #85 (65e) validated and sent to the root chain
02:29 — submit node #2 took 111855 transactions from the pool and forms block #85 (256).
02:36 — block #85 (256) is signed and sent to other nodes for validation
02:36 — server demo script #1 finished working, which sent 1 million transactions in 42.5 seconds
02:38 — block #85 (256) validated and sent to the root chain
03:08 — server script case #2 finished working, which sent 1 million transactions in 47 seconds
03:38 — all nodes received information from the root chain that blocks #85 (f21), #86(65e), #87(256) have been added and start applying 330k, 250k, 111855 transactions
03:49 — the pool was cleared for 330k, 250k, 111855 transactions that were added to blocks #85 (f21), #86(65e), #87(256)
03:59 — submit node #1 took 888145 transactions from the pool and forms block #88 (214), submit node #2 took 750k transactions from the pool and forms block #88 (50a), submit node #3 took 670k transactions from the pool and forms block #88 (d3b)
04:44 — block #88 (d3b) is signed and sent to other nodes for validation
04:58 — block #88 (214) is signed and sent to other nodes for validation
05:11 — block #88 (50a) is signed and sent to other nodes for validation
05:11 — block #85 (d3b) validated and sent to the root chain
05:36 — block #85 (214) validated and sent to the root chain
05:43 — all nodes received information from the root chain that blocks #88 (d3b), #89(214) have been added and start applying 670k, 750k transactions
06:50 — block #85 (50a) was not validated due to a broken connection
06:55 — submit node #2 took 888145 transactions from the pool and forms block #90 (50a)
08:14 — block #90 (50a) is signed and sent to other nodes for validation
09:04 — block #90 (50a) validated and sent to the root chain
11:23 - all nodes received information from the root chain that block #90 (50a) has been added, and begin to apply 888145 transactions. At the same time, server #3 has long applied transactions from blocks #88 (d3b), #89(214)
12:11 - all pools are empty
13:41 - all nodes of server #3 contain 3 million transactions and tokens
14:35 - all nodes of server #1 contain 3 million transactions and tokens
19:24 - all nodes of server #2 contain 3 million transactions and tokens
Obstacles
During the development of Plasma Cash, we encountered the following problems, which we gradually solved and are solving:
1. The conflict of interaction of various functions of the system. For example, the function of adding transactions to the pool blocked the work of submitting and validating blocks, and vice versa, which led to a speed drop.
2. It was not immediately clear how to send a huge number of transactions and at the same time minimize data transfer costs.
3. It was not clear how and where to store the data in order to achieve high results.
4. It was not clear how to organize a network between nodes, since the block size with 1 million transactions takes about 100 MB.
5. Working in single-threaded mode breaks the connection between nodes when there are long calculations (for example, building a Merkle tree and calculating its hash).
How did we deal with all this?
The first version of the Plasma Cash node was a kind of combine that could do everything at the same time: accept transactions, submit and validate blocks, and provide an API for accessing data. Since NodeJS is natively single-threaded, the heavy Merkle tree calculation function was blocking the function of adding a transaction. We have seen two solutions to this problem:
1. Launch several NodeJS processes, each of which performs certain functions.
2. Use worker_threads and move the execution of part of the code to threads.
As a result, we used both options at the same time: we logically divided one node into 3 parts that can work separately, but at the same time synchronously
1. Submit node that accepts transactions to the pool and creates blocks.
2. A validating node that checks the validity of the nodes.
3. API node - provides an API for accessing data.
At the same time, you can connect to each node via unix socket using cli.
Heavy operations, such as the calculation of the Merkle tree, we moved to a separate thread.
Thus, we have achieved the normal operation of all Plasma Cash functions simultaneously and without failures.
As soon as the system was functional, we started testing the speed and, unfortunately, got unsatisfactory results: 5 transactions per second and up to 000 transactions per block. I had to find out what was implemented incorrectly.
To begin with, we started testing the mechanism of communication with Plasma Cash in order to find out the peak capability of the system. Earlier we wrote that the Plasma Cash node provides a unix socket interface. It was originally text. json objects were sent using `JSON.parse()` and `JSON.stringify()`.
```json
{
"action": "sendTransaction",
"payload":{
"prevHash": "0x8a88cc4217745fd0b4eb161f6923235da10593be66b841d47da86b9cd95d93e0",
"prevBlock": 41,
"tokenId": "57570139642005649136210751546585740989890521125187435281313126554130572876445",
"newOwner": "0x200eabe5b26e547446ae5821622892291632d4f4",
"type": "pay",
"data": "",
"signature": "0xd1107d0c6df15e01e168e631a386363c72206cb75b233f8f3cf883134854967e1cd9b3306cc5c0ce58f0a7397ae9b2487501b56695fe3a3c90ec0f61c7ea4a721c"
}
}
```
We measured the transfer speed of such objects and got ~ 130k per second. We tried to replace the standard functions for working with json, but the performance did not improve. The V8 engine must be well optimized for these operations.
We worked with transactions, tokens, blocks through classes. When creating such classes, the performance dipped by 2 times, which indicates that OOP is not suitable for us. I had to rewrite everything to a purely functional approach.
Recording to the database
Initially, Redis was chosen for data storage as one of the most productive solutions that satisfies our requirement: key-value storage, work with hash tables, sets. We ran redis-benchmark and got ~80k operations per second in 1 pipelining mode.
For high performance, we tuned Redis more finely:
- Established a unix socket connection.
- Disabled saving state to disk (for reliability, you can set up a replica and save to disk in a separate Redis).
In Redis, a pool is a hash table, since we need the ability to get all transactions in one request and delete transactions one by one. We tried to use a regular list, but it works slower when unloading the entire list.
When using standard NodeJS, the Redis libraries got a performance of 18k transactions per second. The speed dropped by 9 times.
Since the benchmark showed us the possibilities clearly 5 times more, we began to optimize. We changed the library to ioredis and got a performance of 25k per second. We added transactions one by one using the `hset` command. Thus, we generated many requests in Redis. There was an idea to unite transactions in packs and send them with one `hmset` command. The result is 32k per second.
For several reasons, which we will describe below, we work with data using `Buffer` and, as it turned out, if we translate it into text (`buffer.toString('hex')`) before writing, we can get additional performance. Thus, the speed was increased to 35k per second. At the moment, we decided to suspend further optimization.
We had to switch to a binary protocol because:
1. The system often calculates hashes, signatures, etc., and for this it needs data in `Buffer.
2. When sent between services, binary data weighs less than text. For example, when sending a block with 1 million transactions, the data in the text can take more than 300 megabytes.
3. Constant data conversion affects performance.
Therefore, we took as a basis our own binary data storage and transfer protocol, developed on the basis of the wonderful `binary-data` library.
As a result, we have the following data structures:
—Transaction
```json
{
prevHash: BD.types.buffer(20),
prevBlock: BD.types.uint24le,
tokenId: BD.types.string(null),
type: BD.types.uint8,
newOwner: BD.types.buffer(20),
dataLength: BD.types.uint24le,
data: BD.types.buffer(({current}) => current.dataLength),
signature: BD.types.buffer(65),
hash: BD.types.buffer(32),
blockNumber: BD.types.uint24le,
timestamp: BD.types.uint48le,
}
```
—Tokens
```json
{
id: BD.types.string(null),
owner: BD.types.buffer(20),
block: BD.types.uint24le,
amount: BD.types.string(null),
}
```
—Block
```json
{
number: BD.types.uint24le,
merkleRootHash: BD.types.buffer(32),
signature: BD.types.buffer(65),
countTx: BD.types.uint24le,
transactions: BD.types.array(Transaction.Protocol, ({current}) => current.countTx),
timestamp: BD.types.uint48le,
}
```
With the usual commands `BD.encode(block, Protocol).slice();` and ` BD.decode(buffer, Protocol)` we convert the data to `Buffer` to save it to Redis or send it to another node and retrieve the data back.
We also have 2 binary protocols for transferring data between services:
- Protocol for interacting with Plasma Node via unix socket
```json
{
type: BD.types.uint8,
messageId: BD.types.uint24le,
error: BD.types.uint8,
length: BD.types.uint24le,
payload: BD.types.buffer(({node}) => node.length)
}
```
where:
- `type` - action to be performed, for example, 1 - sendTransaction, 2 - getTransaction;
- `payload` - data to be passed to the corresponding function;
- `messageId` — the message id so that the response can be identified.
— Protocol of interaction between nodes
```json
{
code: BD.types.uint8,
versionProtocol: BD.types.uint24le,
seq: BD.types.uint8,
countChunk: BD.types.uint24le,
chunkNumber: BD.types.uint24le,
length: BD.types.uint24le,
payload: BD.types.buffer(({node}) => node.length)
}
```
where:
- `codes` - message code, for example 6 - PREPARE_NEW_BLOCK, 7 - BLOCK_VALID, 8 - BLOCK_COMMIT;
- `versionProtocol` - protocol version, since nodes with different versions can be raised in the network and they can work differently;
- `seq` — message identifier;
- `countChunk` и `chunkNumber` necessary for splitting large messages;
- `length` и `payload` length and the data itself.
Since we pre-typed the data, the end system is much faster than the `rlp` library from Ethereum. Unfortunately, we have not been able to abandon it yet, as it is necessary to finalize the smart contract, which we plan to do in the future.
If we managed to reach the speed 35 000 transactions per second, we also need to process them in optimal time. Since the approximate block formation time is 30 seconds, we need to include in the block 1 000 000 transactions, which means sending more 100 mb data.
Initially, we used the `ethereumjs-devp2p` library for node communication, but it couldn't handle the amount of data. As a result, we used the `ws` library and configured the transfer of binary data via websocket. Of course, we also encountered problems when sending large data packets, but we divided them into chunks and now there are no such problems.
Also merkle tree formation and hash calculation 1 000 000 transactions require about 10 seconds of continuous computing. During this time, the connection with all the nodes has time to break. It was decided to move this calculation to a separate thread.
Conclusions:
In fact, our findings are not new, but for some reason, many experts forget about them when developing.
- Using Functional Programming instead of Object-Oriented Programming improves performance.
- A monolith is worse than a service architecture for a productive NodeJS system.
- Using `worker_threads` for heavy computation improves system responsiveness, especially when dealing with i/o operations.
- unix socket is more stable and faster than http requests.
- If you need to quickly transfer large data over the network, it is better to use websockets and send binary data broken into chunks that can be forwarded if they do not reach, and then combined into one message.
We invite you to visit GitHub project:
The article was co-authored with Alexander Nashivan, senior developer .
Source: habr.com