Blockchain β an innovative technology promising to enhance many areas of human life. It transfers real processes and products into the digital space, ensures speed and reliability of financial operations, reduces their cost, and also allows for the creation of modern DAPP applications using smart contracts in decentralized networks.
Given the numerous advantages and diverse applications of blockchain, it may seem strange that this promising technology has not yet penetrated all industries. The problem is that modern decentralized blockchains lack scalability. Ethereum processes about 20 transactions per second, which is not sufficient to meet the needs of today's dynamic businesses. At the same time, companies using blockchain technology hesitate to abandon Ethereum due to its high level of protection against hacks and network failures.
To ensure decentralization, security, and scalability in blockchain, thereby addressing the Scalability Trilemma, the development team created Plasma Cash β a sidechain consisting of a smart contract and a private network based on Node.js, periodically transmitting its state to the root chain (Ethereum).
Key processes in Plasma Cash
1. A user calls the smart contract function `deposit`, passing the amount in ETH that they wish to place in the Plasma Cash token. The smart contract function creates the token and generates an event regarding this.
2. Plasma Cash nodes, subscribed to events from the smart contract, receive the event of the deposit creation and add the token creation transaction to the pool.
3. Periodically, special Plasma Cash nodes take all transactions from the pool (up to 1 million) and form a block from them, calculating the Merkle tree and, accordingly, the hash. This block is sent to other nodes for verification. The nodes check whether the Merkle hash is valid, whether the transactions are valid (for example, whether the sender of the token is its owner). After block verification, the node calls the `submitBlock` function of the smart contract, which saves the number and Merkle hash of the block to the root chain. The smart contract generates an event about the successful addition of the block. Transactions are removed from the pool.
4. Nodes that received the block submission event begin to apply the transactions that were added to the block.
5. At some point, the owner (or non-owner) of the token wants to withdraw it from Plasma Cash. To do this, they call the `startExit` function, passing it the information about the last 2 transactions for the token, which confirm that they are indeed the owner of the token. The smart contract, using the Merkle hash, checks for the presence of the transactions in the blocks and sends the token for withdrawal, which will occur in two weeks.
6. If the token withdrawal operation occurred with violations (the token was spent after the withdrawal procedure began or the token was already someone else's before withdrawal), the owner of the token can dispute the withdrawal within two weeks.
Privacy is achieved in two ways
1. The root chain has no knowledge of the transactions that are formed and transmitted within the child chain. Public information remains about who deposited and withdrew ETH to/from Plasma Cash.
2. The child chain allows for anonymous transactions using zk-SNARKs.
Technology stack
- NodeJS
- Redis
- Ethereum
- Solid
Testing
While developing Plasma Cash, we tested the system's speed and obtained the following results:
- up to 35,000 transactions per second are added to the pool;
- up to 1,000,000 transactions can be stored in a block.
The tests were conducted on the following 3 servers:
1. Intel Core i7-6700 Quad-Core Skylake incl. NVMe SSD β 512 GB, 64 GB DDR4 RAM
3 validating Plasma Cash nodes were launched.
2. AMD Ryzen 7 1700X Octa-Core "Summit Ridge" (Zen), SATA SSD β 500 GB, 64 GB DDR4 RAM
A Ropsten testnet ETH node was launched.
3 validating Plasma Cash nodes were launched.
3. Intel Core i9-9900K Octa-Core incl. NVMe SSD β 1 TB, 64 GB DDR4 RAM
1 submission Plasma Cash node was launched.
3 validating Plasma Cash nodes were launched.
A test was initiated to add transactions to the Plasma Cash network.
Total: 10 Plasma Cash nodes in a private network.
Test 1
There is a limit of 1 million transactions per block. Therefore, 1 million transactions are spread across 2 blocks (since the system manages to take a portion of the transactions and submit them while they are being sent).
Initial state: last block #7; 1 million transactions and tokens are stored in the database.
00:00 β script for transaction generation started
01:37 β 1 million transactions created and sending to node began
01:46 β the submitting node took 240k transactions from the pool and is forming block #8. We also see that 320k transactions are added to the pool in 10 seconds
01:58 β block #8 signed and sent for validation
02:03 β Block #8 has been validated, and the `submitBlock` function of the smart contract has been called with the Merkle hash and block number.
02:10 β The demo script has finished running, which sent 1 million transactions in 32 seconds.
02:33 β Nodes began receiving information that block #8 was added to the root chain and started processing 240k transactions.
02:40 β 240k transactions that were already in block #8 have been removed from the pool.
02:56 β The submit node took the remaining 760k transactions from the pool and started calculating the Merkle hash and signing block #9.
03:20 β All nodes contain 1 million 240k transactions and tokens.
03:35 β Block #9 has been signed and is being sent for validation to other nodes.
03:41 β A network error occurred.
04:40 β Validation of block #9 has timed out.
04:54 β The submit node took the remaining 760k transactions from the pool and started calculating the Merkle hash and signing block #9.
05:32 β Block #9 has been signed and is being sent for validation to other nodes.
05:53 β Block #9 has been validated and sent to the root chain.
06:17 β Nodes began receiving information that block #9 was added to the root chain and started processing 760k transactions.
06:47 β The pool has been cleared of the transactions that were included in block #9.
09:06 β All nodes contain 2 million transactions and tokens.
Test 2
There is a limit of 350k on the block. As a result, we have 3 blocks.
Initial state: the last block is #9; 2 million transactions and tokens are saved in the database.
00:00 β The transaction generation script is already running.
00:44 β 1 million transactions have been created and are being sent to the node.
00:56 β The submit node took 320k transactions from the pool and is forming block #10. We also see that 320k transactions are being added to the pool in 10 seconds.
01:12 β Block #10 has been signed and is being sent to other nodes for validation.
01:18 β The demo script has finished, which sent 1 million transactions in 34 seconds.
01:20 β Block #10 has been validated and sent to the root chain.
01:51 β All nodes received information from the root chain that block #10 has been added and begin applying 320k transactions.
02:01 β The pool has cleared 320k transactions that were added to block #10.
02:15 β The submit node took 350k transactions from the pool and is forming block #11.
02:34 β Block #11 has been signed and is being sent to other nodes for validation.
02:51 β Block #11 has been validated and sent to the root chain.
02:55 β The last node has executed the transactions from block #10.
10:59 β the transaction with submission of block #9 was executed very slowly in the root chain, but it has been completed, and all nodes received the information and began processing 350k transactions
11:05 β the pool was cleared of 320k transactions that were added to block #11
12:10 β all nodes contain 1 million 670k transactions and tokens
12:17 β the submission node took 330k transactions from the pool and is forming block #12
12:32 β block #12 has been signed and sent to other nodes for validation
12:39 β block #12 has been validated and sent to the root chain
13:44 β all nodes have received information from the root chain that block #12 has been added and begin to apply 330k transactions
14:50 β all nodes contain 2 million transactions and tokens
Test 3
On the first and second servers, one validating node was replaced with a submission node.
Initial state: last block #84; no transactions and tokens are stored in the database
00:00 β 3 scripts have been launched that generate and send 1 million transactions each
01:38 β 1 million transactions have been created and the sending to submission node #3 has started
01:50 β submission node #3 took 330k transactions from the pool and is forming block #85 (f21). We also see that 350k transactions are being added to the pool in 10 seconds
01:53 β 1 million transactions have been created and the sending to submission node #1 has started
01:50 β submission node #3 took 330k transactions from the pool and is forming block #85 (f21). We also see that 350k transactions are being added to the pool in 10 seconds
02:01 β submission node #1 took 250k transactions from the pool and is forming block #85 (65e)
02:06 β block #85 (f21) has been signed and sent to other nodes for validation
02:08 β the demo script of server #3 has finished working, which sent 1 million transactions in 30 seconds
02:14 β block #85 (f21) has been validated and sent to the root chain
02:19 β block #85 (65e) has been signed and sent to other nodes for validation
02:22 β 1 million transactions have been created and the sending to submission node #2 has started
02:27 β block #85 (65e) has been validated and sent to the root chain
02:29 β submission node #2 took 111855 transactions from the pool and is forming block #85 (256).
02:36 β block #85 (256) has been signed and sent to other nodes for validation
02:36 β the demo script of server #1 has finished working, which sent 1 million transactions in 42.5 seconds
02:38 β block #85 (256) has been validated and sent to the root chain
03:08 β the demo script of server #2 has finished working, which sent 1 million transactions in 47 seconds
03:38 β all nodes received information from the root chain that blocks #85 (f21), #86 (65e), #87 (256) have been added and begin to apply 330k, 250k, 111855 transactions
03:49 β the pool cleared 330k, 250k, 111855 transactions that were added to blocks #85 (f21), #86(65e), #87(256)
03:59 β submit node #1 took 888145 transactions from the pool and is forming block #88 (214), submit node #2 took 750k transactions from the pool and is forming block #88 (50a), submit node #3 took 670k transactions from the pool and is forming block #88 (d3b)
04:44 β block #88 (d3b) has been signed and sent to other nodes for validation
04:58 β block #88 (214) has been signed and sent to other nodes for validation
05:11 β block #88 (50a) has been signed and sent to other nodes for validation
05:11 β block #85 (d3b) has been validated and sent to the root chain
05:36 β block #85 (214) has been validated and sent to the root chain
05:43 β all nodes received information from the root chain that blocks #88 (d3b), #89(214) have been added, and they start applying 670k, 750k transactions
06:50 β due to a connection failure, block #85 (50a) was not validated
06:55 β submit node #2 took 888145 transactions from the pool and is forming block #90 (50a)
08:14 β block #90 (50a) has been signed and sent to other nodes for validation
09:04 β block #90 (50a) has been validated and sent to the root chain
11:23 β all nodes received information from the root chain that block #90 (50a) has been added, and they start applying 888145 transactions. Meanwhile, server #3 has long applied transactions from blocks #88 (d3b), #89(214)
12:11 β all pools are empty
13:41 β all nodes of server #3 contain 3 million transactions and tokens
14:35 β all nodes of server #1 contain 3 million transactions and tokens
19:24 β all nodes of server #2 contain 3 million transactions and tokens
Obstacles
During the development of Plasma Cash, we encountered the following problems, which we gradually resolved and are still resolving:
1. Conflict of interaction between various functions of the system. For example, the function of adding transactions to the pool blocked the work of submitting and validating blocks, and vice versa, which led to a drop in speed.
2. It was not immediately clear how to send a huge number of transactions while minimizing data transfer costs.
3. It was unclear how and where to store data to achieve high results.
4. It was unclear how to organize the network among nodes, as the size of a block with 1 million transactions occupies about 100 MB.
5. Working in single-thread mode breaks the connection between nodes during long computations (for example, constructing the Merkle tree and calculating its hash).
How did we handle all of this?
The first version of the Plasma Cash node was a sort of hybrid that could do everything at once: accept transactions, submit and validate blocks, and provide an API for data access. Since NodeJS is originally single-threaded, the heavy function of calculating the Merkle tree blocked the transaction addition function. We saw two possible solutions to this problem:
1. Run multiple NodeJS processes, each performing specific functions.
2. Use worker_threads and move part of the code execution to threads.
In the end, we utilized both options simultaneously: we logically divided a single node into three parts that can operate separately but at the same time synchronously.
1. A submit node that accepts transactions into the pool and handles block creation.
2. A validating node that checks the validity of nodes.
3. API node β provides an API for data access.
At the same time, each node can be connected to via a unix socket using the CLI.
Heavy operations, such as calculating the Merkle tree, were moved to a separate thread.
Thus, we achieved normal operation of all Plasma Cash functions simultaneously and without failures.
As soon as the system was functionally operational, we began testing the speed, and unfortunately, we received unsatisfactory results: 5,000 transactions per second and up to 50,000 transactions per block. We had to investigate what was incorrectly implemented.
To start, we began testing the communication mechanism with Plasma Cash to find out the system's peak capability. Previously, we noted that the Plasma Cash node provides a unix socket interface. Initially, it was text-based. JSON objects were transmitted using `JSON.parse()` and `JSON.stringify()`.
{"action": "sendTransaction","payload":{"prevHash": "0x8a88cc4217745fd0b4eb161f6923235da10593be66b841d47da86b9cd95d93e0","prevBlock": 41,"tokenId": "57570139642005649136210751546585740989890521125187435281313126554130572876445","newOwner": "0x200eabe5b26e547446ae5821622892291632d4f4","type": "pay","data": "","signature": "0xd1107d0c6df15e01e168e631a386363c72206cb75b233f8f3cf883134854967e1cd9b3306cc5c0ce58f0a7397ae9b2487501b56695fe3a3c90ec0f61c7ea4a721c"}}
We measured the speed of sending such objects and obtained ~130k per second. We tried to replace the standard JSON handling functions, but performance did not improve. It seems that the V8 engine is well optimized for these operations.
Working with transactions, tokens, and blocks was done through classes. When creating such classes, the performance dropped by half, indicating that OOP is not suitable for us. We had to rewrite everything with a purely functional approach.
Database entry
Initially, Redis was chosen for data storage as one of the most efficient solutions that meets our requirements: a key-value store, working with hash tables and sets. We ran redis-benchmark and achieved ~80k operations per second in 1 pipelining mode.
For high performance, we configured Redis more finely:
- We set up a UNIX socket connection.
- We disabled state saving to disk (for reliability, a replica can be configured and saving can be done in a separate Redis).
In Redis, a pool is a hash table, as we need the ability to fetch all transactions in one request and delete transactions individually. We tried using a regular list, but it worked slower when unloading the entire list.
With the standard NodeJS Redis library, we achieved a performance of 18k transactions per second. The speed dropped by 9 times.
Since the benchmark showed us capabilities clearly 5 times greater, we began optimizing. We switched the library to ioredis and achieved a performance of 25k per second. We added transactions individually using the `hset` command. Thus, we generated many requests to Redis. The idea arose to combine transactions into batches and send them with one command, `hmset`. The result was 32k per second.
For several reasons which we will describe below, we work with data using `Buffer`, and it turned out that converting it to text (`buffer.toString(βhexβ)`) before writing can yield additional performance. Thus, we managed to increase the speed to 35k per second. At this point, we decided to pause further optimization.
We had to switch to a binary protocol because:
1. The system often computes hashes, signatures, etc., and it needs data in `Buffer.
2. When transmitting between services, binary data weighs less than text. For example, sending a block with 1 million transactions, the data in text can take up more than 300 megabytes.
3. Constant data conversion affects performance.
Therefore, we based our work on our own binary protocol for storing and transmitting data, developed using the remarkable `binary-data` library.
As a result, we created the following data structures:
β Transaction
```json
{
prevHash: BD.types.buffer(20),
prevBlock: BD.types.uint24le,
tokenId: BD.types.string(null),
type: BD.types.uint8,
newOwner: BD.types.buffer(20),
dataLength: BD.types.uint24le,
data: BD.types.buffer(({current}) => current.dataLength),
signature: BD.types.buffer(65),
hash: BD.types.buffer(32),
blockNumber: BD.types.uint24le,
timestamp: BD.types.uint48le,
}
```
β Token
```json
{
id: BD.types.string(null),
owner: BD.types.buffer(20),
block: BD.types.uint24le,
amount: BD.types.string(null),
}
```
β Block
```json
{
number: BD.types.uint24le,
merkleRootHash: BD.types.buffer(32),
signature: BD.types.buffer(65),
countTx: BD.types.uint24le,
transactions: BD.types.array(Transaction.Protocol, ({current}) => current.countTx),
timestamp: BD.types.uint48le,
}
```
With the ordinary commands `BD.encode(block, Protocol).slice();` and ` BD.decode(buffer, Protocol)`, we transform the data into `Buffer` for storage in Redis or sending to another node and extract the data back.
We also have 2 binary protocols for data transmission between services:
β Protocol for interacting with Plasma Node via unix socket
```json
{
type: BD.types.uint8,
messageId: BD.types.uint24le,
error: BD.types.uint8,
length: BD.types.uint24le,
payload: BD.types.buffer(({node}) => node.length)
}
```
where:
- `type` β the action to be performed, for example, 1 β sendTransaction, 2 β getTransaction;
- `payload` β data to be passed to the corresponding function;
- `messageId` β message id for identifying the response.
β Protocol for interaction between nodes
```json
{
code: BD.types.uint8,
versionProtocol: BD.types.uint24le,
seq: BD.types.uint8,
countChunk: BD.types.uint24le,
chunkNumber: BD.types.uint24le,
length: BD.types.uint24le,
payload: BD.types.buffer(({node}) => node.length)
}
```
where:
- `code` β message code, for example 6 β PREPARE_NEW_BLOCK, 7 β BLOCK_VALID, 8 β BLOCK_COMMIT;
- `versionProtocol` β protocol version, since nodes with different versions can be running in the network and behave differently;
- `seq` β message identifier;
- `countChunk` and `chunkNumber` necessary for splitting large messages;
- `length` and `payload` length and the data itself.
Since we pre-typed the data, the final system works much faster than the `rlp` library from Ethereum. Unfortunately, we have not yet been able to discard it, as we need to finalize the smart contract, which we plan to do in the future.
If we manage to achieve speed 35 000 transactions per second, we also need to process them in an optimal time. Since the approximate time to form a block takes 30 seconds, we need to include in the block 1 000 000 transactions, which means transferring more than 100 MB of data.
Initially, we used the `ethereumjs-devp2p` library for node communication, but it could not handle such a volume of data. As a result, we utilized the `ws` library and configured the transfer of binary data over websockets. Of course, we also faced issues when transferring large data packets, but we split them into chunks and now these problems no longer exist.
The construction of the Merkle tree and calculating the hash 1 000 000 of transactions takes about 10 seconds of continuous computation. During this time, the connection to all nodes tends to drop. It was decided to move this computation to a separate thread.
Conclusions:
In fact, our conclusions are not new, but for some reason, many specialists forget about them during development.
- Using Functional Programming instead of Object-Oriented Programming increases performance.
- Monolith is worse than a service-oriented architecture for a productive system on NodeJS.
- Using `worker_threads` for heavy computations improves system responsiveness, especially when dealing with I/O operations.
- Unix sockets are more stable and faster than HTTP requests.
- If you need to quickly transfer large amounts of data over the network, it's better to use websockets and send binary data split into chunks, which can be retransmitted if they do not reach their destination, and then reassembled into a single message.
We invite you to visit GitHub of the project:
The article was co-authored by Alexander Nashiwan, senior developer at .
Source: habr.com
