For two weeks now, Runet has been making noise about Telegram and the situation with its senseless and merciless blocking by Roskomnadzor. The ricochet offended many, but all of these are topics for posts on Geektimes. I was surprised by something else - I still have not seen on Habré a single analysis of the TON network planned to be released on the basis of Telegram - Telegram Open Network. I wanted to make up for this shortcoming, because there is something to study there - even despite the absence of official statements about it.
Let me remind you that there are rumors that Telegram launched a very large-scale closed ICO, having already collected incredible amounts in it. It is assumed that Gram’s own cryptocurrency will be launched this year, and each Telegram user will automatically have a wallet, which in itself creates a considerable advantage over other cryptocurrencies.
Unfortunately, since there are no official statements, I can only proceed from about which I immediately warn you. Of course, it may turn out to be a very skillful fake, but it is also possible that this is the real whitepaper of the future system, written by Nikolai Durov (and probably leaked by one of the investors). But even if this is a fake, no one will forbid us to study and discuss it, right?
What does this document say? I will try to retell it in my own words, close to the text, but in Russian and a little more human (may Nikolay forgive me with his tendency to go into formal mathematics). Keep in mind that even if authentic, this is a draft description of the system and it is very likely to change by the time of the public launch.
We learn that in addition to cryptocurrency, a lot more is supposed to happen. Let's take it in order.
- TON Blockchain. This is the basis of the entire system. If you don't really know what is - I recommend finding out, because there will be a lot of blockchains here. Nested, virtually fragmented and even “vertical” blockchains within blocks of other blockchains. There will also be some cool-sounding terms like Instant Hypercube Routing и Infinite Sharding Paradigm, but more on that later. And, of course, proof-of-stake and smart contracts.
- TON P2P Network. A peer-to-peer network on the basis of which the system will be built. It will be discussed first of all in this part of the story.
- TON Storage. A file storage that will be built on the aforementioned peer-to-peer network regardless of the blockchain. Can be compared with torrents.
- TON Proxy. This is a service whose purpose is to increase the anonymity of network participants. Any packet can be sent not directly, but through intermediary tunnels with additional encryption - like I2P or TOR.
- TON DHT. Distributed hash table for storing arbitrary values. It is also built on top TON Network (but at the same time it is used by him) and helps TON Storage find "distributing" nodes, and TON Proxy - intermediate relays. But it should be noted that, unlike the blockchain, this hash table is not a secure storage - you cannot store important information in it.
- TON Services. Platform for arbitrary services. In fact, this is a new Internet on top of everything described above. Data exchange - via TON Network/TON Proxy, and the logic is in the smart contracts of the TON Blockchain. And an interface with pretty familiar URLs.
- TON DNS. Since we are talking about the usual URLs, we also need a converter from them to 256-bit addresses - accounts, contracts, services and nodes.
- TON Payments. And that's where the money issue comes into play. And it will not only gram - as with ether, any "tokens" will be possible; Grams will be just the “default” currency here.
This is the first part that describes the "ground" level of TON - its network part, built on top of traditional protocols. In the next part, we will talk about the "pulp" - the blockchain, which will be supported by the system described below. Thus, my order of retelling is somewhat different from that used in the above document (which starts immediately at the abstract level).
Basic Concepts
TL (Type Language). It is an abstract binary format for arbitrary data structures. It is used in the Telegram protocol and will be actively used in TON. If you want to learn more about it - .
Hash (hash). A function that performs an irreversible transformation of an arbitrary data structure into a single number of a fixed length. Throughout the documentation, there is a lot of talk about the function .
Network node (urge). The node is the software that will ensure the operation of the system. In particular, it is assumed that each Telegram client application will include a TON node. At a low level, nodes have IPv4/IPv6 addresses and communicate using the UDP protocol, at a higher level they have abstract addresses and implement the ADNL protocol (see below for abstract addresses and ADNL). When it comes to the fact that some parts of the system do something or store some data, it is understood that network nodes do it.
Abstract address (or simply address, address). The address of a node is determined by its public key. More strictly, this is a 256-bit hash (SHA256) from a data structure containing a public key (the specific cryptographic algorithm is not specified in this case - elliptic curves and RSA-2048 are given as an example). In order for one node to communicate with another, it needs to know not only the address of that one, but also this data structure. Theoretically, one physical node can create any number of addresses (corresponding to different keys).
Further, just such a bundle is often used: a “preimage” in the form of a TL structure (containing almost any data), and a 256-bit hash from it used for addressing.
Blockchain (blockchain). Blockchain is a data structure, elements (Blocks) which are ordered into a "chain", and each next block of the chain contains the hash of the previous one. In this way, integrity is achieved - changes can only be made by adding new blocks.
Service (service). Services within TON can be of various types, depending on whether they use the blockchain or not. For example, one (or many) of the network nodes can process some RPC requests using the ADNL protocol described below, without creating any records in the blockchain - like traditional web servers. In particular, the possibility of implementing HTTP over ADNL is being considered, as well as the transition of the messenger itself to this protocol. By analogy with TOR or I2P, this will make it more resistant to various blocking.
At the same time, a number of services imply both interaction with the blockchain and processing requests outside of it. For example, for TON Storage - file storage - it is not very reasonable to store the files themselves on the blockchain. It will contain only file hashes (along with some meta-information about them), and specialized network nodes will act as "file servers", ready to give them to other nodes via ADNL.
fog service (fog service). We are talking about some services that involve decentralization and open participation in them. For example, TON Proxy is a service that can be supported by any participant who wants to provide their node as an intermediary (proxy) that forwards packets between other nodes. If desired, he can charge a fee set by him for this - using the TON Payments system for micropayments (which, in turn, is also a foggy service).
ADNL: Abstract Datagram Network Layer
At the lowest level, communication between nodes will be via UDP (although other options are acceptable).
As mentioned above, in order for one node to send a packet to another, it must know one of its public keys (and hence the address it determines). It encrypts the packet with this key and adds a 256-bit destination address to the beginning of the packet - since one node can have several such addresses, this will allow it to determine which key to use for decryption.
In addition, instead of the recipient's address at the beginning of the data packet, there may be a so-called. identifier channel. In this case, the processing of the packet already depends on specific agreements between the nodes - for example, data sent to a certain channel can be destined for another node and must be redirected to it (this is the service TON Proxy). Another special case can be interaction directly between nodes, but with encryption using an individual key pair for this channel (previously formed using the Diffie-Hellman protocol).
Finally, a special case is the "null" channel - if the node does not yet know the public keys of its "neighbors", it can send them packets without encryption at all. This is only for initialization - as soon as the nodes send information about their keys, they should be used for further interaction.
The protocol described above (256 bits of channel ID + packet content) is called ADNL. The documentation mentions the possibility of implementing an analogue of TCP on top of it or its own add-on - RLDP (Reliable Large Datagram Protocol), but does not go into details about their implementation.
TON DHT: Distributed Hash Table
As in the case of other distributed systems, TON involves the implementation of DHT - . More specifically, the table is . If you are not familiar with this kind of hash tables, do not worry, then I will describe roughly how they are arranged.
In an abstract sense, DHT maps 256-bit keys to binary values of arbitrary length. At the same time, the keys in the table are hashes from a certain TL structure (the structures themselves are also stored together with the DHT). This is very similar to the formation of node addresses - and they can indeed be present in the DHT (for example, such a key can contain the IP address of the node corresponding to the given abstract addressif it doesn't hide it). But in the general case, "pre-images of keys" (their descriptions, key descriptions) is metadata that indicates the “owner” of an entry in the hash table (that is, the public key of some node), the type of value stored, and the rules by which this entry can subsequently be changed. For example, a rule may allow only the owner to change the value - or prohibit changing the value down (to protect against replay attacks).
In addition to 256-bit keys, the concept of DHT addresses is introduced. The difference with regular host addresses is that a DHT address is bound to an IP address. If a node does not hide its IP, it can use a regular address for DHT. But more often, a separate, “semi-permanent” address will be created for the needs of DHT.
Above keys and DHT addresses, the concept of distance is introduced - everything coincides with tables in this - the distance between the keys is equal to XOR (bitwise exclusive OR) from them. As with Kademlia tables, the value corresponding to some key must be stored in s nodes that have the smallest distance to this key (s here is a relatively small number).
In order for a DHT node to communicate with other such nodes, it keeps in memory DHT routing table - DHT and IP addresses of the nodes with which it interacted before, grouped by distance to them. There are 256 such groups (they correspond to the most significant bit set in the distance value - that is, nodes at a distance from 0 to 255 will fall into one group, from 256 to 65535 - into the next, etc.). Within each group, a limited number of "best" nodes are stored (in terms of ping to them).
Each node must support several operations: storing a value for a key, node search и search for values. The search for nodes implies the issuance by a given key of the nodes closest to it from the routing table; looking up values is the same, except when the node knows the value for the key (in which case it simply returns it). Accordingly, if a node wants to find a value in the DHT by a key, it sends requests to a small number of nodes closest to this key from its routing table. If among their answers there is no desired value, but there are other addresses of nodes, then the request is repeated to them.
TON DHT can be used for various purposes, for example, to implement a torrent-like file storage (see. TON Storage); to determine the addresses of nodes that implement certain services; to store information about account holders in the blockchain. But the most important application is the discovery of nodes by their abstract addresses. To do this, the address is used as a key, the value of which is to be found. The query will either find the host itself (if the address being looked up was its semi-permanent DHT address), or the value will be the IP address and port to connect to, or another address to use as the intermediary tunnel.
Overlay networks in TON
The ADNL protocol described above implies the ability for any nodes to exchange information with each other - though not necessarily in optimal ways. We can say that thanks to ADNL, all nodes form a global TON graph (ideally, connected). But additionally, it is possible to create overlay networks - subgraphs inside this graph.
Within such a network, interaction is carried out only directly - through pre-formed links between network member nodes (via the ADNL channels described above). The formation of such connections between neighbors, the search for the neighbors themselves is an automatic process that seeks to maintain the connectivity of the overlay network and minimize delays in the exchange of data in it.
In addition, there is a way to quickly distribute large broadcast updates within the network - they are broken into parts, supplemented with error correction code, and all these parts are sent from one participant to another. Thus, the participant does not have to fully receive all the parts before forwarding them further on the network.
Overlay networks can be public or private. Becoming a member of a public network is not difficult - you need to find a TL structure that describes it (it can be public - or accessible by a certain key in the DHT). In the case of a private network, this structure must be known to the node in advance.
To be continued
I decided to divide the TON review into several articles. This is where this section ends, I turn to the consideration of the structure of the blockchain (more precisely, blockchains) that TON will consist of.
Source: habr.com