For two weeks now, the Runet has been buzzing about Telegram and the situation with its senseless and merciless blocking by Roskomnadzor. Many have felt the repercussions, but all this is material for posts on Geektimes. What surprised me is that I still haven't seen a single analysis on Habr about the upcoming Telegram-based network TON — Telegram Open Network. I wanted to fill this gap, because there is much to explore even despite the absence of official statements about it.
As a reminder, there are rumors that Telegram has launched a massive closed ICO, having already raised incredible sums. It is expected that this year a cryptocurrency called Gram will be launched — and each Telegram user will automatically have a wallet, which in itself creates a considerable advantage over other cryptocurrencies.
Unfortunately, since there are no official statements, I can only rely on , which I warn you about right away. Of course, it may turn out to be a very skillful forgery, but it's also possible that this is the real whitepaper of the future system, written by Nikolai Durov (and leaked, probably, by someone from the investors). But even if it is a fake, no one can forbid us from studying and discussing it, right?
So what does this document say? I will try to summarize it in my own words, closely sticking to the text, but in more human terms (may Nikolai forgive me for his tendency to delve into formal mathematics). Keep in mind that even if it is authentic, this is a draft description of the system, and it is very likely to change by the time of the public launch.
We learn that besides cryptocurrency, many, many other things are anticipated. Let's break it down in order.
- TON Blockchain. This is the foundation of the entire system. If you don't know what a is, I recommend finding out, because there will be many blockchains here. Nested within each other, virtually fragmented, and even 'vertical' blockchains inside the blocks of other blockchains. And there will be several cool-sounding terms like Instant Hypercube Routing and Infinite Sharding Paradigm, but more on that later. And of course, proof-of-stake and smart contracts.
- TON P2P NetworkThe peer-to-peer network on which the system will operate. This will be the primary focus of this part of the narrative.
- TON StorageA file storage system that will be constructed independently of the blockchain on the aforementioned peer-to-peer network. It can be compared to torrents.
- TON ProxyThis service aims to enhance the anonymity of network participants. Any packet can be sent not directly, but through intermediate tunnel proxies with additional encryption—similar to I2P or TOR.
- TON DHTA distributed hash table for storing arbitrary values. It is also built upon TON Network (but is used by it) and helps TON Storage find 'seeder' nodes, and TON Proxy intermediate relays. However, it should be noted that, unlike a blockchain, this hash table is not a secure storage—important information should not be stored in it.
- TON ServicesA platform for arbitrary services. Essentially, this is a new internet built on top of everything described above. Data exchange happens through TON Network/TON Proxy, while the logic resides within the smart contracts themselves TON Blockchain. The interface features fairly familiar URLs.
- TON DNSSince we've mentioned familiar URLs, a converter from these to 256-bit addresses for accounts, contracts, services, and nodes is needed.
- TON PaymentsAnd only here the monetary aspect comes into play. It won't just be gram —as with Ether, any 'tokens' will be possible; grams here will merely be the 'default' currency.
This is the first part, describing the 'grounded' level of TON—the network aspect built on traditional protocols. The next part will delve into the 'heart'—the blockchain that will be supported by the system described below. Thus, my order of retelling slightly differs from the one used in the aforementioned document (which starts immediately with the abstract level).
Basic Concepts
TL (Type Language). This is an abstract binary format for arbitrary data structures. It is used in the Telegram protocol and will be actively utilized in TON. If you want to learn more about it— .
Hash (hash). A function that performs an irreversible transformation of an arbitrary data structure into a single fixed-length number. The documentation frequently discusses the function .
Network Node (node). A node is software that will support the operation of the system. In particular, it is expected that each client application of Telegram will include a node of TON. At a low level, nodes have IPv4/IPv6 addresses and communicate via the UDP protocol; at a higher level, they have abstract addresses and implement the ADNL protocol (see below for more about abstract addresses and ADNL). When it is stated that certain parts of the system perform tasks or store data, it implies that this is done by the network nodes.
Abstract address (or simply , password, address). A node's address is defined by its public key. More strictly, it is a 256-bit hash (SHA256) of the data structure containing the public key (the specific cryptographic algorithm is not specified—examples include elliptic curves and RSA-2048). For one node to interact with another, it needs to know not only the address but also this data structure. Theoretically, one physical node can create any number of addresses (corresponding to different keys).
It is common to use such a combination: a 'prototype' in the form of a TL structure (containing virtually any data) and the 256-bit hash from it, used for addressing.
Blockchain (blockchain). A blockchain is a data structure whose elements (blocks) are organized in a 'chain', with each subsequent block containing a hash of the previous block. This ensures integrity—changes can be made only by adding new blocks.
The service (cat << EOF | sudo tee -a /etc/systemd/system/lxd-hddpool.service [Unit] Description=Losetup LXD Storage Pool (hddpool) After=local-fs.target[Service] Type=oneshot ExecStart=/sbin/losetup /dev/loop1 /mnt/work/lxd/hddpool.img RemainAfterExit=true[Install] WantedBy=local-fs.target EOF). Services within TON can be of various types, depending on whether they use the blockchain or not. For example, one (or several) nodes in the network may handle certain RPC requests via the described ADNL protocol without creating any records in the blockchain—similar to traditional web servers. There is also a consideration for implementing HTTP over ADNL, as well as transitioning the messenger itself to this protocol. This would make it more resilient to various blocks, akin to TOR or I2P.
At the same time, a number of services imply interaction with the blockchain as well as processing requests outside it. For example, for TON Storage—a file storage solution—it is not very reasonable to store the actual files on the blockchain. Only the file hashes (along with some metadata about them) will be stored there, while specialized nodes in the network will act as 'file servers', ready to serve these files to other nodes via ADNL.
Fog Service (fog service). This refers to some services that involve decentralization and open participation. For instance, TON Proxy is a service that any participant can support by providing their node as a mediator (proxy) to forward packets between other nodes. If desired, they may charge a fee for this—using the TON Payments system for micropayments (which, in turn, is also a fog service).
ADNL: Abstract Datagram Network Layer
At the lowest level, interaction between nodes will occur over the UDP protocol (although other options are permitted).
As mentioned above, for one node to send a packet to another, it must know one of its public keys (and, consequently, the address defined by it). It encrypts the packet with this key and adds a 256-bit recipient address to the beginning of the packet—since one node may have multiple such addresses, this will allow it to determine which key to use for decryption.
Additionally, instead of the recipient address at the beginning of the data packet, there may be a so-called identifier channel. In this case, the processing of the packet will depend on specific agreements between the nodes—for example, the data sent to a certain channel may be intended for another node and should be redirected to it (this is the service TON Proxy). Another specific case may involve direct interaction between nodes, but with encryption using an individual pair of keys for that channel (pre-established via the Diffie-Hellman protocol).
Finally, a special case is the 'zero' channel — if a node does not yet know the public keys of its 'neighbors', it can send them packets without any encryption at all. This is intended solely for initialization — once the nodes send information about their keys, those should be used for further interaction.
The aforementioned protocol (256-bit channel identifier + packet content) is called ADNL. The documentation mentions the possibility of implementing a TCP-like layer on top of it or its own protocol — RLDP (Reliable Large Datagram Protocol), but does not go into detail about their implementation.
TON DHT: Distributed Hash Table
As with other distributed systems, TON implies the implementation of DHT — . More specifically — the table is If you are not familiar with this type of hash tables — don't worry, I will roughly describe how they are structured.
In an abstract sense, DHT maps 256-bit keys to some binary values of arbitrary length. At the same time, the keys in the table are hashes from a certain TL-structure (the structures themselves are also stored along with the DHT). This is very much like forming node addresses — and they can indeed be present in the DHT (for example, an IP address of a node corresponding to a given abstract address, if it does not hide it). But generally, 'prototypes of keys' (their descriptions, key descriptions) are metadata that indicate the 'owner' of the record in the hash table (i.e., the public key of some node), the type of stored value, and the rules by which this record can subsequently be modified. For example, a rule might allow only the owner to change the value — or prohibit lowering the value (to protect against replay attacks).
In addition to 256-bit keys, the concept of DHT addresses is introduced. The difference from ordinary node addresses is that the DHT address is necessarily tied to an IP address. If a node does not hide its IP, it can use a regular address for the DHT. However, more often, a separate 'semi-permanent' address will be created for DHT purposes.
The concept of distance is introduced for keys and DHT addresses — in this respect, everything corresponds to Kademlia tables. The distance between keys is equal to the XOR (exclusive OR) of them. Like in Kademlia tables, the value corresponding to a key must be stored on tr1 != str2 nodes that have the smallest distance to this key (tr1 != str2 here — a relatively small number).
In order for a DHT node to interact with other such nodes, it keeps in memory a DHT routing table — DHT and IP addresses of the nodes it has interacted with before, grouped by distance to them. There are 256 such groups (they correspond to the highest set bit in the distance value — meaning nodes at a distance of 0 to 255 fall into one group, from 256 to 65535 into the next, and so on). Within each group, a limited number of the 'best' nodes (in terms of ping) are stored.
Each node must support several operations: storing a value for a key, finding nodes and finding values. Finding nodes involves returning from the routing table the nodes closest to a given key; finding values does the same, except when the node knows the value for the key (then it simply returns it). Accordingly, if a node wants to find a value in the DHT for a key, it sends requests to a small number of nodes closest to that key from its routing table. If among their responses there is no sought value, but there are other node addresses available, the request is repeated to those nodes.
TON DHT can be used for various purposes, such as implementing a torrent-like file storage system (see TON Storage); identifying addresses of nodes implementing certain services; storing information about account holders in the blockchain. But the most important application is discovering nodes by their abstract addresses. For this, the address is used as a key, whose value needs to be found. As a result of the query, either the node itself will be found (if the sought address was its semi-permanent DHT address), or the value will be the IP address and port for connection — or another address that should be used as a proxy tunnel.
Overlay networks in TON
The ADNL protocol described above enables any nodes to exchange information with each other — although not necessarily through optimal routes. It can be said that thanks to ADNL, all nodes form a global TON graph (ideally connected). Additionally, there is the possibility to create overlay networks — subgraphs within this graph.
Within such a network, interaction occurs only directly — through pre-established connections between the participating nodes (via the ADNL channels described above). Establishing such connections between neighbors and finding neighbors themselves is an automated process aimed at maintaining the connectivity of the overlay network and minimizing data exchange delays within it.
Moreover, a method is provided to quickly disseminate large broadcast updates within the network — they are divided into parts, supplemented with error correction code, and all these pieces are sent from one participant to another. Thus, a participant does not have to fully receive all parts before forwarding them further in the network.
Overlay networks can be public or private. Joining a public network is not difficult — one needs to find a TL structure describing it (it can be public — or available through a specific key in the DHT). In the case of a private network, this structure must be known to the node in advance.
To be continued
I decided to divide the overview of TON into several articles. This part concludes here, and I will move on to discussing the structure of the blockchain (more specifically, the blockchains) that will compose TON.
Source: habr.com
