To those who want to understand networks and protocols, this is dedicated.
Summary
The article discusses the basics of reliable data transmission, with examples implemented on , including UDP and TCP. Inspired by , , and the book 'Computer Networks: A Bottom-Up Approach,' since everyone only discusses Tanenbaum and Olifer.
Transport layer protocol
provides a logical connection between application processes running on different hosts. From the application's perspective, a logical connection appears as a channel that directly connects the processes.
are supported by end systems but not by network routers (except for — ). On the sender’s side, the transport layer converts application layer data received from the transmitting application process into transport layer packets called segments.
This is done by splitting (if necessary) application layer messages into fragments and adding a transport layer header to each of them.
Then the transport layer passes the segment to the sender's network layer, where the segment is encapsulated in a network layer packet (datagram) and sent out. On the receiving side, the network layer extracts the transport layer segment from the datagram and passes it up to the transport layer. The transport layer then processes the received segment so that its data becomes available to the receiving application.
Principles of reliable data transmission
Reliable data transmission over a perfectly reliable channel
A simple case. The sending side simply receives data from the upper layer, creates a package containing it, and sends it to the channel.
Server
package main
import (
"log"
"net"
)
func main() {
// Server IP address and port
serverAddr, err := net.ResolveUDPAddr("udp", "127.0.0.1:12000")
if err != nil {
log.Fatal(err)
}
// Create a socket with the port
serverConn, err := net.ListenUDP("udp", serverAddr)
if err != nil {
log.Fatal(err)
}
// Delayed close of the connection
defer serverConn.Close()
// Create a buffer for data
buf := make([]byte, 1024)
// Wait for the connection
for {
// Read the request
n, addr, err := serverConn.ReadFromUDP(buf)
// Pass data to the UPPER level: in our case stdout
println(string(buf[0:n]), " form ", addr.IP.String())
if err != nil {
log.Fatal(err)
}
// No response, since it's UDP + reliable channel
}
}Client
package main
import (
"fmt"
"log"
"net"
"time"
)
func main() {
// Server IP address and port
serverAddr, err := net.ResolveUDPAddr("udp", "127.0.0.1:12000")
if err != nil {
log.Fatal(err)
}
// Local IP address and port
localAddr, err := net.ResolveUDPAddr("udp", "127.0.0.1:0")
if err != nil {
log.Fatal(err)
}
// Establishing connection
conn, err := net.DialUDP("udp", localAddr, serverAddr)
if err != nil {
log.Fatal(err)
}
// Delayed closure of connection
defer conn.Close()
for {
// Receiving data from the application layer
fmt.Print("Enter a string > ")
var msg string
_, err := fmt.Scanf("%s", &msg)
if err != nil {
log.Fatal(err)
}
// A stream of bytes is sent, not a string
buf := []byte(msg)
// Writing (transmitting) to the connection
_, err = conn.Write(buf)
if err != nil {
log.Fatal(err)
}
// One second pause
time.Sleep(time.Second * 1)
}
}Reliable data transmission over channels with possible errors
The next step is the assumption that all transmitted packets are received in the order they were sent, though bits within them may be corrupted, since the channel sometimes transmits data with distortions.
In such cases, mechanisms are applied:
- error detection;
- feedback;
- retransmission.
Protocols for reliable data transmission that possess such mechanisms for multiple retransmission are known as Automatic Repeat reQuest (ARQ) protocols.
Moreover, it is important to account for possible errors in the acknowledgments, when the receiving party does not receive any information about the results of the last packet transmission.
The solution to this task, used also in TCP, involves adding a new field to the data packet, which contains the sequence number of the packet.
Reliable data transmission over an unreliable channel that allows for distortion and packet loss
Unfortunately, alongside distortions, packet loss also occurs in the network.
And to solve this issue, mechanisms are required for:
- detecting the fact of packet loss;
- retransmitting lost packets to the receiving party.
Additionally, besides packet loss, it is necessary to consider the possibility of acknowledgment loss or, if nothing is lost, its delivery with a significant delay. In all cases, the same action is taken: retransmission of the packet. To control the timing in this mechanism, a countdown timer is used, which allows determining the end of the wait interval. Thus, in the packet, the TCPKeepAlive parameter is set to 15 seconds by default:
// defaultTCPKeepAlive is a default constant value for TCPKeepAlive times
// See golang.org/issue/31510
const (
defaultTCPKeepAlive = 15 * time.Second
)The transmitting side must start the timer each time a packet is transmitted (both for the first and for retransmissions), handle interruptions from the timer, and stop it.
So, we have familiarized ourselves with the key concepts of reliable data transmission protocols:
- checksums;
- packet sequence numbers;
- timers;
- positive and negative acknowledgments.
But that's not all!
Reliable data transmission protocol with pipelining
In the version we have already examined, the reliable delivery protocol is very inefficient. It starts to "slow down" the transmission provided by the communication channel as RTT increases. To improve its efficiency and better utilize the bandwidth of the communication channel, pipelining is applied.
The use of pipelining leads to:
- an increase in the range of sequence numbers, since all sent packets (except for retransmissions) must be uniquely identifiable;
- the need to increase buffers on both the transmitting and receiving sides.
The range of sequence numbers and buffer size requirements depend on the actions taken by the protocol in response to distortion, packet loss, and delays. In the case of pipelining, there are two methods for error correction:
- going back N packets;
- selective repeat.
Going back N packets — a sliding window protocol
The sender must maintain three types of events:
- Higher-level protocol call. When the sending function is invoked from 'above', the sender first checks the fill level of the window (i.e., the number of sent messages awaiting acknowledgements). If the window is not full, a new packet is formed and sent, and the variable values are updated. Otherwise, the sender returns the data to the upper level, indicating implicitly that the window is full. Typically, the upper level attempts to retransmit the data after some time. In a real application, the sender would likely either buffer the data (instead of sending it immediately) or have a synchronization mechanism (such as a semaphore or flag) that allows the upper level to call the sending function only when the window is not full.
- acknowledgment receipt. In the protocol, a cumulative acknowledgment is issued for the packet with sequence number N, indicating that all packets with sequence numbers preceding N have been successfully received.
- timeout expiration. The protocol uses a timer to determine loss and delay occurrences of packets and acknowledgments. If the wait interval expires, the sender retransmits all previously sent unacknowledged packets.
Selective Repeat
When the window size and the product of bandwidth and propagation delay are large, a significant number of packets can be in-flight. In this case, an error in a single packet may cause a retransmission of a large number of packets, most of which were not needed.
Example
Best practices are compiled into practical implementation . And if someone knows better — .
Server
package main
import (
"bufio"
"fmt"
"log"
"net"
"strings"
)
func main() {
// create a socket on the port
ln, err := net.Listen("tcp", ":8081")
if err != nil {
log.Fatalln(err)
}
// waiting for a call
conn, _ := ln.Accept()
for {
// reading data
msg, err := bufio.NewReader(conn).ReadString('n')
if err != nil {
log.Fatalln(err)
}
// output message to stdout
fmt.Print("Message Received:", string(msg))
// convert string to uppercase
newMsg := strings.ToUpper(msg)
// sending data
conn.Write([]byte(newMsg + "n"))
}
}Client
package main
import (
"bufio"
"fmt"
"log"
"net"
"os"
)
func main() {
// establishing connection
conn, err := net.Dial("tcp", "127.0.0.1:8081")
if err != nil {
log.Fatalln(err)
}
for {
// reading data from stdin
reader := bufio.NewReader(os.Stdin)
fmt.Print("Text to send: ")
// line by line
text, err := reader.ReadString('n')
if err != nil {
log.Fatalln(err)
}
// sending
fmt.Fprintf(conn, text+"n")
// receiving
msg, err := bufio.NewReader(conn).ReadString('n')
if err != nil {
log.Fatalln(err)
}
// displaying the received response
fmt.Print("Msg from Server: " + msg)
}
}Output
Mechanisms that ensure reliable data transmission and its use
The mechanism
Usage, comment
Checksum
Used to detect bit errors in the transmitted packet
Timer
Timeout interval and indication of its expiration. The latter means that the packet or its acknowledgment has a high probability of being lost during transmission. If the packet is delivered late but not lost (premature timeout), or if the acknowledgment is lost, retransmission results in duplication of the packet on the receiving side.
Sequence number
Used for sequencing data packets transmitted from sender to receiver. Gaps in the sequence numbers of received packets allow the receiver to detect packet loss. Identical sequence numbers for packets indicate that they are duplicates of each other.
Acknowledgment
Generated by the receiving party and indicates to the sending party that the corresponding packet or group of packets has been successfully received. Typically, the acknowledgment contains the sequence numbers of the successfully received packets. Depending on the protocol, individual and group acknowledgments are distinguished.
Negative acknowledgment
Used by the receiver to inform the sender that the packet was received incorrectly. A negative acknowledgment typically includes the sequence number of the packet that was not received correctly.
Window, pipelining
Limit the range of sequence numbers that can be used for packet transmission. Group transmission and handshaking significantly increase the throughput of protocols compared to acknowledgment wait mode. As we will see, the window size can be calculated based on the receiving party's receiving and buffering capabilities, as well as the network load level.
Other examples of using Go for network operations
In .
Source: habr.com
