Introduction
The concept of building a βDigital Substationβ in the electric power industry requires synchronization with an accuracy of 1 Β΅s. Financial transactions also require accuracy in microseconds. In these applications, NTP time accuracy is no longer sufficient.
The PTPv2 synchronization protocol, described by the IEEE 1588v2 standard, allows achieving synchronization accuracy of several tens of nanoseconds. PTPv2 allows synchronization packets to be sent across L2 and L3 networks.
The main areas where PTPv2 is applied are:
- energy;
- control and measuring equipment;
- military-industrial complex;
- telecom;
- financial sector.
This post explains how the PTPv2 synchronization protocol works.
We have more industry experience and see this protocol frequently in power applications. Accordingly, we will do the review with caution .
Why is it needed?
At the moment, STO 34.01-21-004-2019 of PJSC Rosseti and STO 56947007-29.240.10.302-2020 of PJSC FGC UES contain requirements for organizing a process bus with time synchronization via PTPv2.
This is due to the fact that relay protection terminals and measuring devices are connected to the process bus, which transmit instantaneous current and voltage values ββvia the process bus using the so-called SV streams (multicast streams).
Relay terminals use these values ββto implement feeder protections. If the accuracy of measurements over time is small, then some protections may falsely work out.
For example, defenses of absolute selectivity can fall victim to βweakβ time synchronization. Often the logic of such protections is based on a comparison of two values. If the values ββdiffer by a sufficiently large value, then the protection is triggered. If these values ββare measured with a time accuracy of 1 ms, then you can get a big difference where the values ββare actually in the norm, if measured with an accuracy of 1 Β΅s.
PTP Versions
The PTP protocol was originally described in 2002 in the IEEE 1588-2002 standard and was called "Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems". In 2008, an updated IEEE 1588-2008 standard was released that describes PTP Version 2. This version of the protocol improved accuracy and stability, but was not backward compatible with the first version of the protocol. Also, in 2019, a version of the IEEE 1588-2019 standard was released describing PTP v2.1. This version adds minor improvements to PTPv2 and is backward compatible with PTPv2.
In other words, we have the following picture with versions:
PTPv1
(IEEE 1588-2002)
PTPv2
(IEEE 1588-2008)
PTPv2.1
(IEEE 1588-2019)
PTPv1 (IEEE 1588-2002)
β
Incompatible
Incompatible
PTPv2 (IEEE 1588-2008)
Incompatible
β
Compatible
PTPv2.1 (IEEE 1588-2019)
Incompatible
Compatible
β
But, as always, there are nuances.
The incompatibility between PTPv1 and PTPv2 means that a PTPv1 enabled device will not be able to synchronize from an accurate clock running on PTPv2. They use different message formats for synchronization.
But it is still possible to combine devices with PTPv1 and devices with PTPv2 on the same network. To do this, some manufacturers allow you to select the protocol version on the ports of the boundary clock. That is, the boundary clocks can synchronize over PTPv2 while synchronizing other clocks connected to them over both PTPv1 and PTPv2.
PTP devices. What are and how do they differ?
The IEEE 1588v2 standard describes several types of devices. All of them are shown in the table.
Devices communicate with each other over a LAN using PTP.
PTP devices are called clocks. All clocks take the exact time from the grandmaster clock.
There are 5 types of watches:
Grandmaster clock (Grandmaster clock)
The main source of accurate time. Often equipped with an interface for connecting GPS.
Ordinary Clock
Single port device that can be master (master clock) or slave (slave clock)
Master clock (master)
Are the source of the exact time to which other clocks are synchronized
Slave clock
End device that synchronizes from the master clock
Boundary Clock
A device with multiple ports that can be a master or a slave.
That is, these clocks can synchronize from the upstream master clock and synchronize downstream slave clocks.
End-to-end Transparent Clock (Transparent clock operating in End-to-End mode)
A device with multiple ports that is neither a clock master nor a clock slave. It transmits PTP data between two watches.
When transmitting data, the transparent clock corrects all PTP messages.
Correction occurs by adding the delay time on this device to the correction field in the header of the transmitted message.
Peer-to-Peer Transparent Clock
A device with multiple ports that is neither a clock master nor a clock slave.
It transmits PTP data between two watches.
When transmitting data, the transparent clock corrects all Sync and Follow_Up PTP messages (more about them below).
The correction is achieved by adding to the correction field of the transmitted packet a delay on the transmitting device and a delay on the data transmission channel.
Management Node
A device that configures and diagnoses other watches
Master and slave clocks are synchronized using timestamps in PTP messages. There are two types of messages in the PTP protocol:
- Event Messages are synchronized messages that involve the generation of a timestamp at the time the message is sent and at the time it is received.
- General Messages - these messages do not require timestamps, but may contain timestamps for related messages
Event Messages
General Messages
Sync
Delay_Req
Pdelay_Req
Pdelay_Resp
Announce
Follow_Up
Delay_Resp
Pdelay_Resp_Follow_Up
Management
signalling
Each message type will be discussed in more detail below.
Basic sync issues
When a synchronization packet is transmitted over a local network, it is delayed on the switch and in the data transmission channel. Any switch will give a delay of about 10 Β΅s, which is unacceptable for PTPv2. After all, we need to get an accuracy of 1 ΞΌs on the final device. (This is for energy purposes. Other applications may require more accuracy.)
IEEE 1588v2 describes several algorithms that allow you to fix the time delay and correct it.
The algorithm works
During normal operation, the protocol operates in two phases.
- Phase 1 - setting up the master clock - slave clock hierarchy.
- Phase 2 - clock synchronization using the End-to-End or Peer-to-Peer mechanism.
Phase 1 - Establishing the Master-Slave Hierarchy
Each port of a regular or boundary clock has a certain number of states (slave clock and master clock). The standard describes the transition algorithm between these states. In programming, such an algorithm is called a state machine or a state machine (for more details, see the Wiki).
This state machine uses the Best Master Clock Algorithm (BMCA) to set the master when two clocks are connected.
This algorithm allows the watch to assume the responsibility of a grandmaster watch when the superior grandmaster watch loses its GPS signal, disconnects from the network, etc.
State transitions according to the BMCA are summarized in the following diagram:
Information about the clock at the other end of the "wire" is sent in a special message (Announce message). When this information is received, the state machine algorithm works out and a comparison is made which clock is better. The port on the best clock becomes the master clock.
A simple hierarchy is shown in the diagram below. Paths 1, 2, 3, 4, 5 may contain transparent clocks (Transparent clock), but they do not participate in the establishment of the Master Clock - Slave Clock hierarchy.
Phase 2 - Synchronization of normal and boundary clocks
Once the Master Clock - Slave Clock hierarchy is established, the normal and boundary clock synchronization phase begins.
To synchronize, the master clock sends a message to the slave clock containing a timestamp.
The leading clock can be:
- single-stage;
- two-stage.
A single-stage synchronization clock sends one Sync message.
A two-stage clock uses two messages for synchronization - Sync and Follow_Up.
Two mechanisms can be used for the synchronization phase:
- Delay request-response mechanism.
- Peer delay measurement mechanism.
To begin with, we will consider these mechanisms in the simplest case - when transparent clocks are not used.
Delay request-response mechanism
The mechanism involves two steps:
- Measurement of the delay in the transmission of a message between the master clock and the slaves. This is done using the delay request-response mechanism.
- Time offset correction is performed.
Delay measurement
t1 - Time of sending the Sync message by the master clock; t2 - Time of receipt of the Sync message by the slave clock; t3 β Delay request sending time (Delay_Req) ββby the slave clock; t4 - The time the Delay_Req was received by the master clock.
When the slave clocks know the times t1, t2, t3 and t4, they can calculate the average delay in sending the synchronization message (tmpd). It is calculated as follows:
When transmitting the Sync and Follow_Up messages, the time delay from the master to the slave is calculated - t-ms.
When sending Delay_Req and Delay_Resp messages, the time delay from the slave to the master is calculated - t-sm.
If there is any asymmetry between these two values, then a time offset correction error occurs. The error is due to the fact that the calculated delay is the average of the t-ms and t-sm delays. If the delays are not equal to each other, then we will adjust the time inaccurately.
Accurate Time Offset Correction
Once the delay between the master clock and the slave clock is known, the slave clock performs a time correction.
The slave clock uses the Sync message and the optional Follow_Up message to calculate the exact time offset when a packet is sent from the master clock to the slave clocks. The shift is calculated using the following formula:
Peer delay measurement mechanism
This mechanism also uses two steps for synchronization:
- Devices measure the time delay to all neighbors through all ports. To do this, they use a peer delay mechanism.
- Time shift correction.
Measuring Latency Between Peer-to-Peer Devices
Latency between peer-to-peer ports is measured using the following messages:
When port 1 knows the times t1, t2, t3 and t4, it can calculate the average delay (tmld). It is calculated using the following formula:
The port then uses this value when calculating the adjustment field for each Sync message or optional Follow_Up message that passes through the device.
The resulting delay will be equal to the sum of the delay in transmission through this device, the average delay in transmission through the data channel, and the delay already contained in this message, enabled on upstream devices.
The messages Pdelay_Req, Pdelay_Resp and the optional Pdelay_Resp_Follow_Up allow you to get the delay from the master to the slave and from the slave to the master (circular).
Any asymmetry between these two values ββwill introduce a time offset correction error.
Correction of the shift of the exact time
The slave clock uses the Sync message and the optional Follow_Up message to calculate the exact time offset when a packet is sent from the master clock to the slave clocks. The shift is calculated using the following formula:
Benefits Adjusting the peer-to-peer mechanism - the time delay of each Sync or Follow_Up message is calculated as it passes through the network. Therefore, changing the transmission path will not affect the accuracy of the correction in any way.
When using this mechanism, time synchronization does not require the calculation of the time delay on the path traveled by the synchronization packet, as is done with the basic exchange. Those. Delay_Req and Delay_Resp messages are not sent. In this method, the delay between master and slave clocks is simply added up in the adjustment field of each Sync or Follow_Up message.
Another benefit is that the master clock is offloaded from having to process Delay_Req messages.
Transparent watch modes
Accordingly, these were parsed simple examples. Now suppose that switches appear in the synchronization path.
If you use switches without PTPv2 support, then the synchronization packet will be delayed on the switch by about 10 Β΅s.
Switches that support PTPv2 are called transparent clocks in IEEE 1588v2 terminology. Transparent clocks are not synchronized from the master clock and do not participate in the Master clock - Slave clock hierarchy, but when transmitting synchronization messages, they remember how long the message was delayed on them. This allows you to adjust the time delay.
Transparent watches can work in two modes:
- end-to-end.
- Peer to peer.
End-to-End (E2E)
The transparent E2E clock broadcasts Sync messages and accompanying Follow_Up messages on all ports. Even those that are blocked by some protocols (for example, RSTP).
The switch remembers the timestamp when a Sync packet (Follow_Up) was received on a port and when it was sent out of the port. Based on these two timestamps, the time it takes the switch to process the message is calculated. In the standard, this time is called residence time.
The processing time is added to the correctionField of the Sync (single-step clock) or Follow_Up (two-step clock) message.
The transparent E2E clock measures the processing time for Sync and Delay_Req messages passing through the switch. But it is important to understand that the time delay between the master clock and the slave clock is calculated using the delay request-response mechanism. If the master clock changes or the path from the master clock to the slave clock changes, then the delay is measured again. This increases the transition time in case of network changes.
The transparent P2P clock, in addition to measuring the message processing time by the switch, measures the delay on the data transmission channel to the nearest neighbor using the neighbor delay measurement mechanism.
Latency is measured on each link in both directions, including links that are blocked by some protocol (eg RSTP). This allows you to immediately calculate the new delay on the synchronization path if the grandmaster clock or the network topology has changed.
Switches' message processing time and latency are accumulated when sending Sync or Follow_Up messages.
Types of PTPv2 support by switches
Switches can support PTPv2:
- programmatically;
- hardware.
When implementing the PTPv2 protocol in software, the switch requests a timestamp from the firmware. The problem is that the firmware works cyclically, and you will have to wait until it finishes the current cycle, takes the request into processing and, at the end of the next cycle, issues a timestamp. All this will also take time, and we will get a delay, albeit not as significant as without software support for PTPv2.
Only hardware support for PTPv2 allows you to maintain the required accuracy. In this case, the issuance of a timestamp is performed by a special ASIC, which is installed on the port.
Message Format
All PTP messages consist of the following fields:
- Header - 34 bytes.
- Body - the size depends on the message type.
- Suffix is ββoptional.
Header
The Header field is the same for all PTP messages. Its size is 34 bytes.
Header field format:
messageType β contains the type of message being sent, such as Sync, Delay_Req, PDelay_Req, etc.
messageLength β contains the full length of the PTP message, including header, body, and suffix (but excluding padding bytes).
domainNumber β specifies which PTP domain the message belongs to.
Domain Name - these are several different clocks collected in one logical group and synchronized from one master clock, but not necessarily synchronized with clocks belonging to another domain.
flags β This field contains various flags to identify the status of the message.
correctionField β contains the delay time in nanoseconds. The delay time includes the delay when transmitting through a transparent clock, as well as the delay when transmitting through a channel when using Peer-to-Peer mode.
sourcePortIdentity β this field contains information about the port from which the message was originally sent.
sequenceID β contains an identification number for individual messages.
controlField β field-artifact=) It remains from the first version of the standard and contains information about the type of this message. Essentially the same as messageType but with fewer options.
logMessageInterval β This field is determined by the message type.
Bodysuit
As discussed above, there are several types of messages. These types are described below:
Announce message
The Announce message is used to "tell" other watches within the same domain about its settings. This message allows you to set the Master Clock - Slave Clock hierarchy.
Sync message
A synchronization (Sync) message is sent by the master clock and contains the time of the master clock at the time the Sync message was generated. If the master clock is two-stage, then the timestamp in the Sync message will be set to 0, and the current timestamp will be sent in the conjugate Follow_Up message. The Sync message is used for both latency measurement mechanisms.
The message is transmitted using Multicast. Optionally, you can use Unicast.
Delay_Req message
The format of the Delay_Req message is identical to the Sync message. The slave clock sends a Delay_Req. It contains the time the Delay_Req was sent by the slave clock. This message is only used for the delay request-response mechanism.
The message is transmitted using Multicast. Optionally, you can use Unicast.
Follow_Up message
The Follow_Up message is optionally sent by the master clock and contains the send time sync messages master. The Follow_Up message is sent only by two-stage master clocks.
The Follow_Up message is used for both delay measurement mechanisms.
The message is transmitted using Multicast. Optionally, you can use Unicast.
Delay_Resp Message
The Delay_Resp message is sent by the master clock. It contains the time the Delay_Req was received by the master clock. This message is only used for the delay request-response mechanism.
The message is transmitted using Multicast. Optionally, you can use Unicast.
Pdelay_Req Message
The Pdelay_Req message is sent by the device that requests the delay. It contains the time the message was sent from this device's port. Pdelay_Req is only used for the neighbor delay measurement mechanism.
Pdelay_Resp Message
The Pdelay_Resp message is sent by the device that received the delay request. It contains the time at which the Pdelay_Req message was received by this device. Pdelay_Resp messages are only used for the neighbor delay measurement mechanism.
Pdelay_Resp_Follow_Up message
The Pdelay_Resp_Follow_Up message is optionally sent by the device that received the delay request. It contains the time when the Pdelay_Req message was received by this device. The Pdelay_Resp_Follow_Up message is only sent by the two-stage master clock.
Also, this message can be used for runtime instead of a timestamp. The execution time is the time from the receipt of the Pdelay-Req to the sending of the Pdelay_Resp.
Pdelay_Resp_Follow_Up are only used for neighbor delay measurement mechanism.
Management messages (Message Management)
PTP control messages are required to transfer information between one or more clocks and the control node.
Transfer to LV
A PTP message can be sent at two levels:
- Network - as part of IP data.
- Channel - as part of an Ethernet frame.
PTP message transmission over UDP over IP over Ethernet
PTP over UDP over Ethernet
Profiles
PTP has a lot of "flexible" parameters that need to be configured. For example:
- BMC options.
- Delay measurement mechanism.
- Intervals and initial values ββof all configurable parameters, etc.
And despite the fact that we said earlier that PTPv2 devices are compatible with each other, in a good way this is not so. Devices must have the same settings in order to communicate.
Therefore, there are so-called PTPv2 profiles. Profiles are groups of configured settings and defined protocol restrictions so that you can implement time synchronization for a particular application.
The IEEE 1588v2 standard itself describes only one profile, the βDefault Profileβ. All other profiles are created and described by various organizations and associations.
For example, the PTPv2 Power Profile was created by the Power Systems Relaying Committee and the Substation Committee of the IEEE Power and Energy Society. The profile itself is called IEEE C37.238-2011.
The profile describes that PTP can be sent:
- Only via L2 networks (i.e. Ethernet, HSR, PRP, not IP).
- Messages are transmitted only by Multicast.
- The Peer delay measurement mechanism is used as the delay measurement mechanism.
The default domain is 0, the recommended domain is 93.
The philosophy behind the creation of C37.238-2011 was to reduce the number of optional features and leave only the necessary features for reliable communication between devices and improve system stability.
Also, the frequency of message transmission is defined:
In fact, only one parameter is available for selection - the type of master clock (single-stage or two-stage).
The accuracy should be no more than 1 Β΅s. In other words, one synchronization path can contain a maximum of 15 transparent clocks or three boundary clocks.
Source: habr.com