As a developer library (GOST cryptographic primitives in pure Python), I often receive questions about how to implement a simple secure messaging exchange quickly. Many believe that applied cryptography is fairly straightforward, and that a call to .encrypt() on a block cipher is sufficient for secure transmission over a communication channel. Others think that applied cryptography is reserved for a select few, and itβs acceptable that rich companies like Telegram with their Olympic-level mathematicians a secure protocol.
All of this prompted me to write this article to demonstrate that implementing cryptographic protocols and secure IM isn't as daunting a task as it seems. However, thereβs no need to invent your own authentication and key agreement protocols.
The article will detail an , , instant messenger with authentication and key agreement protocol (on the basis of which ) is implemented, using exclusively GOST cryptographic algorithms from the PyGOST library and ASN.1 message encoding from the (which I previously ). A necessary condition: it should be simple enough to write from scratch in one evening (or working day), otherwise itβs no longer a simple program. There will surely be bugs, excessive complexities, and shortcomings, plus it's my first program using the asyncio library.
IM Design
First, we need to understand what our IM will look like. For simplicity, letβs assume itβs a peer-to-peer network without any participant discovery. We will manually specify the address and port to connect to communicate with our interlocutor.
I understand that, at this moment, the assumption of direct connectivity between any two arbitrary computers is a significant limitation in the practical use of IM. But the more developers implement various NAT traversal workarounds, the longer we will remain in the IPv4 Internet, grappling with the unfortunate probability of connectivity between arbitrary computers. How much longer can we endure the absence of IPv6 at home and at work?
We will have a friend-to-friend network: all possible interlocutors must be known in advance. First, this greatly simplifies everything: we introduce ourselves, find or do not find a name/key, disconnect or continue working, knowing our partner. Second, in general, this is safe and eliminates many attacks.
The IM interface will be close to classic solutions. , which I really like for their minimalism and Unix-way philosophy. The IM program creates a directory for each interlocutor with three Unix domain sockets:
- in β this is where the messages sent to the interlocutor are recorded;
- out β this is where the messages received from the interlocutor are read;
- state β by reading from it, we learn whether the interlocutor is currently connected, and their address/port of connection.
Additionally, a conn socket is created; by writing the host port into it, we initiate a connection to the remote interlocutor.
|-- alice
| |-- in
| |-- out
| `-- state
|-- bob
| |-- in
| |-- out
| `-- state
`- conn
This approach allows for independent implementations of the IM transport and user interface, since thereβs no pleasing everyone when it comes to taste and preferences. By using and/or , you can achieve a multi-window interface with syntax highlighting. And with , you can create a GNU Readline-compatible input line for messages.
In reality, suckless projects use FIFO files. Personally, I could not figure out how to work with files concurrently in asyncio without my own dedicated thread support (for such things I have long used the language ). Therefore, I decided to use Unix domain sockets. Unfortunately, this prevents executing echo 2001:470:dead::babe 6666 > conn. I solved this problem by using : echo 2001:470:dead::babe 6666 | socat β UNIX-CONNECT:conn, socat READLINE UNIX-CONNECT:alice/in.
The initial unsafe protocol
TCP is used as the transport: it guarantees delivery and its order. UDP does not guarantee either (which would be useful when applying cryptography), and support for is not available in Python out of the box.
Unfortunately, TCP does not have the concept of a message, only a stream of bytes. Therefore, we need to come up with a format for messages so that they can be distinguished from each other in this stream. We can agree to use a newline character. This will suffice for now, but when we start encrypting our messages, this character can appear anywhere in the ciphertext. Therefore, protocols that initially send the length of the message in bytes are popular in networks. For example, Python comes with xdrlib, which allows you to work with such a format. .
We will not work correctly and efficiently with TCP reading β letβs simplify the code. We read data from the socket in an infinite loop until we decode the complete message. JSON and XML can be used as a format for this approach. However, when cryptography is added, the data will need to be signed and authenticated β which will require a byte-for-byte identical representation of objects, something that JSON/XML do not ensure (the dumps result may differ).
XDR is suitable for this task, however, I choose ASN.1 with DER encoding and the library, as we will have high-level objects that are often more pleasant and convenient to work with. Unlike schemaless , or , ASN.1 will automatically check the data against a rigidly defined schema.
# Msg ::= CHOICE {
# text MsgText,
# handshake [0] EXPLICIT MsgHandshake }
class Msg(Choice):
schema = ((
("text", MsgText()),
("handshake", MsgHandshake(expl=tag_ctxc(0))),
))
# MsgText ::= SEQUENCE {
# text UTF8String (SIZE(1..MaxTextLen))}
class MsgText(Sequence):
schema = ((
("text", UTF8String(bounds=(1, MaxTextLen))),
))
# MsgHandshake ::= SEQUENCE {
# peerName UTF8String (SIZE(1..256)) }
class MsgHandshake(Sequence):
schema = ((
("peerName", UTF8String(bounds=(1, 256))),
))
The incoming message will be Msg: either a text MsgText (for now with one text field), or a handshake message MsgHandshake (which transmits the interlocutor's name). It may seem overly complicated now, but it is a foundation for the future.
βββββββ βββββββ
βPeerAβ βPeerBβ
ββββ¬βββ ββββ¬βββ
βMsgHandshake(IdA) β
ββββββββββββββββββ>|
β β
βMsgHandshake(IdB) β
β|
β β
β MsgText() β
β<βββββββββββββββββ|
β β
IM without cryptography
As I mentioned, the asyncio library will be used for all socket operations. Letβs declare what we expect when we start:
parser = argparse.ArgumentParser(description="GOSTIM")
parser.add_argument(
"--our-name",
required=True,
help="Our peer name",
)
parser.add_argument(
"--their-names",
required=True,
help="Their peer names, comma-separated",
)
parser.add_argument(
"--bind",
default="::1",
help="Address to listen on",
)
parser.add_argument(
"--port",
type=int,
default=6666,
help="Port to listen on",
)
args = parser.parse_args()
OUR_NAME = UTF8String(args.our_name)
THEIR_NAMES = set(args.their_names.split(","))
A custom name is specified (βour-name alice). All expected peers are listed, separated by commas (βtheir-names bob,eve). A directory with Unix sockets is created for each peer, along with a coroutine for each in, out, state:
for peer_name in THEIR_NAMES:
makedirs(peer_name, mode=0o700, exist_ok=True)
out_queue = asyncio.Queue()
OUT_QUEUES[peer_name] = out_queue
asyncio.ensure_future(asyncio.start_unix_server(
partial(unixsock_out_processor, out_queue=out_queue),
path.join(peer_name, "out"),
))
in_queue = asyncio.Queue()
IN_QUEUES[peer_name] = in_queue
asyncio.ensure_future(asyncio.start_unix_server(
partial(unixsock_in_processor, in_queue=in_queue),
path.join(peer_name, "in"),
))
asyncio.ensure_future(asyncio.start_unix_server(
partial(unixsock_state_processor, peer_name=peer_name),
path.join(peer_name, "state"),
))
asyncio.ensure_future(asyncio.start_unix_server(unixsock_conn_processor, "conn"))
Messages incoming from the user via the in socket are sent to the IN_QUEUES queues:
async def unixsock_in_processor(reader, writer, in_queue: asyncio.Queue) -> None:
while True:
text = await reader.read(MaxTextLen)
if text == b"":
break
await in_queue.put(text.decode("utf-8"))
Messages from peers are sent to the OUT_QUEUES queues, from which data is written to the out socket:
async def unixsock_out_processor(reader, writer, out_queue: asyncio.Queue) -> None:
while True:
text = await out_queue.get()
writer.write(("[%s] %s" % (datetime.now(), text)).encode("utf-8"))
await writer.drain()
When reading from the state socket, the program looks for the peer's address in the PEER_ALIVE dictionary. If there is no connection to the peer yet, an empty string is recorded.
async def unixsock_state_processor(reader, writer, peer_name: str) -> None:
peer_writer = PEER_ALIVES.get(peer_name)
writer.write(
b"" if peer_writer is None else (" ".join([
str(i) for i in peer_writer.get_extra_info("peername")[:2]
]).encode("utf-8") + b"n")
)
await writer.drain()
writer.close()
When the address is written to the conn socket, the connection initiator function is launched:
async def unixsock_conn_processor(reader, writer) -> None:
data = await reader.read(256)
writer.close()
host, port = data.decode("utf-8").split(" ")
await initiator(host=host, port=int(port))
Let's take a look at the initiator. First, it obviously opens a connection to the specified host/port and sends a handshake message with its name:
130 async def initiator(host, port):
131 _id = repr((host, port))
132 logging.info("%s: dialing", _id)
133 reader, writer = await asyncio.open_connection(host, port)
134 # Handshake message {{{
135 writer.write(Msg(("handshake", MsgHandshake((
136 ("peerName", OUR_NAME),
137 )))).encode())
138 # }}}
139 await writer.drain()
Then, it waits for a response from the remote side. It attempts to decode the incoming response according to the Msg ASN.1 scheme. We assume that the entire message will be sent in a single TCP segment and that we will receive it atomically when calling .read(). We check that we received the handshake message.
141 # Wait for Handshake message {{{
142 data = await reader.read(256)
143 if data == b"":
144 logging.warning("%s: no answer, disconnecting", _id)
145 writer.close()
146 return
147 try:
148 msg, _ = Msg().decode(data)
149 except ASN1Error:
150 logging.warning("%s: undecodable answer, disconnecting", _id)
151 writer.close()
152 return
153 logging.info("%s: got %s message", _id, msg.choice)
154 if msg.choice != "handshake":
155 logging.warning("%s: unexpected message, disconnecting", _id)
156 writer.close()
157 return
158 # }}}
We check if the incoming peer name is known to us. If not, we terminate the connection. We verify whether we already had a connection with them (the peer gave the command to connect again) and close it. Python strings containing the message text are placed in the IN_QUEUES queue, but there is a special value None, indicating to the msg_sender coroutine to stop working, so it forgets about its writer associated with the outdated TCP connection.
159 msg_handshake = msg.value
160 peer_name = str(msg_handshake["peerName"])
161 if peer_name not in THEIR_NAMES:
162 logging.warning("unknown peer name: %s", peer_name)
163 writer.close()
164 return
165 logging.info("%s: session established: %s", _id, peer_name)
166 # Run text message sender, initialize transport decoder {{{
167 peer_alive = PEER_ALIVES.pop(peer_name, None)
168 if peer_alive is not None:
169 peer_alive.close()
170 await IN_QUEUES[peer_name].put(None)
171 PEER_ALIVES[peer_name] = writer
172 asyncio.ensure_future(msg_sender(peer_name, writer))
173 # }}}
msg_sender handles outgoing messages (queued from the in socket), serializes them into a MsgText message, and sends them over the TCP connection. It may be interrupted at any moment β this is something we explicitly intercept.
async def msg_sender(peer_name: str, writer) -> None:
in_queue = IN_QUEUES[peer_name]
while True:
text = await in_queue.get()
if text is None:
break
writer.write(Msg(("text", MsgText((
("text", UTF8String(text)),
)))).encode())
try:
await writer.drain()
except ConnectionResetError:
del PEER_ALIVES[peer_name]
return
logging.info("%s: sent %d characters message", peer_name, len(text))
At the end, the initiator enters an infinite loop reading messages from the socket. It checks whether these are text messages and places them in the OUT_QUEUES queue, from which they will be sent to the out socket of the corresponding conversation partner. Why not just use .read() and decode the message? Because thereβs a possibility that multiple messages from the user may be aggregated in the operating system's buffer and sent in a single TCP segment. We can decode the first message, but the buffer may still contain part of the subsequent one. In any abnormal situation, we close the TCP connection and stop the msg_sender coroutine (by sending None to the OUT_QUEUES queue).
174 buf = b""
175 # Wait for test messages {{{
176 while True:
177 data = await reader.read(MaxMsgLen)
178 if data == b"":
179 break
180 buf += data
181 if len(buf) > MaxMsgLen:
182 logging.warning("%s: max buffer size exceeded", _id)
183 break
184 try:
185 msg, tail = Msg().decode(buf)
186 except ASN1Error:
187 continue
188 buf = tail
189 if msg.choice != "text":
190 logging.warning("%s: unexpected %s message", _id, msg.choice)
191 break
192 try:
193 await msg_receiver(msg.value, peer_name)
194 except ValueError as err:
195 logging.warning("%s: %s", err)
196 break
197 # }}}
198 logging.info("%s: disconnecting: %s", _id, peer_name)
199 IN_QUEUES[peer_name].put(None)
200 writer.close()
66 async def msg_receiver(msg_text: MsgText, peer_name: str) -> None:
67 text = str(msg_text["text"])
68 logging.info("%s: received %d characters message", peer_name, len(text))
69 await OUT_QUEUES[peer_name].put(text)
Let's return to the main code. After creating all the coroutines at the program start, we launch the TCP server. For each established connection, it creates a responder coroutine.
logging.basicConfig(
level=logging.INFO,
format="%(levelname)s %(asctime)s: %(funcName)s: %(message)s",
)
loop = asyncio.get_event_loop()
server = loop.run_until_complete(asyncio.start_server(responder, args.bind, args.port))
logging.info("Listening on: %s", server.sockets[0].getsockname())
loop.run_forever()
The responder is similar to the initiator and mirrors all the same actions, but the infinite loop for reading messages starts immediately for simplicity. Currently, the handshake protocol sends one message from each side, but in the future, there will be two from the connection initiator, after which text messages can be sent immediately.
72 async def responder(reader, writer):
73 _id = writer.get_extra_info("peername")
74 logging.info("%s: connected", _id)
75 buf = b""
76 msg_expected = "handshake"
77 peer_name = None
78 while True:
79 # Read until we get Msg message {{{
80 data = await reader.read(MaxMsgLen)
81 if data == b"":
82 logging.info("%s: closed connection", _id)
83 break
84 buf += data
85 if len(buf) > MaxMsgLen:
86 logging.warning("%s: max buffer size exceeded", _id)
87 break
88 try:
89 msg, tail = Msg().decode(buf)
90 except ASN1Error:
91 continue
92 buf = tail
93 # }}}
94 if msg.choice != msg_expected:
95 logging.warning("%s: unexpected %s message", _id, msg.choice)
96 break
97 if msg_expected == "text":
98 try:
99 await msg_receiver(msg.value, peer_name)
100 except ValueError as err:
101 logging.warning("%s: %s", err)
102 break
103 # Process Handshake message {{{
104 elif msg_expected == "handshake":
105 logging.info("%s: got %s message", _id, msg_expected)
106 msg_handshake = msg.value
107 peer_name = str(msg_handshake["peerName"])
108 if peer_name not in THEIR_NAMES:
109 logging.warning("unknown peer name: %s", peer_name)
110 break
111 writer.write(Msg(("handshake", MsgHandshake((
112 ("peerName", OUR_NAME),
113 )))).encode())
114 await writer.drain()
115 logging.info("%s: session established: %s", _id, peer_name)
116 peer_alive = PEER_ALIVES.pop(peer_name, None)
117 if peer_alive is not None:
118 peer_alive.close()
119 await IN_QUEUES[peer_name].put(None)
120 PEER_ALIVES[peer_name] = writer
121 asyncio.ensure_future(msg_sender(peer_name, writer))
122 msg_expected = "text"
123 # }}}
124 logging.info("%s: disconnecting", _id)
125 if msg_expected == "text":
126 IN_QUEUES[peer_name].put(None)
127 writer.close()
Secure Protocol
It's time to secure our communication. What do we mean by security and what do we want:
- the confidentiality of transmitted messages;
- the authenticity and integrity of transmitted messages β any alteration must be detected;
- protection against replay attacks β the loss or repetition of messages must be detected (and we decide to terminate the connection);
- the identification and authentication of communicators by predefined public keys β we previously decided to create a friend-to-friend network. Only after authentication will we know who we are communicating with;
- the presence of properties (PFS) β the compromise of our long-lived signature key should not allow access to all previous conversations. The recorded intercepted traffic becomes useless;
- The validity of messages (transport and handshake) is limited to a single TCP session. Inserting correctly signed/authenticated messages from another session (even with the same interlocutor) should not be possible;
- The passive observer should not see any user identifiers, long-lived public keys being transmitted, or their hashes. Some anonymity from the passive observer.
Surprisingly, this minimum is what almost everyone wants in any handshake protocol, yet very little of what is listed is ultimately fulfilled for 'homegrown' protocols. So let's not invent something new right now. I would definitely recommend using to build protocols, but let's choose something simpler.
The two most popular protocols are:
- β a complex protocol with a long history of bugs, flaws, vulnerabilities, poor design, complexity, and shortcomings (however, TLS 1.3 has little to do with this). But we won't consider it due to its overcomplexity.
- with β have no serious cryptographic issues, though they are also not simple. If you read about IKEv1 and IKEv2, their origins are , ISO/IEC IS 9798-3 and SIGMA (SIGn-and-MAc) protocols β quite simple to implement in one evening.
What makes SIGMA, as the latest link in the evolution of STS/ISO protocols, good? It satisfies all our requirements (including the 'hiding' of interlocutor identifiers), has no known cryptographic problems. It is minimalist β removing at least one element from the protocol message will render it unsafe.
Letβs walk from the simplest homegrown protocol to SIGMA. The most basic operation of interest to us is : a function, where both participants will obtain the same value that can be used as a symmetric key. Without going into details: each party generates a temporary (used only within one session) key pair (public and private keys), exchanges public keys, calls the agreement function, to which they pass their private key and the public key of their interlocutor.
βββββββ βββββββ
βPeerAβ βPeerBβ
ββββ¬βββ ββββ¬βββ
β IdA, PubA β ββββββββββββββββββββββ
ββββββββββββββββγβ βPrvA, PubA = DHgen()β
β β ββββββββββββββββββββββ
β IdB, PubB β ββββββββββββββββββββββ
βγββββββββββββββββ βPrvB, PubB = DHgen()β
β β ββββββββββββββββββββββ
βββββ βββββββββ§βββββββββββββ
β βKey = DH(PrvA, PubB)β
γββββ βββββββββ€βββββββββββββ
β β
β β
Anyone can intercept and substitute public keys with their own β there is no authentication of peers in this protocol. Let's add a signature with long-lived keys.
βββββββ βββββββ
βPeerAβ βPeerBβ
ββββ¬βββ ββββ¬βββ
βIdA, PubA, sign(SignPrvA, (PubA)) β βββββββββββββββββββββββββββββ
ββββββββββββββββββββββββββββββββββγβ βSignPrvA, SignPubA = load()β
β β βPrvA, PubA = DHgen() β
β β βββββββββββββββββββββββββββββ
βIdB, PubB, sign(SignPrvB, (PubB)) β βββββββββββββββββββββββββββββ
βγββββββββββββββββββββββββββββββββββ βSignPrvB, SignPubB = load()β
β β βPrvB, PubB = DHgen() β
β β βββββββββββββββββββββββββββββ
βββββ βββββββββββββββββββββββ β
β βverify(SignPubB, ...)β β
γββββ βKey = DH(PrvA, PubB) β β
β βββββββββββββββββββββββ β
β β
Such a signature will not work, as it is not tied to a specific session. Such messages will work for sessions with other participants. The entire context must be signed. This also requires adding another message from A.
Moreover, it is critical to add your own identifier to the signature; otherwise, we may replace IdXXX and re-sign the message with another known peer's key. To prevent , it is necessary for the elements under the label to be located in clearly defined places according to their meaning: if A signs (PubA, PubB), then B must sign (PubB, PubA). This also highlights the importance of selecting the structure and format of serialized data. For instance, sets in ASN.1 DER encoding are sorted: SET OF(PubA, PubB) will be equivalent to SET OF(PubB, PubA).
βββββββ βββββββ βPeerAβ βPeerBβ ββββ¬βββ ββββ¬βββ β IdA, PubA β βββββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββββββββββββββββββ βSignPrvA, SignPubA = load()β β β βPrvA, PubA = DHgen() β β β βββββββββββββββββββββββββββββ βIdB, PubB, sign(SignPrvB, (IdB, PubA, PubB)) β βββββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββββββββββββββββββ βSignPrvB, SignPubB = load()β β β βPrvB, PubB = DHgen() β β β βββββββββββββββββββββββββββββ β sign(SignPrvA, (IdA, PubB, PubA)) β βββββββββββββββββββββββ βββββββββββββββββββββββββββββββββββββββββββββββ βverify(SignPubB, ...)β β β βKey = DH(PrvA, PubB) β β β βββββββββββββββββββββββ β β
However, we still have not "proven" that we have derived the same shared key for this session. In principle, we could do without this step β the first transport message would be invalid, but we want to ensure that once the handshake is complete, everything is truly agreed upon. At this point, we have the ISO/IEC IS 9798-3 protocol in hand.
We could sign the derived key itself. This is risky, as it is possible that the signature algorithm may have leaks (even if it's just bits-for-signature, they are still leaks). We can sign the hash of the derived key, but a leak of even the hash of the derived key may have value during a brute-force attack on the derivation function. SIGMA uses a MAC function to authenticate the sender's identifier.
βββββββ βββββββ βPeerAβ βPeerBβ ββββ¬βββ ββββ¬βββ β IdA, PubA β βββββββββββββββββββββββββββββ ββββββββββββββββββββββββββββββββββββββββββββββββββ>| βSignPrvA, SignPubA = load()β β β βPrvA, PubA = DHgen() β β β βββββββββββββββββββββββββββββ βIdB, PubB, sign(SignPrvB, (PubA, PubB)), MAC(IdB) β βββββββββββββββββββββββββββββ β| βverify(Key, IdB) β β β βverify(SignPubB, ...)β β β βββββββββββββββββββββββ β β
As an optimization, some may want to reuse their ephemeral keys (which, of course, is detrimental to PFS). For example, we generated a key pair, attempted to connect, but TCP was not available or was interrupted somewhere in the middle of the protocol. It's a waste to spend the entropy and CPU resources on a new pair. Therefore, we introduce what is called a cookie β a pseudo-random value that protects against potential random replay attacks when reusing ephemeral public keys. Due to the binding between the cookie and the ephemeral public key, the public key of the opposite participant can be omitted from the signature as unnecessary.
βββββββ βββββββ βPeerAβ βPeerBβ ββββ¬βββ ββββ¬βββ β IdA, PubA, CookieA β βββββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ>β βSignPrvA, SignPubA = load()β β β βPrvA, PubA = DHgen() β β β βββββββββββββββββββββββββββββ βIdB, PubB, CookieB, sign(SignPrvB, (CookieA, CookieB, PubB)), MAC(IdB) β βββββββββββββββββββββββββββββ ββ βverify(Key, IdB) β β β βverify(SignPubB, ...)β β β βββββββββββββββββββββββ β β
Finally, we want to keep the privacy of our interlocutor's identifiers from passive observers. To achieve this, SIGMA suggests first exchanging ephemeral keys, deriving a shared key to encrypt authentication and identification messages. SIGMA describes two options:
- SIGMA-I β protects the initiator from active attacks and the responder from passive ones: the initiator authenticates the responder, and if something doesn't match, they do not reveal their identification. The responder, however, discloses their identification if an active protocol is initiated with them. A passive observer learns nothing;
SIGMA-R β protects the responder from active attacks and the initiator from passive ones. Everything is reversed, but this protocol involves four handshake messages.We choose SIGMA-I as it is more similar to what we expect from traditional client-server systems: the client is only recognized by the authenticated server, while the server knows everything anyway. Moreover, it is simpler to implement due to the reduced number of handshake messages. All we add to the protocol is the encryption of part of the message and the transfer of identifier A into the encrypted part of the last message:
βββββββ βββββββ βPeerAβ βPeerBβ ββββ¬βββ ββββ¬βββ β PubA, CookieA β βββββββββββββββββββββββββββββ ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ>β βSignPrvA, SignPubA = load()β β β βPrvA, PubA = DHgen() β β β βββββββββββββββββββββββββββββ βPubB, CookieB, Enc((IdB, sign(SignPrvB, (CookieA, CookieB, PubB)), MAC(IdB))) β βββββββββββββββββββββββββββββ ββ βverify(Key, IdB) β β β βverify(SignPubB, ...)β β β βββββββββββββββββββββββ β β
- The GOST R standard is used for signing. It uses a 256-bit key algorithm.
- The common key is generated using the 34.10-2012 VKO standard.
- CMAC is used as the MAC. Technically, this is a special mode of operation for a block cipher described in GOST R 34.13-2015. For this mode, the encryption function is (34.12-2015).
- The identifier of the peer is a hash of their public key. The hash used is (34.11-2012 256 bits).
After the handshake, we will have an agreed common key. This key can be used for authenticated encryption of transport messages. This part is quite straightforward, and it's hard to make a mistake: we increment the message counter, encrypt the message, authenticate (MAC) the counter and ciphertext, and send it. Upon receiving a message, we check that the counter has the expected value, authenticate the ciphertext with the counter, and decrypt. What key should we use to encrypt handshake messages, transport messages, and which one to authenticate? Using one key for all these tasks is dangerous and impractical. We need to derive keys using specialized functions (key derivation function). Again, let's not complicate things and invent something: it is well-known, thoroughly researched, and has no known issues. Unfortunately, this function is not available in the standard Python library, so we will use package. HKDF internally uses , which, in turn, uses a hash function. An implementation example in Python on the Wikipedia page takes only a few lines of code. As with 34.10-2012, we will use Streebog-256 as the hash function. The output of our key agreement function will be called the session key, from which the required symmetric keys will be derived:
kdf = Hkdf(None, key_session, hash=GOST34112012256) kdf.expand(b"handshake1-mac-identity") kdf.expand(b"handshake1-enc") kdf.expand(b"handshake1-mac") kdf.expand(b"handshake2-mac-identity") kdf.expand(b"handshake2-enc") kdf.expand(b"handshake2-mac") kdf.expand(b"transport-initiator-enc") kdf.expand(b"transport-initiator-mac") kdf.expand(b"transport-responder-enc") kdf.expand(b"transport-responder-mac")Structures/Schemas
Let's consider what ASN.1 structures we have now obtained for transmitting all this data:
class Msg(Choice): schema = (( ("text", MsgText()), ("handshake0", MsgHandshake0(expl=tag_ctxc(0))), ("handshake1", MsgHandshake1(expl=tag_ctxc(1))), ("handshake2", MsgHandshake2(expl=tag_ctxc(2))), )) class MsgText(Sequence): schema = (( ("payload", MsgTextPayload()), ("payloadMac", MAC()), )) class MsgTextPayload(Sequence): schema = (( ("nonce", Integer(bounds=(0, float("+inf")))), ("ciphertext", OctetString(bounds=(1, MaxTextLen))), )) class MsgHandshake0(Sequence): schema = (( ("cookieInitiator", Cookie()), ("pubKeyInitiator", PubKey()), )) class MsgHandshake1(Sequence): schema = (( ("cookieResponder", Cookie()), ("pubKeyResponder", PubKey()), ("ukm", OctetString(bounds=(8, 8))), ("ciphertext", OctetString()), ("ciphertextMac", MAC()), )) class MsgHandshake2(Sequence): schema = (( ("ciphertext", OctetString()), ("ciphertextMac", MAC()), )) class HandshakeTBE(Sequence): schema = (( ("identity", OctetString(bounds=(32, 32))), ("signature", OctetString(bounds=(64, 64))), ("identityMac", MAC()), )) class HandshakeTBS(Sequence): schema = (( ("cookieTheir", Cookie()), ("cookieOur", Cookie()), ("pubKeyOur", PubKey()), )) class Cookie(OctetString): bounds = (16, 16) class PubKey(OctetString): bounds = (64, 64) class MAC(OctetString): bounds = (16, 16)HandshakeTBS is what will be signed. HandshakeTBE is what will be encrypted. Note the ukm field in MsgHandshake1. 34.10 VKO, for additional randomization of the keys generated, includes the UKM (user keying material) parameter β simply additional entropy.
Adding cryptography to the code
Letβs only consider the changes made to the original code, as the framework remains the same (in fact, the final implementation was written first, and then all cryptography was stripped away from it).
Since authentication and identification of interlocutors will be conducted using public keys, they now need to be stored somewhere for the long term. For simplicity, we will use JSON in the following structure:
{ "our": { "prv": "21254cf66c15e0226ef2669ceee46c87b575f37f9000272f408d0c9283355f98", "pub": "938c87da5c55b27b7f332d91b202dbef2540979d6ceaa4c35f1b5bfca6df47df0bdae0d3d82beac83cec3e353939489d9981b7eb7a3c58b71df2212d556312a1" }, "their": { "alice": "d361a59c25d2ca5a05d21f31168609deeec100570ac98f540416778c93b2c7402fd92640731a707ec67b5410a0feae5b78aeec93c4a455a17570a84f2bc21fce", "bob": "aade1207dd85ecd283272e7b69c078d5fae75b6e141f7649ad21962042d643512c28a2dbdc12c7ba40eb704af920919511180c18f4d17e07d7f5acd49787224a" } }our β our key pair, hexadecimal private and public keys. their β the names of the interlocutors and their public keys. Letβs change the command line arguments and add post-processing of the JSON data:
from pygost import gost3410 from pygost.gost34112012256 import GOST34112012256 CURVE = gost3410.GOST3410Curve( *gost3410.CURVE_PARAMS["GostR3410_2001_CryptoPro_A_ParamSet"] ) parser = argparse.ArgumentParser(description="GOSTIM") parser.add_argument( "--keys-gen", action="store_true", help="Generate JSON with our new keypair", ) parser.add_argument( "--keys", default="keys.json", required=False, help="JSON with our and their keys", ) parser.add_argument( "--bind", default="::1", help="Address to listen on", ) parser.add_argument( "--port", type=int, default=6666, help="Port to listen on", ) args = parser.parse_args() if args.keys_gen: prv_raw = urandom(32) pub = gost3410.public_key(CURVE, gost3410.prv_unmarshal(prv_raw)) pub_raw = gost3410.pub_marshal(pub) print(json.dumps({ "our": {"prv": hexenc(prv_raw), "pub": hexenc(pub_raw)}, "their": {}, })) exit(0) # Parse and unmarshal our and their keys {{{ with open(args.keys, "rb") as fd: _keys = json.loads(fd.read().decode("utf-8")) KEY_OUR_SIGN_PRV = gost3410.prv_unmarshal(hexdec(_keys["our"]["prv"])) _pub = hexdec(_keys["our"]["pub"]) KEY_OUR_SIGN_PUB = gost3410.pub_unmarshal(_pub) KEY_OUR_SIGN_PUB_HASH = OctetString(GOST34112012256(_pub).digest()) for peer_name, pub_raw in _keys["their"].items(): _pub = hexdec(pub_raw) KEYS[GOST34112012256(_pub).digest()] = { "name": peer_name, "pub": gost3410.pub_unmarshal(_pub), } # }}}The private key of the GOST 34.10 algorithm is a random number. It is 256 bits in size for 256-bit elliptic curves. PyGOST does not work with a byte set, but with , so our private key (urandom(32)) must be converted to a number using gost3410.prv_unmarshal(). The public key is deterministically computed from the private key using gost3410.public_key(). The public key of GOST 34.10 consists of two large numbers, which also need to be converted to a byte sequence for convenience in storage and transmission using gost3410.pub_marshal().
After reading the JSON file, the public keys must be converted back using gost3410.pub_unmarshal(). Since we will receive identifiers of peers as hashes from the public key, these can be computed in advance and placed in a dictionary for quick lookup. The Stribog-256 hash is gost34112012256.GOST34112012256(), which fully meets the hashlib interface of hash functions.
How has the initiator coroutine changed? Everything follows the handshake scheme: we generate a cookie (128 bits is quite sufficient), an ephemeral key pair of GOST 34.10, which will be used for the VKO key agreement function.
395 async def initiator(host, port): 396 _id = repr((host, port)) 397 logging.info("%s: dialing", _id) 398 reader, writer = await asyncio.open_connection(host, port) 399 # Generate our ephemeral public key and cookie, send Handshake 0 message {{{ 400 cookie_our = Cookie(urandom(16)) 401 prv = gost3410.prv_unmarshal(urandom(32)) 402 pub_our = gost3410.public_key(CURVE, prv) 403 pub_our_raw = PubKey(gost3410.pub_marshal(pub_our)) 404 writer.write(Msg(("handshake0", MsgHandshake0(( 405 ("cookieInitiator", cookie_our), 406 ("pubKeyInitiator", pub_our_raw), 407 )))).encode()) 408 # }}} 409 await writer.drain()- waiting for a response and decoding the received Msg message;
- ensuring that we received handshake1;
- decoding the ephemeral public key of the other party and computing the session key;
- deriving the symmetric keys necessary for handling the TBE part of the message.
423 logging.info("%s: got %s message", _id, msg.choice) 424 if msg.choice != "handshake1": 425 logging.warning("%s: unexpected message, disconnecting", _id) 426 writer.close() 427 return 428 # }}} 429 msg_handshake1 = msg.value 430 # Validate Handshake message {{{ 431 cookie_their = msg_handshake1["cookieResponder"] 432 pub_their_raw = msg_handshake1["pubKeyResponder"] 433 pub_their = gost3410.pub_unmarshal(bytes(pub_their_raw)) 434 ukm_raw = bytes(msg_handshake1["ukm"]) 435 ukm = ukm_unmarshal(ukm_raw) 436 key_session = kek_34102012256(CURVE, prv, pub_their, ukm, mode=2001) 437 kdf = Hkdf(None, key_session, hash=GOST34112012256) 438 key_handshake1_mac_identity = kdf.expand(b"handshake1-mac-identity") 439 key_handshake1_enc = kdf.expand(b"handshake1-enc") 440 key_handshake1_mac = kdf.expand(b"handshake1-mac")UKM is a 64-bit number (urandom(8)), which also requires deserialization from bytes using gost3410_vko.ukm_unmarshal(). The VKO function for GOST 34.10-2012 256-bit is gost3410_vko.kek_34102012256() (KEK β key encryption key).
The derived session key is already a 256-bit byte pseudorandom sequence. Therefore, it can be immediately used in the HKDF function. Since GOST34112012256 satisfies the hashlib interface, it can be directly used in the Hkdf class. We do not specify a salt (the first argument of Hkdf) because the derived key, due to the ephemerality of the participating key pairs, will be different for each session and already contains sufficient entropy. kdf.expand() by default already outputs keys of 256 bits in length, which are required for the subsequent calculations.
Next, the TBE and TBS parts of the incoming message are checked:
- MAC is computed and verified over the incoming ciphertext;
- the ciphertext is decrypted;
- the TBE structure is decoded;
- from it, the identifier of the counterpart is taken and checked if it is known to us at all;
- MAC is computed and verified over this identifier;
- The signature over the TBS structure is being verified, which includes the cookies from both sides and the public ephemeral key of the opposing side. The signature is verified using the long-lived signing key of the interlocutor.
441 try: 442 peer_name = validate_tbe( 443 msg_handshake1, 444 key_handshake1_mac_identity, 445 key_handshake1_enc, 446 key_handshake1_mac, 447 cookie_our, 448 cookie_their, 449 pub_their_raw, 450 ) 451 except ValueError as err: 452 logging.warning("%s: %s, disconnecting", _id, err) 453 writer.close() 454 return 455 # }}} 128 def validate_tbe( 129 msg_handshake: Union[MsgHandshake1, MsgHandshake2], 130 key_mac_identity: bytes, 131 key_enc: bytes, 132 key_mac: bytes, 133 cookie_their: Cookie, 134 cookie_our: Cookie, 135 pub_key_our: PubKey, 136 ) -> str: 137 ciphertext = bytes(msg_handshake["ciphertext"]) 138 mac_tag = mac(GOST3412Kuznechik(key_mac).encrypt, KUZNECHIK_BLOCKSIZE, ciphertext) 139 if not compare_digest(mac_tag, bytes(msg_handshake["ciphertextMac"])): 140 raise ValueError("invalid MAC") 141 plaintext = ctr( 142 GOST3412Kuznechik(key_enc).encrypt, 143 KUZNECHIK_BLOCKSIZE, 144 ciphertext, 145 8 * b"x00", 146 ) 147 try: 148 tbe, _ = HandshakeTBE().decode(plaintext) 149 except ASN1Error: 150 raise ValueError("can not decode TBE") 151 key_sign_pub_hash = bytes(tbe["identity"]) 152 peer = KEYS.get(key_sign_pub_hash) 153 if peer is None: 154 raise ValueError("unknown identity") 155 mac_tag = mac( 156 GOST3412Kuznechik(key_mac_identity).encrypt, 157 KUZNECHIK_BLOCKSIZE, 158 key_sign_pub_hash, 159 ) 160 if not compare_digest(mac_tag, bytes(tbe["identityMac"])): 161 raise ValueError("invalid identity MAC") 162 tbs = HandshakeTBS(( 163 ("cookieTheir", cookie_their), 164 ("cookieOur", cookie_our), 165 ("pubKeyOur", pub_key_our), 166 )) 167 if not gost3410.verify( 168 CURVE, 169 peer["pub"], 170 GOST34112012256(tbs.encode()).digest(), 171 bytes(tbe["signature"]), 172 ): 173 raise ValueError("invalid signature") 174 return peer["name"]As mentioned earlier, 34.13-2015 describes various from 34.12-2015. Among them, there is a mode for generating a message authentication code (MAC). In PyGOST, this is done with gost3413.mac(). This mode requires the transmission of an encryption function (taking and returning a single block of data), the block size, and the actual data. Why can't the block size be hardcoded? 34.12-2015 describes not only the 128-bit Kuznechik cipher but also a 64-bit β a slightly modified GOST 28147-89, created back in the KGB and still having one of the highest security thresholds.
The Kuznechik is initialized using gost.3412.GOST3412Kuznechik(key) and returns an object with .encrypt()/.decrypt() methods suitable for use in 34.13 functions. The MAC is computed as follows: gost3413.mac(GOST3412Kuznechik(key).encrypt, KUZNECHIK_BLOCKSIZE, ciphertext). To compare the computed MAC with the received MAC, one cannot use simple equality (==) of byte strings, as this operation can leak time comparisons, potentially leading to severe vulnerabilities such as attacks on TLS. Python has a special hmac.compare_digest function for this.
The block cipher function can only encrypt one block of data. For a larger amount, and not in a multiple of the length, an encryption mode must be used. The following are described in 34.13-2015: ECB, CTR, OFB, CBC, CFB. Each has its own permissible areas of application and characteristics. Unfortunately, we still do not have standardized (like CCM, OCB, GCM, and similar) β we are forced to at least add a MAC ourselves. I choose the (CTR): it does not require padding to block size, can be parallelized, uses only the encryption function, and can be safely used for encrypting a large number of messages (unlike CBC, which relatively quickly encounters collisions).
Like .mac(), .ctr() accepts similar input data: ciphertext = gost3413.ctr(GOST3412Kuznechik(key).encrypt, KUZNECHIK_BLOCKSIZE, plaintext, iv). It requires an initialization vector that is exactly half the length of the block cipher. If our encryption key is used for encrypting only one message (even if it consists of several blocks), it is safe to set the initialization vector to zero. For encrypting handshake messages, we use a separate key each time.
Verifying the signature with gost3410.verify() is straightforward: we pass the elliptic curve we are working within (which we simply define in our GOSTIM protocol), the public key of the signer (remember that this must be a tuple of two large numbers, not a byte string), the 34.11-2012 hash, and the received signature itself.
Next, in the initiator, we prepare and send the handshake2 message, performing the same actions we took during verification, only symmetrically: signing with our keys instead of verification, and so on...
456 # Prepare and send Handshake 2 message {{{ 457 tbs = HandshakeTBS(( 458 ("cookieTheir", cookie_their), 459 ("cookieOur", cookie_our), 460 ("pubKeyOur", pub_our_raw), 461 )) 462 signature = gost3410.sign( 463 CURVE, 464 KEY_OUR_SIGN_PRV, 465 GOST34112012256(tbs.encode()).digest(), 466 ) 467 key_handshake2_mac_identity = kdf.expand(b"handshake2-mac-identity") 468 mac_tag = mac( 469 GOST3412Kuznechik(key_handshake2_mac_identity).encrypt, 470 KUZNECHIK_BLOCKSIZE, 471 bytes(KEY_OUR_SIGN_PUB_HASH), 472 ) 473 tbe = HandshakeTBE(( 474 ("identity", KEY_OUR_SIGN_PUB_HASH), 475 ("signature", OctetString(signature)), 476 ("identityMac", MAC(mac_tag)), 477 )) 478 tbe_raw = tbe.encode() 479 key_handshake2_enc = kdf.expand(b"handshake2-enc") 480 key_handshake2_mac = kdf.expand(b"handshake2-mac") 481 ciphertext = ctr( 482 GOST3412Kuznechik(key_handshake2_enc).encrypt, 483 KUZNECHIK_BLOCKSIZE, 484 tbe_raw, 485 8 * b"x00", 486 ) 487 mac_tag = mac( 488 GOST3412Kuznechik(key_handshake2_mac).encrypt, 489 KUZNECHIK_BLOCKSIZE, 490 ciphertext, 491 ) 492 writer.write(Msg(("handshake2", MsgHandshake2(( 493 ("ciphertext", OctetString(ciphertext)), 494 ("ciphertextMac", MAC(mac_tag)), 495 )))).encode()) 496 # }}} 497 await writer.drain() 498 logging.info("%s: session established: %s", _id, peer_name)Once the session is established, transport keys are generated (a separate key for encryption, for authentication, for each side), and the Kuznechik is initialized for decryption and MAC checking:
499 # Run text message sender, initialize transport decoder {{{ 500 key_initiator_enc = kdf.expand(b"transport-initiator-enc") 501 key_initiator_mac = kdf.expand(b"transport-initiator-mac") 502 key_responder_enc = kdf.expand(b"transport-responder-enc") 503 key_responder_mac = kdf.expand(b"transport-responder-mac") ... 509 asyncio.ensure_future(msg_sender( 510 peer_name, 511 key_initiator_enc, 512 key_initiator_mac, 513 writer, 514 )) 515 encrypter = GOST3412Kuznechik(key_responder_enc).encrypt 516 macer = GOST3412Kuznechik(key_responder_mac).encrypt 517 # }}} 519 nonce_expected = 0 520 # Wait for test messages {{{ 521 while True: 522 data = await reader.read(MaxMsgLen) ... 530 msg, tail = Msg().decode(buf) ... 537 try: 538 await msg_receiver( 539 msg.value, 540 nonce_expected, 541 macer, 542 encrypter, 543 peer_name, 544 ) 545 except ValueError as err: 546 logging.warning("%s: %s", err) 547 break 548 nonce_expected += 1 549 # }}}The msg_sender coroutine now encrypts messages before sending them over the TCP connection. Each message has a monotonically increasing nonce, which also serves as the initialization vector for counter mode encryption. Each message and block of the message will have guaranteed different counter values.
async def msg_sender(peer_name: str, key_enc: bytes, key_mac: bytes, writer) -> None: nonce = 0 encrypter = GOST3412Kuznechik(key_enc).encrypt macer = GOST3412Kuznechik(key_mac).encrypt in_queue = IN_QUEUES[peer_name] while True: text = await in_queue.get() if text is None: break ciphertext = ctr( encrypter, KUZNECHIK_BLOCKSIZE, text.encode("utf-8"), long2bytes(nonce, 8), ) payload = MsgTextPayload(( ("nonce", Integer(nonce)), ("ciphertext", OctetString(ciphertext)), )) mac_tag = mac(macer, KUZNECHIK_BLOCKSIZE, payload.encode()) writer.write(Msg(("text", MsgText(( ("payload", payload), ("payloadMac", MAC(mac_tag)), )))).encode()) nonce += 1Incoming messages are processed by the msg_receiver coroutine, which handles authentication and decryption:
async def msg_receiver( msg_text: MsgText, nonce_expected: int, macer, encrypter, peer_name: str, ) -> None: payload = msg_text["payload"] if int(payload["nonce"]) != nonce_expected: raise ValueError("unexpected nonce value") mac_tag = mac(macer, KUZNECHIK_BLOCKSIZE, payload.encode()) if not compare_digest(mac_tag, bytes(msg_text["payloadMac"])): raise ValueError("invalid MAC") plaintext = ctr( encrypter, KUZNECHIK_BLOCKSIZE, bytes(payload["ciphertext"]), long2bytes(nonce_expected, 8), ) text = plaintext.decode("utf-8") await OUT_QUEUES[peer_name].put(text)Conclusion
GOSTIM is intended for educational purposes only (as it is not covered by tests, at least)! The source code can be downloaded (Stribog-256 hash: 995bbd368c04e50a481d138c5fa2e43ec7c89bc77743ba8dbabee1fde45de120). Like all my projects, type , , , , GOSTIM is completely , distributed under the terms of .
, , member of , Python/Go developer, chief specialist .
Source: habr.com
