🥇Fundamentals of Working with zmq in Python, Creating a Simple Key/Value Store

Introduction

Let's examine an example of a simple key/value store, such as memcache. It's structured simply — data is stored in memory in a hashmap format. Accessing it is done via a TCP socket. In Python, a hashmap is a regular dict. We will use zeromq for access.

Settings

To install this package on Debian/Ubuntu, simply enter the following in the console:
sudo apt-get install libzmq-dev
sudo pip install zmq

Code

We'll write a class to work with our proxy server:
The type of zmq socket used is REQ (REQuest, request); we send a request and wait for a response.
To store and transmit any type of data over the network, we use the standard pickle module. The "protocol" consists of a tuple of three values: (command, key, data)

import zmq
import pickle

class SuperCacher:
    def __init__(self):
        context = zmq.Context()
        self.socket = context.socket(zmq.REQ)
        self.socket.connect('tcp://127.0.0.1:43000')

    def get(self, key):
        self.socket.send(pickle.dumps(('get', key, None)))
        return pickle.loads(self.socket.recv())

    def set(self, key, data):
        self.socket.send(pickle.dumps(('set', key, data)))
        return self.socket.recv() == b'ok'

Using

cache = SuperCacher()
cache.set('key', 'value')
cache.get('key')

As a homework assignment — improve the implementation by adding the ability to specify the address/port when creating an instance of the class.

Now let's write the actual server.
This time we use a REP (REPlY, response) socket — we wait for a request, and then send a response. We parse the request, responding with either 'ok' if writing, or with the data / None if reading.

import pickle
import json
import zmq

def run_daemon():
    memory = {}

    context = zmq.Context()
    socket = context.socket(zmq.REP)
    socket.bind('tcp://127.0.0.1:43000')

    while True:
        try:
            command, key, data = pickle.loads(socket.recv())
            if command == 'set':
                memory[key] = data
                socket.send(b'ok')
            elif command == 'get':
                result = memory.get(key, None)
                socket.send(pickle.dumps(result))
        except Exception as e:
            print(e)

if __name__ == '__main__':
    run_daemon()

To test everything together, we start the server with the command
python daemon.py

In a neighboring tab, we run Python in interactive mode.

>>> from lib import SuperCacher
>>> cache = SuperCacher()
>>> cache.set('key', 'value')
True
>>> cache.get('key')
'value'

Oh miracle, it works! Now you can confidently put "development of a key-value store using the zmq protocol" in your resume.

Source: habr.com

Fundamentals of zmq in Python, creating a simple key/value store

Introduction

Settings

Code

Using