Qhov project configuration sab hauv thiab sab nraud Kubernetes

Kuv nyuam qhuav sau teb txog qhov project lub neej hauv Docker thiab debugging code sab nraum nws, qhov uas nws tau hais luv luv tias koj tuaj yeem ua rau koj tus kheej teeb tsa kom cov kev pabcuam ua haujlwm zoo hauv Kuber, rub tawm cov lus zais, thiab khiav hauv zos yooj yim, txawm tias sab nraud ntawm Docker tag nrho. Tsis muaj dab tsi nyuab, tab sis cov lus piav qhia "daim ntawv qhia" yuav muaj txiaj ntsig rau ib tus neeg :) Cov lej yog nyob rau hauv Python, tab sis cov logic tsis khi rau cov lus.

Qhov project configuration sab hauv thiab sab nraud Kubernetes

Cov keeb kwm ntawm cov lus nug yog qhov no: ib zaug ib zaug muaj ib qhov project, thaum xub thawj nws yog ib qho me me monolith nrog cov khoom siv thiab cov ntawv sau, tab sis dhau sij hawm nws loj hlob, muab faib ua cov kev pabcuam, uas tau pib muab faib ua microservices, thiab. ces scaled. Thaum xub thawj, tag nrho cov no tau ua tiav ntawm qhov liab qab VPS, cov txheej txheem ntawm kev teeb tsa thiab xa cov lej uas tau siv los siv Ansible, thiab txhua qhov kev pabcuam tau suav nrog YAML config nrog rau qhov tsim nyog thiab cov yuam sij, thiab cov ntaub ntawv zoo sib xws tau siv rau lub zos launches, uas yog heev yooj yim, vim hais tias .k no config yog loaded rau hauv lub ntiaj teb no cov khoom, siv tau los ntawm txhua qhov chaw hauv qhov project.

Txawm li cas los xij, kev loj hlob ntawm cov microservices, lawv cov kev sib txuas, thiab xav tau kev txiav txim siab hauv nruab nrab thiab saib xyuas, foreshadowed txav mus rau Kuber, uas tseem tab tom ua tiav. Ua ke nrog kev pab los daws cov teeb meem hais, Kubernetes muab nws txoj hauv kev rau kev tswj hwm vaj tse, suav nrog thiaj li hu ua Secrets ΠΈ txoj kev ua haujlwm nrog lawv. Cov txheej txheem yog tus qauv thiab txhim khu kev qha, yog li nws yog qhov ua txhaum cai tsis txhob siv nws! Tab sis nyob rau tib lub sijhawm, kuv xav khaws kuv cov qauv tam sim no rau kev ua haujlwm nrog kev teeb tsa: thawj zaug, siv nws sib txawv hauv cov microservices sib txawv ntawm qhov project, thiab thib ob, kom muaj peev xwm khiav cov cai ntawm lub tshuab hauv zos siv ib qho yooj yim. config cov ntaub ntawv.

Nyob rau hauv no hais txog, lub mechanism rau kev tsim ib tug configuration khoom raug hloov kho kom muaj peev xwm ua hauj lwm nrog peb classic config ntaub ntawv thiab nrog secrets ntawm Kuber. Ib qho kev nruj dua config kuj tau teev tseg, hauv cov lus ntawm peb Python, raws li hauv qab no:

Dict[str, Dict[str, Union[str, int, ntab]]]

Ntawd yog, cogfig kawg yog phau ntawv txhais lus nrog cov npe npe, txhua yam yog phau ntawv txhais lus nrog cov txiaj ntsig los ntawm hom yooj yim. Thiab cov ntu piav qhia txog kev teeb tsa thiab kev nkag mus rau cov peev txheej ntawm qee yam. Ib qho piv txwv ntawm peb cov config:

adminka:
  django_secret: "ExtraLongAndHardCode"

db_main:
  engine: mysql
  host: 256.128.64.32
  user: cool_user
  password: "SuperHardPassword"

redis:
  host: 256.128.64.32
  pw: "SuperHardPassword"
  port: 26379

smtp:
  server: smtp.gmail.com
  port: 465
  email: [email protected]
  pw: "SuperHardPassword"

Nyob rau tib lub sij hawm, teb engine databases tuaj yeem ntsia tau rau ntawm SQLite, thiab redis teem rau mock, tseem qhia lub npe ntawm cov ntaub ntawv khaws tseg - cov kev txwv no raug lees paub thiab ua tiav, uas ua rau nws yooj yim los khiav cov cai hauv zos rau kev debugging, chav kuaj thiab lwm yam kev xav tau. Qhov no yog qhov tseem ceeb tshwj xeeb rau peb vim tias muaj ntau lwm yam kev xav tau - ib feem ntawm peb cov cai yog npaj rau ntau yam kev ntsuas kev ntsuas, nws khiav tsis tau tsuas yog ntawm cov servers nrog orchestration, tab sis kuj nrog ntau cov ntawv sau, thiab hauv cov khoos phis tawj ntawm cov kws tshuaj ntsuam xyuas uas yuav tsum tau ua haujlwm dhau los. thiab debug complex cov ntaub ntawv ua cov kav dej tsis muaj kev txhawj xeeb txog teeb meem backend. Los ntawm txoj kev, nws yuav tsis raug mob los qhia tias peb cov cuab yeej tseem ceeb, suav nrog cov txheej txheem teeb tsa, raug teeb tsa ntawm setup.py - Ua ke qhov no sib sau ua ke peb cov cai rau hauv ib qho ecosystem, ywj siab ntawm lub platform thiab txoj kev siv.

Cov lus piav qhia ntawm Kubernetes pod zoo li no:

containers:
  - name : enter-api
    image: enter-api:latest
    ports:
      - containerPort: 80
    volumeMounts:
      - name: db-main-secret-volume
        mountPath: /etc/secrets/db-main

volumes:
  - name: db-main-secret-volume
    secret:
      secretName: db-main-secret

Ntawd yog, txhua qhov zais cia piav qhia ib ntu. Cov secrets lawv tus kheej yog tsim li no:

apiVersion: v1
kind: Secret
metadata:
  name: db-main-secret
type: Opaque
stringData:
  db_main.yaml: |
    engine: sqlite
    filename: main.sqlite3

Ua ke qhov no ua rau tsim cov ntaub ntawv YAML raws txoj kev /etc/secrets/db-main/section_name.yaml

Thiab rau lub zos launches, lub config yog siv, nyob rau hauv lub hauv paus directory ntawm qhov project los yog raws li txoj kev teev nyob rau hauv ib puag ncig kuj sib txawv. Cov cai tswj xyuas cov kev yooj yim no tuaj yeem pom nyob rau hauv spoiler.

config.py

__author__ = 'AivanF'
__copyright__ = 'Copyright 2020, AivanF'

import os
import yaml

__all__ = ['config']
PROJECT_DIR = os.path.abspath(__file__ + 3 * '/..')
SECRETS_DIR = '/etc/secrets'
KEY_LOG = '_config_log'
KEY_DBG = 'debug'

def is_yes(value):
    if isinstance(value, str):
        value = value.lower()
        if value in ('1', 'on', 'yes', 'true'):
            return True
    else:
        if value in (1, True):
            return True
    return False

def update_config_part(config, key, data):
    if key not in config:
        config[key] = data
    else:
        config[key].update(data)

def parse_big_config(config, filename):
    '''
    Parse YAML config with multiple section
    '''
    if not os.path.isfile(filename):
        return False
    with open(filename) as f:
        config_new = yaml.safe_load(f.read())
        for key, data in config_new.items():
            update_config_part(config, key, data)
        config[KEY_LOG].append(filename)
        return True

def parse_tiny_config(config, key, filename):
    '''
    Parse YAML config with a single section
    '''
    with open(filename) as f:
        config_tiny = yaml.safe_load(f.read())
        update_config_part(config, key, config_tiny)
        config[KEY_LOG].append(filename)

def combine_config():
    config = {
        # To debug config load code
        KEY_LOG: [],
        # To debug other code
        KEY_DBG: is_yes(os.environ.get('DEBUG')),
    }
    # For simple local runs
    CONFIG_SIMPLE = os.path.join(PROJECT_DIR, 'config.yaml')
    parse_big_config(config, CONFIG_SIMPLE)
    # For container's tests
    CONFIG_ENVVAR = os.environ.get('CONFIG')
    if CONFIG_ENVVAR is not None:
        if not parse_big_config(config, CONFIG_ENVVAR):
            raise ValueError(
                f'No config file from EnvVar:n'
                f'{CONFIG_ENVVAR}'
            )
    # For K8s secrets
    for path, dirs, files in os.walk(SECRETS_DIR):
        depth = path[len(SECRETS_DIR):].count(os.sep)
        if depth > 1:
            continue
        for file in files:
            if file.endswith('.yaml'):
                filename = os.path.join(path, file)
                key = file.rsplit('.', 1)[0]
                parse_tiny_config(config, key, filename)
    return config

def build_config():
    config = combine_config()
    # Preprocess
    for key, data in config.items():
        if key.startswith('db_'):
            if data['engine'] == 'sqlite':
                data['filename'] = os.path.join(PROJECT_DIR, data['filename'])
    # To verify correctness
    if config[KEY_DBG]:
        print(f'** Loaded config:n{yaml.dump(config)}')
    else:
        print(f'** Loaded config from: {config[KEY_LOG]}')
    return config

config = build_config()

Lub logic ntawm no yog qhov yooj yim heev: peb muab cov configs loj los ntawm qhov project directory thiab txoj hauv kev los ntawm ib puag ncig hloov pauv, thiab me me config seem los ntawm Kuber secrets, thiab tom qab ntawd ua ntej lawv me ntsis. Ntxiv rau qee qhov sib txawv. Kuv nco ntsoov tias thaum tshawb nrhiav cov ntaub ntawv los ntawm cov ntaub ntawv tsis pub lwm tus paub, kev txwv qhov tob yog siv, vim K8s tsim cov ntawv zais zais hauv txhua qhov zais cia uas lawv tus kheej khaws cia, thiab tsuas yog qhov txuas nyob rau theem siab dua.

Kuv vam tias qhov tau piav qhia yuav muaj txiaj ntsig zoo rau ib tus neeg :) Cov lus pom thiab cov lus pom zoo txog kev ruaj ntseg lossis lwm qhov chaw rau kev txhim kho tau txais. Lub zej zog txoj kev xav kuj tseem nthuav, tej zaum nws tsim nyog ntxiv kev txhawb nqa rau ConfigMaps (peb qhov project tseem tsis tau siv) thiab tshaj tawm cov cai ntawm GitHub / PyPI? Tus kheej, kuv xav tias tej yam zoo li no yog ib tus neeg heev rau tej yaam num yuav tsum universal, thiab ib tug me ntsis peeking ntawm lwm tus neeg cov kev siv, xws li ib tug muab rau ntawm no, thiab kev sib tham ntawm nuances, cov lus qhia thiab cov kev coj zoo tshaj plaws, uas kuv vam tias yuav pom hauv cov lus. , puas πŸ˜‰

Tsuas yog cov neeg siv sau npe tuaj yeem koom nrog hauv daim ntawv ntsuam xyuas. Kos npe rau hauvthov.

Kuv puas yuav tsum luam tawm raws li qhov project / tsev qiv ntawv?

  • 0,0%Yog lawm, kuv yuav siv /pab txhawb0

  • 33,3%Yog, nws suab zoo heev 4

  • 41,7%Tsis yog, leej twg yuav tsum ua lawv tus kheej hauv lawv tus kheej hom thiab kom haum lawv cov kev xav tau5

  • 25,0%Kuv yuav tsis teb 3

12 cov neeg siv pov npav. 3 cov neeg siv tau txwv.

Tau qhov twg los: www.hab.com

Ntxiv ib saib