Los ntawm "startup" mus rau ntau txhiab tus servers hauv lub kaum os cov ntaub ntawv chaw. Yuav ua li cas peb caum txoj kev loj hlob ntawm Linux infrastructure

Yog tias koj IT infrastructure loj hlob sai dhau lawm, koj yuav tau ntsib nrog kev xaiv sai sai: linearly nce tib neeg cov peev txheej los txhawb nws lossis pib automation. Txog rau qee qhov taw tes, peb nyob hauv thawj lub tswv yim, thiab tom qab ntawd txoj kev ntev mus rau Infrastructure-raws-Code pib.

Los ntawm "startup" mus rau ntau txhiab tus servers hauv lub kaum os cov ntaub ntawv chaw. Yuav ua li cas peb caum txoj kev loj hlob ntawm Linux infrastructure

Ntawm chav kawm, NSPK tsis yog ib qho pib, tab sis xws li cov huab cua tau kav hauv lub tuam txhab hauv thawj xyoo ntawm nws lub neej, thiab cov xyoo no tau nthuav dav heev. Kuv lub npe yog Kornyakov Dmitry, Kuv tau txhawb nqa Linux infrastructure nrog cov kev xav tau siab rau ntau tshaj 10 xyoo. Nws tau koom nrog pab pawg NSPK thaum Lub Ib Hlis 2016 thiab, hmoov tsis, tsis pom qhov pib ntawm lub tuam txhab muaj nyob, tab sis tuaj ntawm theem ntawm kev hloov pauv loj.

Feem ntau, peb tuaj yeem hais tias peb pab neeg muab 2 khoom rau lub tuam txhab. Qhov thib ib yog infrastructure. Xa ntawv yuav tsum ua haujlwm, DNS yuav tsum ua haujlwm, thiab cov tswj hwm sau npe yuav tsum cia koj mus rau hauv servers uas yuav tsum tsis txhob poob. Lub tuam txhab IT toj roob hauv pes loj heev! Cov no yog kev lag luam & lub luag haujlwm tseem ceeb, cov kev xav tau muaj rau qee qhov yog 99,999. Cov khoom thib ob yog cov servers lawv tus kheej, lub cev thiab virtual. Cov uas twb muaj lawm yuav tsum tau saib xyuas, thiab cov tshiab yuav tsum tsis tu ncua rau cov neeg siv khoom los ntawm ntau lub tuam tsev. Nyob rau hauv tsab xov xwm no kuv xav tsom mus rau yuav ua li cas peb tsim cov infrastructure uas yog lub luag hauj lwm rau lub server lub neej voj voog.

Pib ntawm txoj kev taug

Thaum pib ntawm peb txoj kev taug, peb cov txheej txheem thev naus laus zis zoo li no:
OS CentOS 7
FreeIPA Domain Controllers
Automation - Ansible(+Tower), Cobbler

Tag nrho cov no tau nyob rau hauv 3 lub teb chaws, kis thoob plaws ntau lub chaw cov ntaub ntawv. Hauv ib lub chaw cov ntaub ntawv muaj cov chaw ua haujlwm thiab cov chaw sim, nyob rau hauv tas li muaj PROD.

Tsim cov servers ntawm ib kis zoo li no:

Los ntawm "startup" mus rau ntau txhiab tus servers hauv lub kaum os cov ntaub ntawv chaw. Yuav ua li cas peb caum txoj kev loj hlob ntawm Linux infrastructure

Hauv VM template, CentOS yog qhov tsawg heev thiab qhov tsawg kawg nkaus yuav tsum yog zoo li qhov tseeb /etc/resolv.conf, tus so los ntawm Ansible.

CMDB - Excel.

Yog tias tus neeg rau zaub mov yog lub cev, ces tsis txhob theej lub tshuab virtual, OS tau nruab rau ntawm nws siv Cobbler - MAC chaw nyob ntawm lub hom phiaj server raug ntxiv rau Cobbler config, tus neeg rau zaub mov tau txais IP chaw nyob ntawm DHCP, thiab tom qab ntawd OS yog ntxiv.

Thaum xub thawj peb txawm sim ua qee yam kev tswj hwm kev teeb tsa hauv Cobbler. Tab sis dhau sij hawm, qhov no tau pib coj teeb meem nrog kev txav mus los ntawm kev teeb tsa ob qho tib si rau lwm lub chaw cov ntaub ntawv thiab rau Ansible code rau kev npaj VMs.

Lub sijhawm ntawd, peb coob leej pom Ansible yog qhov yooj yim txuas ntxiv ntawm Bash thiab tsis skimp ntawm cov qauv siv lub plhaub thiab sed. Zuag qhia tag nrho Bashsible. Qhov no thaum kawg ua rau lub fact tias yog hais tias lub playbook rau ib co yog vim li cas tsis ua hauj lwm nyob rau hauv lub server, nws yog yooj yim dua rau rho tawm lub server, kho lub playbook thiab khiav nws dua. Muaj qhov tseem ceeb tsis muaj kev hloov pauv ntawm cov ntawv sau, tsis muaj kev txav mus los ntawm kev teeb tsa.

Piv txwv li, peb xav hloov ib co config ntawm tag nrho cov servers:

  1. Peb hloov cov kev teeb tsa ntawm cov servers uas twb muaj lawm nyob rau hauv cov txheej txheem / cov ntaub ntawv chaw. Qee lub sij hawm tsis nyob rau hauv ib hnub - kev nkag mus tau yooj yim thiab kev cai lij choj ntawm cov neeg coob tsis tso cai rau txhua qhov kev hloov pauv mus siv ib zaug. Thiab qee qhov kev hloov pauv tuaj yeem ua rau puas tsuaj thiab yuav tsum tau rov pib dua ib yam - los ntawm cov kev pabcuam rau OS nws tus kheej.
  2. Kho nws hauv Ansible
  3. Peb kho nws hauv Cobbler
  4. Rov ua dua N lub sij hawm rau txhua qhov laj thawj / cov ntaub ntawv chaw

Txhawm rau kom txhua qhov kev hloov pauv mus tau zoo, nws yuav tsum tau coj mus rau hauv tus account ntau yam, thiab kev hloov pauv tshwm sim tas li.

  • Refactoring ansible code, configuration ntaub ntawv
  • Hloov cov kev coj ua zoo tshaj plaws sab hauv
  • Kev hloov pauv raws li cov txiaj ntsig ntawm kev txheeb xyuas qhov xwm txheej / xwm txheej
  • Hloov cov qauv kev ruaj ntseg, ob qho tib si sab hauv thiab sab nraud. Piv txwv li, PCI DSS tau hloov kho nrog cov cai tshiab txhua xyoo

Infrastructure kev loj hlob thiab pib ntawm txoj kev mus

Tus naj npawb ntawm cov servers / lub ntsiab lus tseem ceeb / cov chaw zov me nyuam tau loj hlob, thiab nrog lawv cov lej yuam kev hauv kev teeb tsa. Qee lub sij hawm, peb tuaj rau peb cov lus qhia uas kev tswj hwm kev teeb tsa yuav tsum tau tsim:

  1. Automation. Tib neeg yuam kev hauv kev ua haujlwm rov ua dua yuav tsum zam kom ntau li ntau tau.
  2. Rov ua dua. Nws yooj yim dua los tswj kev tsim kho vaj tse thaum nws kwv yees tau. Kev teeb tsa ntawm cov servers thiab cov cuab yeej rau lawv cov kev npaj yuav tsum zoo ib yam txhua qhov chaw. Qhov no tseem yog ib qho tseem ceeb rau pawg khoom - tom qab kev sim, daim ntawv thov yuav tsum tau lees tias yuav xaus rau hauv ib puag ncig kev tsim khoom teeb tsa zoo ib yam li qhov chaw sim.
  3. Simplicity thiab transparency ntawm kev hloov mus rau configuration tswj.

Nws tseem yuav ntxiv ob peb yam cuab yeej.

Peb xaiv GitLab CE raws li peb qhov chaw khaws cia, tsis tsawg kawg rau nws cov khoom tsim hauv CI / CD.

Vault ntawm secrets - Hashicorp Vault, incl. rau qhov zoo API.

Kev ntsuas kev teeb tsa thiab lub luag haujlwm tseem ceeb - Molecule + Testinfra. Kev ntsuam xyuas mus sai dua yog tias koj txuas rau ansible mitogen. Nyob rau tib lub sijhawm, peb pib sau peb tus kheej CMDB thiab tus kws tshaj lij rau kev xa tawm tsis siv neeg (hauv daim duab saum toj no Cobbler), tab sis qhov no yog ib zaj dab neeg sib txawv kiag li, uas kuv cov npoj yaig thiab tus tsim tawm tseem ceeb ntawm cov tshuab no yuav qhia yav tom ntej.

Peb qhov kev xaiv:

Molecule + Testinfra
Ansible + Tower + AWX
Ntiaj teb ntawm Servers + DITNET (Kev txhim kho tus kheej)
Cob Tsib
Gitlab + GitLab khiav
Hashicorp Vault

Los ntawm "startup" mus rau ntau txhiab tus servers hauv lub kaum os cov ntaub ntawv chaw. Yuav ua li cas peb caum txoj kev loj hlob ntawm Linux infrastructure

Los ntawm txoj kev, txog ansible luag hauj lwm. Thaum xub thawj tsuas muaj ib qho xwb, tab sis tom qab ob peb refactorings muaj 17 ntawm lawv. Kuv xav kom tawg lub monolith rau hauv lub luag hauj lwm idempotent, uas tuaj yeem raug tso tawm cais; ntxiv rau, koj tuaj yeem ntxiv cov cim npe. Peb faib lub luag haujlwm los ntawm kev ua haujlwm - network, kaw, pob khoom, kho vajtse, molecule thiab lwm yam. Feem ntau, peb ua raws li lub tswv yim hauv qab no. Kuv tsis hais tias qhov no tsuas yog qhov tseeb xwb, tab sis nws ua haujlwm rau peb.

  • Luam cov servers los ntawm "cov duab golden" yog qhov phem!Qhov tsis zoo tseem ceeb yog tias koj tsis paub meej tias lub xeev cov duab nyob rau hauv tam sim no, thiab txhua qhov kev hloov pauv yuav tuaj rau tag nrho cov duab hauv txhua qhov virtualization ua liaj ua teb.
  • Siv default configuration cov ntaub ntawv kom tsawg thiab pom zoo nrog rau lwm lub tuam tsev uas koj yog lub luag hauj lwm rau lub ntsiab system cov ntaub ntawv, piv txwv:
    1. Tawm /etc/sysctl.conf khoob, cov chaw yuav tsum tsuas yog nyob rau hauv /etc/sysctl.d/. Koj lub neej ntawd nyob rau hauv ib cov ntaub ntawv, kev cai rau daim ntawv thov nyob rau hauv lwm yam.
    2. Siv cov ntaub ntawv override los kho cov systemd units.
  • Template tag nrho configs thiab suav nrog tag nrho; yog tias ua tau, tsis muaj sed lossis nws cov analogues hauv cov ntawv ua si
  • Refactoring configuration tswj system code:
    1. Txiav cov dej num mus rau hauv cov chaw muaj kev sib cav thiab rov sau dua lub monolith rau hauv lub luag haujlwm
    2. Siv cov ntaub ntawv! Ansible-lint, yaml-lint, thiab lwm yam
    3. Hloov koj txoj hauv kev! Tsis muaj bashible. Nws yog tsim nyog los piav txog lub xeev ntawm lub system
  • Rau txhua lub luag haujlwm Ansible koj yuav tsum tau sau cov ntawv xeem hauv molecule thiab tsim cov ntawv ceeb toom ib hnub ib zaug.
  • Hauv peb cov ntaub ntawv, tom qab npaj cov kev xeem (uas muaj ntau tshaj 100), txog 70000 qhov yuam kev tau pom. Nws siv ob peb lub hlis los kho nws.Los ntawm "startup" mus rau ntau txhiab tus servers hauv lub kaum os cov ntaub ntawv chaw. Yuav ua li cas peb caum txoj kev loj hlob ntawm Linux infrastructure

Peb qhov kev siv

Yog li ntawd, lub luag hauj lwm ansible tau npaj, templated thiab xyuas los ntawm linters. Thiab txawm gits raug tsa nyob txhua qhov chaw. Tab sis cov lus nug ntawm txhim khu kev qha code xa mus rau ntau ntu tseem qhib. Peb txiav txim siab synchronize nrog scripts. Zoo li ntawd:

Los ntawm "startup" mus rau ntau txhiab tus servers hauv lub kaum os cov ntaub ntawv chaw. Yuav ua li cas peb caum txoj kev loj hlob ntawm Linux infrastructure

Tom qab qhov kev hloov pauv tuaj txog, CI raug tso tawm, ib qho kev sim neeg rau zaub mov raug tsim, lub luag haujlwm raug dov tawm, thiab kuaj los ntawm cov molecule. Yog tias txhua yam ua tau zoo, tus lej nkag mus rau cov khoom lag luam. Tab sis peb tsis siv cov cai tshiab rau cov servers uas twb muaj lawm hauv lub tshuab. Qhov no yog ib hom stopper uas tsim nyog rau siab muaj ntawm peb cov tshuab. Thiab thaum cov kev tsim kho vaj tse loj heev, txoj cai ntawm cov neeg coob tuaj rau hauv kev ua si - txawm tias koj paub tseeb tias qhov kev hloov pauv tsis muaj teeb meem, nws tuaj yeem ua rau muaj qhov tshwm sim loj heev.

Kuj tseem muaj ntau txoj hauv kev los tsim cov servers. Peb xaus rau xaiv Python scripts kev cai. Thiab rau CI ansible:

- name: create1.yml - Create a VM from a template
  vmware_guest:
    hostname: "{{datacenter}}".domain.ru
    username: "{{ username_vc }}"
    password: "{{ password_vc }}"
    validate_certs: no
    cluster: "{{cluster}}"
    datacenter: "{{datacenter}}"
    name: "{{ name }}"
    state: poweredon
    folder: "/{{folder}}"
    template: "{{template}}"
    customization:
      hostname: "{{ name }}"
      domain: domain.ru
      dns_servers:
        - "{{ ipa1_dns }}"
        - "{{ ipa2_dns }}"
    networks:
      - name: "{{ network }}"
        type: static
        ip: "{{ip}}"
        netmask: "{{netmask}}"
        gateway: "{{gateway}}"
        wake_on_lan: True
        start_connected: True
        allow_guest_control: True
    wait_for_ip_address: yes
    disk:
      - size_gb: 1
        type: thin
        datastore: "{{datastore}}"
      - size_gb: 20
        type: thin
        datastore: "{{datastore}}"

Qhov no yog qhov peb tau tuaj, qhov system tseem nyob thiab txhim kho.

  • 17 Lub luag haujlwm tseem ceeb rau kev teeb tsa lub server. Txhua lub luag haujlwm yog tsim los daws cov haujlwm sib txawv (kev txiav txim siab, kev tshuaj xyuas, kev tso cai rau tus neeg siv, saib xyuas, thiab lwm yam).
  • Kev ntsuam xyuas lub luag haujlwm. Molecule + TestInfra.
  • Tus kheej txoj kev loj hlob: CMDB + Orchestrator.
  • Lub sijhawm tsim neeg rau zaub mov yog ~ 30 feeb, automated thiab xyaum ua haujlwm ywj pheej ntawm cov neeg ua haujlwm.
  • Tib lub xeev / npe ntawm cov txheej txheem hauv txhua ntu - playbooks, repositories, virtualization ntsiab.
  • Kev tshuaj xyuas txhua hnub ntawm tus neeg rau zaub mov xwm txheej nrog cov ntawv qhia txog qhov tsis sib xws nrog tus qauv.

Kuv vam tias kuv zaj dab neeg yuav muaj txiaj ntsig zoo rau cov neeg uas pib lawv txoj kev mus. Dab tsi automation pawg koj siv?

Tau qhov twg los: www.hab.com