Tsis siv neeg tiam thiab sau cov khoom siv network teeb tsa siv Nornir

Tsis siv neeg tiam thiab sau cov khoom siv network teeb tsa siv Nornir

Hlo Habr!

Tsis ntev los no ib tsab xov xwm tau tshwm sim ntawm no Mikrotik thiab Linux. Kev niaj hnub thiab automation qhov twg ib qho teeb meem zoo sib xws tau daws siv cov fossil txhais tau tias. Thiab txawm hais tias txoj haujlwm yog ib txwm raug, tsis muaj ib yam dab tsi zoo sib xws ntawm nws ntawm Habre. Kuv twv muab kuv lub tsheb kauj vab rau cov zej zog IT uas hwm.

Qhov no tsis yog thawj lub tsheb kauj vab rau txoj haujlwm no. Thawj qhov kev xaiv tau siv ntau xyoo dhau los rov qab rau hauv teb tau version 1.x.x. Lub tsheb kauj vab tsis tshua siv thiab yog li ntawd tsis tu ncua rusted. Hauv kev nkag siab tias txoj haujlwm nws tus kheej tsis tshwm sim ntau npaum li cov hloov kho tshiab teb tau. Thiab txhua zaus koj yuav tsum tau tsav tsheb, cov saw hlau poob los yog lub log poob. Txawm li cas los xij, thawj ntu, tsim kev teeb tsa, ib txwm ua haujlwm kom meej meej, hmoov zoo jinj 2 Lub cav yog tsim los ntev. Tab sis qhov thib ob - dov tawm configs - feem ntau coj surprises. Thiab txij li thaum kuv yuav tsum dov tawm lub config remotely mus rau ib nrab ib puas pab kiag li lawm, ib co ntawm uas nyob rau hauv ntau txhiab kilometers deb, siv cov cuab yeej no yog ib tug me ntsis tho txawv.

Ntawm no kuv yuav tsum lees tias kuv qhov kev tsis paub tseeb feem ntau yuav nyob hauv kuv qhov tsis paub nrog teb tautshaj nyob rau hauv nws shortcomings. Thiab qhov no, los ntawm txoj kev, yog ib qho tseem ceeb. teb tau yog ib qho kev sib cais tag nrho, nws tus kheej thaj tsam ntawm kev paub nrog nws tus kheej DSL (Domain Specific Language), uas yuav tsum tau khaws cia ntawm ib qho kev ntseeg siab. Zoo, lub sijhawm ntawd teb tau Nws tab tom txhim kho sai heev, thiab tsis muaj kev saib xyuas tshwj xeeb rau kev rov qab sib raug zoo, nws tsis ntxiv kev ntseeg siab.

Yog li ntawd, tsis ntev dhau los ib tug thib ob version ntawm lub tsheb kauj vab tau siv. Lub sijhawm no nab hab sej, los yog theej ntawm lub moj khaum sau rau hauv nab hab sej thiab rau nab hab sej hu ua Nornir

Yog li ntawd - Nornir yog microframework sau rau hauv nab hab sej thiab rau nab hab sej thiab tsim rau automation. Ib yam li hauv rooj plaub nrog teb tau, los daws cov teeb meem ntawm no, muaj peev xwm npaj cov ntaub ntawv yuav tsum tau, i.e. Cov khoom muag ntawm cov tswv thiab lawv cov kev txwv, tab sis cov ntawv sau tsis yog nyob rau hauv ib qho DSL cais, tab sis tib yam tsis qub, tab sis zoo heev p[i|i]ton.

Cia peb saib seb nws siv li cas hauv cov piv txwv nyob hauv qab no.

Kuv muaj ib ceg koom tes nrog ntau lub chaw ua haujlwm hauv lub tebchaws. Txhua qhov chaw ua haujlwm muaj WAN router uas txiav tawm ntau txoj kev sib txuas lus los ntawm cov neeg ua haujlwm sib txawv. Txoj cai routing yog BGP. WAN routers tuaj nyob rau hauv ob hom: Cisco ISG lossis Juniper SRX.

Tam sim no txoj haujlwm: koj yuav tsum teeb tsa lub vev xaib tshwj xeeb rau Video Saib Xyuas ntawm qhov chaw nres nkoj cais ntawm txhua WAN routers ntawm ceg network - tshaj tawm cov subnet hauv BGP - teeb tsa qhov ceev txwv ntawm qhov chaw nres nkoj tshwj xeeb.

Ua ntej, peb yuav tsum tau npaj ob peb tus qauv, raws li qhov kev teeb tsa yuav raug tsim tawm cais rau Cisco thiab Juniper. Nws tseem yog ib qho tsim nyog los npaj cov ntaub ntawv rau txhua lub ntsiab lus thiab kev sib txuas tsis sib xws, i.e. sau tib yam khoom muag

Npaj template rau Cisco:

$ cat templates/ios/base.j2 
class-map match-all VIDEO_SURV
 match access-group 111

policy-map VIDEO_SURV
 class VIDEO_SURV
    police 1500000 conform-action transmit  exceed-action drop

interface {{ host.task_data.ifname }}
  description VIDEOSURV
  ip address 10.10.{{ host.task_data.ipsuffix }}.254 255.255.255.0
  service-policy input VIDEO_SURV

router bgp {{ host.task_data.asn }}
  network 10.40.{{ host.task_data.ipsuffix }}.0 mask 255.255.255.0

access-list 11 permit 10.10.{{ host.task_data.ipsuffix }}.0 0.0.0.255
access-list 111 permit ip 10.10.{{ host.task_data.ipsuffix }}.0 0.0.0.255 any

Template rau Juniper:

$ cat templates/junos/base.j2 
set interfaces {{ host.task_data.ifname }} unit 0 description "Video surveillance"
set interfaces {{ host.task_data.ifname }} unit 0 family inet filter input limit-in
set interfaces {{ host.task_data.ifname }} unit 0 family inet address 10.10.{{ host.task_data.ipsuffix }}.254/24
set policy-options policy-statement export2bgp term 1 from route-filter 10.10.{{ host.task_data.ipsuffix }}.0/24 exact
set security zones security-zone WAN interfaces {{ host.task_data.ifname }}
set firewall policer policer-1m if-exceeding bandwidth-limit 1m
set firewall policer policer-1m if-exceeding burst-size-limit 187k
set firewall policer policer-1m then discard
set firewall policer policer-1.5m if-exceeding bandwidth-limit 1500000
set firewall policer policer-1.5m if-exceeding burst-size-limit 280k
set firewall policer policer-1.5m then discard
set firewall filter limit-in term 1 then policer policer-1.5m
set firewall filter limit-in term 1 then count limiter

Templates, ntawm chav kawm, tsis tuaj tawm ntawm huab cua nyias. Cov no yog qhov tseem ceeb sib txawv ntawm cov kev teeb tsa ua haujlwm uas yog thiab tau tom qab daws cov haujlwm ntawm ob lub routers tshwj xeeb ntawm cov qauv sib txawv.

Los ntawm peb cov qauv peb pom tias txhawm rau daws qhov teeb meem, peb tsuas yog xav tau ob qhov kev txwv rau Juniper thiab 3 qhov tsis zoo rau Cisco. ntawm no lawv yog:

  • if npe
  • ipsuffix ua
  • asn

Tam sim no peb yuav tsum teeb tsa cov kev txwv no rau txhua lub cuab yeej, i.e. ua tib yam nkaus cov lus nug.

rau cov lus nug Peb yuav nruj me ntsis ua raws li cov ntaub ntawv Pib pib Norir

uas yog, cia peb tsim tib cov ntaub ntawv skeleton:

.
β”œβ”€β”€ config.yaml
β”œβ”€β”€ inventory
β”‚   β”œβ”€β”€ defaults.yaml
β”‚   β”œβ”€β”€ groups.yaml
β”‚   └── hosts.yaml

Cov ntaub ntawv config.yaml yog tus qauv nonir configuration file

$ cat config.yaml 
---
core:
    num_workers: 10

inventory:
    plugin: nornir.plugins.inventory.simple.SimpleInventory
    options:
        host_file: "inventory/hosts.yaml"
        group_file: "inventory/groups.yaml"
        defaults_file: "inventory/defaults.yaml"

Peb yuav qhia txog qhov tseem ceeb ntawm cov ntaub ntawv hosts yam, pab pawg (hauv kuv cov ntaub ntawv no yog tus ID nkag mus / lo lus zais) hauv pawg.yaml, thiab hauv defaults.yaml Peb yuav tsis qhia dab tsi, tab sis koj yuav tsum nkag mus rau peb qhov minuses - qhia tias nws yog yam l cov ntaub ntawv yog khoob.

Nov yog qhov hosts.yaml zoo li:

---
srx-test:
    hostname: srx-test
    groups: 
        - juniper
    data:
        task_data:
            ifname: fe-0/0/2
            ipsuffix: 111

cisco-test:
    hostname: cisco-test
    groups: 
        - cisco
    data:
        task_data:
            ifname: GigabitEthernet0/1/1
            ipsuffix: 222
            asn: 65111

Thiab ntawm no yog group.yaml:

---
cisco:
    platform: ios
    username: admin1
    password: cisco1

juniper:
    platform: junos
    username: admin2
    password: juniper2

Qhov no yog qhov tshwm sim cov lus nug rau peb txoj haujlwm. Thaum lub sijhawm pib, qhov ntsuas tsis tau los ntawm cov ntaub ntawv khaws cia yog mapped rau cov qauv khoom InventoryElement.

Hauv qab tus spoiler yog daim duab ntawm InventoryElement qauv

print(json.dumps(InventoryElement.schema(), indent=4))
{
    "title": "InventoryElement",
    "type": "object",
    "properties": {
        "hostname": {
            "title": "Hostname",
            "type": "string"
        },
        "port": {
            "title": "Port",
            "type": "integer"
        },
        "username": {
            "title": "Username",
            "type": "string"
        },
        "password": {
            "title": "Password",
            "type": "string"
        },
        "platform": {
            "title": "Platform",
            "type": "string"
        },
        "groups": {
            "title": "Groups",
            "default": [],
            "type": "array",
            "items": {
                "type": "string"
            }
        },
        "data": {
            "title": "Data",
            "default": {},
            "type": "object"
        },
        "connection_options": {
            "title": "Connection_Options",
            "default": {},
            "type": "object",
            "additionalProperties": {
                "$ref": "#/definitions/ConnectionOptions"
            }
        }
    },
    "definitions": {
        "ConnectionOptions": {
            "title": "ConnectionOptions",
            "type": "object",
            "properties": {
                "hostname": {
                    "title": "Hostname",
                    "type": "string"
                },
                "port": {
                    "title": "Port",
                    "type": "integer"
                },
                "username": {
                    "title": "Username",
                    "type": "string"
                },
                "password": {
                    "title": "Password",
                    "type": "string"
                },
                "platform": {
                    "title": "Platform",
                    "type": "string"
                },
                "extras": {
                    "title": "Extras",
                    "type": "object"
                }
            }
        }
    }
}

Cov qauv no tuaj yeem saib me ntsis tsis meej pem, tshwj xeeb tshaj yog thaum xub thawj. Yuav kom xam nws tawm, sib tham sib hom nyob rau hauv ipython.

 $ ipython3
Python 3.6.9 (default, Nov  7 2019, 10:44:02) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.1.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from nornir import InitNornir                                                                           

In [2]: nr = InitNornir(config_file="config.yaml", dry_run=True)                                                

In [3]: nr.inventory.hosts                                                                                      
Out[3]: 
{'srx-test': Host: srx-test, 'cisco-test': Host: cisco-test}

In [4]: nr.inventory.hosts['srx-test'].data                                                                                    
Out[4]: {'task_data': {'ifname': 'fe-0/0/2', 'ipsuffix': 111}}

In [5]: nr.inventory.hosts['srx-test']['task_data']                                                     
Out[5]: {'ifname': 'fe-0/0/2', 'ipsuffix': 111}

In [6]: nr.inventory.hosts['srx-test'].platform                                                                                
Out[6]: 'junos'

Thiab thaum kawg, cia peb mus rau qhov tsab ntawv nws tus kheej. Kuv tsis muaj dab tsi yuav txaus siab tshwj xeeb ntawm no. Kuv nyuam qhuav coj ib qho piv txwv npaj ua los ntawm kev qhia thiab siv nws yuav luag tsis hloov. Nov yog qhov ua tiav daim ntawv ua haujlwm zoo li:

from nornir import InitNornir
from nornir.plugins.tasks import networking, text
from nornir.plugins.functions.text import print_title, print_result

def config_and_deploy(task):
    # Transform inventory data to configuration via a template file
    r = task.run(task=text.template_file,
                 name="Base Configuration",
                 template="base.j2",
                 path=f"templates/{task.host.platform}")

    # Save the compiled configuration into a host variable
    task.host["config"] = r.result

    # Save the compiled configuration into a file
    with open(f"configs/{task.host.hostname}", "w") as f:
        f.write(r.result)

    # Deploy that configuration to the device using NAPALM
    task.run(task=networking.napalm_configure,
             name="Loading Configuration on the device",
             replace=False,
             configuration=task.host["config"])

nr = InitNornir(config_file="config.yaml", dry_run=True) # set dry_run=False, cross your fingers and run again

# run tasks
result = nr.run(task=config_and_deploy)
print_result(result)

Ua tib zoo saib rau qhov parameter dry_run = tseeb hauv kab khoom pib pib nr.
Ntawm no tib yam li hauv teb tau Kev sim khiav tau ua tiav nyob rau hauv qhov kev sib txuas rau lub router tau tsim, kev hloov kho tshiab tau npaj, uas yog tom qab ntawd siv tau los ntawm lub cuab yeej (tab sis qhov no tsis yog qhov tseeb; nws nyob ntawm kev txhawb nqa thiab kev tsav tsheb hauv NAPALM) , tab sis qhov kev teeb tsa tshiab tsis yog siv ncaj qha. Rau kev siv kev sib ntaus sib tua, koj yuav tsum tshem tawm qhov parameter dry_run los yog hloov nws tus nqi rau cuav.

Thaum tsab ntawv raug ua tiav, Nornir tawm cov ncauj lus kom ntxaws rau lub console.

Hauv qab no tus spoiler yog cov zis ntawm kev sib ntaus sib tua ntawm ob lub routers:

config_and_deploy***************************************************************
* cisco-test ** changed : True *******************************************
vvvv config_and_deploy ** changed : True vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- Base Configuration ** changed : True ------------------------------------- INFO
class-map match-all VIDEO_SURV
 match access-group 111

policy-map VIDEO_SURV
 class VIDEO_SURV
    police 1500000 conform-action transmit  exceed-action drop

interface GigabitEthernet0/1/1
  description VIDEOSURV
  ip address 10.10.222.254 255.255.255.0
  service-policy input VIDEO_SURV

router bgp 65001
  network 10.10.222.0 mask 255.255.255.0

access-list 11 permit 10.10.222.0 0.0.0.255
access-list 111 permit ip 10.10.222.0 0.0.0.255 any
---- Loading Configuration on the device ** changed : True --------------------- INFO
+class-map match-all VIDEO_SURV
+ match access-group 111
+policy-map VIDEO_SURV
+ class VIDEO_SURV
+interface GigabitEthernet0/1/1
+  description VIDEOSURV
+  ip address 10.10.222.254 255.255.255.0
+  service-policy input VIDEO_SURV
+router bgp 65001
+  network 10.10.222.0 mask 255.255.255.0
+access-list 11 permit 10.10.222.0 0.0.0.255
+access-list 111 permit ip 10.10.222.0 0.0.0.255 any
^^^^ END config_and_deploy ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* srx-test ** changed : True *******************************************
vvvv config_and_deploy ** changed : True vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- Base Configuration ** changed : True ------------------------------------- INFO
set interfaces fe-0/0/2 unit 0 description "Video surveillance"
set interfaces fe-0/0/2 unit 0 family inet filter input limit-in
set interfaces fe-0/0/2 unit 0 family inet address 10.10.111.254/24
set policy-options policy-statement export2bgp term 1 from route-filter 10.10.111.0/24 exact
set security zones security-zone WAN interfaces fe-0/0/2
set firewall policer policer-1m if-exceeding bandwidth-limit 1m
set firewall policer policer-1m if-exceeding burst-size-limit 187k
set firewall policer policer-1m then discard
set firewall policer policer-1.5m if-exceeding bandwidth-limit 1500000
set firewall policer policer-1.5m if-exceeding burst-size-limit 280k
set firewall policer policer-1.5m then discard
set firewall filter limit-in term 1 then policer policer-1.5m
set firewall filter limit-in term 1 then count limiter
---- Loading Configuration on the device ** changed : True --------------------- INFO
[edit interfaces]
+   fe-0/0/2 {
+       unit 0 {
+           description "Video surveillance";
+           family inet {
+               filter {
+                   input limit-in;
+               }
+               address 10.10.111.254/24;
+           }
+       }
+   }
[edit]
+  policy-options {
+      policy-statement export2bgp {
+          term 1 {
+              from {
+                  route-filter 10.10.111.0/24 exact;
+              }
+          }
+      }
+  }
[edit security zones]
     security-zone test-vpn { ... }
+    security-zone WAN {
+        interfaces {
+            fe-0/0/2.0;
+        }
+    }
[edit]
+  firewall {
+      policer policer-1m {
+          if-exceeding {
+              bandwidth-limit 1m;
+              burst-size-limit 187k;
+          }
+          then discard;
+      }
+      policer policer-1.5m {
+          if-exceeding {
+              bandwidth-limit 1500000;
+              burst-size-limit 280k;
+          }
+          then discard;
+      }
+      filter limit-in {
+          term 1 {
+              then {
+                  policer policer-1.5m;
+                  count limiter;
+              }
+          }
+      }
+  }
^^^^ END config_and_deploy ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Khaws cov passwords hauv ansible_vault

Thaum pib ntawm tsab xov xwm kuv mus me ntsis overboard teb tau, tab sis nws tsis yog txhua yam uas phem. Kuv nyiam lawv heev vault zoo li, uas yog tsim los nkaum cov ntaub ntawv rhiab tawm ntawm qhov pom. Thiab tej zaum ntau tus tau pom tias peb muaj tag nrho cov logins / password rau txhua qhov kev sib ntaus sib tua routers ci ntsa iab hauv daim ntawv qhib hauv cov ntaub ntawv gorups.yaml. Nws tsis zoo nkauj, tau kawg. Wb tiv thaiv cov ntaub ntawv no nrog vault.

Cia peb hloov cov kev txwv los ntawm groups.yaml mus rau creds.yaml, thiab encrypt nws nrog AES256 nrog 20 tus lej lo lus zais:

$ cd inventory
$ cat creds.yaml
---
cisco:
    username: admin1
    password: cisco1

juniper:
    username: admin2
    password: juniper2

$ pwgen 20 -N 1 > vault.passwd
ansible-vault encrypt creds.yaml --vault-password-file vault.passwd  
Encryption successful
$ cat creds.yaml 
$ANSIBLE_VAULT;1.1;AES256
39656463353437333337356361633737383464383231366233386636333965306662323534626131
3964396534396333363939373539393662623164373539620a346565373439646436356438653965
39643266333639356564663961303535353364383163633232366138643132313530346661316533
6236306435613132610a656163653065633866626639613537326233653765353661613337393839
62376662303061353963383330323164633162386336643832376263343634356230613562643533
30363436343465306638653932366166306562393061323636636163373164613630643965636361
34343936323066393763323633336366366566393236613737326530346234393735306261363239
35663430623934323632616161636330353134393435396632663530373932383532316161353963
31393434653165613432326636616636383665316465623036376631313162646435

Nws yog qhov yooj yim. Nws tseem yuav qhia peb Nornir-script los retrieve thiab siv cov ntaub ntawv no.
Ua li no, hauv peb tsab ntawv tom qab kab pib pib nr = InitNornir(config_file=… ntxiv cov cai hauv qab no:

...
nr = InitNornir(config_file="config.yaml", dry_run=True) # set dry_run=False, cross your fingers and run again

# enrich Inventory with the encrypted vault data
from ansible_vault import Vault
vault_password_file="inventory/vault.passwd"
vault_file="inventory/creds.yaml"
with open(vault_password_file, "r") as fp:
    password = fp.readline().strip()   
    vault = Vault(password)
    vaultdata = vault.load(open(vault_file).read())

for a in nr.inventory.hosts.keys():
    item = nr.inventory.hosts[a]
    item.username = vaultdata[item.groups[0]]['username']
    item.password = vaultdata[item.groups[0]]['password']
    #print("hostname={}, username={}, password={}n".format(item.hostname, item.username, item.password))

# run tasks
...

Tau kawg, vault.passwd yuav tsum tsis txhob nyob ib sab ntawm creds.yaml raws li hauv kuv qhov piv txwv. Tab sis nws zoo rau kev ua si.

Qhov ntawd yog tag nrho rau tam sim no. Muaj ob peb tsab xov xwm ntxiv txog Cisco + Zabbix tuaj, tab sis qhov no tsis yog me ntsis txog automation. Thiab nyob rau yav tom ntej no kuv npaj yuav sau txog RESTCONF hauv Cisco.

Tau qhov twg los: www.hab.com

Ntxiv ib saib