Hello, Habr!
Недavno se pojavila članak gdje su slične zadatke rješavali fosilnim sredstvima. I iako je zadatak potpuno tipičan, o njemu se na Habr-u ne može pronaći ništa slično. Usudim se predložiti uvaženom IT zajednici svoj bicikl.
Ovo nije prvi bicikl za sličan zadatak. Prva verzija je realizirana prije nekoliko godina na ansible verziji 1.x.x. Bicikl se rijetko koristio i stoga je stalno hrđavio. U tom smislu, zadatak se ne pojavljuje tako često koliko se verzije ažuriraju ansible. I svaki put kada je potrebno raditi, ili lanac ispada, ili kotač otpada. Ipak, prvi dio, generacija konfiguracija — uvijek radi vrlo točno, srećom jinja2 motor je već dugo utvrđen. Međutim, drugi dio — implementacija konfiguracija, obično donosi iznenađenja. A kako konfiguracije moram implementirati udaljeno na pola stotina uređaja, od kojih se neki nalaze tisućama kilometara daleko, korištenje ovog alata bilo je malo zastrašujuće.
Treba priznati da je moja neodlučnost više vezana uz nedovoljno poznavanje ansible, nego u njegove nedostatke. I to je, usput, važna tačka. ansible — to je potpuno drugačija, vlastita domena znanja sa svojim vlastitim DSL (Jezik specifičan za domen), koji je potrebno održavati na sigurnom nivou. A taj faktor ansible se vrlo brzo razvija, i to bez previše obzira na unazadnu kompatibilnost, ne dodaje povjerenje.
Zbog toga je nedavno realizirana druga varijanta bicikla. Ovaj put na python, tačnije na okviru napisanom na python in za python poimenovano
Dakle — Nornir to je mikro okvir, napisan na python in za python i namijenjen za automatizaciju. Takođe, kao i u slučaju sa ansible, za rješavanje zadataka ovdje je potrebna pravilna priprema podataka, tj. inventarizacija hostova i njihovih parametara, a scenariji se ne pišu na posebnom DSL-u, već svi na istom onom ne tako starom, ali veoma dobrom p[и|й]tonu.
Hajde da pogledamo kakva je to stvar na sljedećem živom primeru.
Imam filijalnu mrežu sa nekoliko desetina kancelarija širom zemlje. U svakoj kancelariji postoji WAN usmjerivač, koji terminira nekoliko komunikacijskih kanala od različitih operatora. Protokol usmjeravanja — BGP. WAN usmjerivači dolaze u dvije vrste: Cisco ISG ili Juniper SRX.
Sedaj je naloga: potrebno je konfigurirati na vseh WAN usmerjevalnikih podrejeno omrežje za video nadzor na ločenem priključku — napovedati to omrežje v BGP — konfigurirati omejitev hitrosti ločenega priključka.
Najprej moramo pripraviti par predlog, na podlagi katerih se bodo generirale konfiguracije posebej za Cisco in Juniper. Prav tako je potrebno pripraviti podatke za vsako točko in parametre povezave, tj. zbrati tisto inventuro.
Pripravljena predloga za Cisco:
$ cat templates/ios/base.j2
class-map match-all VIDEO_SURV
match access-group 111
policy-map VIDEO_SURV
class VIDEO_SURV
police 1500000 conform-action transmit exceed-action drop
interface {{ host.task_data.ifname }}
description VIDEOSURV
ip address 10.10.{{ host.task_data.ipsuffix }}.254 255.255.255.0
service-policy input VIDEO_SURV
router bgp {{ host.task_data.asn }}
network 10.40.{{ host.task_data.ipsuffix }}.0 mask 255.255.255.0
access-list 11 permit 10.10.{{ host.task_data.ipsuffix }}.0 0.0.0.255
access-list 111 permit ip 10.10.{{ host.task_data.ipsuffix }}.0 0.0.0.255 anyPredloga za Juniper:
$ cat templates/junos/base.j2
set interfaces {{ host.task_data.ifname }} unit 0 description "Video surveillance"
set interfaces {{ host.task_data.ifname }} unit 0 family inet filter input limit-in
set interfaces {{ host.task_data.ifname }} unit 0 family inet address 10.10.{{ host.task_data.ipsuffix }}.254/24
set policy-options policy-statement export2bgp term 1 from route-filter 10.10.{{ host.task_data.ipsuffix }}.0/24 exact
set security zones security-zone WAN interfaces {{ host.task_data.ifname }}
set firewall policer policer-1m if-exceeding bandwidth-limit 1m
set firewall policer policer-1m if-exceeding burst-size-limit 187k
set firewall policer policer-1m then discard
set firewall policer policer-1.5m if-exceeding bandwidth-limit 1500000
set firewall policer policer-1.5m if-exceeding burst-size-limit 280k
set firewall policer policer-1.5m then discard
set firewall filter limit-in term 1 then policer policer-1.5m
set firewall filter limit-in term 1 then count limiterPredloge, seveda, ne prihajajo kar tako. To so v bistvu difi med delujočimi konfiguracijami pred-nato in po-rešeni nalogi na dveh konkretnih usmerjevalnikih različnih modelov.
Iz naših predlog vidimo, da za rešitev naloge potrebujemo dva parametra za Juniper in 3 parametre za Cisco. Tukaj so:
- ifname
- ipsuffix
- asn
Sedaj moramo te parametre določiti za vsako napravo, tj. narediti tisto inventar.
Za inventar bomo natančno sledili dokumentaciji
tj. ustvarili bomo enako strukturo datotek:
.
├── config.yaml
├── inventory
│ ├── defaults.yaml
│ ├── groups.yaml
│ └── hosts.yamlDatoteka config.yaml — standardna konfiguracijska datoteka nornir
$ cat config.yaml
---
core:
num_workers: 10
inventory:
plugin: nornir.plugins.inventory.simple.SimpleInventory
options:
host_file: "inventory/hosts.yaml"
group_file: "inventory/groups.yaml"
defaults_file: "inventory/defaults.yaml"Glavne parametre bomo navedli v datoteki hosts.yaml, skupinske (v mojem primeru so to prijave/gesla) v groups.yaml, već u defaults.yaml ničesar ne bomo navedli, a v to je treba vpisati tri minuse — ki označujejo, da gre za yaml datoteko, čeprav je prazna.
Tako izgleda hosts.yaml:
---
srx-test:
hostname: srx-test
groups:
- juniper
data:
task_data:
ifname: fe-0/0/2
ipsuffix: 111
cisco-test:
hostname: cisco-test
groups:
- cisco
data:
task_data:
ifname: GigabitEthernet0/1/1
ipsuffix: 222
asn: 65111Tako izgleda groups.yaml:
---
cisco:
platform: ios
username: admin1
password: cisco1
juniper:
platform: junos
username: admin2
password: juniper2Tako je bilo doseženo inventar za našo nalogo. Pri inicializaciji se parametri iz inventarnih datotek preslikajo na objektni model InventoryElement.
Pod spenjem je shema modela InventoryElement
print(json.dumps(InventoryElement.schema(), indent=4))
{
"title": "InventoryElement",
"type": "object",
"properties": {
"hostname": {
"title": "Hostname",
"type": "string"
},
"port": {
"title": "Port",
"type": "integer"
},
"username": {
"title": "Username",
"type": "string"
},
"password": {
"title": "Password",
"type": "string"
},
"platform": {
"title": "Platform",
"type": "string"
},
"groups": {
"title": "Groups",
"default": [],
"type": "array",
"items": {
"type": "string"
}
},
"data": {
"title": "Data",
"default": {},
"type": "object"
},
"connection_options": {
"title": "Connection_Options",
"default": {},
"type": "object",
"additionalProperties": {
"$ref": "#\/definitions\/ConnectionOptions"
}
}
},
"definitions": {
"ConnectionOptions": {
"title": "ConnectionOptions",
"type": "object",
"properties": {
"hostname": {
"title": "Hostname",
"type": "string"
},
"port": {
"title": "Port",
"type": "integer"
},
"username": {
"title": "Username",
"type": "string"
},
"password": {
"title": "Password",
"type": "string"
},
"platform": {
"title": "Platform",
"type": "string"
},
"extras": {
"title": "Extras",
"type": "object"
}
}
}
}
}Ta model je lahko dokaj zapleten, še posebej na začetku. Z interaktivnim načinom dela se zelo olajša razumevanje. ipython.
$ ipython3
Python 3.6.9 (default, Nov 7 2019, 10:44:02)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.1.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: from nornir import InitNornir
In [2]: nr = InitNornir(config_file="config.yaml", dry_run=True)
In [3]: nr.inventory.hosts
Out[3]:
{'srx-test': Host: srx-test, 'cisco-test': Host: cisco-test}
In [4]: nr.inventory.hosts['srx-test'].data
Out[4]: {'task_data': {'ifname': 'fe-0/0/2', 'ipsuffix': 111}}
In [5]: nr.inventory.hosts['srx-test']['task_data']
Out[5]: {'ifname': 'fe-0/0/2', 'ipsuffix': 111}
In [6]: nr.inventory.hosts['srx-test'].platform
Out[6]: 'junos'
In conclusion, let’s move on to the script itself. There isn’t much to be proud of here. I just took a ready-made example from and used it almost unchanged. Here is how the final working script looks:
from nornir import InitNornir
from nornir.plugins.tasks import networking, text
from nornir.plugins.functions.text import print_title, print_result
def config_and_deploy(task):
# Transform inventory data to configuration via a template file
r = task.run(task=text.template_file,
name="Base Configuration",
template="base.j2",
path=f"templates/{task.host.platform}")
# Save the compiled configuration into a host variable
task.host["config"] = r.result
# Save the compiled configuration into a file
with open(f"configs/{task.host.hostname}", "w") as f:
f.write(r.result)
# Deploy that configuration to the device using NAPALM
task.run(task=networking.napalm_configure,
name="Loading Configuration on the device",
replace=False,
configuration=task.host["config"])
nr = InitNornir(config_file="config.yaml", dry_run=True) # set dry_run=False, cross your fingers and run again
# run tasks
result = nr.run(task=config_and_deploy)
print_result(result)Obratite pažnju na parametar dry_run=True in the object initialization line nr.
Here, as in ansible a test run is implemented where a connection to the router is established, a new modified configuration is prepared, which is then validated by the device (but this is not certain; it depends on the device's support and the implementation of the driver in NAPALM), but the new configuration is not actually applied. For production use, you need to remove the parameter dry_run or change its value to False.
When executing the Nornir script, detailed logs are output to the console.
Below the spoiler is the output of the production run on two test routers:
config_and_deploy***************************************************************
* cisco-test ** changed : True *******************************************
vvvv config_and_deploy ** changed : True vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- Base Configuration ** changed : True ------------------------------------- INFO
class-map match-all VIDEO_SURV
match access-group 111
policy-map VIDEO_SURV
class VIDEO_SURV
police 1500000 conform-action transmit exceed-action drop
interface GigabitEthernet0/1/1
description VIDEOSURV
ip address 10.10.222.254 255.255.255.0
service-policy input VIDEO_SURV
router bgp 65001
network 10.10.222.0 mask 255.255.255.0
access-list 11 permit 10.10.222.0 0.0.0.255
access-list 111 permit ip 10.10.222.0 0.0.0.255 any
---- Loading Configuration on the device ** changed : True --------------------- INFO
+class-map match-all VIDEO_SURV
+ match access-group 111
+policy-map VIDEO_SURV
+ class VIDEO_SURV
+interface GigabitEthernet0/1/1
+ description VIDEOSURV
+ ip address 10.10.222.254 255.255.255.0
+ service-policy input VIDEO_SURV
+router bgp 65001
+ network 10.10.222.0 mask 255.255.255.0
+access-list 11 permit 10.10.222.0 0.0.0.255
+access-list 111 permit ip 10.10.222.0 0.0.0.255 any
^^^^ END config_and_deploy ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* srx-test ** changed : True *******************************************
vvvv config_and_deploy ** changed : True vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
---- Base Configuration ** changed : True ------------------------------------- INFO
set interfaces fe-0/0/2 unit 0 description "Video surveillance"
set interfaces fe-0/0/2 unit 0 family inet filter input limit-in
set interfaces fe-0/0/2 unit 0 family inet address 10.10.111.254/24
set policy-options policy-statement export2bgp term 1 from route-filter 10.10.111.0/24 exact
set security zones security-zone WAN interfaces fe-0/0/2
set firewall policer policer-1m if-exceeding bandwidth-limit 1m
set firewall policer policer-1m if-exceeding burst-size-limit 187k
set firewall policer policer-1m then discard
set firewall policer policer-1.5m if-exceeding bandwidth-limit 1500000
set firewall policer policer-1.5m if-exceeding burst-size-limit 280k
set firewall policer policer-1.5m then discard
set firewall filter limit-in term 1 then policer policer-1.5m
set firewall filter limit-in term 1 then count limiter
---- Loading Configuration on the device ** changed : True --------------------- INFO
[edit interfaces]
+ fe-0/0/2 {
+ unit 0 {
+ description "Video surveillance";
+ family inet {
+ filter {
+ input limit-in;
+ }
+ address 10.10.111.254/24;
+ }
+ }
+ }
[edit]
+ policy-options {
+ policy-statement export2bgp {
+ term 1 {
+ from {
+ route-filter 10.10.111.0/24 exact;
+ }
+ }
+ }
+ }
[edit security zones]
security-zone test-vpn { ... }
+ security-zone WAN {
+ interfaces {
+ fe-0/0/2.0;
+ }
+ }
[edit]
+ firewall {
+ policer policer-1m {
+ if-exceeding {
+ bandwidth-limit 1m;
+ burst-size-limit 187k;
+ }
+ then discard;
+ }
+ policer policer-1.5m {
+ if-exceeding {
+ bandwidth-limit 1500000;
+ burst-size-limit 280k;
+ }
+ then discard;
+ }
+ filter limit-in {
+ term 1 {
+ then {
+ policer policer-1.5m;
+ count limiter;
+ }
+ }
+ }
+ }
^^^^ END config_and_deploy ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Skrivenje lozinki u ansible_vault
Na početku članka malo sam kritikovao ansible, ali nije sve tako loše. Zaista mi se sviđa kod njih vault который предназначен для сокрытия чувствительной информации из виду. И, вероятно, многие заметили, что у нас все логины/пароли ко всем боевым маршрутизаторам отображаются в открытом виде в файле gorups.yaml. Это, конечно, выглядит некрасиво. Давайте защитим эти данные с помощью vault.
Переносим параметры из groups.yaml в creds.yaml и зашифруем его с помощью AES256 с 20-значным паролем:
$ cd inventory
$ cat creds.yaml
---
cisco:
username: admin1
password: cisco1
juniper:
username: admin2
password: juniper2
$ pwgen 20 -N 1 > vault.passwd
ansible-vault encrypt creds.yaml --vault-password-file vault.passwd
Шифрование успешно
$ cat creds.yaml
$ANSIBLE_VAULT;1.1;AES256
39656463353437333337356361633737383464383231366233386636333965306662323534626131
3964396534396333363939373539393662623164373539620a346565373439646436356438653965
39643266333639356564663961303535353364383163633232366138643132313530346661316533
6236306435613132610a656163653065633866626639613537326233653765353661613337393839
62376662303061353963383330323164633162386336643832376263343634356230613562643533
30363436343465306638653932366166306562393061323636636163373164613630643965636361
34343936323066393763323633336366366566393236613737326530346234393735306261363239
35663430623934323632616161636330353134393435396632663530373932383532316161353963
31393434653165613432326636616636383665316465623036376631313162646435Вот так просто. Осталось научить наш Nornir-скрипт извлекать и использовать эти данные.
Для этого в нашем скрипте после строки инициализации nr = InitNornir(config_file=… добавим следующий код:
...
nr = InitNornir(config_file="config.yaml", dry_run=True) # установите dry_run=False, скрестив пальцы, запустите снова
# дополните инвентарь зашифрованными данными
from ansible_vault import Vault
vault_password_file="inventory/vault.passwd"
vault_file="inventory/creds.yaml"
with open(vault_password_file, "r") as fp:
password = fp.readline().strip()
vault = Vault(password)
vaultdata = vault.load(open(vault_file).read())
for a in nr.inventory.hosts.keys():
item = nr.inventory.hosts[a]
item.username = vaultdata[item.groups[0]]['username']
item.password = vaultdata[item.groups[0]]['password']
#print("hostname={}, username={}, password={}n".format(item.hostname, item.username, item.password))
# запустите задачи
...Естественно, vault.passwd не должен лежать рядом с creds.yaml, как в моем примере. Но для экспериментов сойдет.
На этом пока все. В ближайшее время выйдет еще пара статей про Cisco + Zabbix, но это немного не о автоматизации. А в будущем планирую написать о RESTCONF в Cisco.
Izvor: habr.com
