Maanta, waxaa jira xalal diyaarsan (la-hawlgal ah) oo loogu talagalay la socodka socodka IP (TS), tusaale ahaan и , waxay leeyihiin hawlo badan oo qani ah, sida caadiga ah hawl wadeenada waaweyn ee ka shaqeeya adeegyada TV-ga ayaa leh xal isku mid ah. Maqaalkani waxa uu qeexayaa xal ku salaysan mashruuc il furan , loogu talagalay xakamaynta ugu yar ee socodka IP (TS) iyada oo la adeegsanayo xisaabiyaha CC (continuity counter) counter iyo bitrate. Codsiga suurtogalka ah waa la socoshada luminta xirmooyinka ama qulqulka oo dhan iyada oo loo marayo kanaalka L2 ee la kiraysto (kaas oo aan si caadi ah loola socon karin, tusaale ahaan, akhrinta xisaabaadka khasaaraha ee safafka).
Si kooban oo ku saabsan TDuck
TSDuck waa il furan (laysanka BSD 2-Clause) software (oo ah qalab konsole iyo maktabad aad ku horumarinayso yutiilitidaada ama plugins) si aad wax uga qabato durdurrada TS. Fikrad ahaan, waxay la shaqeyn kartaa IP (multicast/unicast), http, hls, dvb tuners, dektec dvb-asi demodulator, waxaa jira koronto-dhaliye TS gudaha ah iyo akhrinta faylasha. Wax soo saarku wuxuu noqon karaa in lagu duubo fayl, IP (multicast/unicast), hls, dektec dvb-asi iyo HiDes modulators, ciyaartoyda (mplayer, vlc, xine) iyo dhibic. Inta u dhaxaysa gelinta iyo soo saarista, waxaad awood u siin kartaa soo-saareyaasha taraafikada kala duwan, tusaale ahaan, dib-u-habaynta PID-yada, samaynta xoqidda/kala-guurka, falanqaynta xisaabiyeyaasha CC, xisaabinta bitrate-ka iyo hawlgallada kale ee caadiga ah ee durdurrada TS.
Maqaalkan, durdurrada IP-ga (multicast) ayaa loo isticmaali doonaa sidii gelinta, soo-saareyaasha bitrate_monitor (magaca way caddahay waxa tani ay tahay) iyo sii wadida ( falanqaynta CC counter) soo-saareyaasha ayaa la isticmaali doonaa. Dhibaato la'aanteed, waxaad ku beddeli kartaa multicast-ka IP-ga nooc kale oo wax-gelinta ah oo ay taageerto TDuck.
La heli karo TSDuck inta badan OS hadda jira. Ma jiro mid loogu talagalay Debian, laakiin waxaan ku guuleysanay inaan u uruurino Debian 8 iyo Debian 10 dhib la'aan.
Marka xigta, nooca TSDuck 3.19-1520 ayaa la isticmaalaa, Linux waxaa loo isticmaalaa OS ahaan (debian 10 waxaa loo adeegsaday diyaarinta xalka, CentOS 7 waxaa loo adeegsaday isticmaalka dhabta ah)
Diyaarinta TSDuck iyo OS
Kahor intaadan la socon socodka dhabta ah, waxaad u baahan tahay inaad hubiso in TDuck ay si sax ah u shaqeyso iyo in dhibciyadu aysan ku dhicin kaarka shabakada ama heerka OS (socket). Tan waxaa loo baahan yahay si aadan u malaynin hadhow meesha dhibco ka dhaceen - shabakadda ama "gudaha server-ka." Waxaad ku hubin kartaa dhibcaha heerka kaadhka shabakada amarka ethtool -S ethX, hagaajinta waxaa lagu sameeyaa isla ethtool (badanaa waxaad u baahan tahay inaad kordhiso kaydiyaha RX (-G) oo marmarka qaarkood joojiso qaar ka mid ah offloads (-K)). Talo guud ahaan, waxaa lagu talinayaa in la isticmaalo deked gaar ah si loo helo taraafikada la falanqeeyay, haddii ay suurtagal tahay, tani waxay yareyn doontaa been-abuurka sababtoo ah xaqiiqda ah in hoos u dhacu uu isku mar ku dhacay dekedda falanqaynta sababtoo ah joogitaanka taraafikada kale. Haddii tani aysan suurtagal ahayn (waxaad isticmaaleysaa mini-computer / NUC oo leh hal deked), markaa waxaa aad loogu talinayaa in la dejiyo mudnaanta taraafikada la falanqeeyay ee la xiriirta inta kale ee aaladda uu falanqeeyuhu ku xiran yahay. Ku saabsan bay'adaha farsamada, halkan waxaad u baahan tahay inaad taxaddar muujiso oo aad awoodo inaad hesho dhibco baakidh ah oo ka bilaabmaya dekedda jireed oo ku dhammaanaysa codsiga gudaha mashiinka farsamada.
Abuuritaanka iyo helitaanka qulqulka dhex-dhexaadiyaha
Talaabada ugu horeysa ee diyaarinta TDuck, waxaanu soo saari doonaa oo ku heli doonaa taraafikada hal marti-geliyaha anagoo adeegsanayna shabakadaha.
Diyaarinta deegaanka:
ip netns add P #создаём netns P, в нём будет происходить анализ трафика
ip link add type veth #создаём veth-пару - veth0 оставляем в netns по умолчанию (в этот интерфейс будет генерироваться трафик)
ip link set dev veth1 netns P #veth1 - помещаем в netns P (на этом интерфейсе будет приём трафика)
ip netns exec P ifconfig veth1 192.0.2.1/30 up #поднимаем IP на veth1, не имеет значения какой именно
ip netns exec P ip ro add default via 192.0.2.2 #настраиваем маршрут по умолчанию внутри nents P
sysctl net.ipv6.conf.veth0.disable_ipv6=1 #отключаем IPv6 на veth0 - это делается для того, чтобы в счётчик TX не попадал посторонний мусор
ifconfig veth0 up #поднимаем интерфейс veth0
ip route add 239.0.0.1 dev veth0 #создаём маршрут, чтобы ОС направляла трафик к 239.0.0.1 в сторону veth0Deegaanku waa diyaar. Bilow falanqaynta taraafikada:
ip netns exec P tsp --realtime -t
-I ip 239.0.0.1:1234
-P continuity
-P bitrate_monitor -p 1 -t 1
-O dropMeesha "-p 1 -t 1" macnaheedu waa inaad u baahan tahay inaad xisaabiso bitrate ilbiriqsi kasta oo aad muujiso macluumaadka ku saabsan bitrate ilbiriqsi kasta
Waxaan bilownaa koronto-dhaliye taraafiko ah oo xawlihiisu yahay 10 Mbit/s:
tsp -I craft
-P regulate -b 10000000
-O ip -p 7 -e --local-port 6000 239.0.0.1:1234meesha "-p 7 -e" macnaheedu waa inaad u baahan tahay inaad ku xidhid 7 baakidh TS 1 baakidh IP ah oo aad si adag u sameyso (-e), i.e. had iyo jeer sug xirmooyinka 7 TS ee processor-ka ugu dambeeya ka hor inta aadan dirin sameynta xirmo IP ah.
Falanqeeyaha ayaa bilaabaya soo bandhigida fariimaha la filayo:
* 2020/01/03 14:55:44 - bitrate_monitor: 2020/01/03 14:55:44, TS bitrate: 9,970,016 bits/s
* 2020/01/03 14:55:45 - bitrate_monitor: 2020/01/03 14:55:45, TS bitrate: 10,022,656 bits/s
* 2020/01/03 14:55:46 - bitrate_monitor: 2020/01/03 14:55:46, TS bitrate: 9,980,544 bits/sHadda aan ku darno xoogaa dhibco ah:
ip netns exec P iptables -I INPUT -d 239.0.0.1 -m statistic --mode random --probability 0.001 -j DROPwaxaana soo baxay fariimahan oo kale:
* 2020/01/03 14:57:11 - continuity: packet index: 80,745, PID: 0x0000, missing 7 packets
* 2020/01/03 14:57:11 - continuity: packet index: 83,342, PID: 0x0000, missing 7 packets taasoo la filayo. Waxaan joojinaa khasaaraha xirmada (ip netns exec P iptables -F) waxaanan isku daynaa in aan kordhino koronto-dhaliyaha ilaa 100 Mbit/s. Falanqeeyaha wuxuu soo sheegaa khaladaad badan oo CC ah iyo qiyaastii 75 Mbit/s halkii 100. Waxaan isku dayeynaa inaan ogaano cidda eedda leh - koronto-dhaliye ma ilaalinayo ama dhibaatadu kuma jirto, si tan loo sameeyo waxaan bilaabeynaa abuurista baakado go'an (700000 xirmooyinka TS = 100000 baakidh IP):
# ifconfig veth0 | grep TX
TX packets 151825460 bytes 205725459268 (191.5 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
# tsp -I craft -c 700000 -P regulate -b 100000000 -P count -O ip -p 7 -e --local-port 6000 239.0.0.1:1234
* count: PID 0 (0x0000): 700,000 packets
# ifconfig veth0 | grep TX
TX packets 151925460 bytes 205861259268 (191.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0Sida aad arki karto, 100000 oo xirmo IP ah ayaa la sameeyay (151925460-151825460). Marka waxaan ogaanay waxa ku dhacaya falanqeeyayaasha, si aan tan u sameyno waxaan ku hubineynaa miiska RX ee veth1, waxay si adag ula mid tahay miiska TX ee veth0, ka dib waxaan eegnaa waxa ka dhacaya heerka godka:
# ip netns exec P cat /proc/net/udp
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
133: 010000EF:04D2 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 72338 2 00000000e0a441df 24355 Halkan waxaad ku arki kartaa tirada dhibcaha = 24355. Baakadaha TS tani waa 170485 ama 24.36% ee 700000, markaa waxaan aragnaa in isla 25% ee bitrate-ka lumay ay yihiin dhibco ku jira godka UDP. Ku-dhicista godka UDP waxay badanaa ku dhacdaa bakhti la'aan awgeed, aynu aragno waxa cabbirka barafeeyaha caadiga ah iyo cabbirka ugu sarreeya ee godadku ay yihiin:
# sysctl net.core.rmem_default
net.core.rmem_default = 212992
# sysctl net.core.rmem_max
net.core.rmem_max = 212992Markaa, haddii codsiyadu aanay si cad u codsan cabbirka bakhaarrada, saldhigyada waxa lagu abuuray kayd 208 KB ah, laakiin haddii ay wax ka badan codsadaan, weli ma heli doonaan waxay codsadeen. Maaddaama gudaha tsp aad u dejin karto cabbirka wax-ka-qabashada ee galitaanka IP-ga (--size-size), ma taaban doono cabbirka godka caadiga ah, laakiin waxaan dejin doonaa oo keliya cabbirka bareegga ugu sarreeya oo u qeex cabbirka cabbirka si cad iyada oo loo marayo doodaha tsp:
sysctl net.core.rmem_max=8388608
ip netns exec P tsp --realtime -t -I ip 239.0.0.1:1234 -b 8388608 -P continuity -P bitrate_monitor -p 1 -t 1 -O dropIyada oo la hagaajinayo baqshadda godka, qiyaasta bitrate-ka la soo sheegay hadda waa ku dhawaad 100Mbps, ma jiraan khaladaad CC ah.
Iyada oo ku saleysan isticmaalka CPU ee codsiga tsp laftiisa. Marka laga hadlayo hal xudunta u ah i5-4260U CPU @ 1.40GHz, si loo falanqeeyo socodka 10Mbit/s, 3-4% CPU ayaa loo baahan doonaa, 100Mbit/s - 25%, 200Mbit/s - 46%. Marka la dejinayo % khasaaraha xirmada, culeyska CPU ficil ahaan ma kordhiyo (laakiin waxaa laga yaabaa inuu yaraado).
Qalab wax soo saar badan leh, waxaa suurtagal ahayd in la soo saaro oo la falanqeeyo durdurrada in ka badan 1Gb/s dhib la'aan.
Tijaabinta kaararka shabakada dhabta ah
Ka dib markaad tijaabiso lammaane veth, waxaad u baahan tahay inaad qaadato laba marti-geliyaha ama laba dekedood oo hal marti-geliyaha ah, ku xidh dekedaha midba midka kale, ku socodsii koronto-dhaliyaha mid, iyo falanqaynta labaad. Ma jiraan wax la yaab leh halkan, laakiin dhab ahaantii waxay dhammaan ku xiran tahay qalabka, daciifnimada, ayaa aad u xiiso badan halkan.
Isticmaalka xogta la helay ee nidaamka la socodka (Zabbix)
tsp ma laha API mashiinka lagu akhriyi karo sida SNMP ama wax la mid ah. Farriimaha CC waxay u baahan yihiin in la isku geeyo ugu yaraan 1 ilbiriqsi markiiba (iyadoo ay jirto boqolkiiba sare ee khasaaraha xirmada, waxaa jiri kara boqollaal/kun/tobanaan kun ilbiriqsikii, iyadoo ku xiran qiyaasta bitrate).
Markaa, si loo badbaadiyo macluumaadka oo loo sawiro garaafyada khaladaadka CC iyo bitrate oo loo sameeyo nooc ka mid ah shilalka, waxaa jiri kara xulashooyinka soo socda:
- Kala saar oo isku geyn (by CC) wax soo saarka tsp, i.e. u beddel qaabka la rabo.
- Ku dar tsp laftiisa iyo/ama bitrate_monitor-ka iyo sii wadista borotiinka si ay natiijadu ugu soo baxdo qaab mishiin-la akhriyi karo oo ku habboon nidaamka la socodka.
- Codsigaga ku dul qor maktabadda tsduck.
Sida iska cad, marka la eego kharashka shaqada, doorashada 1 ayaa ah tan ugu fudud, gaar ahaan iyadoo la tixgelinayo in tsduck lafteedu ay ku qoran tahay heer hoose (heerarka casriga ah) luqadda (C++)
Nooca fudud ee parser + aggregator ee bash ayaa muujiyay in socodka 10 Mbit / s iyo 50% luminta baakidhka (kiiskii ugu xumaa), habka bashku wuxuu cunay 3-4 jeer ka badan CPU marka loo eego habka tsp laftiisa. Dhacdadani waa wax aan la aqbali karin. Dhab ahaantii qayb ka mid ah tusaalahani waa hoos
Baasto on basha
#!/usr/bin/env bash
missingPackets=0
ccErrorSeconds=0
regexMissPackets='^* (.+) - continuity:.*missing ([0-9]+) packets$'
missingPacketsTime=""
ip netns exec P tsp --realtime -t -I ip -b 8388608 "239.0.0.1:1234" -O drop -P bitrate_monitor -p 1 -t 1 -P continuity 2>&1 |
while read i
do
#line example:* 2019/12/28 23:41:14 - continuity: packet index: 6,078, PID: 0x0100, missing 5 packets
#line example 2: * 2019/12/28 23:55:11 - bitrate_monitor: 2019/12/28 23:55:11, TS bitrate: 4,272,864 bits/s
if [[ "$i" == *continuity:* ]]
then
if [[ "$i" =~ $regexMissPackets ]]
then
missingPacketsTimeNew="${BASH_REMATCH[1]}" #timestamp (seconds)
if [[ "$missingPacketsTime" != "$missingPacketsTimeNew" ]] #new second with CC error
then
((ccErrorSeconds += 1))
fi
missingPacketsTime=$missingPacketsTimeNew
packets=${BASH_REMATCH[2]} #TS missing packets
((missingPackets += packets))
fi
elif [[ "$i" == *bitrate_monitor:* ]]
then
: #...
fi
doneMarka lagu daro xaqiiqda ah in tani ay si tartiib tartiib ah u shaqeyso, ma jiraan xargaha caadiga ah ee bash, shaqooyinka bashku waa habab madax-bannaan oo waxaan ku qasbanaa inaan qoro qiimaha maqan ee Packets hal mar oo ku saabsan saameynta dhinaca (marka la helo farriimaha bitrate ee yimaada ilbiriqsi kasta). Natiijo ahaan, bash keligiis ayaa laga tagay waxaana la go'aansaday in lagu qoro duub (parser + aggregator) golang. Isticmaalka CPU ee koodka la midka ah ee golang waa 4-5 jeer ka yar habka tsp laftiisa. Dardargelinta duubabka iyadoo lagu beddelayo bash golang waxay ahayd qiyaastii 16 jeer guud ahaanna natiijadu waa mid la aqbali karo (CPU-da waxaa ka sarreeya 25% kiiskii ugu xumaa). Faylka isha golang ayaa ku yaal .
Daahfurka duubka
Si loo bilaabo duubka, qaab adeeg fudud oo loogu talagalay systemd ayaa la sameeyay (). Waxaa loo malaynayaa in duubabka laftiisa lagu ururiyay faylka binary (go build tsduck-stat.go), oo ku yaal /opt/tsduck-stat/. Waxaa loo maleynayaa in golang loo isticmaalo taageerada saacadda monotonic (> = 1.9).
Si aad u abuurto tusaale adeeg waxaad u baahan tahay inaad socodsiiso systemctl awood amarka [emailka waa la ilaaliyay]:1234, ka dibna ku orod systemctl start [emailka waa la ilaaliyay]: 1234.
Helitaanka Zabbix
Markaa zabbix waxay samayn kartaa daahfurka adeegyada socodsiinta, (discovery.sh), qaabka looga baahan yahay helitaanka Zabbix, waxaa loo malaynayaa inay ku taal meel isku mid ah - gudaha /opt/tsduck-stat. Si aad u socodsiiso helitaanka iyada oo loo marayo zabbix-agent, waxaad u baahan tahay inaad ku darto tusaha leh qaabaynta wakiilka zabbix si loogu daro qiyaasta isticmaale.
Qaabka Zabbix
(tsduck_stat_template.xml) waxa uu ka kooban yahay qaanuunka daahfurka iswada, curiyaha, garaafka, iyo prototypes kiciye.
Liis kooban oo hubin ah (ka warran haddii qof go'aansado inuu isticmaalo)
- Hubi in tsp uusan hoos u dhigin xirmooyinka "ku habboon" (dhaliye iyo falanqeeye ayaa si toos ah ugu xiran), haddii ay jiraan dhibco, fiiri qodobka 2 ama qoraalka maqaalka mawduucan.
- Samee hagaajinta godadka ugu sarreeya (net.core.rmem_max=8388608).
- Samee tsduck-stat.go (go build tsduck-stat.go).
- Dhig qaabka adeega gudaha /lib/systemd/system.
- Ku bilow adeegyada addoo isticmaalaya systemctl, hubi in xisaabiyayaashu ay bilaabeen inay soo baxaan (grep "" /dev/shm/tsduck-stat/*). Tirada adeegyada ee tirada durdurrada cajaladaha badan leh. Halkan waxaa laga yaabaa inaad u baahato inaad abuurto waddo aad u marto kooxda cajaladaha badan leh, laga yaabee in aad joojiso rp_filter ama abuurto dariiqa isha ip.
- Ku orod discovery.sh, hubi inay soo saarto json.
- Dhig wakiilka zabbix config, dib u bilow wakiilka zabbix.
- Upload template to zabbix, codsan in ay martida loo kaas oo la socodka la fuliyay iyo zabbix-wakiil la rakibay, sug ilaa 5 daqiiqo, arag in xubno xogta cusub, garaafyada iyo kiciyeyaasha soo muuqday.
natiijada
Hawsha lagu ogaanayo khasaaraha baakadaha, waa ku filan tahay, ugu yaraan way ka fiican tahay la-socod la'aan.
Dhab ahaantii, CC "khasaaraha" waxay dhici kartaa marka la kala saaro jajabyada fiidiyowga (inta aan ogahay, tani waa sida loo dhejiyo xarumaha telefishinka ee gudaha Ruushka, ie iyada oo aan dib loo xisaabin miiska CC), tani waa in la xasuusto. Xallinta lahaanshaha, dhibaatadan qayb ahaan waa la hareer maray iyadoo la ogaanayo calaamadaha SCTE-35 (haddii lagu daro koronto dhaliyaha durdurrada).
Marka laga eego dhinaca la socodka tayada gaadiidka, la socodka jitter (IAT) kuma filna, sababtoo ah Qalabka TV-ga (ha ahaadeen modulators ama aaladaha dhamaadka) waxay leeyihiin shuruudo loogu talagalay halbeegyadan oo mar walba suurtagal maaha in la buufiyo jitbuffer si aan xad lahayn. Jitter-kuna wuu sabeyn karaa marka gaadiidku isticmaalo qalab leh bakhaarro waaweyn iyo QoS aan la habeynin ama aan si fiican loo habeynin si loo gudbiyo taraafikada waqtiga dhabta ah.
Source: www.habr.com