Taa, enwere ihe ngwọta nke emebere (ihe onwunwe) maka nlekota IP (TS), dịka ọmụmaatụ и , Ha nwere ọtụtụ ọrụ bara ụba nke ukwuu ma na-abụkarị ndị ọrụ buru ibu na-arụ ọrụ na TV nwere ngwọta yiri nke ahụ. Edemede a na-akọwa ngwọta dabere na ọrụ mepere emepe , Emebere maka njikwa ntakịrị nke IP (TS) na-asọ site na iji counter CC (continuity counter) counter na bitrate. Ngwa enwere ike na-enyocha mfu nke ngwugwu ma ọ bụ ihe niile na-erute site na ọwa L2 mgbazinye (nke enweghị ike nyochaa ya nke ọma, dịka ọmụmaatụ, site n'ịgụ ọnụọgụ ọnwụ na ahịrị).
Nnọọ nkenke banyere TDuck
TSDuck bụ ngwanrọ mepere emepe (ikike BSD 2-Clause BSD) (usoro akụrụngwa njikwa yana ọbá akwụkwọ maka imepụta akụrụngwa ma ọ bụ plugins nke gị) maka ijikwa iyi iyi TS. Dị ka ntinye, ọ nwere ike na-arụ ọrụ na IP (multicast/unicast), http, hls, dvb tuners, dektec dvb-asi demodulator, e nwere n'ime TS iyi generator na ịgụ site na faịlụ. Mmepụta nwere ike ịdekọ na faịlụ, IP (multicast/unicast), hls, dektec dvb-asi na HiDes modulators, ndị egwuregwu (mplayer, vlc, xine) na dobe. N'etiti ntinye na mmepụta, ị nwere ike ime ka ndị na-arụ ọrụ okporo ụzọ dị iche iche, dịka ọmụmaatụ, ịmegharị PIDs, na-eme scrambling / descrambling, nyochaa CC counters, na-agbakọ bitrate na ọrụ ndị ọzọ a na-ahụkarị maka iyi TS.
N'isiokwu a, a ga-eji iyi IP (multicast) mee ihe dị ka ntinye, bitrate_monitor processors (site na aha ahụ doro anya ihe nke a bụ) na continuity (CC counter analysis) ga-eji ihe nhazi. Enweghị nsogbu ọ bụla, ị nwere ike dochie multicast IP na ụdị ntinye ọzọ nke TDuck kwadoro.
Dị TDuck maka ọtụtụ OS dị ugbu a. Enweghị maka Debian, mana anyị jisiri ike chịkọta ha maka Debian 8 na Debian 10 n'enweghị nsogbu ọ bụla.
Na-esote, a na-eji ụdị TSDuck 3.19-1520 mee ihe, a na-eji Linux mee ihe dị ka OS (ejiri debian 10 iji dozie ngwọta ahụ, ejiri CentOS 7 mee ihe maka iji ya eme ihe).
Na-akwado TSDuck na OS
Tupu nyochaa ezigbo ebili mmiri, ịkwesịrị ijide n'aka na TDuck na-arụ ọrụ nke ọma yana ọdịda anaghị eme na kaadị netwọkụ ma ọ bụ ọkwa OS (socket). Achọrọ nke a ka ị ghara ịkọ ma emechaa ebe ọdịda ahụ mere - na netwọk ma ọ bụ "n'ime ihe nkesa." Ị nwere ike ịlele ọdịda na ọkwa kaadị netwọk na iwu ethtool -S ethX, a na-eme ya site na otu ethtool (na-emekarị ị ga-abawanye RX buffer (-G) na mgbe ụfọdụ gbanyụọ ụfọdụ offloads (-K)). Dị ka ndụmọdụ n'ozuzu, ọ bụ ihe amamihe dị na ya iji ọdụ ụgbọ mmiri dị iche iche na-enweta okporo ụzọ nyochara, ọ bụrụ na ọ ga-ekwe omume, nke a ga-ebelata ihe ndị na-ezighị ezi n'ihi eziokwu ahụ bụ na ọdịda ahụ mere n'otu oge na ọdụ ụgbọ mmiri nyocha n'ihi ọnụnọ nke okporo ụzọ ndị ọzọ. Ọ bụrụ na nke a agaghị ekwe omume (ị na-eji obere-kọmputa / NUC nwere otu ọdụ ụgbọ mmiri), mgbe ahụ, ọ bụ ihe amamihe dị na ya ịhazi ihe kacha mkpa nke okporo ụzọ nyochara n'ihe metụtara ndị ọzọ na ngwaọrụ nke ejikọtara onye nyocha. Banyere mebere gburugburu, ebe a ịkwesịrị ịkpachara anya ma nwee ike ịchọta ngwugwu ngwugwu na-amalite site na ọdụ ụgbọ mmiri na-ejedebe na ngwa n'ime igwe mebere.
Ịmepụta na ịnweta iyi n'ime onye ọbịa
Dị ka nzọụkwụ mbụ na ịkwadebe TDuck, anyị ga-emepụta ma nata okporo ụzọ n'ime otu onye ọbịa site na iji netn.
Na-akwado gburugburu:
ip netns add P #создаём netns P, в нём будет происходить анализ трафика
ip link add type veth #создаём veth-пару - veth0 оставляем в netns по умолчанию (в этот интерфейс будет генерироваться трафик)
ip link set dev veth1 netns P #veth1 - помещаем в netns P (на этом интерфейсе будет приём трафика)
ip netns exec P ifconfig veth1 192.0.2.1/30 up #поднимаем IP на veth1, не имеет значения какой именно
ip netns exec P ip ro add default via 192.0.2.2 #настраиваем маршрут по умолчанию внутри nents P
sysctl net.ipv6.conf.veth0.disable_ipv6=1 #отключаем IPv6 на veth0 - это делается для того, чтобы в счётчик TX не попадал посторонний мусор
ifconfig veth0 up #поднимаем интерфейс veth0
ip route add 239.0.0.1 dev veth0 #создаём маршрут, чтобы ОС направляла трафик к 239.0.0.1 в сторону veth0Gburugburu ebe obibi adịla njikere. Mepee ihe nyocha okporo ụzọ:
ip netns exec P tsp --realtime -t
-I ip 239.0.0.1:1234
-P continuity
-P bitrate_monitor -p 1 -t 1
-O dropebe "-p 1 -t 1" pụtara na ịkwesịrị ịgbakọ bitrate kwa sekọnd ma gosipụta ozi gbasara bitrate kwa sekọnd.
Anyị na-ebupụta generator nke nwere ọsọ nke 10Mbit / s:
tsp -I craft
-P regulate -b 10000000
-O ip -p 7 -e --local-port 6000 239.0.0.1:1234ebe "-p 7 -e" pụtara na ịkwesịrị ịkwanye ngwugwu 7 TS n'ime ngwugwu IP 1 wee mee ya siri ike (-e), i.e. na-eche mgbe niile maka ngwugwu 7 TS site na nhazi ikpeazụ tupu izipu nhazi nke ngwugwu IP.
Onye nyocha na-amalite igosipụta ozi a tụrụ anya:
* 2020/01/03 14:55:44 - bitrate_monitor: 2020/01/03 14:55:44, TS bitrate: 9,970,016 bits/s
* 2020/01/03 14:55:45 - bitrate_monitor: 2020/01/03 14:55:45, TS bitrate: 10,022,656 bits/s
* 2020/01/03 14:55:46 - bitrate_monitor: 2020/01/03 14:55:46, TS bitrate: 9,980,544 bits/sUgbu a, ka anyị gbakwunye ntụ ntụ:
ip netns exec P iptables -I INPUT -d 239.0.0.1 -m statistic --mode random --probability 0.001 -j DROPna ozi dị ka nke a na-apụta:
* 2020/01/03 14:57:11 - continuity: packet index: 80,745, PID: 0x0000, missing 7 packets
* 2020/01/03 14:57:11 - continuity: packet index: 83,342, PID: 0x0000, missing 7 packets nke a na-atụ anya ya. Anyị na-ewepụ mfu ngwungwu (ip netns exec P iptables -F) wee gbalịa ịbawanye bitrate generator ka ọ bụrụ 100 Mbit/s. The analyzer na-akọ ụyọkọ nke njehie CC na ihe dị ka 75 Mbit / s kama 100. Anyị na-agbalị ịchọta onye ga-ata ụta - generator adịghị edebe ma ọ bụ nsogbu adịghị na ya, iji mee nke a, anyị na-amalite ịmepụta a. Ọnụ ọgụgụ nke ngwugwu (ngwugwu 700000 TS = 100000 IP ngwugwu):
# ifconfig veth0 | grep TX
TX packets 151825460 bytes 205725459268 (191.5 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
# tsp -I craft -c 700000 -P regulate -b 100000000 -P count -O ip -p 7 -e --local-port 6000 239.0.0.1:1234
* count: PID 0 (0x0000): 700,000 packets
# ifconfig veth0 | grep TX
TX packets 151925460 bytes 205861259268 (191.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0Dị ka ị pụrụ ịhụ, kpọmkwem 100000 IP ngwugwu emepụtara (151925460-151825460). Ya mere, anyị na-achọpụta ihe na-eme na analyzer, ime nke a, anyị na-elele na RX counter on veth1, ọ bụ nnọọ hà TX counter on veth0, mgbe ahụ, anyị na-ele ihe na-eme na oghere larịị:
# ip netns exec P cat /proc/net/udp
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
133: 010000EF:04D2 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 72338 2 00000000e0a441df 24355 N'ebe a, ị nwere ike ịhụ ọnụ ọgụgụ nke ọdịda = 24355. Na ngwugwu TS nke a bụ 170485 ma ọ bụ 24.36% nke 700000, ya mere, anyị na-ahụ na otu 25% nke bitrate efu na-adaba na oghere UDP. Ntube na oghere UDP na-emekarị n'ihi enweghị ihe nchekwa, ka anyị hụ ihe nkwụnye nkwụnye nkwụnye ndabara na oke ihe nkwụnye oghere bụ:
# sysctl net.core.rmem_default
net.core.rmem_default = 212992
# sysctl net.core.rmem_max
net.core.rmem_max = 212992Ya mere, ọ bụrụ na ngwa anaghị arịọ n'ụzọ doro anya nha nchekwa, sockets na-eji ihe nchekwa 208 KB mepụta, mana ọ bụrụ na ha arịọ karịa, ha agaghị enweta ihe ha rịọrọ. Ebe ọ bụ na na tsp ị nwere ike ịtọ nha nchekwa maka ntinye IP (--buffer-size), anyị agaghị emetụ nha oghere ndabara, kama anyị ga-edozi nha ihe nkwụnye ihe kachasị elu wee kọwaa nha nkwụnye ahụ n'ụzọ doro anya site na arụmụka tsp:
sysctl net.core.rmem_max=8388608
ip netns exec P tsp --realtime -t -I ip 239.0.0.1:1234 -b 8388608 -P continuity -P bitrate_monitor -p 1 -t 1 -O dropSite na ngbanwe nke oghere oghere a, ọnụọgụ bitrate a na-akọ ugbu a bụ ihe dịka 100Mbps, enweghị njehie CC.
Dabere na oriri CPU site na ngwa tsp n'onwe ya. Banyere otu isi i5-4260U CPU @ 1.40GHz, iji nyochaa ọsọ 10Mbit/s, 3-4% nke CPU ga-achọ, 100Mbit/s - 25%, 200Mbit/s - 46%. Mgbe ịtọba % mfu ngwugwu, ibu CPU anaghị abawanye (mana ọ nwere ike ibelata).
Na ngwaike na-arụpụta ihe, ọ ga-ekwe omume ịmepụta na nyochaa iyi nke karịrị 1Gb/s na-enweghị nsogbu ọ bụla.
Nnwale na kaadị netwọkụ n'ezie
Mgbe anwalechara na veth ụzọ abụọ, ịkwesịrị iburu ndị ọbịa abụọ ma ọ bụ ọdụ ụgbọ mmiri abụọ nke otu onye ọbịa, jikọọ ọdụ ụgbọ mmiri n'otu n'otu, gbanye generator na otu, yana onye nyocha na nke abụọ. Enweghị ihe ijuanya ebe a, ma n'eziokwu, ihe niile dabere na ngwaike, ike na-esikwu ike, otú ahụ ka ọ ga-adọrọ mmasị karị ebe a.
Iji data enwetara site na sistemụ nleba anya (Zabbix)
tsp enweghị API igwe nwere ike ịgụ dị ka SNMP ma ọ bụ ihe yiri ya. Ekwesịrị ịchịkọta ozi CC ma ọ dịkarịa ala 1 sekọnd n'otu oge (na ọnụọgụ dị elu nke mfu ngwugwu, enwere ike ịnwe narị otu narị / puku / iri puku kwuru iri kwa nkeji, dabere na bitrate).
Ya mere, iji chekwaa ozi na eserese eserese maka njehie CC na bitrate ma mee ka ụfọdụ ụdị ihe mberede gaa n'ihu, enwere ike ịnwe nhọrọ ndị a:
- Wepu ma chịkọta (site na CC) mmepụta tsp, ya bụ. gbanwee ya n'ụdị achọrọ.
- Tinye tsp n'onwe ya na / ma ọ bụ bitrate_monitor na continuity processor plugins ka nsonaazụ ya pụta n'ụdị igwe-agụ nke dabara adaba maka usoro nlekota.
- Dee ngwa gị n'elu ọba akwụkwọ tsduck.
N'ụzọ doro anya, n'ihe gbasara ụgwọ ọrụ, nhọrọ 1 bụ nke kachasị mfe, karịsịa na-atụle na tsduck n'onwe ya ka edere n'asụsụ dị ala (site n'ụkpụrụ ọgbara ọhụrụ) (C++)
Ihe nlere dị mfe nke parser + aggregator na bash gosiri na na nrịgo nke 10 Mbit / s na mfu ngwugwu 50% (ikpe kacha njọ), usoro bash riri 3-4 ugboro karịa CPU karịa usoro tsp n'onwe ya. Nke a dị n'ọnọdụ adịghị anabata. N'ezie otu mpempe prototype a dị n'okpuru
Noodles na basha
#!/usr/bin/env bash
missingPackets=0
ccErrorSeconds=0
regexMissPackets='^* (.+) - continuity:.*missing ([0-9]+) packets$'
missingPacketsTime=""
ip netns exec P tsp --realtime -t -I ip -b 8388608 "239.0.0.1:1234" -O drop -P bitrate_monitor -p 1 -t 1 -P continuity 2>&1 |
while read i
do
#line example:* 2019/12/28 23:41:14 - continuity: packet index: 6,078, PID: 0x0100, missing 5 packets
#line example 2: * 2019/12/28 23:55:11 - bitrate_monitor: 2019/12/28 23:55:11, TS bitrate: 4,272,864 bits/s
if [[ "$i" == *continuity:* ]]
then
if [[ "$i" =~ $regexMissPackets ]]
then
missingPacketsTimeNew="${BASH_REMATCH[1]}" #timestamp (seconds)
if [[ "$missingPacketsTime" != "$missingPacketsTimeNew" ]] #new second with CC error
then
((ccErrorSeconds += 1))
fi
missingPacketsTime=$missingPacketsTimeNew
packets=${BASH_REMATCH[2]} #TS missing packets
((missingPackets += packets))
fi
elif [[ "$i" == *bitrate_monitor:* ]]
then
: #...
fi
doneNa mgbakwunye na eziokwu na nke a na-arụ ọrụ na-adịghị anabata nwayọọ nwayọọ, ọ dịghị nkịtị eri na bash, bash ọrụ bụ onwe usoro na m ga-ede na-efuPackets uru otu ugboro na nke abụọ na akụkụ mmetụta (mgbe ị na-enweta bitrate ozi na-abịa kwa abụọ). N'ihi ya, a hapụrụ bash naanị ya wee kpebie ide ihe mkpuchi (parser + aggregator) na golang. Oriri CPU nke koodu yiri ya na golang bụ 4-5 ugboro erughị usoro tsp n'onwe ya. Mmụba nke ihe mkpuchi ahụ site na iji golang dochie bash bụ ihe dị ka ugboro 16 na n'ozuzu ya bụ ihe a na-anabata (CPU n'elu site na 25% n'ọnọdụ kachasị njọ). Faịlụ isi mmalite golang dị .
Ịmalite ihe mkpuchi ahụ
Iji malite ihe mkpuchi ahụ, e mere template ọrụ dị mfe maka systemd (). A na-eche na a na-achịkọta ihe mkpuchi ahụ n'onwe ya na faịlụ ọnụọgụ abụọ (go build tsduck-stat.go), nke dị na /opt/tsduck-stat/. A na-eche na a na-eji golang na nkwado elekere monotonic (> = 1.9).
Iji mepụta ihe atụ ọrụ, ịkwesịrị ịgbanye iwu systemctl [email protected]:1234, wee na-agba ọsọ na systemctl mmalite [email protected]: 1234.
Nchọpụta sitere na Zabbix
Ka zabbix wee mee nchọpụta nke ọrụ ịgba ọsọ, (discovery.sh), na usoro achọrọ maka nchọpụta Zabbix, a na-eche na ọ dị n'otu ebe - na /opt/tsduck-stat. Iji mee nchọpụta site na zabbix-agent, ịkwesịrị ịgbakwunye na ndekọ nwere zabbix-agent nhazi ka ịgbakwunye onye ọrụ paramita.
Zabbix template
(tsduck_stat_template.xml) nwere ụkpụrụ nchọpụta akpaaka, mmewere, eserese na ụdị akpalite.
Ndepụta nkenke (gịnị ma ọ bụrụ na mmadụ kpebie iji ya)
- Jide n'aka na tsp adịghị ahapụ ngwugwu n'okpuru ọnọdụ "ezigbo" (a na-ejikọta generator na analyzer ozugbo), ọ bụrụ na enwere ọdịda, lee isi 2 ma ọ bụ ederede nke isiokwu na isiokwu a.
- Mee nzigharị nke ihe nchekwa oghere kacha (net.core.rmem_max=8388608).
- Mepụta tsduck-stat.go (gaa wuo tsduck-stat.go).
- Tinye ndebiri ọrụ na /lib/systemd/system.
- Malite ọrụ site na iji systemctl, lelee na counters amalitela ịpụta (grep "" /dev/shm/tsduck-stat/*). Ọnụọgụ nke ọrụ site na ọnụọgụ nke iyi multicast. N'ebe a, ị nwere ike ịmepụta ụzọ na-aga otu multicast, ikekwe gbanyụọ rp_filter ma ọ bụ mepụta ụzọ isi ip.
- Gbaa discovery.sh, hụ na ọ na-ewepụta json.
- Debe onye nnọchi anya zabbix, malitegharịa onye nnọchi anya zabbix.
- Bulite template na zabbix, tinye ya na onye ọbịa nke a na-eme nlekota na arụ ọrụ zabbix-agent, chere ihe dị ka nkeji 5, hụ na ihe ọhụrụ data, eserese na ndị na-akpali akpali apụtawo.
N'ihi
Maka ọrụ nke ịchọpụta mfu ngwugwu, ọ fọrọ nke nta ka ọ bụrụ nke zuru oke, ọ dịkarịa ala ọ dị mma karịa enweghị nlekota.
N'ezie, CC "ofufu" nwere ike ime mgbe ị na-ekesa iberibe vidiyo (dị ka m maara, nke a bụ otú e si etinye ntinye na ụlọ ọrụ telivishọn mpaghara na Russian Federation, ya bụ na-enweghị ịtụgharị CC counter), a ghaghị icheta nke a. N'ihe ngwọta nke ụlọ, nsogbu a na-agafe akụkụ ụfọdụ site na ịchọpụta mkpado SCTE-35 (ọ bụrụ na ndị na-emepụta iyi na-agbakwunye ha).
Site n'echiche nke nlekota ihe nleba anya iga njem, nlekota oru jitter (IAT) ezughị oke, n'ihi na Akụrụngwa TV (ma modulators ma ọ bụ ngwaọrụ njedebe) nwere ihe achọrọ maka oke a na ọ naghị ekwe omume mgbe niile ịgbanye jitbuffer ruo mgbe ebighị ebi. Na jitter nwere ike ise n'elu mgbe njem na-eji akụrụngwa nwere nnukwu ihe nchekwa na QoS ahazighị ma ọ bụ ahazighị ya nke ọma iji bufee ụdị okporo ụzọ ozugbo.
isi: www.habr.com