🥇Ukusebenzisa i-TSDuck ukujonga i-IP(TS) imisinga

Namhlanje, kukho izisombululo esele zenziwe (zobunini) zokubeka iliso kwimijelo ye-IP (TS), umzekelo. VB и iQ, banesethi yemisebenzi etyebileyo kwaye ngokuqhelekileyo abaqhubi abakhulu abajongene neenkonzo zeTV banezisombululo ezinjalo. Eli nqaku lichaza isisombululo esisekelwe kwiprojekthi yomthombo ovulekileyo TSDuck, eyenzelwe ulawulo oluncinci lwe-IP (TS) imilambo ngeCC (ikhawuntari yokuqhubeka) ikhawuntara kunye ne-bitrate. Isicelo esinokwenzeka kukulawula ukulahleka kweepakethi okanye ukuhamba konke nge-channel ye-L2 eqeshiweyo (engenako ukubeka iliso ngokuqhelekileyo, umzekelo, ngokufunda izinto zokubala ezilahlekileyo kwimigca).

Ngokufutshane kakhulu malunga neTSDuck

I-TSDuck ngumthombo ovulekileyo (i-2-Igatya lelayisensi ye-BSD) isofthiwe (iseti yezixhobo ze-console kunye nethala leencwadi lokuphuhlisa izinto ezisetyenziswayo okanye iiplagi) zokulawula imilambo ye-TS. Njengegalelo, inokusebenza nge-IP (multicast/unicast), http, hls, dvb tuners, dektec dvb-asi demodulator, kukho i-TS-stream generator yangaphakathi kunye nokufunda kwiifayile. Imveliso ingabhalwa kwifayile, IP (multicast/unicast), hls, dektec dvb-asi kunye neemodyuli zeHiDes, abadlali (mplayer, vlc, xine) kunye nokulahla. Iiprosesa ezahlukeneyo zetrafikhi zinokuqukwa phakathi kwegalelo kunye nemveliso, umzekelo, uhlengahlengiso lwe-PID, ukukrazula / ukukrazula, uhlalutyo lwekhawunta yeCC, ukubalwa kwe-bitrate, kunye neminye imisebenzi eqhelekileyo yemilambo ye-TS.

Kule nqaku, imijelo ye-IP (i-multicast) iya kusetyenziswa njengegalelo, i-processors bitrate_monitor (ukusuka kwigama kucacile ukuba yintoni) kunye nokuqhubeka (uhlalutyo lwe-CC counters) lusetyenziswa. Ungayitshintsha ngokulula i-IP multicast ngolunye uhlobo lwegalelo oluxhaswa yiTSDuck.

Iyafumaneka Ulwakhiwo olusemthethweni / iipakethi I-TSDuck yeendlela ezininzi zokusebenza zangoku. Azifumaneki kwi-Debian, kodwa sikwazile ukuzakha phantsi kwe-debian 8 kunye ne-debian 10 ngaphandle kweengxaki.

Okulandelayo, inguqulo ye-TSDuck 3.19-1520 isetyenziswa, i-Linux isetyenziswa njenge-OS (i-debian 10 yayisetyenziselwa ukulungisa isisombululo, i-CentOS 7 yayisetyenziselwa ukusetyenziswa kwangempela)

Ukulungiselela i-TSDuck kunye ne-OS

Ngaphambi kokubeka iliso ukuhamba kwangempela, kufuneka uqinisekise ukuba i-TSDuck isebenza ngokuchanekileyo kwaye akukho kwehla kwikhadi lenethiwekhi okanye kwinqanaba le-OS (socket). Oku kuyafuneka ukuze ungaqashi kamva apho amathontsi enzeka khona - kwinethiwekhi okanye "ngaphakathi kwiseva". Unokujonga ukuhla kwinqanaba lekhadi lenethiwekhi kunye nomyalelo we-ethtool -S ethX, ukulungiswa kwenziwa yi-ethtool efanayo (ngokuqhelekileyo, kufuneka ukhulise i-RX buffer (-G) kwaye ngamanye amaxesha ukhubaze ukulayisha okunye (-K)). Njengesincomo esiqhelekileyo, kunokucetyiswa ukuba kusetyenziswe i-port ehlukeneyo yokufumana i-traffic ehlalutyiweyo, ukuba kunokwenzeka, oku kunciphisa iimpawu zobuxoki ezihambelana nokuba ukuhla kwenzeka ngokuchanekileyo kwi-port analyzer ngenxa yobukho bezinye izithuthi. Ukuba oku akunakwenzeka (i-mini-computer / i-NUC ene-port enye isetyenzisiweyo), ngoko kunqweneleka kakhulu ukuseta ukubeka phambili ukubekwa phambili kwetrafikhi ehlalutyiweyo ngokubhekiselele kuyo yonke into kwisixhobo apho i-analyzer ixhunyiwe. Ngokumalunga neemeko-bume zenyani, apha kufuneka ulumke kwaye ukwazi ukufumana ipakethe ehlayo eqala kwizibuko elibonakalayo kwaye iphele ngesicelo ngaphakathi kumatshini wenyani.

Isizukulwana kunye nolwamkelo lomlambo ngaphakathi kombuki zindwendwe

Njengenyathelo lokuqala ekulungiseleleni i-TSDuck, siya kuvelisa kwaye sifumane i-traffic ngaphakathi komkhosi omnye sisebenzisa iinethi.

Ukulungisa okusingqongileyo:

ip netns add P #создаём netns P, в нём будет происходить анализ трафика
ip link add type veth #создаём veth-пару - veth0 оставляем в netns по умолчанию (в этот интерфейс будет генерироваться трафик)
ip link set dev veth1 netns P #veth1 - помещаем в netns P (на этом интерфейсе будет приём трафика)
ip netns exec P ifconfig veth1 192.0.2.1/30 up #поднимаем IP на veth1, не имеет значения какой именно
ip netns exec P ip ro add default via 192.0.2.2 #настраиваем маршрут по умолчанию внутри nents P
sysctl net.ipv6.conf.veth0.disable_ipv6=1 #отключаем IPv6 на veth0 - это делается для того, чтобы в счётчик TX не попадал посторонний мусор
ifconfig veth0 up #поднимаем интерфейс veth0
ip route add 239.0.0.1 dev veth0 #создаём маршрут, чтобы ОС направляла трафик к 239.0.0.1 в сторону veth0

Indawo engqongileyo ilungile. Siqala umhlalutyi wendlela:

ip netns exec P tsp --realtime -t 
 -I ip 239.0.0.1:1234 
 -P continuity 
 -P bitrate_monitor -p 1 -t 1 
 -O drop

apho "-p 1 -t 1" ithetha ukuba ufuna ukubala ibitrate rhoqo umzuzwana kwaye ubonise ulwazi malunga nebitrate nganye umzuzwana.
Siqala ijenereyitha yetrafikhi ngesantya se-10Mbps:

tsp -I craft 
 -P regulate -b 10000000 
 -O ip -p 7 -e --local-port 6000 239.0.0.1:1234

apho "-p 7 -e" kuthetha ukuba kufuneka upakishe iipakethi ze-7 TS kwi-1 IP ipakethe kwaye wenze nzima (-e), i.e. hlala ulinde iipakethi ezi-7 ze-TS ukusuka kwiprosesa yokugqibela ngaphambi kokuthumela ipakethi ye-IP.

Umhlalutyi uqalisa ukukhupha imiyalezo elindelekileyo:

* 2020/01/03 14:55:44 - bitrate_monitor: 2020/01/03 14:55:44, TS bitrate: 9,970,016 bits/s
* 2020/01/03 14:55:45 - bitrate_monitor: 2020/01/03 14:55:45, TS bitrate: 10,022,656 bits/s
* 2020/01/03 14:55:46 - bitrate_monitor: 2020/01/03 14:55:46, TS bitrate: 9,980,544 bits/s

Ngoku yongeza amathontsi:

ip netns exec P iptables -I INPUT -d 239.0.0.1 -m statistic --mode random --probability 0.001 -j DROP

kwaye imiyalezo efana nale ibonakala:

* 2020/01/03 14:57:11 - continuity: packet index: 80,745, PID: 0x0000, missing 7 packets
* 2020/01/03 14:57:11 - continuity: packet index: 83,342, PID: 0x0000, missing 7 packets

elindelekileyo. Khubaza ilahleko yepakethe (ip netns exec P iptables -F) kwaye uzame ukwandisa i-bitrate ye-generator kwi-100Mbps. I-analyzer ibika iqela leempazamo zeCC kunye ne-75 Mbps endaweni ye-100. Sizama ukufumanisa ukuba ngubani onetyala - i-generator ayinalo ixesha okanye ingxaki ayikho kuyo, kuba oku siqala ukuvelisa inani elimiselweyo iipakethe (700000 TS ipakethe = 100000 IP pakethe):

# ifconfig veth0 | grep TX
       TX packets 151825460  bytes 205725459268 (191.5 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
# tsp -I craft -c 700000 -P regulate -b 100000000 -P count -O ip -p 7 -e --local-port 6000 239.0.0.1:1234
* count: PID    0 (0x0000):    700,000 packets
# ifconfig veth0 | grep TX
        TX packets 151925460  bytes 205861259268 (191.7 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Njengoko ubona, iipakethi ze-IP ze-100000 zenziwa (151925460-151825460). Ke makhe sijonge ukuba kwenzeka ntoni nge-analyzer, kule nto sijonga nge-RX counter kwi-veth1, ilingana ngokungqongqo ne-TX counter kwi-veth0, emva koko sijonge okwenzekayo kwinqanaba le-socket:

# ip netns exec P cat /proc/net/udp                                                                                                           
  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode ref pointer drops             
  133: 010000EF:04D2 00000000:0000 07 00000000:00000000 00:00000000 00000000     0        0 72338 2 00000000e0a441df 24355

Apha ungabona inani lamaconsi = 24355. Kwiipakethi ze-TS, le yi-170485 okanye i-24.36% ye-700000, ngoko siyabona ukuba loo 25% ye-bitrate elahlekileyo iyancipha kwi-socket ye-udp. Ukuhla kwisokethi ye-UDP ngokuqhelekileyo kwenzeka ngenxa yokunqongophala kwesithinteli, jonga ubungakanani besokethi engagqibekanga kunye nobukhulu bobungakanani besokethi yebuffer:

# sysctl net.core.rmem_default
net.core.rmem_default = 212992
# sysctl net.core.rmem_max
net.core.rmem_max = 212992

Ke ngoko, ukuba izicelo aziceli ngokucacileyo isayizi ye-buffer, iisokethi zenziwe zine-buffer ye-208 KB, kodwa ukuba zicela ngaphezulu, abasayi kukufumana oko bekuceliweyo. Ekubeni ungacwangcisa ubungakanani bebuffer kwi tsp yegalelo le IP (-buffer-size), asiyi kubamba ubungakanani besokethi engagqibekanga, kodwa seta kuphela ubungakanani besokethi yebuffer kwaye uchaze ubungakanani bebuffer ngokucacileyo ngokusebenzisa iimpikiswano zetsp:

sysctl net.core.rmem_max=8388608
ip netns exec P tsp --realtime -t -I ip 239.0.0.1:1234 -b 8388608 -P continuity -P bitrate_monitor -p 1 -t 1 -O drop

Ngolu lungelelwaniso lwe-socket buffer, ngoku i-bitrate exeliweyo imalunga ne-100Mbps, akukho ziphoso zeCC.

Ngokutsho kokusetyenziswa kwe-CPU yesicelo se-tsp ngokwayo. Ngokumalunga ne-core enye i5-4260U CPU @ 1.40GHz, uhlalutyo lokuhamba kwe-10Mbps luya kufuna i-3-4% ye-CPU, i-100Mbps - i-25%, i-200Mbps - i-46%. Xa ucwangcisa i% Ilahleko yePakethe, umthwalo kwi-CPU ngokwenyani awunyuki (kodwa unokuhla).

Kwi-hardware enemveliso ngakumbi, bekunokwenzeka ukuvelisa kunye nokuhlalutya imijelo engaphezulu kwe-1Gb / s ngaphandle kweengxaki.

Uvavanyo kumakhadi womnatha wokwenene

Emva kokuvavanya kwi-pair ye-veth, kufuneka uthathe imikhosi emibini okanye izibuko ezimbini zomkhosi omnye, udibanise amachweba omnye komnye, uqale i-generator kwenye, kunye ne-analyzer okwesibini. Kwakungekho nto imangalisayo apha, kodwa ngokwenene konke kuxhomekeke kwintsimbi, ibuthathaka, iya kuba yinto enomdla ngakumbi.

Ukusebenzisa idatha efunyenweyo yinkqubo yokubeka iliso (Zabbix)

I-tsp ayinayo nayiphi na i-API efundeka ngomatshini njenge-SNMP okanye efanayo. Imiyalezo yeCC kufuneka idityaniswe ubuncinane i-1 isekhondi (ngepesenti ephezulu yokulahleka kwepakethi, kunokuba ngamakhulu / amawaka / amashumi amawaka ngomzuzwana, ngokuxhomekeke kwi-bitrate).

Ke, ukuze ugcine zombini iinkcukacha kunye nokuzoba iigrafu zeempazamo zeCC kunye ne-bitrate kwaye wenze uhlobo oluthile lweengozi, kusenokubakho olu khetho lulandelayo:

Ukuhlalutya kunye nokudibanisa (nge-CC) imveliso ye-tsp, i.e. yiguqulele kwifomu efunwayo.
Gqibezela i-tsp ngokwayo kunye / okanye iiplagi zeprosesa bitrate_monitor kunye nokuqhubeka ukuze umphumo unikwe kwifom efundekayo ngomatshini efanelekileyo kwinkqubo yokubeka iliso.
Bhala isicelo sakho phezu kwethala leencwadi le-tsduck.

Ngokucacileyo, ukhetho 1 lolona lulula ngokubhekiselele kumgudu, ngakumbi ukuqwalasela ukuba i-tsduck ngokwayo ibhalwe kwinqanaba eliphantsi (ngokwemigangatho yanamhlanje) ulwimi (C ++)

I-bash parser elula + i-aggregator prototype ibonise ukuba kumlambo we-10Mbps kunye ne-50% yokulahlekelwa kwepakethe (imeko embi kakhulu), inkqubo ye-bash idliwe amaxesha angama-3-4 ngaphezulu kwe-CPU kunenkqubo ye-tsp ngokwayo. Le meko ayimkelekanga. Ngokwenene isiqwenga sale prototype ingezantsi

IiNoodles phezulu

#!/usr/bin/env bash

missingPackets=0
ccErrorSeconds=0
regexMissPackets='^* (.+) - continuity:.*missing ([0-9]+) packets$'
missingPacketsTime=""

ip netns exec P tsp --realtime -t -I ip -b 8388608 "239.0.0.1:1234" -O drop -P bitrate_monitor -p 1 -t 1  -P continuity 2>&1 | 
while read i
do
    #line example:* 2019/12/28 23:41:14 - continuity: packet index: 6,078, PID: 0x0100, missing 5 packets
    #line example 2: * 2019/12/28 23:55:11 - bitrate_monitor: 2019/12/28 23:55:11, TS bitrate: 4,272,864 bits/s
    if [[ "$i" == *continuity:* ]] 
    then
        if [[ "$i" =~ $regexMissPackets ]]
        then
            missingPacketsTimeNew="${BASH_REMATCH[1]}" #timestamp (seconds)
            if [[ "$missingPacketsTime" != "$missingPacketsTimeNew" ]] #new second with CC error
            then
                ((ccErrorSeconds += 1))
            fi
            missingPacketsTime=$missingPacketsTimeNew
            packets=${BASH_REMATCH[2]} #TS missing packets
            ((missingPackets += packets))
        fi
    elif [[ "$i" == *bitrate_monitor:* ]]
    then
        : #...
    fi
done

Ukongeza kokucotha ngendlela engamkelekanga, akukho misonto iqhelekileyo kwi-bash, imisebenzi ye-bash ziinkqubo ezahlukeneyo, kwaye kuye kwafuneka ndibhale ixabiso lePackets ezilahlekileyo kanye ngomzuzwana kwicala lesiphumo (xa sifumana imiyalezo ye-bitrate ezayo rhoqo ngesekondi). Ngenxa yoko, i-bash yayishiywe yodwa kwaye kwagqitywa ekubeni kubhalwe i-wrapper (parser + aggregator) kwi-golang. Ukusetyenziswa kwe-CPU yekhowudi ye-golang efanayo ngamaxesha angama-4-5 ngaphantsi kwenkqubo ye-tsp ngokwayo. I-speedup ye-wrapper ngenxa yokutshintshwa kwe-bash nge-golang ibe malunga namaxesha angama-16 kwaye ngokubanzi umphumo uyamkeleka (i-CPU ngaphezulu kwe-25% kwimeko embi kakhulu). Ifayile yemvelaphi ye golang ikhona apha.

Qhuba isonga

Ukuqala isisonga, eyona template ilula yenkonzo ye-systemd yenziwe (apha). Isisongelo ngokwaso simelwe kukuhlanganiswa sibe yifayile yokubini (hamba ukwakha tsduck-stat.go) ebekwe kwi /opt/tsduck-stat/. Kucingelwa ukuba usebenzisa igolang ngenkxaso yewotshi ye-monotonic (>=1.9).

Ukwenza umzekelo wenkonzo, kufuneka usebenzise i-systemctl vumela umyalelo [imeyile ikhuselwe]:1234 emva koko ubaleke nge-systemctl qala [imeyile ikhuselwe]: 1234.

Ukufunyanwa kweZabbix

Ukuze i-zabbix ikwazi ukufumana iinkonzo ezisebenzayo, iyenziwa uluhlu lweqela umvelisi (discovery.sh), kwifomathi efunekayo ekufumaneni iZabbix, kucingelwa ukuba ibekwe kwindawo enye - kwi /opt/tsduck-stat. Ukuqhuba ukufunyanwa nge-zabbix-arhente, kufuneka udibanise .conf ifayile kulawulo lwe-zabbix-arhente yoqwalaselo ukongeza iparameter yomsebenzisi.

Zabbix Template

Yenziwe itemplate (tsduck_stat_template.xml) iqulathe umthetho wokuzifumanisa ngokuzenzekelayo, iiprototypes zento, iigrafu, kunye nezichukumisi.

Uluhlu lokukhangela olufutshane (kakuhle, kuthekani ukuba umntu uthatha isigqibo sokuyisebenzisa)

Qinisekisa ukuba i-tsp ayilahli iipakethi phantsi kweemeko "ezifanelekileyo" (i-generator kunye ne-analyzer ixhunyiwe ngokuthe ngqo), ukuba kukho amaconsi, jonga isiqendu 2 okanye isicatshulwa senqaku kulo mbandela.
Yenza uhlengahlengiso olona thintelo luphezulu lwesokethi (net.core.rmem_max=8388608).
Qokelela i-tsduck-stat.go (hamba uye ukwakha tsduck-stat.go).
Beka itemplate yenkonzo kwi/lib/systemd/system.
Qala iinkonzo nge-systemctl, khangela ukuba izixhobo zokubala ziqalile ukubonakala (grep "" /dev/shm/tsduck-stat/*). Inani leenkonzo ngenani losasazo oluninzi. Apha unokufuna ukwenza indlela eya kwiqela lokuphoswa kwezininzi, mhlawumbi khubaza i-rp_filter okanye wenze indlela eya kwimvelaphi ip.
Qhuba i-discovery.sh, qiniseka ukuba ivelisa i-json.
Yongeza i-agent ye-zabbix config, qala kwakhona i-agent ye-zabbix.
Layisha itemplate kwi-zabbix, yifake kwi-host host ebekwe esweni kwaye i-zabbix-agent ifakwe, linda malunga nemizuzu emi-5, ubone ukuba kukho izinto ezintsha, iigrafu kunye ne-triggers.

Isiphumo

Ngomsebenzi wokubona ilahleko yepakethe, iphantse yanele, ubuncinci ingcono kunokubeka iliso.

Enyanisweni, i-CC "ilahleko" inokuthi yenzeke xa idibanisa amaqhekeza evidiyo (ngoko ndikwaziyo, yile ndlela ukufakwa kwe-TV eyenziwa ngayo kumaziko e-TV asekuhlaleni kwi-Russian Federation, oko kukuthi ngaphandle kokubala kwakhona i-counter counter yeCC), oku kufuneka kukhunjulwe. Izisombululo zobunini zithintela le ngxaki ngokuyinxenye ngokufumana iileyibhile ze-SCTE-35 (ukuba zongezwe ngujenereyitha womsinga).

Ngokumalunga nokujongwa komgangatho wezothutho, kukho ukunqongophala kokujongwa kwe-jitter (IAT). Izixhobo ze-TV (ingaba zimodyuli okanye izixhobo zokuphela) zineemfuno zale parameter kwaye akusoloko kunokwenzeka ukunyusela i-jitbuffer ngokungapheliyo. Kwaye i-jitter inokudada xa izixhobo ezinezithinteli ezinkulu zisetyenziswa kuhambo kwaye i-QoS ayiqwalaselwanga okanye ayilungiswanga kakuhle ngokwaneleyo ukuhambisa itrafikhi yexesha lokwenyani.

umthombo: www.habr.com

Ukusebenzisa i-TSDuck ukujonga i-IP(TS) imijelo