A yau, akwai shirye-shiryen da aka yi (na mallaka) mafita don kula da rafukan IP (TS), alal misali и , suna da kyawawan ayyuka masu arziƙi kuma yawanci manyan masu aiki da ke hulɗa da ayyukan TV suna da irin waɗannan mafita. Wannan labarin yana bayyana mafita dangane da aikin buɗaɗɗen tushe , An tsara don ƙananan iko na rafukan IP (TS) ta CC (counter counter) counter da bitrate. Mai yuwuwar aikace-aikacen ita ce sarrafa asarar fakiti ko gabaɗaya ta hanyar tashar L2 da aka yi hayar (wanda ba za a iya saka idanu akai-akai ba, misali, ta hanyar karanta lissafin asarar a cikin jerin gwano).
A taƙaice game da TDuck
TSDuck tushen buɗaɗɗen software ne (lasisin BSD 2-Clause) (saitin kayan aikin wasan bidiyo da ɗakin karatu don haɓaka abubuwan amfani na al'ada ko plugins) don sarrafa rafukan TS. A matsayin shigarwa, yana iya aiki tare da IP (multicast/unicast), http, hls, dvb tuners, dektec dvb-asi demodulator, akwai janareta na TS-rafi na ciki da karantawa daga fayiloli. Fitarwa na iya zama rubutawa zuwa fayil, IP (multicast/unicast), hls, dektec dvb-asi da HiDes modulators, yan wasa (mplayer, vlc, xine) da sauke. Ana iya haɗa na'urori daban-daban na zirga-zirga tsakanin shigarwa da fitarwa, misali, sake taswirar PID, ɓarnawa / ɓarna, ƙididdigar ƙididdigar CC, lissafin bitrate, da sauran ayyuka na yau da kullun don rafukan TS.
A cikin wannan labarin, za a yi amfani da rafukan IP (multicast) azaman shigarwa, ana amfani da masu sarrafawa bitrate_monitor (daga sunan ya bayyana abin da yake) da kuma ci gaba (bincike na ƙididdigar CC). Kuna iya sauƙin maye gurbin multicast IP tare da wani nau'in shigarwa wanda TDuck ke tallafawa.
Akwai TSDuck don yawancin tsarin aiki na yanzu. Ba su samuwa ga Debian, amma mun sami nasarar gina su a ƙarƙashin debian 8 da debian 10 ba tare da wata matsala ba.
Na gaba, ana amfani da sigar TSDuck 3.19-1520, ana amfani da Linux azaman OS (an yi amfani da debian 10 don shirya maganin, an yi amfani da CentOS 7 don amfani na gaske)
Ana shirya TSDuck da OS
Kafin saka idanu ainihin kwararar ruwa, kuna buƙatar tabbatar da cewa TDuck yana aiki daidai kuma babu digo a katin cibiyar sadarwa ko matakin OS (socket). Ana buƙatar wannan don kar a yi la'akari daga baya inda faɗuwar ta faru - akan hanyar sadarwa ko "cikin uwar garken". Kuna iya duba digo a matakin katin cibiyar sadarwa tare da umarnin ethtool -S ethX, kunnawa ana yin ta ta hanyar ethtool iri ɗaya (yawanci, kuna buƙatar haɓaka buffer RX (-G) kuma wani lokacin kashe wasu abubuwan kashewa (-K)). A matsayin shawarwarin gabaɗaya, ana iya ba da shawarar yin amfani da tashar jiragen ruwa daban don karɓar zirga-zirgar da aka bincika, idan zai yiwu, wannan yana rage ƙimar ƙaryar da ke da alaƙa da gaskiyar cewa faɗuwar ta faru daidai a tashar tashar mai nazari saboda kasancewar sauran zirga-zirga. Idan wannan ba zai yiwu ba (ana amfani da mini-kwamfuta / NUC tare da tashar jiragen ruwa ɗaya), to yana da kyau sosai don saita fifiko na zirga-zirgar da aka bincika dangane da sauran akan na'urar da aka haɗa mai nazarin. Game da mahallin kama-da-wane, a nan kuna buƙatar yin hankali kuma ku sami damar samun fakitin fakiti farawa daga tashar jiragen ruwa ta zahiri da ƙarewa tare da aikace-aikacen cikin injin kama-da-wane.
Ƙirƙiri da liyafar rafi a cikin mai watsa shiri
A matsayin mataki na farko na shirya TSDuck, za mu ƙirƙira da karɓar zirga-zirga tsakanin runduna guda ta amfani da netns.
Shirya muhalli:
ip netns add P #создаём netns P, в нём будет происходить анализ трафика
ip link add type veth #создаём veth-пару - veth0 оставляем в netns по умолчанию (в этот интерфейс будет генерироваться трафик)
ip link set dev veth1 netns P #veth1 - помещаем в netns P (на этом интерфейсе будет приём трафика)
ip netns exec P ifconfig veth1 192.0.2.1/30 up #поднимаем IP на veth1, не имеет значения какой именно
ip netns exec P ip ro add default via 192.0.2.2 #настраиваем маршрут по умолчанию внутри nents P
sysctl net.ipv6.conf.veth0.disable_ipv6=1 #отключаем IPv6 на veth0 - это делается для того, чтобы в счётчик TX не попадал посторонний мусор
ifconfig veth0 up #поднимаем интерфейс veth0
ip route add 239.0.0.1 dev veth0 #создаём маршрут, чтобы ОС направляла трафик к 239.0.0.1 в сторону veth0Yanayin yana shirye. Muna fara nazarin zirga-zirga:
ip netns exec P tsp --realtime -t
-I ip 239.0.0.1:1234
-P continuity
-P bitrate_monitor -p 1 -t 1
-O dropinda "-p 1 -t 1" yana nufin cewa kana buƙatar lissafin bitrate kowane daƙiƙa kuma ka nuna bayanin game da bitrate kowane sakan.
Muna fara janareta na zirga-zirga da saurin 10Mbps:
tsp -I craft
-P regulate -b 10000000
-O ip -p 7 -e --local-port 6000 239.0.0.1:1234inda "-p 7 -e" yana nufin cewa kana buƙatar shirya fakiti 7 TS a cikin fakitin IP guda 1 kuma yi shi da wuya (-e), watau. koyaushe jira fakiti 7 TS daga na'urar sarrafawa ta ƙarshe kafin aika fakitin IP.
Mai nazari yana fara fitar da saƙon da ake sa ran:
* 2020/01/03 14:55:44 - bitrate_monitor: 2020/01/03 14:55:44, TS bitrate: 9,970,016 bits/s
* 2020/01/03 14:55:45 - bitrate_monitor: 2020/01/03 14:55:45, TS bitrate: 10,022,656 bits/s
* 2020/01/03 14:55:46 - bitrate_monitor: 2020/01/03 14:55:46, TS bitrate: 9,980,544 bits/sYanzu ƙara wasu digo:
ip netns exec P iptables -I INPUT -d 239.0.0.1 -m statistic --mode random --probability 0.001 -j DROPkuma sakonni kamar haka suna bayyana:
* 2020/01/03 14:57:11 - continuity: packet index: 80,745, PID: 0x0000, missing 7 packets
* 2020/01/03 14:57:11 - continuity: packet index: 83,342, PID: 0x0000, missing 7 packets wanda ake sa ran. Kashe asarar fakiti (ip netns exec P iptables -F) kuma gwada ƙara bitrate na janareta zuwa 100Mbps. Mai nazari ya ba da rahoton bunch of CC kurakurai kuma game da 75 Mbps maimakon 100. Muna ƙoƙarin gano wanda ke da laifi - janareta ba shi da lokaci ko matsalar ba ta cikinsa, saboda wannan mun fara samar da ƙayyadaddun adadin adadin. fakiti (fakiti 700000 TS = fakitin IP 100000):
# ifconfig veth0 | grep TX
TX packets 151825460 bytes 205725459268 (191.5 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
# tsp -I craft -c 700000 -P regulate -b 100000000 -P count -O ip -p 7 -e --local-port 6000 239.0.0.1:1234
* count: PID 0 (0x0000): 700,000 packets
# ifconfig veth0 | grep TX
TX packets 151925460 bytes 205861259268 (191.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0Kamar yadda kuke gani, an samar da fakitin IP daidai 100000 (151925460-151825460). Don haka bari mu gano abin da ke faruwa tare da mai nazari, saboda wannan muna bincika tare da ma'aunin RX akan veth1, daidai yake da ma'aunin TX akan veth0, sannan mu kalli abin da ke faruwa a matakin soket:
# ip netns exec P cat /proc/net/udp
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
133: 010000EF:04D2 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 72338 2 00000000e0a441df 24355 Anan zaka iya ganin adadin raguwa = 24355. A cikin fakitin TS, wannan shine 170485 ko 24.36% na 700000, don haka muna ganin cewa waɗannan 25% na asarar bitrate sun ragu a cikin soket na udp. Saukowa a cikin soket na UDP yawanci yana faruwa ne saboda rashin buffer, duba girman buffer tsoho da matsakaicin girman buffer soket:
# sysctl net.core.rmem_default
net.core.rmem_default = 212992
# sysctl net.core.rmem_max
net.core.rmem_max = 212992Don haka, idan aikace-aikacen ba su fito fili suna buƙatar girman buffer ba, ana ƙirƙira kwasfa tare da buffer na 208 KB, amma idan sun nemi ƙari, har yanzu ba za su sami abin da aka nema ba. Tun da zaku iya saita girman buffer a cikin tsp don shigar da IP (-size-size), ba za mu taɓa girman soket ɗin tsoho ba, amma kawai saita matsakaicin girman buffer soket kuma saka girman buffer a sarari ta cikin muhawarar tsp:
sysctl net.core.rmem_max=8388608
ip netns exec P tsp --realtime -t -I ip 239.0.0.1:1234 -b 8388608 -P continuity -P bitrate_monitor -p 1 -t 1 -O dropTare da wannan kunnawa na soket buffer, yanzu rahoton bitrate ya kusan 100Mbps, babu kurakurai CC.
Dangane da yawan amfani da CPU na aikace-aikacen tsp kanta. Dangane da i5-4260U CPU mai mahimmanci guda ɗaya @ 1.40GHz, nazarin kwararar 10Mbps zai buƙaci 3-4% CPU, 100Mbps - 25%, 200Mbps - 46%. Lokacin saita asarar fakiti %, nauyin da ke kan CPU a zahiri baya karuwa (amma yana iya raguwa).
A kan ƙarin kayan masarufi, yana yiwuwa a ƙirƙira da bincika rafukan sama da 1Gb/s ba tare da wata matsala ba.
Gwaji akan katunan cibiyar sadarwa na gaske
Bayan gwaji a kan veth biyu, kuna buƙatar ɗaukar runduna biyu ko tashoshi biyu na runduna ɗaya, haɗa tashoshin jiragen ruwa da juna, fara janareta akan ɗaya, da na'urar nazari akan na biyu. Babu wani abin mamaki a nan, amma a gaskiya duk ya dogara da ƙarfe, mai rauni, mafi ban sha'awa zai kasance a nan.
Amfani da bayanan da aka karɓa ta tsarin kulawa (Zabbix)
tsp ba shi da API mai karanta na'ura kamar SNMP ko makamancin haka. Dole ne a tara saƙonnin CC na akalla daƙiƙa 1 (tare da yawan asarar fakiti, za a iya samun ɗaruruwa/dubbai/dubban dubbai a cikin sakan daya, dangane da bitrate).
Don haka, don adana bayanan biyu da zana hotuna don kurakuran CC da bitrate da yin wasu nau'ikan hatsarori, ana iya samun zaɓuɓɓuka masu zuwa:
- Tsaya da tara (ta CC) fitar da tsp, watau. canza shi zuwa sigar da ake so.
- Gama tsp kanta da/ko processor plugins bitrate_monitor da ci gaba don a ba da sakamakon a cikin nau'i mai karantawa na inji wanda ya dace da tsarin sa ido.
- Rubuta aikace-aikacen ku a saman ɗakin karatu na tsduck.
Babu shakka, zaɓi na 1 shine mafi sauƙi dangane da ƙoƙari, musamman la'akari da cewa tsduck da kansa an rubuta shi a cikin ƙananan matakan (ta tsarin zamani) (C ++)
Wani samfurin bash parser + aggregator samfurin ya nuna cewa akan rafi 10Mbps da asarar fakiti 50% (mafi munin shari'ar), tsarin bash ya cinye CPU sau 3-4 fiye da tsarin tsp kanta. Ba za a yarda da wannan yanayin ba. Haƙiƙa wani yanki na wannan samfuri a ƙasa
Noodles a saman
#!/usr/bin/env bash
missingPackets=0
ccErrorSeconds=0
regexMissPackets='^* (.+) - continuity:.*missing ([0-9]+) packets$'
missingPacketsTime=""
ip netns exec P tsp --realtime -t -I ip -b 8388608 "239.0.0.1:1234" -O drop -P bitrate_monitor -p 1 -t 1 -P continuity 2>&1 |
while read i
do
#line example:* 2019/12/28 23:41:14 - continuity: packet index: 6,078, PID: 0x0100, missing 5 packets
#line example 2: * 2019/12/28 23:55:11 - bitrate_monitor: 2019/12/28 23:55:11, TS bitrate: 4,272,864 bits/s
if [[ "$i" == *continuity:* ]]
then
if [[ "$i" =~ $regexMissPackets ]]
then
missingPacketsTimeNew="${BASH_REMATCH[1]}" #timestamp (seconds)
if [[ "$missingPacketsTime" != "$missingPacketsTimeNew" ]] #new second with CC error
then
((ccErrorSeconds += 1))
fi
missingPacketsTime=$missingPacketsTimeNew
packets=${BASH_REMATCH[2]} #TS missing packets
((missingPackets += packets))
fi
elif [[ "$i" == *bitrate_monitor:* ]]
then
: #...
fi
doneBaya ga kasancewa da jinkirin da ba a yarda da shi ba, babu zaren al'ada a cikin bash, ayyukan bash matakai ne daban, kuma dole ne in rubuta ƙimar ɓacewarPackets sau ɗaya a sakan daya akan sakamako na gefe (lokacin karɓar saƙonnin bitrate waɗanda ke zuwa kowane sakan). Sakamakon haka, an bar bash shi kaɗai kuma an yanke shawarar rubuta abin rufe (parser + aggregator) a cikin golang. Amfanin CPU na lambar golan irin wannan shine sau 4-5 ƙasa da tsarin tsp kanta. Saurin abin rufewa saboda maye gurbin bash tare da golang ya zama kusan sau 16 kuma gabaɗaya sakamakon yana karɓa (CPU sama da 25% a cikin mafi munin yanayi). Fayil na tushen golang yana nan .
Run abin rufe fuska
Don fara kunsa, an yi samfurin sabis mafi sauƙi na systemd (). Ya kamata a haɗa kundi da kanta cikin fayil ɗin binary (go build tsduck-stat.go) dake cikin /opt/tsduck-stat/. Ana ɗauka cewa kuna amfani da golan tare da goyan bayan agogon monotonic (> = 1.9).
Don ƙirƙirar misalin sabis ɗin, kuna buƙatar gudanar da tsarin ba da damar systemctl [email kariya]:1234 sannan kuyi aiki tare da systemctl start [email kariya]: 1234.
Gano daga Zabbix
Domin zabbix ya sami damar gano ayyukan aiki, an yi (discovery.sh), a cikin tsarin da ake buƙata don gano Zabbix, ana ɗauka cewa yana cikin wuri ɗaya - a /opt/tsduck-stat. Don gudanar da bincike ta hanyar zabbix-agent, kuna buƙatar ƙarawa zuwa ga tsarin daidaitawa na zabbix-agent don ƙara siginar mai amfani.
Samfuran Zabbix
(tsduck_stat_template.xml) yana ƙunshe da ƙa'idar ganowa ta atomatik, samfuran abubuwa, jadawalai, da abubuwan jan hankali.
Takaitaccen lissafi (da kyau, menene idan wani ya yanke shawarar amfani da shi)
- Tabbatar cewa tsp ba ya sauke fakiti a ƙarƙashin yanayin "madaidaici" (an haɗa janareta da na'urar nazari kai tsaye), idan akwai faɗuwar, duba sakin layi na 2 ko rubutun labarin akan wannan batun.
- Yi daidaita madaidaicin buffer soket (net.core.rmem_max=8388608).
- Haɗa tsduck-stat.go (je gina tsduck-stat.go).
- Saka samfurin sabis a /lib/systemd/system.
- Fara ayyuka tare da systemctl, duba cewa masu lissafin sun fara bayyana (grep "" /dev/shm/tsduck-stat/*). Adadin ayyuka ta adadin rafukan watsa labarai da yawa. Anan kuna iya buƙatar ƙirƙirar hanya zuwa rukunin multicast, watakila kashe rp_filter ko ƙirƙirar hanya zuwa tushen ip.
- Gudun discovery.sh, tabbatar yana haifar da json.
- Ƙara saitin wakili na zabbix, sake kunna wakilin zabbix.
- Loda samfurin zuwa zabbix, yi amfani da shi ga mai watsa shirye-shiryen da ake kulawa kuma an shigar da zabbix-agent, jira kimanin minti 5, duba idan akwai sababbin abubuwa, jadawalai da abubuwan jawo.
sakamakon
Don aikin gano asarar fakiti, ya kusan isa, aƙalla yana da kyau fiye da saka idanu.
Lalle ne, CC "asarar" na iya faruwa a lokacin da ake haɗa gutsuttsura bidiyo (kamar yadda na sani, wannan shine yadda ake sakawa a cibiyoyin TV na gida a cikin Tarayyar Rasha, watau ba tare da sake lissafin CC counter ba), dole ne a tuna da wannan. Maganganun mallakar mallakar wani yanki sun kewaye wannan matsala ta hanyar gano alamun SCTE-35 (idan mai samar da rafi ya ƙara).
Dangane da sa ido kan ingancin sufuri, akwai rashin kulawar jitter (IAT). Kayan aikin talbijin (ya kasance masu daidaitawa ko na'urori na ƙarshe) suna da buƙatu don wannan siga kuma ba koyaushe zai yiwu a kunna jitbuffer zuwa rashin iyaka ba. Kuma jitter na iya yin iyo lokacin da ake amfani da kayan aiki tare da manyan buffers a jigilar kaya kuma ba a saita QoS ko kuma ba a tsara shi sosai don watsa irin wannan zirga-zirgar lokaci ba.
source: www.habr.com