I kēia lā, aia nā mea i hoʻomākaukau ʻia (proprietary) no ka nānā ʻana i nā kahawai IP (TS), no ka laʻana и , loaʻa iā lākou kahi hoʻonohonoho pono o nā hana a ʻo ka mea maʻamau nā mea hoʻohana nui e pili ana i nā lawelawe TV i loaʻa nā hopena like. Hōʻike kēia ʻatikala i kahi hoʻonā e pili ana i kahi papahana open source , i hoʻolālā ʻia no ka mana liʻiliʻi o nā kahawai IP(TS) e CC(continuity counter) counter a bitrate. ʻO kahi noi hiki ke hoʻomalu i ka nalowale o nā ʻeke a i ʻole ke kahe holoʻokoʻa ma o kahi ala L2 i hoʻolimalima ʻia (ʻaʻole hiki ke nānā maʻamau, no ka laʻana, ma ka heluhelu ʻana i nā helu poho ma nā pila).
He pōkole loa e pili ana iā TSDuck
ʻO TSDuck kahi polokalamu wehe (2-Clause BSD license) lako polokalamu (kahi hoʻonohonoho o nā mea hoʻohana console a me kahi waihona no ka hoʻomohala ʻana i nā pono hana maʻamau a i ʻole nā plugins) no ka hoʻopunipuni ʻana i nā kahawai TS. Ma keʻano he hoʻokomo, hiki iā ia ke hana me IP (multicast/unicast), http, hls, dvb tuners, dektec dvb-asi demodulator, aia kahi TS-stream generator a heluhelu mai nā faila. Hiki ke kākau ʻia ka hopena i kahi faila, IP (multicast/unicast), hls, dektec dvb-asi a me HiDes modulators, nā mea pāʻani (mplayer, vlc, xine) a hāʻule. Hiki ke hoʻokomo ʻia nā kaʻina hana kaʻa like ʻole ma waena o ka hoʻokomo a me ka hoʻopuka, no ka laʻana, PID remapping, scrambling / descrambling, CC counter analysis, bitrate helu, a me nā hana maʻamau no nā kahawai TS.
Ma kēia ʻatikala, e hoʻohana ʻia nā kahawai IP (multicast) ma ke ʻano he hoʻokomo, hoʻohana ʻia nā kaʻina hana bitrate_monitor (mai ka inoa ʻike ʻia he aha ia) a me ka hoʻomau (ka nānā ʻana o nā helu CC). Hiki iā ʻoe ke hoʻololi maʻalahi i ka IP multicast me kahi ʻano hoʻokomo ʻē aʻe i kākoʻo ʻia e TSDuck.
Loaʻa TSDuck no ka hapanui o nā ʻōnaehana hana o kēia manawa. ʻAʻole i loaʻa iā Debian, akā hiki iā mākou ke kūkulu iā lākou ma lalo o debian 8 a me debian 10 me ka pilikia ʻole.
A laila, hoʻohana ʻia ka mana TSDuck 3.19-1520, hoʻohana ʻia ʻo Linux ma ke ʻano he OS (ua hoʻohana ʻia ʻo debian 10 e hoʻomākaukau i ka hopena, hoʻohana ʻia ʻo CentOS 7 no ka hoʻohana maoli)
E hoʻomākaukau ana iā TSDuck a me OS
Ma mua o ka nānā ʻana i nā kahe maoli, pono ʻoe e hōʻoia i ka hana pololei ʻana o TSDuck a ʻaʻole hiki ke hāʻule i ke kāleka pūnaewele a i ʻole ka pae OS (socket). Pono kēia i ʻole ʻoe e koho ma hope i kahi i hāʻule ai nā hāʻule - ma ka pūnaewele a i ʻole "i loko o ke kikowaena." Hiki iā ʻoe ke nānā i nā hāʻule ma ka pae kāleka pūnaewele me ke kauoha ethtool -S ethX, hana ʻia ke kani ʻana e ka ethtool like (maʻamau pono ʻoe e hoʻonui i ka RX buffer (-G) a i kekahi manawa e hoʻopau i kekahi offloads (-K)). Ma ke ʻano he ʻōlelo aʻoaʻo maʻamau, ʻoi aku ka maikaʻi o ka hoʻohana ʻana i kahi awa ʻokoʻa no ka loaʻa ʻana o ka huakaʻi i kālailai ʻia, inā hiki, e hōʻemi kēia i nā hopena maikaʻi ʻole ma muli o ka hāʻule ʻana o ka hāʻule ʻana i ka manawa like ma ke awa analyer ma muli o ke alo o nā kaʻa ʻē aʻe. Inā ʻaʻole hiki kēia (ke hoʻohana nei ʻoe i kahi mini-computer / NUC me hoʻokahi awa), a laila ʻoi aku ka maikaʻi o ka hoʻonohonoho ʻana i ka mea nui o ka huakaʻi i hoʻopaʻa ʻia e pili ana i ke koena ma ka hāmeʻa kahi i hoʻopili ʻia ai ka mea anaana. E pili ana i nā kaiapuni virtual, pono ʻoe e makaʻala a hiki ke ʻike i nā hāʻule packet e hoʻomaka ana mai ke awa kino a hoʻopau me ka noi i loko o ka mīkini virtual.
Ka hana ʻana a me ka loaʻa ʻana o kahi kahawai i loko o ka host
Ma ke ʻano he hana mua i ka hoʻomākaukau ʻana i ka TSDuck, e hana mākou a loaʻa i nā kaʻa i loko o kahi hoʻokipa hoʻokahi me ka hoʻohana ʻana i nā netns.
Hoʻomākaukau i ke kaiapuni:
ip netns add P #создаём netns P, в нём будет происходить анализ трафика
ip link add type veth #создаём veth-пару - veth0 оставляем в netns по умолчанию (в этот интерфейс будет генерироваться трафик)
ip link set dev veth1 netns P #veth1 - помещаем в netns P (на этом интерфейсе будет приём трафика)
ip netns exec P ifconfig veth1 192.0.2.1/30 up #поднимаем IP на veth1, не имеет значения какой именно
ip netns exec P ip ro add default via 192.0.2.2 #настраиваем маршрут по умолчанию внутри nents P
sysctl net.ipv6.conf.veth0.disable_ipv6=1 #отключаем IPv6 на veth0 - это делается для того, чтобы в счётчик TX не попадал посторонний мусор
ifconfig veth0 up #поднимаем интерфейс veth0
ip route add 239.0.0.1 dev veth0 #создаём маршрут, чтобы ОС направляла трафик к 239.0.0.1 в сторону veth0Ua mākaukau ke kaiapuni. Hoʻomaka mākou i ka mea hoʻoponopono kaʻa:
ip netns exec P tsp --realtime -t
-I ip 239.0.0.1:1234
-P continuity
-P bitrate_monitor -p 1 -t 1
-O dropkahi "-p 1 -t 1" pono ʻoe e helu i ka bitrate i kēlā me kēia kekona a hōʻike i ka ʻike e pili ana i ka bitrate i kēlā me kēia kekona.
Hoʻomaka mākou i ka mea hana kaʻa me ka wikiwiki o 10Mbps:
tsp -I craft
-P regulate -b 10000000
-O ip -p 7 -e --local-port 6000 239.0.0.1:1234kahi "-p 7 -e" 'o ia ho'i, pono 'oe e ho'okomo i 7 mau 'eke TS i loko o ka 'eke IP 1 a pa'akikī (-e), 'o ia ho'i. e kali mau i nā ʻeke TS 7 mai ka mea hana hope ma mua o ka hoʻouna ʻana i kahi ʻeke IP.
Hoʻomaka ka mea anaana e hoʻopuka i nā memo i manaʻo ʻia:
* 2020/01/03 14:55:44 - bitrate_monitor: 2020/01/03 14:55:44, TS bitrate: 9,970,016 bits/s
* 2020/01/03 14:55:45 - bitrate_monitor: 2020/01/03 14:55:45, TS bitrate: 10,022,656 bits/s
* 2020/01/03 14:55:46 - bitrate_monitor: 2020/01/03 14:55:46, TS bitrate: 9,980,544 bits/sI kēia manawa e hoʻohui i kekahi mau kulu:
ip netns exec P iptables -I INPUT -d 239.0.0.1 -m statistic --mode random --probability 0.001 -j DROPa puka mai nā memo e like me kēia:
* 2020/01/03 14:57:11 - continuity: packet index: 80,745, PID: 0x0000, missing 7 packets
* 2020/01/03 14:57:11 - continuity: packet index: 83,342, PID: 0x0000, missing 7 packets ka mea i manaoia. Hoʻopau i ka poho packet (ip netns exec P iptables -F) a hoʻāʻo e hoʻonui i ka bitrate generator i 100Mbps. Ke hōʻike nei ka mea hōʻike i kahi hui o nā hewa CC a ma kahi o 75 Mbps ma kahi o 100. Ke ho'āʻo nei mākou e noʻonoʻo i ka mea hewa - ʻaʻohe manawa o ka generator a i ʻole ka pilikia i loko, no kēia mea mākou e hoʻomaka ai e hana i kahi helu paʻa o ʻeke (700000 ʻeke TS = 100000 ʻeke IP):
# ifconfig veth0 | grep TX
TX packets 151825460 bytes 205725459268 (191.5 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
# tsp -I craft -c 700000 -P regulate -b 100000000 -P count -O ip -p 7 -e --local-port 6000 239.0.0.1:1234
* count: PID 0 (0x0000): 700,000 packets
# ifconfig veth0 | grep TX
TX packets 151925460 bytes 205861259268 (191.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0E like me kāu e ʻike ai, ua hana ʻia nā ʻeke IP 100000 (151925460-151825460). No laila e noʻonoʻo kākou i ka mea e hana nei me ka mea anana, no kēia mea mākou e nānā me ka RX counter ma veth1, ua like loa ia me ka counter TX ma veth0, a laila nānā mākou i ka mea e hana ana ma ka pae kumu:
# ip netns exec P cat /proc/net/udp
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
133: 010000EF:04D2 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 72338 2 00000000e0a441df 24355 Maanei hiki iā ʻoe ke ʻike i ka helu o nā hāʻule = 24355. Ma nā ʻeke TS, ʻo ia ka 170485 a i ʻole 24.36% o 700000, no laila ke ʻike nei mākou ʻo kēlā mau 25% o ka bitrate i nalowale he hāʻule i ke kumu udp. ʻO ka hāʻule ʻana i loko o kahi kumu UDP ma muli o ka nele o ka buffer, e nānā i ka nui o ke kumu paʻa paʻamau a me ka nui o ka nui buffer socket:
# sysctl net.core.rmem_default
net.core.rmem_default = 212992
# sysctl net.core.rmem_max
net.core.rmem_max = 212992No laila, inā ʻaʻole noi pololei nā noi i ka nui buffer, hana ʻia nā kumu me ka buffer o 208 KB, akā inā noi hou lākou, ʻaʻole e loaʻa iā lākou ka mea i noi ʻia. No ka mea hiki iā ʻoe ke hoʻonohonoho i ka nui buffer i ka tsp no ka hoʻokomo IP (-buffer-size), ʻaʻole mākou e hoʻopā i ka nui socket paʻamau, akā hoʻonohonoho wale i ka nui o ka nui buffer a kuhikuhi i ka nui buffer ma o nā ʻōlelo tsp:
sysctl net.core.rmem_max=8388608
ip netns exec P tsp --realtime -t -I ip 239.0.0.1:1234 -b 8388608 -P continuity -P bitrate_monitor -p 1 -t 1 -O dropMe kēia hoʻopaʻa ʻana o ke kumu hoʻopaʻa, ʻo ka bitrate i hōʻike ʻia i kēia manawa ma kahi o 100Mbps, ʻaʻohe hewa CC.
E like me ka hoʻohana ʻana o ka CPU o ka noi tsp ponoʻī. E pili ana i hoʻokahi kumu i5-4260U CPU @ 1.40GHz, 10Mbps kahe ana e koi ana i ka 3-4% CPU, 100Mbps - 25%, 200Mbps - 46%. I ka hoʻonohonoho ʻana i ka % Packet Loss, ʻaʻole i hoʻonui ʻia ka ukana ma ka CPU (akā hiki ke emi).
Ma nā lako hana ʻoi aku ka maikaʻi, hiki ke hana a hoʻopaʻa i nā kahawai ʻoi aku ma mua o 1Gb / s me ka pilikia ʻole.
Ke hoʻāʻo ʻana ma nā kāleka pūnaewele maoli
Ma hope o ka hoʻāʻo ʻana i kahi pālua veth, pono ʻoe e lawe i ʻelua mau pūʻali a i ʻole ʻelua mau awa o hoʻokahi host, e hoʻohui i nā awa i kekahi i kekahi, e hoʻomaka i ka generator ma kekahi, a me ka mea anana ma ka lua. ʻAʻohe mea kupanaha ma aneʻi, akā ʻo ka mea ʻoiaʻiʻo e pili ana i ka hao, ʻoi aku ka nāwaliwali, ʻoi aku ka hoihoi.
Ke hoʻohana nei i ka ʻikepili i loaʻa e ka ʻōnaehana nānā (Zabbix)
tsp ʻaʻohe API hiki ke heluhelu ʻia e ka mīkini e like me SNMP a i ʻole like. Pono e hōʻuluʻulu ʻia nā memo CC no ka liʻiliʻi loa 1 kekona (me ka hapa nui o ka poho ʻeke, hiki ke loaʻa he mau haneli/tausani/ʻumi kaukani i kēlā me kēia kekona, ma muli o ka bitrate).
No laila, i mea e mālama ai i ka ʻike a huki i nā kiʻi no nā hewa CC a me ka bitrate a hana hou i kekahi ʻano pōʻino, aia paha nā koho penei:
- Hoʻopili a hōʻuluʻulu (e CC) i ka hua tsp, i.e. e hoʻololi i ke ʻano i makemake ʻia.
- E hoʻopau i ka tsp iā ia iho a/a i ʻole nā plugins processor bitrate_monitor a me ka hoʻomau i hāʻawi ʻia ka hopena ma kahi palapala hiki ke heluhelu ʻia e ka mīkini kūpono no ka ʻōnaehana nānā.
- Kākau i kāu palapala noi ma luna o ka waihona tsduck.
ʻOiaʻiʻo, ʻo ka koho 1 ka mea maʻalahi loa e pili ana i ka hoʻoikaika ʻana, ʻoi aku ka nui o ka noʻonoʻo ʻana ua kākau ʻia ʻo tsduck iā ia iho ma kahi haʻahaʻa haʻahaʻa (e nā kūlana hou) (C ++)
Ua hōʻike ʻia kahi prototype maʻalahi o kahi parser + aggregator i ka bash i ke kahe o 10 Mbit/s a me 50% packet loss (ʻoi loa ka hihia), ua hoʻopau ke kaʻina bash i 3-4 mau manawa ʻoi aku ka CPU ma mua o ke kaʻina hana tsp. ʻAʻole ʻae ʻia kēia hiʻohiʻona. ʻOiaʻiʻo, aia kahi ʻāpana o kēia prototype ma lalo
Noodles ma luna
#!/usr/bin/env bash
missingPackets=0
ccErrorSeconds=0
regexMissPackets='^* (.+) - continuity:.*missing ([0-9]+) packets$'
missingPacketsTime=""
ip netns exec P tsp --realtime -t -I ip -b 8388608 "239.0.0.1:1234" -O drop -P bitrate_monitor -p 1 -t 1 -P continuity 2>&1 |
while read i
do
#line example:* 2019/12/28 23:41:14 - continuity: packet index: 6,078, PID: 0x0100, missing 5 packets
#line example 2: * 2019/12/28 23:55:11 - bitrate_monitor: 2019/12/28 23:55:11, TS bitrate: 4,272,864 bits/s
if [[ "$i" == *continuity:* ]]
then
if [[ "$i" =~ $regexMissPackets ]]
then
missingPacketsTimeNew="${BASH_REMATCH[1]}" #timestamp (seconds)
if [[ "$missingPacketsTime" != "$missingPacketsTimeNew" ]] #new second with CC error
then
((ccErrorSeconds += 1))
fi
missingPacketsTime=$missingPacketsTimeNew
packets=${BASH_REMATCH[2]} #TS missing packets
((missingPackets += packets))
fi
elif [[ "$i" == *bitrate_monitor:* ]]
then
: #...
fi
doneMa waho aʻe o ka lohi ʻole, ʻaʻohe mau kaula maʻamau i ka bash, ʻo nā hana bash nā kaʻina kaʻawale, a pono wau e kākau i ka waiwai o missingPackets i hoʻokahi kekona ma ka hopena ʻaoʻao (i ka wā e loaʻa ai nā leka bitrate e hiki mai ana i kēlā me kēia kekona). ʻO ka hopena, ua waiho wale ʻia ka bash a ua hoʻoholo ʻia e kākau i kahi wīwī (parser + aggregator) ma golang. ʻO ka hoʻohana ʻana i ka CPU o ke code golang like he 4-5 mau manawa ma mua o ke kaʻina hana tsp ponoʻī. ʻO ka wikiwiki o ka wīwī ma muli o ka hoʻololi ʻana o ka bash me ka golang ma kahi o 16 mau manawa a ma ke ʻano maʻamau ua ʻae ʻia ka hopena (CPU ma luna o 25% i ka hihia ʻino loa). Aia ka waihona kumu golang .
Holoi ʻōwili
No ka hoʻomaka ʻana i ka wīwī, ua hana ʻia ke ʻano lawelawe maʻalahi loa no systemd (). Manaʻo ʻia e hoʻopili ʻia ka mea hoʻopili ponoʻī i loko o kahi faila binary (hele kūkulu tsduck-stat.go) aia ma /opt/tsduck-stat/. Manaʻo ʻia ke hoʻohana nei ʻoe i ka golang me ke kākoʻo no ka uaki monotonic (>=1.9).
No ka hana ʻana i kahi hiʻohiʻona o ka lawelawe, pono ʻoe e holo i ke kauoha systemctl enable [pale ʻia ka leka uila]: 1234 a laila holo me systemctl hoʻomaka [pale ʻia ka leka uila]: 1234.
ʻIke ʻia mai Zabbix
I mea e hiki ai iā zabbix ke ʻike i nā lawelawe holo, ua hana ʻia (discovery.sh), ma ke ʻano i koi ʻia no ka ʻike ʻana iā Zabbix, ua manaʻo ʻia aia ma kahi hoʻokahi - ma /opt/tsduck-stat. E holo i ka ʻike ma o zabbix-agent, pono ʻoe e hoʻohui i ka papa kuhikuhi hoʻonohonoho zabbix-agent e hoʻohui i ka mea hoʻohana.
ʻO ka laʻana Zabbix
(tsduck_stat_template.xml) loaʻa ke kānāwai ʻike ʻokoʻa, nā prototype ʻikamu, nā kiʻi, a me nā mea hoʻomaka.
He papa helu pōkole (pehea inā hoʻoholo kekahi e hoʻohana ia mea)
- E hōʻoia ʻaʻole e hāʻule ka tsp i nā ʻeke ma lalo o nā kūlana "kūpono" (pili pololei ʻia ka mea hana a me ka mea anaana), inā he hāʻule, e ʻike i ka paukū 2 a i ʻole ke kikokikona o ka ʻatikala e pili ana i kēia mea.
- E hoʻokani i ka mea hoʻopaʻa kumu kiʻekiʻe loa (net.core.rmem_max=8388608).
- E hōʻuluʻulu i tsduck-stat.go (hele e kūkulu i tsduck-stat.go).
- E hoʻokomo i ka hoʻohālike lawelawe i /lib/systemd/system.
- E hoʻomaka i nā lawelawe me systemctl, e nānā ua hoʻomaka nā helu helu (grep "" /dev/shm/tsduck-stat/*). ʻO ka helu o nā lawelawe ma ka helu o nā kahawai multicast. Pono paha ʻoe e hana i kahi ala i ka hui multicast, hoʻopau paha i ka rp_filter a i ʻole e hana i kahi ala i ke kumu ip.
- E holo i ka discovery.sh, e hōʻoia i ka hana ʻana i ka json.
- E hoʻohui i ka config agent zabbix, e hoʻomaka hou i ka mea hana zabbix.
- E hoʻouka i ka template i zabbix, e hoʻopili iā ia i ka mea hoʻokipa e nānā ʻia a hoʻokomo ʻia ka zabbix-agent, e kali ma kahi o 5 mau minuke, e ʻike inā he mau mea hou, nā kiʻi a me nā mea hoʻoiho.
hopena
No ka hana o ka ʻike ʻana i ka poho packet, ʻaneʻane lawa, ʻoi aku ka maikaʻi ma mua o ka nānā ʻole ʻana.
ʻOiaʻiʻo, hiki ke loaʻa nā "losses" CC i ka hoʻohui ʻana i nā ʻāpana wikiō (e like me kaʻu i ʻike ai, eia ke ʻano o ka hoʻokomo ʻana i nā kikowaena TV kūloko i ka Russian Federation, ʻo ia hoʻi me ka helu ʻole ʻana i ka counter CC), pono e hoʻomanaʻo ʻia kēia. ʻO nā haʻina ponoʻī e pale i kēia pilikia ma ka ʻike ʻana i nā lepili SCTE-35 (inā hoʻohui ʻia e ka mea hoʻoheheʻe kahawai).
Ma ke ʻano o ka nānā ʻana i ka maikaʻi o ka halihali, aia ka nele o ka nānā ʻana i ka jitter (IAT). Loaʻa i nā lako TV (ʻo ia nā modulators a i ʻole nā mea hoʻopau) i nā koi no kēia ʻāpana a ʻaʻole hiki ke hoʻonui i ka jitbuffer i ka pau ʻole. A hiki i ka jitter ke lana i ka wā e hoʻohana ʻia ai nā lako me nā buffers nui i ka transit a ʻaʻole i hoʻonohonoho pono ʻia ʻo QoS a ʻaʻole i hoʻonohonoho maikaʻi ʻia e hoʻouna i kēlā mau kaʻa manawa maoli.
Source: www.habr.com