Kajeno, ho na le tharollo e lokiselitsoeng (ea mong'a ntlo) bakeng sa ho lekola melapo ea IP (TS), mohlala. и , ba na le mesebetsi e mengata haholo 'me hangata basebetsi ba kholo ba sebetsanang le litšebeletso tsa TV ba na le tharollo e joalo. Sehlooho sena se hlalosa tharollo e thehiloeng morerong oa mohloli o bulehileng , e etselitsoeng taolo e fokolang ea melapo ea IP(TS) ka CC(continuity counter) counter le bitrate. Kopo e ka khonehang ke ho laola tahlehelo ea lipakete kapa phallo eohle ka mocha oa L2 o hiriloeng (o ke keng oa behoa leihlo ka tloaelo, mohlala, ka ho bala li-counters tsa tahlehelo ka mela).
Ka bokhutšoanyane haholo ka TSDuck
TSDuck ke mohloli o bulehileng (laesense ea 2-Clause BSD) software (sete sa lisebelisoa tsa li-console le laeborari ea ho nts'etsapele lits'ebeletso kapa li-plugins) bakeng sa ho thetsa melapo ea TS. E le ho kenya letsoho, e ka sebetsa le IP (multicast/unicast), http, hls, dvb tuners, dektec dvb-asi demodulator, ho na le jenereithara ea ka hare ea TS-stream le ho bala ho tsoa lifaeleng. Sephetho se ka ngolla faele, IP (multicast/unicast), hls, dektec dvb-asi le HiDes modulators, libapali (mplayer, vlc, xine) le drop. Li-processor tse fapaneng tsa sephethe-phethe li ka kenyelletsoa lipakeng tsa ho kenya le ho tsoa, mohlala, ho pheta-pheta PID, ho qhekella / ho theola, tlhahlobo ea li-counter tsa CC, lipalo tsa bitrate, le lits'ebetso tse ling tse tloaelehileng bakeng sa melapo ea TS.
Sehloohong sena, IP streams (multicast) e tla sebelisoa e le ho kenya letsoho, li-processor bitrate_monitor (ho tloha ka lebitso ho hlakile hore na ke eng) le ho tsoelapele (tlhahlobo ea li-counters tsa CC) li sebelisoa. U ka nkela sebaka sa IP multicast habonolo ka mofuta o mong oa ho kenya o tšehetsoeng ke TSDuck.
E fumaneha TSDuck bakeng sa litsamaiso tse ngata tsa hajoale tse sebetsang. Ha li fumanehe bakeng sa Debian, empa re khonne ho li haha ka tlas'a debian 8 le debian 10 ntle le mathata.
Ka mor'a moo, ho sebelisoa mofuta oa TSDuck 3.19-1520, Linux e sebelisoa e le OS (debian 10 e ne e sebelisetsoa ho lokisa tharollo, CentOS 7 e ne e sebelisetsoa tšebeliso ea sebele)
Ho lokisa TSDuck le OS
Pele o shebella phallo ea 'nete, o hloka ho etsa bonnete ba hore TSDuck e sebetsa ka nepo mme ha ho na marotholi ho karete ea marang-rang kapa boemo ba OS (socket). Sena sea hlokahala e le hore u se ke ua hakanya hamorao moo marotholi a hlahileng - marang-rang kapa "ka hare ho seva". U ka hlahloba marotholi boemong ba karete ea marang-rang ka taelo ea ethtool -S ethX, tokiso e etsoa ke ethtool e tšoanang (hangata, ho hlokahala hore u eketse RX buffer (-G) 'me ka linako tse ling u tima lintho tse ling (-K)). E le khothaletso e akaretsang, e ka eletsoa ho sebelisa kou e arohaneng bakeng sa ho amohela sephethephethe se hlahlobiloeng, haeba ho khonahala, sena se fokotsa lintlha tse fosahetseng tse amanang le taba ea hore lerotholi le etsahetse hantle boema-kepeng ba analyzer ka lebaka la boteng ba sephethephethe se seng. Haeba sena se ke ke sa khoneha (k'homphieutha e nyenyane / NUC e nang le koung e le 'ngoe e sebelisoa), joale ke ntho e lakatsehang haholo ho theha pele ho sephethephethe se hlahlobiloeng mabapi le tse ling ho sesebelisoa seo mohlahlobi a amanang le sona. Mabapi le tikoloho ea sebele, mona u hloka ho ba hlokolosi 'me u khone ho fumana marotholi a pakete ho tloha boema-kepe ba' mele le ho qetella ka kopo ka har'a mochini o sebetsang.
Moloko le kamohelo ea molapo ka hare ho moamoheli
E le mohato oa pele oa ho lokisa TSDuck, re tla hlahisa le ho amohela sephethephethe ka har'a moamoheli a le mong re sebelisa marang-rang.
Ho lokisa tikoloho:
ip netns add P #создаём netns P, в нём будет происходить анализ трафика
ip link add type veth #создаём veth-пару - veth0 оставляем в netns по умолчанию (в этот интерфейс будет генерироваться трафик)
ip link set dev veth1 netns P #veth1 - помещаем в netns P (на этом интерфейсе будет приём трафика)
ip netns exec P ifconfig veth1 192.0.2.1/30 up #поднимаем IP на veth1, не имеет значения какой именно
ip netns exec P ip ro add default via 192.0.2.2 #настраиваем маршрут по умолчанию внутри nents P
sysctl net.ipv6.conf.veth0.disable_ipv6=1 #отключаем IPv6 на veth0 - это делается для того, чтобы в счётчик TX не попадал посторонний мусор
ifconfig veth0 up #поднимаем интерфейс veth0
ip route add 239.0.0.1 dev veth0 #создаём маршрут, чтобы ОС направляла трафик к 239.0.0.1 в сторону veth0Tikoloho e lokile. Re qala tlhahlobo ea sephethephethe:
ip netns exec P tsp --realtime -t
-I ip 239.0.0.1:1234
-P continuity
-P bitrate_monitor -p 1 -t 1
-O dropmoo "-p 1 -t 1" e bolelang hore o hloka ho bala bitrate motsotsoana o mong le o mong ebe o bonts'a tlhahisoleseling mabapi le bitrate motsotsoana o mong le o mong.
Re qala jenereithara ea sephethephethe ka lebelo la 10Mbps:
tsp -I craft
-P regulate -b 10000000
-O ip -p 7 -e --local-port 6000 239.0.0.1:1234moo "-p 7 -e" e bolelang hore o hloka ho kenya lipakete tsa 7 TS ka har'a pakete ea 1 IP le ho e etsa ka thata (-e), i.e. kamehla emela lipakete tse 7 TS ho tsoa ho processor ea ho qetela pele u romella pakete ea IP.
Analyzer e qala ho hlahisa melaetsa e lebelletsoeng:
* 2020/01/03 14:55:44 - bitrate_monitor: 2020/01/03 14:55:44, TS bitrate: 9,970,016 bits/s
* 2020/01/03 14:55:45 - bitrate_monitor: 2020/01/03 14:55:45, TS bitrate: 10,022,656 bits/s
* 2020/01/03 14:55:46 - bitrate_monitor: 2020/01/03 14:55:46, TS bitrate: 9,980,544 bits/sJoale eketsa marotholi:
ip netns exec P iptables -I INPUT -d 239.0.0.1 -m statistic --mode random --probability 0.001 -j DROPmme ho hlaha melaetsa e kang ena:
* 2020/01/03 14:57:11 - continuity: packet index: 80,745, PID: 0x0000, missing 7 packets
* 2020/01/03 14:57:11 - continuity: packet index: 83,342, PID: 0x0000, missing 7 packets e lebeletsoeng. Tlosa tahlehelo ea pakete (ip netns exec P iptables -F) 'me u leke ho eketsa bitrate ea jenereithara ho 100Mbps. Analyzer e tlaleha liphoso tse ngata tsa CC le hoo e ka bang 75 Mbps ho e-na le 100. Re leka ho fumana hore na ke mang ea molato - jenereithara ha e na nako kapa bothata ha bo ho eona, bakeng sa sena re qala ho hlahisa palo e tsitsitseng ea lipakete (lipakete tse 700000 TS = 100000 IP pakete):
# ifconfig veth0 | grep TX
TX packets 151825460 bytes 205725459268 (191.5 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
# tsp -I craft -c 700000 -P regulate -b 100000000 -P count -O ip -p 7 -e --local-port 6000 239.0.0.1:1234
* count: PID 0 (0x0000): 700,000 packets
# ifconfig veth0 | grep TX
TX packets 151925460 bytes 205861259268 (191.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0Joalokaha u bona, lipakete tsa IP tsa 100000 hantle li entsoe (151925460-151825460). Kahoo a re boneng hore na ho etsahalang ka mohlahlobi, bakeng sa sena re hlahloba le RX counter ho veth1, e lekana hantle le TX counter on veth0, ebe re sheba se etsahalang boemong ba sokete:
# ip netns exec P cat /proc/net/udp
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
133: 010000EF:04D2 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 72338 2 00000000e0a441df 24355 Mona u ka bona palo ea marotholi = 24355. Lipaketeng tsa TS, sena ke 170485 kapa 24.36% ea 700000, kahoo re bona hore 25% e tšoanang ea bitrate e lahlehileng ke marotholi ka har'a sokete ea udp. Marotholi ka soketeng ea UDP hangata a etsahala ka lebaka la khaello ea buffer, sheba boholo ba sokete ea sokete le boholo bo boholo ba sokete:
# sysctl net.core.rmem_default
net.core.rmem_default = 212992
# sysctl net.core.rmem_max
net.core.rmem_max = 212992Kahoo, haeba lits'ebetso li sa kope ka ho hlaka boholo ba buffer, li-sockets li entsoe ka "buffer" ea 208 KB, empa haeba ba kopa ho feta, ba ntse ba ke ke ba fumana se kopiloeng. Kaha o ka seta boholo ba buffer ka tsp bakeng sa tlhahiso ea IP (-buffer-size), re ke ke ra ama saese ea sokete e sa feleng, empa re behe feela boholo ba sokete ea sokete ebe re bolela boholo ba buffer ka ho hlaka ka likhang tsa tsp:
sysctl net.core.rmem_max=8388608
ip netns exec P tsp --realtime -t -I ip 239.0.0.1:1234 -b 8388608 -P continuity -P bitrate_monitor -p 1 -t 1 -O dropKa tokiso ena ea socket buffer, hona joale bitrate e tlalehiloeng e ka ba 100Mbps, ha ho na liphoso tsa CC.
Ho latela ts'ebeliso ea CPU ea tsp tsp ka boeona. Ho latela core e le 'ngoe i5-4260U CPU @ 1.40GHz, tlhahlobo ea phallo ea 10Mbps e tla hloka 3-4% CPU, 100Mbps - 25%, 200Mbps - 46%. Ha u beha % Packet Loss, mojaro ho CPU ha e eketsehe (empa e ka fokotseha).
Ho lisebelisoa tse ngata tse hlahisang litholoana, ho ne ho khoneha ho hlahisa le ho sekaseka melapo e fetang 1Gb / s ntle le mathata.
Ho etsa liteko ka likarete tsa 'nete tsa marang-rang
Ka mor'a ho etsa liteko ka veth pair, u lokela ho nka mabotho a mabeli kapa likou tse peli tsa moeti a le mong, ho hokahanya likou ho e mong, ho qala jenereithara ho e 'ngoe, le analyzer ho ea bobeli. Ho ne ho se na lintho tse makatsang mona, empa ha e le hantle tsohle li itšetlehile ka tšepe, e fokolang, e tla ba e thahasellisang haholoanyane mona.
Ho sebelisa data e fumanoeng ke sistimi ea ho beha leihlo (Zabbix)
tsp ha e na API efe kapa efe e baloang ke mochini joalo ka SNMP kapa e tšoanang. Melaetsa ea CC e tlameha ho kopanngoa bonyane motsotsoana oa 1 (ka tekanyo e phahameng ea tahlehelo ea pakete, ho ka ba le makholo / likete / mashome a likete ka motsotsoana, ho itšetlehile ka bitrate).
Kahoo, bakeng sa ho boloka tlhahisoleseling le ho taka li-graph bakeng sa liphoso tsa CC le bitrate le ho etsa likotsi tsa mofuta o itseng, ho ka ba le likhetho tse latelang:
- Hlahisa le ho kopanya (ka CC) tlhahiso ea tsp, i.e. e fetolela ho sebopeho se batloang.
- Qetella tsp ka boeona le / kapa li-plugins tsa processor bitrate_monitor le ho tsoela pele e le hore sephetho se fanoe ka mokhoa o baloang ke mochine o loketseng mokhoa oa ho shebella.
- Ngola kopo ea hau holim'a laeborari ea tsduck.
Ho totobetse hore khetho ea 1 ke eona e bonolo ka ho fetisisa mabapi le boiteko, haholo-holo ha ho nahanoa hore tsduck ka boeona e ngotsoe ka puo ea boemo bo tlaase (ka litekanyetso tsa morao-rao) (C ++)
Mohlala o bonolo oa bash parser + aggregator o bonts'itse hore molapong oa 10Mbps le tahlehelo ea pakete ea 50% (maemo a mabe ka ho fetesisa), ts'ebetso ea bash e sebelisitse makhetlo a 3-4 ho feta ts'ebetso ea tsp ka boeona. Boemo bona ha bo amohelehe. Ha e le hantle, karolo ea prototype ena e ka tlase
Li-noodle holimo
#!/usr/bin/env bash
missingPackets=0
ccErrorSeconds=0
regexMissPackets='^* (.+) - continuity:.*missing ([0-9]+) packets$'
missingPacketsTime=""
ip netns exec P tsp --realtime -t -I ip -b 8388608 "239.0.0.1:1234" -O drop -P bitrate_monitor -p 1 -t 1 -P continuity 2>&1 |
while read i
do
#line example:* 2019/12/28 23:41:14 - continuity: packet index: 6,078, PID: 0x0100, missing 5 packets
#line example 2: * 2019/12/28 23:55:11 - bitrate_monitor: 2019/12/28 23:55:11, TS bitrate: 4,272,864 bits/s
if [[ "$i" == *continuity:* ]]
then
if [[ "$i" =~ $regexMissPackets ]]
then
missingPacketsTimeNew="${BASH_REMATCH[1]}" #timestamp (seconds)
if [[ "$missingPacketsTime" != "$missingPacketsTimeNew" ]] #new second with CC error
then
((ccErrorSeconds += 1))
fi
missingPacketsTime=$missingPacketsTimeNew
packets=${BASH_REMATCH[2]} #TS missing packets
((missingPackets += packets))
fi
elif [[ "$i" == *bitrate_monitor:* ]]
then
: #...
fi
doneNtle le ho lieha ka mokhoa o sa amoheleheng, ha ho na likhoele tse tloaelehileng ho bash, mesebetsi ea bash ke lits'ebetso tse arohaneng mme ke ile ka tlameha ho ngola boleng ba li- missingPackets hang ka motsotsoana lehlakoreng la lehlakoreng (ha ke amohela melaetsa ea bitrate e tlang motsotsoana o mong le o mong). Ka lebaka leo, bash o ile a sala a le mong 'me ho ile ha etsoa qeto ea ho ngola wrapper (parser + aggregator) ka golang. Ts'ebeliso ea CPU ea khoutu e tšoanang ea golang ke makhetlo a 4-5 ho feta tsp tsp ka boeona. Ho potlaka ha sephutheloana ka lebaka la ho nkeloa sebaka ha bash ka golang ho ile ha fetoha ka makhetlo a ka bang 16 'me ka kakaretso sephetho se amoheleha (CPU ka holimo ho 25% boemong bo bobe ka ho fetisisa). The golang source file is located .
Matha wrapper
Ho qala wrapper, template e bonolo ka ho fetisisa ea tšebeletso ea systemd e entsoe (). Sephutheloana ka boeona se tlameha ho hlophisoa hore e be faele ea binary (ea ho haha tsduck-stat.go) e teng /opt/tsduck-stat/. Ho nahanoa hore u sebelisa golang ka tšehetso bakeng sa oache ea monotonic (>=1.9).
Ho theha mohlala oa ts'ebeletso, o hloka ho tsamaisa taelo ea systemctl [imeile e sirelelitsoe]:1234 ebe o matha le systemctl qala [imeile e sirelelitsoe]: 1234.
Ho sibolloa ho tsoa ho Zabbix
E le hore zabbix e tsebe ho sibolla lits'ebeletso tse sebetsang, e etsoa (discovery.sh), ka mokhoa o hlokahalang bakeng sa ho sibolloa ha Zabbix, ho nahanoa hore e sebakeng se le seng - ho /opt/tsduck-stat. Ho tsamaisa phumano ka zabbix-agent, o hloka ho eketsa bukeng ea tlhophiso ea zabbix-agent ho kenya paramethara ea mosebelisi.
Setšoantšo sa Zabbix
(tsduck_stat_template.xml) e na le molao oa autodiscover, prototypes tsa lintho, li-graph, le li-trigger.
Lethathamo le lekhutšoane la tlhahlobo (hantle, ho thoe'ng haeba motho e mong a etsa qeto ea ho e sebelisa)
- Etsa bonnete ba hore tsp ha e tlohele lipakete tlas'a maemo a "loketseng" (jenereithara le analyzer li kopantsoe ka ho toba), haeba ho na le marotholi, sheba serapa sa 2 kapa temana ea sehlooho tabeng ena.
- Etsa tokiso ea boholo ba sokete buffer (net.core.rmem_max=8388608).
- Kopanya tsduck-stat.go (ea haha tsduck-stat.go).
- Beha template ea tšebeletso ho /lib/systemd/system.
- Qala lits'ebeletso ka systemctl, hlahloba hore na li-counter li se li qalile ho hlaha (grep "" /dev/shm/tsduck-stat/*). Palo ea litšebeletso ho latela palo ea li-multicast streams. Mona o kanna oa hloka ho theha tsela ho sehlopha sa multicast, mohlomong tima rp_filter kapa ho theha tsela e eang mohloling oa ip.
- Matha discovery.sh, etsa bonnete ba hore e hlahisa json.
- Eketsa zabbix agent config, qala hape moemeli oa zabbix.
- Kenya template ho zabbix, e sebelise ho moamoheli ea ntseng a behiloe leihlo mme moemeli oa zabbix a kentsoe, ema metsotso e ka bang 5, bona hore na ho na le lintho tse ncha, li-graph le li-triggers.
sephetho
Bakeng sa mosebetsi oa ho lemoha tahlehelo ea pakete, e batla e lekana, bonyane e molemo ho feta ho se be le leihlo.
Ka sebele, "litahlehelo" tsa CC li ka etsahala ha ho kopanya likhechana tsa video (ho ea kamoo ke tsebang, ke kamoo ho kenngoa ho etsoang kateng litsing tsa TV tsa sebaka sa Russia Federation, ke hore ntle le ho tsosolosa k'hamphani ea CC), sena se tlameha ho hopoloa. Litharollo tsa beng ba matlo li qoba bothata bona ka ho fumana mangolo a SCTE-35 (haeba a kentsoe ke jenereithara ea molapo).
Mabapi le tlhahlobo ea boleng ba lipalangoang, ho na le khaello ea jitter monitoring (IAT). Thepa ea TV (e ka ba li-modulator kapa lisebelisoa tsa ho qetela) li na le litlhoko tsa paramethara ena 'me ha se kamehla ho ka khonehang ho kenya jitbuffer ho infinity. 'Me jitter e ka phaphamala ha lisebelisoa tse nang le li-buffers tse kholo li sebelisoa ha li tsamaea 'me QoS e sa hlophisoa kapa e sa hlophisoa hantle ho lekana ho fetisa sephethephethe sa nako ea nnete.
Source: www.habr.com