ProHoster > Π‘Π»ΠΎΠ³ > Kev tswj hwm > Siv TSDuck los saib xyuas IP (TS) ntws

Siv TSDuck los saib xyuas IP (TS) ntws

Niaj hnub no, muaj cov kev daws teeb meem npaj txhij (tus tswv) rau kev saib xyuas IP (TS) ntws, piv txwv li VB ΠΈ iQ, lawv muaj cov txheej txheem nplua nuj thiab feem ntau cov tswv lag luam loj cuam tshuam nrog cov kev pabcuam TV muaj cov kev daws teeb meem zoo sib xws. Kab lus no piav qhia txog kev daws teeb meem raws li qhov project qhib TSDuck, tsim los rau kev tswj tsawg kawg nkaus ntawm IP (TS) ntws siv CC (txuas ntxiv txee) txee thiab bitrate. Ib daim ntawv thov ua tau yog saib xyuas cov pob ntawv poob lossis tag nrho cov dej ntws los ntawm L2 channel (uas tsis tuaj yeem saib xyuas ib txwm, piv txwv li, los ntawm kev nyeem cov ntawv poob hauv cov kab).

Luv luv txog TSDuck

TSDuck yog qhib qhov chaw (2-Clause BSD daim ntawv tso cai) software (ib txheej ntawm cov khoom siv hluav taws xob thiab lub tsev qiv ntawv rau kev tsim koj tus kheej cov khoom siv hluav taws xob lossis plugins) rau kev tswj hwm TS kwj. Raws li kev tawm tswv yim, nws tuaj yeem ua haujlwm nrog IP (multicast / unicast), http, hls, dvb tuners, dektec dvb-asi demodulator, muaj TS kwj hluav taws xob sab hauv thiab nyeem los ntawm cov ntaub ntawv. Cov zis tuaj yeem kaw rau cov ntaub ntawv, IP (multicast / unicast), hls, dektec dvb-asi thiab HiDes modulators, players (mplayer, vlc, xine) thiab poob. Nruab nrab ntawm cov tswv yim thiab cov zis, koj tuaj yeem ua kom muaj ntau yam kev khiav tsheb khiav, piv txwv li, rov ua PIDs, ua scrambling / descrambling, txheeb xyuas CC cov txee, suav cov bitrate thiab lwm yam haujlwm raug rau TS kwj.

Hauv tsab xov xwm no, IP kwj (multicast) yuav siv los ua cov tswv yim, bitrate_monitor processors (los ntawm lub npe nws paub meej tias qhov no yog dab tsi) thiab txuas ntxiv (CC counter tsom xam) processors yuav raug siv. Yog tsis muaj teeb meem, koj tuaj yeem hloov IP multicast nrog lwm hom kev txhawb nqa los ntawm TSDuck.

Muaj official builds/packages TSDuck rau feem ntau OS tam sim no. Tsis muaj ib qho rau Debian, tab sis peb tau tswj hwm los sau lawv rau Debian 8 thiab Debian 10 yam tsis muaj teeb meem.

Tom ntej no, TSDuck version 3.19-1520 yog siv, Linux yog siv raws li OS (debian 10 tau siv los npaj cov tshuaj, CentOS 7 tau siv rau kev siv tiag tiag)

Npaj TSDuck thiab OS

Ua ntej saib xyuas cov dej ntws tiag tiag, koj yuav tsum xyuas kom tseeb tias TSDuck ua haujlwm kom raug thiab cov tee ntawd tsis tshwm sim ntawm daim npav network lossis OS (socket) qib. Qhov no yog qhov yuav tsum tau ua kom koj tsis tas yuav twv tom qab qhov twg cov tee tshwm sim - ntawm lub network lossis "hauv lub server." Koj tuaj yeem tshawb xyuas cov tee ntawm daim npav network nrog cov lus txib ethtool -S ethX, tuning yog ua los ntawm tib ethtool (feem ntau koj yuav tsum tau nce RX tsis (-G) thiab qee zaum lov tes taw qee qhov offloads (-K)). Raws li kev pom zoo dav dav, nws raug nquahu kom siv qhov chaw nres nkoj cais kom tau txais kev txheeb xyuas cov tsheb khiav, yog tias ua tau, qhov no yuav txo qis qhov tsis zoo vim tias qhov poob tshwm sim ib txhij ntawm qhov chaw nres nkoj analyzer vim muaj lwm cov tsheb khiav. Yog tias qhov no tsis tuaj yeem ua tau (koj siv lub khoos phis tawj me me / NUC nrog ib qho chaw nres nkoj), ces nws yog qhov tsim nyog los teeb tsa qhov tseem ceeb ntawm kev txheeb xyuas cov tsheb sib txuas nrog rau cov khoom seem ntawm cov cuab yeej ntsuas kev sib txuas. Hais txog ib puag ncig virtual, ntawm no koj yuav tsum tau ceev faj thiab nrhiav tau pob ntawv poob pib los ntawm qhov chaw nres nkoj lub cev thiab xaus nrog daim ntawv thov hauv lub tshuab virtual.

Tsim thiab tau txais kwj nyob hauv tus tswv tsev

Raws li thawj kauj ruam hauv kev npaj TSDuck, peb yuav tsim thiab tau txais kev khiav tsheb hauv ib tus tswv tsev siv netns.

Npaj ib puag ncig:

ip netns add P #создаём netns P, Π² Π½Ρ‘ΠΌ Π±ΡƒΠ΄Π΅Ρ‚ ΠΏΡ€ΠΎΠΈΡΡ…ΠΎΠ΄ΠΈΡ‚ΡŒ Π°Π½Π°Π»ΠΈΠ· Ρ‚Ρ€Π°Ρ„ΠΈΠΊΠ°
ip link add type veth #создаём veth-ΠΏΠ°Ρ€Ρƒ - veth0 оставляСм Π² netns ΠΏΠΎ ΡƒΠΌΠΎΠ»Ρ‡Π°Π½ΠΈΡŽ (Π² этот интСрфСйс Π±ΡƒΠ΄Π΅Ρ‚ Π³Π΅Π½Π΅Ρ€ΠΈΡ€ΠΎΠ²Π°Ρ‚ΡŒΡΡ Ρ‚Ρ€Π°Ρ„ΠΈΠΊ)
ip link set dev veth1 netns P #veth1 - ΠΏΠΎΠΌΠ΅Ρ‰Π°Π΅ΠΌ Π² netns P (Π½Π° этом интСрфСйсС Π±ΡƒΠ΄Π΅Ρ‚ ΠΏΡ€ΠΈΡ‘ΠΌ Ρ‚Ρ€Π°Ρ„ΠΈΠΊΠ°)
ip netns exec P ifconfig veth1 192.0.2.1/30 up #ΠΏΠΎΠ΄Π½ΠΈΠΌΠ°Π΅ΠΌ IP Π½Π° veth1, Π½Π΅ ΠΈΠΌΠ΅Π΅Ρ‚ значСния ΠΊΠ°ΠΊΠΎΠΉ ΠΈΠΌΠ΅Π½Π½ΠΎ
ip netns exec P ip ro add default via 192.0.2.2 #настраиваСм ΠΌΠ°Ρ€ΡˆΡ€ΡƒΡ‚ ΠΏΠΎ ΡƒΠΌΠΎΠ»Ρ‡Π°Π½ΠΈΡŽ Π²Π½ΡƒΡ‚Ρ€ΠΈ nents P
sysctl net.ipv6.conf.veth0.disable_ipv6=1 #ΠΎΡ‚ΠΊΠ»ΡŽΡ‡Π°Π΅ΠΌ IPv6 Π½Π° veth0 - это дСлаСтся для Ρ‚ΠΎΠ³ΠΎ, Ρ‡Ρ‚ΠΎΠ±Ρ‹ Π² счётчик TX Π½Π΅ ΠΏΠΎΠΏΠ°Π΄Π°Π» посторонний мусор
ifconfig veth0 up #ΠΏΠΎΠ΄Π½ΠΈΠΌΠ°Π΅ΠΌ интСрфСйс veth0
ip route add 239.0.0.1 dev veth0 #создаём ΠΌΠ°Ρ€ΡˆΡ€ΡƒΡ‚, Ρ‡Ρ‚ΠΎΠ±Ρ‹ ОБ направляла Ρ‚Ρ€Π°Ρ„ΠΈΠΊ ΠΊ 239.0.0.1 Π² сторону veth0

Ib puag ncig yog npaj. Peb pib lub tsheb analyzer:

ip netns exec P tsp --realtime -t 
 -I ip 239.0.0.1:1234 
 -P continuity 
 -P bitrate_monitor -p 1 -t 1 
 -O drop

qhov twg "-p 1 -t 1" txhais tau hais tias koj yuav tsum xam cov bitrate txhua ob thiab tso saib cov ntaub ntawv hais txog tus bitrate txhua thib ob
Peb tso lub tshuab hluav taws xob tsheb nrog qhov ceev ntawm 10 Mbit / s:

tsp -I craft 
 -P regulate -b 10000000 
 -O ip -p 7 -e --local-port 6000 239.0.0.1:1234

qhov twg "-p 7 -e" txhais tau tias koj yuav tsum tau ntim 7 TS packets rau hauv 1 IP pob ntawv thiab ua nws nyuaj (-e), i.e. ib txwm tos rau 7 TS pob ntawv los ntawm cov txheej txheem kawg ua ntej xa qhov tsim ntawm pob ntawv IP.

Tus kws tshuaj ntsuam xyuas pib nthuav tawm cov lus xav tau:

* 2020/01/03 14:55:44 - bitrate_monitor: 2020/01/03 14:55:44, TS bitrate: 9,970,016 bits/s
* 2020/01/03 14:55:45 - bitrate_monitor: 2020/01/03 14:55:45, TS bitrate: 10,022,656 bits/s
* 2020/01/03 14:55:46 - bitrate_monitor: 2020/01/03 14:55:46, TS bitrate: 9,980,544 bits/s

Tam sim no cia peb ntxiv qee qhov tee:

ip netns exec P iptables -I INPUT -d 239.0.0.1 -m statistic --mode random --probability 0.001 -j DROP

thiab cov lus zoo li no tshwm sim:

* 2020/01/03 14:57:11 - continuity: packet index: 80,745, PID: 0x0000, missing 7 packets
* 2020/01/03 14:57:11 - continuity: packet index: 83,342, PID: 0x0000, missing 7 packets

uas xav tau. Peb lov tes taw pob ntawv poob (ip netns exec P iptables -F) thiab sim ua kom lub tshuab hluav taws xob nce mus txog 100 Mbit / s. Cov kws tshuaj ntsuam xyuas qhia txog ib pawg ntawm CC yuam kev thiab txog 75 Mbit / s es tsis txhob 100. Peb tab tom sim xyuas seb leej twg yuav liam - lub tshuab hluav taws xob tsis khaws lossis qhov teeb meem tsis nyob hauv nws, ua qhov no peb pib tsim ib qho Cov naj npawb ruaj khov (700000 TS pob ntawv = 100000 IP pob ntawv):

# ifconfig veth0 | grep TX
       TX packets 151825460  bytes 205725459268 (191.5 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
# tsp -I craft -c 700000 -P regulate -b 100000000 -P count -O ip -p 7 -e --local-port 6000 239.0.0.1:1234
* count: PID    0 (0x0000):    700,000 packets
# ifconfig veth0 | grep TX
        TX packets 151925460  bytes 205861259268 (191.7 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Raws li koj tuaj yeem pom, raws nraim 100000 IP pob ntawv tau tsim tawm (151925460-151825460). Yog li peb txheeb xyuas seb puas muaj dab tsi tshwm sim nrog lub ntsuas ntsuas, ua qhov no peb xyuas nrog RX txee ntawm veth1, nws nruj me ntsis sib npaug rau TX txee ntawm veth0, ces peb saib dab tsi tshwm sim ntawm lub qhov (socket) qib:

# ip netns exec P cat /proc/net/udp                                                                                                           
  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode ref pointer drops             
  133: 010000EF:04D2 00000000:0000 07 00000000:00000000 00:00000000 00000000     0        0 72338 2 00000000e0a441df 24355

Ntawm no koj tuaj yeem pom cov naj npawb ntawm cov tee = 24355. Hauv TS pob ntawv no yog 170485 lossis 24.36% ntawm 700000, yog li peb pom tias tib 25% ntawm qhov poob bitrate yog tee hauv UDP qhov (socket). Kev poob rau ntawm UDP lub qhov (socket) feem ntau tshwm sim vim qhov tsis muaj qhov tsis muaj, cia peb pom dab tsi lub qhov (socket) tsis muaj qhov loj me thiab qhov siab tshaj qhov tsis sib xws yog:

# sysctl net.core.rmem_default
net.core.rmem_default = 212992
# sysctl net.core.rmem_max
net.core.rmem_max = 212992

Yog li, yog tias daim ntawv thov tsis qhia meej txog qhov loj me, qhov (sockets) raug tsim nrog 208 KB tsis, tab sis yog tias lawv thov ntau dua, lawv tseem yuav tsis tau txais qhov lawv thov. Txij li thaum nyob rau hauv tsp koj tuaj yeem teeb tsa qhov loj me rau IP cov tswv yim (--buffer-loj), peb yuav tsis kov lub qhov (socket) qhov loj me, tab sis tsuas yog yuav teeb tsa qhov siab tshaj plaws qhov tsis muaj qhov loj me thiab qhia qhov tsis pom qhov loj me me los ntawm cov lus sib cav:

sysctl net.core.rmem_max=8388608
ip netns exec P tsp --realtime -t -I ip 239.0.0.1:1234 -b 8388608 -P continuity -P bitrate_monitor -p 1 -t 1 -O drop

Nrog rau qhov kev hloov kho ntawm lub qhov (socket) tsis, qhov kev tshaj tawm tam sim no yog kwv yees li 100Mbps, tsis muaj CC yuam kev.

Raws li CPU noj los ntawm daim ntawv thov tsp nws tus kheej. Hais txog ib tus tub ntxhais i5-4260U CPU @ 1.40GHz, txhawm rau txheeb xyuas 10Mbit / s ntws, 3-4% ntawm CPU yuav tsum, 100Mbit / s - 25%, 200Mbit / s - 46%. Thaum teeb tsa % pob ntawv poob, CPU load xyaum tsis nce (tab sis yuav txo qis).

Ntawm cov khoom tsim tau ntau dua, nws tuaj yeem tsim thiab txheeb xyuas cov kwj ntawm ntau dua 1Gb / s yam tsis muaj teeb meem.

Kev sim ntawm daim npav network tiag

Tom qab kev sim ntawm ib khub veth, koj yuav tsum coj ob lub tswv yim lossis ob qhov chaw nres nkoj ntawm ib tus tswv tsev, txuas cov chaw nres nkoj mus rau ib leeg, khiav lub tshuab hluav taws xob ntawm ib qho, thiab lub ntsuas ntsuas ntawm qhov thib ob. Tsis muaj qhov xav tsis thoob ntawm no, tab sis qhov tseeb nws txhua yam nyob ntawm qhov kho vajtse, qhov tsis muaj zog, qhov nthuav ntau nws yuav nyob ntawm no.

Siv cov ntaub ntawv tau txais los ntawm kev saib xyuas (Zabbix)

tsp tsis muaj lub tshuab nyeem tau API zoo li SNMP lossis zoo sib xws. CC cov lus yuav tsum tau sib sau ua ke tsawg kawg yog 1 thib ob ntawm ib lub sijhawm (nrog rau feem pua ​​​​ntawm cov pob ntawv poob, tuaj yeem muaj ntau pua / txhiab / kaum tawm txhiab ib ob, nyob ntawm tus nqi qis).

Yog li, txhawm rau txuag cov ntaub ntawv thiab kos duab rau CC yuam kev thiab bitrate thiab ua qee yam xwm txheej ntxiv, tej zaum yuav muaj cov kev xaiv hauv qab no:

  1. Parse thiab aggregate (los ntawm CC) lub tsp tso zis, i.e. hloov nws mus rau hauv daim ntawv xav tau.
  2. Ntxiv cov tsp nws tus kheej thiab / lossis cov bitrate_monitor thiab txuas ntxiv processor plugins kom cov txiaj ntsig tau tso tawm hauv daim ntawv tshuab nyeem tau tsim nyog rau kev saib xyuas.
  3. Sau koj daim ntawv thov rau saum lub tsev qiv ntawv tsduck.

Pom tseeb, hais txog cov nqi zog, kev xaiv 1 yog qhov yooj yim tshaj plaws, tshwj xeeb tshaj yog xav tias tsduck nws tus kheej tau sau rau hauv qib qis (los ntawm cov qauv niaj hnub) hom lus (C ++)

Ib qho piv txwv yooj yim ntawm parser + aggregator hauv bash tau pom tias ntawm qhov ntws ntawm 10 Mbit / s thiab 50% pob ntawv poob (qhov xwm txheej phem tshaj), cov txheej txheem bash tau noj 3-4 zaug ntau dua CPU dua li cov txheej txheem tsp nws tus kheej. Qhov xwm txheej no tsis tsim nyog. Qhov tseeb, ib daim ntawm cov qauv no yog hauv qab no

Noodles rau saum

#!/usr/bin/env bash

missingPackets=0
ccErrorSeconds=0
regexMissPackets='^* (.+) - continuity:.*missing ([0-9]+) packets$'
missingPacketsTime=""

ip netns exec P tsp --realtime -t -I ip -b 8388608 "239.0.0.1:1234" -O drop -P bitrate_monitor -p 1 -t 1  -P continuity 2>&1 | 
while read i
do
    #line example:* 2019/12/28 23:41:14 - continuity: packet index: 6,078, PID: 0x0100, missing 5 packets
    #line example 2: * 2019/12/28 23:55:11 - bitrate_monitor: 2019/12/28 23:55:11, TS bitrate: 4,272,864 bits/s
    if [[ "$i" == *continuity:* ]] 
    then
        if [[ "$i" =~ $regexMissPackets ]]
        then
            missingPacketsTimeNew="${BASH_REMATCH[1]}" #timestamp (seconds)
            if [[ "$missingPacketsTime" != "$missingPacketsTimeNew" ]] #new second with CC error
            then
                ((ccErrorSeconds += 1))
            fi
            missingPacketsTime=$missingPacketsTimeNew
            packets=${BASH_REMATCH[2]} #TS missing packets
            ((missingPackets += packets))
        fi
    elif [[ "$i" == *bitrate_monitor:* ]]
    then
        : #...
    fi
done

Ntxiv rau qhov tseeb tias qhov no ua haujlwm qeeb qeeb, tsis muaj ib txwm xov hauv bash, bash txoj haujlwm yog cov txheej txheem ywj pheej thiab kuv yuav tsum tau sau tus nqi plojPackets ib zaug thib ob ntawm qhov tshwm sim (thaum tau txais cov lus bitrate uas tuaj txhua ob). Yog li ntawd, bash tau nyob ib leeg thiab nws tau txiav txim siab los sau ib daim ntawv qhwv (parser + aggregator) hauv golang. CPU noj ntawm golang code zoo sib xws yog 4-5 zaug tsawg dua li cov txheej txheem tsp nws tus kheej. Kev nrawm ntawm lub wrapper los ntawm kev hloov bash nrog golang yog kwv yees li 16 lub sij hawm thiab tag nrho cov txiaj ntsig tau txais (CPU nyiaj siv ua haujlwm los ntawm 25% hauv qhov phem tshaj). Cov ntaub ntawv golang qhov chaw nyob no.

Launching lub wrapper

Txhawm rau tso lub wrapper, ib qho kev pabcuam yooj yim rau systemd tau tsim (no). Nws yog xav tias lub wrapper nws tus kheej tau muab tso ua ke rau hauv cov ntaub ntawv binary (mus tsim tsduck-stat.go), nyob rau hauv /opt/tsduck-stat/. Nws yog assumed tias golang yog siv nrog monotonic moos txhawb (>= 1.9).

Txhawm rau tsim ib qho kev pabcuam koj yuav tsum tau khiav qhov systemctl pab kom hais kom ua [email tiv thaiv]: 1234, ces khiav nrog systemctl pib [email tiv thaiv]: 1234.

Discovery los ntawm Zabbix

Yog li ntawd zabbix tuaj yeem tshawb pom cov kev pabcuam khiav, pawg generator (discovery.sh), nyob rau hauv hom xav tau rau Zabbix discovery, nws yog assumed tias nws yog nyob rau hauv tib qhov chaw - nyob rau hauv /opt/tsduck-stat. Txhawm rau khiav nrhiav pom ntawm zabbix-tus neeg sawv cev, koj yuav tsum tau ntxiv .conf cov ntaub ntawv mus rau cov npe nrog zabbix-tus neeg sawv cev configurations ntxiv ib tus neeg siv tsis tau.

Zabix template

Tsim qauv (tsduck_stat_template.xml) muaj txoj cai autodiscovery, lub ntsiab lus, daim duab, thiab cov qauv tsim.

Daim ntawv teev npe luv (yuav ua li cas yog tias ib tus neeg txiav txim siab siv nws)

  1. Nco ntsoov tias tsp tsis tso cov pob ntawv raws li "zoo tagnrho" cov xwm txheej (lub tshuab hluav taws xob thiab cov ntsuas ntsuas tau txuas ncaj qha), yog tias muaj cov tee, saib cov ntsiab lus 2 lossis cov ntawv nyeem ntawm cov ntsiab lus no.
  2. Ua kom kho qhov siab tshaj plaws qhov tsis muaj (net.core.rmem_max=8388608).
  3. Sau tsduck-stat.go (mus tsim tsduck-stat.go).
  4. Muab qhov kev pabcuam template rau hauv /lib/systemd/system.
  5. Pib cov kev pabcuam siv systemctl, xyuas tias cov txee tau pib tshwm sim (grep "" /dev/shm/tsduck-stat/*). Tus naj npawb ntawm cov kev pabcuam los ntawm tus lej ntawm cov kwj tawm multicast. Ntawm no koj yuav tsum tau tsim ib txoj hauv kev rau pawg multicast, tej zaum lov tes taw rp_filter lossis tsim ib txoj hauv kev mus rau qhov chaw ip.
  6. Khiav discovery.sh, xyuas kom nws generates json.
  7. Muab tus neeg saib xyuas zabbix config, rov pib tus neeg saib xyuas zabbix.
  8. Upload lub template rau zabbix, siv nws rau tus tswv tsev uas kev soj ntsuam yog nqa tawm thiab zabbix-tus neeg saib xyuas raug teeb tsa, tos li 5 feeb, pom tias cov ntaub ntawv tshiab, cov duab thiab cov cim tshwm sim.

tshwm sim

Siv TSDuck los saib xyuas IP (TS) ntws

Rau txoj haujlwm ntawm kev txheeb xyuas cov pob ntawv poob, nws yuav luag txaus, yam tsawg kawg nws zoo dua tsis muaj kev saib xyuas.

Qhov tseeb, CC "kev poob" tuaj yeem tshwm sim thaum splicing video fragments (raws li kuv paub, qhov no yog li cas inserts yog ua nyob rau hauv lub zos TV chaw nyob rau hauv Lavxias teb sab Federation, piv txwv li tsis reccalculating lub CC txee), qhov no yuav tsum nco ntsoov. Hauv cov kev daws teeb meem, qhov teeb meem no yog ib feem los ntawm kev kuaj xyuas SCTE-35 cim npe (yog tias lawv ntxiv los ntawm lub tshuab hluav taws xob kwj).

Los ntawm qhov pom ntawm kev saib xyuas kev thauj mus los zoo, kev saib xyuas kev ntxhov siab (IAT) tsis txaus, vim Cov khoom siv TV (txawm yog cov khoom siv modulators lossis cov khoom siv kawg) muaj qhov yuav tsum tau ua rau qhov ntsuas no thiab nws tsis yog ib txwm ua tau kom ua rau lub jitbuffer tsis tas li. Thiab jitter tuaj yeem ntab thaum kev thauj mus los siv cov cuab yeej siv nrog cov buffers loj thiab QoS tsis teeb tsa lossis tsis tau teeb tsa zoo txaus los xa cov tsheb khiav hauv lub sijhawm.

Tau qhov twg los: www.hab.com

Ntxiv ib saib