{"id":75722,"date":"2020-03-28T07:42:04","date_gmt":"2020-03-28T05:42:04","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/kak-my-uskorili-kodirovanie-video-v-vosem-raz"},"modified":"2020-03-28T07:42:04","modified_gmt":"2020-03-28T05:42:04","slug":"kak-my-uskorili-kodirovanie-video-v-vosem-raz","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-my-uskorili-kodirovanie-video-v-vosem-raz","title":{"rendered":"Si e p\u00ebrshpejtuam kodimin e videove n\u00eb tet\u00eb her\u00eb","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Si
\n
\n\u00c7do dit\u00eb, miliona shikues shohin video n\u00eb internet. Por p\u00ebr t\u00eb b\u00ebr\u00eb videon t\u00eb aksesueshme, ajo duhet jo vet\u00ebm t\u00eb ngarkohet n\u00eb server, por edhe t\u00eb p\u00ebrpunoj\u00eb. Sa m\u00eb shpejt t\u00eb ndodh\u00eb kjo, aq m\u00eb mir\u00eb \u00ebsht\u00eb p\u00ebr sh\u00ebrbimin dhe p\u00ebrdoruesit e tij. <\/p>\n

Un\u00eb quhem Askar Kamalov, nj\u00eb vit m\u00eb par\u00eb iu bashkova ekipit t\u00eb teknologjive video t\u00eb Yandex. Sot do t'u tregoj lexuesve t\u00eb Habr\u00ebs se si arrit\u00ebm t\u00eb p\u00ebrshpejtonim d\u00ebrgimin e videove te p\u00ebrdoruesit p\u00ebrmes paralelizimit t\u00eb procesit t\u00eb kodifikimit. <\/p>\n

Ky postim n\u00eb radh\u00eb t\u00eb par\u00eb do t\u00eb jet\u00eb i interesant p\u00ebr ata q\u00eb nuk kan\u00eb menduar ndonj\u00ebher\u00eb se \u00e7far\u00eb ndodh n\u00ebn kapak\u00ebt e sh\u00ebrbimeve video. N\u00eb komentet mund t\u00eb b\u00ebni pyetje dhe t\u00eb sugjeroni tema p\u00ebr postime t\u00eb ardhshme.
\n<\/a><\/noindex>
\nPak fjal\u00eb mbi detyr\u00ebn vet\u00eb. Yandex jo vet\u00ebm q\u00eb ndihmon p\u00ebr t\u00eb k\u00ebrkuar video n\u00eb faqe t\u00eb tjera, por gjithashtu ruan video p\u00ebr sh\u00ebrbimet e tij. Qoft\u00eb nj\u00eb program autor\u00ebsh, nj\u00eb ndeshje sportive n\u00eb eter, nj\u00eb film n\u00eb KinoPoisk, apo klipet n\u00eb Zen dhe Lajme \u2014 gjith\u00e7ka ngarkohet n\u00eb server\u00ebt tan\u00eb. P\u00ebr t\u00eb par\u00eb videon, ajo duhet p\u00ebrgatitur: t\u00eb konvertohet n\u00eb formatin e nevojsh\u00ebm, t\u00eb krijohet nj\u00eb miniatur\u00eb apo madje t\u00eb kaloj\u00eb p\u00ebrmes teknologjis\u00eb DeepHD<\/a><\/noindex>. Nj\u00eb skedar i pap\u00ebrgatitur thjesht z\u00eb vend. Dhe nuk b\u00ebhet fjal\u00eb vet\u00ebm p\u00ebr p\u00ebrdorimin optimal t\u00eb harduerit, por gjithashtu p\u00ebr shpejt\u00ebsin\u00eb e d\u00ebrgimit t\u00eb p\u00ebrmbajtjes te p\u00ebrdoruesit. P\u00ebr shembull: nj\u00eb regjistrim me momentin vendimtar t\u00eb nj\u00eb ndeshjeje hokej mund t\u00eb k\u00ebrkohet n\u00eb k\u00ebrkimin brenda nj\u00eb minute pas ngjarjes.<\/p>\n

Kodimi i radh\u00ebs<\/h3>\n

\nPra, lumturia e p\u00ebrdoruesit varet shum\u00eb nga sa shpejt b\u00ebhet video e aksesueshme. Dhe kjo p\u00ebrcaktohet kryesisht nga shpejt\u00ebsia e transkodimit. Kur nuk ka k\u00ebrkesa t\u00eb rrepta p\u00ebr shpejt\u00ebsin\u00eb e publikimit t\u00eb videos, nuk ka asnj\u00eb problem. Ne marrim nj\u00eb skedar t\u00eb vet\u00ebm, t\u00eb pandar\u00eb, e konvertojm\u00eb at\u00eb dhe e ngrem\u00eb. N\u00eb fillim ne punonim k\u00ebshtu:<\/p>\n

\"Si
\n
\nKlienti ngarkon videon n\u00eb ruajtje, komponenti Analyzer mbledh meta-informacione dhe e d\u00ebrgon videon p\u00ebr konvertim n\u00eb komponentin Worker. T\u00eb gjitha hapat kryhen n\u00eb radh\u00eb. Edhe pse mund t\u00eb ket\u00eb shum\u00eb server\u00eb p\u00ebr kodim, vet\u00ebm nj\u00eb \u00ebsht\u00eb i angazhuar n\u00eb p\u00ebrpunimin e videos specifike. Nj\u00eb skem\u00eb e thjesht\u00eb, transparente. N\u00eb k\u00ebt\u00eb pik\u00eb mbarojn\u00eb avantazhet e saj. Kjo skem\u00eb mund t\u00eb shkall\u00ebzohet vet\u00ebm verticalisht (duke bler\u00eb server\u00eb m\u00eb t\u00eb fuqish\u00ebm). <\/p>\n

Kodimi i radh\u00ebs me rezultat t\u00eb nd\u00ebrmjet\u00ebm<\/h3>\n

\nP\u00ebr t\u00eb zbutur pritjen e mundimshme, industria ka shpikur nj\u00eb variant t\u00eb kodimit t\u00eb shpejt\u00eb. Nj\u00eb em\u00ebr mashtrues, sepse n\u00eb t\u00eb v\u00ebrtet\u00eb kodimi i plot\u00eb ndodh n\u00eb radh\u00eb dhe po aq ngadal\u00eb. Por p\u00ebrve\u00e7 k\u00ebsaj, me nj\u00eb rezultat t\u00eb nd\u00ebrmjet\u00ebm. Ideja \u00ebsht\u00eb k\u00ebshtu: sa m\u00eb shpejt t\u00eb p\u00ebrgatitet dhe t\u00eb ngarkohet nj\u00eb version video merezolucion t\u00eb ul\u00ebt, dhe pastaj \u2014 versione me rezolucione m\u00eb t\u00eb larta. <\/p>\n

N\u00eb nj\u00eb an\u00eb, video b\u00ebhet m\u00eb shpejt e aksesueshme. Dhe kjo \u00ebsht\u00eb e dobishme p\u00ebr ngjarjet e r\u00ebnd\u00ebsishme. Por nga ana tjet\u00ebr, skena rezulton t\u00eb jet\u00eb e paqart\u00eb, dhe kjo e nervozon shikuesit.<\/p>\n

Pra, nuk duhet vet\u00ebm t\u00eb p\u00ebrpunoni video shpejt, por edhe t\u00eb ruani cil\u00ebsin\u00eb e saj. Kjo \u00ebsht\u00eb ajo q\u00eb p\u00ebrdoruesit presin nga sh\u00ebrbimi video tani. Mund t\u00eb duket se mjafton t\u00eb bleni server\u00ebt m\u00eb t\u00eb fuqish\u00ebm (dhe t'i p\u00ebrmir\u00ebsoni ato rregullisht). Por kjo \u00ebsht\u00eb nj\u00eb rrug\u00eb pa dalje, sepse gjithmon\u00eb do t\u00eb ket\u00eb video q\u00eb do t\u00eb ngadal\u00ebsojn\u00eb edhe harduerin m\u00eb t\u00eb fuqish\u00ebm. <\/p>\n

Kodimi paralel<\/h3>\n

\nM\u00eb efikase \u00ebsht\u00eb t\u00eb ndash nj\u00eb detyr\u00eb t\u00eb komplikuar n\u00eb shum\u00eb detyra m\u00eb pak t\u00eb komplikuara dhe t'i zgjidh\u00ebsh ato paralelisht n\u00eb servera t\u00eb ndrysh\u00ebm. Nj\u00eb lloj MapReduce p\u00ebr video. N\u00eb k\u00ebt\u00eb rast, ne nuk p\u00ebrballemi me performanc\u00ebn e nj\u00eb serveri dhe mund t\u00eb shkall\u00ebzojm\u00eb horizontalisht (duke shtuar makineri t\u00eb reja). <\/p>\n

P\u00ebr ta th\u00ebn\u00eb ndryshe, ideja p\u00ebr t\u00eb ndar\u00eb videon n\u00eb copa t\u00eb vogla, p\u00ebr ta p\u00ebrpunuar ato paralelisht dhe p\u00ebr t'i ngjitur, nuk \u00ebsht\u00eb ndonj\u00eb sekret. Ju mund t\u00eb gjeni shum\u00eb p\u00ebrmendje t\u00eb k\u00ebtij qasje (p\u00ebr shembull, n\u00eb Habr sugjeroj nj\u00eb postim mbi projektin DistVIDc<\/a><\/noindex>). Por kjo nuk e b\u00ebn t\u00eb leht\u00eb, sepse nuk mund t\u00eb merrni thjesht nj\u00eb zgjidhje t\u00eb gatshme dhe ta integroni. K\u00ebrkohet nj\u00eb adaptim p\u00ebr infrastruktur\u00ebn ton\u00eb, videon ton\u00eb dhe madje edhe ngarkes\u00ebn ton\u00eb. N\u00eb p\u00ebrgjith\u00ebsi, \u00ebsht\u00eb m\u00eb e leht\u00eb t\u00eb shkruash vet\u00eb. <\/p>\n

Pra, n\u00eb arkitektur\u00ebn e re, blloku monolitik Worker me kodimin e radh\u00ebs e kemi ndar\u00eb n\u00eb mikrosh\u00ebrbime Segmenter, Tcoder, Combiner.<\/p>\n

\"Si
\n<\/p>\n

    \n
  1. Segmenter ndan videon n\u00eb fragmente t\u00eb p\u00ebraf\u00ebrsisht 10 sekondash. Fragmente p\u00ebrb\u00ebhen nga nj\u00eb ose m\u00eb shum\u00eb GOP (group of pictures<\/a><\/noindex>). \u00c7do GOP \u00ebsht\u00eb e pavarur dhe kodifikohet ve\u00e7mas, k\u00ebshtu q\u00eb mund t\u00eb dekodifikohet pa u referuar n\u00eb kadrin nga GOP t\u00eb tjera. Pra, fragmentet mund t\u00eb luajn\u00eb pavar\u00ebsisht nga nj\u00ebri-tjetri. Kjo segmentim zvog\u00eblon vones\u00ebn, duke lejuar q\u00eb p\u00ebrpunimi t\u00eb filloj\u00eb m\u00eb her\u00ebt.<\/li>\n
  2. Tcoder p\u00ebrpunon \u00e7do fragment. Ai merr detyr\u00ebn nga rradha, shkarkon fragmentin nga depoja, kodon n\u00eb rezoluta t\u00eb ndryshme (mos harroni, q\u00eb playeri mund t\u00eb zgjedh versionin n\u00eb baz\u00eb t\u00eb shpejt\u00ebsis\u00eb s\u00eb lidhjes), pastaj e rivendos rezultatin prap\u00eb n\u00eb depo dhe e sh\u00ebnon fragmentin si t\u00eb p\u00ebrpunuar n\u00eb baz\u00ebn e t\u00eb dh\u00ebnave. Pasi t\u00eb ket\u00eb p\u00ebrpunuar t\u00eb gjith\u00eb fragmentet, Tcoder d\u00ebrgon detyr\u00ebn p\u00ebr formimin e rezultateve p\u00ebr komponentin e ardhsh\u00ebm.<\/li>\n
  3. Combiner bashkon rezultatet: shkarkon t\u00eb gjith\u00eb fragmentet e realizuara nga Tcoder, formon rrjedha p\u00ebr rezoluta t\u00eb ndryshme.<\/li>\n<\/ol>\n

    \nDisa fjal\u00eb p\u00ebr z\u00ebrin. Kodiku m\u00eb popullor t\u00eb audios, AAC, ka nj\u00eb disavantazh t\u00eb pad\u00ebshiruar. Nse kodifikoni fragmentet ve\u00e7mas, ato nuk mund t\u00eb bashkohen pa probleme. Kalimet do t\u00eb jen\u00eb t\u00eb dukshme. Video kodik\u00ebve nuk kan\u00eb k\u00ebt\u00eb problem. Teoretikisht, mund t\u00eb k\u00ebrkohet nj\u00eb zgjidhje teknike m\u00eb komplekse, por kjo loj\u00eb p\u00ebr momentin nuk ia vlen (audio peson ndjesh\u00ebm m\u00eb pak se video). Prandaj, ne kodojm\u00eb video vet\u00ebm paralelisht, nd\u00ebrsa shtegu audio p\u00ebrpunohen n\u00eb m\u00ebnyr\u00eb t\u00eb plot\u00eb.<\/p>\n

    Rezultatet<\/h3>\n

    \nFal\u00eb p\u00ebrpunimit paralel t\u00eb videos, ne kemi reduktuar ndjesh\u00ebm vones\u00ebn midis ngarkes\u00ebs s\u00eb videos tek ne dhe disponueshm\u00ebris\u00eb p\u00ebr p\u00ebrdoruesit. P\u00ebr shembull, m\u00eb par\u00eb, krijimi i disa versioneve t\u00eb plota me cil\u00ebsi t\u00eb ndryshme p\u00ebr nj\u00eb film FullHD q\u00eb zgjaste nj\u00eb or\u00eb e gjysm\u00eb mund t\u00eb merrte deri n\u00eb dy or\u00eb. Tani e gjitha kjo zgjat 15 minuta. P\u00ebr m\u00eb tep\u00ebr, me p\u00ebrpunimin paralel, ne krijojm\u00eb nj\u00eb version n\u00eb rezolucion t\u00eb lart\u00eb edhe m\u00eb shpejt se versionin n\u00eb rezolucion t\u00eb ul\u00ebt me qasjen e vjet\u00ebr me rezultatet e p\u00ebrkohshme. <\/p>\n

    Dhe di\u00e7ka tjet\u00ebr. Me qasjen e vjet\u00ebr, ndoshta nuk kishim mjaft server\u00eb, ose ata ishin t\u00eb papun\u00eb pa detyra. Kodifikimi paralel lejon rritjen e p\u00ebrqindjes s\u00eb shfryt\u00ebzimit t\u00eb pajisjeve. Tani klasteri yn\u00eb prej m\u00eb shum\u00eb se nj\u00eb mij\u00eb server\u00ebve \u00ebsht\u00eb gjithmon\u00eb i angazhuar me di\u00e7ka. <\/p>\n

    N\u00eb t\u00eb v\u00ebrtet\u00eb, akoma ka shum\u00eb p\u00ebr t\u00eb arritur. P\u00ebr shembull, ne mund t\u00eb kursejm\u00eb ndjesh\u00ebm koh\u00ebn n\u00ebse fillojm\u00eb t\u00eb p\u00ebrpunojm\u00eb fragmentet e videos para se ajo t\u00eb arrij\u00eb tek ne plot\u00ebsisht. Si\u00e7 thon\u00eb, p\u00ebrpara \u00ebsht\u00eb m\u00eb shum\u00eb. <\/p>\n

    Shkruani n\u00eb komente p\u00ebr \u00e7far\u00eb detyrash n\u00eb fush\u00ebn e p\u00ebrpunimit t\u00eb videos do t\u00eb d\u00ebshironit t\u00eb lexonit. <\/p>\n

    Linket e dobishme mbi p\u00ebrvoj\u00ebn e koleg\u00ebve n\u00eb industri<\/h3>\n

    <\/p>\n