Chikamu chekutanga chinotsanangura chishuwo chakaoma chekugadzirisa mavhidhiyo emhuri ekare uye kuaputsa kuita zvikamu zvakasiyana.. Mushure mekugadzirisa zvimedu zvese, ndaida kuronga kuona kwavo pamhepo zviri nyore sepaYouTube. Sezvo idzi dziri ndangariro dzemhuri, hadzigone kutumirwa paYouTube pachayo. Isu tinoda imwe yakavanzika yekutambira iyo iri nyore uye yakachengeteka.

Danho rechitatu: Kutsikisa

ClipBucket, yakavhurika sosi YouTube clone inogona kuisirwa pane yako sevha

Chinhu chekutanga ndakaedza ClipBucket, iyo inozvidaidza kuti yakavhurika-sosi YouTube clone yaunogona kuisa pane yako server.

Zvinoshamisa kuti ClipBucket haina chero mirairo yekuisa. Thanks to kuvatungamiri vekunze я otomatiki maitiro ekuisa nerubatsiro Anable, server configuration management tool.

Chikamu chekuomerwa ndechekuti ClipBucket yekuisa zvinyorwa zvakaputswa zvachose. Panguva iyoyo ini akashanda paGoogle uye zvinoenderana nemamiriro echibvumirano ini ndakanga ndisina kodzero yekupa kune yakavhurika sosi YouTube clone, asi ini akaisa mushumo webug, kubva pairi kururamisa kunodiwa kwaigona kuitwa nyore nyore. Mwedzi yakapera, uye vakanga vasati vanzwisisa kuti dambudziko raiva rei. Pane kudaro vakawedzera zvose kupfuura bugs mukuburitswa kwese.

ClipBucket yakashanda pamuenzaniso wekubvunza-vakaburitsa kodhi yavo mahara uye kubhadhariswa rubatsiro rwekutumira. Zvakabva zvajeka kwandiri zvishoma nezvishoma kuti kambani inoita mari kubva kune inobhadharwa rutsigiro pamwe haifarire kuti vatengi vaise chigadzirwa ivo pachavo.

MediaGoblin, imwe nzira yemazuva ano

Mushure memwedzi yekushungurudzika neClipBucket, ndakatarisa mune sarudzo dziripo ndikawana MediaGoblin.

MediaGoblin ipuratifomu yekugovana midhiya isina Indaneti

MediaGoblin ine zvakawanda zvakanaka. Kusiyana neClipBucket mune isingataridzike PHP, MediaGoblin yakanyorwa muPython, mutauro wandine ruzivo rwakawanda rwekunyora kodhi mukati. Eat command line interface, iyo inokutendera kuti ugone kurodha vhidhiyo nyore nyore. Kunyanya kukosha, MediaGoblin inouya nayo Docker mufananidzo, iyo inobvisa chero matambudziko ekuisa.

Docker tekinoroji inogadzira nzvimbo inozvimiririra yekuti application ishande chero kupi. Ini ndinoshandisa Docker mukati akawanda emapurojekiti ake.

Iyo inokatyamadza kuomerwa kwe-dockerizing MediaGoblin

Ndakafunga kuti kuisa iyo MediaGoblin docker mufananidzo raizova basa diki. Zvakanaka, hazvina kunyatsoshanda nenzira iyoyo.

Mufananidzo wapera wanga usina mabasa maviri anodiwa:

• Kusimbiswa
  • MediaGoblin inogadzira yeruzhinji midhiya portal nekusarudzika, uye ini ndaida nzira yekurambidza kupinda kune vekunze.
• Transcoding
  • Pese paunodhawunirodha vhidhiyo, MediaGoblin inoedza kuichinjisa kuti inyatso kutenderera. Kana vhidhiyo yakatanga kugadzirira kutenderera, transcoding inodzikisira mhando.
  • MediaGoblin inopa kudzima transcoding kuburikidza nekugadzirisa sarudzo, asi izvi hazvigone kuitwa mune iripo Docker mufananidzo.

Zvakanaka, hapana dambudziko. Mufananidzo weDocker unouya nawo open source, saka zvinogoneka uzvivake iwe pachako.

Nehurombo, iyo Docker mufananidzo hauchavakwa kubva kune yazvino. MediaGoblin repository. Ndakaedza kuiwiriranisa neshanduro kubva kune yekupedzisira yakabudirira kuvaka, asi izvo zvakare zvakatadza. Kunyangwe ini ndakashandisa iyo chaiyo kodhi kodhi, MediaGoblin yekunze inotsamira yakachinja, ichipwanya chivakwa. Mushure memaawa akawanda, ndakamhanya kuburikidza ne10-15 miniti yeMediaGoblin yekuvaka maitiro kakawanda uyezve kusvika yakazoshanda.

Mwedzi mishomanana gare gare chinhu chimwe chetecho chakaitika. Pakazara, iyo MediaGoblin inotsamira cheni yakapwanya kuvaka kwangu kakawanda mumakore akati wandei apfuura, uye nguva yekupedzisira pazvakaitika ndichiri kunyora chinyorwa ichi. Ndakazoburitsa yako forogo yeMediaGoblin c hard-coded dependencies uye neshanduro dzakanyatsotsanangurwa dzemaraibhurari. Mune mamwe mazwi, pachinzvimbo chekupokana kuti MediaGoblin inoshanda nechero vhezheni celery > = 3.0, ini ndakaisa yakatarwa vhezheni kutsamira celery 4.2.1nekuti ndakaedza MediaGoblin neiyi vhezheni. Zvinotaridzika sechigadzirwa chinoda reproducible kuvaka mechanism, asi handisati ndazviita.

Zvakadaro, mushure memaawa mazhinji ekutambura, ini pakupedzisira ndakakwanisa kuvaka nekugadzirisa MediaGoblin muDocker mufananidzo. Zvainge zvatove nyore ipapo svetuka zvisina kufanira transcoding и isa Nginx yechokwadi.

Danho 4. Hosting

Sezvo MediaGoblin yaimhanyisa Docker pakombuta yangu yemuno, nhanho yakatevera yaive yekuiisa kune sevha yegore kuitira kuti mhuri ione vhidhiyo.

MediaGoblin uye dambudziko rekuchengetedza vhidhiyo

Kune akawanda mapuratifomu anotora Docker mufananidzo uye anoitora pane yeruzhinji URL. Iyo inobata ndeyekuti mukuwedzera kune iyo app pachayo, 33 GB yemavhidhiyo mafaera aifanira kuburitswa. Zvaigoneka kuvaomesera-kodhi mumufananidzo weDocker, asi izvo zvaizonetsa uye zvakashata. Kuchinja mutsara mumwe wekumisikidza kunoda kuendesazve 33 GB yedata.

Pandakashandisa ClipBucket ndakagadzirisa dambudziko ne gcsfuse - chishandiso chinobvumira sisitimu yekushandisa kurodha madhairekitori kuGoogle Cloud Cloud kuchengetedza senzira dzenguva dzose kune faira system. Ini ndakabata mafaera evhidhiyo paGoogle Cloud uye ndikashandisa gcsfuse kuita kuti aoneke muClipBucket semafaira emuno.

Musiyano waive wekuti ClipBucket yaimhanya mumuchina chaiwo, uku MediaGoblin ichimhanya mumudziyo weDocker. Pano, kukwidza mafaera kubva kuchengetedzo yegore kwakave kwakaoma zvakanyanya. Ndakapedza maawa akawanda ndichigadzirisa matambudziko ose uye ndakanyora nezvazvo yakazara blog post.

MediaGoblin yekutanga kubatanidzwa neGoogle Cloud kuchengetedza, izvo zvandiri kutaura nezvazvo yakataurwa muna 2018

Mushure memavhiki akati wandei ekugadzirisa zvese zvikamu, zvese zvakashanda. Pasina kuita chero shanduko kuMediaGoblin's kodhi, ndakaibiridzira kuverenga nekunyora midhiya mafaera kuGoogle Cloud kuchengetedza.

Dambudziko chete raive rekuti MediaGoblin yakave inonyadzisa. Zvakatora masekonzi makumi maviri akazara kuti zvidhori zvidhori zvitakurwe papeji yekumba. Kana iwe ukasvetukira kumberi uchiona vhidhiyo, MediaGoblin yaizomira kweasingaperi masekonzi gumi isati yatangazve kutamba.

Dambudziko guru nderekuti mavhidhiyo nemifananidzo zvakatora nzira refu, yekutenderera kumushandisi. Vaifanira kubva kuGoogle cloud storage kuburikidza negcsfuse kuenda kuMediaGoblin, Nginx - uye ndipo pavakazosvika kubrowser yemushandisi. Iyo huru bhodhoro yaive gcsfuse, iyo isina kugadziridzwa kumhanya. Vagadziri vanonyevera nezve kunonoka kukuru mukushanda kweiyo utility pane peji reprojekiti:

Yambiro nezvekuita kwakaderera mune gcsfuse zvinyorwa

Nenzira yakanaka, bhurawuza rinofanira kutora mafaera zvakananga kubva kuGoogle Cloud, ichipfuura chero yepakati. Ungaita sei izvi pasina kunyura muMediaGoblin codebase kana kuwedzera yakaoma Google Cloud yekubatanidza logic?

Sub_filter trick mu nginx

Sezvineiwo ndakawana mhinduro iri nyore, kunyangwe zvishoma zvakashata. Ndakawedzera kune default.conf configuration muNginx sefa yakadaro:

sub_filter "/mgoblin_media/media_entries/" "https://storage.googleapis.com/MY-GCS-BUCKET/media_entries/";
sub_filter_once off;

Mukuseta kwangu, Nginx yakaita semumiriri pakati peMediaGoblin nemushandisi wekupedzisira. Murairo wepamusoro unoraira Nginx kutsvaga nekutsiva mhinduro dzese dzeMediaGoblin HTML isati yadzipa kumushandisi wekupedzisira. Nginx inotsiva nzira dzese dzeMediaGoblin midhiya mafaera ane ma URL kubva kuGoogle Cloud kuchengetedza.

Semuenzaniso, MediaGoblin inogadzira HTML seizvi:

<video width="720" height="480" controls autoplay>
  <source
    src="/mgoblin_media/media_entries/16/Michael-riding-a-bike.mp4"
    type="video/mp4">
</video>

Nginx inoshandura mhinduro:

<video width="720" height="480" controls autoplay>
  <source
    src="https://storage.googleapis.com/MY-GCS-BUCKET/media_entries/16/Michael-riding-a-bike.mp4"
    type="video/mp4">
</video>

Iye zvino zvinhu zvose zviri kufamba sezvinotarisirwa:

Nginx inonyorazve mhinduro kubva kuMediaGoblin kuitira kuti vatengi vakwanise kukumbira midhiya mafaera zvakananga kubva kuGoogle Cloud kuchengetedza

Chinhu chakanakisa nezve mhinduro yangu ndechekuti haidi chero shanduko kuMediaGoblin kodhi. Nginx's maviri-mitsara dhairekitori inobatanidza zvisina mutsetse MediaGoblin neGoogle Cloud, kunyangwe masevhisi asingazivi chinhu nezvemumwe nemumwe.

taura pfungwa: Mhinduro iyi inoda kuti mafaera ari muGoogle cloud storage averengeke nemunhu wese. Kuti ndideredze njodzi yekuwana kusingatenderwe, ini ndinoshandisa refu, isina kurongeka bhaketi zita (e.g. mediagoblin-39dpduhfz1wstbprmyk5ak29) uye simbisa kuti mutemo wekutonga webucket hautenderi vashandisi vasina mvumo kuratidza zviri mukati medhairekitori.

Pakupedzisira chigadzirwa

Panguva ino ndaiva nemhinduro yakakwana, inoshanda. MediaGoblin yakamhanya nemufaro mumudziyo wayo wega papuratifomu yegore reGoogle, saka yaisada kugarwa kana kuvandudzwa nguva zhinji. Zvese zviri mumaitiro angu zvaive otomatiki uye zvichigoneka, zvichibvumira kurongeka kwakapusa kana kudzoreredza kune dzakapfuura shanduro.

Mhuri yangu yaifarira chaizvo kuti kwaiva nyore kuona mavhidhiyo. Nerubatsiro rweNginx hack yakatsanangurwa pamusoro, kushanda nemavhidhiyo kwakakurumidza kunge paYouTube.

Iyo yekuona skrini inoita seizvi:

Zviri mukati mekatalogi yevhidhiyo yemhuri ne tag "Zvakanakisa"

Kudzvanya pane thumbnail kunounza skrini seizvi:

Kuona clip imwe chete pane media server

Mushure memakore ebasa, ndakafara zvisingaite kupa mhuri mukana wekuona mavhidhiyo edu mune imwecheteyo mushandisi-inoshamwaridzika interface seYouTube, ndizvo zvandaida pakutanga.

Bhonasi: Deredza mari isingasviki $1 pamwedzi

Hautarise mavhidhiyo ekumba kazhinji, mwedzi mishoma yega yega. Mhuri yangu pamwe chete yakagadzira anenge maawa makumi maviri emotokari mugore, asi sevha yaimhanya 20/15. Ndakabhadhara $ 99,7 pamwedzi kune sevha yaive pasi XNUMX% yenguva.

Pakupera kwa2018, Google yakaburitsa chigadzirwa Cloud Run. Chinhu chemhondi chaimhanyisa midziyo yeDocker nekukurumidza zvekuti application yaigona kupindura zvikumbiro zveHTTP. Izvi zvinoreva kuti, sevha inogona kuramba iri mumodhi yekumira uye inotanga chete kana mumwe munhu aida kuiwana. Pazvisingawanzo mhanya seyangu, mutengo wakadzikira kubva pamadhora gumi nemashanu pamwedzi kuenda kumasendi mashoma pagore.

Nezvikonzero zvandisingachayeuki, Cloud Run haina kushanda nemufananidzo wangu weMediaGoblin. Asi nekuuya kwe Cloud Run ndakarangarira izvozvo Heroku inopa sevhisi yakafanana yemahara, uye maturusi avo akanyanya mushandisi-ane hushamwari kupfuura eGoogle.

Iine yemahara application server, iyo chete mutengo ndeyekuchengetedza data. Nzvimbo yekuchengetera yeGoogle yakajairika inodhura 2,3 cents/GB. Vhidhiyo yakachengetwa inotora 33 GB, saka ini ndinongobhadhara 77 cents pamwedzi.

Iyi mhinduro inongoda $0,77 chete pamwedzi

Mazano kune avo vari kuronga kuedza

Zviri pachena kuti hurongwa hwacho hwakanditorera nguva. Asi ndinovimba chinyorwa ichi chichakubatsira iwe kuchengetedza 80-90% yekuedza kwedhijitari uye kuburitsa mavhidhiyo ako epamba. Muchikamu chakasiyana unogona kuwana yakadzama nhanho-nhanho gwara mukati mekuita, asi heano mamwe matipi akajairwa:

• Munguva yedhijitari uye editing chikamu, chengetedza yakawanda metadata sezvinobvira.
  • Mashoko anokosha anowanzorekodhwa pamakaseti evhidhiyo.
  • Chengetedza kuti ndeipi clip yakapfurwa kubva patepi uye nehurongwa hupi.
  • Nyora pasi zuva rekupfura, iro rinogona kuoneka pavhidhiyo.
• Funga kubhadhara kune hunyanzvi digitization masevhisi.
  • Uchadaro zvakanyanya zvakaoma uye zvinodhura kuti zvifanane navo maererano nedigitization quality.
  • Asi gara kure nekambani inonzi EverPresent (nditumire meseji kana uchida ruzivo).
• Kana iwe uchiita digitization iwe pachako, tenga HDD.
  • Uncompressed standard tsananguro vhidhiyo inotora 100-200 MB paminiti.
  • Ndakachengeta zvese zvangu Synology DS412 + (10 TB).
• Rekodha metadata mune imwe yakajairika fomati isina kusungirirwa kune chaiyo application.
  • Clip tsananguro, macode enguva, misi, nezvimwe.
  • Kana iwe ukachengeta metadata mune imwe application-chaiyo fomati (kana zvakanyanya, usaichengete zvachose), hauzokwanisi kuitazve basa racho kana ukafunga kushandisa imwe mhinduro.
  • Paunenge uchigadzirisa, unoona yakawanda inobatsira metadata pavhidhiyo. Ucharasikirwa nazvo kana ukasavaponesa.
    • Chii chiri kuitika muvhidhiyo?
    • Ndiani akanyoreswa ipapo?
    • Izvi zvakanyorwa rini?
• Maka mavhidhiyo aunofarira.
  • Kutaura chokwadi, mavhidhiyo mazhinji epamba anofinha.
  • Ini ndinoisa iyo "yakanakisa" tag kune zvandinofarira zvimedu uye ndinovhura pandinoda kuona mavhidhiyo anosekesa.
• Ronga mhinduro yakazara nekukurumidza sezvinobvira kuitira kuti maitiro aende zvakananga kubva pakutanga kusvika pakupedzisira.
  • Ndakaedza kubatanidza matepi ese kutanga, ndozogadzirisa matepi ese, nezvimwe.
  • Ndinoshuva kuti dai ndakatanga netepi imwe chete uye ndaita basa rose nayo. Ipapo ndaizonzwisisa kuti ndedzipi sarudzo uye pamatanho api ane chekuita nemhedzisiro.
• Ramba uchikodha kusvika padiki.
  • Pese paunogadzirisa kana kukodha zvakare clip, unoderedza kunaka kwayo.
  • Chengetedza mifananidzo yako yakasvibirira pamhando yepamusoro, wobva washandura clip yega yega kamwe mufomati inopihwa nemabhurawuza.
• Shandisa mhinduro yakapusa inokwanisika kuburitsa mavhidhiyo.
  • Mukutarisa, MediaGoblin inoita senge chishandiso chakaomesesa cheiyo yakapusa mamiriro ekugadzira mapeji ewebhu ane static seti yemafaira evhidhiyo.
  • Dai ndanga ndichitanga, ndaizoshandisa static saiti jenareta senge Hugo, Jekyll kana Gridsome.
• Ita kuisa.
  • Kugadzirisa vhidhiyo inzira inonakidza yekubatanidza nguva dzakanakisa kubva kumavhidhiyo akawanda.
  • Chinhu chikuru mukugadzirisa mimhanzi. Semuenzaniso, dingindira rinoshamisa Slow Snow kubva kuNational, uku ndiko kuwanikwa kwangu.

Source: www.habr.com