{"id":84416,"date":"2020-06-07T13:42:50","date_gmt":"2020-06-07T11:42:50","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/formaty-fajlov-v-bolshih-dannyh-kratkij-likbez"},"modified":"2020-06-07T13:42:50","modified_gmt":"2020-06-07T11:42:50","slug":"formaty-fajlov-v-bolshih-dannyh-kratkij-likbez","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/formaty-fajlov-v-bolshih-dannyh-kratkij-likbez","title":{"rendered":"Formatet e skedar\u00ebve n\u00eb t\u00eb dh\u00ebnat e m\u00ebdha: nj\u00eb p\u00ebrmbledhje e shkurt\u00ebr","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Formatet
\nZoti i Motit nga Remarin<\/i><\/a><\/noindex> <\/p>\n

Ekipa Zgjidhjet Cloud t\u00eb Mail.ru<\/a><\/noindex> ofron p\u00ebrkthimi i artikullit<\/a><\/noindex> i inxhinierit Rahul Bhatia nga kompania Clairvoyant mbi formatet e skedar\u00ebve n\u00eb t\u00eb dh\u00ebnat e m\u00ebdha, funksionet m\u00eb t\u00eb zakonshme t\u00eb formatit Hadoop dhe cilin format \u00ebsht\u00eb m\u00eb i mir\u00eb t\u00eb p\u00ebrdoret.
\n<\/a><\/noindex><\/p>\n

Pse jan\u00eb t\u00eb nevojshme formate t\u00eb ndryshme skedar\u00ebsh<\/h2>\n

\nNj\u00eb kufizim i r\u00ebnd\u00ebsish\u00ebm n\u00eb performanc\u00ebn e aplikacioneve q\u00eb mb\u00ebshtesin HDFS, si MapReduce dhe Spark \u2014 koha e k\u00ebrkimit, leximit dhe shkrimit t\u00eb t\u00eb dh\u00ebnave. K\u00ebto probleme p\u00ebrkeq\u00ebsohen nga v\u00ebshtir\u00ebsit\u00eb n\u00eb menaxhimin e grupeve t\u00eb m\u00ebdha t\u00eb t\u00eb dh\u00ebnave, n\u00ebse kemi nj\u00eb skem\u00eb q\u00eb evoluon dhe ka disa kufizime p\u00ebr ruajtjen.<\/p>\n

P\u00ebrpunimi i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha rrit ngarkes\u00ebn n\u00eb n\u00ebn-sistemet e ruajtjes \u2014 Hadoop ruan t\u00eb dh\u00ebnat n\u00eb m\u00ebnyr\u00eb t\u00eb tepruar p\u00ebr t\u00eb arritur q\u00ebndrueshm\u00ebrin\u00eb. P\u00ebrve\u00e7 disqeve, ngarkohen procesori, rrjeti, sistemi i hyrjes-daljes dhe k\u00ebshtu me radh\u00eb. Nd\u00ebrsa rritet sasia e t\u00eb dh\u00ebnave, p\u00ebrmir\u00ebsohet gjithashtu kostoja e p\u00ebrpunimit dhe ruajtjes s\u00eb tyre.<\/p>\n

Formatet e ndryshme t\u00eb skedar\u00ebve n\u00eb Hadoop<\/a><\/noindex> jan\u00eb krijuar p\u00ebr t\u00eb zgjidhur k\u00ebto probleme. Zgjedhja e formatit t\u00eb duhur t\u00eb skedarit mund t\u00eb sjell\u00eb disa p\u00ebrfitime t\u00eb konsiderueshme:<\/p>\n

    \n
  1. Koha m\u00eb e shpejt\u00eb e leximit.<\/li>\n
  2. Koha m\u00eb e shpejt\u00eb e shkrimit.<\/li>\n
  3. Skedar\u00eb t\u00eb ndar\u00eb.<\/li>\n
  4. Mb\u00ebshtetje p\u00ebr evolucionin e skemave.<\/li>\n
  5. Mb\u00ebshtetje e zgjeruar p\u00ebr kompresimin.<\/li>\n<\/ol>\n

    \nDisa formate skedar\u00ebsh jan\u00eb p\u00ebr p\u00ebrdorim t\u00eb p\u00ebrgjithsh\u00ebm, t\u00eb tjerat p\u00ebr variante m\u00eb specifike, dhe disa jan\u00eb zhvilluar me karakteristika specifike t\u00eb t\u00eb dh\u00ebnave. Pra, zgjedhja \u00ebsht\u00eb v\u00ebrtet\u00eb e madhe.<\/p>\n

    Formati i skedar\u00ebve Avro<\/h2>\n

    \nP\u00ebr serializimi i t\u00eb dh\u00ebnave <\/i>Avro p\u00ebrdoret gjer\u00ebsisht \u2014 ky \u00ebsht\u00eb format i bazuar n\u00eb rreshta<\/i>, q\u00eb do t\u00eb thot\u00eb, formati i t\u00eb dh\u00ebnave t\u00eb ruajtura n\u00eb Hadoop. Ai ruan skem\u00ebn n\u00eb format JSON, duke e leht\u00ebsuar leximin dhe interpretimin nga \u00e7do program. T\u00eb dh\u00ebnat vet\u00eb ruhen n\u00eb format binar, q\u00eb \u00ebsht\u00eb kompakt dhe efikas.<\/p>\n

    Sistemi i serializimit Avro \u00ebsht\u00eb neutral ndaj gjuh\u00ebve. Skedar\u00ebt mund t\u00eb p\u00ebrpunohen nga gjuh\u00eb t\u00eb ndryshme, n\u00eb k\u00ebt\u00eb moment jan\u00eb C, C++, C#, Java, Python dhe Ruby.<\/p>\n

    Nj\u00eb ve\u00e7ori ky\u00e7e e Avro \u00ebsht\u00eb mb\u00ebshtetja e besueshme p\u00ebr skemat e t\u00eb dh\u00ebnave, t\u00eb cilat ndryshojn\u00eb me kalimin e koh\u00ebs, pra evoluojn\u00eb. Avro e kupton ndryshimin e skem\u00ebs \u2014 fshirjen, shtimin ose modifikimin e fushave.<\/p>\n

    Avro mb\u00ebshtet struktura t\u00eb ndryshme t\u00eb t\u00eb dh\u00ebnave. P\u00ebr shembull, mund t\u00eb krijoni nj\u00eb regjist\u00ebr q\u00eb p\u00ebrmban nj\u00eb list\u00eb, nj\u00eb tip t\u00eb enumeruar dhe nj\u00eb regjist\u00ebr n\u00ebn.<\/p>\n

    \"Formatet
    \nKy format \u00ebsht\u00eb ideal p\u00ebr regjistrimin n\u00eb zon\u00ebn e uljes (n\u00eb zon\u00ebn kaluese) t\u00eb liqenit t\u00eb t\u00eb dh\u00ebnave (    liqeni i t\u00eb dh\u00ebnave<\/a><\/noindex>, ose data lake \u2014 nj\u00eb koleksion instancash p\u00ebr ruajtjen e llojeve t\u00eb ndryshme t\u00eb t\u00eb dh\u00ebnave p\u00ebrve\u00e7 burimeve t\u00eb t\u00eb dh\u00ebnave q\u00eb lidhen drejtp\u00ebrdrejt). <\/p>\n

    Pra, p\u00ebr regjistrimin n\u00eb zon\u00ebn e uljes t\u00eb liqenit t\u00eb t\u00eb dh\u00ebnave, ky format \u00ebsht\u00eb m\u00eb i p\u00ebrshtatsh\u00ebm p\u00ebr arsyet e m\u00ebposhtme:<\/p>\n

      \n
    1. T\u00eb dh\u00ebnat nga kjo zon\u00eb zakonisht lexohen t\u00ebr\u00ebsisht p\u00ebr p\u00ebrpunim t\u00eb m\u00ebtejsh\u00ebm nga sistemet posht\u00eb \u2014 dhe formati i bazuar n\u00eb rreshta n\u00eb k\u00ebt\u00eb rast \u00ebsht\u00eb m\u00eb efikas.<\/li>\n
    2. Sistemet posht\u00eb mund t\u00eb nxjerrin leht\u00ebsisht tabelat e skemave nga skedar\u00ebt \u2014 nuk \u00ebsht\u00eb nevoja t\u00eb ruhen skemat ve\u00e7mas n\u00eb nj\u00eb metastore t\u00eb jasht\u00ebm.<\/li>\n
    3. \u00c7do ndryshim n\u00eb skem\u00ebn fillestare p\u00ebrpunohen leht\u00ebsisht (evolucioni i skem\u00ebs).<\/li>\n<\/ol>\n

      <\/p>\n

      Formati i skedave Parquet<\/h2>\n

      \nParquet \u00ebsht\u00eb nj\u00eb format skedash open source p\u00ebr Hadoop, i cili ruan strukturat e brendshme t\u00eb t\u00eb dh\u00ebnave n\u00eb nj\u00eb format t\u00eb shesht\u00eb kolonor<\/i>.<\/p>\n

      N\u00eb krahasim me qasjen tradicionale lineare, Parquet \u00ebsht\u00eb m\u00eb efikas n\u00eb aspektin e ruajtjes dhe performanc\u00ebs.<\/p>\n

      Kjo \u00ebsht\u00eb ve\u00e7an\u00ebrisht e dobishme p\u00ebr k\u00ebrkesat q\u00eb lexojn\u00eb kolona t\u00eb caktuara nga nj\u00eb tabel\u00eb e gjer\u00eb (me shum\u00eb kolona). Fal\u00eb formatit t\u00eb skedave, lexohen vet\u00ebm kolonat e nevojshme, duke reduktuar k\u00ebshtu input-in dhe output-in n\u00eb minimum.<\/p>\n

      Nj\u00eb shpjegim i vog\u00ebl<\/strong>: p\u00ebr t\u00eb kuptuar m\u00eb mir\u00eb formatin e skedave Parquet n\u00eb Hadoop, le t\u00eb shohim se \u00e7far\u00eb \u00ebsht\u00eb nj\u00eb format i bazuar n\u00eb kolona \u2014 pra, nj\u00eb format kolonal. N\u00eb k\u00ebt\u00eb format, vlerat homogjene t\u00eb \u00e7do kolone ruhen s\u00eb bashku. <\/p>\n

      P\u00ebr shembull<\/a><\/noindex>, shkrimi p\u00ebrfshin fushat ID, Name dhe Department. N\u00eb k\u00ebt\u00eb rast, t\u00eb gjitha vlerat e kolones ID do t\u00eb ruhen s\u00eb bashku, ashtu si edhe vlerat e kolones Name e k\u00ebshtu me radh\u00eb. Tabela do t\u00eb duket af\u00ebrsisht k\u00ebshtu:<\/p>\n

      ID<\/strong>
      \nEmri<\/strong>
      \nDepartment<\/strong><\/p>\n

      1
      \nemp1
      \nd1<\/p>\n

      2
      \nemp2
      \nd2<\/p>\n

      3
      \nemp3
      \nd3<\/p>\n

      \nN\u00eb formatin e vargjeve, t\u00eb dh\u00ebnat do t\u00eb ruhen si m\u00eb posht\u00eb:<\/p>\n

      1
      \nemp1
      \nd1
      \n2
      \nemp2
      \nd2
      \n3
      \nemp3
      \nd3<\/p>\n

      \nN\u00eb formatin kolonal, t\u00eb nj\u00ebjtat t\u00eb dh\u00ebna do t\u00eb ruhet k\u00ebshtu:<\/p>\n

      1
      \n2
      \n3
      \nemp1
      \nemp2
      \nemp3
      \nd1
      \nd2
      \nd3<\/p>\n

      \nFormati kolonal \u00ebsht\u00eb m\u00eb efikas kur duhet t\u00eb k\u00ebrkoni disa kolona nga tabela. Ai do t\u00eb lexoj\u00eb vet\u00ebm kolonat e nevojshme, sepse ato ndodhen ngjitur. K\u00ebshtu, operacionet e input-it dhe output-it reduktohen n\u00eb minimum.<\/p>\n

      P\u00ebr shembull, ju nevojitet vet\u00ebm kolona NAME. N\u00eb formatin e vargjeve<\/a><\/noindex> \u00c7do regjistrim n\u00eb grupin e t\u00eb dh\u00ebnave duhet t\u00eb ngarkohet, t\u00eb analizohet sipas fushave, dhe pastaj t\u00eb nxirren t\u00eb dh\u00ebnat NAME. Formati kolon\u00eb lejon kalimin direkt n\u00eb kolon\u00ebn Name, pasi t\u00eb gjitha vlerat p\u00ebr k\u00ebt\u00eb kolon\u00eb ruhen s\u00eb bashku. Nuk do t\u00eb ket\u00eb nevoj\u00eb t\u00eb skanohet e gjith\u00eb regjistrimi.<\/p>\n

      K\u00ebshtu, formati kolon\u00eb rrit performanc\u00ebn e pyetjeve, pasi p\u00ebr t\u00eb kaluar n\u00eb kolonat e nevojshme k\u00ebrkohet m\u00eb pak koh\u00eb k\u00ebrkimi dhe reduktohet numri i operacioneve t\u00eb hyrjes-daljes, pasi lexohet vet\u00ebm kolonat e nevojshme.<\/p>\n

      Nj\u00eb nga ve\u00e7orit\u00eb unike Parquet<\/a><\/noindex> \u00ebsht\u00eb q\u00eb n\u00eb nj\u00eb format t\u00eb till\u00eb ai mund t\u00eb ruaj\u00eb t\u00eb dh\u00ebna me struktura t\u00eb brendshme<\/i>. Kjo do t\u00eb thot\u00eb se n\u00eb skedarin Parquet, edhe fushat e brendshme mund t\u00eb lexohen ve\u00e7mas pa pasur nevoj\u00eb t\u00eb lexohen t\u00eb gjitha fushat n\u00eb struktur\u00ebn e brendshme. P\u00ebr ruajtjen e strukturave t\u00eb brendshme, Parquet p\u00ebrdor algoritmin e shkat\u00ebrrimit dhe montimit (shredding and assembly).<\/p>\n

      \"Formatet
      \nP\u00ebr t\u00eb kuptuar formatin e skedarit Parquet n\u00eb Hadoop, \u00ebsht\u00eb e nevojshme t\u00eb dihen termat e m\u00ebposhtme:<\/p>\n

        \n
      1. Grupi i rreshtave<\/strong> (row group): nj\u00eb ndarje logjike horizontale e t\u00eb dh\u00ebnave n\u00eb rreshta. Grupi i rreshtave p\u00ebrb\u00ebhet nga nj\u00eb fragment i secil\u00ebs kolon\u00eb n\u00eb grupin e t\u00eb dh\u00ebnave.<\/li>\n
      2. Fragmenti i kolon\u00ebs<\/strong> (column chunk): nj\u00eb fragment i nj\u00eb kolone specifike. K\u00ebto fragmente kolone jetojn\u00eb n\u00eb nj\u00eb grup t\u00eb caktuar rreshtash dhe garantuar do t\u00eb jen\u00eb t\u00eb ngjitur n\u00eb skedar.<\/li>\n
      3. Faqja<\/strong> (page): fragmentet e kolonave ndahen n\u00eb faqe, t\u00eb regjistruara nj\u00ebra pas tjetr\u00ebs. Faqet kan\u00eb nj\u00eb titull t\u00eb p\u00ebrbashk\u00ebt, k\u00ebshtu q\u00eb gjat\u00eb leximit mund t\u00eb kaloni t\u00eb panevojshmet.<\/li>\n<\/ol>\n

        \n\"Formatet
        \nK\u00ebtu titulli thjesht p\u00ebrmban nj\u00eb num\u00ebr magjik PAR1<\/i> (4 bytes), i cili identifikon skedarin si t\u00eb formatit Parquet.<\/p>\n

        N\u00eb fund shkruhet si vijon:<\/p>\n

          \n
        1. Metadat e skedarit, t\u00eb cilat p\u00ebrmbajn\u00eb koordinatat fillestare t\u00eb metadata t\u00eb secil\u00ebs kolon\u00eb. Gjat\u00eb leximit, s\u00eb pari duhet t\u00eb lexoni metadat e skedarit p\u00ebr t\u00eb gjetur t\u00eb gjitha fragmentet e kolonave t\u00eb interesit. Pastaj, fragmentet e kolonave duhet t\u00eb lexohen radhazi. Metadat gjithashtu p\u00ebrfshijn\u00eb versionin e formatit, skem\u00ebn dhe \u00e7do \u00e7ift shtes\u00eb \u00e7el\u00ebs-vler\u00eb.<\/li>\n
        2. Gjat\u00ebsia e metadatas (4 bytes).<\/li>\n
        3. Numri magjik PAR1<\/i> (4 bytes).<\/li>\n<\/ol>\n

          <\/p>\n

          Formati i skedar\u00ebve ORC<\/h2>\n

          \nFormati i skedar\u00ebve t\u00eb optimizuar rresht-kolon\u00eb<\/i> (Optimized Row Columnar, ORC<\/a><\/noindex>) ofron nj\u00eb m\u00ebnyr\u00eb shum\u00eb efikase p\u00ebr t\u00eb ruajtur t\u00eb dh\u00ebna dhe \u00ebsht\u00eb projektuar p\u00ebr t\u00eb tejkaluar kufizimet e formateve t\u00eb tjera. Ruajtjen e t\u00eb dh\u00ebnave n\u00eb nj\u00eb format t\u00eb p\u00ebrsosur kompakt, duke lejuar kalimin e detajeve t\u00eb panevojshme \u2014 n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, nuk k\u00ebrkon nd\u00ebrtimin e indekseve t\u00eb m\u00ebdha, t\u00eb komplikuara ose t\u00eb menaxhueshme manualisht. <\/p>\n

          P\u00ebrfitimet e formatit ORC:<\/p>\n

            \n
          1. Nj\u00eb skedar n\u00eb daljen e \u00e7do detyre, q\u00eb redukton ngarkes\u00ebn n\u00eb NameNode (nyja e emrave).<\/li>\n
          2. Mb\u00ebshtetje p\u00ebr lloje t\u00eb dh\u00ebnash Hive, duke p\u00ebrfshir\u00eb DateTime, numra decimal\u00eb dhe lloje komplekse t\u00eb dh\u00ebnash (struktur\u00eb, list\u00eb, hart\u00eb dhe bashkim).<\/li>\n
          3. Leximi i nj\u00ebjtit skedar nga procese t\u00eb ndryshme RecordReader nj\u00ebkoh\u00ebsisht.<\/li>\n
          4. Mund\u00ebsia p\u00ebr t\u00eb ndar\u00eb skedar\u00ebt pa skanuar p\u00ebr ndonj\u00eb shenj\u00eb.<\/li>\n
          5. Vler\u00ebsimi i ndarjes maksimale t\u00eb kujtes\u00ebs heap p\u00ebr proceset e leximit\/shkruarjes sipas informacionit n\u00eb fundin e skedarit.<\/li>\n
          6. Metadatot ruhen n\u00eb nj\u00eb format binar t\u00eb serializimit t\u00eb Protocol Buffers, i cili lejon shtimin dhe fshirjen e fushave.<\/li>\n<\/ol>\n

            \n\"Formatet
            \nORC ruan koleksione rreshtash n\u00eb nj\u00eb skedar, dhe brenda koleksionit, t\u00eb dh\u00ebnat rreshtore ruhen n\u00eb format kolonesh.<\/p>\n

            Fajl ORC ruan grupe rreshtash, t\u00eb cilat quhen stripe (stripes), si dhe informacion ndihm\u00ebs n\u00eb fund t\u00eb fajlit. Postscript n\u00eb fund t\u00eb fajlit p\u00ebrmban parametrat e kompresimit dhe madh\u00ebsin\u00eb e fundit t\u00eb kompresuar.<\/p>\n

            P\u00ebr default, madh\u00ebsia e stripe \u00ebsht\u00eb 250 MB. Me stripe t\u00eb till\u00eb t\u00eb m\u00ebdha, leximi nga HDFS realizohet m\u00eb efektivisht: n\u00eb blloqe t\u00eb m\u00ebdha dhe t\u00eb pand\u00ebrprera.<\/p>\n

            N\u00eb fundin e fajlit \u00ebsht\u00eb shkruar nj\u00eb list\u00eb e stripeve n\u00eb fajl, numri i rreshtave p\u00ebr \u00e7do stripe dhe lloji i t\u00eb dh\u00ebnave p\u00ebr \u00e7do kolone. Aty \u00ebsht\u00eb shkruar gjithashtu vlera p\u00ebrfundimtare count, min, max dhe sum p\u00ebr \u00e7do kolone.<\/p>\n

            Fundi i stripe p\u00ebrmban nj\u00eb katalog pozita rrjedhe.<\/p>\n

            T\u00eb dh\u00ebnat rreshtore p\u00ebrdoren gjat\u00eb skanimit t\u00eb tabelave.<\/p>\n

            T\u00eb dh\u00ebnat e indeksit p\u00ebrfshijn\u00eb vlerat minimale dhe maksimale p\u00ebr \u00e7do kolone dhe pozitat e rreshtave n\u00eb \u00e7do kolone. Indeks\u00ebt ORC p\u00ebrdoren vet\u00ebm p\u00ebr t\u00eb zgjedhur stripe dhe grupe rreshtash, dhe jo p\u00ebr t'u p\u00ebrgjigjur ndaj k\u00ebrkesave.<\/p>\n

            Krahasimi i formateve t\u00eb ndryshme t\u00eb fayllave<\/h2>\n

            <\/p>\n

            Avro n\u00eb krahasim me Parquet<\/h3>\n

            <\/p>\n

              \n
            1. Avro \u00ebsht\u00eb nj\u00eb format ruajtjeje sipas rreshtave, nd\u00ebrsa Parquet ruan t\u00eb dh\u00ebnat sipas kolonave.<\/li>\n
            2. Parquet \u00ebsht\u00eb m\u00eb i p\u00ebrshtatsh\u00ebm p\u00ebr k\u00ebrkesat analitike, dometh\u00ebn\u00eb operacionet e leximit dhe k\u00ebrkimit t\u00eb t\u00eb dh\u00ebnave jan\u00eb shum\u00eb m\u00eb efikase se shkrimi.<\/li>\n
            3. Operacionet e shkrimit n\u00eb Avro kryhen m\u00eb efektivisht se n\u00eb Parquet.<\/li>\n
            4. Avro funksionon m\u00eb qart\u00eb me evolucionin e skemave. Parquet mb\u00ebshtet vet\u00ebm shtimin e skemave, nd\u00ebrsa n\u00eb Avro \u00ebsht\u00eb implementuar nj\u00eb evolucion multifunksional, dometh\u00ebn\u00eb shtimi ose modifikimi i kolonave.<\/li>\n
            5. Parquet \u00ebsht\u00eb ideal p\u00ebr t\u00eb k\u00ebrkuar n\u00ebngrupe kolonash n\u00eb nj\u00eb tabel\u00eb me shum\u00eb kolona. Avro \u00ebsht\u00eb i p\u00ebrshtatsh\u00ebm p\u00ebr operacione ETL, ku k\u00ebrkojm\u00eb t\u00eb gjitha kolonat.<\/li>\n<\/ol>\n

              <\/p>\n

              ORC krahasuar me Parquet<\/h3>\n

              <\/p>\n

                \n
              1. Parquet ruan m\u00eb mir\u00eb t\u00eb dh\u00ebnat e brendshme.<\/li>\n
              2. ORC \u00ebsht\u00eb m\u00eb i p\u00ebrshtatsh\u00ebm p\u00ebr shtypjen e predikateve (predicate pushdown).<\/li>\n
              3. ORC mb\u00ebshtet pronat ACID.<\/li>\n
              4. ORC thjeshton m\u00eb mir\u00eb t\u00eb dh\u00ebnat.<\/li>\n<\/ol>\n

                \n\u00c7far\u00eb tjet\u00ebr t\u00eb lexoni n\u00eb k\u00ebt\u00eb tem\u00eb<\/strong>:<\/p>\n

                  \n
                1. Analiza e t\u00eb dh\u00ebnave t\u00eb m\u00ebdha n\u00eb cloud: si t\u00eb b\u00ebhen kompanit\u00eb t\u00eb orientuara nga t\u00eb dh\u00ebnat<\/a><\/noindex>.<\/li>\n
                2. Nj\u00eb udh\u00ebzues modest p\u00ebr skemat e bazave t\u00eb t\u00eb dh\u00ebnave<\/a><\/noindex>.<\/li>\n
                3. Kanalin ton\u00eb n\u00eb Telegram p\u00ebr transformimin digjital<\/a><\/noindex>. \n<\/li>\n<\/ol>\n

                  Burimi: habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

                  Weather Deity by Remarin \u041a\u043e\u043c\u0430\u043d\u0434\u0430 Mail.ru Cloud Solutions \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u0442 \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u0441\u0442\u0430\u0442\u044c\u0438 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u0430 \u0420\u0430\u0445\u0443\u043b\u0430 \u0411\u0445\u0430\u0442\u0438\u0438 \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Clairvoyant \u043e \u0442\u043e\u043c, \u043a\u0430\u043a\u0438\u0435 \u0435\u0441\u0442\u044c \u0444\u043e\u0440\u043c\u0430\u0442\u044b \u0444\u0430\u0439\u043b\u043e\u0432 \u0432 \u0431\u043e\u043b\u044c\u0448\u0438\u0445 \u0434\u0430\u043d\u043d\u044b\u0445, \u043a\u0430\u043a\u0438\u0435 \u0441\u0430\u043c\u044b\u0435 \u0440\u0430\u0441\u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0435\u043d\u043d\u044b\u0435 \u0444\u0443\u043d\u043a\u0446\u0438\u0438 \u0444\u043e\u0440\u043c\u0430\u0442\u043e\u0432 Hadoop \u0438 \u043a\u0430\u043a\u043e\u0439 \u0444\u043e\u0440\u043c\u0430\u0442 \u043b\u0443\u0447\u0448\u0435 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c. \u0417\u0430\u0447\u0435\u043c \u043d\u0443\u0436\u043d\u044b \u0440\u0430\u0437\u043d\u044b\u0435 \u0444\u043e\u0440\u043c\u0430\u0442\u044b \u0444\u0430\u0439\u043b\u043e\u0432 \u0421\u0435\u0440\u044c\u0435\u0437\u043d\u043e\u0435 \u0443\u0437\u043a\u043e\u0435 \u043c\u0435\u0441\u0442\u043e \u0432 \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u0439 \u0441 \u043f\u043e\u0434\u0434\u0435\u0440\u0436\u043a\u043e\u0439 HDFS, \u0442\u0430\u043a\u0438\u0445 \u043a\u0430\u043a MapReduce \u0438 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":84417,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-84416","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Formatet e skedar\u00ebve n\u00eb t\u00eb dh\u00ebnat e m\u00ebdha: nj\u00eb e shpjeguar e shkurt\u00ebr | ProHoster","description":"Zoti i Motit nga Remarin Ekipa e Mail.ru Cloud Solutions ofron p\u00ebrkthimin e artikullit nga inxhinieri Rahul Bhatia nga kompania Clairvoyant mbi formatet e skedar\u00ebve n\u00eb t\u00eb dh\u00ebnat e m\u00ebdha, cilat jan\u00eb funksionet m\u00eb t\u00eb zakonshme t\u00eb formateve Hadoop dhe cili format \u00ebsht\u00eb m\u00eb i mir\u00eb p\u00ebr t'u p\u00ebrdorur. Pse jan\u00eb t\u00eb nevojshme formate t\u00eb ndryshme skedar\u00ebsh Nj\u00eb ngushtic\u00eb serioze n\u00eb performanc\u00ebn e aplikacioneve q\u00eb mb\u00ebshtesin HDFS, t\u00eb tilla si MapReduce dhe","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/formaty-fajlov-v-bolshih-dannyh-kratkij-likbez","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0424\u043e\u0440\u043c\u0430\u0442\u044b \u0444\u0430\u0439\u043b\u043e\u0432 \u0432 \u0431\u043e\u043b\u044c\u0448\u0438\u0445 \u0434\u0430\u043d\u043d\u044b\u0445: \u043a\u0440\u0430\u0442\u043a\u0438\u0439 \u043b\u0438\u043a\u0431\u0435\u0437 | ProHoster","og:description":"Weather Deity by Remarin \u041a\u043e\u043c\u0430\u043d\u0434\u0430 Mail.ru Cloud Solutions \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u0442 \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u0441\u0442\u0430\u0442\u044c\u0438 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u0430 \u0420\u0430\u0445\u0443\u043b\u0430 \u0411\u0445\u0430\u0442\u0438\u0438 \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Clairvoyant \u043e \u0442\u043e\u043c, \u043a\u0430\u043a\u0438\u0435 \u0435\u0441\u0442\u044c \u0444\u043e\u0440\u043c\u0430\u0442\u044b \u0444\u0430\u0439\u043b\u043e\u0432 \u0432 \u0431\u043e\u043b\u044c\u0448\u0438\u0445 \u0434\u0430\u043d\u043d\u044b\u0445, \u043a\u0430\u043a\u0438\u0435 \u0441\u0430\u043c\u044b\u0435 \u0440\u0430\u0441\u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0435\u043d\u043d\u044b\u0435 \u0444\u0443\u043d\u043a\u0446\u0438\u0438 \u0444\u043e\u0440\u043c\u0430\u0442\u043e\u0432 Hadoop \u0438 \u043a\u0430\u043a\u043e\u0439 \u0444\u043e\u0440\u043c\u0430\u0442 \u043b\u0443\u0447\u0448\u0435 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c. \u0417\u0430\u0447\u0435\u043c \u043d\u0443\u0436\u043d\u044b \u0440\u0430\u0437\u043d\u044b\u0435 \u0444\u043e\u0440\u043c\u0430\u0442\u044b \u0444\u0430\u0439\u043b\u043e\u0432 \u0421\u0435\u0440\u044c\u0435\u0437\u043d\u043e\u0435 \u0443\u0437\u043a\u043e\u0435 \u043c\u0435\u0441\u0442\u043e \u0432 \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u0439 \u0441 \u043f\u043e\u0434\u0434\u0435\u0440\u0436\u043a\u043e\u0439 HDFS, \u0442\u0430\u043a\u0438\u0445 \u043a\u0430\u043a MapReduce \u0438","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/formaty-fajlov-v-bolshih-dannyh-kratkij-likbez","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-06-07T11:42:50+00:00","article:modified_time":"2020-06-07T11:42:50+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"84416","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 14:58:40","updated":"2022-09-28 08:24:46"},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/84416","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=84416"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/84416\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/84417"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=84416"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=84416"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=84416"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}