{"id":30548,"date":"2019-10-31T21:36:08","date_gmt":"2019-10-31T18:36:08","guid":{"rendered":"https:\/\/prohoster.info\/blog\/konfigurirovanie-spark-na-yarn\/"},"modified":"2019-10-31T21:36:08","modified_gmt":"2019-10-31T18:36:08","slug":"konfigurirovanie-spark-na-yarn","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/news\/konfigurirovanie-spark-na-yarn","title":{"rendered":"Konfigurimi i Spark n\u00eb YARN","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

P\u00ebrsh\u00ebndetje, Habr! Ddje n\u00eb mitapin e dedikuar Apache Spark<\/a><\/noindex>, \u043e\u0442 \u0440\u0435\u0431\u044f\u0442 \u0438\u0437 Rambler&Co, \u0431\u044b\u043b\u043e \u0434\u043e\u0432\u043e\u043b\u044c\u043d\u043e \u043c\u043d\u043e\u0433\u043e \u0432\u043e\u043f\u0440\u043e\u0441\u043e\u0432 \u043e\u0442 \u0443\u0447\u0430\u0441\u0442\u043d\u0438\u043a\u043e\u0432, \u0441\u0432\u044f\u0437\u0430\u043d\u043d\u044b\u0445 \u0441 \u043a\u043e\u043d\u0444\u0438\u0433\u0443\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u044d\u0442\u043e\u0433\u043e \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u0430. \u0420\u0435\u0448\u0438\u043b\u0438 \u043f\u043e \u0435\u0433\u043e \u0441\u043b\u0435\u0434\u0430\u043c \u043f\u043e\u0434\u0435\u043b\u0438\u0442\u044c\u0441\u044f \u0441\u0432\u043e\u0438\u043c \u043e\u043f\u044b\u0442\u043e\u043c. \u0422\u0435\u043c\u0430 \u043d\u0435\u043f\u0440\u043e\u0441\u0442\u0430\u044f \u2014 \u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c \u0434\u0435\u043b\u0438\u0442\u044c\u0441\u044f \u043e\u043f\u044b\u0442\u043e\u043c \u0442\u043e\u0436\u0435 \u0432 \u043a\u043e\u043c\u043c\u0435\u043d\u0442\u0430\u0440\u0438\u044f\u0445, \u043c\u043e\u0436\u0435\u0442 \u0431\u044b\u0442\u044c, \u043c\u044b \u0442\u043e\u0436\u0435 \u0447\u0442\u043e-\u0442\u043e \u043d\u0435 \u0442\u0430\u043a \u043f\u043e\u043d\u0438\u043c\u0430\u0435\u043c \u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c.
\n<\/a><\/noindex>
\nNj\u00eb hyrje e vog\u00ebl \u2014 si e p\u00ebrdorim Spark. Ne kemi nj\u00eb program tre-mujor \u201cSpecialist p\u00ebr t\u00eb dh\u00ebna t\u00eb m\u00ebdha\u201d<\/a><\/noindex>, dhe gjat\u00eb modulit t\u00eb dyt\u00eb, pjes\u00ebmarr\u00ebsit tan\u00eb punojn\u00eb me k\u00ebt\u00eb instrument. Si organizator\u00eb, detyra jon\u00eb \u00ebsht\u00eb t\u00eb p\u00ebrgatisim nj\u00eb klaster p\u00ebr p\u00ebrdorim n\u00eb k\u00ebt\u00eb rast.<\/p>\n

Ve\u00e7oria e p\u00ebrdorimit ton\u00eb \u00ebsht\u00eb se numri i njer\u00ebzve q\u00eb punojn\u00eb n\u00eb m\u00ebnyr\u00eb t\u00eb nj\u00ebkohshme me Spark mund t\u00eb jet\u00eb i barabart\u00eb me t\u00eb gjith\u00eb grupin. P\u00ebr shembull, n\u00eb seminar, kur t\u00eb gjith\u00eb provojn\u00eb di\u00e7ka n\u00eb nj\u00eb koh\u00eb dhe p\u00ebrs\u00ebritin pas m\u00ebsuesit ton\u00eb. Dhe kjo \u00ebsht\u00eb jo pak \u2014 deri n\u00eb 40 njer\u00ebz ndonj\u00ebher\u00eb. Ndoshta nuk ka shum\u00eb kompani n\u00eb bot\u00eb q\u00eb p\u00ebrballen me nj\u00eb skenar p\u00ebrdorimi t\u00eb till\u00eb.<\/p>\n

M\u00eb tej do t\u00eb flas se si dhe pse p\u00ebrzgjedh\u00ebm parametrat e caktuar t\u00eb konfigurimit.<\/p>\n

T\u00eb fillojm\u00eb nga fillimi. Spark ka 3 mund\u00ebsi p\u00ebr t\u00eb punuar n\u00eb klaster: standalone, duke p\u00ebrdorur Mesos dhe duke p\u00ebrdorur YARN. Ne vendos\u00ebm t\u00eb zgjidhim opsionin e tret\u00eb, sepse p\u00ebr ne ishte logjik.<\/p>\n

spark.master=yarn<\/code><\/pre>\n
\nM\u00eb pas b\u00ebhet m\u00eb interesante. \u00c7do nj\u00eb nga k\u00ebto 3 variante t\u00eb vendosjes ka 2 variante t\u00eb shkarkimit: client dhe cluster. Duke u bazuar n\u00eb dokumentacionin<\/a><\/noindex> dhe lidhjet e ndryshme n\u00eb internet, mund t\u00eb arrijm\u00eb n\u00eb p\u00ebrfundimin se client \u00ebsht\u00eb i p\u00ebrshtatsh\u00ebm p\u00ebr pun\u00eb interaktive \u2014 p\u00ebr shembull, p\u00ebrmes jupyter notebook, nd\u00ebrsa cluster \u00ebsht\u00eb m\u00eb i p\u00ebrshtatsh\u00ebm p\u00ebr zgjidhje produksioni. N\u00eb rastin ton\u00eb, na interesonte puna interaktive, k\u00ebshtu q\u00eb:<\/p>\n
spark.deploy-mode=client<\/code><\/pre>\n
\nN\u00eb t\u00eb v\u00ebrtet\u00eb nga ky moment, Spark do t\u00eb funksionoj\u00eb n\u00eb YARN, por kjo nuk ishte e mjaftueshme p\u00ebr ne. Duke qen\u00eb se kemi nj\u00eb program p\u00ebr t\u00eb dh\u00ebna t\u00eb m\u00ebdha, p\u00ebr disa pjes\u00ebmarr\u00ebs ndonj\u00ebher\u00eb u mungonin burimet q\u00eb fitoheshin nga ndarja e rregullt t\u00eb tyre. Dhe k\u00ebtu gjet\u00ebm nj\u00eb gj\u00eb interesante \u2014 alokimin dinamik t\u00eb burimeve. N\u00eb shprehje t\u00eb thjesht\u00eb, n\u00ebse keni nj\u00eb detyr\u00eb t\u00eb r\u00ebnd\u00eb dhe klasteri \u00ebsht\u00eb i lir\u00eb (p\u00ebr shembull, m\u00ebngjesin), at\u00ebher\u00eb me k\u00ebt\u00eb opsion, Spark mund t'ju ofroj\u00eb burime shtes\u00eb. Nevojat llogariten sipas nj\u00eb formule t\u00eb zgjuar. Nuk do t\u00eb hyjm\u00eb n\u00eb detaje \u2014 punon mir\u00eb.<\/p>\n
spark.dynamicAllocation.enabled=true<\/code><\/pre>\n
\nE vendos\u00ebm k\u00ebt\u00eb paramet\u00ebr, dhe kur e filluam Spark, ai u ankuar dhe nuk nisi. E drejt\u00eb, sepse duhej t\u00eb lexohej dokumentacion<\/a><\/noindex> m\u00eb me v\u00ebmendje. Atje thot\u00eb se p\u00ebr ta pasur gjith\u00e7ka n\u00eb rregull, duhet t\u00eb aktivizoni nj\u00eb paramet\u00ebr shtes\u00eb.<\/p>\n
spark.shuffle.service.enabled=true<\/code><\/pre>\n
\nPse \u00ebsht\u00eb e nevojshme? Kur puna jon\u00eb nuk k\u00ebrkon m\u00eb kaq shum\u00eb burime, Spark duhet t'i kthej\u00eb ato n\u00eb rezerv\u00eb t\u00eb p\u00ebrgjithshme. Faza m\u00eb e pun\u00ebs intensiven n\u00eb \u00e7do detyr\u00eb MapReduce \u00ebsht\u00eb faza Shuffle. Ky paramet\u00ebr lejon q\u00eb t\u00eb ruhen t\u00eb dh\u00ebnat q\u00eb krijohen n\u00eb k\u00ebt\u00eb faz\u00eb dhe p\u00ebr pasoj\u00eb t\u00eb \u00e7liron executor\u00ebt. Nj\u00eb executor \u00ebsht\u00eb nj\u00eb proces q\u00eb llogarit gjith\u00e7ka n\u00eb pun\u00ebtor\u00eb. Ai ka nj\u00eb num\u00ebr t\u00eb caktuar b\u00ebrthama procesori dhe nj\u00eb sasi t\u00eb caktuar memories.<\/p>\n
E vendos\u00ebm k\u00ebt\u00eb paramet\u00ebr. Gjith\u00e7ka dukej se po funksiononte. U v\u00ebrejt se pjes\u00ebmarr\u00ebsit merrnin m\u00eb shum\u00eb burime kur u nevoiteshin. Por ndodhi nj\u00eb problem tjet\u00ebr \u2014 n\u00eb nj\u00eb moment, pjes\u00ebmarr\u00ebsit e tjer\u00eb u zgjuan dhe gjithashtu donin t\u00eb p\u00ebrdornin Spark, por e gjith\u00eb kapaciteti ishte i z\u00ebn\u00eb, dhe ata ishin t\u00eb pak\u00ebnaqur. Mund t\u00eb kuptohen. Filluam t\u00eb shikojm\u00eb dokumentacionin. Atje doli se kishte disa parametra t\u00eb tjer\u00eb, me t\u00eb cil\u00ebt mund t\u00eb ndikojm\u00eb n\u00eb proces. P\u00ebr shembull, n\u00ebse nj\u00eb executor \u00ebsht\u00eb n\u00eb modalitetin e zgjedhjes \u2014 pas sa kohe mund t'i merret burimet?<\/p>\n
spark.dynamicAllocation.executorIdleTimeout=120s<\/code><\/pre>\n
\nN\u00eb rastin ton\u00eb \u2014 n\u00ebse ekzekutor\u00ebt tuaj nuk b\u00ebjn\u00eb asgj\u00eb p\u00ebr dy minuta, ju lutemi, kthejini ata n\u00eb pool-in e p\u00ebrgjithsh\u00ebm. Por as ky paramet\u00ebr nuk ishte gjithmon\u00eb i mjaftuesh\u00ebm. Ishte e dukshme q\u00eb nj\u00eb njeri nuk po b\u00ebnte asgj\u00eb p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb, por burimet nuk po liroheshin. U zbulua se kishte edhe nj\u00eb paramet\u00ebr t\u00eb ve\u00e7ant\u00eb \u2014 pas sa kohe t\u00eb merreshin ekzekutor\u00ebt q\u00eb p\u00ebrmbanin t\u00eb dh\u00ebna t\u00eb ruajtura n\u00eb cache. N\u00eb default, ky paramet\u00ebr ishte vendosur \u2014 pafund\u00ebsi! Ne e ndryshuam at\u00eb.<\/p>\n
spark.dynamicAllocation.cachedExecutorIdleTimeout=600s<\/code><\/pre>\n
\nPra, n\u00ebse p\u00ebr 5 minuta ekzekutor\u00ebt tuaj nuk b\u00ebjn\u00eb asgj\u00eb, ktheni ata n\u00eb pool-in e p\u00ebrgjithsh\u00ebm. N\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb, shpejt\u00ebsia e lirimit dhe ndarjes s\u00eb burimeve p\u00ebr nj\u00eb num\u00ebr t\u00eb madh p\u00ebrdoruesish u b\u00eb e k\u00ebnaqshme. Numri i pak\u00ebnaq\u00ebsive u reduktua. Por ne vendos\u00ebm t\u00eb shkojm\u00eb p\u00ebrpara dhe t\u00eb kufizojm\u00eb numrin maksimal t\u00eb ekzekutor\u00ebve p\u00ebr nj\u00eb aplikacion \u2014 n\u00eb thelb p\u00ebr nj\u00eb pjes\u00ebmarr\u00ebs n\u00eb program.<\/p>\n
spark.dynamicAllocation.maxExecutors=19<\/code><\/pre>\n
\nTani, sigurisht, kan\u00eb dal\u00eb t\u00eb pak\u00ebnaqur nga ana tjet\u00ebr \u2014 \u201cklusteri \u00ebsht\u00eb i papun\u00eb, dhe un\u00eb kam vet\u00ebm 19 ekzekutor\u00eb\u201d, por \u00e7far\u00eb mund t\u00eb b\u00ebjm\u00eb \u2014 na nevojitet nj\u00eb balancim i duhur. T\u00eb gjith\u00eb t\u00eb b\u00ebhen t\u00eb lumtur nuk do t\u00eb jet\u00eb e mundur.<\/p>\n
Dhe nj\u00eb histori e vog\u00ebl tjet\u00ebr, e lidhur me specifik\u00ebn e rastit ton\u00eb. Disa njer\u00ebz mbet\u00ebn pas n\u00eb nj\u00eb praktik\u00eb, dhe p\u00ebr nj\u00eb arsye, Spark nuk nisi. Ne shikojm\u00eb numrin e burimeve t\u00eb lira \u2014 duket se ka. Spark duhet t\u00eb nis\u00eb. Fatmir\u00ebsisht, n\u00eb ato momente dokumentacioni tashm\u00eb ishte regjistruar n\u00eb n\u00ebnnd\u00ebrgjegje, dhe ne e kujtuam se kur Spark nis, ai k\u00ebrkon nj\u00eb port p\u00ebr t\u00eb nisur. N\u00ebse porta e par\u00eb nga diapazoni \u00ebsht\u00eb e z\u00ebn\u00eb, ai kalon n\u00eb portin tjet\u00ebr n\u00eb rend. N\u00ebse ai \u00ebsht\u00eb i lir\u00eb, ai e merr. Dhe ka nj\u00eb parametr q\u00eb tregon numrin maksimal t\u00eb p\u00ebrpjekjeve p\u00ebr k\u00ebt\u00eb. N\u00eb default \u2014 \u00ebsht\u00eb 16. Numri \u00ebsht\u00eb m\u00eb i vog\u00ebl se njer\u00ebzit n\u00eb grupin ton\u00eb n\u00eb seanc\u00eb. Si rezultat, pas 16 p\u00ebrpjekjesh, Spark dor\u00ebzonte k\u00ebt\u00eb pun\u00eb dhe thoshte se nuk mund t\u00eb nisi. Ne e kemi rregulluar k\u00ebt\u00eb parametr.<\/p>\n
spark.port.maxRetries=50<\/code><\/pre>\n
\nM\u00eb pas do t'ju tregoj p\u00ebr disa konfigurime, t\u00eb cilat nuk jan\u00eb shum\u00eb t\u00eb lidhura me specifik\u00ebn e rastit ton\u00eb.<\/p>\n
P\u00ebr nj\u00eb nisje m\u00eb t\u00eb shpejt\u00eb t\u00eb Spark, ka nj\u00eb rekomandim q\u00eb folderin jars, q\u00eb ndodhet n\u00eb direktorin\u00eb sht\u00ebpiake SPARK_HOME, ta kompresoni dhe ta vendosni n\u00eb HDFS. At\u00ebher\u00eb ai nuk do t\u00eb humbas\u00eb koh\u00eb n\u00eb shkarkimin e k\u00ebtyre jar-\u00ebve n\u00eb pun\u00ebtor\u00eb.<\/p>\n
spark.yarn.archive=hdfs:\/\/\/tmp\/spark-archive.zip<\/code><\/pre>\n
\nPo gjithashtu, p\u00ebr nj\u00eb pun\u00eb m\u00eb t\u00eb shpejt\u00eb, rekomandohet t\u00eb p\u00ebrdoret kryo si serializues. Ai \u00ebsht\u00eb m\u00eb i optimizuar se ai q\u00eb \u00ebsht\u00eb nga parazgjedhja.<\/p>\n
spark.serializer=org.apache.spark.serializer.KryoSerializer<\/code><\/pre>\n
\nDhe ka nj\u00eb problem t\u00eb vjet\u00ebr me Spark, se shpesh d\u00ebshton p\u00ebr shkak t\u00eb memory. Kjo ndodh shpesh n\u00eb momentin kur pun\u00ebtor\u00ebt kan\u00eb p\u00ebrfunduar gjith\u00e7ka dhe d\u00ebrgojn\u00eb rezultatin n\u00eb drejtues. Ne e kemi rritur k\u00ebt\u00eb paramet\u00ebr. Nga parazgjedhja \u00ebsht\u00eb 1GB, ne b\u00ebm\u00eb \u2014 3.<\/p>\n
spark.driver.maxResultSize=3072<\/code><\/pre>\n
\nDhe e fundit, si nj\u00eb \u00ebmb\u00eblsir\u00eb. Si t\u00eb p\u00ebrmir\u00ebsoni Spark n\u00eb versionin 2.1 mbi distribucionin HortonWorks \u2014 HDP 2.5.3.0. Ky version i HDP p\u00ebrmban versionin e parainstaluar 2.0, por nj\u00eb her\u00eb vendos\u00ebm se Spark po zhvillohet mjaft aktivisht, dhe \u00e7do version i ri rregullon disa gabime dhe gjithashtu ofron mund\u00ebsi t\u00eb tjera, p\u00ebrfshir\u00eb p\u00ebr API-n\u00eb python, prandaj vendos\u00ebm se duhet t\u00eb b\u00ebnim azhurnim.<\/p>\n
Kemi shkarkuar versionin nga faqja zyrtare p\u00ebr Hadoop 2.7. E kemi zgjidhur, e kemi futur n\u00eb dosjen me HDP. Vendos\u00ebm lidhjet si\u00e7 duhet. E nisim \u2014 nuk fillon. Shkruan nj\u00eb gabim shum\u00eb t\u00eb paqart\u00eb.<\/p>\n
java.lang.NoClassDefFoundError: com\/sun\/jersey\/api\/client\/config\/ClientConfig<\/code><\/pre>\n
\nPas k\u00ebrkimeve n\u00eb Google, zbuluam se Spark vendosi t\u00eb mos pres\u00eb derisa Hadoop t\u00eb ndihmoj\u00eb dhe vendosi t\u00eb p\u00ebrdor\u00eb versionin e ri t\u00eb jersey. Ata vet\u00eb diskutuan p\u00ebr k\u00ebt\u00eb tem\u00eb n\u00eb JIRA. \u00c7elja ishte \u2014 shkarko jersey versioni 1.17.1<\/a><\/noindex>. E vendos\u00ebm k\u00ebt\u00eb n\u00eb dosjen jars n\u00eb SPARK_HOME, p\u00ebrs\u00ebri b\u00ebm\u00eb zip dhe e d\u00ebrguam n\u00eb HDFS.<\/p>\n
K\u00ebt\u00eb gabim e kaluam, por u shfaq nj\u00eb e re dhe mjaft e paqart\u00eb.<\/p>\n
org.apache.spark.SparkException: Yarn application has already ended! Mund t\u00eb jet\u00eb mbyllur ose nuk ka mundur t\u00eb nis\u00eb master-in e aplikacionit.<\/code><\/pre>\n
\nNd\u00ebrkoh\u00eb, provojm\u00eb t\u00eb nisim versionin 2.0 \u2014 gjith\u00e7ka \u00ebsht\u00eb n\u00eb rregull. E kupto si duket. Ne u ngjit\u00ebm n\u00eb logjet e k\u00ebtij aplikacioni dhe pash\u00eb di\u00e7ka t\u00eb till\u00eb:<\/p>\n
\/usr\/hdp\/${hdp.version}\/hadoop\/lib\/hadoop-lzo-0.6.0.${hdp.version}.jar<\/code><\/pre>\n
\nN\u00eb p\u00ebrgjith\u00ebsi, p\u00ebr disa arsye hdp.version nuk u zgjidh. Pas k\u00ebrkimeve, gjet\u00ebm zgjidhjen. Duhet t\u00eb hyjm\u00eb n\u00eb Ambari n\u00eb cil\u00ebsimet YARN dhe t\u00eb shtojm\u00eb atje parametrin n\u00eb yarn-site-un e personalizuar:<\/p>\n
hdp.version=2.5.3.0-37<\/code><\/pre>\n
\nKjo magji ndihmoi, dhe Spark fluturoi. E provuam disa nga jupyter-notebook-et tona. Gjith\u00e7ka funksionon. Jemi gati p\u00ebr mbledhjen e par\u00eb p\u00ebr Spark t\u00eb shtun\u00ebn (tashm\u00eb nes\u00ebr)!<\/p>\n
UPD<\/b>. N\u00eb mbledhje doli nj\u00eb problem tjet\u00ebr. N\u00eb nj\u00eb moment YARN ndaloi s\u00eb dh\u00ebni kontejner\u00eb p\u00ebr Spark. N\u00eb YARN ne pat\u00ebm nevoj\u00eb t\u00eb rregullojm\u00eb parametrin, i cili p\u00ebr defolt ishte 0.2:<\/p>\n
yarn.scheduler.capacity.maximum-am-resource-percent=0.8<\/code><\/pre>\n
\nPra t'u th\u00ebn\u00eb, vet\u00ebm 20% e burimeve ishin t\u00eb angazhuara n\u00eb shp\u00ebrndarjen e burimeve. Duke ndryshuar parametrat, ne riluam YARN. Problemi u zgjidh dhe pjes\u00ebmarr\u00ebsit e tjer\u00eb gjithashtu mund\u00ebn t\u00eb nisnin kontekstin e spark.
\n
Burimi: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"
\u0425\u0430\u0431\u0440, \u043f\u0440\u0438\u0432\u0435\u0442! \u0412\u0447\u0435\u0440\u0430 \u043d\u0430 \u043c\u0438\u0442\u0430\u043f\u0435, \u043f\u043e\u0441\u0432\u044f\u0449\u0435\u043d\u043d\u043e\u043c Apache Spark, \u043e\u0442 \u0440\u0435\u0431\u044f\u0442 \u0438\u0437 Rambler&Co, \u0431\u044b\u043b\u043e \u0434\u043e\u0432\u043e\u043b\u044c\u043d\u043e \u043c\u043d\u043e\u0433\u043e \u0432\u043e\u043f\u0440\u043e\u0441\u043e\u0432 \u043e\u0442 \u0443\u0447\u0430\u0441\u0442\u043d\u0438\u043a\u043e\u0432, \u0441\u0432\u044f\u0437\u0430\u043d\u043d\u044b\u0445 \u0441 \u043a\u043e\u043d\u0444\u0438\u0433\u0443\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u044d\u0442\u043e\u0433\u043e \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u0430. \u0420\u0435\u0448\u0438\u043b\u0438 \u043f\u043e \u0435\u0433\u043e \u0441\u043b\u0435\u0434\u0430\u043c \u043f\u043e\u0434\u0435\u043b\u0438\u0442\u044c\u0441\u044f \u0441\u0432\u043e\u0438\u043c \u043e\u043f\u044b\u0442\u043e\u043c. \u0422\u0435\u043c\u0430 \u043d\u0435\u043f\u0440\u043e\u0441\u0442\u0430\u044f \u2014 \u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c \u0434\u0435\u043b\u0438\u0442\u044c\u0441\u044f \u043e\u043f\u044b\u0442\u043e\u043c \u0442\u043e\u0436\u0435 \u0432 \u043a\u043e\u043c\u043c\u0435\u043d\u0442\u0430\u0440\u0438\u044f\u0445, \u043c\u043e\u0436\u0435\u0442 \u0431\u044b\u0442\u044c, \u043c\u044b \u0442\u043e\u0436\u0435 \u0447\u0442\u043e-\u0442\u043e \u043d\u0435 \u0442\u0430\u043a \u043f\u043e\u043d\u0438\u043c\u0430\u0435\u043c \u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c. \u041d\u0435\u0431\u043e\u043b\u044c\u0448\u0430\u044f \u0432\u0432\u043e\u0434\u043d\u0430\u044f \u2014 \u043a\u0430\u043a \u043c\u044b […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-30548","post","type-post","status-publish","format-standard","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Konfigurimi i Spark n\u00eb YARN | ProHoster","description":"Habr, p\u00ebrsh\u00ebndetje!","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/news\/konfigurirovanie-spark-na-yarn","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041a\u043e\u043d\u0444\u0438\u0433\u0443\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 Spark \u043d\u0430 YARN | ProHoster","og:description":"\u0425\u0430\u0431\u0440, \u043f\u0440\u0438\u0432\u0435\u0442!","og:url":"https:\/\/prohoster.info\/sq\/blog\/news\/konfigurirovanie-spark-na-yarn","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T18:36:08+00:00","article:modified_time":"2019-10-31T18:36:08+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"30548","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-21 01:45:22","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 03:33:26","updated":"2026-01-21 01:45:22","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/30548","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=30548"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/30548\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=30548"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=30548"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=30548"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}