{"id":82038,"date":"2020-05-19T01:42:26","date_gmt":"2020-05-18T23:42:26","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/pogruzhenie-v-delta-lake-prinuditelnoe-primenenie-i-evolyucziya-shemy"},"modified":"2020-05-19T01:42:26","modified_gmt":"2020-05-18T23:42:26","slug":"pogruzhenie-v-delta-lake-prinuditelnoe-primenenie-i-evolyucziya-shemy","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/pogruzhenie-v-delta-lake-prinuditelnoe-primenenie-i-evolyucziya-shemy","title":{"rendered":"The Role of Delta Lake: Schema Enforcement and Evolution","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><i><b>P\u00ebrsh\u00ebndetje, Habr! Po ju prezantoj nj\u00eb p\u00ebrkthim t\u00eb artikullit <noindex><a rel=\"nofollow\" href=\"https:\/\/databricks.com\/blog\/2019\/09\/24\/diving-into-delta-lake-schema-enforcement-evolution.html\">\u00abImmersing Into Delta Lake: Schema Enforcement &amp; Evolution\u00bb<\/a><\/noindex> by Burak Yavuz, Brenner Heintz, and Denny Lee, prepared ahead of the course launch <noindex><a rel=\"nofollow\" href=\"https:\/\/otus.pw\/J1P5\/\">Inxhinier i t\u00eb Dh\u00ebnave<\/a><\/noindex> from OTUS.<\/b><\/i><\/p>\n<p><img decoding=\"async\" alt=\"The Role of Delta Lake: Schema Enforcement and Evolution\" src=\"\/wp-content\/uploads\/2020\/05\/1d20921d8d4a1c1e0f8afee4fc019024.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>Data, like our experience, is constantly accumulating and evolving. To keep pace, our mental models of the world must adapt to new data, some of which contain new dimensions \u2014 new ways to observe things we previously had no idea about. These mental models are little different from table schemas that define how we classify and process new information.<\/p>\n<p>This brings us to the question of schema management. As business tasks and requirements change over time, so does the structure of your data. Delta Lake allows new dimensions to be easily introduced as data changes. Users have access to simple semantics for managing the schemas of their tables. These tools include schema enforcement, which protects users from inadvertently cluttering their tables with mistakes or unnecessary data, and schema evolution, which allows the automatic addition of new columns with valuable data in appropriate places. In this article, we will delve into the use of these tools.<\/p>\n<h2>Understanding Table Schemas<\/h2>\n<p>\nEvery DataFrame in Apache Spark has a schema that defines the form of the data, such as data types, columns, and metadata. With Delta Lake, the table schema is stored in JSON format within the transaction log.<br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2>What is Schema Enforcement?<\/h2>\n<p>\nSchema enforcement, also known as schema validation, is a protective mechanism in Delta Lake that ensures data quality by rejecting records that do not conform to the table schema. Much like a host at a restaurant who only accepts reservations, it checks if each data column being entered into the table is on the expected list of columns (in other words, if each has a 'reservation'), and rejects any records with columns not on the list.<\/p>\n<h2>How does schema enforcement work?<\/h2>\n<p>\nDelta Lake p\u00ebrdor verifikimin e skem\u00ebs gjat\u00eb shkrimit, q\u00eb do t\u00eb thot\u00eb se t\u00eb gjitha shkrimet e reja n\u00eb tabel\u00eb verifikohen p\u00ebr p\u00ebrputhshm\u00ebri me skem\u00ebn e tabel\u00ebs s\u00eb q\u00ebllimit gjat\u00eb shkrimit. N\u00ebse skema nuk \u00ebsht\u00eb e p\u00ebrputhshme, Delta Lake anulon plot\u00ebsisht transaksionin (t\u00eb dh\u00ebnat nuk shkruhen) dhe krijon nj\u00eb p\u00ebrjashtim p\u00ebr t\u00eb informuar p\u00ebrdoruesin mbi mos p\u00ebrputhshm\u00ebrin\u00eb.<br \/>\nP\u00ebr t\u00eb p\u00ebrcaktuar p\u00ebrputhshm\u00ebrin\u00eb e shkrimit me tabel\u00ebn, Delta Lake p\u00ebrdor rregullat e m\u00ebposhtme. DataFrame q\u00eb po shkruhet:<\/p>\n<ul>\n<li>nuk mund t\u00eb p\u00ebrmbaj\u00eb kolona shtes\u00eb, t\u00eb cilat nuk gjenden n\u00eb skem\u00ebn e tabel\u00ebs s\u00eb q\u00ebllimit. Anasjelltas, \u00ebsht\u00eb n\u00eb rregull n\u00ebse t\u00eb dh\u00ebnat e ardhura nuk p\u00ebrmbajn\u00eb t\u00eb gjitha kolonat nga tabela \u2013 k\u00ebtyre kolonave thjesht do t'u jepet nj\u00eb vler\u00eb zero.<\/li>\n<li>nuk mund t\u00eb ket\u00eb tipe t\u00eb dh\u00ebnash t\u00eb kolonave q\u00eb diferencohen nga tipet e t\u00eb dh\u00ebnave t\u00eb kolonave n\u00eb tabel\u00ebn e q\u00ebllimit. N\u00ebse nj\u00eb kolon\u00eb n\u00eb tabel\u00ebn e q\u00ebllimit p\u00ebrmban t\u00eb dh\u00ebna t\u00eb tipit StringType, por kolona p\u00ebrkat\u00ebse n\u00eb DataFrame p\u00ebrmban t\u00eb dh\u00ebna t\u00eb tipit IntegerType, imponimi i skem\u00ebs do t\u00eb shkaktoj\u00eb nj\u00eb p\u00ebrjashtim dhe do t\u00eb parandaloj\u00eb ekzekutimin e operacionit t\u00eb shkrimit.<\/li>\n<li>\u043d\u0435 \u043c\u043e\u0436\u0435\u0442 \u0441\u043e\u0434\u0435\u0440\u0436\u0430\u0442\u044c \u0438\u043c\u0435\u043d\u0430 \u0441\u0442\u043e\u043b\u0431\u0446\u043e\u0432, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043e\u0442\u043b\u0438\u0447\u0430\u044e\u0442\u0441\u044f \u0442\u043e\u043b\u044c\u043a\u043e \u0440\u0435\u0433\u0438\u0441\u0442\u0440\u043e\u043c. \u042d\u0442\u043e \u0437\u043d\u0430\u0447\u0438\u0442, \u0447\u0442\u043e \u0432\u044b \u043d\u0435 \u043c\u043e\u0436\u0435\u0442\u0435 \u0438\u043c\u0435\u0442\u044c \u0441\u0442\u043e\u043b\u0431\u0446\u044b \u0441 \u0438\u043c\u0435\u043d\u0430\u043c\u0438 &#8216;Foo&#8217; \u0438 &#8216;foo&#8217;, \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u043d\u044b\u0435 \u0432 \u043e\u0434\u043d\u043e\u0439 \u0442\u0430\u0431\u043b\u0438\u0446\u0435. \u0425\u043e\u0442\u044f Spark \u043c\u043e\u0436\u043d\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u0432 \u0447\u0443\u0432\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u043c \u0438\u043b\u0438 \u043d\u0435\u0447\u0443\u0432\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u043c (\u043f\u043e \u0443\u043c\u043e\u043b\u0447\u0430\u043d\u0438\u044e) \u043a \u0440\u0435\u0433\u0438\u0441\u0442\u0440\u0443 \u0440\u0435\u0436\u0438\u043c\u0435, Delta Lake \u0441\u043e\u0445\u0440\u0430\u043d\u044f\u0435\u0442 \u0440\u0435\u0433\u0438\u0441\u0442\u0440, \u043d\u043e \u043d\u0435\u0447\u0443\u0432\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u0435\u043d \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0438 \u0441\u0445\u0435\u043c\u044b. Parquet \u0447\u0443\u0432\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u0435\u043d \u043a \u0440\u0435\u0433\u0438\u0441\u0442\u0440\u0443 \u043f\u0440\u0438 \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0438 \u0438 \u0432\u043e\u0437\u0432\u0440\u0430\u0442\u0435 \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u0438 \u0441\u0442\u043e\u043b\u0431\u0446\u0430. \u0427\u0442\u043e\u0431\u044b \u0438\u0437\u0431\u0435\u0436\u0430\u0442\u044c \u0432\u043e\u0437\u043c\u043e\u0436\u043d\u044b\u0445 \u043e\u0448\u0438\u0431\u043e\u043a, \u043f\u043e\u0432\u0440\u0435\u0436\u0434\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445 \u0438\u043b\u0438 \u0438\u0445 \u043f\u043e\u0442\u0435\u0440\u0438 (\u0441 \u0447\u0435\u043c \u043c\u044b \u043b\u0438\u0447\u043d\u043e \u0441\u0442\u0430\u043b\u043a\u0438\u0432\u0430\u043b\u0438\u0441\u044c \u0432 Databricks), \u043c\u044b \u0440\u0435\u0448\u0438\u043b\u0438 \u0434\u043e\u0431\u0430\u0432\u0438\u0442\u044c \u044d\u0442\u043e \u043e\u0433\u0440\u0430\u043d\u0438\u0447\u0435\u043d\u0438\u0435.<\/li>\n<\/ul>\n<p>\nP\u00ebr t\u00eb ilustruar k\u00ebt\u00eb, le t\u00eb shohim se \u00e7far\u00eb ndodh n\u00eb kodin e m\u00ebposht\u00ebm kur p\u00ebrpiqemi t\u00eb shtojm\u00eb disa kolona t\u00eb reja t\u00eb gjeneruara n\u00eb tabel\u00ebn Delta Lake, e cila ende nuk \u00ebsht\u00eb konfiguruar p\u00ebr t'i pranuar ato.<\/p>\n<pre><code class=\"apache\"># \u0421\u0433\u0435\u043d\u0435\u0440\u0438\u0440\u0443\u0435\u043c DataFrame \u0441\u0441\u0443\u0434, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u043c\u044b \u0434\u043e\u0431\u0430\u0432\u0438\u043c \u0432 \u043d\u0430\u0448\u0443 \u0442\u0430\u0431\u043b\u0438\u0446\u0443 Delta Lake\nloans = sql(&quot;&quot;&quot;\n            SELECT addr_state, CAST(rand(10)*count as bigint) AS count,\n            CAST(rand(10) * 10000 * count AS double) AS amount\n            FROM loan_by_state_delta\n            &quot;&quot;&quot;)\n\n# \u0412\u044b\u0432\u0435\u0441\u0442\u0438 \u0438\u0441\u0445\u043e\u0434\u043d\u0443\u044e \u0441\u0445\u0435\u043c\u0443 DataFrame\noriginal_loans.printSchema()\n\nroot\n  |-- addr_state: string (nullable = true)\n  |-- count: integer (nullable = true)\n \n# \u0412\u044b\u0432\u0435\u0441\u0442\u0438 \u043d\u043e\u0432\u0443\u044e \u0441\u0445\u0435\u043c\u0443 DataFrame\nloans.printSchema()\n \nroot\n  |-- addr_state: string (nullable = true)\n  |-- count: integer (nullable = true)\n  |-- amount: double (nullable = true) # new column\n \n# \u041f\u043e\u043f\u044b\u0442\u043a\u0430 \u0434\u043e\u0431\u0430\u0432\u0438\u0442\u044c \u043d\u043e\u0432\u044b\u0439 DataFrame (\u0441 \u043d\u043e\u0432\u044b\u043c \u0441\u0442\u043e\u043b\u0431\u0446\u043e\u043c) \u0432 \u0441\u0443\u0449\u0435\u0441\u0442\u0432\u0443\u044e\u0449\u0443\u044e \u0442\u0430\u0431\u043b\u0438\u0446\u0443\nloans.write.format(&quot;delta&quot;) \n           .mode(&quot;append&quot;) \n           .save(DELTALAKE_PATH)\n\nReturns:\n\nA schema mismatch detected when writing to the Delta table.\n \nTo enable schema migration, please set:\n'.option(&quot;mergeSchema&quot;, &quot;true&quot;)'\n \nTable schema:\nroot\n-- addr_state: string (nullable = true)\n-- count: long (nullable = true)\n \nData schema:\nroot\n-- addr_state: string (nullable = true)\n-- count: long (nullable = true)\n-- amount: double (nullable = true)\n \nIf Table ACLs are enabled, these options will be ignored. Please use the ALTER TABLE command for changing the schema.<\/code><\/pre>\n<p>\nN\u00eb vend q\u00eb t\u00eb shtoj\u00eb automatikisht kolonat e reja, Delta Lake imponon skem\u00ebn dhe ndalon shkrimin. P\u00ebr t\u00eb ndihmuar n\u00eb p\u00ebrcaktimin se cila kolon\u00eb (ose disa prej tyre) \u00ebsht\u00eb shkaku i mos p\u00ebrputhshm\u00ebris\u00eb, Spark nxjerr t\u00eb dy skemat nga stack trace p\u00ebr krahasim.<\/p>\n<h2>Cila \u00ebsht\u00eb dobi e imponimit t\u00eb skem\u00ebs?<\/h2>\n<p>\nPasi aplikimi i detyruesh\u00ebm i skemave paraqet nj\u00eb kontroll t\u00eb mjaftuesh\u00ebm t\u00eb rrept\u00eb, ai \u00ebsht\u00eb nj\u00eb mjet i shk\u00eblqyer p\u00ebr t'u p\u00ebrdorur si nj\u00eb portier i nj\u00eb grupi t\u00eb dh\u00ebnash t\u00eb past\u00ebr dhe t\u00eb transformuar plot\u00ebsisht, i gatsh\u00ebm p\u00ebr prodhim ose konsum. Si rregull, aplikohet n\u00eb tabela q\u00eb drejtp\u00ebrdrejt ofrojn\u00eb t\u00eb dh\u00ebna:<\/p>\n<ul>\n<li>Algoritmet e m\u00ebsimit t\u00eb makinerive<\/li>\n<li>Panele BI<\/li>\n<li>Analiza e t\u00eb dh\u00ebnave dhe mjetet e vizualizimit<\/li>\n<li>\u00c7do sistem prodhimi q\u00eb k\u00ebrkon skema semantike shum\u00eb t\u00eb strukturuara dhe shum\u00eb t\u00eb tipizuara.<\/li>\n<\/ul>\n<p>\nP\u00ebr t\u00eb p\u00ebrgatitur t\u00eb dh\u00ebnat tuaja p\u00ebr k\u00ebt\u00eb barrier\u00eb p\u00ebrfundimtare, shum\u00eb p\u00ebrdorues p\u00ebrdorin nj\u00eb arkitektur\u00eb t\u00eb thjesht\u00eb \u201cmulti-hop\u201d, e cila gradualisht sjell struktur\u00eb n\u00eb tabelat e tyre. P\u00ebr t\u00eb m\u00ebsuar m\u00eb shum\u00eb rreth k\u00ebsaj, mund t\u00eb konsultoheni me artikullin <noindex><a rel=\"nofollow\" href=\"https:\/\/databricks.com\/blog\/2019\/08\/14\/productionizing-machine-learning-with-delta-lake.html\">M\u00ebsimi i makinave n\u00eb nivelin e prodhimit me Delta Lake.<\/a><\/noindex><\/p>\n<p>Sigurisht, aplikimi i detyruesh\u00ebm i skemave mund t\u00eb p\u00ebrdoret kudo n\u00eb pipelin\u00ebn tuaj, por mbani mend se regjistrimi n\u00eb rrjedh\u00eb n\u00eb tabel\u00eb n\u00eb k\u00ebt\u00eb rast mund t\u00eb jet\u00eb frustrues, p\u00ebr shkak se, p\u00ebr shembull, keni harruar se keni shtuar nj\u00eb kolon\u00eb t\u00eb re n\u00eb t\u00eb dh\u00ebnat hyr\u00ebse.<\/p>\n<h2>Parandalimi i hollimit t\u00eb t\u00eb dh\u00ebnave<\/h2>\n<p>\nN\u00eb k\u00ebt\u00eb pik\u00eb, mund t\u00eb pyesni veten se \u00e7far\u00eb \u00ebsht\u00eb gjith\u00eb ky entuziaz\u00ebm? Pas gjith\u00eb k\u00ebsaj, ndonj\u00ebher\u00eb nj\u00eb gabim i papritur \"mosp\u00ebrputhje skemash\" mund t'ju v\u00ebr\u00eb pengesa n\u00eb procesin tuaj t\u00eb pun\u00ebs, ve\u00e7an\u00ebrisht n\u00ebse jeni fillestar n\u00eb Delta Lake. Pse thjesht t\u00eb mos lejojm\u00eb skem\u00ebn t\u00eb ndryshoj\u00eb si\u00e7 \u00ebsht\u00eb e nevojshme p\u00ebr t\u00eb mund\u00ebsuar shkrimin tim t\u00eb DataFrame, pa marr\u00eb parasysh?<\/p>\n<p>Si\u00e7 thot\u00eb nj\u00eb shprehje e vjet\u00ebr, \"nj\u00eb ons\u00eb parandalimi \u00ebsht\u00eb nj\u00eb pound kurimi.\" N\u00eb nj\u00eb moment, n\u00ebse nuk kujdeseni p\u00ebr aplikimin e skem\u00ebs tuaj, do t\u00eb lindin probleme t\u00eb dhimbshme me p\u00ebrputhshm\u00ebrin\u00eb e tipeve t\u00eb t\u00eb dh\u00ebnave \u2014 gjithashtu burimet e dukshme t\u00eb papiruar t\u00eb dh\u00ebnash mund t\u00eb p\u00ebrmbajn\u00eb raste t\u00eb kufizuara, kolona t\u00eb d\u00ebmtuara, p\u00ebrfytyrime t\u00eb formuara keq ose gj\u00ebra t\u00eb tjera t\u00eb frikshme q\u00eb ndodhin n\u00eb pesimizmat tuaj. Qasja m\u00eb e mir\u00eb \u00ebsht\u00eb t\u00eb ndaloni k\u00ebta armiq te portat \u2014 p\u00ebrmes aplikimit t\u00eb detyruesh\u00ebm t\u00eb skemave \u2014 dhe t\u00eb merremi me ta n\u00eb drit\u00eb, dhe jo m\u00eb von\u00eb, kur ata fillojn\u00eb t\u00eb p\u00ebrhapen n\u00eb thell\u00ebsit\u00eb e err\u00ebta t\u00eb kodit tuaj t\u00eb pun\u00ebs.<\/p>\n<p>Aplikimi i detyruesh\u00ebm i skem\u00ebs ofron sigurin\u00eb q\u00eb skema e tabel\u00ebs suaj nuk do t\u00eb ndryshoj\u00eb, p\u00ebrve\u00e7 n\u00ebse e konfirmoni vet\u00eb ndryshimin. Kjo parandalon 'njollosjen' (dilution) e t\u00eb dh\u00ebnave, e cila ndodh kur kolonat e reja shtohen aq shpesh, sa tabelat e m\u00ebparshme, t\u00eb vlefshme dhe t\u00eb kompresuara humbasin vler\u00ebn dhe p\u00ebrdorshm\u00ebrin\u00eb e tyre p\u00ebr shkak t\u00eb p\u00ebrmbytjes nga t\u00eb dh\u00ebnat. Duke ju inkurajuar t\u00eb jeni t\u00eb q\u00ebllimsh\u00ebm, t\u00eb vendosni standarde t\u00eb larta dhe t\u00eb prisni cil\u00ebsi t\u00eb lart\u00eb, aplikimi i detyruesh\u00ebm i skem\u00ebs e b\u00ebn pik\u00ebrisht at\u00eb p\u00ebr \u00e7far\u00eb \u00ebsht\u00eb t\u00eb destinuar - t\u00eb ndihmoj\u00eb q\u00eb ju t\u00eb mbani nj\u00eb q\u00ebndrim t\u00eb ndersh\u00ebm dhe tabelat tuaja t\u00eb mbeten t\u00eb pastra.<\/p>\n<p>N\u00ebse gjat\u00eb shqyrtimit t\u00eb m\u00ebtejsh\u00ebm vendosni se ju nevojitet <i>t\u00eb shtoni nj\u00eb kolon\u00eb t\u00eb re - asnj\u00eb problem, m\u00eb posht\u00eb \u00ebsht\u00eb nj\u00eb zgjidhje e thjesht\u00eb nj\u00eb-linj\u00ebshe. Zgjidhja \u00ebsht\u00eb evolucioni i skem\u00ebs!<\/i> \u00c7far\u00eb \u00ebsht\u00eb evolucioni i skem\u00ebs?<\/p>\n<h2>Evolucioni i skem\u00ebs \u00ebsht\u00eb nj\u00eb funksion q\u00eb lejon p\u00ebrdoruesit t\u00eb ndryshojn\u00eb leht\u00ebsisht skem\u00ebn aktuale t\u00eb tabel\u00ebs n\u00eb p\u00ebrputhje me t\u00eb dh\u00ebnat q\u00eb ndryshojn\u00eb me kalimin e koh\u00ebs. P\u00ebrdoret m\u00eb s\u00eb shpeshti n\u00eb operacionet e shtimit ose t\u00eb rid\u00ebshirimit, p\u00ebr t\u00eb p\u00ebrshtatur automatikisht skem\u00ebn p\u00ebr t\u00eb p\u00ebrfshir\u00eb nj\u00eb ose disa kolona t\u00eb reja.<\/h2>\n<p>\nSi funksionon evolucioni i skem\u00ebs?<\/p>\n<h2>Duke ndjekur shembullin nga seksioni i m\u00ebparsh\u00ebm, zhvilluesit mund t\u00eb p\u00ebrdorin leht\u00ebsisht evolucionin e skem\u00ebs p\u00ebr t\u00eb shtuar kolona t\u00eb reja q\u00eb m\u00eb par\u00eb ishin refuzuar p\u00ebr shkak t\u00eb moskonsistenc\u00ebs me skem\u00ebn. Evolucioni i skem\u00ebs aktivizohet duke shtuar<\/h2>\n<p>\n.option('mergeSchema', 'true') <code>n\u00eb komand\u00ebn tuaj Spark<\/code> .write ose .writeStream. <code>P\u00ebr t\u00eb par\u00eb grafikun, ju lutemi ekzekutoni k\u00ebt\u00eb k\u00ebrkes\u00eb Spark SQL<\/code><\/p>\n<pre><code class=\"apache\"># \u0414\u043e\u0431\u0430\u0432\u044c\u0442\u0435 \u043f\u0430\u0440\u0430\u043c\u0435\u0442\u0440 mergeSchema\nloans.write.format(&quot;delta&quot;) \n           .option(&quot;mergeSchema&quot;, &quot;true&quot;) \n           .mode(&quot;append&quot;) \n           .save(DELTALAKE_SILVER_PATH)<\/code><\/pre>\n<p>\nSi nj\u00eb alternativ\u00eb, mund ta vendosni k\u00ebt\u00eb opsion p\u00ebr t\u00ebr\u00eb sesionin Spark, duke shtuar<\/p>\n<pre><code class=\"apache\"># \u0421\u043e\u0437\u0434\u0430\u0439\u0442\u0435 \u0433\u0440\u0430\u0444\u0438\u043a \u0441 \u043d\u043e\u0432\u044b\u043c \u0441\u0442\u043e\u043b\u0431\u0446\u043e\u043c, \u0447\u0442\u043e\u0431\u044b \u043f\u043e\u0434\u0442\u0432\u0435\u0440\u0434\u0438\u0442\u044c, \u0447\u0442\u043e \u0437\u0430\u043f\u0438\u0441\u044c \u043f\u0440\u043e\u0448\u043b\u0430 \u0443\u0441\u043f\u0435\u0448\u043d\u043e\n%sql\nSELECT addr_state, sum(`amount`) AS amount\nFROM loan_by_state_delta\nGROUP BY addr_state\nORDER BY sum(`amount`)\nDESC LIMIT 10<\/code><\/pre>\n<p>\n<img decoding=\"async\" alt=\"The Role of Delta Lake: Schema Enforcement and Evolution\" src=\"\/wp-content\/uploads\/2020\/05\/873ce67da00f69696d5b9328bd90793a.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\nspark.databricks.delta.schema.autoMerge = True <code>n\u00eb konfigurimin e Spark. Por p\u00ebrdoreni k\u00ebt\u00eb me kujdes, pasi aplikimi i detyruesh\u00ebm i skem\u00ebs nuk do t'ju paralajm\u00ebroj\u00eb m\u00eb p\u00ebr moskonsistencat e paq\u00ebllimshme me skem\u00ebn.<\/code> Duke p\u00ebrfshir\u00eb parametrin<\/p>\n<p>mergeSchema <code>n\u00eb k\u00ebrkes\u00eb, t\u00eb gjitha kolonat q\u00eb jan\u00eb t\u00eb pranishme n\u00eb DataFrame, por mungojn\u00eb n\u00eb tabel\u00ebn e synuar, shtohen automatikisht n\u00eb fund t\u00eb skem\u00ebs si pjes\u00eb e transaksionit t\u00eb shkrimit. Po ashtu mund t\u00eb shtohen fusha t\u00eb p\u00ebrfshira, dhe ato gjithashtu do t\u00eb shtohen n\u00eb fund t\u00eb kolonave p\u00ebrkat\u00ebse t\u00eb struktur\u00ebs.<\/code>, \u0432\u0441\u0435 \u0441\u0442\u043e\u043b\u0431\u0446\u044b, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043f\u0440\u0438\u0441\u0443\u0442\u0441\u0442\u0432\u0443\u044e\u0442 \u0432 DataFrame, \u043d\u043e \u043e\u0442\u0441\u0443\u0442\u0441\u0442\u0432\u0443\u044e\u0442 \u0432 \u0446\u0435\u043b\u0435\u0432\u043e\u0439 \u0442\u0430\u0431\u043b\u0438\u0446\u0435, \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0447\u0435\u0441\u043a\u0438 \u0434\u043e\u0431\u0430\u0432\u043b\u044f\u044e\u0442\u0441\u044f \u0432 \u043a\u043e\u043d\u0435\u0446 \u0441\u0445\u0435\u043c\u044b \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 \u0442\u0440\u0430\u043d\u0437\u0430\u043a\u0446\u0438\u0438 \u0437\u0430\u043f\u0438\u0441\u0438. \u0422\u0430\u043a\u0436\u0435 \u043c\u043e\u0433\u0443\u0442 \u0431\u044b\u0442\u044c \u0434\u043e\u0431\u0430\u0432\u043b\u0435\u043d\u044b \u0432\u043b\u043e\u0436\u0435\u043d\u043d\u044b\u0435 \u043f\u043e\u043b\u044f, \u0438 \u043e\u043d\u0438 \u0442\u0430\u043a\u0436\u0435 \u0431\u0443\u0434\u0443\u0442 \u0434\u043e\u0431\u0430\u0432\u043b\u0435\u043d\u044b \u0432 \u043a\u043e\u043d\u0435\u0446 \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0443\u044e\u0449\u0438\u0445 \u0441\u0442\u043e\u043b\u0431\u0446\u043e\u0432 \u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u044b.<\/p>\n<p>Inxhinier\u00ebt dhe shkenc\u00ebtar\u00ebt mund t\u00eb p\u00ebrdorin k\u00ebt\u00eb opsion p\u00ebr t\u00eb shtuar kolonat e reja (ndoshta nj\u00eb metrik\u00eb t\u00eb ndjekur rishtazi ose nj\u00eb kolone treguesish p\u00ebr shitjet k\u00ebt\u00eb muaj) n\u00eb tabelat e tyre ekzistuese t\u00eb prodhimit t\u00eb m\u00ebsimit t\u00eb makinerive, pa e prishur modelet ekzistuese q\u00eb bazohen n\u00eb kolonat e vjetra. <\/p>\n<p>Llojet e m\u00ebposhtme t\u00eb ndryshimeve t\u00eb skem\u00ebs jan\u00eb t\u00eb lejuara n\u00eb kuadrin e evolucionit t\u00eb skem\u00ebs gjat\u00eb shtimit ose rihapjes s\u00eb tabel\u00ebs:<\/p>\n<ul>\n<li>Shtimi i kolonave t\u00eb reja (kjo \u00ebsht\u00eb skenari m\u00eb i zakonsh\u00ebm)<\/li>\n<li>Ndryshimi i llojeve t\u00eb t\u00eb dh\u00ebnave nga NullType -&gt; \u00e7do lloj tjet\u00ebr ose rritja nga ByteType -&gt; ShortType -&gt; IntegerType<\/li>\n<\/ul>\n<p>\nNdryshime t\u00eb tjera, t\u00eb papranueshme n\u00eb kuadrin e evolucionit t\u00eb skem\u00ebs, k\u00ebrkojn\u00eb q\u00eb skema dhe t\u00eb dh\u00ebnat t\u00eb rivendosen duke shtuar <code>.option(&quot;overwriteSchema&quot;, &quot;true&quot;)<\/code>. P\u00ebr shembull, n\u00eb rastin kur kolona \"Foo\" fillimisht ishte integer dhe skema e re do t\u00eb ishte e tipit string, at\u00ebher\u00eb t\u00eb gjith\u00eb skedar\u00ebt Parquet (t\u00eb dh\u00ebnat) duhet t\u00eb rivendosen. Ndryshimet e tilla p\u00ebrfshijn\u00eb:<\/p>\n<ul>\n<li>fshirjen e kolon\u00ebs<\/li>\n<li>ndryshimin e llojit t\u00eb t\u00eb dh\u00ebnave t\u00eb nj\u00eb kolone ekzistuese (n\u00eb vend)<\/li>\n<li>rilodhjen e kolonave q\u00eb ndryshojn\u00eb vet\u00ebm n\u00eb regjistrin (p.sh., \"Foo\" dhe \"foo\")<\/li>\n<\/ul>\n<p>\nS\u00eb fundi, me l\u00ebshimin e ardhsh\u00ebm t\u00eb Spark 3.0, do t\u00eb p\u00ebrkrah\u00eb plot\u00ebsisht DDL t\u00eb qart\u00eb (duke p\u00ebrdorur ALTER TABLE), q\u00eb do t\u00eb lejoj\u00eb p\u00ebrdoruesit t\u00eb realizojn\u00eb veprimet e m\u00ebposhtme mbi skemat e tabelave:<\/p>\n<ul>\n<li>shtimi i kolonave<\/li>\n<li>ndryshimi i komenteve p\u00ebr kolonat<\/li>\n<li>konfigurimi i pronave t\u00eb tabel\u00ebs q\u00eb p\u00ebrcaktojn\u00eb sjelljen e tabel\u00ebs, p\u00ebr shembull, vendosja e koh\u00ebzgjatjes s\u00eb ruajtjes s\u00eb regjistrit t\u00eb transaksioneve.<\/li>\n<\/ul>\n<p><\/p>\n<h2>Cila \u00ebsht\u00eb dobi e evolucionit t\u00eb skem\u00ebs?<\/h2>\n<p>\nEvolucioni i skem\u00ebs mund t\u00eb p\u00ebrdoret gjithmon\u00eb kur ju <i>keni nd\u00ebrmend<\/i> t\u00eb ndryshoni skem\u00ebn e tabel\u00ebs tuaj (n\u00eb kontrast me ato raste kur aksidentalisht keni shtuar n\u00eb DataFrame-in tuaj kolona q\u00eb nuk duhej t\u00eb ishin atje). Kjo \u00ebsht\u00eb m\u00ebnyra m\u00eb e thjesht\u00eb p\u00ebr t\u00eb migruar skem\u00ebn tuaj, sepse automatikisht shton emrat e duhur t\u00eb kolonave dhe llojet e t\u00eb dh\u00ebnave pa nevoj\u00ebn p\u00ebr t'i shpallur ato qart\u00eb.<\/p>\n<h2>P\u00ebrfundim<\/h2>\n<p>\nZbatimi i detyruesh\u00ebm i skem\u00ebs hedh posht\u00eb \u00e7do kolone t\u00eb re ose ndryshime t\u00eb tjera t\u00eb skem\u00ebs q\u00eb nuk jan\u00eb t\u00eb p\u00ebrshtatshme p\u00ebr tabel\u00ebn tuaj. Duke vendosur dhe mbajtur k\u00ebto standarde t\u00eb larta, analist\u00ebt dhe inxhinier\u00ebt mund t\u00eb mb\u00ebshteten n\u00eb t\u00eb dh\u00ebnat e tyre q\u00eb kan\u00eb nj\u00eb nivel t\u00eb lart\u00eb integriteti, duke e argumentuar k\u00ebt\u00eb qart\u00eb dhe sakt\u00eb, duke u mund\u00ebsuar atyre t\u00eb marrin vendime m\u00eb efektive biznesi.<\/p>\n<p>Nga ana tjet\u00ebr, evolucioni i skem\u00ebs plot\u00ebson zbatimin e detyruesh\u00ebm, duke e b\u00ebr\u00eb m\u00eb t\u00eb leht\u00eb <i>shkall\u00ebzimin<\/i> automatik t\u00eb ndryshimeve t\u00eb skem\u00ebs. N\u00eb fund t\u00eb fundit, kjo nuk duhet t\u00eb jet\u00eb nj\u00eb kompleksitet \u2014 p\u00ebr t\u00eb shtuar nj\u00eb kolon\u00eb.<\/p>\n<p>Zbatimi i detyruesh\u00ebm i skem\u00ebs \u00ebsht\u00eb jani, ku evolucioni i skem\u00ebs \u00ebsht\u00eb inni. Kur p\u00ebrdoren s\u00eb bashku, k\u00ebto funksione e thjeshtojn\u00eb ndihm\u00ebn dhe konfigurimin e sinjalit si asnj\u00ebher\u00eb m\u00eb par\u00eb.<\/p>\n<p><i>Gjithashtu, do donim t\u00eb falenderonim Mukul Murti dhe Pranav Anand p\u00ebr kontributin e tyre n\u00eb k\u00ebt\u00eb artikull.<\/i><\/p>\n<p>Artikuj t\u00eb tjer\u00eb nga kjo seri:<\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/databricks.com\/blog\/2019\/08\/21\/diving-into-delta-lake-unpacking-the-transaction-log.html\">Zhytesha n\u00eb Delta Lake: shp\u00ebrndarja e regjistrit t\u00eb transaksioneve<\/a><\/noindex><\/p>\n<p><center><div class=\"youtube-placeholder\" data-id=\"tjb10n5wVs8\" onclick=\"loadVideo(this)\">\r\n        <img decoding=\"async\" src=\"https:\/\/img.youtube.com\/vi\/tjb10n5wVs8\/hqdefault.jpg\" alt=\"Luaj videon\" loading=\"lazy\" width=\"480\" height=\"360\" style=\"width:100%;height:auto;\">\r\n        <div class=\"play-button\"><\/div>\r\n    <\/div><\/center><\/p>\n<h2>Artikuj lidhur me tem\u00ebn<\/h2>\n<p>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/databricks.com\/blog\/2019\/08\/14\/productionizing-machine-learning-with-delta-lake.html\">M\u00ebsimi i makineris\u00eb n\u00eb nivel prodhimi me Delta Lake<\/a><\/noindex><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/databricks.com\/discover\/data-lakes\/introduction\">\u00c7far\u00eb \u00ebsht\u00eb nj\u00eb liqen t\u00eb dh\u00ebnash?<\/a><\/noindex><\/p>\n<p>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/otus.pw\/J1P5\/\">M\u00ebsoni m\u00eb shum\u00eb rreth kursit<\/a><\/noindex><\/p>\n<p>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/502324\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440! \u041f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u044f\u044e \u0432\u0430\u0448\u0435\u043c\u0443 \u0432\u043d\u0438\u043c\u0430\u043d\u0438\u044e \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u0441\u0442\u0430\u0442\u044c\u0438 \u00abDiving Into Delta Lake: Schema Enforcement &amp; Evolution\u00bb \u0430\u0432\u0442\u043e\u0440\u043e\u0432 Burak Yavuz, Brenner Heintz and Denny Lee, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0431\u044b\u043b \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043b\u0435\u043d \u0432 \u043f\u0440\u0435\u0434\u0434\u0432\u0435\u0440\u0438\u0438 \u0441\u0442\u0430\u0440\u0442\u0430 \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb \u043e\u0442 OTUS. \u0414\u0430\u043d\u043d\u044b\u0435, \u043a\u0430\u043a \u0438 \u043d\u0430\u0448 \u043e\u043f\u044b\u0442, \u043f\u043e\u0441\u0442\u043e\u044f\u043d\u043d\u043e \u043d\u0430\u043a\u0430\u043f\u043b\u0438\u0432\u0430\u044e\u0442\u0441\u044f \u0438 \u0440\u0430\u0437\u0432\u0438\u0432\u0430\u044e\u0442\u0441\u044f. \u0427\u0442\u043e\u0431\u044b \u043d\u0435 \u043e\u0442\u0441\u0442\u0430\u0432\u0430\u0442\u044c, \u043d\u0430\u0448\u0438 \u043c\u0435\u043d\u0442\u0430\u043b\u044c\u043d\u044b\u0435 \u043c\u043e\u0434\u0435\u043b\u0438 \u043c\u0438\u0440\u0430 \u0434\u043e\u043b\u0436\u043d\u044b \u0430\u0434\u0430\u043f\u0442\u0438\u0440\u043e\u0432\u0430\u0442\u044c\u0441\u044f \u043a \u043d\u043e\u0432\u044b\u043c \u0434\u0430\u043d\u043d\u044b\u043c, [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":82039,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-82038","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.1.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440!\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/pogruzhenie-v-delta-lake-prinuditelnoe-primenenie-i-evolyucziya-shemy\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.1.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041f\u043e\u0433\u0440\u0443\u0436\u0435\u043d\u0438\u0435 \u0432 Delta Lake: \u043f\u0440\u0438\u043d\u0443\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0435 \u043f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0438\u0435 \u0438 \u044d\u0432\u043e\u043b\u044e\u0446\u0438\u044f \u0441\u0445\u0435\u043c\u044b | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440!\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/pogruzhenie-v-delta-lake-prinuditelnoe-primenenie-i-evolyucziya-shemy\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-05-18T23:42:26+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-05-18T23:42:26+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Zhytesha n\u00eb Delta Lake: zbatimi i detyruesh\u00ebm dhe evolucioni i skem\u00ebs | ProHoster","description":"P\u00ebrsh\u00ebndetje, Habr!","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/pogruzhenie-v-delta-lake-prinuditelnoe-primenenie-i-evolyucziya-shemy","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041f\u043e\u0433\u0440\u0443\u0436\u0435\u043d\u0438\u0435 \u0432 Delta Lake: \u043f\u0440\u0438\u043d\u0443\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0435 \u043f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0438\u0435 \u0438 \u044d\u0432\u043e\u043b\u044e\u0446\u0438\u044f \u0441\u0445\u0435\u043c\u044b | ProHoster","og:description":"\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440!","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/pogruzhenie-v-delta-lake-prinuditelnoe-primenenie-i-evolyucziya-shemy","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-05-18T23:42:26+00:00","article:modified_time":"2020-05-18T23:42:26+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"82038","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 15:45:28","updated":"2022-09-29 13:43:45","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/82038","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=82038"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/82038\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/82039"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=82038"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=82038"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=82038"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}