Ebe mepere emepe DataHub: Ọchụchọ metadata nke LinkedIn na Platform nchọpụta

Ebe mepere emepe DataHub: Ọchụchọ metadata nke LinkedIn na Platform nchọpụta

Ịchọta data ị chọrọ ngwa ngwa dị mkpa maka ụlọ ọrụ ọ bụla na-adabere na nnukwu data iji mee mkpebi ndị dabeere na data. Ọ bụghị naanị na nke a na-emetụta nrụpụta nke ndị ọrụ data (gụnyere ndị nyocha, ndị mmepe igwe, ndị sayensị data, na ndị injinia data), mana ọ nwekwara mmetụta ozugbo na ngwaahịa ikpeazụ dabere na pipeline igwe mmụta (ML). Na mgbakwunye, omume maka mmejuputa ma ọ bụ iwulite usoro mmụta igwe na-ewelite ajụjụ a: kedu usoro gị maka ịchọpụta njirimara, ụdị, metrik, datasets, wdg.

N'isiokwu a, anyị ga-ekwu maka otu anyị siri bipụta isi iyi data n'okpuru ikikere mepere emepe DataHub n'ime usoro nyocha na nchọpụta metadata anyị, malite n'ụbọchị mbụ nke ọrụ ahụ Ebe Olee. LinkedIn na-edobe ụdị DataHub nke ya iche na ụdị isi mmalite. Anyị ga-amalite site n'ịkọwa ihe kpatara anyị ji chọọ gburugburu mmepe abụọ dị iche iche, wee kwurịta ụzọ mmalite iji jiri ebe mepere emepe WhereHows wee tụnyere ụdị ime (mmepụta) nke DataHub na ụdị na GitHub. Anyị ga-ekekọrịta nkọwa gbasara ngwọta akpaghị aka ọhụrụ anyị maka ịkwanye na ịnweta mmelite isi mmalite ka idowe ebe nchekwa abụọ ahụ na mmekọrịta. N'ikpeazụ, anyị ga-enye ntụziaka ka esi malite iji DataHub mepere emepe wee kwurịta ya na nkenke.

Ebe mepere emepe DataHub: Ọchụchọ metadata nke LinkedIn na Platform nchọpụta

Ebe Hows bụ DataHub ugbu a!

Otu metadata nke LinkedIn ewepụtara na mbụ DataHub (onye ga-anọchi WhereHows), nchọta LinkedIn na nchọta metadata, yana atụmatụ imeghe ya. N'oge na-adịghị anya ka ọkwa a gasịrị, anyị wepụtara ụdị alfa nke DataHub wee kesaa ya na ndị obodo. Kemgbe ahụ, anyị anọgidewo na-enye aka na ebe nchekwa ahụ ma soro ndị ọrụ nwere mmasị rụkọọ ọrụ ịgbakwunye atụmatụ ndị a na-arịọkarị na dozie nsogbu. Obi dị anyị ụtọ ugbu a ịkpọsa mwepụta gọọmentị DataHub na GitHub.

Ụzọ mepere emepe

WhereHows, Portal mbụ nke LinkedIn maka ịchọta data na ebe o si bịa, malitere dị ka ọrụ ime; otu metadata meghere ya Koodu nzipu ozi na 2016. Kemgbe ahụ, ndị otu ahụ na-edobe codebases abụọ dị iche iche mgbe niile - otu maka isi mmalite na otu maka ojiji nke LinkedIn - n'ihi na ọ bụghị njirimara ngwaahịa niile emepụtara maka iji okwu LinkedIn na-emetụtakarị ndị na-ege ntị. Na mgbakwunye, WhereHows nwere ụfọdụ adabere n'ime (ihe akụrụngwa, ọba akwụkwọ, wdg) na-abụghị ebe mepere emepe. N'ime afọ ndị sochirinụ, WhereHows gafere ọtụtụ iterations na usoro mmepe, na-eme ka idobe codebases abụọ na mmekọrịta bụrụ nnukwu ihe ịma aka. Ndị otu metadata anwalela ụzọ dị iche iche n'ime ọtụtụ afọ iji nwaa idobe mmepe ime na mepere emepe na mmekọrịta.

Nwaa mbụ: "Bu ụzọ meghee isi mmalite"

Anyị gbasoro usoro mmepe “Oghere ụzọ mepere emepe nke mbụ”, ebe ọtụtụ mmepe na-eme na ebe nchekwa mepere emepe ma na-eme mgbanwe maka mbugharị n'ime. Nsogbu dị na ụzọ a bụ na a na-ebugharị koodu ahụ mgbe niile na GitHub tupu enyocha ya nke ọma n'ime. Ruo mgbe a na-eme mgbanwe site na ebe nchekwa ihe na-emeghe ma tinye ntinye ọhụrụ n'ime, anyị agaghị ahụ ihe ọ bụla mmepụta ihe. N'ihe banyere ibuga na-adịghị mma, ọ dịkwa ezigbo ike ịchọpụta onye mere mpụ n'ihi na e mere mgbanwe na batches.

Na mgbakwunye, ihe nlereanya a belatara nrụpụta otu ahụ mgbe ha na-emepụta atụmatụ ọhụrụ chọrọ mmegharị ngwa ngwa, ebe ọ bụ na ọ manyere mgbanwe niile ka ebu ụzọ kwaba n'ime ebe nchekwa mepere emepe wee kwaga na ebe nchekwa dị n'ime. Iji belata oge nhazi, enwere ike ime ndozi ma ọ bụ mgbanwe achọrọ na ebe nchekwa dị n'ime mbụ, mana nke a ghọrọ nnukwu nsogbu mgbe ọ bịara na-ejikọta mgbanwe ndị ahụ na-emepe emepe n'ihi na ebe nchekwa abụọ ahụ enweghị mmekọrịta.

Ihe nlereanya a dị nnọọ mfe iji mejuputa maka ikpo okwu nkekọrịta, ụlọ akwụkwọ ọbá akwụkwọ, ma ọ bụ ọrụ akụrụngwa karịa maka ngwa weebụ omenala nwere njiri mara. Tụkwasị na nke a, ihe nlereanya a dị mma maka ọrụ na-amalite oghere site na ụbọchị mbụ, mana WhereHows wuru dị ka ngwa weebụ kpamkpam. Ọ bụ ihe siri ike n'ezie ịpụpụ ihe ndabere niile dị n'ime, yabụ anyị kwesịrị idobe ndụdụ dị n'ime, mana idobe ndụdụ dị n'ime na imepe emepe emepe emepeghị nke ọma.

Mgbalị nke abụọ: "N'ime mbụ"

**Dịka mbọ nke abụọ, anyị kwagara n'ụdị mmepe "ime mbụ", ebe ọtụtụ mmepe na-eme n'ime ụlọ ma na-agbanwe na koodu isi mmalite mgbe niile. Ọ bụ ezie na ihe nlereanya a kachasị mma maka ikpe eji anyị eme ihe, ọ nwere nsogbu ndị sitere n'okike. Ịkwanye esemokwu niile ozugbo na ebe nchekwa ihe mepere emepe wee gbalị idozi esemokwu jikọrọ ọnụ ma emechaa bụ nhọrọ, mana ọ na-ewe oge. Ndị mmepe n'ọtụtụ oge na-agbalị ka ha ghara ime nke a oge ọ bụla ha na-enyocha koodu ha. N'ihi ya, a ga-eme nke a obere ugboro ugboro, na batches, ma si otú a mee ka ọ dịkwuo mfe idozi esemokwu jikọrọ ọnụ ma emechaa.

Oge nke atọ ọ rụrụ ọrụ!

Mgbalị abụọ ahụ dara ada a kpọtụrụ aha n'elu mere ka ebe nchekwa WhereHows GitHub fọdụ ogologo oge. Ndị otu ahụ gara n'ihu na-emeziwanye atụmatụ na ụkpụrụ ụlọ nke ngwaahịa ahụ, nke mere na ụdị ime nke WhereHows maka LinkedIn bịara nwee ọganihu karịa ụdị isi mmalite mepere emepe. O nwekwara aha ọhụrụ - DataHub. Dabere na mbọ ndị gara aga dara ada, ndị otu ahụ kpebiri imepụta usoro nwere ike ịgbatị, ogologo oge.

Maka oru ngo ọ bụla mepere emepe, ndị otu oghere oghere LinkedIn na-adụ ọdụ ma na-akwado ụdị mmepe nke a na-emepụta modul ọrụ ahụ kpamkpam n'isi mmalite. A na-ebuga arịa ndị ederede n'ụdị n'ebe nchekwa ọha wee leleeghachi n'ime arịa LinkedIn dị n'ime. arịrịọ ọbá akwụkwọ mpụga (ELR). Ịgbaso ụdị mmepe a abụghị naanị mma maka ndị na-eji isi mmalite mepere emepe, ma na-ebutekwa ụkpụrụ modular, extensible, na pluggable architecture.

Agbanyeghị, ngwa azụ azụ tozuru oke dị ka DataHub ga-achọ nnukwu oge iji ruo steeti a. Nke a na-egbochikwa ohere nke imepe emepe mmejuputa iwu nke na-arụ ọrụ n'ụzọ zuru oke tupu ewepụla ndabere niile nke ime n'ụzọ zuru ezu. Ọ bụ ya mere anyị ji mepụta ngwa ọrụ ndị na-enyere anyị aka inye onyinye mepere emepe ngwa ngwa yana obere mgbu. Ihe ngwọta a na-erite uru ma ndị otu metadata (Onye nrụpụta DataHub) yana obodo mepere emepe. Akụkụ ndị na-esonụ ga-atụle ụzọ ọhụrụ a.

Mepee Isi mmalite mbipụta akpaaka

Ụzọ kachasị ọhụrụ nke otu metadata si nweta DataHub mepere emepe bụ ịmepụta ngwa ọrụ na-emekọrịta koodu ime na ebe nchekwa oghere mepere emepe na-akpaghị aka. Akụkụ dị elu nke ngwa ngwa a gụnyere:

  1. Mekọrịta koodu LinkedIn na/site na isi mmalite mepere emepe, yiri ya rsync.
  2. Ọgbọ nkụnye eji isi mee ikike, yiri Apache oke.
  3. Na akpaghị aka wepụta ndekọ idebanye aha mepere emepe site na ndekọ ime ime ime.
  4. Gbochie mgbanwe dị n'ime nke na-emebi isi mmalite mepere emepe site na nnwale dabere.

Akụkụ ndị na-esonụ ga-abanye n'ime ọrụ ndị a kpọtụrụ aha n'elu nwere nsogbu na-adọrọ mmasị.

Mmekọrịta koodu isi mmalite

N'adịghị ka ụdị DataHub mepere emepe, nke bụ otu ebe nchekwa GitHub, ụdị LinkedIn nke DataHub bụ ngwakọta nke ọtụtụ ebe nchekwa (a na-akpọ n'ime ụlọ). multiproducts). The DataHub interface, metadata ụdị ọba akwụkwọ, metadata ụlọ nkwakọba ihe ndabere ọrụ, na nkwanye ọrụ na-ebi na iche iche nchekwa na LinkedIn. Agbanyeghị, iji mee ka ọ dịrị ndị ọrụ mepere emepe mfe, anyị nwere otu ebe nchekwa maka ụdị DataHub mepere emepe.

Ebe mepere emepe DataHub: Ọchụchọ metadata nke LinkedIn na Platform nchọpụta

Ọgụgụ 1: Mmekọrịta n'etiti ebe nchekwa LinkedIn DataHub na otu ebe nchekwa DataHub isi mmalite

Iji kwado nrụpụta akpaaka, ịkwanye, na dọpụta usoro ọrụ, ngwa ọhụrụ anyị na-emepụta nkewa ọkwa faịlụ na-akpaghị aka na faịlụ isi mmalite ọ bụla. Agbanyeghị, ngwa ngwa chọrọ nhazi mbụ yana ndị ọrụ ga-enye maapụ modul dị elu dị ka egosiri n'okpuru.

{
  "datahub-dao": [
    "${datahub-frontend}/datahub-dao"
  ],
  "gms/impl": [
    "${dataset-gms}/impl",
    "${user-gms}/impl"
  ],
  "metadata-dao": [
    "${metadata-models}/metadata-dao"
  ],
  "metadata-builders": [
    "${metadata-models}/metadata-builders"
  ]
}

Maapụ ọkwa modul bụ JSON dị mfe nke igodo ya bụ modul ebumnuche na ebe nchekwa ihe mepere emepe yana ụkpụrụ bụ ndepụta nke modul isi mmalite na ebe nchekwa LinkedIn. Enwere ike inye modul ọ bụla ezubere iche na ebe nchekwa ebe mepere emepe site na ọnụọgụ modul isi mmalite ọ bụla. Iji gosi aha ime ụlọ nchekwa na modul isi mmalite, jiri eriri interpolation n'ụdị Bash. N'iji faịlụ nhazi ọkwa modul, ngwaọrụ ndị ahụ na-emepụta faịlụ nhazi ọkwa ọkwa site na nyocha faịlụ niile dị na akwụkwọ ndekọ aha emetụtara.

{
  "${metadata-models}/metadata-builders/src/main/java/com/linkedin/Foo.java":
"metadata-builders/src/main/java/com/linkedin/Foo.java",
  "${metadata-models}/metadata-builders/src/main/java/com/linkedin/Bar.java":
"metadata-builders/src/main/java/com/linkedin/Bar.java",
  "${metadata-models}/metadata-builders/build.gradle": null,
}

A na-emepụta maapụ ọkwa faịlụ na-akpaghị aka site na ngwaọrụ; Otú ọ dị, onye ọrụ nwekwara ike iji aka melite ya. Nke a bụ maapụ 1:1 nke faịlụ isi iyi LinkedIn na faịlụ dị na ebe nchekwa isi mmalite. Enwere ọtụtụ iwu jikọtara ya na imepụta njikọ faịlụ akpaka a:

  • N'ihe banyere ọtụtụ modul isi mmalite maka modul ebumnuche na ebe mepere emepe, esemokwu nwere ike ibilite, dịka otu FQCN, dị na ihe karịrị otu modul isi mmalite. Dị ka atụmatụ mkpebi esemokwu, ngwá ọrụ anyị na-adaba na nhọrọ "onye ikpeazụ meriri".
  • "null" pụtara na faịlụ isi mmalite abụghị akụkụ nke ebe nchekwa isi mmalite.
  • Mgbe ntinye ma ọ bụ mmịpụta nke ọ bụla mepere emepe, a na-emelite maapụ a na-akpaghị aka ma mepụta foto ọ bụla. Nke a dị mkpa iji chọpụta mgbakwunye na ihichapụ site na koodu isi mmalite kemgbe mmemme ikpeazụ.

Ịmepụta ndekọ ndekọ

A na-emepụtakwa ndekọ ndekọ maka mmeghe isi mmalite na-akpaghị aka site na ijikọ ndekọ ndekọ nke ebe nchekwa dị n'ime. N'okpuru ebe a bụ ihe nlele ndekọ ndekọ iji gosi nhazi nke akwụkwọ ntinye akwụkwọ nke ngwa ọrụ anyị mepụtara. Nkwenye na-egosi n'ụzọ doro anya ụdị nsụgharị nke ebe nchekwa isi mmalite na-achịkọta na nkwa ahụ ma na-enye nchịkọta nke ndekọ ntinye aka. Lelee nke a eme na-eji ezigbo ihe atụ nke akwụkwọ ntinye aka nke ngwa ngwa anyị mepụtara.

metadata-models 29.0.0 -> 30.0.0
    Added aspect model foo
    Fixed issue bar

dataset-gms 2.3.0 -> 2.3.4
    Added rest.li API to serve foo aspect

MP_VERSION=dataset-gms:2.3.4
MP_VERSION=metadata-models:30.0.0

Nnwale dabere

LinkedIn nwere akụrụngwa ule dabere, nke na-enyere aka hụ na mgbanwe na multiproduct dị n'ime adịghị emebi mgbakọ nke multiproducts dabere. Ebe nchekwa dataHub mepere emepe abụghị ọtụtụ ngwaahịa, ọ nweghịkwa ike ịdabere ozugbo nke ngwaahịa ọ bụla, mana site n'enyemaka nke ihe mkpuchi ọtụtụ ngwaahịa nke na-ewepụta koodu isi mmalite DataHub mepere emepe, anyị ka nwere ike iji nnwale ndabere a. Ya mere, mgbanwe ọ bụla (nke nwere ike mechaa kpughee) na nke ọ bụla nke multiproducts na-eri nri na-emeghe ebe nchekwa DataHub na-akpalite ihe omume ụlọ na shei multiproduct. Ya mere, mgbanwe ọ bụla nke na-emeghị ka ịmepụta ngwaahịa mkpuchi na-adaba ule ndị ahụ tupu ịme ngwaahịa mbụ wee laghachi azụ.

Nke a bụ usoro bara uru nke na-enyere aka igbochi ime ime ọ bụla nke na-agbaji isi mmalite mepere emepe ma chọpụta ya n'oge. Na-enweghị nke a, ọ ga-esi ike ikpebi nke ime ime mere ka ebe nchekwa ihe mepere emepe daa, n'ihi na anyị na-etinye mgbanwe dị n'ime na ebe nchekwa data DataHub.

Ọdịiche dị n'etiti DataHub mepere emepe na ụdị mmepụta anyị

Ruo n'oge a, anyị atụlewo ihe ngwọta anyị maka ịmekọrịta ụdị abụọ nke DataHub repositories, ma anyị ka akọwapụtaghị ihe kpatara anyị ji chọọ iyi mmepe abụọ dị iche iche na mbụ. Na ngalaba a, anyị ga-edepụta ọdịiche dị n'etiti ụdị ọha nke DataHub na mmepụta mmepụta na sava LinkedIn, ma kọwaa ihe kpatara esemokwu ndị a.

Otu isi iyi nke esemokwu sitere na eziokwu ahụ bụ na ụdị mmepụta anyị nwere ndabere na koodu nke na-emeghebeghị, dị ka ụmụ LinkedIn (LinkedIn's internal dependency injection framework). A na-ejikarị ụmụ eme ihe n'ime codebases n'ihi na ọ bụ usoro kachasị mma maka ijikwa nhazi dị ike. Mana ọ bụghị ebe mepere emepe; yabụ anyị kwesịrị ịchọta ụzọ mepere emepe maka isi mmalite DataHub.

E nwekwara ihe ndị ọzọ kpatara ya. Ka anyị na-emepụta ndọtị na ụdị metadata maka mkpa LinkedIn, ndọtị ndị a na-abụkarị kpọmkwem maka LinkedIn na ọ nwere ike ọ gaghị emetụta ozugbo na gburugburu ndị ọzọ. Dịka ọmụmaatụ, anyị nwere akara aha kpọmkwem maka NJ ndị so na ya na ụdị metadata ndị ọzọ dakọtara. Yabụ, anyị ewepụla ndọtị ndị a na ụdị metadata mepere emepe nke DataHub. Ka anyị na ndị obodo na-emekọrịta ihe ma ghọta mkpa ha, anyị ga-arụ ọrụ na ụdị isi mmalite mepere emepe nke ndọtị ndị a ebe ọ dị mkpa.

Ọ dị mfe iji yana ngbanwe dị mfe maka obodo mepere emepe mekwara ụfọdụ ọdịiche dị n'etiti ụdị abụọ nke DataHub. Ọdịiche dị na akụrụngwa nhazi iyi bụ ezigbo ihe atụ nke a. Ọ bụ ezie na ụdị ime anyị na-eji usoro nhazi iyi a na-achịkwa, anyị họọrọ iji nhazi iyi arụnyere n'ime (nkeonwe) maka ụdị mepere emepe n'ihi na ọ na-ezere ịmepụta ndabere akụrụngwa ọzọ.

Ihe atụ ọzọ nke dị iche bụ inwe otu GMS (Generalized Metadata Store) na mmejuputa isi mmalite karịa ọtụtụ GMS. GMA (Generalized Metadata Architecture) bụ aha ụlọ azụ azụ maka DataHub, na GMS bụ ụlọ ahịa metadata na gburugburu GMA. GMA bụ ihe owuwu na-agbanwe agbanwe nke na-enye gị ohere ikesa ihe nrụpụta data ọ bụla (dịka ọmụmaatụ datasets, ndị ọrụ, wdg) n'ime ụlọ ahịa metadata nke ya, ma ọ bụ chekwaa ọtụtụ ihe nrụpụta data n'otu ụlọ ahịa metadata ọ bụrụhaala na ndekọ nwere maapụ nhazi data na A na-emelite GMS. Maka ịdị mfe nke ojiji, anyị họọrọ otu ihe atụ GMS nke na-echekwa data niile dị iche iche n'ime ebe mepere emepe DataHub.

Enyere ndepụta zuru oke nke ọdịiche dị n'etiti mmejuputa abụọ a na tebụl n'okpuru.

Product Atụmatụ
LinkedIn DataHub
Mepee Isi mmalite DataHub

Nrụpụta data akwadoro
1) Datasets 2) Ndị ọrụ 3) Metrics 4) Njirimara ML 5) Charts 6) Dashboards.
1) Datasets 2) Ndị ọrụ

Isi mmalite metadata akwadoro maka nhazi data
1) Ambry 2) Couchbase 3) Dalids 4) espresso 5) HDFS 6) Hive 7) Kafka 8) MongoDB 9) MySQL 10) Oracle 11) Pinot 12) Presto 12) Osimiri 13) Teradata 13) Vector 14) Venice
Hive Kafka RDBMS

Pub-sub
LinkedIn Kafka
Confluent Kafka

Nhazi Stream
jisiri
agbakwunyere (onwe ya)

Ntụnye ndabere & Nhazi dị omimi
Ụmụ LinkedIn
mmiri

Mee Ngwá Ọrụ
Ligradle (ihe mkpuchi ime ụlọ nke LinkedIn)
Gradlew

CI / CD
CRT (Ci/CD dị n'ime LinkedIn)
TravisCI na Ogwe Docker

Ụlọ ahịa metadata
Kesara otutu GMS: 1) GMS Dataset 2) GMS onye ọrụ 3) Metric GMS 4) Njirimara GMS 5) Chart/Dashboard GMS
Otu GMS maka: 1) Datasets 2) Ndị ọrụ

Microservices n'ime arịa Docker

Docker na-eme ka ntinye ngwa na nkesa dị mfe containerization. Akụkụ ọ bụla nke ọrụ dị na DataHub bụ ebe mepere emepe, gụnyere akụrụngwa akụrụngwa dịka Kafka, Elasticsearch, neo4j и MySQL, nwere onyonyo Docker nke ya. Iji hazie arịa Docker anyị ji Docker Dezie.

Ebe mepere emepe DataHub: Ọchụchọ metadata nke LinkedIn na Platform nchọpụta

Onyonyo 2: Architecture DataHub *Ebe mepere emepe**

Ị nwere ike ịhụ ụkpụrụ ụlọ dị elu nke DataHub na foto dị n'elu. Ewezuga akụrụngwa akụrụngwa, ọ nwere igbe Docker anọ dị iche iche:

datahub-gms: ọrụ nchekwa metadata

datahub-frontend: ngwa Play, na-eje ozi na interface DataHub.

datahub-mce-consumer: ngwa Kafka iyi, nke na-eji ihe omume mgbanwe metadata (MCE) ma na-emelite ụlọ ahịa metadata.

datahub-mae-consumer: ngwa Kafka iyi, nke na-eji iyi ihe omume nyocha metadata (MAE) wee mepụta ndepụta ọchụchọ na nchekwa data eserese.

Mepee akwụkwọ ebe nchekwa na mbụ DataHub blọọgụ nwere ozi zuru oke gbasara ọrụ nke ọrụ dị iche iche.

CI/CD na DataHub bụ ebe mepere emepe

Ebe nchekwa dataHub mepere emepe na-eji TravisCI maka na-aga n'ihu mwekota na Ogwe Docker maka ntinye aka na-aga n'ihu. Ha abụọ nwere ezigbo njikọ GitHub ma dị mfe ịtọlite. Maka ọtụtụ akụrụngwa mepere emepe nke obodo ma ọ bụ ụlọ ọrụ nkeonwe mepụtara (dịka ọmụmaatụ. Mgbagha), A na-emepụta ihe oyiyi Docker na ebuga ya na Docker Hub maka ịdị mfe nke obodo. Enwere ike iji onyonyo Docker ọ bụla achọtara na Docker Hub site na iji iwu dị mfe docker ịdọrọ.

Site n'itinye aka na ebe nchekwa data mepere emepe DataHub, a na-ewu ihe onyonyo Docker niile na-akpaghị aka wee bufee ya na Docker Hub na mkpado "ọcha ọhụrụ". Ọ bụrụ na ejiri ụfọdụ hazie Docker Hub Ịkpọ alaka okwu mgbe nile, a na-ewepụtakwa mkpado niile dị na ebe nchekwa isi mmalite nwere aha mkpado kwekọrọ na Docker Hub.

Iji DataHub

Ịtọlite ​​​​DataHub dị nnọọ mfe ma mejupụtara usoro atọ dị mfe:

  1. Mechie ebe nchekwa ihe mepere emepe wee jiri docker-edepụta arịa Docker niile site na iji ederede docker-edepụta maka mmalite ngwa ngwa.
  2. Budata data nlele enyere na ebe nchekwa site na iji ngwa ahịrị iwu nke enyere.
  3. Chọgharịa DataHub na ihe nchọgharị gị.

Esochiri nke ọma Gitter nkata ahaziri maka ajụjụ ngwa ngwa. Ndị ọrụ nwekwara ike ịmepụta okwu ozugbo na ebe nchekwa GitHub. Nke kachasị mkpa, anyị na-anabata ma nwee ekele maka nzaghachi na aro niile!

Atụmatụ maka ọdịnihu

Ugbu a, akụrụngwa ma ọ bụ microservice ọ bụla maka ebe mepere emepe DataHub ka ewuru dị ka akpa Docker, a na-ahazikwa sistemụ niile site na iji. docker-ide. Nyere ewu ewu na ebe nile Kubernetes, anyị ga-amasịkwa ịnye ihe ngwọta dabere na Kubernetes n'ọdịnihu dị nso.

Anyị na-eme atụmatụ ịnye ngwọta ntụgharị maka ibunye DataHub na ọrụ igwe ojii dị ka Azure, AWS ma ọ bụ Google Cloud. N'inye ọkwa ọkwa na nso nso a nke mbugharị LinkedIn na Azure, nke a ga-adaba na ihe ndị otu metadata dị n'ime ụzọ.

N'ikpeazụ ma ọ dịghị ihe ọzọ, ekele maka ndị mbụ nakweere DataHub na obodo mepere emepe bụ ndị nyere DataHub alphas ma nyere anyị aka ịchọpụta okwu ma melite akwụkwọ.

isi: www.habr.com

Tinye a comment