Zvichida hachisi chakavanzika kuti gore rapfuura raive gore rekuchinja kukuru kweApache Hadoop. Gore rakapera, Cloudera neHortonworks vakabatanidza (chaizvoizvo, kutora kwekupedzisira), uye Mapr, nekuda kwematambudziko makuru emari, akatengeswa kuna Hewlett Packard. Uye kana makore mashoma apfuura, mune yekumisikidzwa kwenzvimbo, sarudzo kazhinji yaifanira kuitwa pakati peCloudera neHortonworks, nhasi, nhamo, isu hatina sarudzo iyi. Chimwe chakashamisa inyaya yekuti Cloudera yakazivisa muna Kukadzi wegore rino kuti yaizomira kuburitsa mabhanari magungano ekugovera munzvimbo inochengeterwa veruzhinji, uye ave kuwanikwa chete kuburikidza nekubhadhara kwakabhadharwa. Ehe, zvichiri kugoneka kurodha yazvino vhezheni yeCDH neHDP yakaburitswa isati yapera 2019, uye rutsigiro rwavo runotarisirwa kwegore rimwe kusvika maviri. Asi chii chinotevera? Kune avo vakambobhadhara kubhadhara, hapana chakachinja. Uye kune avo vasingadi kushandura kune yakabhadharwa vhezheni yekugovera, asi panguva imwecheteyo vanoda kukwanisa kugashira yazvino mavhezheni ezvikamu zvemasumbu, pamwe nezvigamba uye zvimwe zvigadziriso, takagadzirira chinyorwa ichi. Mariri tichaona zvingasarudzwa zvekubuda mumamiriro ezvinhu aya.
Chinyorwa chacho chakanyanya kudzokorora. Izvo hazvizove nekuenzanisa kwekugovera uye ongororo yakadzama yavo, uye hapazove nemabikirwo ekuisa nekugadzirisa iwo. Chii chichaitika? Tichataura muchidimbu nezvekugovera kwakadai seArenadata Hadoop, iyo inofanirwa kutariswa nesu nekuda kwekuvapo kwayo, iyo isingawanzoitiki nhasi. Uye tozotaura nezve Vanilla Hadoop, kunyanya nezvekuti "ingabikwa" sei uchishandisa Apache Bigtop. Wakagadzirira? Wobva wagamuchirwa kukati.
Arenadata Hadoop
Iyi inyowani chose uye, parizvino, idiki-inozivikanwa yekugovera kit yekuvandudza kwemba. Nehurombo, parizvino paHabrΓ© pane chete .
Rumwe ruzivo runogona kuwanikwa pane mukuru project. Iyo yazvino vhezheni yekugovera yakavakirwa paHadoop 3.1.2 yevhezheni 3, uye 2.8.5 yevhezheni 2.
Ruzivo nezve nzira yemugwagwa inogona kuwanikwa .
Arenadata Cluster Manager Interface
Arenadata's core product is , iyo inoshandiswa kuisa, kugadzirisa uye kutarisa akasiyana-siyana emakambani software mhinduro. ADCM inogoverwa mahara, uye kushanda kwayo kunowedzerwa nekuwedzera mabundle, ari seti yeanonzwisisika-mabhuku ekutamba. Mabundle akakamurwa kuva marudzi maviri: bhizinesi uye nharaunda. Iwo ekupedzisira anowanikwa emahara kurodha kubva kuArenadata webhusaiti. Izvo zvakare zvinogoneka kugadzira yako bundle uye kuibatanidza neADCM.
Kuendeswa uye manejimendi eHadoop 3, nharaunda vhezheni yebundle inopihwa pamwe neADCM, asi yeHadoop 2 iripo chete. seimwe nzira. Kana ari marepositori ane mapakeji, akavhurika kuti asvike neveruzhinji, anogona kutorwa uye kuiswa nenzira yakajairwa kune ese zvikamu zve cluster. Pakazara, kugovera kunoratidzika kunonakidza zvikuru. Ndine chokwadi chekuti kuchave nevaya vakajaira mhinduro senge Cloudera Manager uye Ambari, uye vachada ADCM pachayo. Kune vamwe, ichavawo yakakura pamwe nekugovera yekutsiva kunze kwenyika.
Kana tikataura pamusoro pezvisina kunaka, zvichave zvakafanana kune mamwe ose kugoverwa kweHadoop. Zvinonzi:
- Izvo zvinonzi "vendor lock-in". Tichishandisa mienzaniso yeCloudera neHortonworks, takatoona kuti pane nguva dzose njodzi yekuchinja mutemo wekambani.
- Yakakosha kusarira kuseri kweApache kumusoro kwerwizi.
Vanilla Hadoop
Sezvaunoziva, Hadoop haisi monolithic chigadzirwa, asi, kutaura zvazviri, galaxy yose yebasa rakapoteredza yakagoverwa faira system HDFS. Vashoma vanhu vanozove nezvakakwana zveimwe faira cluster. Vamwe vanoda Hive, vamwe Presto, uyezve kune HBase nePhoenix; Spark iri kuwedzera kushandiswa. Kune orchestration uye kurodha data, Oozie, Sqoop uye Flume dzimwe nguva inowanikwa. Uye kana nyaya yekuchengeteka ikamuka, ipapo Kerberos pamwe chete neRanger pakarepo inouya mupfungwa.
Binary vhezheni dzeHadoop zvikamu zviripo pawebhusaiti yega yega ecosystem mapurojekiti muchimiro chetarballs. Unogona kuzvidhawunirodha uye wotanga kuisirwa, asi nechimiro chimwe chete: pamusoro pekuunganidza zvakasununguka mapakeji kubva ku "mbishi" mabhinari, izvo zvaungangoda kuita, haungavi nechivimbo mukuenderana kweshanduro dzakadhawunirodha dzezvikamu nechimwe nechimwe. zvimwe. Sarudzo yakasarudzika ndeyekuvaka uchishandisa Apache Bigtop. Bigtop ichakubvumidza iwe kuvaka kubva kuApache maven repositories, mhanyisa bvunzo uye kuvaka mapakeji. Asi, izvo zvakakosha kwatiri, Bigtop ichaunganidza iwo mavhezheni ezvikamu izvo zvinozoenderana. Tichataura nezvazvo zvakadzama pazasi.
Apache Bigtop
Apache Bigtop chishandiso chekuvaka, kurongedza uye kuyedza akati wandei
mapurojekiti akavhurika sosi, akadai seHadoop neGreenplum. Bigtop ine zvakawanda
zvinoburitswa. Panguva yekunyora, yazvino yakagadzikana kuburitswa yaive vhezheni 1.4,
uye tenzi aiva 1.5. Mhando dzakasiyana dzekuburitswa dzinoshandisa shanduro dzakasiyana
zvikamu. Semuenzaniso, ye1.4 Hadoop core components ine version 2.8.5, uye mune master
2.10.0. Kuumbwa kwezvikamu zvinotsigirwa zviri kuchinjawo. Chimwe chinhu chekare uye
iyo isingagadziriswi inoenda, uye panzvimbo yayo inouya chimwe chinhu chitsva, chinonyanya kudiwa, uye
hachisi hazvo chinhu kubva kumhuri yeApache pachayo.
Mukuwedzera, Bigtop ine zvakawanda .
Patakatanga kujairana neBigtop, isu takatanga takashamiswa nehunyoro hwayo, mukuenzanisa nemamwe mapurojekiti eApache, kuwanda uye mukurumbira, pamwe nenharaunda diki kwazvo. Izvi zvinotevera kubva pane izvi kuti kune ruzivo rushoma pane chigadzirwa, uye kutsvaga mhinduro dzezvinetso zvakamuka pamaforamu uye zvinyorwa zvekutumira zvinogona kusaburitsa chero chinhu. Pakutanga, rakava basa rakaoma kuti isu tipedze kuungana kwakazara kwekugovera nekuda kwezvimiro zvechishandiso pachacho, asi isu tichazotaura nezvazvo gare gare.
Semuseki, avo pane imwe nguva vaifarira mapurojekiti akadaro eLinux universe seGentoo neLFS vangazviona zvichinakidza kushanda nechinhu ichi uye nekurangarira idzo "epic" nguva isu pachedu taitsvaga (kana kunyora) ebuilds uye anogara achivakazve Mozilla nezvigamba zvitsva.
Mukana wakakura weBigtop kuvhurika uye kuita zvakasiyana-siyana kwezvishandiso pazviri. Iyo yakavakirwa paGradle uye Apache Maven. Gradle inonyanya kuzivikanwa sechinhu chinoshandiswa neGoogle kuvaka Android. Inochinjika, uye, sezvavanotaura, "yakaedzwa-kurwa." Maven chishandiso chakajairwa chekuvaka mapurojekiti muApache pachayo, uye sezvo mazhinji ezvigadzirwa zvawo achiburitswa kuburikidza neMaven, hazvaigona kuitwa pasina iwo pano futi. Zvakakodzera kutarisisa iyo POM (project object model) - iyo "yakakosha" xml faira inotsanangura zvese zvinodikanwa kuti Maven ashande nepurojekiti yako, panovakwa basa rese. Ndizvozvo pa
zvikamu zveMaven uye kune zvimwe zvipingamupinyi zvinowanzosangana nevashandisi veBigtop kekutanga.
Dzidzira
Saka unofanira kutanga kupi? Enda kune yekurodha peji uye dhawunirodha yazvino yakagadzikana vhezheni seyakachengetwa. Iwe unogona zvakare kuwana mabinary artifacts akaunganidzwa neBigtop ipapo. Nenzira, pakati peakajairika mapakeji maneja, YUM uye APT inotsigirwa.
Neimwe nzira, unogona kudhawunirodha yazvino yakagadzikana kuburitswa zvakananga kubva
github:
$ git clone --branch branch-1.4 https://github.com/apache/bigtop.gitKuvhara mu "bigtop" ...
remote: Enumerating objects: 46, done.
remote: Counting objects: 100% (46/46), done.
remote: Compressing objects: 100% (41/41), done.
remote: Total 40217 (delta 14), reused 10 (delta 1), pack-reused 40171
ΠΠΎΠ»ΡΡΠ΅Π½ΠΈΠ΅ ΠΎΠ±ΡΠ΅ΠΊΡΠΎΠ²: 100% (40217/40217), 43.54 MiB | 1.05 MiB/s, Π³ΠΎΡΠΎΠ²ΠΎ.
ΠΠΏΡΠ΅Π΄Π΅Π»Π΅Π½ΠΈΠ΅ ΠΈΠ·ΠΌΠ΅Π½Π΅Π½ΠΈΠΉ: 100% (20503/20503), Π³ΠΎΡΠΎΠ²ΠΎ.
Updating files: 100% (1998/1998), Π³ΠΎΡΠΎΠ²ΠΎ.Iyo inoguma ./bigtop dhairekitori inotaridzika seizvi:
./bigtop-bigpetstore - demo application, synthetic mienzaniso
./bigtop-ci - CI zvishandiso, jenkins
./bigtop-data-generators - kugadzirwa kwedata, synthetics, yekuongorora utsi, nezvimwe.
./bigtop-deploy - midziyo yekushandisa
./bigtop-packages - configs, zvinyorwa, zvigamba zvegungano, chikamu chikuru chechishandiso
./bigtop-test-framework - test framework
./bigtop-tests - miedzo pachayo, mutoro uye utsi
./bigtop_toolchain - nharaunda yekuungana, kugadzirira nharaunda kuti chishandiso chishande
./build - gadzira dhairekitori rekushanda
./dl - dhairekitori kune zvakadhawunirwa zvinyorwa
./docker - kuvaka mune docker mifananidzo, kuyedza
./gradle - Gradle config
./output - dhairekitori uko kuvaka zvigadzirwa zvinoenda
./provisioner - kupa
Chinhu chinonyanya kufadza kwatiri panguva ino ndiyo huru config ./bigtop/bigtop.bom, umo tinoona zvese zvinotsigirwa zvikamu zvine shanduro. Apa ndipo patinogona kutsanangura imwe vhezheni yechigadzirwa (kana isu takangoerekana tada kuedza kuivaka) kana vhezheni yekuvaka (kana, semuenzaniso, takawedzera chigamba chakakosha).
Iyo subdirectory zvakare inofarira zvikuru ./bigtop/bigtop-packages, iyo yakanangana nemaitiro ekuunganidza zvikamu uye mapakeji navo.
Saka, isu takadhawunirodha archive, kuiburitsa kana kuita clone kubva github, tinogona kutanga kuvaka?
Kwete, ngatigadzirirei nharaunda kutanga.
Kugadzirira Zvakatipoteredza
Uye pano tinoda kuzorora kudiki. Kuti uvake chero chakanyanya kana chishoma chigadzirwa chigadzirwa, iwe unoda imwe nharaunda - mune yedu, iyi ndiyo JDK, yakafanana yakagovaniswa maraibhurari, misoro mafaera, nezvimwe, zvishandiso, semuenzaniso, svosve, ivy2 uye zvimwe zvakawanda. Imwe yesarudzo yekuwana nharaunda yaunoda yeBigtop ndeyekuisa iyo inodiwa zvinhu pane yekuvaka host. Ndinogona kunge ndakakanganisa mukuverenga nguva, asi zvinoita sekunge nevhezheni 1.0 pakanga painewo sarudzo yekuvaka mune pre-yakagadzirirwa uye inowanikwa Docker mifananidzo, inogona kuwanikwa pano.
Kana zviri zvekugadzirira zvakatipoteredza, pane mubatsiri weizvi - Puppet.
Iwe unogona kushandisa iyo inotevera mirairo, mhanya kubva kumudzi dhairekitori
mudziyo, ./bigtop:
./gradlew toolchain
./gradlew toolchain-devtools
./gradlew toolchain-puppetmodules
Kana zvakananga kuburikidza nepuppet:
puppet apply --modulepath=<path_to_bigtop> -e "include bigtop_toolchain::installer"
puppet apply --modulepath=<path_to_bigtop> -e "include bigtop_toolchain::deployment-tools"
puppet apply --modulepath=<path_to_bigtop> -e "include bigtop_toolchain::development-tools"Nehurombo, matambudziko anogona kumuka panguva ino. Zano rakajairwa pano ndere kushandisa inotsigirwa kugovera, kusvika parizvino pane yekuvaka host, kana kuyedza docker nzira.
Gungano
Chii chatingaedza kuunganidza? Mhinduro kumubvunzo uyu ichapihwa nekubuda kwemurairo
./gradlew tasks
Muchikamu chePackage mabasa pane akati wandei zvigadzirwa zviri zvekupedzisira zveBigtop.
Ivo vanogona kuzivikanwa ne suffix -rpm kana -pkg-ind (panyaya yekuvaka
mu docker). Muchiitiko chedu, inonyanya kufadza ndeyeHadoop.
Ngatiedzei kuvaka munharaunda yevake server yedu:
./gradlew hadoop-rpmBigtop pachayo inodhawunirodha masosi anodiwa anodiwa kune chaiyo chikamu uye otanga kusangana. Saka, kushanda kwechishandiso kunoenderana neMaven repositories uye mamwe masosi, ndiko kuti, inoda kuwanikwa kweInternet.
Panguva yekushanda, chiyero chinobuda chinogadzirwa. Dzimwe nguva iwo uye mameseji ekukanganisa anogona kukubatsira kuti unzwisise zvakakanganisika. Uye dzimwe nguva unoda kuwana rumwe ruzivo. Muchiitiko ichi zvakakodzera kuwedzera nharo --info kana --debug, uye inogonawo kubatsira βstacktrace. Pane nzira iri nyore yekugadzira data seti yekuzotevera kuwana kune yetsamba zvinyorwa, kiyi --scan.
Nerubatsiro rwayo, bigtop inounganidza ruzivo rwese nekuiisa mu gradle, mushure mezvo ichapa chinongedzo,
nekutevera izvo, munhu anokwanisa achakwanisa kunzwisisa kuti sei gungano rakakundikana.
Ndapota ziva kuti sarudzo iyi inogona kuburitsa ruzivo rwausingadi, senge mazita ekushandisa, node, nharaunda dzakasiyana, nezvimwe, saka chenjera.
Kazhinji kukanganisa kunokonzerwa nekutadza kuwana chero zvikamu zvakakosha pakuunganidza. Kazhinji, unogona kugadzirisa dambudziko nekugadzira chigamba chekugadzirisa chimwe chinhu mumatsime, semuenzaniso, kero mu pom.xml mumudzi wezvinyorwa. Izvi zvinoitwa nekugadzira nekuiisa mudhairekitori rakakodzera ./bigtop/bigtop-packages/src/common/oozie/ chigamba, semuenzaniso, mune fomu patch2-fix.diff.
--- a/pom.xml
+++ b/pom.xml
@@ -136,7 +136,7 @@
<repositories>
<repository>
<id>central</id>
- <url>http://repo1.maven.org/maven2</url>
+ <url>https://repo1.maven.org/maven2</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
Zvingangodaro, panguva yekuverenga chinyorwa ichi, haufanirwe kuita zviri pamusoro gadzirisa iwe pachako.
Paunenge uchiunza chero zvigamba uye shanduko kune dhizaini yegungano, ungangoda "kugadzirisa" gungano uchishandisa iyo yekuchenesa kuraira:
./gradlew hadoop-clean
> Task :hadoop_vardefines
> Task :hadoop-clean
BUILD SUCCESSFUL in 5s
2 actionable tasks: 2 executedUku kuvhiya kunodzosera kumashure shanduko dzese kumusangano wechikamu ichi, mushure mezvo gungano richaitwa zvakare. Panguva ino tichaedza kuvaka purojekiti mumufananidzo wedocker:
./gradlew -POS=centos-7 -Pprefix=1.2.1 hadoop-pkg-ind
> Task :hadoop-pkg-ind
Building 1.2.1 hadoop-pkg on centos-7 in Docker...
+++ dirname ./bigtop-ci/build.sh
++ cd ./bigtop-ci/..
++ pwd
+ BIGTOP_HOME=/tmp/bigtop
+ '[' 6 -eq 0 ']'
+ [[ 6 -gt 0 ]]
+ key=--prefix
+ case $key in
+ PREFIX=1.2.1
+ shift
+ shift
+ [[ 4 -gt 0 ]]
+ key=--os
+ case $key in
+ OS=centos-7
+ shift
+ shift
+ [[ 2 -gt 0 ]]
+ key=--target
+ case $key in
+ TARGET=hadoop-pkg
+ shift
+ shift
+ [[ 0 -gt 0 ]]
+ '[' -z x ']'
+ '[' -z x ']'
+ '[' '' == true ']'
+ IMAGE_NAME=bigtop/slaves:1.2.1-centos-7
++ uname -m
+ ARCH=x86_64
+ '[' x86_64 '!=' x86_64 ']'
++ docker run -d bigtop/slaves:1.2.1-centos-7 /sbin/init
+
CONTAINER_ID=0ce5ac5ca955b822a3e6c5eb3f477f0a152cd27d5487680f77e33fbe66b5bed8
+ trap 'docker rm -f
0ce5ac5ca955b822a3e6c5eb3f477f0a152cd27d5487680f77e33fbe66b5bed8' EXIT
....
ΠΌΠ½ΠΎΠ³ΠΎ Π²ΡΠ²ΠΎΠ΄Π°
....
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-mapreduce-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-namenode-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-secondarynamenode-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-zkfc-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-journalnode-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-datanode-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-httpfs-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-resourcemanager-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-nodemanager-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-proxyserver-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-timelineserver-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-mapreduce-historyserver-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-client-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-conf-pseudo-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-doc-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-libhdfs-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-libhdfs-devel-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-fuse-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-debuginfo-2.8.5-1.el7.x86_64.rpm
+ umask 022
+ cd /bigtop/build/hadoop/rpm//BUILD
+ cd hadoop-2.8.5-src
+ /usr/bin/rm -rf /bigtop/build/hadoop/rpm/BUILDROOT/hadoop-2.8.5-1.el7.x86_64
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.uQ2FCn
+ exit 0
+ umask 022
Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.CwDb22
+ cd /bigtop/build/hadoop/rpm//BUILD
+ rm -rf hadoop-2.8.5-src
+ exit 0
[ant:touch] Creating /bigtop/build/hadoop/.rpm
:hadoop-rpm (Thread[Task worker for ':',5,main]) completed. Took 38 mins 1.151 secs.
:hadoop-pkg (Thread[Task worker for ':',5,main]) started.
> Task :hadoop-pkg
Task ':hadoop-pkg' is not up-to-date because:
Task has not declared any outputs despite executing actions.
:hadoop-pkg (Thread[Task worker for ':',5,main]) completed. Took 0.0 secs.
BUILD SUCCESSFUL in 40m 37s
6 actionable tasks: 6 executed
+ RESULT=0
+ mkdir -p output
+ docker cp
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb:/bigtop/build .
+ docker cp
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb:/bigtop/output .
+ docker rm -f ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
+ '[' 0 -ne 0 ']'
+ docker rm -f ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
Error: No such container:
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
BUILD SUCCESSFUL in 41m 24s
1 actionable task: 1 executedIyo kuvaka yakaitwa pasi peCentOS, asi inogonawo kuitwa pasi peUbuntu:
./gradlew -POS=ubuntu-16.04 -Pprefix=1.2.1 hadoop-pkg-indPamusoro pekuvaka mapakeji ekugoverwa kwakasiyana kweLinux, chishandiso chinogona kugadzira repository nemapakeji akaunganidzwa, semuenzaniso:
./gradlew yumIwe unogona zvakare kurangarira nezve utsi bvunzo uye kutumirwa muDocker.
Gadzira sumbu remanodhi matatu:
./gradlew -Pnum_instances=3 docker-provisionerIta bvunzo dzeutsi muboka remanodhi matatu:
./gradlew -Pnum_instances=3 -Prun_smoke_tests docker-provisionerDelete a cluster:
./gradlew docker-provisioner-destroyTora mirairo yekubatanidza mukati memidziyo yedocker:
./gradlew docker-provisioner-sshRatidza chimiro:
./gradlew docker-provisioner-statusIwe unogona kuverenga zvakawanda nezve Deployment mabasa mune zvinyorwa.
Kana tikataura nezve bvunzo, kune akati wandei nhamba yavo, kunyanya kusvuta uye kubatanidzwa. Ongororo yavo iri kunze kwechikamu chechinyorwa chino. Rega ndingotaura kuti kuunganidza kit yekugovera harisi basa rakaoma sezvaringaite sekutanga kuona. Isu takakwanisa kuunganidza uye kupasa bvunzo pazvinhu zvese zvatinoshandisa mukugadzira kwedu, uye isu takanga tisina matambudziko nekuatumira nekuita mabasa ekutanga munzvimbo yekuyedza.
Pamusoro pezvinhu zviripo muBigtop, zvinokwanisika kuwedzera chero chimwe chinhu, kunyangwe yako wega software yekuvandudza. Zvese izvi zvakangoita otomatiki uye zvinokwana mune iyo CI / CD pfungwa.
mhedziso
Zviripachena, kugovera kwakaunganidzwa nenzira iyi hakufanirwe kutumirwa nekukurumidza kukugadzira. Iwe unofanirwa kunzwisisa kuti kana paine kudikanwa kwechokwadi kwekuvaka uye kutsigira kugovera kwako, saka iwe unofanirwa kuisa mari uye nguva mune izvi.
Nekudaro, mukubatana neiyo nzira uye timu yehunyanzvi, zvinokwanisika kuita pasina mhinduro dzekutengesa.
Izvo zvakakosha kuti uzive kuti iyo Bigtop purojekiti pachayo iri kuda kuvandudzwa uye hairatidzike seiri kugadzirwa nhasi. Tarisiro yeHadoop 3 inoonekwa mairi haina kujeka. Nenzira, kana iwe uine chido chechokwadi chekuvaka Hadoop 3, unogona kutarisa kubva kuArenadata, umo, kuwedzera kune yakajairwa
Kune akati wandei ekuwedzera zvikamu (Ranger, Knox, NiFi).
Kana iri Rostelecom, kwatiri Bigtop ndeimwe yesarudzo dziri kufungwa nhasi. Pasinei nokuti tinozvisarudza here kana kuti kwete, nguva ndiyo ichataura.
chinamatidzwa
Kuti ubatanidze chinhu chitsva mugungano, unofanirwa kuwedzera tsananguro yayo ku bigtop.bom uye ./bigtop-packages. Iwe unogona kuedza kuita izvi nekuenzanisa nezvikamu zviripo. Edza kuzvinzwisisa. Hazvina kuoma sezvazvinoita pakuona kwekutanga.
Unofungei? Tichafara kuona maonero ako mumashoko uye tinokutendai nekuteerera kwako!
Chinyorwa chakagadzirwa neRostelecom data management timu
Source: www.habr.com