Mohlomong ha se lekunutu hore selemong se fetileng e bile selemo sa liphetoho tse kholo bakeng sa Apache Hadoop. Selemong se fetileng, Cloudera le Hortonworks ba ile ba kopanya (ha e le hantle, ho fumanoa ha morao), 'me Mapr, ka lebaka la mathata a tebileng a lichelete, a rekisoa ho Hewlett Packard. 'Me haeba lilemo tse' maloa pele ho moo, tabeng ea ho kenya mehaho, hangata khetho e ne e lokela ho etsoa pakeng tsa Cloudera le Hortonworks, kajeno, oho, ha re na khetho ena. Ntho e 'ngoe e makatsang ke taba ea hore Cloudera e phatlalalitse ka Hlakola selemong sena hore e tla emisa ho lokolla likopano tsa binary tsa ho ajoa sebakeng sa polokelo ea sechaba, mme joale li fumaneha feela ka peeletso e lefelloang. Ha e le hantle, ho ntse ho ka khoneha ho khoasolla liphetolelo tsa morao-rao tsa CDH le HDP tse lokollotsoeng pele ho bofelo ba 2019, 'me tšehetso bakeng sa bona e lebeletsoe selemo se le seng ho isa ho tse peli. Empa ke eng seo u lokelang ho se etsa ka mor'a moo? Bakeng sa ba kileng ba lefa bakeng sa peeletso, ha ho letho le fetohileng. 'Me bakeng sa ba sa batleng ho fetolela phetolelong e lefshoang ea kabo, empa ka nako e ts'oanang ba batla ho khona ho fumana liphetolelo tsa morao-rao tsa likarolo tsa lihlopha, hammoho le li-patches le lisebelisoa tse ling, re lokiselitse sehlooho sena. Ho eona re tla hlahloba mekhoa e ka khonehang ea ho tsoa boemong bona.
Sengoloa ke tlhahlobo e ngata. E ke ke ea ba le papiso ea likhatiso le tlhahlobo e qaqileng ea tsona, 'me ho ke ke ha ba le litlolo tsa ho li kenya le ho li hlophisa. Ho tla etsahala'ng? Re tla bua ka bokhutšoanyane ka kabo e joalo e kang Arenadata Hadoop, eo ka nepo e tšoaneloang ke tlhokomelo ea rona ka lebaka la ho fumaneha ha eona, e leng ntho e sa tloaelehang haholo kajeno. 'Me joale re tla bua ka Vanilla Hadoop, haholo-holo mabapi le hore na e ka "phehoa" joang ho sebelisa Apache Bigtop. U se u loketse? Ebe u amohela katse.
Arenadata Hadoop
Ena ke sephutheloana se secha ka ho felletseng, 'me hajoale, ha se tsejoe hanyane ka hanyane kabo ea ntlafatso ea lehae. Ka bomalimabe, hajoale ho Habré ho na le feela .
Lintlha tse ling li ka fumanoa ho ofisiri morero. Liphetolelo tsa morao-rao tsa kabo li thehiloe ho Hadoop 3.1.2 bakeng sa mofuta oa 3, le 2.8.5 bakeng sa mofuta oa 2.
Lintlha tse mabapi le 'mapa oa litsela li ka fumanoa .
Arenadata Cluster Manager Interface
Sehlahisoa sa mantlha sa Arenadata ke , e sebelisetsoang ho kenya, ho hlophisa le ho beha leihlo litharollo tse fapaneng tsa software ea k'hamphani. ADCM e ajoa ntle ho tefo, 'me ts'ebetso ea eona e atolosoa ka ho eketsa liphutheloana, e leng sehlopha sa libuka tsa ho bapala tse utloahalang. Li-bundle li arotsoe ka mefuta e 'meli: khoebo le sechaba. Tse morao li fumaneha bakeng sa ho khoasolla mahala ho webosaete ea Arenadata. Hape hoa khoneha ho iketsetsa sephutheloana sa hau le ho se hokahanya le ADCM.
Bakeng sa ho tsamaisoa le tsamaiso ea Hadoop 3, mofuta oa sechaba oa bongata o fanoa hammoho le ADCM, empa bakeng sa Hadoop 2 ho na le feela. joalo ka mokhoa o mong. Ha e le li-repositories tse nang le liphutheloana, li bulehile ho fihlella sechaba, li ka kopitsoa le ho kenngoa ka mokhoa o tloaelehileng bakeng sa likarolo tsohle tsa sehlopha. Ka kakaretso, kabo e shebahala e thahasellisa haholo. Kea kholoa ho tla ba le ba tloaetseng litharollo tse kang Cloudera Manager le Ambari, le ba tla rata ADCM ka boeona. Bakeng sa ba bang, e tla boela e be kholo e kholo le hore kabo bakeng sa ho kenya sebaka.
Haeba re bua ka bofokoli, li tla tšoana le tse ling kaofela tsa liphallelo tsa Hadoop. E leng:
- Seo ho thoeng ke "morekisi lock-in". Re sebelisa mehlala ea Cloudera le Hortonworks, re se re hlokometse hore kamehla ho na le kotsi ea ho fetola pholisi ea khampani.
- Ho na le sebaka se seholo ka morao ho Apache holimo.
Vanilla Hadoop
Joalo ka ha u tseba, Hadoop ha se sehlahisoa sa monolithic, empa, ha e le hantle, sehlopha sa linaleli sa lits'ebeletso ho potoloha sistimi ea eona ea faele ea HDFS. Ke batho ba fokolang ba tla ba le sehlopha se lekaneng sa faele se le seng. Ba bang ba hloka Hive, ba bang Presto, ebe ho na le HBase le Phoenix; Spark e ntse e sebelisoa haholo. Bakeng sa 'mino oa liletsa le ho kenya data, Oozie, Sqoop le Flume ka linako tse ling lia fumanoa. 'Me haeba potso ea ts'ireletso e hlaha, joale Kerberos hammoho le Ranger e tla kelellong hang-hang.
Liphetolelo tsa Binary tsa likarolo tsa Hadoop li fumaneha webosaeteng ea morero o mong le o mong oa ecosystem ka mokhoa oa li-tarball. U ka li khoasolla 'me ua qala ho li kenya, empa ka boemo bo le bong: ntle le ho bokella liphutheloana ka boithatelo ho tsoa ho li-binary "tse tala", tseo o kanna oa batla ho li etsa, u ke ke ua ba le ts'epo ea ho lumellana ha liphetolelo tse jarollotsoeng tsa likarolo le e' ngoe le e 'ngoe. tse ding. Khetho e ratoang ke ho haha ho sebelisa Apache Bigtop. Bigtop e tla u lumella ho aha ho tsoa ho polokelo ea maven ea Apache, ho etsa liteko le ho aha liphutheloana. Empa, se bohlokoa haholo ho rona, Bigtop e tla bokella mefuta eo ea likarolo tse tla lumellana. Re tla bua ka eona ka botlalo ka tlase.
Apache Bigtop
Apache Bigtop ke sesebelisoa sa ho aha, ho paka le ho leka palo ea
merero e bulehileng ea mohloli, joalo ka Hadoop le Greenplum. Bigtop e na le tse ngata
litokollo. Nakong ea ho ngola, tokollo ea morao-rao e tsitsitseng e ne e le mofuta oa 1.4,
mme ho master e ne e le 1.5. Litokomane tse fapaneng li sebelisa liphetolelo tse fapaneng
dikarolo. Ka mohlala, bakeng sa likarolo tsa mantlha tsa 1.4 Hadoop li na le version 2.8.5, le ka master
2.10.0. Sebopeho sa likarolo tse tšehetsoeng se boetse se fetoha. Ntho e siiloeng ke nako le
e sa tsosolositsoeng e ea tloha, 'me sebakeng sa eona ho tla ntho e ncha, e hlokahalang haholoanyane, le
ha ho hlile ha se ntho e tsoang ho lelapa la Apache ka bolona.
Ho feta moo, Bigtop e na le tse ngata .
Ha re qala ho tloaelana le Bigtop, re ile ra qala ho makatsoa ke boinyenyefatso ba eona, ha re bapisoa le merero e meng ea Apache, ho ata le ho tsebahala, hammoho le sechaba se senyenyane haholo. Ho latela ho tsoa ho sena hore ho na le tlhahisoleseding e fokolang ka sehlahisoa, 'me ho batla tharollo ea mathata a hlahileng liforamong le lethathamong la mangolo ho ka' na ha se ke ha hlahisa letho ho hang. Qalong, e ile ea e-ba mosebetsi o boima ho rona ho phethela kopano e feletseng ea kabo ka lebaka la likarolo tsa sesebelisoa ka boeona, empa re tla bua ka sena hamorao.
Joaloka motlatsi, bao ka nako e 'ngoe ba neng ba thahasella merero e joalo ea bokahohle ba Linux e kang Gentoo le LFS ba ka fumana ho le monate ho sebetsa le ntho ena le ho hopola linako tseo "tse mahlonoko" tseo rona ka borona re neng re batla (kapa re ngola) boilds le ho aha Mozilla khafetsa ka lipache tse ncha.
Molemo o moholo oa Bigtop ke ho buleha le ho feto-fetoha ha lisebelisoa tseo e itšetlehileng ka tsona. E thehiloe ho Gradle le Apache Maven. Gradle e tsebahala haholo joalo ka sesebelisoa seo Google e se sebelisang ho aha Android. E ea tenyetseha, ’me, joalokaha ba bolela, “e lekoa ntoeng.” Maven ke sesebelisoa se tloaelehileng sa merero ea kaho ho Apache ka boeona, 'me kaha boholo ba lihlahisoa tsa eona li lokolloa ka Maven, e ne e ke ke ea etsoa ntle le eona le mona. Ke habohlokoa ho ela hloko POM (mohlala oa morero) - "faele ea motheo" ea xml e hlalosang ntho e 'ngoe le e' ngoe e hlokahalang hore Maven a sebetse le morero oa hau, oo mosebetsi oohle o hahiloeng ho oona. Hantle feela ho
likarolo tsa Maven 'me ho na le litšitiso tseo basebelisi ba pele ba Bigtop ba atisang ho kopana le tsona.
Itloaetse
Joale u lokela ho qala hokae? Eya leqepheng la ho jarolla 'me u khoasolle mofuta oa morao-rao o tsitsitseng e le polokelo ea litaba. U ka boela ua fumana li-artifacts tsa binary tse bokelletsoeng ke Bigtop moo. Ka tsela, har'a batsamaisi ba liphutheloana tse tloaelehileng, YUM le APT lia tšehetsoa.
Ntle le moo, o ka khoasolla tokollo ea morao-rao e tsitsitseng ka kotloloho ho tsoa
github:
$ git clone --branch branch-1.4 https://github.com/apache/bigtop.gitHo kopanya "Bigtop"…
remote: Enumerating objects: 46, done.
remote: Counting objects: 100% (46/46), done.
remote: Compressing objects: 100% (41/41), done.
remote: Total 40217 (delta 14), reused 10 (delta 1), pack-reused 40171
Получение объектов: 100% (40217/40217), 43.54 MiB | 1.05 MiB/s, готово.
Определение изменений: 100% (20503/20503), готово.
Updating files: 100% (1998/1998), готово.Sephetho sa ./bigtop directory se shebahala tjena:
./bigtop-bigpetstore - lits'ebetso tsa demo, mehlala ea maiketsetso
./bigtop-ci - CI toolkit, jenkins
./bigtop-data-generators - tlhahiso ea data, synthetics, liteko tsa mosi, joalo-joalo.
./bigtop-deploy - lisebelisoa tsa ho tsamaisa
./bigtop-packages - configs, scripts, patches bakeng sa kopano, karolo e ka sehloohong ea sesebelisoa
./bigtop-test-framework - moralo oa teko
./bigtop-tests - liteko ka botsona, mojaro le ho tsuba
./bigtop_toolchain - tikoloho bakeng sa kopano, ho lokisa tikoloho bakeng sa sesebelisoa sa ho sebetsa
./build - theha bukana e sebetsang
./dl — directory bakeng sa mehloli e jarollotsoeng
./docker - ho aha litšoantšo tsa docker, liteko
./gradle - Gradle config
./output - bukana moo li-artifacts li eang teng
./provisioner - ho fana
Ntho e khahlisang haholo ho rona mothating ona ke config e kholo ./bigtop/bigtop.bom, eo ho eona re bonang likarolo tsohle tse tšehetsoeng ka liphetolelo. Mona ke moo re ka hlakisang mofuta o fapaneng oa sehlahisoa (haeba ka tšohanyetso re batla ho leka ho se haha) kapa mofuta oa kaho (haeba, ka mohlala, re kentse patch ea bohlokoa).
Bukana e nyenyane le eona e na le thahasello e kholo ./bigtop/bigtop-packages, e amanang ka ho toba le mokhoa oa ho bokella likarolo le liphutheloana le tsona.
Kahoo, re khoasolla li-archive, ra li lahlela kapa ra etsa clone ho tloha github, na re ka qala ho haha?
Che, ha re lokiseng tikoloho pele.
Ho Lokisetsa Tikoloho
'Me mona re hloka phomolo e nyane. Ho haha sehlahisoa se seng le se seng kapa se seng se rarahaneng, o hloka tikoloho e itseng - ho rona, ena ke JDK, lilaebrari tse tšoanang tse arolelanoang, lifaele tsa lihlooho, joalo-joalo, lisebelisoa, mohlala, bohloa, ivy2 le tse ling tse ngata. E 'ngoe ea likhetho tsa ho fumana tikoloho eo u e hlokang bakeng sa Bigtop ke ho kenya likarolo tse hlokahalang ho moamoheli oa kaho. Nka fosahetse ka tatellano ea liketsahalo, empa ho bonahala eka ka mofuta oa 1.0 ho ne ho boetse ho na le khetho ea ho aha litšoantšong tsa Docker tse hlophisitsoeng esale pele le tse fumanehang, tse ka fumanoang mona.
Ha e le ho lokisa tikoloho, ho na le mothusi oa sena - Puppet.
U ka sebelisa litaelo tse latelang, matha ho tloha bukeng ea motso
sesebelisoa, ./bigtop:
./gradlew toolchain
./gradlew toolchain-devtools
./gradlew toolchain-puppetmodules
Kapa ka ho toba ka popi:
puppet apply --modulepath=<path_to_bigtop> -e "include bigtop_toolchain::installer"
puppet apply --modulepath=<path_to_bigtop> -e "include bigtop_toolchain::deployment-tools"
puppet apply --modulepath=<path_to_bigtop> -e "include bigtop_toolchain::development-tools"Ka bomalimabe, mathata a se a ntse a ka hlaha nakong ena. Keletso e akaretsang mona ke ho sebelisa phepelo e tšehelitsoeng, ea morao-rao ho moamoheli oa kaho, kapa leka tsela ea docker.
Kopano
Re ka leka ho bokella eng? Karabo ea potso ena e tla fanoa ke tlhahiso ea taelo
./gradlew tasks
Karolong ea "Package tasks" ho na le lihlahisoa tse 'maloa tseo e leng li-artifact tsa ho qetela tsa Bigtop.
Li ka tsejoa ka sehlongoana -rpm kapa -pkg-ind (tabeng ea moaho
boema-kepe). Tabeng ea rona, e thahasellisang ka ho fetisisa ke Hadoop.
Ha re leke ho aha tikolohong ea seva ea rona ea kaho:
./gradlew hadoop-rpmBigtop ka boeona e tla khoasolla mehloli e hlokahalang bakeng sa karolo e itseng ebe e qala kopano. Kahoo, ts'ebetso ea sesebelisoa e itšetlehile ka polokelo ea Maven le mehloli e meng, ke hore, e hloka ho kena Inthaneteng.
Nakong ea ts'ebetso, tlhahiso e tloaelehileng e hlahisoa. Ka linako tse ling eona le melaetsa ea liphoso li ka u thusa ho utloisisa hore na ho ile ha etsahala eng. 'Me ka linako tse ling u hloka ho fumana lintlha tse eketsehileng. Tabeng ena ho loketse ho eketsa likhang --info kapa --debug, hape e ka ba molemo –stacktrace. Ho na le mokhoa o bonolo oa ho hlahisa sete ea data bakeng sa phihlello e latelang ea manane a poso, senotlolo --scan.
Ka thuso ea eona, bigtop e tla bokella tlhaiso-leseling eohle ebe e e beha butle, kamora moo e tla fana ka sehokelo,
ka ho latela seo, motho ea nang le bokhoni o tla khona ho utloisisa hore na ke hobane'ng ha kopano e hlōlehile.
Ka kopo hlokomela hore khetho ena e ka hlahisa tlhahisoleseling eo u sa e batleng, joalo ka mabitso a basebelisi, li-node, mefuta-futa ea tikoloho, joalo-joalo, kahoo ela hloko.
Hangata liphoso li bakoa ke ho se khone ho fumana likarolo tse hlokahalang bakeng sa kopano. Ka tloaelo, o ka lokisa bothata ka ho theha patch ho lokisa ntho e itseng mehloling, mohlala, liaterese ho pom.xml bukeng ea metso ea mehloli. Sena se etsoa ka ho bopa le ho se beha bukeng e nepahetseng ./bigtop/bigtop-packages/src/common/oozie/ patch, mohlala, ka sebopeho patch2-fix.diff.
--- a/pom.xml
+++ b/pom.xml
@@ -136,7 +136,7 @@
<repositories>
<repository>
<id>central</id>
- <url>http://repo1.maven.org/maven2</url>
+ <url>https://repo1.maven.org/maven2</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
Ho ka etsahala hore, nakong ea ho bala sengoloa sena, ha ho hlokahale hore u itokisetse ka holimo.
Ha u hlahisa li-patches le liphetoho leha e le life mochineng oa kopano, ho ka 'na ha hlokahala hore u "seta bocha" kopano u sebelisa taelo ea ho hloekisa:
./gradlew hadoop-clean
> Task :hadoop_vardefines
> Task :hadoop-clean
BUILD SUCCESSFUL in 5s
2 actionable tasks: 2 executedTs'ebetso ena e tla khutlisetsa liphetoho tsohle ho kopano ea karolo ena, ka mor'a moo kopano e tla etsoa hape. Lekhetlong lena re tla leka ho aha morero ka setšoantšo sa docker:
./gradlew -POS=centos-7 -Pprefix=1.2.1 hadoop-pkg-ind
> Task :hadoop-pkg-ind
Building 1.2.1 hadoop-pkg on centos-7 in Docker...
+++ dirname ./bigtop-ci/build.sh
++ cd ./bigtop-ci/..
++ pwd
+ BIGTOP_HOME=/tmp/bigtop
+ '[' 6 -eq 0 ']'
+ [[ 6 -gt 0 ]]
+ key=--prefix
+ case $key in
+ PREFIX=1.2.1
+ shift
+ shift
+ [[ 4 -gt 0 ]]
+ key=--os
+ case $key in
+ OS=centos-7
+ shift
+ shift
+ [[ 2 -gt 0 ]]
+ key=--target
+ case $key in
+ TARGET=hadoop-pkg
+ shift
+ shift
+ [[ 0 -gt 0 ]]
+ '[' -z x ']'
+ '[' -z x ']'
+ '[' '' == true ']'
+ IMAGE_NAME=bigtop/slaves:1.2.1-centos-7
++ uname -m
+ ARCH=x86_64
+ '[' x86_64 '!=' x86_64 ']'
++ docker run -d bigtop/slaves:1.2.1-centos-7 /sbin/init
+
CONTAINER_ID=0ce5ac5ca955b822a3e6c5eb3f477f0a152cd27d5487680f77e33fbe66b5bed8
+ trap 'docker rm -f
0ce5ac5ca955b822a3e6c5eb3f477f0a152cd27d5487680f77e33fbe66b5bed8' EXIT
....
много вывода
....
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-mapreduce-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-namenode-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-secondarynamenode-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-zkfc-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-journalnode-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-datanode-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-httpfs-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-resourcemanager-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-nodemanager-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-proxyserver-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-timelineserver-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-mapreduce-historyserver-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-client-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-conf-pseudo-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-doc-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-libhdfs-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-libhdfs-devel-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-fuse-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-debuginfo-2.8.5-1.el7.x86_64.rpm
+ umask 022
+ cd /bigtop/build/hadoop/rpm//BUILD
+ cd hadoop-2.8.5-src
+ /usr/bin/rm -rf /bigtop/build/hadoop/rpm/BUILDROOT/hadoop-2.8.5-1.el7.x86_64
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.uQ2FCn
+ exit 0
+ umask 022
Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.CwDb22
+ cd /bigtop/build/hadoop/rpm//BUILD
+ rm -rf hadoop-2.8.5-src
+ exit 0
[ant:touch] Creating /bigtop/build/hadoop/.rpm
:hadoop-rpm (Thread[Task worker for ':',5,main]) completed. Took 38 mins 1.151 secs.
:hadoop-pkg (Thread[Task worker for ':',5,main]) started.
> Task :hadoop-pkg
Task ':hadoop-pkg' is not up-to-date because:
Task has not declared any outputs despite executing actions.
:hadoop-pkg (Thread[Task worker for ':',5,main]) completed. Took 0.0 secs.
BUILD SUCCESSFUL in 40m 37s
6 actionable tasks: 6 executed
+ RESULT=0
+ mkdir -p output
+ docker cp
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb:/bigtop/build .
+ docker cp
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb:/bigtop/output .
+ docker rm -f ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
+ '[' 0 -ne 0 ']'
+ docker rm -f ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
Error: No such container:
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
BUILD SUCCESSFUL in 41m 24s
1 actionable task: 1 executedMoaho o entsoe tlasa CentOS, empa o ka etsoa hape tlasa Ubuntu:
./gradlew -POS=ubuntu-16.04 -Pprefix=1.2.1 hadoop-pkg-indNtle le liphutheloana tsa kaho bakeng sa liphallelo tse fapaneng tsa Linux, sesebelisoa se ka theha polokelo e nang le liphutheloana tse hlophisitsoeng, mohlala:
./gradlew yumU ka hopola hape ka liteko tsa mosi le ho romelloa ho Docker.
Etsa sehlopha sa li-node tse tharo:
./gradlew -Pnum_instances=3 docker-provisionerEtsa liteko tsa mosi ka sehlopha sa li-node tse tharo:
./gradlew -Pnum_instances=3 -Prun_smoke_tests docker-provisionerHlakola sehlopha:
./gradlew docker-provisioner-destroyFumana litaelo tsa ho hokela ka har'a lijana tsa docker:
./gradlew docker-provisioner-sshHlahisa boemo:
./gradlew docker-provisioner-statusU ka bala haholoanyane ka mesebetsi ea Deployment litokomaneng.
Haeba re bua ka liteko, ho na le palo e kholo ea tsona, haholo-holo ho tsuba le ho kopanya. Tlhahlobo ea bona e ka holimo ho sebaka sa sehlooho sena. E re ke bolele feela hore ho kopanya sephutheloana sa kabo ha se mosebetsi o boima joalo ka ha o ka bonahala o qala. Re khonne ho bokella le ho fetisa liteko ho likarolo tsohle tseo re li sebelisang tlhahisong ea rona, hape re ne re se na mathata a ho li tsamaisa le ho etsa ts'ebetso ea mantlha tikolohong ea liteko.
Ntle le likarolo tse teng ho Bigtop, hoa khoneha ho eketsa eng kapa eng, esita le nts'etsopele ea software ea hau. Sena sohle se itirile hantle 'me se lumellana le mohopolo oa CI/CD.
fihlela qeto e
Ho hlakile hore kabo e hlophisitsoeng ka tsela ena ha ea lokela ho romelloa hanghang tlhahisong. U lokela ho utloisisa hore haeba ho na le tlhokahalo ea sebele ea ho haha le ho tšehetsa kabo ea hau, joale u lokela ho kenya chelete le nako ho sena.
Leha ho le joalo, hammoho le mokhoa o nepahetseng le sehlopha sa litsebi, ho ka khoneha ho etsa ntle le tharollo ea khoebo.
Ke habohlokoa ho hlokomela hore morero oa Bigtop ka boeona o hloka ntlafatso 'me ha o bonahale o ntse o tsoela pele ka mafolofolo kajeno. Tšepo ea Hadoop 3 e hlahang ho eona le eona ha e hlake.Ka tsela, haeba u na le tlhoko ea sebele ea ho haha Hadoop 3, u ka sheba ho tloha Arenadata, eo ho eona, ho phaella ho maemo a tloaelehileng
Ho na le palo ea likarolo tse ling (Ranger, Knox, NiFi).
Ha e le Rostelecom, ho rona Bigtop ke e 'ngoe ea likhetho tse ntseng li nahanoa kajeno. Hore na rea e khetha kapa che, nako e tla bolela.
sehlomathiso
Ho kenyelletsa karolo e ncha kopanong, o hloka ho kenyelletsa tlhaloso ea eona ho bigtop.bom le ./bigtop-packages. U ka leka ho etsa sena ka papiso le likarolo tse teng. Leka ho e utloisisa. Ha e thata joalo ka ha e bonahala qalong.
U nahana eng? Re tla thabela ho bona maikutlo a hau ho maikutlo mme re leboha tlhokomelo ea hau!
Sengoloa se lokiselitsoe ke sehlopha sa taolo ea data ea Rostelecom
Source: www.habr.com