O nwere ike ọ bụghị ihe nzuzo na afọ gara aga bụ afọ nke nnukwu mgbanwe maka Apache Hadoop. N'afọ gara aga, Cloudera na Hortonworks jikọtara (n'ụzọ bụ isi, inweta nke ikpeazụ), na Mapr, n'ihi nnukwu nsogbu ego, e rere ya Hewlett Packard. Ma ọ bụrụ na afọ ole na ole gara aga, n'ihe banyere nrụnye ụlọ, a ga-emerịrị nhọrọ n'etiti Cloudera na Hortonworks, taa, Ewoo, anyị enweghị nhọrọ a. Ihe ọzọ dị ịtụnanya bụ eziokwu ahụ bụ na Cloudera kwupụtara na February nke afọ a na ọ ga-akwụsị ịhapụ mgbakọ ọnụọgụ abụọ nke nkesa ya na ebe nchekwa ọha, na ha dị ugbu a naanị site na ndenye aha akwụ ụgwọ. N'ezie, ọ ka nwere ike ibudata ụdị CDH na HDP kachasị ọhụrụ ewepụtara tupu njedebe nke 2019, a na-atụkwa anya nkwado maka ha otu afọ ruo afọ abụọ. Ma gịnị ka a ga-eme ọzọ? Maka ndị akwụburu ụgwọ maka ndenye aha, ọ nweghị ihe gbanwere. Na maka ndị na-achọghị ịgbanwe gaa na ụdị nkesa a na-akwụ ụgwọ, ma n'otu oge ahụ chọrọ ka ha nwee ike ịnweta ụdị ụyọkọ kachasị ọhụrụ, yana patches na mmelite ndị ọzọ, anyị akwadola isiokwu a. N'ime ya, anyị ga-atụle nhọrọ ndị nwere ike ịpụ na ọnọdụ a.
Edemede a bụ nyocha karịa. Ọ gaghị enwe ntụnyere nkesa na nyocha zuru ezu banyere ha, ọ gaghị enwekwa ntụziaka maka ịwụnye na ịhazi ha. Gịnị ga-eme? Anyị ga-ekwu nkenke maka nkesa dị otú ahụ dị ka Arenadata Hadoop, nke kwesịrị ka anyị lebara anya n'ihi na ọ dị, nke dị ụkọ taa. Ma mgbe ahụ, anyị ga-ekwu maka Vanilla Hadoop, tumadi maka otu esi esi “esi ya” site na iji Apache Bigtop. Ọ dịla njikere? Mgbe ahụ, nabata nwamba.
Arenadata Hadoop
Nke a bụ ihe ọhụrụ kpamkpam na, ka ọ dị ugbu a, ngwa nkesa a na-amachaghị maka mmepe ụlọ. Ọ dị nwute na ugbu a na Habré enwere naanị .
Enwere ike ịchọta ozi ndị ọzọ na gọọmentị oru ngo. Ụdị nkesa kachasị ọhụrụ dabere na Hadoop 3.1.2 maka ụdị 3, yana 2.8.5 maka ụdị 2.
Enwere ike ịchọta ozi gbasara map okporo ụzọ .
Interface njikwa ụyọkọ Arenadata
Isi ngwaahịa Arenadata bụ , nke a na-eji wụnye, hazie na nyochaa ngwọta ngwanrọ ụlọ ọrụ dị iche iche. A na-ekesa ADCM n'efu, a na-agbasawanyekwa ọrụ ya site na ịgbakwunye ngwugwu, nke bụ akwụkwọ mpịakọta nwere ike ime. A na-ekewa ùkwù ụzọ abụọ: ụlọ ọrụ na obodo. Nke ikpeazụ dị maka nbudata n'efu site na webụsaịtị Arenadata. Ọ dịkwa ike ịmepụta ùkwù nke gị wee jikọọ ya na ADCM.
Maka ibuga na njikwa nke Hadoop 3, a na-enye ụdị ngwugwu a na njikọ ADCM, mana maka Hadoop 2 enwere naanị dị ka ihe ọzọ. Banyere ebe nchekwa nwere ngwugwu, ha na-emeghe ohere ọha, enwere ike ibudata ma tinye ya n'ụzọ ọ na-adịbu maka akụkụ niile nke ụyọkọ ahụ. N'ozuzu, nkesa na-ele anya nke ukwuu. Eji m n'aka na a ga-enwe ndị maara ihe ngwọta dịka Cloudera Manager na Ambri, na onye ga-amasị ADCM n'onwe ya. Maka ụfọdụ, ọ ga-abụkwa nnukwu gbakwunyere na nkesa maka nnọchi mbubata.
Ọ bụrụ na anyị na-ekwu maka ọghọm dị, ha ga-adị ka nkesa Hadoop ndị ọzọ niile. Ya bụ:
- Ihe a na-akpọ "mkpọchi onye na-ere ahịa". N'iji ihe atụ nke Cloudera na Hortonworks, anyị achọpụtalarị na enwere ihe ize ndụ nke ịgbanwe iwu ụlọ ọrụ.
- Ọdịda dị mkpa n'azụ Apache mgbago elu.
Vanilla Hadoop
Dị ka ị maara, Hadoop abụghị ngwaahịa monolithic, mana, n'ezie, otu ụyọkọ kpakpando nke ọrụ gburugburu sistemụ faịlụ HDFS ya kesara. Mmadụ ole na ole ga-ezuru otu ụyọkọ faịlụ. Ụfọdụ chọrọ Hive, ndị ọzọ Presto, mgbe ahụ enwere HBase na Phoenix; A na-eji Spark eme ihe. Maka nhazi na nbudata data, Oozie, Sqoop na Flume na-ahụ mgbe ụfọdụ. Ma ọ bụrụ na okwu nke nchekwa bilitere, mgbe ahụ Kerberos na njikọ Ranger na-abata n'uche ozugbo.
Ụdị ọnụọgụ abụọ nke akụrụngwa Hadoop dị na webụsaịtị nke ọrụ gburugburu ebe obibi ọ bụla n'ụdị tarballs. Ị nwere ike budata ha wee malite nrụnye, mana n'otu ọnọdụ: na mgbakwunye na ịchịkọta ngwugwu n'onwe ya site na ọnụọgụ abụọ "raw", nke ị nwere ike ime, ị gaghị enwe obi ike na ndakọrịta nke ụdị ngwa ndị ebudatara na nke ọ bụla. ọzọ. Nhọrọ kacha mma bụ iji Apache Bigtop wuo. Bigtop ga-enye gị ohere ịrụ site na ebe nchekwa Apache maven, mee ule ma wuo ngwugwu. Mana, ihe dị anyị ezigbo mkpa, Bigtop ga-achịkọta ụdị ngwa ndị ahụ ga-adakọbara na ibe ha. Anyị ga-ekwu maka ya n'ụzọ zuru ezu n'okpuru.
Apache Bigtop
Apache Bigtop bụ ngwá ọrụ maka iwulite, nkwakọ ngwaahịa na ịnwale ọnụọgụgụ
oru ngo mepere emepe, dị ka Hadoop na Greenplum. Bigtop nwere ọtụtụ
ewepụtara. N'oge edere, ntọhapụ kwụsiri ike kachasị ọhụrụ bụ ụdị 1.4,
na nna ukwu enwere 1.5. Ụdị ntọhapụ dị iche iche na-eji ụdị dị iche iche
components. Dịka ọmụmaatụ, maka 1.4 Hadoop isi components nwere ụdị 2.8.5, yana na nna ukwu
2.10.0. Ngwakọta nke akụrụngwa akwadoro na-agbanwekwa. Ihe merela ochie na
ndị na-adịghị ọhụrụ na-apụ apụ, na n'ọnọdụ ya na-abịa ihe ọhụrụ, ọzọ na-achọ, na
ọ bụchaghị ihe sitere na ezinụlọ Apache n'onwe ya.
Na mgbakwunye, Bigtop nwere ọtụtụ .
Mgbe anyị malitere ịmara Bigtop, ọ tụrụ anyị n'anya nke ukwuu maka ịdị umeala n'obi ya, ma e jiri ya tụnyere ọrụ Apache ndị ọzọ, mgbasa na ewu ewu, yana obere obodo. Ọ na-esite na nke a na enwere ntakịrị ozi na ngwaahịa ahụ, na ịchọ ngwọta maka nsogbu ndị bilitere na forums na ndepụta nzipu ozi nwere ike ọ gaghị emepụta ihe ọ bụla. Na mbụ, ọ bịara bụrụ ọrụ siri ike maka anyị iji mezue mgbakọ zuru ezu nke nkesa n'ihi njirimara nke ngwá ọrụ ahụ n'onwe ya, ma anyị ga-ekwu banyere nke a obere oge.
Dị ka teaser, ndị nwere mmasị n'otu oge na ọrụ ndị dị otú ahụ nke eluigwe na ala Linux dị ka Gentoo na LFS nwere ike ịhụ na ọ na-atọ ụtọ ịrụ ọrụ na ihe a ma cheta oge "epic" mgbe anyị onwe anyị na-achọ (ma ọ bụ ọbụna na-ede) ebuils ma jiri patches ọhụrụ na-ewughachi Mozilla mgbe niile.
Nnukwu uru nke Bigtop bụ imeghe na ntụgharị nke ngwaọrụ ndị ọ dabere na ya. Ọ dabere na Gradle na Apache Maven. A maara Gradle nke ọma dị ka ngwa Google na-eji wuo gam akporo. Ọ na-agbanwe agbanwe, na, dị ka ha na-ekwu, "agha nwalere." Maven bụ ọkọlọtọ eji arụ ọrụ ụlọ na Apache n'onwe ya, ebe ọ bụ na ewepụtara ọtụtụ ngwaahịa ya site na Maven, enweghị ike ịme ya na-enweghị ya ebe a. Ọ bara uru ịṅa ntị na POM (ihe nlereanya ihe oru ngo) - faịlụ xml "isi" na-akọwa ihe niile dị mkpa maka Maven na-arụ ọrụ na ọrụ gị, nke a na-arụ ọrụ niile. Kpọmkwem na
akụkụ nke Maven na enwere ụfọdụ ihe mgbochi ndị ọrụ Bigtop oge mbụ na-ezutekarị.
Mee ihe
Yabụ kedu ebe ị ga-ebido? Gaa na ibe nbudata wee budata ụdị kwụsiri ike kachasị ọhụrụ dị ka ebe nchekwa. Ị nwekwara ike ịhụ arịa ọnụọgụ abụọ nke Bigtop nakọtara ebe ahụ. Site n'ụzọ, n'etiti ndị njikwa ngwugwu nkịtị, YUM na APT na-akwado.
N'aka nke ọzọ, ịnwere ike ibudata ntọhapụ kwụsiri ike kachasị ọhụrụ ozugbo na
github:
$ git clone --branch branch-1.4 https://github.com/apache/bigtop.gitNa-emechi na "bigtop"…
remote: Enumerating objects: 46, done.
remote: Counting objects: 100% (46/46), done.
remote: Compressing objects: 100% (41/41), done.
remote: Total 40217 (delta 14), reused 10 (delta 1), pack-reused 40171
Получение объектов: 100% (40217/40217), 43.54 MiB | 1.05 MiB/s, готово.
Определение изменений: 100% (20503/20503), готово.
Updating files: 100% (1998/1998), готово.Akwụkwọ ndekọ aha ./bigtop pụtara ihe dị ka nke a:
./bigtop-bigpetstore - ngwa ngosi, ihe atụ sịntetik
./bigtop-ci - CI ngwaọrụ, jenkins
./bigtop-data-generators - ọgbọ data, sịntetik, maka ule anwụrụ ọkụ, wdg.
./bigtop-deploy - ntinye ngwaọrụ
./bigtop-packages - configs, scripts, patches maka mgbakọ, akụkụ bụ isi nke ngwá ọrụ
./bigtop-test-framework - usoro nyocha
./bigtop-tests - ule onwe ha, ibu na anwụrụ ọkụ
./bigtop_toolchain - gburugburu ebe obibi maka mgbakọ, na-akwadebe gburugburu maka ngwá ọrụ na-arụ ọrụ
./build - wuo akwụkwọ ndekọ aha
./dl - ndekọ maka isi mmalite ebudatara
./docker - ewu n'ime ihe onyonyo docker, nnwale
./gradle - gradle config
./output - akwụkwọ ndekọ ebe ihe nrụpụta na-aga
./provisioner - ndokwa
Ihe kacha amasị anyị n'oge a bụ isi nhazi ./bigtop/bigtop.bom, nke anyị na-ahụ ihe niile akwadoro nwere nsụgharị. Nke a bụ ebe anyị nwere ike ịkọwapụta ụdị ngwaahịa dị iche iche (ma ọ bụrụ na anyị chọrọ na mberede na-agbalị ịmepụta ya) ma ọ bụ ihe nrụpụta (ma ọ bụrụ na, dịka ọmụmaatụ, anyị gbakwunyere patch dị ịrịba ama).
Subdirectory nwekwara mmasị dị ukwuu ./bigtop/bigtop-packages, nke na-emetụta kpọmkwem na usoro nke ikpokọta components na ngwugwu na ha.
Yabụ, anyị budata ebe nchekwa ahụ, bupu ya ma ọ bụ mee clone site na github, anyị nwere ike ịmalite iwu ụlọ?
Mba, ka anyị buru ụzọ kwadebe gburugburu ebe obibi.
Ịkwadebe gburugburu
Na ebe a, anyị chọrọ obere mgbagha. Iji wuo ihe fọrọ nke nta ka ọ bụrụ ngwaahịa ọ bụla ma ọ bụ na-adịchaghị mgbagwoju anya, ịchọrọ mpaghara ụfọdụ - n'ọnọdụ anyị, nke a bụ JDK, otu ọba akwụkwọ, faịlụ nkụnye eji isi mee, wdg, ngwaọrụ, dịka ọmụmaatụ, ant, ivy2 na ọtụtụ ndị ọzọ. Otu nhọrọ iji nweta gburugburu ebe ị chọrọ maka Bigtop bụ ịwụnye ihe ndị dị mkpa na onye na-ewu ụlọ. Enwere m ike ịmehie na usoro oge, mana ọ dị ka na ụdị 1.0 enwekwara nhọrọ iji wuo na onyonyo Docker ahazigoro na ịnweta, nke enwere ike ịhụ ebe a.
Maka ịkwadebe gburugburu ebe obibi, enwere onye inyeaka maka nke a - Puppet.
Ị nwere ike iji iwu ndị a, na-agba ọsọ site na ndekọ ndekọ
ngwá ọrụ, ./bigtop:
./gradlew toolchain
./gradlew toolchain-devtools
./gradlew toolchain-puppetmodules
Ma ọ bụ ozugbo site na puppet:
puppet apply --modulepath=<path_to_bigtop> -e "include bigtop_toolchain::installer"
puppet apply --modulepath=<path_to_bigtop> -e "include bigtop_toolchain::deployment-tools"
puppet apply --modulepath=<path_to_bigtop> -e "include bigtop_toolchain::development-tools"O di nwute, ihe isi ike nwere ike ibilite ugbua n'oge a. Ndụmọdụ izugbe ebe a bụ iji nkesa akwadoro, emelitere na onye nrụpụta ụlọ, ma ọ bụ nwalee ụzọ docker.
Mgbakọ
Kedu ihe anyị nwere ike ịnwa ịnakọta? A ga-enye azịza nke ajụjụ a site na mmepụta nke iwu ahụ
./gradlew tasks
Na ngalaba ihe aga-eme ngwugwu enwere ọtụtụ ngwaahịa bụ arịa ikpeazụ nke Bigtop.
Enwere ike mata ha site na suffix -rpm ma ọ bụ -pkg-ind (n'ihe gbasara ụlọ
na docker). N'ọnọdụ anyị, ihe kacha adọrọ mmasị bụ Hadoop.
Ka anyị gbalịa wuo na gburugburu ebe nchekwa ihe nkesa anyị:
./gradlew hadoop-rpmBigtop n'onwe ya ga-ebudata isi mmalite ndị dị mkpa maka otu akụrụngwa wee malite mgbakọ. Ya mere, ọrụ ngwá ọrụ ahụ dabere na ebe nchekwa Maven na isi mmalite ndị ọzọ, ya bụ, ọ chọrọ ịnweta ịntanetị.
N'oge ọrụ, a na-emepụta mmepụta ọkọlọtọ. Mgbe ụfọdụ, ya na ozi njehie nwere ike inyere gị aka ịghọta ihe na-ezighi ezi. Ma mgbe ụfọdụ ị ga-achọ inweta ozi ndị ọzọ. N'okwu a, ọ bara uru ịgbakwunye arụmụka --info ma ọ bụ --debug, ma nwekwara ike ịba uru –stacktrace. Enwere ụzọ dabara adaba iji wepụta ntọala data maka ịnweta ndepụta nzipu ozi na-esote, igodo --scan.
Site n'enyemaka ya, bigtop ga-anakọta ozi niile ma tinye ya na gradle, mgbe nke ahụ gasịrị, ọ ga-enye njikọ.
site n'ime nke a, onye tozuru etozu ga-enwe ike ịghọta ihe mere mgbakọ ahụ dara.
Biko mara na nhọrọ a nwere ike ikpughe ozi ị na-achọghị, dị ka aha njirimara, ọnụ, mgbanwe gburugburu ebe obibi, wdg, yabụ kpachara anya.
Ọtụtụ mgbe, njehie na-esite na enweghị ike ịnweta ihe ọ bụla dị mkpa maka mgbakọ. Dịka, ị nwere ike idozi nsogbu ahụ site na ịmepụta patch iji dozie ihe dị na isi mmalite, dịka ọmụmaatụ, adreesị na pom.xml na ndekọ mgbọrọgwụ nke isi mmalite. A na-eme nke a site na ịmepụta na itinye ya na ndekọ kwesịrị ekwesị ./bigtop/bigtop-packages/src/common/oozie/ patch, ọmụmaatụ, n'ụdị patch2-fix.diff.
--- a/pom.xml
+++ b/pom.xml
@@ -136,7 +136,7 @@
<repositories>
<repository>
<id>central</id>
- <url>http://repo1.maven.org/maven2</url>
+ <url>https://repo1.maven.org/maven2</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
O yikarịrị, n'oge ị na-agụ akụkọ a, ị gaghị eme nke a n'elu dozie onwe gị.
Mgbe ị na-ewebata patches ọ bụla na mgbanwe na usoro mgbakọ, ị nwere ike iji iwu nhichapụ "tọgharịa" mgbakọ ahụ:
./gradlew hadoop-clean
> Task :hadoop_vardefines
> Task :hadoop-clean
BUILD SUCCESSFUL in 5s
2 actionable tasks: 2 executedỌrụ a ga-atụgharị mgbanwe niile na mgbakọ nke akụrụngwa a, mgbe nke ahụ gasịrị, a ga-eme mgbakọ ahụ ọzọ. Oge a, anyị ga-agbalị ịrụ ọrụ a na foto docker:
./gradlew -POS=centos-7 -Pprefix=1.2.1 hadoop-pkg-ind
> Task :hadoop-pkg-ind
Building 1.2.1 hadoop-pkg on centos-7 in Docker...
+++ dirname ./bigtop-ci/build.sh
++ cd ./bigtop-ci/..
++ pwd
+ BIGTOP_HOME=/tmp/bigtop
+ '[' 6 -eq 0 ']'
+ [[ 6 -gt 0 ]]
+ key=--prefix
+ case $key in
+ PREFIX=1.2.1
+ shift
+ shift
+ [[ 4 -gt 0 ]]
+ key=--os
+ case $key in
+ OS=centos-7
+ shift
+ shift
+ [[ 2 -gt 0 ]]
+ key=--target
+ case $key in
+ TARGET=hadoop-pkg
+ shift
+ shift
+ [[ 0 -gt 0 ]]
+ '[' -z x ']'
+ '[' -z x ']'
+ '[' '' == true ']'
+ IMAGE_NAME=bigtop/slaves:1.2.1-centos-7
++ uname -m
+ ARCH=x86_64
+ '[' x86_64 '!=' x86_64 ']'
++ docker run -d bigtop/slaves:1.2.1-centos-7 /sbin/init
+
CONTAINER_ID=0ce5ac5ca955b822a3e6c5eb3f477f0a152cd27d5487680f77e33fbe66b5bed8
+ trap 'docker rm -f
0ce5ac5ca955b822a3e6c5eb3f477f0a152cd27d5487680f77e33fbe66b5bed8' EXIT
....
много вывода
....
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-mapreduce-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-namenode-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-secondarynamenode-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-zkfc-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-journalnode-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-datanode-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-httpfs-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-resourcemanager-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-nodemanager-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-proxyserver-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-yarn-timelineserver-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-mapreduce-historyserver-2.8.5-
1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-client-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-conf-pseudo-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-doc-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-libhdfs-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-libhdfs-devel-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-hdfs-fuse-2.8.5-1.el7.x86_64.rpm
Wrote: /bigtop/build/hadoop/rpm/RPMS/x86_64/hadoop-debuginfo-2.8.5-1.el7.x86_64.rpm
+ umask 022
+ cd /bigtop/build/hadoop/rpm//BUILD
+ cd hadoop-2.8.5-src
+ /usr/bin/rm -rf /bigtop/build/hadoop/rpm/BUILDROOT/hadoop-2.8.5-1.el7.x86_64
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.uQ2FCn
+ exit 0
+ umask 022
Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.CwDb22
+ cd /bigtop/build/hadoop/rpm//BUILD
+ rm -rf hadoop-2.8.5-src
+ exit 0
[ant:touch] Creating /bigtop/build/hadoop/.rpm
:hadoop-rpm (Thread[Task worker for ':',5,main]) completed. Took 38 mins 1.151 secs.
:hadoop-pkg (Thread[Task worker for ':',5,main]) started.
> Task :hadoop-pkg
Task ':hadoop-pkg' is not up-to-date because:
Task has not declared any outputs despite executing actions.
:hadoop-pkg (Thread[Task worker for ':',5,main]) completed. Took 0.0 secs.
BUILD SUCCESSFUL in 40m 37s
6 actionable tasks: 6 executed
+ RESULT=0
+ mkdir -p output
+ docker cp
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb:/bigtop/build .
+ docker cp
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb:/bigtop/output .
+ docker rm -f ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
+ '[' 0 -ne 0 ']'
+ docker rm -f ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
Error: No such container:
ac46014fd9501bdc86b6c67d08789fbdc6ee46a2645550ff6b6712f7d02ffebb
BUILD SUCCESSFUL in 41m 24s
1 actionable task: 1 executedEmere ihe owuwu ahụ n'okpuru CentOS, mana enwere ike ịme ya n'okpuru Ubuntu:
./gradlew -POS=ubuntu-16.04 -Pprefix=1.2.1 hadoop-pkg-indNa mgbakwunye na ngwungwu ewu maka nkesa Linux dị iche iche, ngwá ọrụ ahụ nwere ike ịmepụta ebe nchekwa na ngwugwu agbakọtara, dịka ọmụmaatụ:
./gradlew yumỊ nwekwara ike icheta maka ule anwụrụ ọkụ na mbugharị na Docker.
Mepụta ụyọkọ ọnụ ụzọ atọ:
./gradlew -Pnum_instances=3 docker-provisionerGbaa ule anwụrụ ọkụ n'ụyọkọ nke ọnụ atọ:
./gradlew -Pnum_instances=3 -Prun_smoke_tests docker-provisionerHichapụ ụyọkọ:
./gradlew docker-provisioner-destroyNweta iwu maka ijikọ n'ime arịa docker:
./gradlew docker-provisioner-sshGosi ọkwa:
./gradlew docker-provisioner-statusỊ nwere ike gụkwuo gbasara ọrụ ntinye n'ime akwụkwọ.
Ọ bụrụ na anyị na-ekwu banyere ule, e nwere nnọọ nnukwu ọnụ ọgụgụ nke ha, tumadi anwụrụ ọkụ na mwekota. Nyocha ha karịrị akarị nke isiokwu a. Ka m kwuo na ịchịkọta ngwa nkesa abụghị ọrụ siri ike dịka ọ nwere ike iyi na nlele mbụ. Anyị jisiri ike gbakọta ma gafere ule na ihe niile anyị na-eji na mmepụta anyị, anyị enweghịkwa nsogbu ibugharị ha na ịrụ ọrụ ndị bụ isi na gburugburu ule.
Na mgbakwunye na ihe ndị dị na Bigtop, enwere ike ịgbakwunye ihe ọ bụla ọzọ, ọbụlagodi mmepe ngwanrọ nke gị. Ihe ndị a niile bụ akpaaka zuru oke ma dabara na echiche CI/CD.
nkwubi
N'ụzọ doro anya, nkesa achịkọtara n'ụzọ dị otú a ekwesịghị iziga ozugbo na mmepụta. Ịkwesịrị ịghọta na ọ bụrụ na ọ dị ezigbo mkpa iji wuo ma kwado nkesa gị, mgbe ahụ ịkwesịrị itinye ego na oge na nke a.
Otú ọ dị, n'ijikọta ya na ụzọ ziri ezi na otu ọkachamara, ọ ga-ekwe omume ime na-enweghị ngwọta azụmahịa.
Ọ dị mkpa ịmara na ọrụ Bigtop n'onwe ya chọrọ mmepe na ọ naghị adị ka a na-arụsi ọrụ ike taa. Atụmanya nke Hadoop 3 na-apụta na ya edoghịkwa anya. Site n'ụzọ, ọ bụrụ na ị nwere ezigbo mkpa iji wuo Hadoop 3, ị nwere ike ile anya. site na Arenadata, nke, na mgbakwunye na ọkọlọtọ
Enwere ọtụtụ ngwa mgbakwunye (Ranger, Knox, NiFi).
Maka Rostelecom, maka anyị Bigtop bụ otu n'ime nhọrọ ndị a na-atụle taa. Ma ànyị ahọrọ ya ma ọ bụ na anyị ahọrọghị, oge ga-egosi.
Odide Ntụkwasị
Iji tinye akụkụ ọhụrụ na mgbakọ ahụ, ịkwesịrị ịgbakwunye nkọwa ya na bigtop.bom na ./bigtop-packages. Ị nwere ike ịgbalị ime nke a site na ntụnyere ihe ndị dị adị. Gbalịa chọpụta ya. Ọ bụghị ihe siri ike dị ka ọ dị na nlele mbụ.
Kedu ihe ị chere? Anyị ga-enwe obi ụtọ ịhụ echiche gị na nkwupụta ma kelee gị maka nlebara anya gị!
Ndị otu njikwa data Rostelecom kwadoro akụkọ a
isi: www.habr.com