Tso tawm ntawm lub platform rau kev faib cov ntaub ntawv ua haujlwm Apache Hadoop 3.3

Tom qab ib xyoos thiab ib nrab ntawm txoj kev loj hlob, Apache Software Foundation luam tawm tso tawm Apache Hadoop 3.3.0, lub platform pub dawb rau kev teeb tsa kev faib tawm ntawm cov ntaub ntawv loj loj uas siv lub ntsiab lus map / txo, nyob rau hauv uas cov hauj lwm tau muab faib ua ntau me me sib cais, txhua yam uas yuav tsum tau launched rau ib tug cais pawg node. Hadoop-based cia tuaj yeem hla ntau txhiab ntawm cov nodes thiab muaj cov ntaub ntawv exabytes.

Hadoop suav nrog kev siv ntawm Hadoop Distributed Filesystem (HDFS), uas cia li muab cov ntaub ntawv thaub qab thiab ua kom zoo rau daim ntawv thov MapReduce. Txhawm rau kom yooj yim nkag mus rau cov ntaub ntawv hauv Hadoop cia, HBase database thiab SQL-zoo li hom lus npua tau tsim, uas yog hom SQL rau MapReduce, cov lus nug uas tuaj yeem sib luag thiab ua tiav los ntawm ntau lub Hadoop platforms. Qhov project yog ntsuas raws li ruaj khov thiab npaj txhij rau kev lag luam. Hadoop tau nquag siv hauv kev lag luam loj, muab lub peev xwm zoo ib yam li Google Bigtable / GFS / MapReduce platform, thaum Google tau ua tiav. tso cai Hadoop thiab lwm txoj haujlwm Apache muaj cai siv cov thev naus laus zis uas tau them los ntawm patents ntsig txog MapReduce txoj kev.

Hadoop ranked thawj ntawm Apache repositories nyob rau hauv cov nqe lus ntawm tus naj npawb ntawm cov kev hloov pauv tau ua thiab thib tsib nyob rau hauv cov nqe lus ntawm codebase loj (txog 4 lab kab ntawm code). Kev siv Hadoop tseem ceeb suav nrog Netflix (ntau dua 500 billion txheej xwm hauv ib hnub tau khaws cia), Twitter (ib pawg ntawm 10 txhiab nodes khw muag khoom ntau dua li zettabyte ntawm cov ntaub ntawv hauv lub sijhawm tiag tiag thiab cov txheej txheem ntau dua 5 billion zaug ib hnub), Facebook (ib pawg. ntawm 4 txhiab nodes khw muag khoom ntau dua 300 petabytes thiab nce txhua hnub los ntawm 4 PB ib hnub).

ntsiab hloov Hauv Apache Hadoop 3.3:

• Ntxiv kev txhawb nqa rau lub platform raws li ARM architecture.
• Kev ua raws li hom ntawv Protobuf (Txoj Cai buffers), siv rau serializing cov ntaub ntawv tsim qauv, tau hloov kho kom tso tawm 3.7.1 vim qhov kawg ntawm lub neej voj voog ntawm protobuf-2.5.0 ceg.
• Lub peev xwm ntawm S3A txuas tau nthuav dav: kev txhawb nqa rau kev lees paub siv tokens tau ntxiv (Delegation Token), txhim kho kev txhawb nqa rau caching cov lus teb nrog code 404, nce S3guard kev ua tau zoo, thiab ua kom muaj kev ntseeg siab ntxiv.
• Cov teeb meem nrog kev hloov kho tsis siv neeg tau raug daws hauv ABFS cov ntaub ntawv.
• Ntxiv ib qho kev txhawb nqa rau Tencent Cloud COS cov ntaub ntawv rau kev nkag mus rau COS cov khoom khaws cia.
• Ntxiv kev txhawb nqa tag nrho rau Java 11.
• Kev siv ntawm HDFS RBF (Router-based Federation) tau ruaj khov. Kev tswj hwm kev ruaj ntseg tau ntxiv rau HDFS Router.
• Ntxiv qhov kev pabcuam DNS daws teeb meem rau tus neeg siv khoom los txiav txim siab cov servers ntawm DNS los ntawm tus tswv lub npe, tso cai rau koj ua yam tsis sau tag nrho cov tswv hauv qhov chaw.
• Ntxiv kev txhawb nqa lub sijhawm teem sijhawm opportunistic ntim los ntawm tus thawj tswj hwm kev pabcuam hauv nruab nrab (ResourceManager), suav nrog lub peev xwm los faib cov ntim khoom noj rau hauv tus account lub nra ntawm txhua qhov ntawm.
• Ntxiv tshawb nrhiav YARN (Tau Lwm Tus Neeg Sib Tham) daim ntawv thov npe.

Tau qhov twg los: opennet.ru