Tom qab ib xyoos thiab ib nrab ntawm kev txhim kho, Apache Software Foundation tso tawm , ib lub platform pub dawb rau kev teeb tsa cov txheej txheem faib tawm ntawm cov ntaub ntawv loj siv cov qauv , uas ib txoj haujlwm raug muab faib ua ntau daim me me, sib cais, txhua daim tuaj yeem khiav ntawm ib lub cluster node sib cais. Ib qho chaw cia khoom raws li Hadoop tuaj yeem nthuav dav ntau txhiab lub nodes thiab muaj exabytes ntawm cov ntaub ntawv.
Hadoop muaj xws li kev siv Hadoop Distributed Filesystem (HDFS), uas muab cov ntaub ntawv rov ua dua thiab tau kho kom zoo rau MapReduce daim ntawv thov. Txhawm rau kom yooj yim nkag mus rau cov ntaub ntawv hauv Hadoop cia, lub database HBase thiab SQL-zoo li lus Pig tau tsim. Pig yog ib hom SQL rau MapReduce, uas nws cov lus nug tuaj yeem sib luag thiab ua tiav los ntawm ntau lub platform Hadoop. Qhov project raug soj ntsuam tias ruaj khov thiab npaj txhij rau kev siv hauv kev lag luam. Hadoop siv nquag hauv cov haujlwm lag luam loj, muab cov peev xwm zoo ib yam li Google Bigtable / GFS / MapReduce platform, thiab Google tau ua tiav ... Hadoop thiab lwm yam Apache projects tau txais txoj cai siv cov thev naus laus zis uas muaj cov ntawv pov thawj cuam tshuam nrog MapReduce method.
Hadoop yog thawj qhov chaw khaws cia ntawm Apache los ntawm tus lej ntawm kev cog lus thiab thib tsib los ntawm qhov loj ntawm codebase (kwv yees li 4 lab kab ntawm cov lej). Cov kev siv Hadoop tseem ceeb suav nrog Netflix (uas khaws ntau dua 500 billion qhov xwm txheej ib hnub), Twitter (10-node cluster khaws ntau dua ib zettabyte ntawm cov ntaub ntawv tiag tiag thiab cov txheej txheem ntau dua 5 billion sessions ib hnub), thiab Facebook (4-node cluster khaws ntau dua 300 petabytes thiab loj hlob los ntawm 4 petabytes ib hnub).
ntsiab Hauv Apache Hadoop 3.3:
- Ntxiv kev txhawb nqa rau cov platforms raws li ARM architecture.
- Kev siv hom ntawv (Protocol buffers), siv rau kev ua cov ntaub ntawv teeb tsa, tau hloov kho tshiab rau qhov tso tawm 3.7.1 vim yog qhov kawg ntawm lub neej ntawm ceg protobuf-2.5.0.
- Lub S3A connector tau nthuav dav los txhawb kev lees paub siv cov tokens (), kev txhawb nqa zoo dua rau cov lus teb caching nrog 404 code, kev ua haujlwm S3guard nce ntxiv, thiab kev ntseeg tau zoo dua.
- Lub kaw lus ntaub ntawv ABFS tau daws cov teeb meem nrog kev kho qhov tsis siv neeg.
- Ntxiv kev txhawb nqa rau Tencent Cloud COS cov ntaub ntawv system kom nkag mus rau COS khoom cia.
- Ntxiv kev txhawb nqa tag nrho rau Java 11.
- Qhov kev siv HDFS RBF (Router-based Federation) tau ruaj khov lawm. Kev tswj kev ruaj ntseg tau ntxiv rau HDFS Router.
- Ntxiv qhov kev pabcuam DNS Resolution kom tus neeg siv khoom txiav txim siab cov servers ntawm DNS los ntawm cov npe host, tshem tawm qhov xav tau los teev txhua tus hosts hauv qhov chaw.
- Ntxiv kev txhawb nqa rau kev teem sijhawm tso tawm los ntawm tus thawj tswj hwm cov peev txheej hauv nruab nrab (ResourceManager), suav nrog lub peev xwm faib cov thawv raws li qhov thauj khoom ntawm txhua lub node.
- Ntxiv ib daim ntawv teev npe thov YARN (Yet Another Resource Negotiator) uas tuaj yeem tshawb nrhiav tau.
Tau qhov twg los: opennet.ru
