Cov ntaub ntawv compression hauv Apache Ignite. Sber kev paub

Cov ntaub ntawv compression hauv Apache Ignite. Sber kev paubThaum ua haujlwm nrog cov ntaub ntawv loj, qhov teeb meem ntawm qhov tsis muaj qhov chaw disk tuaj yeem tshwm sim qee zaum. Ib txoj hauv kev los daws qhov teeb meem no yog compression, ua tsaug rau cov khoom siv tib yam, koj tuaj yeem them taus kom nce ntim ntim. Hauv tsab xov xwm no, peb yuav saib seb cov ntaub ntawv compression ua haujlwm li cas hauv Apache Ignite. Kab lus no yuav piav qhia tsuas yog cov txheej txheem compression disk siv hauv cov khoom. Lwm txoj hauv kev ntawm cov ntaub ntawv compression (dhau lub network, hauv nco), txawm tias siv los yog tsis, yuav nyob sab nraud.

Yog li, nrog rau hom kev mob siab rau, raws li kev hloov pauv hauv cov ntaub ntawv hauv lub caches, Ignite pib sau rau disk:

  1. Cov ntsiab lus ntawm caches
  2. Sau Npe Ua Ntej (tom qab no tsuas yog WAL)

Muaj ib lub tshuab rau WAL compression rau qee lub sijhawm tam sim no, hu ua WAL compaction. Qhov tsis ntev los no tau tso tawm Apache Ignite 2.8 qhia ob lub tswv yim ntxiv uas tso cai rau koj los nthuav cov ntaub ntawv ntawm disk: disk nplooj compression rau compressing cov ntsiab lus ntawm caches thiab WAL nplooj snapshot compression rau compressing qee qhov WAL nkag. Paub meej ntxiv txog tag nrho peb ntawm cov txheej txheem hauv qab no.

Disk nplooj compression

Yuav ua li cas li no ua hauj lwm

Ua ntej, cia peb saib luv luv li cas Ignite khaws cov ntaub ntawv. Nplooj ntawv nco yog siv los khaws cia. Nplooj ntawv loj yog teem rau ntawm qhov pib ntawm node thiab tsis tuaj yeem hloov pauv tom qab theem; kuj, nplooj ntawv loj yuav tsum yog lub zog ntawm ob thiab ntau yam ntawm cov ntaub ntawv kaw qhov loj. Cov nplooj ntawv raug thauj mus rau hauv RAM los ntawm disk raws li xav tau; qhov loj ntawm cov ntaub ntawv ntawm disk tuaj yeem tshaj li qhov faib RAM. Yog tias tsis muaj chaw txaus hauv RAM kom thauj cov nplooj ntawv los ntawm disk, qub, cov nplooj ntawv tsis siv lawm yuav raug tshem tawm ntawm RAM.

Cov ntaub ntawv khaws cia rau hauv disk hauv daim ntawv hauv qab no: cov ntaub ntawv sib cais yog tsim rau txhua qhov kev faib tawm ntawm txhua pawg cache; hauv cov ntaub ntawv no, nplooj ntawv tshwm ib tom qab ntawm qhov kev txiav txim nce mus. Cov nplooj ntawv tag nrho muaj cov cache pab pawg neeg txheeb xyuas, muab faib tus lej, thiab nplooj ntawv ntsuas hauv cov ntaub ntawv. Yog li, siv cov nplooj ntawv tag nrho, peb tuaj yeem txiav txim siab cov ntaub ntawv thiab qhov offset hauv cov ntaub ntawv rau txhua nplooj ntawv. Koj tuaj yeem nyeem ntxiv txog kev nco qab hauv Apache Ignite Wiki tsab xov xwm: Ignite Persistent Store - hauv qab hood.

Lub disk nplooj compression mechanism, raws li koj tuaj yeem twv los ntawm lub npe, ua haujlwm ntawm nplooj ntawv qib. Thaum cov txheej txheem no tau qhib, cov ntaub ntawv hauv RAM tau ua tiav raws li yog, yam tsis muaj compression, tab sis thaum nplooj ntawv tau txais kev cawmdim los ntawm RAM mus rau disk, lawv raug compressed.

Tab sis compressing txhua nplooj ntawv ib tus zuj zus tsis yog ib qho kev daws teeb meem rau qhov teeb meem; koj yuav tsum tau qee qhov txo qhov loj ntawm cov ntaub ntawv cov ntaub ntawv. Yog tias nplooj ntawv loj tsis tas lawm, peb tsis tuaj yeem sau cov nplooj ntawv rau cov ntaub ntawv ib zaug ntxiv, vim qhov no tuaj yeem tsim ntau yam teeb meem:

  • Siv cov nplooj ntawv Performance index, peb yuav tsis tuaj yeem xam qhov offset uas nws nyob hauv cov ntaub ntawv.
  • Nws tsis paub meej tias yuav ua li cas nrog cov nplooj ntawv uas tsis yog qhov kawg ntawm cov ntaub ntawv thiab hloov lawv qhov loj. Yog tias nplooj ntawv loj zuj zus, qhov chaw nws tso tawm ploj mus. Yog tias nplooj ntawv loj tuaj, koj yuav tsum nrhiav qhov chaw tshiab hauv cov ntaub ntawv rau nws.
  • Yog tias nplooj ntawv txav los ntawm ntau tus bytes uas tsis yog ntau yam ntawm cov ntaub ntawv kaw qhov loj, ces nyeem lossis sau nws yuav tsum tau kov ib qho ntxiv cov ntaub ntawv thaiv, uas tuaj yeem ua rau kev ua haujlwm tsis zoo.

Txhawm rau kom tsis txhob daws cov teeb meem no ntawm nws tus kheej theem, disk nplooj compression hauv Apache Ignite siv cov txheej txheem txheej txheem hu ua sparse files. Ib cov ntaub ntawv sparse yog ib qho uas qee thaj tsam xoom puv tuaj yeem raug cim ua "qhov". Hauv qhov no, tsis muaj cov ntaub ntawv kaw lus yuav raug faib los khaws cov qhov no, ua rau kev txuag ntawm qhov chaw disk.

Nws yog qhov laj thawj uas txhawm rau txhawm rau pub dawb cov ntaub ntawv kaw lus, qhov loj ntawm lub qhov yuav tsum siab dua lossis sib npaug ntawm cov ntaub ntawv kaw lus, uas ua rau muaj kev txwv ntxiv ntawm nplooj ntawv loj thiab Apache Ignite: rau compression kom muaj txiaj ntsig, nplooj ntawv loj yuav tsum nruj me ntsis loj dua qhov loj ntawm cov ntaub ntawv kaw lus. Yog tias nplooj ntawv loj sib npaug ntawm qhov loj me, ces peb yuav tsis muaj peev xwm pub dawb ib qho thaiv, txij li txhawm rau txhawm rau pub dawb ib qho thaiv, nplooj ntawv compressed yuav tsum nyob 0 bytes. Yog tias nplooj ntawv loj yog sib npaug ntawm qhov loj ntawm 2 lossis 4 blocks, peb twb tuaj yeem tso tawm tsawg kawg ib ntu yog tias peb nplooj ntawv raug compressed tsawg kawg yog 50% lossis 75%, feem.

Yog li, qhov kawg piav qhia txog yuav ua li cas cov txheej txheem ua haujlwm: Thaum sau ib nplooj ntawv rau disk, sim ua kom nrawm nplooj ntawv. Yog hais tias qhov loj ntawm nplooj ntawv compressed tso cai rau ib los yog ntau tshaj cov ntaub ntawv kaw lus kom tso tawm, ces nplooj ntawv yog sau nyob rau hauv compressed daim ntawv, thiab ib tug "lub qhov" yog ua nyob rau hauv qhov chaw ntawm lub freed blocks (ib tug system hu yog tua. fallocate() nrog lub punch qhov chij). Yog hais tias qhov loj ntawm nplooj ntawv compressed tsis tso cai rau cov blocks tau tso tawm, nplooj ntawv tau txais kev cawmdim raws li yog, uncompressed. Tag nrho cov nplooj ntawv offsets raug xam tib yam li tsis muaj compression, los ntawm kev muab cov nplooj ntawv txheeb xyuas los ntawm nplooj ntawv loj. Tsis muaj kev hloov chaw ntawm nplooj ntawv yog xav tau ntawm koj tus kheej. Nplooj ntawv offsets, ib yam li tsis muaj compression, poob rau ntawm thaj tsam ntawm cov ntaub ntawv kaw lus.

Cov ntaub ntawv compression hauv Apache Ignite. Sber kev paub

Hauv kev siv tam sim no, Ignite tsuas tuaj yeem ua haujlwm nrog cov ntaub ntawv tsis tshua muaj nyob hauv Linux OS; raws li, disk nplooj compression tsuas tuaj yeem qhib thaum siv Ignite ntawm qhov kev ua haujlwm no.

Compression algorithms uas tuaj yeem siv rau disk nplooj compression: ZSTD, LZ4, Snappy. Tsis tas li ntawd, muaj ib qho kev khiav hauj lwm hom (SKIP_GARBAGE), uas tsuas yog qhov chaw tsis siv nyob rau hauv nplooj ntawv raug muab pov tseg yam tsis siv compression ntawm cov ntaub ntawv seem, uas txo cov load ntawm CPU piv rau yav dhau los algorithms.

Kev cuam tshuam kev ua haujlwm

Hmoov tsis zoo, kuv tsis tau ua qhov kev ntsuas kev ua tau zoo ntawm qhov chaw tiag tiag, vim tias peb tsis npaj yuav siv cov txheej txheem no hauv kev tsim khoom, tab sis peb tuaj yeem xav tias peb yuav poob qhov twg thiab qhov twg peb yuav yeej.

Txhawm rau ua qhov no, peb yuav tsum nco ntsoov tias nplooj ntawv nyeem thiab sau li cas thaum nkag mus:

  • Thaum ua haujlwm nyeem ntawv, nws yog thawj zaug tshawb hauv RAM; yog tias qhov kev tshawb fawb tsis ua tiav, nplooj ntawv raug thauj khoom rau hauv RAM ntawm disk los ntawm tib cov xov uas ua tiav cov ntawv nyeem.
  • Thaum ua haujlwm sau ua tiav, nplooj ntawv hauv RAM raug cim tias qias neeg, tab sis nplooj ntawv tsis tau khaws cia rau hauv disk tam sim ntawd los ntawm xov ua qhov sau. Tag nrho cov nplooj ntawv qias neeg raug khaws cia rau disk tom qab hauv cov txheej txheem kuaj xyuas hauv cov xov sib cais.

Yog li qhov cuam tshuam rau kev nyeem ntawv yog:

  • Zoo (disk IO), vim qhov txo qis ntawm cov ntawv nyeem cov ntaub ntawv kaw.
  • Tsis zoo (CPU), vim yog qhov ntxiv load yuav tsum tau los ntawm lub operating system ua hauj lwm nrog sparse ntaub ntawv. Nws tseem muaj peev xwm hais tias ntxiv IO kev khiav hauj lwm yuav implicitly tshwm sim ntawm no kom txuag tau ib tug complex sparse cov ntaub ntawv qauv (hmoov, kuv tsis paub tag nrho cov ntsiab lus ntawm yuav ua li cas sparse ntaub ntawv ua hauj lwm).
  • Negative (CPU), vim qhov xav tau los decompress nplooj ntawv.
  • Tsis muaj kev cuam tshuam rau kev sau ntawv.
  • Kev cuam tshuam rau cov txheej txheem kuaj xyuas (txhua yam ntawm no zoo ib yam li nyeem cov haujlwm):
  • Qhov zoo (disk IO), vim qhov txo qis ntawm cov ntaub ntawv kaw lus kaw.
  • Tsis zoo (CPU, tejzaum nws yog disk IO), vim ua haujlwm nrog cov ntaub ntawv tsis tshua muaj.
  • Tsis zoo (CPU), vim xav tau nplooj ntawv compression.

Qhov twg ntawm qhov ntsuas yuav taw qhia qhov ntsuas? Qhov no tag nrho yog nyob ntawm ib puag ncig, tab sis kuv xav ntseeg tias disk nplooj compression feem ntau yuav ua rau kev ua haujlwm tsis zoo ntawm ntau lub tshuab. Ntxiv mus, kev ntsuam xyuas ntawm lwm yam DBMSs uas siv txoj hauv kev zoo sib xws nrog cov ntaub ntawv me me qhia qhov kev poob qis hauv kev ua haujlwm thaum compression qhib.

Yuav ua li cas qhib thiab teeb tsa

Raws li tau hais los saum no, qhov tsawg kawg nkaus version ntawm Apache Ignite uas txhawb nqa disk nplooj compression yog 2.8 thiab tsuas yog Linux kev ua haujlwm tau txais kev txhawb nqa. Qhib thiab teeb tsa raws li hauv qab no:

  • Yuav tsum muaj qhov ignite-compression module hauv chav kawm-txoj kev. Los ntawm lub neej ntawd, nws nyob hauv Apache Ignite faib hauv libs/optional directory thiab tsis suav nrog hauv chav kawm-txoj kev. Koj tuaj yeem yooj yim txav cov ntawv qhia txog ib qib mus rau libs thiab tom qab ntawd thaum koj khiav nws los ntawm ignite.sh nws yuav tau txais kev tso cai.
  • Kev ua siab ntev yuav tsum tau qhib (Enabled ntawm DataRegionConfiguration.setPersistenceEnabled(true)).
  • Nplooj ntawv loj yuav tsum loj dua cov ntaub ntawv kaw qhov loj (koj tuaj yeem teeb tsa siv DataStorageConfiguration.setPageSize() ).
  • Rau txhua cache uas nws cov ntaub ntawv yuav tsum tau compressed, koj yuav tsum configure lub compression txoj kev thiab (optional) lub compression theem (txoj kev. CacheConfiguration.setDiskPageCompression() , CacheConfiguration.setDiskPageCompressionLevel()).

WAL compaction

Yuav ua li cas li no ua hauj lwm

WAL yog dab tsi thiab vim li cas nws thiaj xav tau? Luv luv heev: qhov no yog lub cav uas muaj tag nrho cov xwm txheej uas thaum kawg hloov nplooj ntawv cia. Nws yog qhov yuav tsum tau ua kom muaj peev xwm rov qab tau thaum lub caij nplooj zeeg. Txhua qhov kev ua haujlwm, ua ntej muab kev tswj hwm rau tus neeg siv, yuav tsum xub sau ib qho xwm txheej hauv WAL, yog li ntawd, thaum tsis ua haujlwm, nws tuaj yeem rov ua haujlwm hauv lub cav thiab rov qab ua txhua yam haujlwm uas tus neeg siv tau txais cov lus teb zoo, txawm tias cov haujlwm no. tsis muaj sijhawm los xav txog hauv nplooj ntawv khaws cia ntawm disk (tam sim no tau piav qhia tias qhov kev sau ntawv tiag tiag rau nplooj ntawv khw yog ua tiav hauv cov txheej txheem hu ua "checkpointing" nrog qee qhov qeeb los ntawm cov xov sib cais).

Kev nkag hauv WAL tau muab faib ua kev xav thiab lub cev. Cov Boolean yog cov yuam sij thiab qhov tseem ceeb ntawm lawv tus kheej. Lub cev - qhia txog kev hloov pauv rau nplooj ntawv hauv nplooj ntawv khw. Thaum cov ntaub ntawv pov thawj tuaj yeem pab tau rau qee qhov xwm txheej, cov ntaub ntawv lub cev tsuas yog xav tau rau kev rov qab los thaum muaj kev sib tsoo thiab cov ntaub ntawv xav tau tsuas yog txij li qhov chaw kuaj xyuas ua tiav kawg. Ntawm no peb yuav tsis nkag mus rau hauv kev nthuav dav thiab piav qhia vim li cas nws ua haujlwm li no, tab sis cov neeg nyiam tuaj yeem xa mus rau cov lus hais dhau los ntawm Apache Ignite Wiki: Ignite Persistent Store - hauv qab hood.

Muaj ntau ntau cov ntaub ntawv lub cev rau ib cov ntaub ntawv pov thawj. Ntawd yog, piv txwv li, ib qho kev ua haujlwm rau hauv cache cuam tshuam rau ntau nplooj ntawv hauv nplooj ntawv nco (ib nplooj ntawv nrog cov ntaub ntawv nws tus kheej, nplooj ntawv nrog indexes, nplooj ntawv nrog cov npe dawb). Hauv qee qhov kev ntsuam xyuas hluavtaws, kuv pom tias cov ntaub ntawv lub cev muaj txog li 90% ntawm WAL cov ntaub ntawv. Txawm li cas los xij, lawv xav tau rau lub sijhawm luv luv (los ntawm lub neej ntawd, lub sijhawm nruab nrab ntawm cov chaw kuaj xyuas yog 3 feeb). Nws yuav yog qhov laj thawj kom tshem tawm cov ntaub ntawv no tom qab poob nws qhov tseeb. Qhov no yog raws nraim li WAL compaction mechanism ua: nws tau tshem tawm ntawm lub cev cov ntaub ntawv thiab compresses cov ntaub ntawv tseem ceeb uas siv zip, thaum cov ntaub ntawv loj txo qis heev (qee zaum los ntawm kaum tawm zaus).

Lub cev, WAL muaj ntau ntu (10 los ntawm lub neej ntawd) ntawm qhov loj me (64MB los ntawm lub neej ntawd), uas tau sau dhau los ua ib puag ncig. Sai li qhov ntu tam sim no tau sau, ntu txuas ntxiv yog muab raws li tam sim no, thiab ntu sau tau raug theej rau hauv archive los ntawm cov xov sib cais. WAL compaction twb ua haujlwm nrog ntu archive. Tsis tas li ntawd, raws li ib txoj xov sib cais, nws saib xyuas qhov ua tiav ntawm qhov chaw kuaj xyuas thiab pib compression hauv cov ntu ntu uas cov ntaub ntawv lub cev tsis xav tau ntxiv lawm.

Cov ntaub ntawv compression hauv Apache Ignite. Sber kev paub

Kev cuam tshuam kev ua haujlwm

Txij li thaum WAL compaction khiav raws li cov xov sib cais, yuav tsum tsis muaj kev cuam tshuam ncaj qha rau kev ua haujlwm. Tab sis nws tseem tso cov ntaub ntawv keeb kwm yav dhau los ntxiv rau ntawm CPU (compression) thiab disk (nyeem txhua ntu WAL los ntawm cov ntaub ntawv khaws cia thiab sau cov ntu compressed), yog li yog tias lub kaw lus khiav ntawm nws lub peev xwm siab tshaj plaws, nws tseem yuav ua rau kev ua haujlwm tsis zoo.

Yuav ua li cas qhib thiab teeb tsa

Koj tuaj yeem ua kom WAL compaction siv cov cuab yeej WalCompactionEnabled Π² DataStorageConfiguration (DataStorageConfiguration.setWalCompactionEnabled(true)). Tsis tas li ntawd, siv DataStorageConfiguration.setWalCompactionLevel() txoj kev, koj tuaj yeem teeb tsa qib compression yog tias koj tsis txaus siab rau tus nqi qub (BEST_SPEED).

WAL nplooj ntawv snapshot compression

Yuav ua li cas li no ua hauj lwm

Peb twb pom lawm hais tias nyob rau hauv WAL cov ntaub ntawv tau muab faib ua logical thiab lub cev. Rau txhua qhov kev hloov pauv rau txhua nplooj ntawv, cov ntaub ntawv WAL lub cev raug tsim tawm hauv nplooj ntawv nco. Cov ntaub ntawv cev nqaij daim tawv, nyob rau hauv lem, kuj muab faib ua 2 subtypes: nplooj ntawv snapshot cov ntaub ntawv thiab delta cov ntaub ntawv. Txhua zaus peb hloov qee yam ntawm nplooj ntawv thiab hloov nws los ntawm lub xeev huv mus rau lub xeev qias neeg, ib daim ntawv luam ntawm nplooj ntawv no tau muab khaws cia rau hauv WAL (nplooj snapshot record). Txawm hais tias peb hloov tsuas yog ib byte hauv WAL, cov ntaub ntawv yuav loj dua me ntsis ntawm nplooj ntawv. Yog tias peb hloov qee yam ntawm nplooj ntawv qias neeg lawm, cov ntaub ntawv delta tau tsim hauv WAL, uas qhia txog kev hloov pauv tsuas yog piv rau lub xeev dhau los ntawm nplooj ntawv, tab sis tsis yog tag nrho nplooj ntawv. Txij li thaum rov pib dua lub xeev cov nplooj ntawv los ntawm qias neeg kom huv yog ua thaum lub sijhawm kuaj xyuas, tam sim tom qab pib qhov chaw kuaj xyuas, yuav luag tag nrho cov ntaub ntawv ntawm lub cev yuav tsuas muaj cov duab thaij duab ntawm nplooj ntawv (vim txhua nplooj ntawv tam sim tom qab pib qhov chaw kuaj xyuas huv) , tom qab ntawd thaum peb mus txog qhov chaw kuaj xyuas tom ntej, cov ntaub ntawv delta feem pib loj hlob thiab rov pib dua thaum pib ntawm qhov chaw kuaj tom ntej. Kev ntsuas hauv qee qhov kev ntsuam xyuas hluavtaws pom tau tias qhov sib faib ntawm nplooj ntawv snapshots hauv tag nrho cov ntaub ntawv lub cev mus txog 90%.

Lub tswv yim ntawm WAL nplooj ntawv snapshot compression yog txhawm rau txhawm rau nplooj ntawv snapshots siv cov cuab yeej npaj ua nplooj ntawv compression (saib disk nplooj compression ). Nyob rau tib lub sijhawm, hauv WAL, cov ntaub ntawv tau txais kev cawmdim nyob rau hauv append-tsuas yog hom thiab tsis tas yuav khi cov ntaub ntawv rau thaj tsam ntawm cov ntaub ntawv kaw lus, yog li ntawm no, tsis zoo li disk nplooj compression mechanism, peb tsis xav tau cov ntaub ntawv sib cais ntawm tag nrho; raws li, qhov txheej txheem no yuav ua haujlwm tsis yog ntawm OS Linux nkaus xwb. Tsis tas li ntawd xwb, nws tsis muaj teeb meem rau peb npaum li cas peb tuaj yeem nthuav tawm nplooj ntawv. Txawm hais tias peb tso tawm 1 byte, qhov no yog qhov txiaj ntsig zoo thiab peb tuaj yeem txuag cov ntaub ntawv compressed hauv WAL, tsis zoo li disk nplooj compression, qhov twg peb khaws cov nplooj compressed tsuas yog tias peb tso ntau tshaj 1 cov ntaub ntawv kaw lus.

Nplooj ntawv yog cov ntaub ntawv compressible heev, lawv qhov sib koom hauv tag nrho WAL ntim yog siab heev, yog li tsis hloov WAL cov ntaub ntawv peb tuaj yeem tau txais qhov txo qis hauv nws qhov loj. Compression, suav nrog cov ntaub ntawv pov thawj, yuav xav tau kev hloov pauv ntawm hom ntawv thiab tsis muaj kev sib raug zoo, piv txwv li, rau cov neeg siv khoom sab nraud uas yuav nyiam cov ntaub ntawv pov thawj, tab sis yuav tsis ua rau txo qis ntawm cov ntaub ntawv loj.

Raws li nrog disk nplooj compression, WAL nplooj snapshot compression tuaj yeem siv ZSTD, LZ4, Snappy compression algorithms, nrog rau SKIP_GARBAGE hom.

Kev cuam tshuam kev ua haujlwm

Nws tsis yooj yim rau kev pom tias ncaj qha ua rau WAL nplooj ntawv snapshot compression tsuas yog cuam tshuam cov xov uas sau cov ntaub ntawv rau nplooj ntawv nco, uas yog, cov xov uas hloov cov ntaub ntawv hauv caches. Kev nyeem cov ntaub ntawv lub cev los ntawm WAL tsuas yog tshwm sim ib zaug xwb, thaum lub sijhawm node tau tsa tom qab lub caij nplooj zeeg (thiab tsuas yog nws ntog thaum lub sijhawm kuaj xyuas).

Qhov no cuam tshuam cov xov uas hloov cov ntaub ntawv raws li hauv qab no: peb tau txais cov txiaj ntsig tsis zoo (CPU) vim qhov xav tau los nthuav cov nplooj ntawv txhua zaus ua ntej sau ntawv rau disk, thiab muaj txiaj ntsig zoo (disk IO) vim qhov txo qis ntawm cov ntaub ntawv. cov ntaub ntawv sau. Raws li, txhua yam yog qhov yooj yim ntawm no: yog tias qhov kev ua haujlwm raug txwv los ntawm CPU, peb tau txais kev degradation me ntsis, yog tias nws txwv los ntawm disk I / O, peb tau txais kev nce ntxiv.

Kev tsis ncaj, txo qhov loj ntawm WAL tseem cuam tshuam (zoo) cov kwj deg uas pov tseg WAL ntu rau hauv cov ntaub ntawv khaws cia thiab WAL compaction kwj.

Kev ntsuas kev ua tau zoo tiag tiag hauv peb ib puag ncig siv cov ntaub ntawv hluavtaws pom tau tias nce me ntsis (los ntawm kev nce ntxiv los ntawm 10% -15%, latency txo los ntawm 10% -15%).

Yuav ua li cas qhib thiab teeb tsa

Yam tsawg kawg Apache Ignite version: 2.8. Qhib thiab teeb tsa raws li hauv qab no:

  • Yuav tsum muaj qhov ignite-compression module hauv chav kawm-txoj kev. Los ntawm lub neej ntawd, nws nyob hauv Apache Ignite faib hauv libs/optional directory thiab tsis suav nrog hauv chav kawm-txoj kev. Koj tuaj yeem yooj yim txav cov ntawv qhia txog ib qib mus rau libs thiab tom qab ntawd thaum koj khiav nws los ntawm ignite.sh nws yuav tau txais kev tso cai.
  • Kev ua siab ntev yuav tsum tau qhib (Enabled ntawm DataRegionConfiguration.setPersistenceEnabled(true)).
  • Lub compression hom yuav tsum tau teem siv txoj kev DataStorageConfiguration.setWalPageCompression(), compression yog neeg xiam los ntawm lub neej ntawd (DISABLED hom).
  • Optionally, koj tuaj yeem teeb tsa qib compression siv txoj kev DataStorageConfiguration.setWalPageCompression(), saib javadoc rau cov txheej txheem rau cov nqi siv tau rau txhua hom.

xaus

Cov kev xav txog cov ntaub ntawv compression mechanisms hauv Apache Ignite tuaj yeem siv nws tus kheej ntawm ib leeg, tab sis ib qho kev sib xyaw ua ke ntawm lawv kuj tau txais. Kev nkag siab tias lawv ua haujlwm li cas yuav tso cai rau koj los txiav txim siab seb lawv tsim nyog rau koj cov dej num hauv koj ib puag ncig thiab koj yuav tau txi dab tsi thaum siv lawv. Disk nplooj compression yog tsim los rau compress lub ntsiab cia thiab tuaj yeem muab qhov nruab nrab compression piv. WAL nplooj ntawv snapshot compression yuav muab qhov nruab nrab ntawm compression rau WAL cov ntaub ntawv, thiab feem ntau yuav txawm tias txhim kho kev ua haujlwm. WAL compaction yuav tsis muaj txiaj ntsig zoo rau kev ua haujlwm, tab sis yuav txo qhov loj ntawm WAL cov ntaub ntawv ntau li ntau tau los ntawm kev tshem tawm cov ntaub ntawv lub cev.

Tau qhov twg los: www.hab.com

Ntxiv ib saib