Kev npau taws, kev sib cav thiab kev nyuaj siab thaum ua haujlwm nrog InfluxDB

Yog tias koj siv lub sijhawm series database (timeseries db, wiki) raws li lub ntsiab cia rau ib qhov chaw nrog kev txheeb cais, ces es tsis txhob daws qhov teeb meem koj tuaj yeem tau txais kev mob taub hau ntau. Kuv tab tom ua haujlwm ntawm qhov project uas siv cov ntaub ntawv zoo li no, thiab qee zaum InfluxDB, uas yuav tau tham txog, nthuav tawm qhov xav tsis thoob kiag li.

Disclaimer: Cov teeb meem teev tseg siv rau InfluxDB version 1.7.4.

Vim li cas lub sij hawm series?

Qhov project yog los taug qab kev lag luam ntawm ntau yam blockchains thiab tso saib cov txheeb cais. Tshwj xeeb, peb saib cov emissions thiab hlawv cov nyiaj npib ruaj khov (wiki). Raws li cov kev hloov pauv no, koj yuav tsum tsim cov duab kos thiab qhia cov lus xaus.

Thaum soj ntsuam kev lag luam, ib lub tswv yim tuaj txog: siv InfluxDB lub sij hawm series database ua lub ntsiab cia. Kev ua lag luam yog cov ntsiab lus nyob rau hauv lub sij hawm thiab lawv haum zoo rau lub sij hawm series qauv.

Kev sib sau ua ke kuj zoo li yooj yim heev - zoo tagnrho rau kev ua cov kab kos nrog lub sijhawm ntev. Tus neeg siv xav tau daim ntawv qhia rau ib xyoos, thiab cov ntaub ntawv muaj cov ntaub ntawv teev nrog lub sijhawm ntawm tsib feeb. Nws tsis muaj txiaj ntsig xa nws tag nrho ib puas txhiab dots - sib nrug los ntawm kev ua haujlwm ntev, lawv yuav tsis haum rau ntawm qhov screen. Koj tuaj yeem sau koj tus kheej qhov kev siv ntawm kev nce lub sijhawm, lossis siv cov haujlwm sib sau ua ke hauv Influx. Nrog lawv cov kev pab, koj tuaj yeem pab pawg cov ntaub ntawv los ntawm hnub thiab xa cov ntsiab lus xav tau 365.

Nws yog qhov tsis meej pem me ntsis tias cov ntaub ntawv no feem ntau yog siv rau lub hom phiaj ntawm kev sau cov ntsuas. Kev saib xyuas cov servers, iot li, txhua yam los ntawm ntau lab ntawm cov ntsiab lus ntawm daim ntawv "ntws": [<time> - <metric value>]. Tab sis yog tias cov ntaub ntawv ua haujlwm zoo nrog cov ntaub ntawv loj, ces vim li cas thiaj li yuav tsum muaj qhov ntim me me ua teeb meem? Nrog rau qhov no hauv siab, peb coj InfluxDB los ua haujlwm.

Dab tsi ntxiv yog yooj yim hauv InfluxDB

Sib nrug los ntawm cov haujlwm sib sau tau hais, muaj ib qho zoo dua - cov lus nug tsis tu ncua (doc). Qhov no yog lub sijhawm teem rau hauv cov ntaub ntawv uas tuaj yeem ua cov ntaub ntawv ntawm lub sijhawm. Piv txwv li, txhua txhua 24 teev koj tuaj yeem pab pawg tag nrho cov ntaub ntawv rau ib hnub, suav qhov nruab nrab thiab sau ib qho ntsiab lus tshiab hauv lwm lub rooj yam tsis tau sau koj lub tsheb kauj vab.

Kuj tseem muaj tuav txoj cai (doc) - teeb tsa cov ntaub ntawv tshem tawm tom qab qee lub sijhawm. Nws yog qhov muaj txiaj ntsig zoo thaum, piv txwv li, koj yuav tsum khaws CPU load rau ib lub lim tiam nrog kev ntsuas ib zaug ib ob, tab sis dhau ntawm qhov deb ntawm ob peb lub hlis xws li qhov tseeb tsis xav tau. Hauv qhov xwm txheej zoo li no, koj tuaj yeem ua qhov no:

1. tsim cov lus nug txuas ntxiv kom sau cov ntaub ntawv mus rau lwm lub rooj;
2. rau thawj lub rooj, txhais ib txoj cai rau kev tshem tawm cov metrics uas laus dua li tib lub lim tiam.

Thiab Influx yuav nws tus kheej txo qhov loj ntawm cov ntaub ntawv thiab tshem tawm yam tsis tsim nyog.

Hais txog cov ntaub ntawv khaws cia

Tsis muaj ntau cov ntaub ntawv khaws cia: txog 70 txhiab kev lag luam thiab lwm lab cov ntsiab lus nrog cov ntaub ntawv lag luam. Ntxiv cov kev nkag tshiab - tsis pub ntau tshaj 3000 cov ntsiab lus hauv ib hnub. Kuj tseem muaj kev ntsuas rau lub xaib, tab sis muaj cov ntaub ntawv me me nyob ntawd thiab, raws li txoj cai tswj hwm, lawv tau khaws cia tsis pub dhau ib hlis.

Teeb meem

Thaum lub sijhawm txhim kho thiab kev sim tom qab ntawm kev pabcuam, ntau thiab ntau qhov teeb meem tseem ceeb tshwm sim hauv kev ua haujlwm ntawm InfluxDB.

1. Tshem tawm cov ntaub ntawv

Muaj ntau cov ntaub ntawv nrog kev hloov pauv:

SELECT time, amount, block, symbol FROM transactions WHERE symbol='USDT'

Tshwm sim:

Kuv tab tom xa cov lus txib kom rho tawm cov ntaub ntawv:

DELETE FROM transactions WHERE symbol=’USDT’

Tom ntej no kuv thov kom tau txais cov ntaub ntawv deleted lawm. Thiab es tsis txhob teb qhov khoob, Influx rov qab ib feem ntawm cov ntaub ntawv uas yuav tsum tau muab tshem tawm.

Kuv tabtom sim rho tawm tag nrho cov lus:

DROP MEASUREMENT transactions

Kuv xyuas lub rooj deletion:

SHOW MEASUREMENTS

Kuv tsis pom cov lus hauv daim ntawv teev npe, tab sis cov lus nug cov ntaub ntawv tshiab tseem rov qab rau tib txheej kev lag luam.

Qhov teeb meem tsuas yog tshwm sim rau kuv ib zaug xwb, txij li cov ntaub ntawv tshem tawm yog ib rooj plaub cais. Tab sis qhov kev coj cwj pwm no ntawm cov ntaub ntawv qhia meej tsis haum rau hauv lub moj khaum ntawm kev ua haujlwm "yog". Tom qab ntawd kuv pom nws qhib ntawm github daim pib yuav luag ib xyoos dhau los ntawm lub ncauj lus no.

Yog li ntawd, rho tawm thiab rov qab kho tag nrho cov ntaub ntawv tau pab.

2. Floating point tooj

Kev suav lej thaum siv cov haujlwm ua haujlwm hauv InfluxDB muaj qhov raug yuam kev. Tsis yog hais tias qhov no yog dab tsi txawv txawv, tab sis nws tsis kaj siab.

Hauv kuv qhov xwm txheej, cov ntaub ntawv muaj kev tiv thaiv nyiaj txiag thiab kuv xav ua kom tiav nrog qhov raug siab. Vim li no, peb npaj yuav tso tseg cov lus nug tsis tu ncua.

3. Cov lus nug tsis tu ncua tsis tuaj yeem hloov mus rau cov sijhawm sib txawv

Cov kev pabcuam muaj ib lub rooj nrog kev txheeb xyuas txhua hnub. Rau txhua hnub, koj yuav tsum tau pab pawg tag nrho cov kev hloov pauv rau hnub ntawd. Tab sis txhua tus neeg siv hnub yuav pib ntawm lub sijhawm sib txawv, thiab yog li cov txheej txheem kev lag luam yuav txawv. Los ntawm UTC yog 37 variants hloov uas koj yuav tsum tau sau cov ntaub ntawv.

Nyob rau hauv InfluxDB, thaum pab pawg los ntawm lub sijhawm, koj tuaj yeem hais ntxiv txog kev hloov pauv, piv txwv li Moscow lub sijhawm (UTC + 3):

SELECT MEAN("supply") FROM transactions GROUP BY symbol, time(1d, 3h) fill(previous)

Tab sis cov lus nug tshwm sim yuav tsis raug. Rau qee qhov laj thawj, cov ntaub ntawv sib koom ua ke los ntawm hnub yuav pib txhua txoj kev rov qab mus rau 1677 (InfluxDB txhawb nqa lub sijhawm ncua ntawm xyoo no):

Txhawm rau ua haujlwm ntawm qhov teeb meem no, peb hloov qhov kev pabcuam mus rau UTC + 0 ib ntus.

4. Kev ua tau zoo

Muaj ntau cov qauv ntsuas hauv Is Taws Nem uas sib piv InfluxDB thiab lwm cov ntaub ntawv. Thaum xub thawj siab ib muag, lawv zoo li cov khoom lag luam, tab sis tam sim no kuv xav tias muaj qee qhov tseeb hauv lawv.

Kuv mam qhia koj li cas.

Cov kev pabcuam muab cov txheej txheem API uas xa rov qab cov txheeb cais rau hnub kawg. Thaum ua kev suav, txoj kev nug cov ntaub ntawv peb zaug nrog cov lus nug hauv qab no:

SELECT * FROM coins_info WHERE time <= NOW() GROUP BY symbol ORDER BY time DESC LIMIT 1

SELECT * FROM dominance_info ORDER BY time DESC LIMIT 1

SELECT * FROM transactions WHERE time >= NOW() - 24h ORDER BY time DESC

Kev piav qhia:

1. Hauv thawj qhov kev thov, peb tau txais cov ntsiab lus kawg rau txhua lub npib nrog cov ntaub ntawv ua lag luam. Yim lub ntsiab lus rau yim npib hauv kuv rooj plaub.
2. Qhov kev thov thib ob tau txais ib qho ntawm cov ntsiab lus tshiab tshaj plaws.
3. Qhov thib peb thov ib daim ntawv teev kev lag luam rau XNUMX teev dhau los; tej zaum yuav muaj ntau pua ntawm lawv.

Cia kuv qhia meej tias InfluxDB cia li tsim qhov ntsuas raws li cov cim npe thiab lub sijhawm, uas ua rau cov lus nug nrawm. Hauv thawj qhov kev thov cim yog tag.

Kuv tau khiav qhov kev ntsuam xyuas kev ntxhov siab ntawm txoj kev API no. Rau 25 RPS, tus neeg rau zaub mov tau ua kom pom tag nrho ntawm XNUMX CPUs:

Nyob rau tib lub sijhawm, cov txheej txheem NodeJs tsis muaj kev thauj khoom txhua.

Qhov kev ua tiav ceev twb tau degraded los ntawm 7-10 RPS: yog tias ib tus neeg siv tau txais cov lus teb hauv 200 ms, ces 10 cov neeg siv khoom yuav tsum tau tos ib pliag. 25 RPS yog qhov txwv ntawm kev ruaj ntseg raug kev txom nyem; 500 qhov yuam kev raug xa rov qab rau cov neeg siv khoom.

Nrog rau kev ua tau zoo li no nws tsis tuaj yeem siv Influx hauv peb qhov project. Ntxiv mus: nyob rau hauv ib qhov project uas yuav tsum tau soj ntsuam xyuas rau ntau tus neeg siv, cov teeb meem zoo sib xws yuav tshwm sim thiab cov metrics server yuav overloaded.

xaus

Qhov kev txiav txim siab tseem ceeb tshaj plaws los ntawm cov kev paub dhau los yog tias koj tsis tuaj yeem nqa cov thev naus laus zis tsis paub rau hauv qhov project yam tsis muaj kev tshuaj xyuas txaus. Kev tshuaj xyuas yooj yim ntawm cov teeb meem qhib ntawm github tuaj yeem muab cov ntaub ntawv kom tsis txhob xaiv InfluxDB ua cov ntaub ntawv tseem ceeb.

InfluxDB yuav tsum tau ua kom zoo rau cov dej num ntawm kuv qhov project, tab sis raws li kev xyaum tau pom, cov ntaub ntawv no tsis ua raws li qhov xav tau thiab muaj ntau yam kab mob.

Koj twb tuaj yeem pom version 2.0.0-beta hauv qhov project repository; peb tuaj yeem cia siab tias qhov thib ob version yuav muaj kev txhim kho tseem ceeb. Lub sijhawm no, kuv yuav mus kawm cov ntaub ntawv TimescaleDB.

Tau qhov twg los: www.hab.com