Nyob zoo sawv daws! Kuv lub npe yog Golov Nikolay. Yav dhau los, kuv tau ua haujlwm ntawm Avito thiab tswj hwm Cov Ntaub Ntawv Platform rau rau xyoo, uas yog, kuv tau ua haujlwm ntawm txhua lub databases: analytical (Vertica, ClickHouse), streaming thiab OLTP (Redis, Tarantool, VoltDB, MongoDB, PostgreSQL). Lub sijhawm no, kuv tau hais txog ntau cov ntaub ntawv khaws tseg - sib txawv thiab txawv txawv, thiab nrog cov xwm txheej tsis zoo ntawm lawv siv.
Tam sim no kuv tab tom ua haujlwm ntawm ManyChat. Nyob rau hauv qhov tseeb, qhov no yog ib tug startup - tshiab, ambitious thiab loj hlob sai. Thiab thaum kuv thawj zaug koom nrog lub tuam txhab, cov lus nug classic tau tshwm sim: "Dab tsi yuav tsum tau pib tam sim no los ntawm DBMS thiab kev lag luam database?"
Hauv tsab xov xwm no, raws li kuv daim ntawv qhia ntawm , Kuv yuav teb lo lus nug no. Ib qho video version ntawm tsab ntawv ceeb toom muaj nyob ntawm .
Feem ntau paub databases 2020
Nws yog 2020, Kuv tau saib ib ncig thiab pom peb hom databases.
Thawj hom - classic OLTP databases: PostgreSQL, SQL Server, Oracle, MySQL. Lawv tau sau ntev dhau los, tab sis tseem muaj feem cuam tshuam vim tias lawv paub zoo rau cov neeg tsim tawm.
Hom ob yog bases los ntawm "zero". Lawv tau sim txav deb ntawm cov qauv qub los ntawm kev tso tseg SQL, cov qauv ib txwm muaj thiab ACID, los ntawm kev ntxiv cov kev sib txuas ua ke thiab lwm yam zoo nkauj. Piv txwv li, qhov no yog Cassandra, MongoDB, Redis lossis Tarantool. Tag nrho cov kev daws teeb meem no xav muab lub lag luam ib yam dab tsi tshiab thiab nyob hauv lawv lub niche vim tias lawv tau ua kom yooj yim heev rau qee yam haujlwm. Kuv yuav qhia cov databases nrog lub kaus lub sij hawm NOSQL.
Cov "zeros" dhau lawm, peb tau siv rau NOSQL databases, thiab lub ntiaj teb, los ntawm kuv qhov kev xav, coj cov kauj ruam tom ntej - mus rau tswj databases. Cov databases no muaj tib lub ntsiab lus xws li classic OLTP databases lossis NoSQL tshiab. Tab sis lawv tsis xav tau DBA thiab DevOps thiab khiav ntawm kev tswj hwm kho vajtse hauv huab. Rau tus tsim tawm, qhov no yog "tsuas yog lub hauv paus" uas ua haujlwm rau qhov chaw, tab sis tsis muaj leej twg saib xyuas nws li cas rau nruab rau ntawm lub server, leej twg teeb tsa lub server thiab leej twg hloov kho nws.
Piv txwv ntawm xws li databases:
- AWS RDS is a managed wrapper for PostgreSQL/MySQL.
- DynamoDB yog AWS analogue ntawm cov ntaub ntawv raws li database, zoo ib yam li Redis thiab MongoDB.
- Amazon Redshift yog ib qho kev tswj xyuas cov ntaub ntawv pov thawj.
Cov no yog cov qub databases, tab sis tsa nyob rau hauv ib puag ncig tswj, tsis tas yuav ua hauj lwm nrog kho vajtse.
Nco tseg. Cov piv txwv raug coj mus rau AWS ib puag ncig, tab sis lawv cov analogues kuj muaj nyob hauv Microsoft Azure, Google Cloud, lossis Yandex.Cloud.
Dab tsi tshiab txog qhov no? Hauv 2020, tsis muaj qhov no.
Serverless tswvyim
Dab tsi tshiab ntawm kev ua lag luam xyoo 2020 yog serverless lossis serverless kev daws teeb meem.
Kuv yuav sim piav qhia tias qhov no txhais li cas yog siv cov piv txwv ntawm cov kev pabcuam tsis tu ncua lossis daim ntawv thov rov qab.
Txhawm rau xa daim ntawv thov rov qab tsis tu ncua, peb yuav lossis xauj ib lub server, luam cov lej rau nws, tshaj tawm qhov kawg sab nraud thiab tsis tu ncua them nqi xauj tsev, hluav taws xob thiab cov chaw pabcuam cov ntaub ntawv. Qhov no yog tus qauv txheej txheem.
Puas muaj lwm txoj kev? Nrog kev pabcuam serverless koj tuaj yeem ua tau.
Dab tsi yog qhov tseem ceeb ntawm txoj hauv kev no: tsis muaj neeg rau zaub mov, tsis muaj txawm tias xauj ib qho virtual hauv huab. Txhawm rau xa cov kev pabcuam, luam cov lej (kev ua haujlwm) mus rau qhov chaw khaws cia thiab tshaj tawm mus rau qhov kawg. Tom qab ntawd peb tsuas them rau txhua qhov kev hu mus rau qhov haujlwm no, tsis quav ntsej txog qhov kho vajtse qhov twg nws raug tua.
Kuv yuav sim piav qhia txog txoj hauv kev no nrog cov duab.
Classic kev xa tawm. Peb muaj kev pabcuam nrog ib qho kev thauj khoom. Peb tsa ob qho piv txwv: lub cev servers lossis piv txwv hauv AWS. Kev thov sab nraud raug xa mus rau cov xwm txheej no thiab ua tiav rau ntawd.
Raws li koj tuaj yeem pom hauv daim duab, cov servers tsis raug pov tseg ntawm qhov sib npaug. Ib qho yog 100% siv, muaj ob qhov kev thov, thiab ib qho tsuas yog 50% - ib nrab tsis ua haujlwm. Yog tias tsis yog peb qhov kev thov tuaj txog, tab sis 30, tom qab ntawd tag nrho cov kab ke yuav tsis tuaj yeem tiv nrog cov load thiab yuav pib qeeb.
Serverless deployment. Nyob rau hauv ib puag ncig serverless, xws li cov kev pab cuam tsis muaj piv txwv lossis servers. Muaj qee lub pas dej ua ke ntawm cov khoom siv cua sov - me me npaj Docker ntim nrog rau kev ua haujlwm code. Lub kaw lus tau txais kev thov sab nraud thiab rau txhua tus ntawm lawv lub serverless lub moj khaum tsa lub thawv me me nrog cov lej: nws ua cov lus thov tshwj xeeb no thiab tua lub thawv.
Ib qho kev thov - ib lub thawv tsa, 1000 thov - 1000 ntim. Thiab kev xa tawm ntawm hardware servers twb yog kev ua haujlwm ntawm huab muab kev pabcuam. Nws yog kiag li zais los ntawm serverless lub moj khaum. Hauv lub tswv yim no peb them rau txhua qhov kev hu. Piv txwv li, ib tug hu tuaj ib hnub - peb them rau ib tug hu, ib lab tuaj ib feeb - peb them rau ib lab. Los yog hauv ib ob, qhov no kuj tshwm sim.
Lub tswv yim ntawm kev tshaj tawm kev ua haujlwm tsis muaj server yog qhov tsim nyog rau kev pabcuam tsis muaj tebchaws. Thiab yog tias koj xav tau ib qho kev pabcuam (xeev) kev pabcuam, ces peb ntxiv cov ntaub ntawv rau qhov kev pabcuam. Nyob rau hauv cov ntaub ntawv no, thaum nws los txog rau kev ua hauj lwm nrog lub xeev, txhua lub xeev muaj nuj nqi tsuas yog sau thiab nyeem los ntawm lub database. Ntxiv mus, los ntawm cov ntaub ntawv ntawm ib qho ntawm peb hom tau piav qhia thaum pib ntawm tsab xov xwm.
Dab tsi yog qhov kev txwv ntawm tag nrho cov ntaub ntawv no? Cov no yog cov nqi ntawm ib ntus siv huab lossis kho vajtse server (lossis ntau lub servers). Nws tsis muaj teeb meem txawm tias peb siv cov ntaub ntawv qub lossis kev tswj hwm, txawm tias peb muaj Devops thiab tus thawj tswj hwm lossis tsis, peb tseem them nyiaj rau kho vajtse, hluav taws xob thiab cov ntaub ntawv xauj tsev 24/7. Yog tias peb muaj lub hauv paus classic, peb them rau tus tswv thiab qhev. Yog hais tias nws yog ib tug heev loaded sharded database, peb them rau 10, 20 los yog 30 servers, thiab peb them tas li.
Lub xub ntiag ntawm cov servers ruaj khov nyob rau hauv tus nqi qauv yav dhau los tau pom tias yog qhov tsim nyog phem. Cov ntaub ntawv ib txwm muaj kuj muaj lwm yam teeb meem, xws li kev txwv ntawm tus naj npawb ntawm kev sib txuas, kev txwv kev txwv, kev pom zoo ntawm thaj chaw - lawv tuaj yeem daws tau qee yam ntawm cov ntaub ntawv, tab sis tsis yog tag nrho ib zaug thiab tsis yog qhov zoo.
Serverless database - kev xav
Lo lus nug ntawm 2020: nws puas tuaj yeem ua rau database serverless thiab? Txhua tus tau hnov ββββtxog serverless backend ... cia sim ua kom cov database serverless?
Qhov no suab coj txawv txawv, vim hais tias lub database yog ib tug stateful kev pab cuam, tsis heev haum rau serverless infrastructure. Nyob rau tib lub sijhawm, lub xeev ntawm cov ntaub ntawv loj heev: gigabytes, terabytes, thiab hauv cov ntaub ntawv txheeb xyuas txawm petabytes. Nws tsis yooj yim los tsa nws hauv lub teeb yuag Docker ntim.
Ntawm qhov tod tes, yuav luag tag nrho cov ntaub ntawv niaj hnub no muaj ntau qhov laj thawj thiab cov khoom siv: kev lag luam, kev sib koom tes ncaj ncees, cov txheej txheem, kev sib raug zoo ntawm kev sib raug zoo thiab ntau qhov laj thawj. Rau ntau ntau cov ntaub ntawv logic, lub xeev me me txaus. Gigabytes thiab Terabytes yog siv ncaj qha los ntawm tsuas yog ib feem me me ntawm cov ntaub ntawv logic koom nrog ncaj qha ua cov lus nug.
Raws li, lub tswv yim yog: yog ib feem ntawm lub logic tso cai rau kev tua neeg tsis muaj tebchaws, vim li cas ho tsis faib lub hauv paus rau hauv Stateful thiab Stateless qhov chaw.
Serverless rau OLAP cov kev daws teeb meem
Cia peb pom dab tsi txiav ib lub database rau hauv Stateful thiab Stateless seem yuav zoo li siv cov piv txwv tswv yim.
Piv txwv li, peb muaj ib tug analytical database: Cov ntaub ntawv sab nraud (lub tog raj kheej liab ntawm sab laug), cov txheej txheem ETL uas thauj cov ntaub ntawv mus rau hauv cov ntaub ntawv, thiab tus kws tshuaj ntsuam uas xa cov lus nug SQL rau hauv cov ntaub ntawv. Qhov no yog ib qho classic data warehouse operation scheme.
Nyob rau hauv cov tswv yim no, ETL yog conditionally ua ib zaug. Tom qab ntawd koj yuav tsum tau them nyiaj tas li rau cov servers uas cov ntaub ntawv sau nrog cov ntaub ntawv sau nrog ETL, kom muaj qee yam xa cov lus nug.
Cia peb saib lwm txoj hauv kev uas tau siv hauv AWS Athena Serverless. Tsis muaj ib qho kev kho vajtse ruaj khov uas rub tawm cov ntaub ntawv khaws cia. Tsis tas li ntawd:
- Tus neeg siv xa cov lus nug SQL rau Athena. Lub Athena optimizer soj ntsuam cov lus nug SQL thiab tshawb xyuas lub khw muag khoom metadata (Metadata) rau cov ntaub ntawv tshwj xeeb xav tau los ua cov lus nug.
- Tus optimizer, raws li cov ntaub ntawv khaws tseg, rub tawm cov ntaub ntawv tsim nyog los ntawm cov chaw sab nraud mus rau qhov chaw cia ib ntus (cov ntaub ntawv ib ntus).
- Ib qho lus nug SQL los ntawm tus neeg siv raug tua nyob rau hauv ib ntus cia thiab cov txiaj ntsig tau xa rov qab rau tus neeg siv.
- Kev khaws cia ib ntus raug tshem tawm thiab cov peev txheej raug tso tawm.
Hauv no architecture, peb tsuas yog them rau cov txheej txheem ntawm kev ua raws li qhov kev thov. Tsis muaj kev thov - tsis muaj nqi.
Qhov no yog txoj hauv kev ua haujlwm thiab siv tsis yog hauv Athena Serverless nkaus xwb, tab sis kuj hauv Redshift Spectrum (hauv AWS).
Piv txwv Athena qhia tau hais tias Serverless database ua haujlwm ntawm cov lus nug tiag tiag nrog kaum thiab pua pua ntawm cov ntaub ntawv Terabytes. Pua pua ntawm Terabytes yuav xav tau ntau pua servers, tab sis peb tsis tas yuav them rau lawv - peb them rau qhov kev thov. Qhov ceev ntawm txhua qhov kev thov yog (hloov) tsawg piv rau cov kev txheeb xyuas tshwj xeeb xws li Vertica, tab sis peb tsis them rau lub sijhawm poob.
Xws li cov ntaub ntawv siv tau rau cov lus nug tsis tshua muaj kev tshuaj ntsuam ad-hoc. Piv txwv li, thaum peb txiav txim siab spontaneously sim ib qho kev xav ntawm qee cov ntaub ntawv loj heev. Athena yog zoo meej rau cov rooj plaub no. Rau kev thov tsis tu ncua, xws li ib qho system yog kim. Hauv qhov no, cache cov ntaub ntawv hauv qee qhov kev daws teeb meem tshwj xeeb.
Serverless rau OLTP cov kev daws teeb meem
Cov piv txwv yav dhau los tau saib ntawm OLAP (analytical) cov haujlwm. Tam sim no cia saib OLTP cov haujlwm.
Cia peb xav txog qhov scalable PostgreSQL lossis MySQL. Cia peb tsa ib qho kev tswj hwm ib txwm muaj PostgreSQL lossis MySQL nrog cov peev txheej tsawg. Thaum qhov piv txwv tau txais kev thauj khoom ntau dua, peb yuav txuas ntxiv cov replicas uas peb yuav faib ib feem ntawm kev nyeem ntawv. Yog tias tsis muaj kev thov lossis thauj khoom, peb muab cov ntawv luam tawm. Thawj qhov piv txwv yog tus tswv, thiab tus so yog replicas.
Lub tswv yim no yog siv rau hauv database hu ua Aurora Serverless AWS. Lub hauv paus ntsiab lus yog qhov yooj yim: kev thov los ntawm cov ntawv thov sab nraud tau txais los ntawm lub npe hu ua nkoj. Pom qhov kev thauj khoom nce ntxiv, nws faib cov khoom siv suav nrog los ntawm qhov ua ntej ua kom sov tsawg - qhov kev sib txuas tau ua kom sai li sai tau. Kev xiam oob qhab tshwm sim ib yam nkaus.
Hauv Aurora muaj lub tswvyim ntawm Aurora Capacity Unit, ACU. Qhov no yog (conditionally) ib qho piv txwv (server). Txhua qhov tshwj xeeb ACU tuaj yeem yog tus tswv lossis tus qhev. Txhua chav muaj peev xwm muaj nws tus kheej RAM, processor thiab tsawg kawg disk. Raws li, ib tug yog tus tswv, tus so yog nyeem tsuas yog replicas.
Tus naj npawb ntawm cov Aurora Capacity Units khiav yog ib qho kev teeb tsa. Qhov tsawg kawg nkaus kom muaj nuj nqis tuaj yeem yog ib lossis xoom (hauv qhov no, cov ntaub ntawv tsis ua haujlwm yog tias tsis muaj kev thov).
Thaum lub hauv paus tau txais kev thov, tus neeg tsav nkoj lub npe nce Aurora CapacityUnits, ua kom lub cev muaj peev xwm ua tau zoo. Lub peev xwm los nce thiab txo cov peev txheej tso cai rau lub kaw lus "juggle" cov peev txheej: cia li tso tawm ib tus neeg ACUs (hloov lawv nrog cov tshiab) thiab yob tawm tag nrho cov kev hloov tshiab tam sim no rau cov peev txheej rho tawm.
Aurora Serverless puag tuaj yeem ntsuas qhov kev nyeem ntawv. Tab sis cov ntaub ntawv tsis hais qhov no ncaj qha. Tej zaum nws yuav xav tias lawv tuaj yeem nqa tus tswv ntau. Tsis muaj khawv koob.
Cov ntaub ntawv no yog qhov tsim nyog kom tsis txhob siv nyiaj ntau ntawm cov kab ke nrog kev nkag mus tsis tau. Piv txwv li, thaum tsim MVP lossis lag luam daim npav lag luam chaw, peb feem ntau tsis xav tias yuav muaj kev ruaj khov. Raws li, yog tias tsis muaj kev nkag mus, peb tsis them rau cov piv txwv. Thaum tsis muaj kev cia siab tshwm sim, piv txwv li tom qab lub rooj sib tham lossis kev tshaj tawm kev tshaj tawm, cov neeg coob coob tuaj xyuas lub xaib thiab cov khoom thauj tau nce ntau, Aurora Serverless tau txais cov khoom thauj no thiab txuas sai sai rau cov peev txheej uas ploj lawm (ACU). Tom qab ntawd lub rooj sib tham dhau mus, txhua tus tsis nco qab txog tus qauv, cov servers (ACU) mus tsaus ntuj, thiab cov nqi poob rau xoom - yooj yim.
Qhov kev daws teeb meem no tsis haum rau kev ruaj khov siab vim nws tsis ntsuas qhov kev sau ntawv. Tag nrho cov kev sib txuas thiab kev cuam tshuam ntawm cov peev txheej no tshwm sim ntawm qhov hu ua "scale point" - ib qho taw qhia nyob rau hauv lub sij hawm thaum cov ntaub ntawv tsis tau txais kev txhawb nqa los ntawm kev sib pauv lossis cov rooj ib ntus. Piv txwv li, nyob rau hauv ib lub lis piam qhov ntsuas qhov ntsuas yuav tsis tshwm sim, thiab lub hauv paus ua haujlwm ntawm tib cov peev txheej thiab tsuas yog tsis tuaj yeem nthuav lossis cog lus.
Tsis muaj khawv koob - nws yog ib txwm PostgreSQL. Tab sis cov txheej txheem ntawm kev ntxiv cov cav tov thiab disconnecting lawv yog ib nrab automated.
Serverless los ntawm kev tsim
Aurora Serverless yog cov ntaub ntawv qub uas tau sau dua tshiab rau huab kom tau txais txiaj ntsig ntawm qee qhov txiaj ntsig ntawm Serverless. Thiab tam sim no kuv mam li qhia rau koj txog lub hauv paus, uas yog Ameslikas sau rau huab, rau serverless mus kom ze - Serverless-los-tsim. Nws tau tsim tam sim ntawd yam tsis muaj kev xav tias nws yuav khiav ntawm lub cev servers.
Lub hauv paus no hu ua Snowflake. Nws muaj peb qhov tseem ceeb blocks.
Qhov thib ib yog metadata thaiv. Qhov no yog qhov kev pabcuam ceev ceev hauv lub cim xeeb uas daws teeb meem nrog kev ruaj ntseg, metadata, kev hloov pauv, thiab kev nug kom zoo (pom hauv cov duab kos sab laug).
Qhov thib ob thaiv yog ib txheej ntawm cov cuab yeej siv hauv virtual rau kev suav (hauv cov duab qhia muaj ib txheej ntawm lub voj voog xiav).
Qhov thaiv thib peb yog cov ntaub ntawv khaws cia raws li S3. S3 yog dimensionless khoom cia hauv AWS, zoo li dimensionless Dropbox rau kev lag luam.
Cia peb saib seb Snowflake ua haujlwm li cas, piv txwv li pib txias. Uas yog, muaj ib tug database, cov ntaub ntawv yog loaded rau hauv nws, tsis muaj khiav queries. Raws li, yog tias tsis muaj kev thov rau cov ntaub ntawv, ces peb tau tsa cov kev pabcuam ceev ceev hauv Metadata (thawj thaiv). Thiab peb muaj S3 cia, qhov twg cov ntaub ntawv yog khaws cia, muab faib ua cov micropartitions. Rau kev yooj yim: yog tias lub rooj muaj kev lag luam, ces micropartitions yog hnub ntawm kev lag luam. Txhua hnub yog cais micropartition, cais cov ntaub ntawv. Thiab thaum lub database ua haujlwm hauv hom no, koj tsuas yog them rau qhov chaw nyob los ntawm cov ntaub ntawv. Ntxiv mus, tus nqi ib lub rooj yog tsawg heev (tshwj xeeb tshaj yog noj mus rau hauv tus account qhov tseem ceeb compression). Cov kev pabcuam metadata kuj tseem ua haujlwm tas li, tab sis koj tsis xav tau ntau cov peev txheej los txhim kho cov lus nug, thiab cov kev pabcuam tuaj yeem suav tias yog shareware.
Tam sim no cia peb xav txog tias tus neeg siv tuaj rau peb cov ntaub ntawv thiab xa cov lus nug SQL. Cov lus nug SQL raug xa mus rau Metadata kev pabcuam tam sim. Yog li, thaum tau txais kev thov, qhov kev pabcuam no txheeb xyuas qhov kev thov, cov ntaub ntawv muaj, cov neeg siv kev tso cai thiab, yog tias txhua yam ua tau zoo, kos cov phiaj xwm los ua cov ntawv thov.
Tom qab ntawd, qhov kev pabcuam pib pib lub tshuab xam phaj. Lub tshuab xam zauv yog ib pawg ntawm cov servers uas ua kev suav. Ntawd yog, qhov no yog pawg uas tuaj yeem muaj 1 server, 2 servers, 4, 8, 16, 32 - ntau npaum li koj xav tau. Koj pov ib qho kev thov thiab kev tso tawm ntawm pawg no tam sim ntawd pib. Nws yeej siv sijhawm vib nas this.
Tom qab ntawd, tom qab pawg tau pib, cov micropartitions xav tau los ua cov ntawv koj thov pib muab luam rau hauv pawg los ntawm S3. Ntawd yog, cia peb xav txog tias txhawm rau ua tiav cov lus nug SQL koj xav tau ob ntu ntawm ib lub rooj thiab ib qho ntawm qhov thib ob. Hauv qhov no, tsuas yog peb qhov tsim nyog yuav tsum tau theej rau hauv pawg, thiab tsis yog tag nrho cov ntxhuav nkaus xwb. Tias yog vim li cas, thiab yog vim li cas txhua yam nyob rau hauv ib lub chaw cov ntaub ntawv thiab kev cob cog rua los ntawm ceev heev raws, tag nrho cov txheej txheem hloov lwm lub tsev tshwm sim sai heev: nyob rau hauv vib nas this, tsis tshua muaj nyob rau hauv feeb, tshwj tsis yog peb tab tom tham txog ib co monstrous thov. Raws li, micropartitions raug theej rau hauv pawg suav, thiab, thaum ua tiav, cov lus nug SQL raug ua tiav ntawm pawg suav no. Qhov tshwm sim ntawm qhov kev thov no tuaj yeem yog ib kab, ob peb kab lossis ib lub rooj - lawv raug xa tawm sab nraud rau tus neeg siv kom nws tuaj yeem rub tawm, tso tawm hauv nws lub cuab yeej BI, lossis siv lwm txoj hauv kev.
Txhua cov lus nug SQL tuaj yeem tsis tsuas yog nyeem cov ntawv sau los ntawm cov ntaub ntawv yav dhau los, tab sis kuj thauj khoom / tsim cov ntaub ntawv tshiab hauv cov ntaub ntawv. Ntawd yog, nws tuaj yeem yog cov lus nug uas, piv txwv li, ntxig cov ntaub ntawv tshiab rau hauv lwm lub rooj, uas ua rau pom qhov kev faib tawm tshiab ntawm pawg suav, uas, dhau los, tau txais kev cawmdim hauv ib qho S3 cia.
Qhov xwm txheej tau piav qhia saum toj no, los ntawm kev tuaj txog ntawm tus neeg siv mus rau kev nce ntawm pawg, thauj cov ntaub ntawv, ua tiav cov lus nug, tau txais cov txiaj ntsig, yog them tus nqi rau feeb ntawm kev siv cov khoom siv hauv virtual, lub tsev khaws khoom virtual. Tus nqi sib txawv nyob ntawm thaj tsam AWS thiab pawg loj, tab sis qhov nruab nrab nws yog ob peb las ib teev. Ib pawg ntawm plaub lub tshuab yog ob zaug kim li ib pawg ntawm ob lub tshuab, thiab ib pawg ntawm yim lub tshuab tseem kim dua ob zaug. Kev xaiv ntawm 16, 32 lub tshuab muaj, nyob ntawm qhov nyuaj ntawm kev thov. Tab sis koj tsuas yog them rau cov feeb thaum pawg ua haujlwm tiag tiag, vim tias thaum tsis muaj kev thov, koj zoo li tshem koj txhais tes, thiab tom qab 5-10 feeb ntawm kev tos (ib qho kev teeb tsa) nws yuav tawm ntawm nws tus kheej, tso cov peev txheej thiab ua dawb.
Qhov xwm txheej tiag tiag yog thaum koj xa ib daim ntawv thov, pawg pops tuaj, kuj hais lus, hauv ib feeb, nws suav lwm feeb, tom qab ntawd tsib feeb los kaw, thiab koj kawg them rau xya feeb ntawm kev ua haujlwm ntawm pawg no, thiab tsis yog rau hli thiab xyoo.
Thawj qhov xwm txheej tau piav qhia siv Snowflake hauv kev teeb tsa ib tus neeg siv. Tam sim no cia peb xav txog tias muaj ntau tus neeg siv, uas yog ze rau qhov tseeb scenario.
Cia peb hais tias peb muaj ntau tus kws tshuaj ntsuam thiab Tableau cov lus ceeb toom uas niaj hnub foob peb cov ntaub ntawv nrog ntau cov lus nug yooj yim SQL analytical.
Tsis tas li ntawd, cia peb hais tias peb muaj cov kws tshawb fawb cov ntaub ntawv uas tau sim ua ntau yam nrog cov ntaub ntawv, ua haujlwm nrog kaum tawm Terabytes, txheeb xyuas billions thiab trillions ntawm kab ntawm cov ntaub ntawv.
Rau ob hom kev ua haujlwm tau piav qhia saum toj no, Snowflake tso cai rau koj los tsa ob peb lub khoos phis tawj ywj pheej ntawm cov peev txheej sib txawv. Tsis tas li ntawd, cov khoos phis tawj no ua haujlwm ntawm nws tus kheej, tab sis nrog cov ntaub ntawv sib xws.
Rau ntau cov lus nug lub teeb, koj tuaj yeem tsa 2-3 pawg me me, kwv yees li 2 lub tshuab txhua. Qhov kev coj cwj pwm no tuaj yeem siv tau, ntawm lwm yam, siv qhov chaw tsis siv neeg. Yog li koj hais tias, βSnowflake, tsa ib pawg me me. Yog hais tias lub load rau nws nce siab tshaj ib tug tej yam parameter, tsa ib tug zoo sib xws thib ob, peb. Thaum lub load pib down, extinguish tus tshaj. " Yog li ntawd txawm tias muaj pes tsawg tus kws tshuaj ntsuam tuaj thiab pib saib cov ntawv ceeb toom, txhua tus muaj peev txheej txaus.
Nyob rau tib lub sijhawm, yog tias cov kws tshuaj ntsuam tau tsaug zog thiab tsis muaj leej twg saib cov ntawv ceeb toom, cov pawg yuav tsaus nti, thiab koj tsis them nyiaj rau lawv.
Nyob rau tib lub sijhawm, rau cov lus nug hnyav (los ntawm Cov Kws Tshawb Fawb Cov Ntaub Ntawv), koj tuaj yeem tsa ib pawg loj heev rau 32 lub tshuab. Cov pawg no tseem yuav raug them rau cov feeb thiab teev thaum koj qhov kev thov loj tseem tab tom khiav ntawm ntawd.
Lub cib fim tau piav qhia saum toj no tso cai rau koj faib tsis tau tsuas yog 2, tab sis kuj muaj ntau hom kev ua haujlwm rau hauv pawg (ETL, saib xyuas, tshaj tawm cov khoom siv, ...).
Cia peb sau txog Snowflake. Lub hauv paus ua ke nrog lub tswv yim zoo nkauj thiab ua haujlwm tau zoo. Ntawm ManyChat, peb siv Snowflake los tshuaj xyuas tag nrho cov ntaub ntawv peb muaj. Peb tsis muaj peb pawg, xws li hauv qhov piv txwv, tab sis los ntawm 5 txog 9, ntawm qhov sib txawv. Peb muaj cov pa 16-tshuab, 2-tshuab, thiab kuj super-me 1-tshuab sawv daws yuav rau qee yam haujlwm. Lawv ntse faib cov load thiab cia peb txuag tau ntau heev.
Lub database ntse ntsuas qhov kev nyeem ntawv thiab sau ntawv. Qhov no yog qhov sib txawv loj heev thiab kev tawg loj heev piv rau tib "Aurora", uas tsuas yog nqa cov ntawv nyeem xwb. Snowflake tso cai rau koj los ntsuas koj cov ntawv sau ua haujlwm nrog cov lej suav no. Ntawd yog, raws li kuv tau hais, peb siv ntau pawg hauv ManyChat, pawg me me thiab super-me me feem ntau yog siv rau ETL, rau kev thauj cov ntaub ntawv. Thiab cov kws tshuaj ntsuam twb nyob rau hauv nruab nrab pawg, uas yog kiag li tsis cuam tshuam los ntawm ETL load, yog li lawv ua hauj lwm sai heev.
Yog li ntawd, cov ntaub ntawv khaws cia tau zoo rau cov haujlwm OLAP. Txawm li cas los xij, hmoov tsis, nws tseem tsis tau siv rau OLTP cov haujlwm ua haujlwm. Ua ntej, cov ntaub ntawv no yog columnar, nrog rau tag nrho cov kev tshwm sim tshwm sim. Qhov thib ob, txoj hauv kev ntawm nws tus kheej, thaum twg rau txhua qhov kev thov, yog tias tsim nyog, koj tsa lub khoos phis tawj thiab dej nyab nrog cov ntaub ntawv, hmoov tsis, tseem tsis tau nrawm txaus rau OLTP loads. Tos vib nas this rau OLAP cov dej num yog ib txwm, tab sis rau cov hauj lwm OLTP nws yog tsis tsim nyog; 100 ms yuav zoo dua, los yog 10 ms yuav zoo dua.
Qhov no
Lub serverless database yog ua tau los ntawm kev faib cov database rau hauv Stateless thiab Stateful qhov chaw. Tej zaum koj yuav tau pom tias nyob rau hauv tag nrho cov piv txwv saum toj no, lub Stateful ib feem yog, kuj hais lus, khaws cia micro-partitions nyob rau hauv S3, thiab Stateless yog lub optimizer, ua hauj lwm nrog metadata, tuav cov teeb meem kev ruaj ntseg uas yuav tsum tau tsa ua ywj siab lub teeb hnyav Stateless kev pab cuam.
Kev ua tiav SQL queries kuj tuaj yeem pom tau tias yog cov kev pabcuam hauv lub xeev uas tuaj yeem tshwm sim hauv serverless hom, xws li Snowflake suav cov pawg, rub tawm tsuas yog cov ntaub ntawv tsim nyog, ua cov lus nug thiab "tawm mus."
Serverless qib ntau lawm databases twb muaj rau siv, lawv ua hauj lwm. Cov serverless databases twb tau npaj los ua haujlwm OLAP. Hmoov tsis zoo, rau cov haujlwm OLTP lawv tau siv ... nrog nuances, vim muaj kev txwv. Ntawm qhov tod tes, qhov no yog qhov rho tawm. Tab sis, ntawm qhov tod tes, qhov no yog lub sijhawm. Tej zaum ib tug ntawm cov nyeem yuav nrhiav tau ib txoj hauv kev los ua OLTP database kiag li serverless, tsis muaj kev txwv ntawm Aurora.
Kuv vam tias koj pom nws nthuav. Serverless yog yav tom ntej :)
Tau qhov twg los: www.hab.com