Xyoo 2017, peb yeej qhov kev sib tw los tsim lub hauv paus kev lag luam rau Alfa-Bank txoj kev lag luam peev thiab pib ua haujlwm (ntawm HighLoad ++ 2018 nrog daim ntawv qhia txog lub hauv paus kev lag luam peev). Vladimir Drynkin, Tus Thawj Coj ntawm Kev Ua Lag Luam rau Alfa-Bank's Investment Business). Lub kaw lus no tau tsim los sau cov ntaub ntawv kev lag luam los ntawm ntau qhov chaw hauv ntau hom ntawv, ua kom cov ntaub ntawv zoo li qub, khaws cia, thiab muab kev nkag mus rau nws.
Thaum lub sijhawm tsim kho, lub kaw lus tau hloov zuj zus thiab tau txais cov haujlwm tshiab, thiab qee lub sijhawm peb pom tias peb tab tom tsim ib yam dab tsi loj dua li tsuas yog software application uas tsim los daws cov teeb meem uas tau teev tseg nruj me ntsis: peb muaj ib lub kaw lus rau kev tsim cov ntawv thov faib tawm nrog kev khaws cia tas mus liQhov kev paub uas peb tau txais tau tsim lub hauv paus ntawm cov khoom tshiab - (TDG).
Kuv xav tham txog TDG lub architecture thiab cov kev daws teeb meem uas peb tau txais thaum lub sijhawm tsim kho, qhia koj txog nws cov haujlwm tseem ceeb, thiab qhia koj tias peb cov khoom tuaj yeem ua lub hauv paus rau kev tsim cov kev daws teeb meem tiav li cas.
Nyob rau hauv kev tsim vaj tsev, peb tau faib lub kaw lus ua cais luag hauj lwm, txhua tus muaj lub luag haujlwm daws ib pawg haujlwm tshwj xeeb. Ib qho piv txwv ntawm daim ntawv thov khiav siv ib lossis ntau hom lub luag haujlwm. Ib pawg tuaj yeem muaj ntau lub luag haujlwm ntawm tib hom:
connector
Tus connector yog lub luag haujlwm rau kev sib txuas lus nrog lub ntiaj teb sab nraud; nws txoj haujlwm yog txais ib qho kev thov, txheeb xyuas nws, thiab, yog tias ua tiav, xa cov ntaub ntawv mus rau lub processor input rau kev ua tiav. Peb txhawb HTTP, SOAP, Kafka, thiab FIX. Lub architecture tso cai rau kev ntxiv yooj yim ntawm cov hom ntawv tshiab, nrog rau kev txhawb nqa IBM MQ yuav los sai sai no. Yog tias kev txheeb xyuas ib qho kev thov ua rau muaj qhov yuam kev, tus connector yuav rov qab qhov yuam kev; txwv tsis pub, nws yuav teb tias qhov kev thov tau ua tiav, txawm tias muaj qhov yuam kev tshwm sim thaum lub sijhawm ua tiav ntxiv. Qhov no yog ua tiav los ua haujlwm nrog cov kab ke uas tsis tuaj yeem rov sim dua cov kev thov - lossis, ntawm qhov tod tes, ua li ntawd tas li. Txhawm rau kom tsis txhob poob cov ntaub ntawv, kab kho tau siv: ib yam khoom raug ntxiv rau nws thawj zaug thiab tsuas yog tom qab ua tiav nws raug tshem tawm. Tus thawj coj tuaj yeem tau txais cov ntawv ceeb toom txog cov khoom uas tseem nyob hauv kab kho thiab, tom qab qhov yuam kev software lossis kho vajtse tsis ua haujlwm tau daws, ua qhov rov sim dua.
Tus neeg ua haujlwm nkag
Tus processor input faib cov ntaub ntawv tau txais raws li cov yam ntxwv thiab hu rau cov handlers tsim nyog. Handlers yog Lua code khiav hauv lub sandbox, yog li lawv tsis tuaj yeem cuam tshuam rau kev ua haujlwm ntawm lub system. Nyob rau theem no, cov ntaub ntawv tuaj yeem hloov pauv mus rau hauv hom ntawv xav tau, thiab, yog tias tsim nyog, txhua tus lej ntawm cov haujlwm tuaj yeem tso tawm los siv cov logic xav tau. Piv txwv li, hauv MDM (Master Data Management) cov khoom tsim los ntawm Tarantool Data Grid, thaum ntxiv tus neeg siv tshiab, peb tso tawm kev tsim cov ntaub ntawv kub ua ib txoj haujlwm sib cais kom tsis txhob qeeb qeeb ntawm kev ua cov lus thov. Lub sandbox txhawb nqa nyeem, hloov kho, thiab ntxiv cov lus nug, tso cai rau kev ua tiav ntawm qee yam haujlwm thoob plaws txhua lub luag haujlwm khaws cia, thiab cov txiaj ntsig sib sau ua ke (daim ntawv qhia / txo).
Cov neeg ua haujlwm tuaj yeem piav qhia hauv cov ntaub ntawv:
sum.lua
local x, y = unpack(...)
return x + yThiab tom qab ntawd tshaj tawm hauv qhov configuration:
functions:
sum: { __file: sum.lua }
Vim li cas thiaj siv Lua? Lua yog ib hom lus yooj yim heev. Hauv peb qhov kev paub, tib neeg pib sau cov code uas daws lawv cov teeb meem hauv ob peb teev tom qab kawm nws. Thiab qhov no tsis yog siv rau cov kws tsim khoom tshaj lij xwb, tab sis kuj siv rau cov kws tshuaj ntsuam xyuas, piv txwv li. Ntxiv mus, ua tsaug rau JIT compiler, Lua ceev heev.
cia
Qhov chaw cia khoom khaws cov ntaub ntawv uas nyob mus ib txhis. Ua ntej khaws cia, cov ntaub ntawv raug kuaj xyuas raws li cov qauv ntaub ntawv. Peb siv hom ntawv txuas ntxiv los piav qhia txog cov qauv. ... Piv txwv:
{
"name": "User",
"type": "record",
"logicalType": "Aggregate",
"fields": [
{ "name": "id", "type": "string"},
{"name": "first_name", "type": "string"},
{"name": "last_name", "type": "string"}
],
"indexes": ["id"]
}Raws li cov lus piav qhia no, DDL (Data Definition Language) yog tsim los rau Tarantula DBMS thiab txoj kev nkag mus rau cov ntaub ntawv.
Kev txhawb nqa cov ntaub ntawv asynchronous replication (npaj ntxiv synchronous replication).
Tus processor tso zis
Qee zaum cov neeg siv khoom sab nraud yuav tsum tau ceeb toom txog cov ntaub ntawv tshiab tuaj txog, thiab lub luag haujlwm Output Processor muaj nyob rau lub hom phiaj no. Tom qab cov ntaub ntawv raug khaws cia, nws tuaj yeem xa mus rau lub processor tsim nyog (piv txwv li, los teeb tsa nws raws li cov neeg siv khoom xav tau) thiab tom qab ntawd xa mus rau lub connector rau kev xa tawm. Ib kab kho kuj siv ntawm no: yog tias tsis muaj leej twg tau txais cov khoom, tus thawj coj tuaj yeem sim dua tom qab.
Scaling
Cov lub luag haujlwm ntawm lub connector, input processor, thiab output processor yog stateless, uas tso cai rau peb kom scale lub system horizontally los ntawm kev ntxiv cov application instances tshiab nrog rau hom role type uas xav tau. Storage siv rau scaling kab rov tav. Txhawm rau teeb tsa ib pawg siv cov thoob virtual. Tom qab ntxiv lub server tshiab, qee cov thoob los ntawm cov servers qub raug tsiv mus rau lub server tshiab hauv keeb kwm yav dhau; qhov no tshwm sim rau cov neeg siv thiab tsis cuam tshuam rau tag nrho lub kaw lus.
Cov khoom ntawm cov ntaub ntawv
Cov khoom yuav loj heev thiab muaj lwm yam khoom. Peb xyuas kom meej tias cov ntaub ntawv atomic ntxig thiab hloov kho los ntawm kev khaws cia ib yam khoom nrog txhua yam nws nyob ntawm ib lub thoob virtual. Qhov no tiv thaiv ib yam khoom kom tsis txhob raug "smearned" hla ntau lub servers.
Kev txhawb nqa versioning: txhua qhov hloov tshiab rau ib yam khoom tsim ib qho version tshiab, thiab peb tuaj yeem siv sijhawm thiab saib seb lub ntiaj teb zoo li cas thaum ntawd. Rau cov ntaub ntawv uas tsis xav tau keeb kwm ntev, peb tuaj yeem txwv tus naj npawb ntawm cov versions lossis txawm tias khaws cia tsuas yog ib qho - qhov tshiab tshaj plaws - ua kom zoo rau kev kaw versioning rau ib hom tshwj xeeb. Peb kuj tuaj yeem txwv keeb kwm los ntawm lub sijhawm: piv txwv li, rho tawm tag nrho cov khoom ntawm ib hom tshwj xeeb uas laus dua ib xyoos. Kev khaws cia kuj tseem txhawb nqa: peb tuaj yeem tshem tawm cov khoom uas laus dua lub sijhawm teev tseg, ua kom muaj chaw seem hauv pawg.
paub tab
Ntawm cov yam ntxwv nthuav, nws tsim nyog sau tseg lub peev xwm los khiav cov haujlwm raws sijhawm, ntawm tus neeg siv qhov kev thov, lossis programmatically los ntawm lub sandbox:
Ntawm no peb pom lwm lub luag haujlwm - tus neeg khiav. Lub luag haujlwm no tsis muaj xeev, thiab cov ntawv thov ntxiv nrog lub luag haujlwm no tuaj yeem ntxiv rau pawg yog tias xav tau. Lub luag haujlwm ntawm tus neeg khiav yog ua tiav cov haujlwm. Raws li tau hais, cov haujlwm tshiab tuaj yeem tsim los ntawm lub sandbox; lawv raug khaws cia rau hauv kab cia thiab tom qab ntawd ua tiav ntawm tus neeg khiav. Hom haujlwm no hu ua Txoj Haujlwm. Peb kuj muaj hom haujlwm hu ua Task - cov haujlwm uas tus neeg siv txhais uas khiav ntawm lub sijhawm teem sijhawm (siv cron syntax) lossis thaum xav tau. Txhawm rau pib thiab saib xyuas cov haujlwm zoo li no, peb muaj tus thawj tswj haujlwm yooj yim. Rau qhov kev ua haujlwm no kom muaj, lub luag haujlwm teem sijhawm yuav tsum tau qhib. Lub luag haujlwm no yog stateful thiab yog li ntawd tsis tuaj yeem ntsuas tau, uas tsis tas yuav tsum muaj. Txawm li cas los xij, zoo li txhua lub luag haujlwm lwm yam, nws tuaj yeem muaj tus qauv uas siv yog tias tus tswv tsis ua haujlwm tam sim ntawd.
Tus neeg sau ntawv
Lwm lub luag haujlwm hu ua logger. Nws sau cov cav los ntawm txhua tus tswv cuab ntawm pawg thiab muab ib qho interface rau rub tawm thiab saib lawv los ntawm lub web interface.
Kev pabcuam
Nws tsim nyog hais tias lub kaw lus ua rau nws yooj yim los tsim cov kev pabcuam. Cov ntaub ntawv teeb tsa tuaj yeem siv los qhia meej tias cov kev thov twg yuav tsum tau xa mus rau tus neeg siv sau tus neeg ua haujlwm khiav hauv lub sandbox. Tus neeg ua haujlwm no tuaj yeem, piv txwv li, ua tiav cov lus nug analytical thiab xa rov qab cov txiaj ntsig.
Cov kev pabcuam tau piav qhia nyob rau hauv cov ntaub ntawv teeb tsa:
services:
sum:
doc: "adds two numbers"
function: sum
return_type: int
args:
x: int
y: int
Lub GraphQL API raug tsim tawm yam tsis tas siv neeg thiab qhov kev pabcuam no yuav siv tau rau kev thov:
query {
sum(x: 1, y: 2)
} Qhov no yuav ua rau tus neeg ua haujlwm raug hu ua sum, uas yuav rov qab los qhov tshwm sim:
3
Kev txheeb xyuas cov lus nug thiab cov ntsuas
Yuav kom nkag siab txog kev ua haujlwm ntawm lub system thiab kev txheeb xyuas cov lus nug, peb tau siv kev txhawb nqa rau OpenTracing protocol. Lub system tuaj yeem xa cov ntaub ntawv raws li qhov xav tau rau cov cuab yeej uas txhawb nqa cov protocol no, xws li Zipkin, uas cia peb nkag siab txog kev ua tiav ntawm cov lus nug:
Ntawm chav kawm, lub kaw lus muab cov ntsuas sab hauv uas tuaj yeem sau tau siv Prometheus thiab pom tau siv Grafana.
Muab tso rau
Tarantool Data Grid tuaj yeem xa tawm los ntawm RPM pob khoom lossis cov ntaub ntawv khaws cia, siv cov khoom siv muab los yog Ansible, thiab kuj tseem muaj kev txhawb nqa rau Kubernetes ().
Daim ntawv thov uas siv cov logic ua lag luam (kev teeb tsa, cov neeg ua haujlwm) raug thauj mus rau hauv Tarantool Data Grid cluster uas tau xa tawm ua ib qho archive ntawm UI lossis siv cov ntawv sau ntawm API uas peb muab.
Piv txwv ntawm daim ntawv thov
Cov ntawv thov twg tuaj yeem tsim nrog Tarantool Data Grid? Qhov tseeb, feem ntau cov haujlwm lag luam cuam tshuam nrog kev ua, khaws cia, thiab nkag mus rau cov ntaub ntawv ntws. Yog li ntawd, yog tias koj muaj cov ntaub ntawv ntws loj uas yuav tsum tau khaws cia thiab nkag mus rau qhov ruaj ntseg, peb cov khoom tuaj yeem txuag koj lub sijhawm tsim kho tseem ceeb, cia koj tsom mus rau koj txoj kev lag luam.
Piv txwv li, peb xav sau cov ntaub ntawv hais txog kev ua lag luam vaj tse kom tom qab, piv txwv li, paub cov nqe zoo tshaj plaws. Hauv qhov no, peb yuav txheeb xyuas cov haujlwm hauv qab no:
- Cov neeg hlau uas sau cov ntaub ntawv los ntawm cov chaw qhib yuav yog peb cov chaw khaws cov ntaub ntawv. Koj tuaj yeem daws qhov teeb meem no siv cov kev daws teeb meem uas tau npaj tseg lossis los ntawm kev sau cov lej hauv txhua hom lus.
- Tarantool Data Grid mam li txais thiab khaws cov ntaub ntawv. Yog tias hom ntawv cov ntaub ntawv los ntawm ntau qhov chaw sib txawv, koj tuaj yeem sau Lua code los hloov nws mus rau hauv hom ntawv sib koom ua ke. Thaum lub sijhawm ua ntej, koj tuaj yeem lim cov kev muab khoom sib txawv lossis hloov kho lub ntaub ntawv nrog cov ntaub ntawv hais txog cov neeg sawv cev ua haujlwm hauv kev ua lag luam.
- Tam sim no koj muaj ib qho kev daws teeb meem uas tuaj yeem nthuav dav hauv ib pawg uas tuaj yeem muaj cov ntaub ntawv thiab rov qab tau. Tom ntej no, koj tuaj yeem siv cov haujlwm tshiab, piv txwv li, sau ib qho kev pabcuam uas nug cov ntaub ntawv thiab xa rov qab qhov zoo tshaj plaws rau hnub ntawd - qhov no yuav xav tau ob peb kab hauv cov ntaub ntawv teeb tsa thiab qee cov lej Lua.
Yuav ua li cas yog tom ntej no?
Peb qhov tseem ceeb yog los txhim kho qhov yooj yim ntawm kev txhim kho nrog kev pab los ntawm Piv txwv li, qhov no yog IDE nrog kev txhawb nqa rau profiling thiab debugging handlers khiav hauv lub sandbox.
Peb kuj tseem ceeb heev rau kev ruaj ntseg. Tam sim no peb tab tom tau txais daim ntawv pov thawj los ntawm Lavxias Federal Service for Technical and Export Control (FSTEC) los xyuas kom meej tias peb muaj kev ruaj ntseg zoo thiab ua tau raws li cov cai ntawm daim ntawv pov thawj rau cov khoom siv software uas siv rau hauv cov ntaub ntawv tus kheej thiab cov ntaub ntawv tsoomfwv.
Tau qhov twg los: www.hab.com
