Kuchokera mufilimuyi "Chilengedwe Chathu Chobisika: Moyo Wobisika wa Cell"

Bizinesi yamalonda ndi imodzi mwamadera ovuta kwambiri m'mabanki, chifukwa palibe ngongole, kubwereketsa ndi ma depositi okha, komanso zotetezedwa, ndalama, katundu, zotumphukira ndi mitundu yonse ya zovuta mu mawonekedwe azinthu zopangidwa.

Posachedwapa, tawona kuwonjezeka kwa luso la zachuma la anthu. Anthu ochulukirachulukira akutenga nawo gawo pazamalonda m'misika yachitetezo. Maakaunti amunthu payekha adawonekera osati kale kwambiri. Amakulolani kuti mugulitse misika yachitetezo ndikulandila kuchotsera msonkho kapena kupewa kulipira misonkho. Ndipo makasitomala onse omwe amabwera kwa ife amafuna kuyang'anira mbiri yawo ndikuwona malipoti munthawi yeniyeni. Kuphatikiza apo, nthawi zambiri izi zimakhala ndi zinthu zambiri, ndiye kuti, anthu amakhala makasitomala amitundu yosiyanasiyana yamabizinesi.

Kuphatikiza apo, zosowa za owongolera, onse aku Russia ndi akunja, akukula.

Kuti tikwaniritse zosowa zapano ndikuyala maziko okweza mtsogolo, tapanga bizinesi yoyambira pa Tarantool.

Ziwerengero zina. Bizinesi yazachuma ya Alfa-Bank imapereka ntchito zobwereketsa kwa anthu ndi mabungwe ovomerezeka kuti apereke mwayi wochita malonda pamisika yosiyanasiyana yotetezedwa, ntchito zosungirako zosungirako zosungirako zotetezedwa, ntchito zowongolera zikhulupiliro za anthu omwe ali ndi ndalama zapadera komanso zazikulu, ntchito zoperekera zitetezo zamakampani ena. . Bizinesi yazachuma ya Alfa-Bank imaphatikizapo ma quotes opitilira 3 pamphindikati, omwe amatsitsidwa pamapulatifomu osiyanasiyana. Patsiku logwira ntchito, zogulitsa zoposa 300 zikwizikwi zimamalizidwa pamisika m'malo mwa banki kapena makasitomala ake. Kufikira ku 5 zikwi kuphedwa pamphindikati kumachitika pamapulatifomu akunja ndi amkati. Panthawi imodzimodziyo, makasitomala onse, mkati ndi kunja, akufuna kuwona malo awo mu nthawi yeniyeni.

prehistory

Kwinakwake kuyambira koyambirira kwa zaka za m'ma 2000, madera athu abizinesi yogulitsa ndalama adakula modziyimira pawokha: malonda osinthanitsa, ntchito zamabizinesi, malonda andalama, kugulitsa m'mabizinesi achitetezo ndi zotuluka zosiyanasiyana. Zotsatira zake, tagwa mumsampha wa zitsime zogwira ntchito. Ndi chiyani? Mzere uliwonse wa bizinesi uli ndi machitidwe ake omwe amafanana ndi ntchito za wina ndi mzake. Dongosolo lililonse liri ndi mtundu wake wa data, ngakhale limagwira ntchito ndi malingaliro omwewo: zochitika, zida, zofananira, zolemba, ndi zina zotero. Ndipo pamene dongosolo lililonse lidayamba kusinthika lodziyimira pawokha, mitundu yosiyanasiyana yaukadaulo idatulukira.

Kuphatikiza apo, ma code m'munsi mwa machitidwewa ndi akale kwambiri, chifukwa zinthu zina zidayamba m'ma 1990. Ndipo m'madera ena izi zinachedwetsa ndondomeko ya chitukuko, ndipo panali zovuta zogwirira ntchito.

Zofunikira payankho latsopano

Mabizinesi azindikira kuti kusintha kwaukadaulo ndikofunikira kuti chitukuko chipitirire. Tinapatsidwa ntchito:

1. Sonkhanitsani deta yonse yamabizinesi m'modzi, posungira mwachangu komanso mumtundu umodzi wa data.
2. Sitiyenera kutaya kapena kusintha chidziwitsochi.
3. Ndikofunika kumasulira deta, chifukwa nthawi iliyonse woyang'anira angafunse ziwerengero za zaka zapitazo.
4. Sitiyenera kungobweretsa DBMS yatsopano, yodziwika bwino, koma kupanga nsanja yothanirana ndi zovuta zamabizinesi.

Kuphatikiza apo, omanga athu amakhazikitsa zinthu zawo:

1. Yankho latsopano liyenera kukhala lamakampani, ndiye kuti, liyenera kuyesedwa kale m'makampani ena akuluakulu.
2. Njira yothetsera vutoli iyenera kukhala yofunika kwambiri. Izi zikutanthauza kuti tiyenera kukhalapo m'malo angapo a data nthawi imodzi ndikupulumuka modekha pakutha kwa malo amodzi a data.
3. Dongosolo liyenera kukhala lokhazikika mozungulira. Chowonadi ndi chakuti machitidwe athu onse omwe alipo tsopano amangowonjezereka, ndipo tikugunda kale padenga chifukwa cha kuchepa kwa mphamvu ya hardware. Chifukwa chake, nthawi yafika yoti tifunika kukhala ndi njira yowongoka kuti tipulumuke.
4. Mwa zina, tinauzidwa kuti njira yothetsera vutoli iyenera kukhala yotchipa.

Tinatsatira njira yokhazikika: tinapanga zofunikira ndikulumikizana ndi dipatimenti yogula. Kuchokera kumeneko tinalandira mndandanda wa makampani omwe, ambiri, ali okonzeka kutichitira izi. Tinauza aliyense za vutolo, ndipo tidalandira mayankho kuchokera kwa asanu ndi mmodzi mwa iwo.

Ku banki, sititenga mawu a aliyense; timakonda kuyesa chilichonse tokha. Chifukwa chake, chofunikira pa mpikisano wathu wamatenda chinali kuti tipambane mayeso a katundu. Tidapanga ntchito zoyeserera zolemetsa, ndipo makampani atatu mwa asanu ndi mmodzi agwirizana kale kuti agwiritse ntchito njira yoyeserera potengera matekinoloje apamtima ndi ndalama zawo kuti ayese.

Sindikuwuzani momwe tidayesera chilichonse komanso kuti zidatenga nthawi yayitali bwanji, ndingofotokoza mwachidule: magwiridwe antchito abwino kwambiri pamayeso olemetsa adawonetsedwa ndi yankho lachitsanzo lochokera ku Tarantool kuchokera ku gulu lachitukuko la Mail.ru Gulu. Tinasaina mgwirizano ndikuyamba chitukuko. Panali anthu anayi ochokera ku Mail.ru Group, ndipo kuchokera ku Alfa-Bank panali atatu opanga, akatswiri atatu a machitidwe, wokonza njira zothetsera mavuto, mwiniwake wa malonda ndi Scrum master.

Kenako ndikuuzeni momwe dongosolo lathu linakulira, momwe lidasinthira, zomwe tidachita komanso chifukwa chake izi.

Development

Funso loyamba lomwe tinadzifunsa ndi momwe tingapezere deta kuchokera ku machitidwe athu amakono. Tinaganiza kuti HTTP inali yoyenera kwa ife, chifukwa makina onse omwe alipo tsopano amalumikizana wina ndi mzake potumiza XML kapena JSON pa HTTP.

Timagwiritsa ntchito seva ya HTTP yomangidwa mu Tarantool chifukwa sitiyenera kuletsa magawo a SSL, ndipo ntchito yake ndi yokwanira kwa ife.

Monga ndanenera kale, machitidwe athu onse amakhala mumitundu yosiyanasiyana ya deta, ndipo pazolowera tifunika kubweretsa chinthucho ku chitsanzo chomwe timadzifotokozera tokha. Chilankhulo chinafunika kuti chilole deta kusinthidwa. Tinasankha Lua wofunikira. Timayendetsa khodi yonse yosinthira deta mu sandbox - awa ndi malo otetezeka kupyola kumene khodi yothamanga sapita. Kuti tichite izi, timangoyika nambala yofunikira, ndikupanga malo okhala ndi ntchito zomwe sizingalepheretse kapena kusiya chilichonse.

Pambuyo pa kutembenuka, deta iyenera kufufuzidwa kuti ikutsatira chitsanzo chomwe tikupanga. Tinakambirana kwa nthawi yaitali chomwe chitsanzocho chiyenera kukhala komanso chinenero chomwe tingachigwiritse ntchito pochifotokozera. Tinasankha Apache Avro chifukwa chinenerocho ndi chosavuta ndipo chili ndi chithandizo kuchokera ku Tarantool. Mitundu yatsopano yachitsanzo ndi kachidindo kameneka ikhoza kuchitidwa kangapo patsiku, ngakhale pansi pa katundu kapena popanda, nthawi iliyonse ya tsiku, ndikusintha kusintha mofulumira kwambiri.

Pambuyo potsimikizira, deta iyenera kusungidwa. Timachita izi pogwiritsa ntchito vshard (tili ndi ma geo-dispersed replicas of shards).

Kuphatikiza apo, kutsimikizika kwake ndikuti makina ambiri omwe amatitumizira deta samasamala kaya talandira kapena ayi. Ichi ndichifukwa chake tidakhazikitsa mzere wokonza kuyambira pachiyambi pomwe. Ndi chiyani? Ngati pazifukwa zina chinthu sichimasinthidwa kapena kutsimikiziridwa kwa deta, timatsimikizirabe kulandira, koma nthawi yomweyo sungani chinthucho pamzere wokonza. Ndizosasinthasintha ndipo zili mu malo osungiramo zinthu zazikulu zamabizinesi. Nthawi yomweyo tidalemba mawonekedwe owongolera, ma metric osiyanasiyana ndi zidziwitso. Zotsatira zake, sititaya deta. Ngakhale chinachake chasintha mu gwero, ngati chitsanzo cha deta chasintha, tidzachizindikira nthawi yomweyo ndipo tikhoza kusintha.

Tsopano muyenera kuphunzira mmene akatenge opulumutsidwa deta. Tidasanthula mosamalitsa makina athu ndikuwona kuti mulu wa Java ndi Oracle uli ndi mtundu wina wa ORM womwe umasintha deta kuchoka pa ubale kupita ku zotsutsa. Ndiye bwanji osapereka zinthu nthawi yomweyo ku machitidwe ngati graph? Chifukwa chake tidalandira GraphQL mokondwa, yomwe idakwaniritsa zosowa zathu zonse. Zimakulolani kuti mulandire deta mu mawonekedwe a ma graph ndikutulutsa zomwe mukufuna pakali pano. Mutha kusinthanso API ndi kusinthasintha kwakukulu.

Pafupifupi nthawi yomweyo tinazindikira kuti zomwe tinkatulutsa sizinali zokwanira. Tinapanga ntchito zomwe zingagwirizane ndi zinthu zomwe zili mu chitsanzo - makamaka, magawo owerengeka. Ndiko kuti, timagwirizanitsa ntchito inayake kumunda, yomwe, mwachitsanzo, imawerengera mtengo wamtengo wapatali. Ndipo wogula wakunja yemwe amapempha deta sadziwa nkomwe kuti iyi ndi gawo lowerengedwa.

Anakhazikitsa dongosolo lotsimikizira.

Kenako tinaona kuti pali maudindo angapo amene akuonekera bwino m’chigamulo chathu. Ntchito ndi mtundu wa aggregator wa ntchito. Nthawi zambiri, maudindo amakhala ndi mbiri yogwiritsa ntchito zida zosiyanasiyana:

• T-Connect: imagwira zolumikizira zomwe zikubwera, CPU yochepa, kugwiritsa ntchito kukumbukira pang'ono, kosawerengeka.
• IB-Core: imasintha deta yomwe imalandira kudzera mu protocol ya Tarantool, ndiko kuti, imagwira ntchito ndi matebulo. Komanso sikusunga boma ndipo ndi scalable.
• Kusungirako: amangosunga deta, sagwiritsa ntchito malingaliro aliwonse. Ntchitoyi imagwiritsa ntchito zolumikizira zosavuta. Scalable chifukwa cha vshard.

Ndiko kuti, pogwiritsa ntchito maudindo, tidagawa magawo osiyanasiyana a masango kuchokera kwa wina ndi mnzake, omwe amatha kuwongoleredwa popanda wina ndi mnzake.

Chifukwa chake, tapanga chojambulira chosasinthika cha data komanso mzere wokonza wokhala ndi mawonekedwe a admin. Zolembazo ndizosasinthika kuchokera ku bizinesi: ngati tatsimikiziridwa kuti tidzilembera tokha deta, mosasamala kanthu komwe, ndiye kuti tidzatsimikizira. Ngati sichinatsimikizidwe, ndiye kuti china chake chalakwika ndipo deta iyenera kutumizidwa. Uku ndiye kujambula kosasinthika.

Kuyesa

Kuyambira pachiyambi cha polojekitiyi, tinaganiza kuti tiyesetse kukhazikitsa chitukuko choyendetsedwa ndi mayeso. Timalemba mayeso a mayunitsi ku Lua pogwiritsa ntchito tarantool/tap framework, ndikuyesa kuphatikiza ku Python pogwiritsa ntchito pytest framework. Panthawi imodzimodziyo, timaphatikizapo onse opanga ndi ofufuza polemba mayeso ophatikizana.

Kodi timagwiritsa ntchito bwanji chitukuko choyendetsedwa ndi mayeso?

Ngati tikufuna zina zatsopano, timayesa kulemba mayeso ake poyamba. Tikazindikira cholakwika, timaonetsetsa kuti talemba mayeso kaye, kenako ndikukonza. Poyamba zimakhala zovuta kugwira ntchito motere, pali kusamvetsetsana kwa ogwira ntchito, ngakhale kuwononga: "Tiyeni tikonze mwamsanga tsopano, tichite chinachake chatsopano, ndikuphimba ndi mayesero." Izi zokha "pambuyo pake" pafupifupi sizibwera konse.

Chifukwa chake, muyenera kudzikakamiza kuti mulembe mayeso kaye ndikupempha ena kuti achite. Ndikhulupirireni, chitukuko choyendetsedwa ndi mayeso chimabweretsa phindu ngakhale kwakanthawi kochepa. Mudzaona kuti moyo wanu wakhala wosavuta. Timamva kuti 99% ya code tsopano yaphimbidwa ndi mayesero. Izi zikuwoneka ngati zambiri, koma tilibe vuto lililonse: mayeso amayendera chilichonse.

Komabe, chomwe timakonda kwambiri ndikuyesa katundu; timawona kuti ndikofunikira kwambiri ndikuzichita pafupipafupi.

Ndikuuzani kankhani kakang'ono momwe tidachitira gawo loyamba la kuyezetsa katundu wa mtundu woyamba. Tidayika pulogalamuyo pa laputopu ya wopangayo, tidayatsa katunduyo ndipo tidapeza ndalama 4 pa sekondi iliyonse. Zotsatira zabwino za laputopu. Tidayiyika pa benchi yolemetsa ya ma seva anayi, ofooka kuposa kupanga. Zaperekedwa pang'ono. Timayendetsa, ndipo timapeza zotsatira zoyipa kuposa pa laputopu mu ulusi umodzi. Zodabwitsa.

Tinali achisoni kwambiri. Timayang'ana kuchuluka kwa seva, koma zikuwoneka kuti ndi zopanda pake.

Timatcha okonza, ndipo amatifotokozera ife, anthu ochokera ku dziko la Java, kuti Tarantool ndi ulusi umodzi. Itha kugwiritsidwa ntchito moyenera ndi purosesa imodzi yokhayo yomwe ili ndi katundu. Kenako tidatumiza kuchuluka kokwanira kwa zochitika za Tarantool pa seva iliyonse, tidayatsa katunduyo ndipo talandira kale 14,5 pa sekondi iliyonse.

Ndiroleni ndifotokozenso. Chifukwa cha kugawikana mu maudindo omwe amagwiritsa ntchito zinthu mosiyana, maudindo athu omwe ali ndi udindo wokonza malumikizidwe ndi kusintha kwa deta amangodzaza purosesa, komanso molingana ndi katundu.

Pankhaniyi, kukumbukira kumagwiritsidwa ntchito pokonza zolumikizira zomwe zikubwera ndi zinthu zosakhalitsa.

M'malo mwake, pa ma seva osungira, kuchuluka kwa purosesa kunakula, koma pang'onopang'ono kuposa ma seva omwe amakonza zolumikizira.

Ndipo kugwiritsa ntchito kukumbukira kunakula molingana ndi kuchuluka kwa deta yomwe idakwezedwa.

Ntchito

Kuti tipange zinthu zathu zatsopano ngati nsanja yogwiritsira ntchito, tidapanga gawo loperekera ntchito ndi malaibulale pamenepo.

Ntchito sizinthu zazing'ono chabe zomwe zimagwira ntchito m'magawo ena. Zitha kukhala zazikulu komanso zovuta zomwe zili m'gulu lamagulu, fufuzani zowerengera, yendetsani malingaliro abizinesi ndikubweza mayankho. Timatumizanso schema yautumiki ku GraphQL, ndipo wogula amalandira malo onse olowera ku data, ndikuwunikira mtundu wonsewo. Ndi bwino kwambiri.

Popeza kuti mautumikiwa ali ndi ntchito zambiri, tidaganiza kuti pakhale malaibulale momwe timasunthira ma code omwe amagwiritsidwa ntchito pafupipafupi. Tinawawonjezera ku malo otetezeka, titafufuza kale kuti sizikusokoneza chilichonse kwa ife. Ndipo tsopano titha kugawa madera owonjezera kuti azigwira ntchito ngati malaibulale.

Tinkafuna kukhala ndi nsanja osati yosungirako, komanso makompyuta. Ndipo popeza tinali kale ndi ma replicas ndi shards, tidakhazikitsa mtundu wa kompyuta yogawidwa ndikuyitcha mapu kuchepetsa, chifukwa idakhala yofanana ndi mapu oyambira kuchepetsa.

Machitidwe akale

Sikuti machitidwe athu onse olowa angatiyimbire pa HTTP ndikugwiritsa ntchito GraphQL, ngakhale amathandizira protocol. Chifukwa chake, tapanga njira yomwe imalola kuti deta ibwerezedwe m'makinawa.

Ngati china chake chikusintha kwa ife, zoyambitsa zapadera zimayambitsidwa mu gawo la Kusunga ndipo uthenga womwe uli ndi zosintha umathera pamzere wokonza. Imatumizidwa ku dongosolo lakunja pogwiritsa ntchito gawo losiyana la replicator. Udindowu susunga boma.

Kusintha kwatsopano

Monga mukukumbukira, pamalingaliro abizinesi, tidajambula mosagwirizana. Koma kenako adazindikira kuti sizingakhale zokwanira, chifukwa pali gulu la machitidwe omwe amayenera kulandira nthawi yomweyo yankho la momwe ntchitoyi ikuyendera. Chifukwa chake tidakulitsa GraphQL yathu ndikuwonjezera masinthidwe. Iwo organically amagwirizana mu paradigm alipo ntchito ndi deta. Kwa ife, iyi ndi mfundo imodzi yowerengera ndi kulemba kwa gulu lina la machitidwe.

Tidazindikiranso kuti ntchito zokha sizingakwanire kwa ife, chifukwa pali malipoti olemetsa omwe amafunika kumangidwa kamodzi patsiku, sabata, mwezi. Izi zitha kutenga nthawi yayitali, ndipo malipoti amathanso kuletsa zochitika za Tarantool. Chifukwa chake, tidapanga maudindo osiyanasiyana: scheduler ndi wothamanga. Othamanga samasunga boma. Amagwira ntchito zolemetsa zomwe sitingathe kuziwerengera pouluka. Ndipo gawo la scheduler limayang'anira ndandanda yotsegulira ntchito izi, zomwe zafotokozedwa mu kasinthidwe. Ntchitozo zimasungidwa pamalo omwewo ngati data yabizinesi. Nthawi yoyenera ikafika, wokonza mapulani amatenga ntchitoyo, ndikuipereka kwa wothamanga wina, yemwe amawerengera ndikusunga zotsatira zake.

Sikuti ntchito zonse ziyenera kuchitika pa ndandanda. Malipoti ena amafunikira kuwerengedwa pofunidwa. Izi zikangofika, ntchito imapangidwa mu sandbox ndikutumizidwa kwa wothamanga kuti aphedwe. Patapita nthawi, wogwiritsa ntchito amalandira yankho la asynchronous kuti zonse zawerengedwa ndipo lipoti lakonzeka.

Poyamba, tinkatsatira mfundo yosunga deta yonse, kuisintha osati kuichotsa. Koma m'moyo, nthawi ndi nthawi muyenera kufufuta chinachake, makamaka zina zaiwisi kapena zapakatikati. Kutengera zomwe zidatha, tidapanga njira yotsuka zosungira kuchokera ku data yakale.

Timamvetsetsanso kuti posachedwa zinthu zidzafika pamene sipadzakhala malo okwanira kusunga deta mu kukumbukira, komabe deta iyenera kusungidwa. Pazifukwa izi, posachedwapa tidzapanga disk yosungirako.

Pomaliza

Tinayamba ndi ntchito yokweza deta mu chitsanzo chimodzi ndipo tinakhala miyezi itatu tikuyipanga. Tinali ndi machitidwe asanu ndi limodzi operekera deta. Khodi yonse yosinthira kukhala chitsanzo chimodzi ndi pafupifupi mizere 30 zikwi ku Lua. Ndipo ntchito yambiri idakali m’tsogolo. Nthawi zina magulu oyandikana nawo amakhala opanda chilimbikitso, ndipo pamakhala zinthu zambiri zomwe zimasokoneza ntchitoyo. Ngati mutakumana ndi ntchito yofananayo, chulukitsani nthawi yomwe ikuwoneka ngati yabwino kwa inu kuti muigwiritse ntchito ndi atatu, kapena anayi.

Kumbukiraninso kuti mavuto omwe alipo muzinthu zamabizinesi sangathetsedwe pogwiritsa ntchito DBMS yatsopano, ngakhale yopindulitsa kwambiri. Kodi ndikutanthauza chiyani? Kumayambiriro kwa polojekiti yathu, tidapanga malingaliro pakati pa makasitomala kuti tsopano tibweretsa database yatsopano yofulumira ndipo tidzakhala ndi moyo! Njira zidzapita mofulumira, zonse zikhala bwino. M'malo mwake, ukadaulo suthetsa mavuto omwe njira zamabizinesi zili nazo, chifukwa njira zamabizinesi ndi anthu. Ndipo muyenera kugwira ntchito ndi anthu, osati ukadaulo.

Kukula koyendetsedwa ndi mayeso kumatha kukhala kowawa komanso kuwononga nthawi koyambirira. Koma zotsatira zake zabwino zidzawonekera ngakhale pakapita nthawi, pamene simukusowa kuchita kalikonse kuti muyesere kuyambiranso.

Ndikofunikira kwambiri kuyezetsa katundu pazigawo zonse za chitukuko. Mukangowona zolakwika zina muzomangamanga, zimakhala zosavuta kuzikonza, zomwe zingakupulumutseni nthawi yambiri m'tsogolomu.

Palibe cholakwika ndi Lua. Aliyense atha kuphunzira kulembamo: Wopanga Java, wopanga JavaScript, wopanga Python, kutsogolo kapena kumbuyo. Ngakhale akatswiri athu amalembapo.

Tikamalankhula zakuti tilibe SQL, zimawopseza anthu. "Mumapeza bwanji deta popanda SQL? Kodi izi zingatheke? Ndithudi. Mu dongosolo la kalasi ya OLTP, SQL siyofunika. Pali njira ina mumtundu wa chilankhulo china chomwe chimakubwezerani nthawi yomweyo kumawonedwe otengera zolemba. Mwachitsanzo, GraphQL. Ndipo pali njira ina mu mawonekedwe a computing anagawira.

Ngati mumvetsetsa kuti muyenera kukulitsa, ndiye pangani yankho lanu pa Tarantool m'njira yoti lizitha kuyenda limodzi pamitundu yambiri ya Tarantool. Ngati simuchita izi, zidzakhala zovuta komanso zopweteka pambuyo pake, popeza Tarantool imatha kugwiritsa ntchito purosesa imodzi yokha.

Source: www.habr.com