Tarantool sizga o'ta tezkor ma'lumotlar bazasi va ular bilan ishlash uchun dasturni birlashtirish imkonini beradi. Buni qilish qanchalik oson

Besh yil oldin men Tarantool bilan ishlashga harakat qildim, lekin keyin bu men uchun ishlamadi. Ammo yaqinda men vebinar o'tkazdim, u erda Hadoop va MapReduce qanday ishlashi haqida gapirdim. U erda ular menga savol berishdi: "Nega bu vazifa uchun Tarantool-dan foydalanmaslik kerak?"

Qiziqish tufayli men unga qaytishga, so'nggi versiyani sinab ko'rishga qaror qildim - va bu safar menga loyiha juda yoqdi. Endi men sizga Tarantool-da oddiy dasturni qanday yozishni, uni yuklashni va uning ishlashini tekshirishni ko'rsataman va siz hamma narsa qanchalik oson va ajoyib ekanligini ko'rasiz.

Tarantool nima

Tarantool o'zini juda tezkor ma'lumotlar bazasi sifatida ko'rsatadi. U erda siz xohlagan ma'lumotlarni qo'yishingiz mumkin. Bundan tashqari, ularni takrorlang, shard - ya'ni katta hajmdagi ma'lumotlarni bir nechta serverlar bo'ylab taqsimlang va ulardan olingan natijalarni birlashtiring - xatolarga chidamli master-master ulanishlarini yarating.

Ikkinchidan, bu dastur serveri. Unga ilovalaringizni yozishingiz, ma'lumotlar bilan ishlashingiz mumkin, masalan, ma'lum qoidalarga muvofiq fonda eski yozuvlarni o'chirishingiz mumkin. Siz to'g'ridan-to'g'ri Tarantula-da ma'lumotlar bilan ishlaydigan Http serverini yozishingiz mumkin: ularning miqdorini bering, u erda yangi ma'lumotlarni yozing va barchasini masterga kamaytiring.

Men yigitlar 300 qatordan iborat xabarlar navbatini qanday tuzganligi haqidagi maqolani o'qidim, bu shunchaki yorilib ketadi va shoshiladi - ular soniyasiga 20 000 ta xabarni tashkil qiladi. Bu erda siz haqiqatan ham o'girilib, juda katta hajmdagi dastur yozishingiz mumkin va u PostgreS-dagi kabi saqlash bo'lmaydi.

Men ushbu maqolada ushbu serverga o'xshash narsani tasvirlashga harakat qilaman, faqat oddiy.

sozlama

Sinov uchun men uchta standart virtual mashinani ishga tushirdim - 20 GB qattiq disk, Ubuntu 18.04. 2 virtual protsessor va 4 gigabayt xotira.

Biz Tarantool-ni o'rnatamiz - bash skriptini ishga tushiramiz yoki ombor qo'shamiz va Tarantool-ni o'rnatamiz. Skriptga havola - (curl -L https://tarantool.io/installer.sh | VER=2.4 sudo -E bash). Bizda quyidagi kabi buyruqlar mavjud:

tarantoolctl - Tarantula misollarini boshqarish uchun asosiy buyruq.
/etc/tarantool - bu erda butun konfiguratsiya.
var/log/tarantool - mana jurnallar.
var/lib/tarantool — maʼlumotlar shu yerda yotadi, keyin esa instansiyalarga boʻlinadi.

Instance-mavjud va instance-enable papkalari mavjud - u ishga tushiriladigan narsalarni o'z ichiga oladi - lua kodi bo'lgan namuna konfiguratsiya fayli, u qaysi portlarni tinglashini, qaysi xotira mavjudligini, Vinil dvigatel sozlamalarini, ishga tushirilganda ishlaydigan kodni tavsiflaydi. serverlar, sharding, navbatlar, eskirgan ma'lumotlarni o'chirish va boshqalar.

Misollar PostgreSdagi kabi ishlaydi. Misol uchun, siz turli portlarda osilgan ma'lumotlar bazasining bir nechta nusxalarini ishga tushirishni xohlaysiz. Ma'lum bo'lishicha, bir serverda bir nechta ma'lumotlar bazasi namunalari ishga tushiriladi, ular turli portlarda osilgan. Ular butunlay boshqacha sozlamalarga ega bo'lishi mumkin - bir misol bitta mantiqni amalga oshiradi, ikkinchisi - boshqa.

Namuna boshqaruvi

Bizda Tarantula misollarini boshqarish imkonini beruvchi tarantoolctl buyrug'i mavjud. Misol uchun, tarantoolctl tekshiruvi misoli konfiguratsiya faylini tekshiradi va aytadi - agar u erda sintaksis xatosi bo'lmasa, fayl yaxshi.

Siz misol holatini ko'rishingiz mumkin - tarantoolctl status misoli. Xuddi shu tarzda siz boshlash, to'xtatish, qayta ishga tushirishni amalga oshirishingiz mumkin.

Namuna ishga tushgandan so'ng, unga ikki usulda ulanishingiz mumkin.

1. Ma'muriy konsol

Odatiy bo'lib, Tarantool rozetkani ochadi, Tarantoolni boshqarish uchun oddiy ASCII matni yuboriladi. Konsolga ulanish har doim administrator foydalanuvchisi ostida amalga oshiriladi, autentifikatsiya yo'q, shuning uchun Tarantulani boshqarish uchun konsol portini tashqariga chiqarishning hojati yo'q.

Ushbu usul yordamida ulanish uchun siz Tarantoolctl misol nomini kiritishingiz kerak. Buyruq konsolni ishga tushiradi va administrator foydalanuvchi sifatida ulanadi. Konsol portini hech qachon tashqariga qo'ymang - uni birlik rozetkasi sifatida qoldirgan ma'qul. Shunda faqat rozetkaga yozish huquqiga ega bo'lganlar Tarantulaga ulanishi mumkin bo'ladi.

Bu usul ma'muriy ishlar uchun kerak. Ma'lumotlar bilan ishlash uchun ikkinchi usul - ikkilik protokoldan foydalaning.

2. Muayyan portga ulanish uchun ikkilik protokoldan foydalanish

Konfiguratsiyada tashqi aloqalar uchun portni ochadigan tinglash direktivasi mavjud. Bu port ikkilik protokol bilan ishlatiladi va u yerda autentifikatsiya yoqilgan.

Ushbu ulanish uchun port raqamiga tarantoolctl connect ishlatiladi. Undan foydalanib, siz uzoq serverlarga ulanishingiz, autentifikatsiyadan foydalanishingiz va turli xil kirish huquqlarini berishingiz mumkin.

Ma'lumotlarni yozib olish va quti moduli

Tarantool ham ma'lumotlar bazasi, ham dastur serveri bo'lgani uchun u turli modullarga ega. Bizni box moduli qiziqtiradi - u ma'lumotlar bilan ishlashni amalga oshiradi. Box-ga biror narsa yozganingizda, Tarantool ma'lumotlarni diskka yozadi, xotirada saqlaydi yoki u bilan boshqa biror narsa qiladi.

rekord

Misol uchun, biz box moduliga o'tamiz va box.once funksiyasini chaqiramiz. Bu server ishga tushirilganda Tarantool kodimizni ishga tushirishga majbur qiladi. Biz ma'lumotlarimiz saqlanadigan joy yaratamiz.

local function bootstrap()
    local space = box.schema.create_space('example')
    space:create_index('primary')
    box.schema.user.grant('guest', 'read,write,execute', 'universe')

    -- Keep things safe by default
    --  box.schema.user.create('example', { password = 'secret' })
    --  box.schema.user.grant('example', 'replication')
    --  box.schema.user.grant('example', 'read,write,execute', 'space', 'example')
end

Shundan so'ng biz ma'lumotlarni qidirishimiz mumkin bo'lgan asosiy indeksni - birlamchi - yaratamiz. Odatiy bo'lib, agar siz hech qanday parametrni belgilamasangiz, har bir yozuvdagi birinchi maydon asosiy indeks uchun ishlatiladi.

Keyin biz mehmon foydalanuvchiga grant beramiz, uning ostida biz ikkilik protokol orqali ulanamiz. Biz butun misol bo'ylab o'qish, yozish va bajarishga ruxsat beramiz.

An'anaviy ma'lumotlar bazalari bilan taqqoslaganda, bu erda hamma narsa juda oddiy. Bizda bo'sh joy bor - bu bizning ma'lumotlarimiz oddiygina saqlanadigan maydon. Har bir yozuv kortej deb ataladi. U MessagePack-da qadoqlangan. Bu juda zo'r format - bu ikkilik va kamroq joy egallaydi - 18 baytga nisbatan 27 bayt.

U bilan ishlash juda qulay. Deyarli har bir satr, har bir ma'lumot yozuvi butunlay boshqa ustunlarga ega bo'lishi mumkin.

Box.space buyrug'i yordamida barcha bo'shliqlarni ko'rishimiz mumkin. Muayyan misolni tanlash uchun box.space misolini yozing va u haqida to'liq ma'lumot oling.

Tarantool ikkita o'rnatilgan dvigatelga ega: Xotira va Vinil. Xotira barcha ma'lumotlarni xotirada saqlaydi. Shuning uchun, hamma narsa oddiy va tez ishlaydi. Ma'lumotlar diskka tashlanadi, shuningdek, oldindan yozish jurnali mexanizmi mavjud, shuning uchun server ishlamay qolsa, biz hech narsani yo'qotmaymiz.

Vinil diskdagi ma'lumotlarni bizga ko'proq tanish bo'lgan shaklda saqlaydi - ya'ni siz bizdagi xotiradan ko'proq ma'lumotlarni saqlashingiz mumkin va Tarantula uni diskdan o'qiydi.

Hozircha biz Xotiradan foydalanamiz.

unix/:/var/run/tarantool/example.control> box.space.example
---
- engine: memtx
  before_replace: 'function: 0x41eb02c8'
  on_replace: 'function: 0x41eb0568'
  ck_constraint: []
  field_count: 0
  temporary: false
  index:
    0: &0
      unique: true
      parts:
      - type: unsigned
        is_nullable: false
        fieldno: 1
      id: 0
      space_id: 512
      type: TREE
      name: primary
    primary: *0
  is_local: false
  enabled: true
  name: example
  id: 512
...

unix/:/var/run/tarantool/example.control>

Indeks:

Har qanday makon uchun asosiy indeks yaratilishi kerak, chunki usiz hech narsa ishlamaydi. Har qanday ma'lumotlar bazasida bo'lgani kabi, biz birinchi maydonni - yozuv identifikatorini yaratamiz.

Ehtiyot qismlar:

Bu erda biz indeksimiz nimadan iboratligini ko'rsatamiz. U bir qismdan iborat - biz ishlatadigan birinchi maydon unsigned turi - musbat butun son. Hujjatlardan eslaganimdek, bo'lishi mumkin bo'lgan maksimal raqam 18 kvintillion. Bu juda ko'p.

Keyin insert buyrug'i yordamida ma'lumotlarni kiritishimiz mumkin.

unix/:/var/run/tarantool/example.control> box.space.example:insert{1, 'test1', 'test2'}
---
- [1, 'test1', 'test2']
...

unix/:/var/run/tarantool/example.control> box.space.example:insert{2, 'test2', 'test3', 'test4'}
---
- [2, 'test2', 'test3', 'test4']
...

unix/:/var/run/tarantool/example.control> box.space.example:insert{3, 'test3'}
---
- [3, 'test3']
...

unix/:/var/run/tarantool/example.control> box.space.example:insert{4, 'test4'}
---
- [4, 'test4']
...

unix/:/var/run/tarantool/example.control>

Birinchi maydon asosiy kalit sifatida ishlatiladi, shuning uchun u noyob bo'lishi kerak. Biz ustunlar soni bilan cheklanmaymiz, shuning uchun biz u erga xohlagancha ma'lumotlarni kiritishimiz mumkin. Ular yuqorida tavsiflangan MessagePack formatida ko'rsatilgan.

Ma'lumotlar chiqishi

Keyin tanlash buyrug'i yordamida ma'lumotlarni ko'rsatishimiz mumkin.

{1} tugmasi bilan Box.example.select kerakli yozuvni ko'rsatadi. Agar biz kalitni tushirsak, bizda mavjud bo'lgan barcha yozuvlarni ko'ramiz. Ularning barchasi ustunlar sonida farq qiladi, lekin bu erda, printsipial jihatdan, ustunlar tushunchasi yo'q - maydon raqamlari mavjud.

Mutlaqo har qanday miqdordagi ma'lumotlar bo'lishi mumkin. Va, masalan, biz ularni ikkinchi maydon bo'yicha qidirishimiz kerak. Buning uchun biz yangi ikkilamchi indeks yaratamiz.

box.space.example:create_index( ‘secondary’, { type = ‘TREE’, unique = false, parts = {{field = 2, type =’string’} }}) 

Biz Create_index buyrug'idan foydalanamiz.
Keling, buni ikkinchi darajali deb ataymiz.

Shundan so'ng siz parametrlarni belgilashingiz kerak. Indeks turi - TREE. Bu noyob bo'lmasligi mumkin, shuning uchun Unique = false kiriting.

Keyin indeksimiz qaysi qismlardan iboratligini ko'rsatamiz. Maydon - biz indeksni bog'laydigan maydonning raqami va satr turini belgilang. Va shunday qilib yaratilgan.

unix/:/var/run/tarantool/example.control> box.space.example:create_index('secondary', { type = 'TREE', unique = false, parts = {{field = 2, type = 'string'}}})
---
- unique: false
  parts:
  - type: string
    is_nullable: false
    fieldno: 2
  id: 1
  space_id: 512
  type: TREE
  name: secondary
...

unix/:/var/run/tarantool/example.control>

Endi biz buni shunday deb atashimiz mumkin:

unix/:/var/run/tarantool/example.control> box.space.example.index.secondary:select('test1')
---
- - [1, 'test1', 'test2']
...

Saqlash

Agar biz misolni qayta ishga tushirsak va ma'lumotlarni qayta chaqirishga harakat qilsak, u erda yo'qligini ko'ramiz - hamma narsa bo'sh. Buning sababi, Tarantool nazorat nuqtalarini yaratadi va ma'lumotlarni diskka saqlaydi, lekin agar biz keyingi saqlashgacha ishlashni to'xtatsak, biz barcha operatsiyalarni yo'qotamiz - chunki biz, masalan, ikki soat oldin bo'lgan oxirgi nazorat nuqtasidan tiklanamiz.

Har soniyani tejash ham ishlamaydi, chunki doimiy ravishda 20 Gb ni diskka tushirish yaxshi fikr emas.

Shu maqsadda oldindan yozish jurnali kontseptsiyasi ixtiro qilindi va amalga oshirildi. Uning yordami bilan ma'lumotlarning har bir o'zgarishi uchun kichik oldindan yozish jurnali faylida yozuv yaratiladi.

Tekshirish punktigacha bo'lgan har bir kirish ularda saqlanadi. Ushbu fayllar uchun biz hajmini o'rnatdik - masalan, 64 MB. U to'lganida, yozuv ikkinchi faylga o'tishni boshlaydi. Va qayta ishga tushirilgandan so'ng, Tarantool oxirgi nazorat punktidan tiklanadi va keyin to'xtaguncha barcha keyingi tranzaktsiyalarni aylantiradi.

Bunday yozishni amalga oshirish uchun siz box.cfg sozlamalarida (example.lua faylida) variantni belgilashingiz kerak:

wal_mode = “write”;

ma'lumotlardan foydalanish

Biz hozir yozgan narsalar bilan siz ma'lumotlarni saqlash uchun Tarantuladan foydalanishingiz mumkin va u ma'lumotlar bazasi sifatida juda tez ishlaydi. Endi esa pirojnoe ustidagi muzlash - bularning barchasi bilan nima qilishingiz mumkin.

Ariza yozish

Masalan, Tarantula uchun quyidagi ariza yozamiz

Spoyler ostidagi ilovaga qarang

box.cfg {
    listen = '0.0.0.0:3301';
    io_collect_interval = nil;
    readahead = 16320;
    memtx_memory = 128 * 1024 * 1024; -- 128Mb
    memtx_min_tuple_size = 16;
    memtx_max_tuple_size = 128 * 1024 * 1024; -- 128Mb
    vinyl_memory = 128 * 1024 * 1024; -- 128Mb
    vinyl_cache = 128 * 1024 * 1024; -- 128Mb
    vinyl_max_tuple_size = 128 * 1024 * 1024; -- 128Mb
    vinyl_write_threads = 2;
    wal_mode = "write";
    wal_max_size = 256 * 1024 * 1024;
    checkpoint_interval = 60 * 60; -- one hour
    checkpoint_count = 6;
    force_recovery = true;
    log_level = 5;
    log_nonblock = false;
    too_long_threshold = 0.5;
    read_only   = false
}

local function bootstrap()
    local space = box.schema.create_space('example')
    space:create_index('primary')

    box.schema.user.create('example', { password = 'secret' })
    box.schema.user.grant('example', 'read,write,execute', 'space', 'example')

    box.schema.user.create('repl', { password = 'replication' })
    box.schema.user.grant('repl', 'replication')
end

-- for first run create a space and add set up grants
box.once('replica', bootstrap)

-- enabling console access
console = require('console')
console.listen('127.0.0.1:3302')

-- http config
local charset = {}  do -- [0-9a-zA-Z]
    for c = 48, 57  do table.insert(charset, string.char(c)) end
    for c = 65, 90  do table.insert(charset, string.char(c)) end
    for c = 97, 122 do table.insert(charset, string.char(c)) end
end

local function randomString(length)
    if not length or length <= 0 then return '' end
    math.randomseed(os.clock()^5)
    return randomString(length - 1) .. charset[math.random(1, #charset)]
end

local http_router = require('http.router')
local http_server = require('http.server')
local json = require('json')

local httpd = http_server.new('0.0.0.0', 8080, {
    log_requests = true,
    log_errors = true
})

local router = http_router.new()

local function get_count()
 local cnt = box.space.example:len()
 return cnt
end

router:route({method = 'GET', path = '/count'}, function()
    return {status = 200, body = json.encode({count = get_count()})}
end)

router:route({method = 'GET', path = '/token'}, function()
    local token = randomString(32)
    local last = box.space.example:len()
    box.space.example:insert{ last + 1, token }
    return {status = 200, body = json.encode({token = token})}
end)

prometheus = require('prometheus')

fiber = require('fiber')
tokens_count = prometheus.gauge("tarantool_tokens_count",
                              "API Tokens Count")

function monitor_tokens_count()
  while true do
    tokens_count:set(get_count())
    fiber.sleep(5)
  end
end
fiber.create(monitor_tokens_count)

router:route( { method = 'GET', path = '/metrics' }, prometheus.collect_http)

httpd:set_router(router)
httpd:start()

Biz lua-da belgilarni belgilaydigan ba'zi jadvalni e'lon qilamiz. Ushbu plastinka tasodifiy qatorni yaratish uchun kerak.

local charset = {}  do -- [0-9a-zA-Z]
    for c = 48, 57  do table.insert(charset, string.char(c)) end
    for c = 65, 90  do table.insert(charset, string.char(c)) end
    for c = 97, 122 do table.insert(charset, string.char(c)) end
end

Shundan so'ng biz funksiyani e'lon qilamiz - randomString va uzunlik qiymatini qavs ichida beramiz.

local function randomString(length)
    if not length or length <= 0 then return '' end
    math.randomseed(os.clock()^5)
    return randomString(length - 1) .. charset[math.random(1, #charset)]
end

Keyin http router va http serverni Tarantula serverimiz JSON ga ulaymiz, uni mijozga yuboramiz.

local http_router = require('http.router')
local http_server = require('http.server')
local json = require('json')

Shundan so'ng biz barcha http-server interfeyslarida 8080 portdan boshlaymiz, u barcha so'rovlar va xatolarni qayd qiladi.

local httpd = http_server.new('0.0.0.0', 8080, {
    log_requests = true,
    log_errors = true
})

Keyinchalik, marshrutni e'lon qilamiz, shunda GET usuli bilan so'rov 8080 /count portiga tushsa, biz funktsiyani bir qatordan chaqiramiz. U holatni qaytaradi - 200, 404, 403 yoki biz belgilagan boshqa.

router:route({method = 'GET', path = '/count'}, function()
    return {status = 200, body = json.encode({count = get_count()})}
end)

Tanada biz json.encode ni qaytaramiz, unda biz chaqirilgan va ma'lumotlar bazasidagi yozuvlar sonini ko'rsatadigan count va getcountni ko'rsatamiz.

Ikkinchi usul

router:route({method = 'GET', path = '/token'}, function() 
    local token = randomString(32) 
    local last = box.space.example:len() 
    box.space.example:insert{ last + 1, token } 
    return {status = 200, body = json.encode({token = token})}
end)

Qaerda qatorda router:route({metod = 'GET', path = '/token'}, function() funksiyani chaqiramiz va token hosil qilamiz.

Chiziq mahalliy token = randomString (32) 32 ta belgidan iborat tasodifiy qatordir.
Mos ravishda mahalliy oxirgi = box.space.example:len() biz oxirgi elementni olib tashlaymiz.
Va qatorda box.space.example:insert{ oxirgi + 1, token } biz ma'lumotlarni ma'lumotlar bazasiga yozamiz, ya'ni identifikatorni shunchaki 1 ga oshiramiz. Tarantulada buning uchun ketma-ketliklar mavjud.

Biz u erda belgini yozamiz.

Shunday qilib, biz arizani bitta faylga yozdik. Siz u erda ma'lumotlarni to'g'ridan-to'g'ri manipulyatsiya qilishingiz mumkin va quti moduli siz uchun barcha iflos ishlarni bajaradi.

U http ni tinglaydi va ma'lumotlar bilan ishlaydi, hamma narsa bitta misolda - ham ilova, ham ma'lumotlar. Shuning uchun hamma narsa juda tez sodir bo'ladi.

Boshlash uchun http modulini o'rnatamiz:

Buni qanday qilamiz, spoyler ostiga qarang

root@test2:/# tarantoolctl rocks install http
Installing http://rocks.tarantool.org/http-scm-1.src.rock
Missing dependencies for http scm-1:
   checks >= 3.0.1 (not installed)

http scm-1 depends on checks >= 3.0.1 (not installed)
Installing http://rocks.tarantool.org/checks-3.0.1-1.rockspec

Cloning into 'checks'...
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 28 (delta 1), reused 16 (delta 1), pack-reused 0
Receiving objects: 100% (28/28), 12.69 KiB | 12.69 MiB/s, done.
Resolving deltas: 100% (1/1), done.
Note: checking out '580388773ef11085015b5a06fe52d61acf16b201'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

No existing manifest. Attempting to rebuild...
checks 3.0.1-1 is now installed in /.rocks (license: BSD)

-- The C compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found TARANTOOL: /usr/include (found version "2.4.2-80-g18f2bc82d")
-- Tarantool LUADIR is /.rocks/share/tarantool/rocks/http/scm-1/lua
-- Tarantool LIBDIR is /.rocks/share/tarantool/rocks/http/scm-1/lib
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    version


-- Build files have been written to: /tmp/luarocks_http-scm-1-V4P9SM/http/build.luarocks
Scanning dependencies of target httpd
[ 50%] Building C object http/CMakeFiles/httpd.dir/lib.c.o
In file included from /tmp/luarocks_http-scm-1-V4P9SM/http/http/lib.c:32:0:
/tmp/luarocks_http-scm-1-V4P9SM/http/http/lib.c: In function ‘tpl_term’:
/usr/include/tarantool/lauxlib.h:144:15: warning: this statement may fall through [-Wimplicit-fallthrough=]
    (*(B)->p++ = (char)(c)))
    ~~~~~~~~~~~^~~~~~~~~~~~
/tmp/luarocks_http-scm-1-V4P9SM/http/http/lib.c:62:7: note: in expansion of macro ‘luaL_addchar’
       luaL_addchar(b, '\');
       ^~~~~~~~~~~~
/tmp/luarocks_http-scm-1-V4P9SM/http/http/lib.c:63:6: note: here
      default:
      ^~~~~~~
In file included from /tmp/luarocks_http-scm-1-V4P9SM/http/http/lib.c:39:0:
/tmp/luarocks_http-scm-1-V4P9SM/http/http/tpleval.h: In function ‘tpe_parse’:
/tmp/luarocks_http-scm-1-V4P9SM/http/http/tpleval.h:147:9: warning: this statement may fall through [-Wimplicit-fallthrough=]
    type = TPE_TEXT;
    ~~~~~^~~~~~~~~~
/tmp/luarocks_http-scm-1-V4P9SM/http/http/tpleval.h:149:3: note: here
   case TPE_LINECODE:
   ^~~~
In file included from /tmp/luarocks_http-scm-1-V4P9SM/http/http/lib.c:40:0:
/tmp/luarocks_http-scm-1-V4P9SM/http/http/httpfast.h: In function ‘httpfast_parse’:
/tmp/luarocks_http-scm-1-V4P9SM/http/http/httpfast.h:372:22: warning: this statement may fall through [-Wimplicit-fallthrough=]
                 code = 0;
                 ~~~~~^~~
/tmp/luarocks_http-scm-1-V4P9SM/http/http/httpfast.h:374:13: note: here
             case status:
             ^~~~
/tmp/luarocks_http-scm-1-V4P9SM/http/http/httpfast.h:393:23: warning: this statement may fall through [-Wimplicit-fallthrough=]
                 state = message;
                 ~~~~~~^~~~~~~~~
/tmp/luarocks_http-scm-1-V4P9SM/http/http/httpfast.h:395:13: note: here
             case message:
             ^~~~
[100%] Linking C shared library lib.so
[100%] Built target httpd
[100%] Built target httpd
Install the project...
-- Install configuration: "Debug"
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/VERSION.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lib/http/lib.so
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/server/init.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/server/tsgi_adapter.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/nginx_server/init.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/router/init.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/router/fs.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/router/matching.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/router/middleware.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/router/request.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/router/response.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/tsgi.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/utils.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/mime_types.lua
-- Installing: /.rocks/share/tarantool/rocks/http/scm-1/lua/http/codes.lua
http scm-1 is now installed in /.rocks (license: BSD)

root@test2:/#

Bizga yugurish uchun prometey ham kerak:

root@test2:/# tarantoolctl rocks install prometheus
Installing http://rocks.tarantool.org/prometheus-scm-1.rockspec

Cloning into 'prometheus'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 19 (delta 2), reused 5 (delta 0), pack-reused 0
Receiving objects: 100% (19/19), 10.73 KiB | 10.73 MiB/s, done.
Resolving deltas: 100% (2/2), done.
prometheus scm-1 is now installed in /.rocks (license: BSD)

root@test2:/#

Biz modullarni ishga tushiramiz va ularga kirishimiz mumkin

root@test2:/# curl -D - -s http://127.0.0.1:8080/token
HTTP/1.1 200 Ok
Content-length: 44
Server: Tarantool http (tarantool v2.4.2-80-g18f2bc82d)
Connection: keep-alive

{"token":"e2tPq9l5Z3QZrewRf6uuoJUl3lJgSLOI"}

root@test2:/# curl -D - -s http://127.0.0.1:8080/token
HTTP/1.1 200 Ok
Content-length: 44
Server: Tarantool http (tarantool v2.4.2-80-g18f2bc82d)
Connection: keep-alive

{"token":"fR5aCA84gj9eZI3gJcV0LEDl9XZAG2Iu"}

root@test2:/# curl -D - -s http://127.0.0.1:8080/count
HTTP/1.1 200 Ok
Content-length: 11
Server: Tarantool http (tarantool v2.4.2-80-g18f2bc82d)
Connection: keep-alive

{"count":2}root@test2:/#

/count bizga 200 holatini beradi.
/token token chiqaradi va bu tokenni maʼlumotlar bazasiga yozadi.

Tezlikni sinab ko'rish

Keling, 50 000 ta so'rov uchun benchmarkni o'tkazaylik. 500 ta raqobatdosh so'rovlar bo'ladi.

root@test2:/# ab -c 500 -n 50000 http://127.0.0.1:8080/token
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Completed 30000 requests
Completed 35000 requests
Completed 40000 requests
Completed 45000 requests
Completed 50000 requests
Finished 50000 requests


Server Software:        Tarantool
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /token
Document Length:        44 bytes

Concurrency Level:      500
Time taken for tests:   14.578 seconds
Complete requests:      50000
Failed requests:        0
Total transferred:      7950000 bytes
HTML transferred:       2200000 bytes
Requests per second:    3429.87 [#/sec] (mean)
Time per request:       145.778 [ms] (mean)
Time per request:       0.292 [ms] (mean, across all concurrent requests)
Transfer rate:          532.57 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   10 103.2      0    3048
Processing:    12   69 685.1     15   13538
Waiting:       12   69 685.1     15   13538
Total:         12   78 768.2     15   14573

Percentage of the requests served within a certain time (ms)
  50%     15
  66%     15
  75%     16
  80%     16
  90%     16
  95%     16
  98%     21
  99%     42
 100%  14573 (longest request)
root@test2:/#

Tokenlar chiqariladi. Va biz doimiy ravishda ma'lumotlarni yozib boramiz. So‘rovlarning 99 foizi 42 millisekundda ko‘rib chiqildi. Shunga ko'ra, bizda 3500 yadroli va 2 gigabayt xotiraga ega bo'lgan kichik mashinada soniyada taxminan 4 so'rov bor.

Shuningdek, siz 50000 XNUMX ga yaqin tokenni tanlashingiz va uning qiymatini ko'rishingiz mumkin.

Siz nafaqat http dan foydalanishingiz mumkin, balki ma'lumotlaringizni qayta ishlaydigan fon funktsiyalarini ham ishga tushirishingiz mumkin. Bundan tashqari, turli xil triggerlar mavjud. Masalan, siz yangilanishlar bo'yicha funktsiyalarni chaqirishingiz, biror narsani tekshirishingiz mumkin - to'qnashuvlarni tuzatish.

Siz skript ilovalarini to'g'ridan-to'g'ri ma'lumotlar bazasi serverida yozishingiz mumkin va hech narsa bilan cheklanmaysiz, har qanday modullarni ulashingiz va har qanday mantiqni amalga oshirishingiz mumkin.

Ilova serveri tashqi serverlarga kirishi, ma'lumotlarni olishi va ma'lumotlar bazasiga qo'shishi mumkin. Ushbu ma'lumotlar bazasidagi ma'lumotlar boshqa ilovalar tomonidan qo'llaniladi.

Tarantula buni o'zi qiladi va siz alohida ariza yozishingiz shart emas.

Xulosa

Bu katta ishning faqat birinchi qismi. Ikkinchisi juda tez orada Mail.ru Group blogida e'lon qilinadi va biz ushbu materialga albatta havolani qo'shamiz.

Agar biz ushbu narsalarni onlayn quradigan tadbirlarda qatnashishni va real vaqtda savollar berishni xohlasangiz, sozlang REBRAIN tomonidan DevOps kanali.

Agar siz bulutga o'tmoqchi bo'lsangiz yoki infratuzilmangiz haqida savollaringiz bo'lsa, iltimos qoldiring.

PS Bizda oyiga 2 ta bepul audit bor, ehtimol sizning loyihangiz ulardan biri bo'ladi.

Manba: www.habr.com