Qee lub sij hawm dhau los peb tau ntsib teeb meem ntawm kev tu tuples hauv qhov chaw . Kev ntxuav yuav tsum tau pib tsis yog thaum tarantool twb khiav tawm ntawm lub cim xeeb, tab sis ua ntej thiab ntawm qee zaus. Rau txoj haujlwm no, tarantool muaj cov qauv sau hauv Lua hu ua . Tom qab siv cov qauv no rau lub sijhawm luv luv, peb pom tau tias nws tsis haum rau peb: vim yog kev tu cov ntaub ntawv loj, Lua dai hauv GC. Yog li ntawd, peb xav txog kev tsim peb tus kheej capped expiration module, vam tias cov cai sau ua ib hom lus programming yuav daws tau peb cov teeb meem nyob rau hauv txoj kev zoo tshaj plaws.
Ib qho piv txwv zoo rau peb yog tarantool module hu ua . Txoj kev siv nyob rau hauv nws yog raws li qhov tseeb hais tias ib tug cais teb yog tsim nyob rau hauv qhov chaw, uas qhia lub neej ntawm tuple, nyob rau hauv lwm yam lus, ttl. Lub module hauv keeb kwm yav dhau scans qhov chaw, sib piv TTL nrog lub sijhawm tam sim no thiab txiav txim siab seb puas yuav rho tawm tuple lossis tsis. Lub memcached module code yog yooj yim thiab elegant, tab sis dhau lawm. Ua ntej, nws tsis coj mus rau hauv tus account hom kev ntsuas uas tau nkag mus thiab tshem tawm. Thib ob, ntawm txhua qhov hla tag nrho cov tuples raug txheeb xyuas, tus naj npawb ntawm cov tuaj yeem loj heev. Thiab yog tias nyob rau hauv lub sij hawm tas sij hawm qhov teeb meem thawj zaug tau raug daws (qhov ntsuas tsob ntoo tau muab cais rau hauv ib chav cais), ces qhov thib ob tseem tsis tau txais kev saib xyuas. Peb lub ntsiab lus no tau txiav txim siab qhov kev xaiv hauv kev pom zoo ntawm kev sau kuv tus kheej code.
piav qhia
Cov ntaub ntawv rau tarantool muaj qhov zoo heev hais txog yuav ua li cas sau koj cov txheej txheem khaws cia hauv C. Ua ntej tshaj plaws, kuv xav kom koj paub koj tus kheej nrog nws txhawm rau nkag siab cov lus ntxig nrog cov lus txib thiab cov lej uas yuav tshwm sim hauv qab no. Nws kuj tsim nyog them nqi rau rau cov khoom uas muaj thaum sau koj tus kheej capped module, uas yog , , ΠΈ .
Cia peb pib los ntawm qhov deb thiab saib seb lub capped expirationd module zoo li los ntawm sab nraud:
fiber = require('fiber')
net_box = require('net.box')
box.cfg{listen = 3300}
box.schema.func.create('libcapped-expirationd.start', {language = 'C'})
box.schema.user.grant('guest', 'execute', 'function', 'libcapped-expirationd.start')
box.schema.func.create('libcapped-expirationd.kill', {language = 'C'})
box.schema.user.grant('guest', 'execute', 'function', 'libcapped-expirationd.kill')
box.schema.space.create('tester')
box.space.tester:create_index('primary', {unique = true, parts = {1, 'unsigned'}})
capped_connection = net_box:new(3300)Rau qhov yooj yim, peb tso tawm tarantool hauv cov npe uas peb lub tsev qiv ntawv libcapped-expirationd.so nyob. Ob txoj haujlwm raug xa tawm los ntawm lub tsev qiv ntawv: pib thiab tua. Thawj kauj ruam yog ua kom cov haujlwm no muaj los ntawm Lua siv box.schema.func.create thiab box.schema.user.grant. Tom qab ntawd tsim ib qho chaw uas nws tuples yuav tsuas muaj peb lub teb: thawj zaug yog tus cim tshwj xeeb, qhov thib ob yog email, thiab qhov thib peb yog lub neej ntawm tuple. Peb tsim ib tsob ntoo ntsuas saum toj ntawm thawj daim teb thiab hu nws thawj. Tom ntej no peb tau txais cov khoom txuas mus rau peb lub tsev qiv ntawv ib txwm muaj.
Tom qab kev npaj ua haujlwm, ua haujlwm pib:
capped_connection:call('libcapped-expirationd.start', {'non-indexed', box.space.tester.id, box.space.tester.index.primary, box.space.tester.index.primary, 3, 1024, 3600})Qhov piv txwv no yuav ua hauj lwm thaum lub sij hawm luam theej duab raws nraim tib yam li cov module tas sij hawm, uas tau sau rau hauv Lua. Thawj qhov kev sib cav rau qhov pib ua haujlwm yog lub npe tshwj xeeb ntawm txoj haujlwm. Qhov thib ob yog qhov chaw identifier. Qhov thib peb yog qhov tshwj xeeb Performance index uas tuples yuav raug tshem tawm. Qhov thib plaub yog qhov ntsuas uas cov tuples yuav hla. Qhov thib tsib yog tus naj npawb ntawm tuple teb nrog lub neej (tus lej pib ntawm 1, tsis yog 0!). Qhov thib rau thiab xya yog scanning nqis. 1024 yog qhov siab tshaj plaws ntawm tuples uas tuaj yeem pom hauv ib qho kev hloov pauv. 3600 β tag nrho scan sij hawm hauv vib nas this.
Nco ntsoov tias qhov piv txwv siv tib qhov ntsuas rau nkag thiab rho tawm. Yog hais tias qhov no yog tsob ntoo Performance index, ces qhov traversal yog nqa tawm ntawm tus yuam sij me mus rau qhov loj dua. Yog hais tias muaj lwm yam, piv txwv li, hash index, ces tus traversal yog nqa tawm, raws li txoj cai, nyob rau hauv random kev txiav txim. Tag nrho qhov chaw tuples yog scanned hauv ib qho scan.
Cia peb ntxig ob peb tuples rau hauv qhov chaw nrog lub neej ntawm 60 vib nas this:
box.space.tester:insert{0, '[email protected]', math.floor(fiber.time()) + 60}
box.space.tester:insert{1, '[email protected]', math.floor(fiber.time()) + 60}
box.space.tester:insert{2, '[email protected]', math.floor(fiber.time()) + 60}Cia peb kuaj xyuas tias qhov kev ntxig tau ua tiav:
tarantool> box.space.tester.index.primary:select()
---
- - [0, '[email protected]', 1576418976]
- [1, '[email protected]', 1576418976]
- [2, '[email protected]', 1576418976]
...Cia peb rov ua qhov kev xaiv tom qab 60+ vib nas this (suav los ntawm qhov pib ntawm qhov ntxig ntawm thawj tuple) thiab pom tias lub capped expirationd module tau ua tiav:
tarantool> box.space.tester.index.primary:select()
---
- []
...Cia peb tso tseg txoj haujlwm:
capped_connection:call('libcapped-expirationd.kill', {'non-indexed'})Cia peb saib ib qho piv txwv thib ob uas qhov kev ntsuas cais tau siv rau kev nkag mus:
fiber = require('fiber')
net_box = require('net.box')
box.cfg{listen = 3300}
box.schema.func.create('libcapped-expirationd.start', {language = 'C'})
box.schema.user.grant('guest', 'execute', 'function', 'libcapped-expirationd.start')
box.schema.func.create('libcapped-expirationd.kill', {language = 'C'})
box.schema.user.grant('guest', 'execute', 'function', 'libcapped-expirationd.kill')
box.schema.space.create('tester')
box.space.tester:create_index('primary', {unique = true, parts = {1, 'unsigned'}})
box.space.tester:create_index('exp', {unique = false, parts = {3, 'unsigned'}})
capped_connection = net_box:new(3300)Txhua yam ntawm no yog tib yam li hauv thawj qhov piv txwv, nrog rau qee qhov kev zam. Peb tsim ib tsob ntoo Performance index saum toj peb daim teb thiab hu nws exp. Qhov ntsuas no tsis tas yuav yog qhov tshwj xeeb, tsis zoo li qhov ntsuas hu ua thawj. Traversal yuav ua los ntawm exp index, thiab tshem tawm los ntawm thawj. Peb nco ntsoov tias yav dhau los ob qho tib si tau ua tiav tsuas yog siv qhov ntsuas thawj zaug xwb.
Tom qab kev npaj ua haujlwm, peb pib ua haujlwm nrog cov lus sib cav tshiab:
capped_connection:call('libcapped-expirationd.start', {'indexed', box.space.tester.id, box.space.tester.index.primary, box.space.tester.index.exp, 3, 1024, 3600})Cia peb ntxig ob peb tuples rau hauv qhov chaw dua nrog lub neej ntawm 60 vib nas this:
box.space.tester:insert{0, '[email protected]', math.floor(fiber.time()) + 60}
box.space.tester:insert{1, '[email protected]', math.floor(fiber.time()) + 60}
box.space.tester:insert{2, '[email protected]', math.floor(fiber.time()) + 60}Tom qab 30 vib nas this, los ntawm kev sib piv, peb yuav ntxiv ob peb ntxiv tuples:
box.space.tester:insert{3, '[email protected]', math.floor(fiber.time()) + 60}
box.space.tester:insert{4, '[email protected]', math.floor(fiber.time()) + 60}
box.space.tester:insert{5, '[email protected]', math.floor(fiber.time()) + 60}Cia peb kuaj xyuas tias qhov kev ntxig tau ua tiav:
tarantool> box.space.tester.index.primary:select()
---
- - [0, '[email protected]', 1576421257]
- [1, '[email protected]', 1576421257]
- [2, '[email protected]', 1576421257]
- [3, '[email protected]', 1576421287]
- [4, '[email protected]', 1576421287]
- [5, '[email protected]', 1576421287]
...Cia peb rov ua qhov kev xaiv tom qab 60+ vib nas this (suav los ntawm qhov pib ntawm qhov ntxig ntawm thawj tuple) thiab pom tias lub capped expirationd module tau ua tiav:
tarantool> box.space.tester.index.primary:select()
---
- - [3, '[email protected]', 1576421287]
- [4, '[email protected]', 1576421287]
- [5, '[email protected]', 1576421287]
...Tseem muaj qee qhov tuples nyob hauv qhov chaw uas yuav muaj li 30 vib nas this ntxiv kom nyob. Ntxiv mus, lub scan nres thaum tsiv ntawm ib tug tuple nrog ib tug ID ntawm 2 thiab ib tug lub neej ntawm 1576421257 mus rau ib tug tuple nrog ib tug ID ntawm 3 thiab ib tug lub neej ntawm 1576421287. Tuples nrog ib tug lifetime ntawm 1576421287 los yog ntau tshaj yog tsis scanned vim lub ordering ntawm cov exp index keys. Nov yog qhov kev txuag nyiaj uas peb xav ua kom tiav thaum pib.
Cia peb tso tseg txoj haujlwm:
capped_connection:call('libcapped-expirationd.kill', {'indexed'})Kev siv
Txoj hauv kev zoo tshaj plaws los qhia txog txhua yam ntawm qhov project yog nws qhov qub. ! Raws li ib feem ntawm kev tshaj tawm, peb tsuas yog tsom rau cov ntsiab lus tseem ceeb tshaj plaws, uas yog, qhov chaw bypass algorithms.
Cov lus sib cav peb dhau mus rau txoj kev pib yog khaws cia rau hauv cov qauv hu ua expirationd_task:
struct expirationd_task
{
char name[256];
uint32_t space_id;
uint32_t rm_index_id;
uint32_t it_index_id;
uint32_t it_index_type;
uint32_t field_no;
uint32_t scan_size;
uint32_t scan_time;
};Lub npe attribute yog lub npe ntawm txoj haujlwm. Tus cwj pwm space_id yog tus cim qhov chaw. Tus cwj pwm rm_index_id yog tus cim ntawm qhov ntsuas tshwj xeeb uas tuples yuav raug tshem tawm. Lub it_index_id attribute yog tus txheeb ze ntawm qhov Performance index uas tuples yuav hla. Lub it_index_type attribute yog hom index uas tuples yuav hla. Filed_no attribute yog tus naj npawb ntawm tuple teb nrog lub neej. Tus cwj pwm scan_size yog tus lej siab tshaj plaws ntawm tuples uas tau luam tawm hauv ib qho kev hloov pauv. Tus cwj pwm scan_time yog lub sijhawm scan tag nrho hauv vib nas this.
Peb yuav tsis xav txog kev sib cav sib cav. Qhov no yog ib txoj haujlwm nyuaj tab sis yooj yim, uas lub tsev qiv ntawv yuav pab koj . Cov teeb meem tsuas yog tshwm sim nrog kev ntsuas uas dhau los ntawm Lua raws li cov ntaub ntawv nyuaj nrog hom mp_map, thiab tsis siv hom yooj yim mp_bool, mp_double, mp_int, mp_uint thiab mp_array. Tab sis tsis tas yuav parse tag nrho cov index. Koj tsuas yog yuav tsum tau xyuas nws qhov tshwj xeeb, suav cov hom thiab rho tawm tus cim.
Peb sau cov qauv ntawm tag nrho cov haujlwm uas siv los txheeb xyuas:
bool expirationd_parse_name(struct expirationd_task *task, const char **pos);
bool expirationd_parse_space_id(struct expirationd_task *task, const char **pos);
bool expirationd_parse_rm_index_id(struct expirationd_task *task, const char **pos);
bool expirationd_parse_rm_index_unique(struct expirationd_task *task, const char **pos);
bool expirationd_parse_rm_index(struct expirationd_task *task, const char **pos);
bool expirationd_parse_it_index_id(struct expirationd_task *task, const char **pos);
bool expirationd_parse_it_index_type(struct expirationd_task *task, const char **pos);
bool expirationd_parse_it_index(struct expirationd_task *task, const char **pos);
bool expirationd_parse_field_no(struct expirationd_task *task, const char **pos);
bool expirationd_parse_scan_size(struct expirationd_task *task, const char **pos);
bool expirationd_parse_scan_time(struct expirationd_task *task, const char **pos);Tam sim no cia peb mus rau qhov tseem ceeb tshaj plaws - lub logic ntawm bypassing qhov chaw thiab tshem tawm tuples. Txhua qhov thaiv ntawm tuples tsis loj dua scan_size yog scanned thiab hloov kho nyob rau hauv ib qho kev sib pauv. Yog tias ua tiav, qhov kev sib pauv no tau cog lus; yog tias muaj teeb meem tshwm sim, nws yuav rov qab. Qhov kev sib cav zaum kawg rau lub expirationd_iterate muaj nuj nqi yog ib tug taw tes rau lub iterator los ntawm uas scanning pib los yog mus ntxiv. Qhov no iterator yog incremented nyob rau hauv kom txog rau thaum ib qho kev ua yuam kev tshwm sim, qhov chaw khiav tawm, los yog nws yog tsis muaj peev xwm mus nres tus txheej txheem ua ntej. Cov haujlwm expirationd_expired xyuas lub neej ntawm ib tug tuple, expirationd_delete deletes ib tug tuple, expirationd_breakable checks seb peb yuav tsum tau mus rau.
Expirationd_iterate muaj nuj nqi code:
static bool
expirationd_iterate(struct expirationd_task *task, box_iterator_t **iterp)
{
box_iterator_t *iter = *iterp;
box_txn_begin();
for (uint32_t i = 0; i < task->scan_size; ++i) {
box_tuple_t *tuple = NULL;
if (box_iterator_next(iter, &tuple) < 0) {
box_iterator_free(iter);
*iterp = NULL;
box_txn_rollback();
return false;
}
if (!tuple) {
box_iterator_free(iter);
*iterp = NULL;
box_txn_commit();
return true;
}
if (expirationd_expired(task, tuple))
expirationd_delete(task, tuple);
else if (expirationd_breakable(task))
break;
}
box_txn_commit();
return true;
}Function code expirationd_expired:
static bool
expirationd_expired(struct expirationd_task *task, box_tuple_t *tuple)
{
const char *buf = box_tuple_field(tuple, task->field_no - 1);
if (!buf || mp_typeof(*buf) != MP_UINT)
return false;
uint64_t val = mp_decode_uint(&buf);
if (val > fiber_time64() / 1000000)
return false;
return true;
}Expirationd_delete muaj nuj nqi code:
static void
expirationd_delete(struct expirationd_task *task, box_tuple_t *tuple)
{
uint32_t len;
const char *str = box_tuple_extract_key(tuple, task->space_id, task->rm_index_id, &len);
box_delete(task->space_id, task->rm_index_id, str, str + len, NULL);
}Function code expiration_breakable:
static bool
expirationd_breakable(struct expirationd_task *task)
{
return task->it_index_id != task->rm_index_id && task->it_index_type == ITER_GT;
}Daim ntawv thov
Koj tuaj yeem saib qhov chaws ntawm !
Tau qhov twg los: www.hab.com