Nrhiav kev ua haujlwm tau zoo hauv cov ntaub ntawv yog siv ntau qhov chaw ntawm kev txheeb xyuas cov ntaub ntawv: kev tswj cov ntaub ntawv, kev tu cov ntaub ntawv, cov ntaub ntawv rov qab engineering thiab kev tshawb nrhiav cov ntaub ntawv. Peb twb tau luam tawm txog cov dependencies lawv tus kheej Anastasia Birillo thiab Nikita Bobrov. Lub sijhawm no, Anastasia, kawm tiav ntawm Computer Science Center xyoo no, qhia txog kev txhim kho ntawm txoj haujlwm no ua ib feem ntawm kev tshawb fawb ua haujlwm uas nws tiv thaiv ntawm qhov chaw.
Kev xaiv ua haujlwm
Thaum kawm ntawm CS center, kuv pib kawm cov ntaub ntawv hauv qhov tob, uas yog, kev tshawb fawb txog kev ua haujlwm thiab qhov sib txawv ntawm qhov sib txawv. Lub ntsiab lus no tau hais txog lub ntsiab lus ntawm kuv cov haujlwm hauv tsev kawm ntawv, yog li thaum ua haujlwm ntawm cov haujlwm, kuv pib nyeem cov ntawv hais txog ntau yam kev vam khom hauv cov ntaub ntawv. Kuv tau sau ib qho kev tshuaj xyuas ntawm thaj chaw no - ib qho ntawm kuv thawj zaug ua lus Askiv thiab xa mus rau lub rooj sib tham SEIM-2017. Kuv zoo siab heev thaum kuv pom tias nws tau txais tom qab tag nrho, thiab txiav txim siab mus tob rau hauv lub ncauj lus. Lub tswv yim nws tus kheej tsis yog tshiab - nws tau pib siv rov qab rau xyoo 90s, tab sis txawm tias tam sim no nws tau siv ntau qhov chaw.
Thaum lub sij hawm kuv lub semester thib ob ntawm qhov chaw, kuv tau pib qhov kev tshawb fawb los txhim kho algorithms rau kev nrhiav kev ua haujlwm tau zoo. Nws tau ua haujlwm rau nws ua ke nrog St. Petersburg State University cov tub ntxhais kawm tiav Nikita Bobrov ntawm JetBrains Research.
Computational complexity ntawm searching rau functional dependencies
Qhov teeb meem tseem ceeb yog kev siv computer complexity. Tus naj npawb ntawm qhov ua tau tsawg kawg nkaus thiab tsis muaj qhov tsis tseem ceeb yog txwv saum toj no los ntawm tus nqi qhov twg - tus naj npawb ntawm cov lus qhia. Lub sijhawm ua haujlwm ntawm cov algorithms tsis yog nyob ntawm tus naj npawb ntawm cov cwj pwm, tab sis kuj nyob ntawm tus lej ntawm kab. Nyob rau hauv lub 90s, tsoom fwv teb chaws txoj cai tshawb nrhiav algorithms ntawm lub desktop PC tsis tu ncua tuaj yeem ua cov ntaub ntawv teev muaj txog 20 tus cwj pwm thiab kaum tawm txhiab tus kab hauv mus txog ob peb teev. Niaj hnub nimno algorithms khiav ntawm ntau cov tub ntxhais processors ntes cov kev cia siab rau cov ntaub ntawv teev muaj ntau pua tus cwj pwm (txog 200) thiab ntau pua txhiab kab nyob rau tib lub sijhawm. Txawm li cas los xij, qhov no tsis txaus: lub sijhawm zoo li no tsis tsim nyog rau feem ntau cov ntawv thov hauv ntiaj teb. Yog li ntawd, peb tsim txoj hauv kev los ua kom cov algorithms uas twb muaj lawm.
Caching schemes rau kev sib tshuam kev sib tshuam
Hauv thawj ntu ntawm kev ua haujlwm, peb tau tsim cov txheej txheem caching rau cov chav kawm ntawm algorithms uas siv txoj kev sib faib faib. Ib qho kev faib rau tus cwj pwm yog ib pawg ntawm cov npe, qhov twg txhua daim ntawv teev npe muaj cov kab zauv uas muaj cov txiaj ntsig zoo ib yam rau tus cwj pwm muab. Txhua daim ntawv teev npe hu ua pawg. Ntau cov algorithms niaj hnub siv cov partitions los txiav txim seb puas muaj kev vam khom los yog tsis, uas yog, lawv ua raws li cov lus qhia: Kev vam meej tuav yog tias . Ntawm no ib tug muab faib yog tsim thiab lub tswvyim ntawm muab faib loj yog siv - tus naj npawb ntawm pawg nyob rau hauv nws. Algorithms uas siv cov partitions, thaum lub dependency raug ua txhaum, ntxiv cov yam ntxwv ntxiv rau sab laug ntawm kev vam khom, thiab rov xam nws, ua haujlwm ntawm kev sib tshuam ntawm partitions. Qhov kev ua haujlwm no yog hu ua kev tshwj xeeb hauv cov khoom. Tab sis peb pom tias cov partitions rau dependencies uas tsuas yog yuav tsum tau khaws cia tom qab ob peb rounds ntawm tshwj xeeb reused active reused, uas yuav txo tau lub sij hawm khiav ntawm lub algorithms, vim hais tias cov kev sib tshuam ua hauj lwm yog kim.
Yog li ntawd, peb tau npaj ib qho kev xav raws li Shannon Entropy thiab Ginny Uncertainty, nrog rau peb cov metric, uas peb hu ua Reverse Entropy. Nws yog kev hloov kho me ntsis ntawm Shannon Entropy thiab nce ntxiv raws li qhov tshwj xeeb ntawm cov ntaub ntawv teeb tsa nce. Lub tswv yim heuristic yog raws li nram no:
nws yog - degree ntawm uniqueness ntawm qhov muab faib tsis ntev los no thiab yog qhov nruab nrab ntawm cov degrees ntawm uniqueness rau tus kheej cwj pwm. Tag nrho peb qhov kev ntsuas tau piav qhia saum toj no tau sim ua qhov ntsuas qhov sib txawv. Koj tuaj yeem pom tias muaj ob qhov kev hloov kho hauv cov heuristic. Thawj qhov qhia tau hais tias kaw qhov kev faib tam sim no li cas rau tus yuam sij tseem ceeb thiab tso cai rau koj kom cache kom ntau dua cov partitions uas nyob deb ntawm qhov tseem ceeb. Qhov kev hloov kho thib ob tso cai rau koj los saib xyuas cov neeg nyob hauv cache thiab yog li txhawb kom ntxiv cov partitions rau lub cache yog tias muaj qhov chaw dawb. Txoj kev daws teeb meem zoo ntawm qhov teeb meem no tau tso cai rau peb kom ceev PYRO algorithm los ntawm 10-40%, nyob ntawm cov ntaub ntawv. Nws yog tsim nyog sau cia tias PYRO algorithm yog qhov ua tau zoo tshaj plaws hauv cheeb tsam no.
Nyob rau hauv daim duab hauv qab no koj tuaj yeem pom cov txiaj ntsig ntawm kev siv qhov kev thov heuristic piv rau qhov yooj yim npib npib caching mus kom ze. X axis yog logarithmic.
Lwm txoj hauv kev los khaws cov partitions
Peb mam li npaj lwm txoj hauv kev los khaws cov partitions. Partitions yog ib pawg ntawm pawg, txhua tus uas khaws cov naj npawb ntawm tuples nrog cov txiaj ntsig zoo ib yam rau qee yam cwj pwm. Cov pawg no tuaj yeem muaj cov lej ntev ntawm tuple, piv txwv li yog tias cov ntaub ntawv hauv lub rooj raug txiav txim. Yog li ntawd, peb tau npaj ib lub tswv yim compression rau khaws cia partitions, uas yog lub sij hawm cia ntawm qhov tseem ceeb nyob rau hauv pawg ntawm partitions:
$$display$$pi(X) = {{underbrace{1, 2, 3, 4, 5}_{First interval}, underbrace{7, 8}_{Second interval}, 10}}\ downarrow{ Compression} \ pi(X) = {{underbrace{$, 1, 5}_{First~interval}, underbrace{7, 8}_{Second~interval}, 10}}$$display$$
Txoj kev no tuaj yeem txo qis kev nco thaum lub sijhawm ua haujlwm ntawm TANE algorithm los ntawm 1 txog 25%. TANE algorithm yog cov txheej txheem classic rau kev tshawb nrhiav tsoomfwv txoj cai lij choj; nws siv cov kev faib tawm thaum nws ua haujlwm. Raws li ib feem ntawm kev xyaum, TANE algorithm raug xaiv, vim nws yooj yim dua los siv lub sijhawm cia hauv nws dua li, piv txwv li, hauv PYRO txhawm rau ntsuas seb qhov kev thov ua haujlwm puas ua haujlwm. Cov txiaj ntsig tau muab nthuav tawm hauv daim duab hauv qab no. X axis yog logarithmic.
Lub Rooj Sib Tham ADBIS-2019
Raws li cov txiaj ntsig ntawm kev tshawb fawb, lub Cuaj Hlis 2019 kuv tau tshaj tawm ib tsab xov xwm ntawm 23rd European Conference on Advances in Databases and Information Systems (ADBIS-2019). Thaum lub sijhawm nthuav tawm, txoj haujlwm tau sau tseg los ntawm Bernhard Thalheim, tus neeg tseem ceeb hauv thaj chaw ntawm cov ntaub ntawv. Cov txiaj ntsig kev tshawb fawb tau tsim lub hauv paus ntawm kuv qhov kev tshaj tawm ntawm kev kawm tiav qib siab hauv kev ua lej thiab tshuab tshuab ntawm St. Petersburg State University, thaum lub sij hawm uas ob qho tib si npaj siab (caching thiab compression) tau siv nyob rau hauv ob qho tib si algorithms: TANE thiab PYRO. Tsis tas li ntawd, cov txiaj ntsig tau pom tias cov kev xav tau yog universal, txij li ntawm ob qho tib si algorithms, nrog rau ob qho tib si, kev txo qis hauv kev nco tau pom, nrog rau kev txo qis hauv lub sijhawm ua haujlwm ntawm cov algorithms.
Tau qhov twg los: www.hab.com