{"id":30280,"date":"2019-10-31T21:34:38","date_gmt":"2019-10-31T18:34:38","guid":{"rendered":"https:\/\/prohoster.info\/blog\/potochnaya-konvertatsiya-baz-firebird-2-5-v-format-ods12-firebird-3-0\/"},"modified":"2019-10-31T21:34:38","modified_gmt":"2019-10-31T18:34:38","slug":"potochnaya-konvertatsiya-baz-firebird-2-5-v-format-ods12-firebird-3-0","status":"publish","type":"post","link":"https:\/\/prohoster.info\/cs\/blog\/potochnaya-konvertatsiya-baz-firebird-2-5-v-format-ods12-firebird-3-0","title":{"rendered":"Pr\u016fb\u011b\u017en\u00e1 konverze datab\u00e1z\u00ed Firebird 2.5 do form\u00e1tu ODS12 (Firebird 3.0).","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Ka\u017ed\u00e1 verze Firebird m\u00e1 vlastn\u00ed verzi form\u00e1tu diskov\u00fdch struktur datab\u00e1ze \u2013 O(n)D(isk)S(tructure). A\u017e do verze 2.5 v\u010detn\u011b mohl Firebird pracovat s ODS p\u0159edchoz\u00edch verz\u00ed, co\u017e znamen\u00e1, \u017ee datab\u00e1ze z star\u0161\u00edch verz\u00ed byly otev\u0159eny novou verz\u00ed a fungovaly v re\u017eimu kompatibility. Nicm\u00e9n\u011b Firebird 3.0 pracuje pouze s datab\u00e1zemi v vlastn\u00ed ODS verzi 12.0.<\/p>\n

Abychom p\u0159e\u0161li na 3.0, je nutn\u00e9 p\u0159ev\u00e9st datab\u00e1zi z 2.5 do nov\u00e9ho form\u00e1tu prost\u0159ednictv\u00edm backup\/restore. P\u0159edpokl\u00e1d\u00e1me, \u017ee datab\u00e1ze byla p\u0159edem p\u0159ipravena na konverzi \u2013 tj. metadata a dotazy byly zkontrolov\u00e1ny na kompatibilitu s Firebird 3.0.<\/p>\n

Pokud se dr\u017e\u00edme standardn\u00edho p\u0159\u00edstupu, to znamen\u00e1, \u017ee je t\u0159eba prov\u00e9st z\u00e1lohu na verzi 2.5, pot\u00e9 nainstalovat 3.0 a prov\u00e9st obnoven\u00ed. Tento postup je p\u0159ijateln\u00fd, pokud je dostatek \u010dasu, ale p\u0159i migraci velk\u00fdch datab\u00e1z\u00ed, nebo p\u0159i sou\u010dasn\u00e9 migraci n\u011bkolika des\u00edtek datab\u00e1z\u00ed, kdy je \u010das omezen, je mo\u017en\u00e9 vyu\u017e\u00edt streamovou konverzi, kter\u00e1 je o 30\u201340 % rychlej\u0161\u00ed. Jak to prov\u00e9st (na Windows a Linuxu), si p\u0159e\u010dt\u011bte n\u00ed\u017ee.
\n<\/a><\/noindex>
\nHlavn\u00ed my\u0161lenka spo\u010d\u00edv\u00e1 v tom, \u017ee pro urychlen\u00ed vyu\u017eijeme pipeline:<\/p>\n

gbak -b \u2026 database25 stdout | gbak -c \u2026 stdin database30<\/code><\/pre>\n
\nGbak z 2.5 generuje z\u00e1lohu ve line\u00e1rn\u00edm form\u00e1tu a sm\u011bruje ji do stdout, kter\u00fd hned p\u0159es stdin zachyt\u00ed gbak z 3.0 a vytvo\u0159\u00ed novou datab\u00e1zi.<\/p>\n
Takov\u00fd pipeline je t\u0159eba organizovat nutn\u011b prost\u0159ednictv\u00edm lok\u00e1ln\u00ed (souborov\u00e9) metody p\u0159\u00edstupu, proto\u017ee s\u00ed\u0165ov\u00fd p\u0159\u00edstup (i p\u0159es localhost) v\u00fdznamn\u011b zpomal\u00ed proces.<\/p>\n
N\u00ed\u017ee se zab\u00fdv\u00e1me detaily pro Windows a Linux.<\/p>\n
Windows<\/b><\/p>\n
V p\u0159\u00edpad\u011b Windows je nejjednodu\u0161\u0161\u00ed ud\u011blat zcela autonomn\u00ed sestaven\u00ed Firebird. K tomu vezmeme embed-archiv Firebird 2.5<\/a><\/noindex>, p\u0159ejmenujeme fbembed.dll na fbclient.dll, p\u0159id\u00e1me z archivu \"b\u011b\u017en\u00e9ho\" 2.5 n\u00e1stroje gbak.exe a (nepovinn\u011b) \u2013 isql.exe. <\/p>\n
Firebird 3.0 pou\u017e\u00edv\u00e1 jednotn\u00e9 sestaven\u00ed<\/a><\/noindex> a nevy\u017eaduje \u017e\u00e1dn\u00e9 \u00fapravy.<\/p>\n
Nejmen\u0161\u00ed varianta (nevy\u017eaduj\u00edc\u00ed instalaci runtime knihoven VS2008\/VS2010 na c\u00edlov\u00e9m syst\u00e9mu) obsahuje n\u00e1sleduj\u00edc\u00ed soubory:<\/p>\n
25\/gbak.exe\n25\/fbclient.dll\n25\/firebird.conf\n25\/firebird.log\n25\/firebird.msg\n25\/ib_util.dll\n25\/icudt30.dll\n25\/icuin30.dll\n25\/icuuc30.dll\n25\/Microsoft.VC80.CRT.manifest\n25\/msvcp80.dll\n25\/msvcr80.dll\n\n30\/fbclient.dll\n30\/firebird.conf\n30\/firebird.msg\n30\/gbak.exe\n30\/ib_util.dll\n30\/icudt52.dll\n30\/icudt52l.dat\n30\/icuin52.dll\n30\/icuuc52.dll\n30\/msvcp100.dll\n30\/msvcr100.dll\n30\/intl\/fbintl.conf\n30\/intl\/fbintl.dll\n30\/plugins\/engine12.dll<\/code><\/pre>\n
\nZku\u0161en\u00fd administr\u00e1tor si m\u016f\u017ee v\u0161imnout, \u017ee ve verzi 2.5 nejsou zahrnuty soubory intl\/fbintl.dll a intl\/fbintl.conf. To je opravdu pravda, proto\u017ee gbak nepou\u017e\u00edv\u00e1 charset p\u0159ipojen\u00ed a neprov\u00e1d\u00ed konverzi dat mezi charsets, ale na 'p\u0159ij\u00edmac\u00ed' stran\u011b Firebird 3.0 jsou tyto soubory nezbytn\u00e9 p\u0159i vytv\u00e1\u0159en\u00ed index\u016f.<\/p>\n
V souboru firebird.conf se doporu\u010duje p\u0159idat n\u00e1sleduj\u00edc\u00ed do Firebird 3.0:<\/p>\n
MaxUnflushedWrites = -1\nMaxUnflushedWriteTime = -1<\/code><\/pre>\n
\nJe tak\u00e9 \u017e\u00e1douc\u00ed nastavit r\u016fzn\u00e9 hodnoty IpcName pro verze 2.5 a 3.0. <\/p>\n
P\u0159i volb\u011b hodnot dal\u0161\u00edch parametr\u016f firebird.conf vych\u00e1z\u00edme z jednoduch\u00e9ho p\u0159edpokladu: b\u011bhem na\u010d\u00edt\u00e1n\u00ed dat v jednom procesu gbak pracuje na verzi 2.5, a ve druh\u00e9m \u2013 3.0, pot\u00e9 2.5 dokon\u010d\u00ed pr\u00e1ci, zat\u00edmco 3.0 za\u010dne vytv\u00e1\u0159et indexy.<\/p>\n
Abychom urychlili f\u00e1zi vytv\u00e1\u0159en\u00ed index\u016f ve verzi 3.0, doporu\u010duje se zv\u00fd\u0161it velikost parametru TempCacheLimit na ~40% RAM (pokud se jedn\u00e1 o dedikovan\u00fd server, samoz\u0159ejm\u011b).<\/p>\n
Nap\u0159\u00edklad, pokud m\u00e1 server 16 GB RAM, lze nastavit <\/p>\n
TempCacheLimit=6G<\/code><\/pre>\n
\nSamoz\u0159ejm\u011b, takovou hodnotu lze pou\u017e\u00edt pouze u 64bitov\u00e9ho Firebirdu 3, proto\u017ee jak\u00fdkoli 32bitov\u00fd proces nem\u016f\u017ee alokovat v\u00edce ne\u017e 2 GB pam\u011bti.<\/i><\/p>\n
U 2.5 nen\u00ed t\u0159eba tento parametr m\u011bnit \u2013 i tak nem\u016f\u017ee b\u00fdt v\u011bt\u0161\u00ed ne\u017e 2 GB, a p\u0159i z\u00e1lohov\u00e1n\u00ed na rychlost nem\u00e1 vliv.<\/p>\n
P\u0159ed proveden\u00edm operace je nutn\u00e9 zkontrolovat, \u017ee je str\u00e1nkov\u00fd cache v hlavi\u010dce datab\u00e1ze nastaven na 0 (p\u0159\u00edkaz gstat -h databasename<\/code>, pod\u00edvejte se na \u0159\u00e1dek Page buffers). <\/p>\n
Pokud je cache explicitn\u011b nastavena v hlavi\u010dce DB, p\u0159episuje hodnoty z firebird.conf (a databases.conf ve verzi 3.0) a v p\u0159\u00edpad\u011b neadekv\u00e1tn\u011b vysok\u00fdch hodnot m\u016f\u017ee v\u00e9st k nadm\u011brn\u00e9 spot\u0159eb\u011b pam\u011bti a odesl\u00e1n\u00ed do swapu.<\/p>\n
D\u00e1le zkop\u00edrujeme soubory na c\u00edlov\u00fd syst\u00e9m. <\/p>\n
Konverze se prov\u00e1d\u00ed po zastaven\u00ed 'syst\u00e9mov\u00e9' slu\u017eby Firebird 2.5, v p\u0159\u00edkazov\u00e9m \u0159\u00e1dku s privilegii m\u00edstn\u00edho administr\u00e1tora (p\u0159\u00edklad):<\/p>\n
set ISC_USER=vlastn\u00edk\n\"25\/gbak\" -z -b -g -v -st t -y 25.log datab\u00e1ze25 stdout|^\n\"30\/gbak\" -z -c -v -st t -y 30.log stdin datab\u00e1ze30<\/code><\/pre>\n
\nV tomto p\u0159\u00edkladu se pou\u017e\u00edv\u00e1 'p\u0159\u00edm\u00e1 lom\u00edtka' v uvozovk\u00e1ch (povolen\u00fd 'unix-style'), a 'st\u0159edn\u00edk' (symbol '^') escapuje znak nov\u00e9ho \u0159\u00e1dku, co\u017e je pohodln\u00e9 p\u0159i psan\u00ed dlouh\u00fdch p\u0159\u00edkaz\u016f. Mo\u017enost -st(atus) se objevila ve Firebird 2.5.8 a umo\u017e\u0148uje zaznamenat do protokolu \u00fadaje o \u010dasov\u00e9m v\u00fdkonu procesu gbak (v\u00edce informac\u00ed je v dokumentaci).<\/p>\n
Linux<\/b><\/p>\n
Na Linuxu Firebird 3 z\u00e1vis\u00ed na knihovn\u011b tommath. V CentOS (RHEL) se tato knihovna nach\u00e1z\u00ed v epel repozit\u00e1\u0159i, v Ubuntu (Debian) v \u2013 syst\u00e9mov\u00e9m.<\/p>\n
Pro CentOS je pot\u0159eba nejprve p\u0159ipojit repozit\u00e1\u0159 epel a a\u017e potom prov\u00e9st <\/p>\n
yum install libtommath<\/code><\/pre>\n
\nU Ubuntu nen\u00ed pot\u0159eba p\u0159ipojovat dal\u0161\u00ed repozit\u00e1\u0159e, ale v Ubuntu 16 a Ubuntu 18 se instaluj\u00ed r\u016fzn\u00e9 verze bal\u00ed\u010dk\u016f \u2013 libtommath0 a libtommath1, odpov\u00eddaj\u00edc\u00edm zp\u016fsobem.<\/p>\n
Firebird 3.0 hled\u00e1 tommath.so.0 a pro Ubuntu 18 je nav\u00edc pot\u0159eba vytvo\u0159it odkaz (symlink) c tommath.so.0 na tommath.so.1. Nejprve je t\u0159eba naj\u00edt tommath.so.1. <\/p>\n
Hledan\u00e1 cesta v Ubuntu je \/usr\/lib\/x86_64-linux-gnu\/<\/code>, ale v jin\u00fdch Debian-based distribuc\u00edch to m\u016f\u017ee b\u00fdt jinak.<\/p>\n
Druh\u00fd probl\u00e9m souvis\u00ed s t\u00edm, \u017ee do Firebird 3.0.1, v\u010detn\u011b, nebyl jednoduch\u00fd zp\u016fsob, jak nainstalovat dv\u011b r\u016fzn\u00e9 verze serveru. Variantu \u201ekompilace ze zdrojov\u00fdch k\u00f3d\u016f s pot\u0159ebn\u00fdm prefixem\u201c nezva\u017eujeme kv\u016fli jej\u00ed relativn\u00ed pracnosti.<\/p>\n
Pro Firebird 3.0.2 a vy\u0161\u0161\u00ed je realizov\u00e1no sestaven\u00ed s \u2013enable-binreloc<\/a><\/noindex> a samostatn\u00e1 mo\u017enost instal\u00e1toru (-path cesta).<\/p>\n
P\u0159edpokl\u00e1daje, \u017ee knihovna tommath a, pokud je to nutn\u00e9, symlink pro tommath.so.0 jsou p\u0159id\u00e1ny do syst\u00e9mu, lze doinstalovat aktu\u00e1ln\u00ed (v dob\u011b psan\u00ed tohoto \u010dl\u00e1nku) distribuci Firebird 3.0.4 do, nap\u0159\u00edklad, \\\/opt\\\/fb3:<\/p>\n
.\/install.sh -path \\\/opt\\\/fb3<\/code><\/pre>\n
\nPo tom lze zastavit syst\u00e9movou slu\u017ebu Firebird a spustit kontinu\u00e1ln\u00ed konverzi. <\/p>\n
P\u0159i zastaven\u00ed Firebird je t\u0159eba br\u00e1t v \u00favahu, \u017ee procesy Firebird 2.5 v re\u017eimu Classic obvykle spou\u0161t\u00ed xinetd \u2013 proto je pot\u0159eba bu\u010f zak\u00e1zat slu\u017ebu firebird pro xinetd, nebo zastavit xinetd \u00fapln\u011b.<\/i><\/p>\n
Ve firebird.conf pro 3.0 na Linuxu nen\u00ed pot\u0159eba nastavovat parametry MaxUnflushed (ty funguj\u00ed pouze na Windows) a m\u011bnit nastaven\u00ed Firebird 2.5. <\/p>\n
V Linuxu m\u00edstn\u00ed (souborov\u00fd) p\u0159\u00edstup Firebird 2.5 nen\u00ed ekvivalentn\u00ed embedded variant\u011b na Windows \u2013 server 2.5 bude fungovat v procesu gbak (bez s\u00ed\u0165ov\u00e9 \u010d\u00e1sti), ale pr\u00e1va p\u0159\u00edstupu budou kontrolov\u00e1na podle datab\u00e1ze u\u017eivatel\u016f, a t\u00edm p\u00e1dem bude pot\u0159eba nejen p\u0159ihla\u0161ovac\u00ed jm\u00e9no, ale i heslo: <\/p>\n
export ISC_USER=username ISC_PASSWORD=password\n\\\/opt\\\/firebird\\\/bin\\\/gbak -b \u2026 datab\u00e1ze25 stdout\n|\\\/opt\\\/fb3\\\/bin\\\/gbak -c \u2026 stdin datab\u00e1ze30<\/code><\/pre>\n
\nPo \u00fasp\u011b\u0161n\u00e9 konverzi je t\u0159eba nejprve odstranit \u201edal\u0161\u00ed\u201c Firebird 3.0, pot\u00e9 \u201ehlavn\u00ed\u201c Firebird 2.5 a a\u017e pot\u00e9 prov\u00e9st \u010distou instalaci Firebird 2.5 \u2013 nejl\u00e9pe z ofici\u00e1ln\u00edho instal\u00e1toru tar.gz, nikoli prost\u0159ednictv\u00edm repozit\u00e1\u0159\u016f, proto\u017ee verze v repozit\u00e1\u0159\u00edch m\u016f\u017ee b\u00fdt zastaral\u00e1.<\/p>\n
Tak\u00e9 po restauraci DB na Linuxu a p\u0159einstalaci je t\u0159eba zkontrolovat, aby m\u011bla nov\u00e1 DB vlastn\u00edka u\u017eivatele firebird.<\/p>\n
Pokud tomu tak nen\u00ed, bude pot\u0159eba to opravit<\/p>\n
chown firebird.firebird datab\u00e1ze<\/code><\/pre>\n
\nV\u00fdsledek<\/b><\/p>\n
Krom\u011b \u00faspory \u010dasu a m\u00edsta na disku m\u00e1 tak\u00e9 proudov\u00e1 konverze jednu d\u016fle\u017eitou v\u00fdhodu \u2013 p\u0159etv\u00e1\u0159en\u00ed datab\u00e1ze prob\u00edh\u00e1 bez odstran\u011bn\u00ed st\u00e1vaj\u00edc\u00edho Firebirdu 2.5, co\u017e zna\u010dn\u011b zjednodu\u0161uje n\u00e1vrat k p\u0159edchoz\u00ed verzi p\u0159i ne\u00fasp\u011b\u0161n\u00e9 konverzi (nej\u010dast\u011bji kv\u016fli nedostatku m\u00edsta nebo neo\u010dek\u00e1van\u00e9mu restartu b\u011bhem procesu migrace).<\/p>\n
\u00daspora \u010dasu souvis\u00ed s t\u00edm, \u017ee \"klasick\u00e1\" konverze znamen\u00e1 \"\u010das z\u00e1lohov\u00e1n\u00ed\" plus \"\u010das obnoven\u00ed\". Obnoven\u00ed se skl\u00e1d\u00e1 ze dvou \u010d\u00e1st\u00ed: \u010dten\u00ed dat ze z\u00e1lo\u017en\u00edho souboru a vytv\u00e1\u0159en\u00ed indexu.<\/p>\n
P\u0159i proudov\u00e9 konverzi se celkov\u00fd \u010das z\u00edsk\u00e1 jako \"\u010das z\u00e1lohov\u00e1n\u00ed plus p\u011bt a\u017e deset procent\" a \"\u010das vytv\u00e1\u0159en\u00ed index\u016f\".<\/p>\n
Konkr\u00e9tn\u00ed v\u00fdsledky z\u00e1vis\u00ed na struktu\u0159e datab\u00e1ze, ale pr\u016fm\u011brn\u00e1 doba obnoven\u00ed je p\u0159ibli\u017en\u011b dvojn\u00e1sobkem \u010dasu z\u00e1lohov\u00e1n\u00ed. Proto, pokud vezmeme jako jednotku \u010das z\u00e1lohov\u00e1n\u00ed, m\u00e1 \"klasick\u00e1 konverze\" \u010das t\u0159i jednotky, zat\u00edmco proudov\u00e1 konverze \u010das dv\u011b jednotky. Dal\u0161\u00ed zkr\u00e1cen\u00ed \u010dasu pom\u00e1h\u00e1 zv\u00fd\u0161en\u00ed hodnoty TempCacheLimit.<\/p>\n
Celkov\u011b, proudov\u00e1 konverze prakticky umo\u017e\u0148uje u\u0161et\u0159it 30-40% \u010dasu v porovn\u00e1n\u00ed s postupn\u00fdm z\u00e1lohov\u00e1n\u00edm a obnovov\u00e1n\u00edm.<\/p>\n
M\u00e1te ot\u00e1zky?<\/b><\/p>\n
Pros\u00edm, v\u0161echny dotazy pi\u0161te do koment\u00e1\u0159\u016f nebo je pos\u00edlejte autorovi metodiky a spoluautorovi tohoto \u010dl\u00e1nku \u2013 Vasiliu Sidorovovi, vedouc\u00edmu syst\u00e9mov\u00e9mu in\u017een\u00fdrovi spole\u010dnosti \"iBase\", na adresu bs at ibase ru.<\/p>\n
Pouze registrovan\u00ed u\u017eivatel\u00e9 se mohou \u00fa\u010dastnit ankety. P\u0159ihlaste se<\/a><\/noindex>, pros\u00edm.<\/p>\n
Jakou verzi Firebirdu pou\u017e\u00edv\u00e1te?<\/h2>\n