Awọn alabara Turki wa beere fun wa lati tunto afẹyinti daradara fun ile-iṣẹ data wọn. A n ṣe iru awọn iṣẹ akanṣe ni Russia, ṣugbọn nibi itan naa jẹ diẹ sii nipa ṣiṣe iwadii bii o ṣe dara julọ lati ṣe.
Ti a fun: ibi ipamọ S3 agbegbe kan wa, Veritas NetBackup wa, eyiti o ti gba iṣẹ-ṣiṣe ti o gbooro sii fun gbigbe data si ibi ipamọ ohun, ni bayi pẹlu atilẹyin fun idinku, ati pe iṣoro kan wa pẹlu aaye ọfẹ ni ibi ipamọ agbegbe yii.
Iṣẹ-ṣiṣe: lati ṣe ohun gbogbo ki ilana ti ipamọ awọn ẹda afẹyinti jẹ iyara ati olowo poku.
Lootọ, ṣaaju eyi, ohun gbogbo ni S3 jẹ awọn faili lasan, ati pe iwọnyi jẹ simẹnti pipe ti awọn ẹrọ pataki ti ile-iṣẹ data naa. Iyẹn ni, kii ṣe iṣapeye pupọ, ṣugbọn ohun gbogbo ṣiṣẹ ni ibẹrẹ. Bayi o to akoko lati ro ero rẹ ki o ṣe o tọ.
Aworan naa fihan ohun ti a wa si:
Bi o ti le rii, afẹyinti akọkọ jẹ laiyara (70 Mb/s), ati awọn afẹyinti atẹle ti awọn eto kanna ni iyara pupọ.
Lootọ, siwaju lori awọn alaye diẹ sii wa nipa kini awọn ẹya ti o wa.
Awọn akọọlẹ afẹyinti fun awọn ti o ṣetan lati ka idaji oju-iwe ti idalẹnuNi kikun pẹlu atunwo
Oṣu kejila ọjọ 18, Ọdun 2018 12:09:43 PM — Alaye bpbkar (pid=4452) ohun imuyara firanṣẹ 14883996160 awọn baiti 14883994624 si olupin, iṣapeye 0.0%
Oṣu kejila 18, 2018 12:10:07 PM - Alaye NBCC (pid=23002) StorageServer=PureDisk_rhceph_rawd:s3.cloud.ngn.com.tr; Iroyin=PDDO Awọn iṣiro (oṣan ṣiṣan olona-tẹle ti a lo) fun (NBCC): ṣayẹwo: 14570817 KB, CR firanṣẹ: 1760761 KB, CR firanṣẹ lori FC: 0 KB, dedup: 87.9%, cache alaabo
Full
Oṣu kejila ọjọ 18, Ọdun 2018 12:13:18 PM — Alaye bpbkar (pid=2864) ohun imuyara firanṣẹ 181675008 awọn baiti 14884060160 si olupin, iṣapeye 98.8%
Oṣu kejila ọjọ 18, Ọdun 2018 12:13:40 PM - Alaye NBCC (pid=23527) StorageServer=PureDisk_rhceph_rawd:s3.cloud.ngn.com.tr; Iroyin=PDDO Iṣiro fun (NBCC): ṣayẹwo: 14569706 KB, CR rán: 45145 KB, CR rán lori FC: 0 KB, dedup: 99.7%, cache alaabo
Ti o pọ sii
Oṣu kejila ọjọ 18, Ọdun 2018 12:15:32 PM — Alaye bpbkar (pid=792) ohun imuyara firanṣẹ 9970688 awọn baiti 14726108160 si olupin, iṣapeye 99.9%
Oṣu kejila ọjọ 18, Ọdun 2018 12:15:53 PM - Alaye NBCC (pid=23656) StorageServer=PureDisk_rhceph_rawd:s3.cloud.ngn.com.tr; Iroyin=PDDO Iṣiro fun (NBCC): ṣayẹwo: 14383788 KB, CR rán: 15700 KB, CR rán lori FC: 0 KB, dedup: 99.9%, cache alaabo
Full
Oṣu kejila ọjọ 18, Ọdun 2018 12:18:02 PM — Alaye bpbkar (pid=3496) ohun imuyara firanṣẹ 171746816 awọn baiti 14884093952 si olupin, iṣapeye 98.8%
Oṣu kejila ọjọ 18, Ọdun 2018 12:18:24 PM - Alaye NBCC (pid=23878) StorageServer=PureDisk_rhceph_rawd:s3.cloud.ngn.com.tr; Iroyin=PDDO Iṣiro fun (NBCC): ṣayẹwo: 14569739 KB, CR rán: 34120 KB, CR rán lori FC: 0 KB, dedup: 99.8%, cache alaabo
Kini iṣoro naa
Awọn alabara fẹ lati ṣe awọn afẹyinti ni igbagbogbo bi o ti ṣee ṣe ati tọju wọn ni olowo poku bi o ti ṣee. O dara julọ lati tọju wọn ni olowo poku ni awọn ibi ipamọ ohun elo bii S3, nitori wọn jẹ lawin ni idiyele iṣẹ fun Megabyte lati ibiti o ti le yi afẹyinti pada ni akoko ti o tọ. Nigba ti o ba wa ni ọpọlọpọ awọn afẹyinti, o di ko gidigidi poku, nitori julọ ti awọn ipamọ ti wa ni tẹdo nipasẹ awọn idaako ti kanna data. Ninu ọran HaaS ti awọn ẹlẹgbẹ Turki, ibi ipamọ le jẹ iwuwo nipasẹ isunmọ 80-90%. O han gbangba pe eyi ni ibatan pataki si awọn pato wọn, ṣugbọn Emi yoo dajudaju ka lori o kere ju 50% baba baba.
Lati yanju iṣoro naa, awọn olutaja akọkọ ti pẹ ti ṣe awọn ẹnu-ọna si Amazon S3. Gbogbo awọn ọna wọn wa ni ibamu pẹlu S3 agbegbe niwọn igba ti wọn ṣe atilẹyin Amazon API. Ni ile-iṣẹ data Turki, a ṣe afẹyinti si S3 wa, bakannaa ni T-III "Compressor" ni Russia, niwon eto iṣẹ yii ti ṣiṣẹ daradara fun wa.
Ati pe S3 wa ni ibamu ni kikun pẹlu awọn ọna afẹyinti Amazon S3. Iyẹn ni, gbogbo awọn irinṣẹ afẹyinti ti o ṣe atilẹyin awọn ọna wọnyi gba ọ laaye lati daakọ ohun gbogbo si iru ibi ipamọ “lati inu apoti.”
Veritas NetBackup ṣafikun ẹya CloudCatalyst:
Iyẹn ni, laarin awọn ẹrọ ti o nilo lati ṣe afẹyinti ati ẹnu-ọna ẹnu-ọna, olupin Linux agbedemeji wa nipasẹ eyiti ijabọ afẹyinti lati awọn aṣoju SRK kọja ati pe o ti yọkuro lori fo ṣaaju gbigbe si S3. Ti o ba wa ni iṣaaju awọn afẹyinti 30 ti 20 GB pẹlu titẹkuro, bayi (nitori ibajọra ti awọn ẹrọ) iwọn didun wọn ti di 90% kere si. Enjini yiyọkuro jẹ lilo kanna bii nigba titọju lori awọn disiki deede nipa lilo Netbackup.
Eyi ni ohun ti o ṣẹlẹ ṣaaju olupin agbedemeji:
A ṣe idanwo ati ki o wa si ipari pe nigba ti a ṣe ni awọn ile-iṣẹ data wa, eyi fi aaye pamọ ni ibi ipamọ S3 fun wa ati fun awọn onibara. Gẹgẹbi oniwun ti awọn ile-iṣẹ data iṣowo, nitorinaa, a gba agbara ni ibamu si iwọn didun ti o tẹdo, ṣugbọn o tun jẹ ere pupọ fun wa paapaa - nitori a bẹrẹ ṣiṣe owo lori awọn aaye iwọn diẹ sii ni sọfitiwia, kii ṣe lori yiyalo ohun elo. O dara, ati pe eyi jẹ idinku ninu awọn idiyele inu.
Awọn akọọlẹAwọn iṣẹ 228 (0 Queued 0 Nṣiṣẹ 0 Nduro fun Tun gbiyanju 0 Idaduro 0 Ti ko pari 228 Ti ṣe — 13 ti a yan)
(Aṣẹ ti a lo [13])
Job Id Iru Ipinle Awọn alaye ipo Ipo Ilana Job Iṣeto Iṣeto Iṣẹ Onibara Media Server Ibẹrẹ Akoko Ipari Akoko Ipari Akoko Ipamọ Apapọ Igbiyanju isẹ ti Awọn faili Kilobytes Ọna ọna % Pari (Iroye) Oluṣe PID Idaakọ Obi Job ID KB/Aaya Ibẹrẹ Iṣiṣẹ Ibẹrẹ Ibẹrẹ Iṣiṣẹ Ipari Ikoni Profaili Robot Vault Media ID lati Kọ Iṣipopada Data Paa-Gbalejo Iru Titunto si Iyọkuro Oṣuwọn Ọkọ Imuyara Imudara Imudara tabi Apejọ Pinpin aaye data
- 1358 Snapshot Ti ṣe 0 VMware - NGNCloudADC NBCC Oṣu kejila ọjọ 18, 2018 12:16:19 PM 00:02:18 Oṣu kejila ọjọ 18, 2018 12:18:37 PM STU_DP_S3_**** afẹyinti 1 100% Dec 1358 18 : 2018: 12 PM 16: 27: 00 Disk Imularada Lẹsẹkẹsẹ WIN - *********** 02
1360 Afẹyinti Ti ṣe 0 VMware Kikun NGNCloudADC NBCC Oṣu kejila 18, 2018 12:16:48 PM 00:01:39 Oṣu kejila ọjọ 18, 2018 12:18:27 PM STU_DP_S3_**** afẹyinti 1 14,535,248% 149654 root 100% 23858 Oṣu kejila 1358, 335,098 18:2018:12 PM 16:48:00 Disk Imularada Lẹsẹkẹsẹ WIN-************** 01 39% 0%
1352 Snapshot Ti ṣe 0 VMware - NGNCloudADC NBCC Oṣu kejila ọjọ 18, 2018 12:14:04 PM 00:02:01 Oṣu kejila ọjọ 18, 2018 12:16:05 PM STU_DP_S3_**** afẹyinti 1 100% Dec 1352 18 2018:12 PM 14:14:00 Disk Imularada Lẹsẹkẹsẹ WIN Standard - *********** 01
1354 Afẹyinti Ti ṣe 0 VMware Incremental NGNCloudADC NBCC Oṣu kejila ọjọ 18, 2018 12:14:34 PM 00:01:21 Oṣu kejila ọjọ 18, 2018 12:15:55 PM STU_DP_S3_****afẹyinti 1 14,380,965% 147 root 100 Oṣu kejila 23617, 1352 500,817:18:2018 PM 12:14:34 Disk Imularada Lẹsẹkẹsẹ WIN Standard-************ 00 01% 21%
1347 Snapshot Ti ṣe 0 VMware - NGNCloudADC NBCC Oṣu kejila ọjọ 18, 2018 12:11:45 PM 00:02:08 Oṣu kejila ọjọ 18, 2018 12:13:53 PM STU_DP_S3_**** afẹyinti 1 100% Dec 1347 18 2018:12 PM 11:45:00 Disk Imularada Lẹsẹkẹsẹ WIN Standard - *********** 02
1349 Afẹyinti Ti ṣe 0 VMware Kikun NGNCloudADC NBCC Oṣu kejila 18, 2018 12:12:02 PM 00:01:41 Oṣu kejila ọjọ 18, 2018 12:13:43 PM STU_DP_S3_**** afẹyinti 1 14,535,215% 149653 root 100% 23508 Oṣu kejila 1347, 316,319 18:2018:12 PM 12:02:00 Disk Imularada Lẹsẹkẹsẹ WIN-************** 01 41% 0%
1341 Snapshot Ti ṣe 0 VMware - NGNCloudADC NBCC Oṣu kejila ọjọ 18, 2018 12:05:28 PM 00:04:53 Oṣu kejila ọjọ 18, 2018 12:10:21 PM STU_DP_S3_**** afẹyinti 1 100% Dec 1341 18 2018:12 PM 05:28:00 Disk Imularada Lẹsẹkẹsẹ WIN Standard - *********** 04
1342 Afẹyinti Ti ṣee 0 VMware Full_Rescan NGNCloudADC NBCC Oṣu kejila ọjọ 18, 2018 12:05:47 PM 00:04:24 Oṣu kejila ọjọ 18, 2018 12:10:11 PM STU_DP_S3_1**** afẹyinti 14,535,151 149653% 100 22999 Oṣu kejila 1341, 70,380 18:2018:12 PM 05:47:00 Lẹsẹkẹsẹ Ìgbàpadà Disk Standard WIN-************ 04 24% 0%
1339 Snapshot Ti ṣe 150 VMware - NGNCloudADC NBCC Oṣu kejila ọjọ 18, 2018 11:05:46 AM 00:00:53 Oṣu kejila ọjọ 18, 2018 11:06:39 AM STU_DP_S3_1**** afẹyinti 100 1339% 18:2018% 11:05 AM 46:00:00 Disk Imularada Lẹsẹkẹsẹ WIN Standard - *********** 53
1327 Snapshot Ti ṣe 0 VMware - *******.********.awọsanma NBCC Oṣu kejila ọjọ 17, Ọdun 2018 12:54:42 PM 05:51:38 Oṣu kejila ọjọ 17, Ọdun 2018 6:46:20 PM STU_DP_S3_**** afẹyinti 1 100% root 1327 Dec 17, 2018 12:54:42 PM 05:51:38 Lẹsẹkẹsẹ Ìgbàpadà Disk Standard WIN-************ 0
1328 Afẹyinti Ti ṣe 0 VMware Kikun *******.********.awọsanma NBCC Oṣu kejila ọjọ 17, Ọdun 2018 12:55:10 PM 05:29:21 Oṣu kejila ọjọ 17, Ọdun 2018 6:24:31 PM STU_DP_S3_**** afẹyinti 1 222,602,719 258932 100% 12856 root 1327 11,326 Dec 17, 2018 12:55:10 PM 05:29:21:0:87.9 Lẹsẹkẹsẹ Gbigba Disk Standard WIN-***0 XNUMX%
1136 Snapshot Ti ṣe 0 VMware - *******.********.awọsanma NBCC Oṣu kejila ọjọ 14, Ọdun 2018 4:48:22 PM 04:05:16 Oṣu kejila ọjọ 14, Ọdun 2018 8:53:38 PM STU_DP_S3_**** afẹyinti 1 100% root 1136 Dec 14, 2018 4:48:22 PM 04:05:16 Lẹsẹkẹsẹ Ìgbàpadà Disk Standard WIN-************ 0
1140 Afẹyinti Ti ṣe 0 VMware Full_Scan *******.********.awọsanma NBCC Oṣu kejila ọjọ 14, Ọdun 2018 4:49:14 PM 03:49:58 Oṣu kejila ọjọ 14, Ọdun 2018 8:39:12 PM STU_DP_S3_**** afẹyinti 1 217,631,332 255465 100% 26438 root 1136 15,963 Dec 14, 2018 4:49:14 PM 03:49:58 **** Lẹsẹkẹsẹ Gbigba Disk Standard WIN-% 0. 45.2%
Awọn ohun imuyara faye gba o lati din ijabọ lati òjíṣẹ, nitori Awọn iyipada data nikan ni a gbejade, iyẹn ni, paapaa awọn afẹyinti kikun ko ni gbejade patapata, niwọn igba ti olupin media n gba awọn afẹyinti ni kikun ti o tẹle lati awọn afẹyinti afikun.
Olupin agbedemeji ni ibi ipamọ ti ara rẹ, nibiti o ti kọ “kaṣe” ti data ati ṣetọju ibi ipamọ data fun idinku.
Ẹya ile-iṣẹ ni kikun dabi eyi:
- Olupin oluwa n ṣakoso iṣeto ni, awọn imudojuiwọn, ati bẹbẹ lọ ati pe o wa ninu awọsanma.
- Olupin media (agbedemeji * ẹrọ nix) yẹ ki o wa ni isunmọ si awọn ọna ṣiṣe laiṣe ni awọn ofin iraye si nẹtiwọọki. Nibi, iyọkuro ti awọn afẹyinti lati gbogbo awọn ẹrọ ti a fi pamọ ti ṣe.
- Lori awọn ẹrọ ti a ṣe afẹyinti awọn aṣoju wa ti o firanṣẹ ni gbogbogbo si olupin media nikan ohun ti ko si ni ibi ipamọ rẹ.
Gbogbo rẹ bẹrẹ pẹlu ọlọjẹ kikun - eyi jẹ afẹyinti kikun ti o ni kikun. Ni aaye yii, olupin media gba ohun gbogbo, yọkuro rẹ ati gbe lọ si S3. Iyara si olupin media jẹ kekere, ṣugbọn lati ọdọ rẹ o ga julọ. Idiwọn akọkọ jẹ agbara iširo ti olupin naa.
Awọn afẹyinti atẹle ti wa ni pipe lati oju-ọna ti gbogbo awọn ọna ṣiṣe, ṣugbọn ni otitọ wọn jẹ nkan bi awọn afẹyinti kikun sintetiki. Iyẹn ni, gbigbe gangan ati gbigbasilẹ si olupin media waye nikan ti awọn bulọọki data wọnyẹn ti ko tii pade ni awọn afẹyinti VM ṣaaju. Ati pe awọn bulọọki data wọnyẹn nikan ti hash ko si ni ibi ipamọ data iyọkuro ti olupin media ti gbe ati gbasilẹ ni S3. Ni awọn ọrọ ti o rọrun, eyi jẹ nkan ti a ko rii ni eyikeyi afẹyinti ti VM kan ṣaaju.
Lakoko mimu-pada sipo, olupin media n beere awọn ohun ti o ṣe iyasọtọ ti o yẹ lati S3, tun mu wọn pada ki o gbe wọn lọ si awọn aṣoju IRB, ie. o jẹ dandan lati ṣe akiyesi iwọn didun ti ijabọ lakoko mimu-pada sipo, eyiti yoo jẹ dọgba si iwọn gangan ti data ti a mu pada.
Eyi ni bi o ti ri:
Ati pe eyi ni nkan miiran ti awọn akọọlẹAwọn iṣẹ 169 (0 Queued 0 Nṣiṣẹ 0 Nduro fun Tun gbiyanju 0 Idaduro 0 Ti ko pari 169 Ti ṣe — 1 ti a yan)
Job Id Iru Ipinle Awọn alaye ipo Ipo Ilana Job Iṣeto Iṣeto Iṣẹ Onibara Media Server Ibẹrẹ Akoko Ipari Akoko Ipari Akoko Ipamọ Apapọ Igbiyanju isẹ ti Awọn faili Kilobytes Ọna ọna % Pari (Iroye) Oluṣe PID Idaakọ Obi Job ID KB/Aaya Ibẹrẹ Iṣiṣẹ Ibẹrẹ Ibẹrẹ Iṣiṣẹ Ipari Ikoni Profaili Robot Vault Media ID lati Kọ Iṣipopada Data Paa-Gbalejo Iru Titunto si Iyọkuro Oṣuwọn Ọkọ Imuyara Imudara Imudara tabi Apejọ Pinpin aaye data
- 1372 Mu pada ni 0 NBPR01 Oṣu kejila 19, 2018 1 ọdun 05, 58:00 04 PM 32:19:2018 WIN-************ 1
Iduroṣinṣin data jẹ idaniloju nipasẹ aabo ti S3 funrararẹ - apọju ti o dara wa nibẹ lati daabobo lodi si awọn ikuna ohun elo bii spindle dirafu lile ti o ku.
Olupin media nilo TB 4 ti kaṣe - eyi ni iṣeduro iwọn to kere julọ ti Veritas. Diẹ sii dara julọ, ṣugbọn iyẹn ni ohun ti a ṣe.
Abajade
Nigbati alabaṣepọ kan ju 3 GB sinu S20 wa, a tọju 60 GB, nitori a pese ifiṣura geo-meta ti data. Bayi o wa pupọ diẹ ijabọ, eyiti o dara mejeeji fun ikanni ati fun awọn idiyele ipamọ.
Ni ọran yii, awọn ipa-ọna ti wa ni pipade ti o ti kọja “Internet nla”, ṣugbọn o le wakọ ijabọ nipasẹ VPN L2 lori Intanẹẹti, ṣugbọn o dara lati fi sori ẹrọ olupin media ṣaaju ẹnu-ọna olupese.
Ti o ba nifẹ lati kọ ẹkọ nipa awọn ẹya wọnyi ni awọn ile-iṣẹ data Russia wa tabi ni awọn ibeere nipa imuse ni ile, beere ninu awọn asọye tabi nipasẹ imeeli [imeeli ni idaabobo].
orisun: www.habr.com