Riak Cloud Storage. Chikamu 1: Kumisikidza Riak KV

Riak CS (Cloud Storage) -nyore-kushandisa-software kuronga kuchengetedza chinhu, inomhanya pamusoro peRiak KV. Riak (KV) inzvimbo yakagoverwa yeNoSQL kiyi-value dhatabhesi. Riak CS yakagadzirirwa kupa nyore, kuwanikwa, kugovera kwekuchengetedza kwegore kwechero chiyero, uye inogona kushandiswa kuvaka zvivakwa zvemakore - zveruzhinji uye zvakavanzika - kana sekuchengetedza zvivakwa zvekushandisa zvakanyanya uye masevhisi. Riak CS API inoenderana neAmazon S3 uye inotsigira kugona kugamuchira mishumo pane akasiyana mamiriro.

Ichi chinyorwa ishanduro yemahara yebhuku repamutemo reRiak CS system vhezheni 2.1.1

MuRiak CS yekuchengetedza system, zvinhu zvitatu zvinoshanda zvakabatana, zvinoreva kuti chikamu chimwe nechimwe chinofanirwa kugadzirwa kuti chishande nezvimwe zvikamu:

• Riak (KV) - database system inoshanda seyekupedzisira system.
• Riak CS - Iyo yegore yekuchengetedza layer pamusoro peRiak inopa kuchengetedza uye API kugona, inochengetedza mafaera uye metadata muRiak, uye wozozvigovera kupedzisa vashandisi.
• Stanchion -Inogadzirisa mibvunzo inosanganisira masangano epasirese akasarudzika senge mabhaketi uye vashandisi mune iyo Riak muenzaniso. Semuenzaniso, kugadzira vashandisi, kugadzira kana kudzima mabhakiti.

Pamusoro pezvo, iwe unogona zvakare kugadzirisa iyo S3 mutengi kuti ishandiswe mukutumira mameseji neiyo Riak CS system.

Iwe unofanirwa kuronga kuve neRiak node imwe neimwe yeRiak CS node pane yako system. Riak neRiak CS node dzinogona kumhanyiswa pamichina yemuviri yakasiyana, asi kazhinji zviri nani kumhanyisa imwe Riak node uye imwe Riak CS node pamushini mumwe chete wenyama. Tichifunga kuti muchina mumwe chete wenyama une simba rakakwana rekugutsa zvido zveRiak neRiak CS node, iwe uchaona kuita kurinani nekuda kwekuderedzwa kwetiweki latency.

Kana sisitimu yako iine akati wandei node, gadziriso ndeyekumisa kutaurirana pakati pezvikamu. Mamwe marongero, akadai sekunge achachengetwa mafaera egi, ane default hunhu uye anongoda kuchinjwa kana iwe uchida kushandisa zvisiri-yakajairwa hunhu.

Kugadzirisa zvikamu zvehurongwa. Kumisikidza Riak KV yeCS

Sezvo Riak CS iri application yakavakirwa pamusoro peRiak, zvakakosha kuti utarise kune yako Riak kumisikidzwa paunenge uchimhanyisa Riak CS. Iri gwaro ndereRiak dhizaini yekumisikidza uye gwaro rereferenzi rinotsanangura zvakakosha zvigadziriso paramita.

Usati wamisa, ita shuwa kuti Riak KV neRiak CS yakaiswa pane yega node musumbu rako. Stanchion, kune rumwe rutivi, inofanira kuiswa chete pane imwe node musumbu rose.

Kumashure kweRiak CS

The default backend inoshandiswa naRiak iBitcask, asi Riak CS package inosanganisira yakakosha backend inofanirwa kushandiswa neRiak cluster iyo iri chikamu cheRiak CS system. Iyo yenguva dzose vhezheni ine yakajairwa Multi backend inouya neRiak.

Iwo mabhakiti eRiak akafanana anoshandiswa mukati meRiak CS anoshandisa yechipiri indexes, izvo zvino zvinoda LevelDB backend. Zvimwe zvikamu zveRiak CS system zvinogona kubatsirwa nekushandisa Bticask backend. Kushandiswa kwemuenzaniso Multi backend inosanganisirwa muRiak CS kutora mukana wezvose izvi zvekumashure kuti uwane yakanakisa musanganiswa wekuita uye kushanda. Chikamu chinotevera chinotsanangura maitiro ekugadzirisa zvakanaka Riak kushandisa iyi Multi-backend.

Backend ndiyo ichashandiswa naRiak kuchengetedza data. Riak KV ine akati wandei kumashure mune yayo arsenal: Bitcask, LevelDB, Memory uye Multi.

Pamusoro pezvo, iyo yekuchengetedza kuverenga system inoshandisa Riak MapReduse kuunganidza mafaera mumabhakiti. Izvi zvinoreva kuti iwe unofanirwa kuudza ese maRiak node kwekutsvaga akagadzirira Riak CS mafaera usati waita komputa chengetedzo.

Mamwe maparamita akati wandei anofanirwa kushandurwa kugadzirisa iyo Riak node sechikamu cheRiak CS system, senge IP kero uye IP kero uye chiteshi chekutumira mameseji kuburikidza neProtocol Buffers. Zvimwe zvirongwa zvinogona kuchinjwa kana zvichidiwa. Zvikamu zvinotevera zvinotsanangura maitiro ekugadzirisa node yeRiak kuti ishande sechikamu cheRiak CS system.

Kumisikidza Riak backend

Kutanga, iyo riak.conf kana advanced.config/app.config configuration mafaira anogadziridzwa. Aya mafaera anogona kuwanikwa mukati /etc/riak kana /opt/riak/etc madhairekitori. By default, Riak anoshandisa Bitcask backend. Chinhu chekutanga chatinofanira kuita kushandura faira yekumisikidza nekubvisa mutsara unotevera:

RIAK.CONF

## Delete this line:
storage_backend = bitcask

ADVANCED.CONFIG

{riak_kv,
 [ %% Delete this line: 
{storage_backend, riak_kv_bitcask_backend},
 ]}

APP.CONFIG

{riak_kv, 
  [ %% Delete this line:
    {storage_backend, riak_kv_bitcask_backend},
]}

Tevere, tinoda kuratidza kudiwa kweRiakCS modules yeRiak uye kuraira Riak kushandisa yakagadziriswa backend inopa Riak CS. Isu tinofanirwa kushandisa iyo advanced.config kana app.config faira kune izvi uye kuwedzera zvinotevera sarudzo:

ADVANCED.CONFIG

{eleveldb, [
    {total_leveldb_mem_percent, 30}
    ]},
{riak_kv, [
    %% Other configs
    {add_paths, ["/usr/lib/riak-cs/lib/riak_cs-2.1.1/ebin"]},
    {storage_backend, riak_cs_kv_multi_backend},
    {multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]},
    {multi_backend_default, be_default},
    {multi_backend, [
        {be_default, riak_kv_eleveldb_backend, [
            {data_root, "/var/lib/riak/leveldb"}
        ]},
        {be_blocks, riak_kv_bitcask_backend, [
            {data_root, "/var/lib/riak/bitcask"}
        ]}
    ]},
    %% Other configs
]}

APP.CONFIG

{eleveldb, [
    {total_leveldb_mem_percent, 30}
    ]},
{riak_kv, [
    %% Other configs
    {add_paths, ["/usr/lib/riak-cs/lib/riak_cs-2.1.1/ebin"]},
    {storage_backend, riak_cs_kv_multi_backend},
    {multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]},
    {multi_backend_default, be_default},
    {multi_backend, [
        {be_default, riak_kv_eleveldb_backend, [
            {data_root, "/var/lib/riak/leveldb"}
        ]},
        {be_blocks, riak_kv_bitcask_backend, [
            {data_root, "/var/lib/riak/bitcask"}
        ]}
    ]},
    %% Other configs
]}

Izvo zvakakosha kuti uzive kuti mazhinji eaya maitiro achaenderana nedhairekitori mutsauko kune yako yekushandisa system, saka tevera mirairo zvinoenderana. Semuenzaniso, iyo yekuwedzera_paths sarudzo inofungidzira kuti Riak CS yakaiswa mukati /usr/lib/riak-cs, nepo data_root sarudzo dzinofunga kuti Riak yakaiswa mukati /var/lib. (Cherechedza: Mune yangu yaive add_paths - /usr/lib64/riak-cs/).

Iyi gadziriso inofungidzira kuti Riak CS yakaiswa pamushini mumwechete seRiak. Kana zvisiri, saka iyo package inoda kukopwa kune yakaparadzana host.

Kugadzira kusikwa kwehama

Zvino, isu tinofanirwa kuseta iyo allow_mult paramende kuti iite chokwadi. Tinogona kuwedzera mutsetse mu riak.conf configuration file, kana riak_core chikamu mu advanced.config kana app.config.

RIAK.CONF

buckets.default.allow_mult = true

ADVANCED.CONFIG

{riak_core, [
    %% Other configs
    {default_bucket_props, [{allow_mult, true}]},
    %% Other configs
]}

APP.CONFIG

{riak_core, [
    %% Other configs
    {default_bucket_props, [{allow_mult, true}]},
    %% Other configs
]}

Izvi zvinobvumira Riak kugadzira hama dzinodiwa kuti Riak CS ishande. Kana iwe ukabatana neRiak CS uchishandisa raibhurari yemutengi, usazvinetse: haufanirwe kugadzirisa kusawirirana, sezvo zvese zveRiak CS mashandiro akanyatsoenderana sekutsanangurwa kwazvino.

Sibling inzira yekuchengetedza zvinhu zvakawanda mukiyi imwe kuitira kuti chinhu chacho ive nemhando dzakasiyana pane dzakasiyana node.

Cherechedza: allow_mult
Chero Riak node inotsigirawo Riak CS ichave yakabvumira_mult yakaiswa kune chokwadi nguva dzese. Riak CS ichagadzirisa zvakare kutanga kana kukosha kuri kwenhema.

Kuisa zita reHostname uye IP Kero

Imwe neimwe yeRiak node ine zita, iro rinogona kutsanangurwa mune nodename sarudzo mu riak.conf. Kana iwe uri kushandisa iyo app.config configuration file, unoda kugadzira faira inonzi vm.args mune imwechete dhairekitori se app.config uye tsanangura zita remuenzi uchishandisa -name mureza. Isu tinokurudzira kudoma mazita emanodhi mu @ format. Uye saka kana uine matatu nodes ari kumhanya mumwe mugamuchiri 100.0.0.1, unogona kuvadana [email inodzivirirwa], [email inodzivirirwa]uye [email inodzivirirwa] kana kuti unogona kupa mamwe mazita chaiwo, akadai [email inodzivirirwa], [email inodzivirirwa] zvichingoenda zvakadaro. Muenzaniso uri pazasi unoratidza kushandura zita remuenzi kuti [email inodzivirirwa], iyo inomhanya pane localhost.

RIAK.CONF

 nodename = [email protected] 

VM.ARGS

 -name [email protected]

Iwe unofanirwa kutumidza nodes ese usati watanga iwo uye nekujoinha sumbu.

Setup bvunzo

Zvino kuti ese anodiwa marongero apera, tinogona kuedza kuvhura Riak:

shell

 riak start 

Cherechedza Mhinduro munyaya yangu:

Pano unofanira kumirira zvishoma. Ipapo iwe unogona kutanga kuyedza iyo inomhanya node.

shell

 riak ping

Kana mhinduro iri pong, ipapo Riak iri kumhanya; kana mhinduro iri Node isiri kupindura pings, ipapo chimwe chinhu chakashata.

Cherechedza Mhinduro munyaya yangu:

Kana iyo node isina kutanga zvakanaka, tarisa erlang.log.1 log mu /log dhairekitori yenodhi kana dambudziko richigona kuzivikanwa. Imwe yezvikanganiso zvinowanzoitika invalid_storage_backend. Zvinoratidza kuti nzira inoenda kuraibhurari yeRiak CS mune advanced.config kana muapp.config haina kururama (kana Riak CS haina kuisirwa paseva). Zvisinei nekukanganisa uku, ita shuwa kuti hauna kuchinja kubva pa riak_cs_kv_multi_backend kuenda ku riak_kv_multi_backend.

Kugadzirisa Riak kushandisa protocol buffers

Iyo Riak protocol buffer marongero ari mu riak.conf kana mu riak_api chikamu mune advanced.config kana app.config mafaera, ari mu /etc/riak/ dhairekitori. Nokusingaperi, mugadziri ane IP kero ye127.0.0.1 uye port 8087. Iwe uchada kushandura izvi kana ukaronga kumhanya Riak neRiak CS munzvimbo isiri yenzvimbo. Tsiva 127.0.0.1 neRiak host IP kero uye port 8087 ine yakakodzera.

RIAK.CONF

 listener.protobuf.internal = 10.0.2.10:10001

ADVANCED.CONF

{riak_api, [
    %% Other configs
    {pb, ["10.0.2.10", 10001]},
    %% Other configs
]}

APP.CONFIG

riak_api, [
    %% Other configs
    {pb, ["10.0.2.10", 10001]},
    %% Other configs
]}

Cherechedza:Kukosha kwemutereri.protobuf.yemukati parameter mu riak.conf (kana kukosha kwepb parameter mu advanced.conf/app.config) faira rinofanira kuenderana neukoshi hwe riak_host muRiak CS riak-cs.config uye Stanchion stanchion.conf (kana riak_host zvichiteerana mune advanced .config/app.config) mafaera.

Chinyorwa nezve nhamba yechiteshi
Imwe nhamba yechiteshi inogona kudiwa kana chiteshi ichinetsana nezviteshi zvinoshandiswa neimwe application kana iwe uri kushandisa inoremedza balancer kana proxy server.

Zvinokurudzirwawo kuti vashandisi vaone kuti saizi yeRiak protobuf.backlog (kana mune advanced.config/app.config mafaera iyi pb_backlog) yakaenzana kana yakakura kupfuura pool.request.size yakatsanangurwa Riak CS mu riak-cs. config (kana request_pool_size mune advanced.config/app.conf mafaera).

Kana kukosha kwepool.request.size muRiak CS kwashandurwa, saka kukosha kweprotobuf.backlog kunofanirawo kuvandudzwa muRiak.

Mamwe marongero eRiak

Iyo riak.conf uye advanced.config mafaira anosanganisira mamwe marongero anogadzirisa kusikwa kwemafaira elogi uye kwaanochengetwa. Aya marongero ane default kukosha uye anofanirwa kushanda muzviitiko zvakawanda. Kuti uwane rumwe ruzivo, tinokurudzira kuverenga zvinyorwa zvedu nezve configuration mafaira.

Kumisikidza IP kero yeRiak

Pakumisikidza IP kero yeRiak, unofanirwa kuve nechokwadi chekuti maRiak node ane yakasarudzika IP kero, ungave uri kushanda nenode imwe chete kana kuwedzera mamwe manodhi kuhurongwa. Iyo Riak IP kero iri mu riak.conf kana - kana uri kushandisa iyo app.config faira - mune vm.args configuration faira, iyo iri imomo mu /etc/riak directory (kana /opt/riak/etc/ pane mamwe masisitimu anoshanda).

Pakutanga, mutsetse une iyo Riak IP kero inonongedza kune yemunohost panzvimbo ino:

RIAK.CONF

 nodename = [email protected]

VM.ARGS

 -name [email protected]

Tsiva 127.0.0.1 neyako yaunofarira IP kero kana zita rekutambira reRiak node.

Performance uye Bandwidth Settings

Nekuda kwezvikonzero zvekuita, isu tinokurudzira zvakanyanya kuwedzera kukosha kune iyo Riak gadziriso mafaera riak.conf kana vm.args, iri mu /etc/riak/ kana /opt/riak/etc dhairekitori.

RIAK.CONF

 erlang.max_ports = 65536

VM.ARGS

## This setting should already be present for recent Riak installs.
 -env ERL_MAX_PORTS 65536

Kudzima JavaScript MapReduce

Zvinokurudzirwa kusashandisa legacy JavaScript MapReduce nechero vhezheni yeRiak CS. Nekuda kwezvikonzero zvekuita, unofanira kudzima chaiwo muchina unoita JavaScript MepuDrusa mashandiro nekuisa mu riak.conf configuration faira, kana muchikamu che riak_kv che advanced.conf kana app.config:

RIAK.CONF

 javascript.map_pool_size = 0
 javascript.reduce_pool_size = 0
 javascript.hook_pool_size = 0 

ADVANCED.CONFIG

{riak_kv, [
    %% Other configs
    {map_js_vm_count, 0},
    {reduce_js_vm_count, 0},
    {hook_js_vm_count, 0}
    %% Other configs
]}

APP.CONFIG

{riak_kv, [
    %% Other configs
    {map_js_vm_count, 0},
    {reduce_js_vm_count, 0},
    {hook_js_vm_count, 0}
    %% Other configs
]}

Tevere tinoda kugadzirisa zvakasara zvikamu zveRiak CS system.

Chinyorwa chepakutanga.

Source: www.habr.com