Izi siziri nthabwala, zikuwoneka kuti chithunzichi chikuwonetsa bwino zomwe zili patsamba lino, ndipo pamapeto pake zidzamveka chifukwa chake:
Malinga ndi DB-Engines Ranking, nkhokwe ziwiri zodziwika bwino za NoSQL ndi Cassandra (pano CS) ndi HBase (HB).
Mwachifuniro chake, gulu lathu loyang'anira zonyamula deta ku Sberbank lilipo kale ndipo amagwira ntchito limodzi ndi HB. Panthawi imeneyi, tinaphunzira bwino mphamvu zake ndi zofooka zake ndipo tinaphunzira kuphika. Komabe, kukhalapo kwa njira ina mu mawonekedwe a CS nthawi zonse kumatikakamiza kudzizunza pang'ono ndi kukayikira: kodi tinapanga chisankho choyenera? Komanso, zotsatira , yochitidwa ndi DataStax, adanena kuti CS imamenya HB mosavuta ndi pafupifupi kuphwanya. Kumbali inayi, DataStax ndi chipani chokonda, ndipo simuyenera kutenga mawu awo. Tidasokonezedwanso ndi chidziwitso chaching'ono chokhudza kuyezetsa, kotero tidaganiza zodzipezera tokha mfumu ya BigData NoSql, ndipo zotsatira zomwe zidapezeka zidakhala zosangalatsa kwambiri.
Komabe, musanayambe kupita ku zotsatira za mayesero omwe anachitidwa, m'pofunika kufotokoza mbali zazikulu za masanjidwe a chilengedwe. Chowonadi ndi chakuti CS ingagwiritsidwe ntchito mumalowedwe omwe amalola kutaya deta. Iwo. izi ndi pamene seva imodzi yokha (node) imayang'anira deta ya fungulo linalake, ndipo ngati pazifukwa zina zimalephera, ndiye kuti mtengo wa funguloli udzatayika. Kwa ntchito zambiri izi sizofunikira, koma kwa banki izi ndizosiyana osati lamulo. Kwa ife, ndikofunika kukhala ndi makope angapo a deta kuti asungidwe odalirika.
Choncho, njira yokhayo yogwiritsira ntchito CS mu katatu yobwerezabwereza inaganiziridwa, i.e. Kupanga kwa casespace kunachitika ndi magawo awa:
CREATE KEYSPACE ks WITH REPLICATION = {'class' : 'NetworkTopologyStrategy', 'datacenter1' : 3};
Chotsatira, pali njira ziwiri zowonetsetsa kuti mulingo wofunikira wa kusasinthika. Malamulo onse:
NW + NR > RF
Zomwe zikutanthauza kuti chiwerengero cha zitsimikizo kuchokera ku nodes polemba (NW) kuphatikizapo chiwerengero cha zitsimikizo kuchokera ku nodes powerenga (NR) chiyenera kukhala chachikulu kuposa chobwerezabwereza. Kwa ife, RF = 3, zomwe zikutanthauza kuti zosankha zotsatirazi ndizoyenera:
2 + 2 > 3
3 + 1 > 3
Popeza ndikofunikira kwambiri kuti tisunge deta modalirika momwe tingathere, chiwembu cha 3+1 chinasankhidwa. Kuwonjezera apo, HB imagwira ntchito pa mfundo yofanana, i.e. kuyerekezera koteroko kudzakhala kolungama.
Tiyenera kuzindikira kuti DataStax anachita zosiyana mu phunziro lawo, adayika RF = 1 kwa onse CS ndi HB (kwa otsiriza mwa kusintha makonzedwe a HDFS). Ichi ndi gawo lofunika kwambiri chifukwa zotsatira zake pakuchita kwa CS pankhaniyi ndi zazikulu. Mwachitsanzo, chithunzi chili m'munsichi chikuwonetsa kuwonjezeka kwa nthawi yofunikira kuti muyike deta mu CS:
Apa tikuwona zotsatirazi: ulusi wopikisana kwambiri umalemba zambiri, zimatengera nthawi yayitali. Izi ndizachilengedwe, koma ndikofunikira kuti kuwonongeka kwa magwiridwe antchito a RF=3 ndikokwera kwambiri. Mwa kuyankhula kwina, ngati tilemba ulusi 4 mu matebulo 5 aliyense (20 pamodzi), ndiye RF=3 imataya pafupifupi ka 2 (masekondi 150 a RF=3 motsutsana ndi 75 a RF=1). Koma ngati tiwonjezera katunduyo pokweza deta mu matebulo 8 ndi ulusi 5 aliyense (40 pamodzi), ndiye kuti kutaya kwa RF = 3 kuli kale nthawi 2,7 (masekondi 375 motsutsana ndi 138).
Mwina ichi ndi chinsinsi cha kuyezetsa bwino katundu wopangidwa ndi DataStax kwa CS, chifukwa kwa HB pakuyimilira kwathu kusintha chinthu chobwereza kuchokera ku 2 mpaka 3 sikunakhale ndi zotsatira. Iwo. ma disks sizomwe zimalepheretsa HB kuti tisinthe. Komabe, pali misampha ina yambiri pano, chifukwa ziyenera kudziwidwa kuti mtundu wathu wa HB udasinthidwa pang'ono ndikusinthidwa, malowa ndi osiyana kwambiri, ndi zina zambiri. Ndiyeneranso kuzindikira kuti mwina sindikudziwa momwe ndingakonzekerere CS molondola ndipo pali njira zina zogwirira ntchito, ndipo ndikuyembekeza kuti tidzapeza mu ndemanga. Koma zinthu zoyamba choyamba.
Mayesero onse adachitidwa pagulu la hardware lomwe lili ndi ma seva a 4, iliyonse ili ndi makonzedwe awa:
CPU: Xeon E5-2680 v4 @ 2.40GHz 64 ulusi.
Ma disks: 12 zidutswa SATA HDD
mtundu wa java: 1.8.0_111
CS Mtundu: 3.11.5
magawo a cassandra.ymlnambala_zizindikiro: 256
hinted_handoff_enabled: zoona
hinted_handoff_throttle_in_kb: 1024
max_hints_delivery_threads: 2
hints_directory: /data10/cassandra/hints
hints_flush_period_in_ms: 10000
max_hints_file_size_in_mb: 128
batchlog_replay_throttle_in_kb: 1024
authenticator: AllowAllAuthenticator
authorizer: AllowAllAuthorizer
role_manager: CassandraRoleManager
roles_validity_in_ms: 2000
permits_validity_in_ms: 2000
credentials_validity_in_ms: 2000
wogawa: org.apache.cassandra.dht.Murmur3Partitioner
data_file_directories:
- /data1/cassandra/data # chikwatu chilichonse cha dataN ndi diski yosiyana
- /data2/cassandra/data
- /data3/cassandra/data
- /data4/cassandra/data
- /data5/cassandra/data
- /data6/cassandra/data
- /data7/cassandra/data
- /data8/cassandra/data
commitlog_directory: /data9/cassandra/commitlog
cdc_enabled: zabodza
disk_failure_policy: siyani
commit_failure_policy: siyani
prepared_statements_cache_size_mb:
thrift_prepared_statements_cache_size_mb:
key_cache_size_in_mb:
key_cache_save_period: 14400
mzere_cache_size_mu_mb: 0
row_cache_save_nyengo: 0
counter_cache_size_in_mb:
counter_cache_save_period: 7200
save_caches_directory: /data10/cassandra/saved_caches
commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000
commitlog_segment_size_in_mb: 32
mbewu_wopereka:
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
magawo:
- mbewu: "*,*"
concurrent_reads: 256 # anayesa 64 - palibe kusiyana komwe kunachitika
concurrent_writes: 256 # anayesa 64 - palibe kusiyana komwe kunachitika
concurrent_counter_writes: 256 # anayesa 64 - palibe kusiyana komwe kunazindikirika
concurrent_materialized_view_writes: 32
memtable_heap_space_in_mb: 2048 # anayesa 16 GB - anali pang'onopang'ono
memtable_allocation_type: heap_buffers
index_summary_capacity_mu_mb:
index_summary_resize_interval_mu_minutes: 60
trickle_fsync: zabodza
trickle_fsync_interval_in_kb: 10240
storage_port: 7000
ssl_storage_port: 7001
mverani_adilesi: *
broadcast_address: *
mverani_pa_kulengeza_adilesi: zoona
internode_authenticator: org.apache.cassandra.auth.AllowAllInternodeAuthenticator
start_native_transport: zoona
native_transport_port: 9042
start_rpc: zoona
rpc_address: *
rpc_port: 9160
rpc_keepalive: zoona
rpc_server_mtundu: kulunzanitsa
thrift_framed_transport_size_in_mb: 15
zowonjezera_zosunga zobwezeretsera: zabodza
snapshot_before_compaction: zabodza
auto_snapshot: zoona
column_index_size_in_kb: 64
column_index_cache_size_in_kb: 2
concurrent_compactors: 4
compaction_throughput_mb_per_sec: 1600
sstable_preemptive_open_interval_mu_mb: 50
read_request_timeout_in_ms: 100000
range_request_timeout_in_ms: 200000
kulemba_request_timeout_in_ms: 40000
counter_write_request_timeout_in_ms: 100000
ca_contention_timeout_in_ms: 20000
truncate_request_timeout_in_ms: 60000
request_timeout_in_ms: 200000
slow_query_log_timeout_in_ms: 500
cross_node_timeout: zabodza
endpoint_snitch: GossipingPropertyFileSnitch
dynamic_snitch_update_interval_in_ms: 100
dynamic_snitch_reset_interval_in_ms: 600000
dynamic_snitch_badness_threshold: 0.1
request_scheduler: org.apache.cassandra.scheduler.NoScheduler
seva_encryption_options:
internode_encryption: palibe
client_encryption_options:
thandizira: zabodza
internode_compression: dc
inter_dc_tcp_nodelay: zabodza
tracetype_query_ttl: 86400
tracetype_repair_ttl: 604800
enable_user_defined_functions: zabodza
enable_scripted_user_defined_functions: zabodza
windows_timer_interval: 1
transparent_data_encryption_options:
thandizira: zabodza
tombstone_warn_threshold: 1000
tombstone_failure_threshold: 100000
batch_size_warn_threshold_in_kb: 200
batch_size_fail_threshold_in_kb: 250
unlogged_batch_across_partitions_warn_threshold: 10
compaction_large_partition_warning_threshold_mb: 100
gc_warn_threshold_in_ms: 1000
back_pressure_enabled: zabodza
enable_materialized_views: zoona
enable_sasi_indexes: zoona
Zokonda pa GC:
### Zokonda za CMS-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled
-XX:SurvivorRatio=8
-XX:MaxTenuringThreshold=1
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:CMSWaitDuration=10000
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
-XX:+CMSClassUnloadingEnabled
Chikumbutso cha jvm.options chinapatsidwa 16Gb (tinayesanso 32 Gb, palibe kusiyana komwe kunawonedwa).
Matebulo adapangidwa ndi lamulo:
CREATE TABLE ks.t1 (id bigint PRIMARY KEY, title text) WITH compression = {'sstable_compression': 'LZ4Compressor', 'chunk_length_kb': 64};Mtundu wa HB: 1.2.0-cdh5.14.2 (mu kalasi org.apache.hadoop.hbase.regionserver.HRegion sitinaphatikizepo MetricsRegion zomwe zinatsogolera ku GC pamene chiwerengero cha zigawo chinali choposa 1000 pa RegionServer)
Zosintha zosasinthika za HBasezookeeper.session.timeout: 120000
hbase.rpc.timeout: 2 mphindi
hbase.client.scanner.timeout.period: Mphindi 2 (mphindi)
hbase.master.handler.count: 10
hbase.regionserver.lease.period, hbase.client.scanner.timeout.period: 2 mphindi(s)
hbase.regionserver.handler.count: 160
hbase.regionserver.metahandler.count: 30
hbase.regionserver.logroll.period: 4 maola
hbase.regionserver.maxlogs: 200
hbase.hregion.memstore.flush.size: 1 GiB
hbase.hregion.memstore.block.multiplier: 6
hbase.hstore.compactionThreshold: 5
hbase.hstore.blockingStoreFiles: 200
hbase.hregion.majorcompaction: 1 tsiku(s)
HBase Service Advanced Configuration Snippet (Vavu yachitetezo) ya hbase-site.xml:
hbase.regionserver.wal.codecorg.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec
hbase.master.namespace.init.timeout3600000
hbase.regionserver.optionalcacheflushinterval18000000
hbase.regionserver.thread.compaction.large12
hbase.regionserver.wal.enablecompressiontrue
hbase.hstore.compaction.max.size1073741824
hbase.server.compactchecker.interval.multiplier200
Zosintha za Java za HBase RegionServer:
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:ReservedCodeCacheSize=256m
hbase.snapshot.master.timeoutMillis: 2 min(s)
hbase.snapshot.region.timeout: 2 mphindi(s)
hbase.snapshot.master.timeout.millis: Mphindi 2 (mphindi)
HBase REST Server Max Log Kukula: 100 MiB
HBase REST Server Maximum Log File Backups: 5
HBase Thrift Server Max Log Kukula: 100 MiB
HBase Thrift Server Maximum Log File Backups: 5
Master Max Log Kukula: 100 MiB
Zosungirako Fayilo Yapamwamba Kwambiri: 5
RegionServer Max Log Kukula: 100 MiB
Zosungirako Zosungirako Mafayilo a RegionServer Maximum: 5
HBase Active Master Detection Window: 4 min(s)
dfs.client.hedged.read.threadpool.size: 40
dfs.client.hedged.read.threshold.millis: 10 millisecond(s)
hbase.rest.threads.min: 8
hbase.rest.threads.max: 150
Zofotokozera za Fayilo Yapamwamba: 180000
hbase.thrift.minWorkerThreads: 200
hbase.master.executor.openregion.threads: 30
hbase.master.executor.closeregion.threads: 30
hbase.master.executor.serverops.threads: 60
hbase.regionserver.thread.compaction.ing'ono: 6
hbase.ipc.server.read.threadpool.size: 20
Zigawo Zosuntha: 6
Kukula kwa Client Java Heap mu Bytes: 1 GiB
HBase REST Server Default Gulu: 3 GiB
HBase Thrift Server Default Group: 3 GiB
Java Heap Kukula kwa HBase Master mu Bytes: 16 GiB
Java Heap Kukula kwa HBase RegionServer mu Bytes: 32 GiB
+ ZooKeeper
MaxClientCnxns: 601
MaxSessionTimeout: 120000
Kupanga matebulo:
hbase org.apache.hadoop.hbase.util.RegionSplitter ns:t1 UniformSplit -c 64 -f cf
alter 'ns:t1', {NAME => 'cf', DATA_BLOCK_ENCODING => 'FAST_DIFF', COMPRESSION => 'GZ'}
Pali mfundo imodzi yofunika apa - kufotokozera kwa DataStax sikunena kuti ndi madera angati omwe adagwiritsidwa ntchito popanga matebulo a HB, ngakhale kuti izi ndizofunikira kwambiri pamagulu akuluakulu. Choncho, pa mayesero, kuchuluka = ββ64 kunasankhidwa, komwe kumalola kusunga mpaka 640 GB, i.e. tebulo lapakati.
Pa nthawi ya mayeso, HBase inali ndi matebulo 22 zikwi ndi zigawo 67 zikwi (izi zikanakhala zakupha kwa mtundu wa 1.2.0 ngati si pa chigamba chomwe chatchulidwa pamwambapa).
Tsopano za code. Popeza sizinali zodziwikiratu kuti masinthidwe omwe anali opindulitsa kwambiri pa database inayake, mayeso adachitika mophatikiza zosiyanasiyana. Iwo. m'mayeso ena, matebulo 4 adanyamulidwa nthawi imodzi (mfundo zonse za 4 zidagwiritsidwa ntchito polumikizana). M'mayeso ena tidagwira ntchito ndi matebulo 8 osiyanasiyana. Nthawi zina, kukula kwa batch kunali 100, ena 200 (gawo la batch - onani code pansipa). Kukula kwa data pamtengo wake ndi 10 byte kapena 100 byte (dataSize). Pazonse, zolemba 5 miliyoni zidalembedwa ndikuwerengedwa patebulo lililonse nthawi iliyonse. Nthawi yomweyo, ulusi wa 5 udalembedwa/kuwerengedwa patebulo lililonse (chiwerengero cha ulusi - thNum), iliyonse idagwiritsa ntchito makiyi ake (kuwerengera = 1 miliyoni):
if (opType.equals("insert")) {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
StringBuilder sb = new StringBuilder("BEGIN BATCH ");
for (int i = 0; i < batch; i++) {
String value = RandomStringUtils.random(dataSize, true, true);
sb.append("INSERT INTO ")
.append(tableName)
.append("(id, title) ")
.append("VALUES (")
.append(key)
.append(", '")
.append(value)
.append("');");
key++;
}
sb.append("APPLY BATCH;");
final String query = sb.toString();
session.execute(query);
}
} else {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
StringBuilder sb = new StringBuilder("SELECT * FROM ").append(tableName).append(" WHERE id IN (");
for (int i = 0; i < batch; i++) {
sb = sb.append(key);
if (i+1 < batch)
sb.append(",");
key++;
}
sb = sb.append(");");
final String query = sb.toString();
ResultSet rs = session.execute(query);
}
}
Chifukwa chake, magwiridwe antchito ofanana adaperekedwa kwa HB:
Configuration conf = getConf();
HTable table = new HTable(conf, keyspace + ":" + tableName);
table.setAutoFlush(false, false);
List<Get> lGet = new ArrayList<>();
List<Put> lPut = new ArrayList<>();
byte[] cf = Bytes.toBytes("cf");
byte[] qf = Bytes.toBytes("value");
if (opType.equals("insert")) {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
lPut.clear();
for (int i = 0; i < batch; i++) {
Put p = new Put(makeHbaseRowKey(key));
String value = RandomStringUtils.random(dataSize, true, true);
p.addColumn(cf, qf, value.getBytes());
lPut.add(p);
key++;
}
table.put(lPut);
table.flushCommits();
}
} else {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
lGet.clear();
for (int i = 0; i < batch; i++) {
Get g = new Get(makeHbaseRowKey(key));
lGet.add(g);
key++;
}
Result[] rs = table.get(lGet);
}
}
Popeza mu HB kasitomala ayenera kusamalira kugawa yunifolomu kwa deta, ntchito yofunika kwambiri ya salting imawoneka motere:
public static byte[] makeHbaseRowKey(long key) {
byte[] nonSaltedRowKey = Bytes.toBytes(key);
CRC32 crc32 = new CRC32();
crc32.update(nonSaltedRowKey);
long crc32Value = crc32.getValue();
byte[] salt = Arrays.copyOfRange(Bytes.toBytes(crc32Value), 5, 7);
return ArrayUtils.addAll(salt, nonSaltedRowKey);
}
Tsopano gawo losangalatsa kwambiri - zotsatira:
Zomwezo mu mawonekedwe a graph:
Ubwino wa HB ndi wodabwitsa kwambiri kotero kuti pali kukayikira kuti pali mtundu wina wa botolo pakukhazikitsa kwa CS. Komabe, Googling ndi kufufuza magawo odziwika bwino (monga concurrent_writes or memtable_heap_space_in_mb) sikunafulumizitse zinthu. Panthawi imodzimodziyo, zipikazo zimakhala zoyera ndipo sizimalumbira chilichonse.
Detayo inagawidwa mofanana m'malo onse, ziwerengero zochokera kumagulu onse zinali zofanana.
Izi ndi momwe ziwerengero zatebulo zimawonekera kuchokera kumodzi mwa nodeMalo ofunika: ks
Werengani Chiwerengero: 9383707
Werengani Kuchedwa: 0.04287025042448576 ms
Lembani Chiwerengero: 15462012
Lembani Kuchedwa: 0.1350068438699957 ms
Zomwe Zili Patsogolo: 0
Gulu: t1
Chiwerengero chokhazikika: 16
Malo ogwiritsidwa ntchito (amoyo): 148.59 MiB
Malo ogwiritsidwa ntchito (onse): 148.59 MiB
Malo ogwiritsidwa ntchito ndi zithunzithunzi (zonse): 0 mabayiti
Memory of mulu yogwiritsidwa ntchito (yonse): 5.17 MiB
SSTable Compression Ratio: 0.5720989576459437
Chiwerengero cha magawo (chiwerengero): 3970323
Chiwerengero cha ma cell okumbukira: 0
Kukula kwa data yokumbukira: 0 byte
Memtable off mulu kukumbukira ntchito: 0 mabayiti
Chiwerengero cha kusintha kokumbukika: 5
Chiwerengero chowerengedwa m'deralo: 2346045
Kuchedwerako kowerengera kwanuko: NaN ms
Chiwerengero cholembera m'deralo: 3865503
Kuchedwa kulemba kwanuko: NaN ms
Zomwe zikuyembekezeredwa: 0
Peresenti yokonzedwa: 0.0
Zosefera za Bloom zabodza: ββ25
Chiyerekezo chabodza cha Bloom: 0.00000
Malo osefa a Bloom omwe amagwiritsidwa ntchito: 4.57 MiB
Bloom fyuluta pa mulu kukumbukira ntchito: 4.57 MiB
Chidule cha chidule cha kukumbukira mulu wogwiritsidwa ntchito: 590.02 KB
Kuphatikizika kwa metadata kuchokera mulu wa kukumbukira komwe kumagwiritsidwa ntchito: 19.45 KB
Gawo locheperako ma byte: 36
Gawo lalikulu la ma byte: 42
Gawo lophatikizika limatanthawuza ma byte: 42
Avereji ya maselo amoyo pagawo lililonse (mphindi zisanu zapitazi): NaN
Kuchuluka kwa maselo amoyo pagawo lililonse (mphindi zisanu zapitazi): 0
Pafupifupi miyala yapamanda pa chidutswa chilichonse (mphindi zisanu zapitazi): NaN
Miyala yam'manda yochuluka pagawo lililonse (mphindi zisanu zapitazi): 0
Kusintha kwasintha: 0 mabayiti
Kuyesera kuchepetsa kukula kwa batch (ngakhale kutumiza payekha) kunalibe zotsatira, kunangowonjezereka. N'zotheka kuti kwenikweni izi ndizochita bwino kwambiri kwa CS, popeza zotsatira zopezeka kwa CS ndizofanana ndi zomwe zimapezedwa ku DataStax - pafupifupi mazana zikwi za ntchito pamphindi. Kuphatikiza apo, tikayang'ana kugwiritsa ntchito zida, tiwona kuti CS imagwiritsa ntchito ma CPU ambiri ndi ma disks:
Chithunzichi chikuwonetsa kugwiritsidwa ntchito kwa mayeso onse motsatizana pama database onse awiri.
Ponena za mwayi wowerenga kwambiri wa HB. Apa mutha kuwona kuti pazosungidwa zonse ziwiri, kugwiritsa ntchito disk pakuwerenga kumakhala kotsika kwambiri (mayeso owerengera ndi gawo lomaliza la kuyesa kwa database iliyonse, mwachitsanzo kwa CS izi zikuchokera ku 15:20 mpaka 15:40). Pankhani ya HB, chifukwa chake ndi chodziwikiratu - zambiri za data zimapachikidwa pamtima, mu memstore, ndipo zina zimasungidwa mu blockcache. Ponena za CS, sizikudziwika bwino momwe zimagwirira ntchito, koma kubwezeretsanso disk sikukuwoneka, koma ngati, kuyesa kunapangidwa kuti cache row_cache_size_in_mb = 2048 ndikuyika caching = {'makiyi': 'ALL', 'rows_per_partition': '2000000'}, koma izi zidapangitsa kuti ziipire pang'ono.
Ndikoyeneranso kutchulanso mfundo yofunika yokhudza kuchuluka kwa zigawo mu HB. Kwa ife, mtengowo unatchulidwa kuti 64. Ngati muchepetse ndikupangitsa kuti ikhale yofanana, mwachitsanzo, 4, ndiye powerenga, liwiro limatsika ndi 2 nthawi. Chifukwa chake ndi chakuti memstore idzadzaza mwachangu ndipo mafayilo amasinthidwa pafupipafupi ndipo powerenga, mafayilo ochulukirapo adzafunika kukonzedwa, yomwe ndi ntchito yovuta kwambiri ya HB. M'mikhalidwe yeniyeni, izi zitha kuthandizidwa poganiza kudzera munjira yopangiratu ndi kuphatikizira; makamaka, timagwiritsa ntchito chida chodzilembera chomwe chimatolera zinyalala ndikumakanikiza ma HFiles kumbuyo nthawi zonse. Ndizotheka kuti pamayesero a DataStax adagawa chigawo chimodzi chokha pa tebulo (zomwe sizolondola) ndipo izi zingamveke bwino chifukwa chake HB inali yotsika kwambiri pamayesero awo owerengera.
Zotsatira zotsatirazi zikuchokera ku izi. Poganiza kuti palibe zolakwa zazikulu zomwe zidapangidwa panthawi yoyesedwa, ndiye Cassandra amawoneka ngati colossus ndi mapazi a dongo. Mwatsatanetsatane, pamene akulinganiza mwendo umodzi, monga momwe zilili pachithunzichi kumayambiriro kwa nkhaniyo, amasonyeza zotsatira zabwino, koma pomenyana pansi pa mikhalidwe yomweyi amalephera. Nthawi yomweyo, poganizira kutsika kwa CPU pazida zathu, tidaphunzira kubzala ma RHB a RegionServer pamunthu aliyense ndipo potero kuchulukitsa magwiridwe antchito. Iwo. Poganizira za kagwiritsidwe ntchito ka zinthu, mkhalidwe wa CS ndi woipa kwambiri.
Zoonadi, mayeserowa ndi opangidwa ndithu ndipo kuchuluka kwa deta yomwe idagwiritsidwa ntchito pano ndi yochepa. Ndizotheka kuti ngati titasinthira ku ma terabytes, zinthu zitha kukhala zosiyana, koma kwa HB titha kukweza ma terabytes, kwa CS izi zidakhala zovuta. Nthawi zambiri idaponya OperationTimedOutException ngakhale ndi ma voliyumu awa, ngakhale magawo odikirira kuyankha anali atawonjezeka kangapo poyerekeza ndi osakhazikika.
Ndikuyembekeza kuti kupyolera mu mgwirizano tidzapeza zovuta za CS ndipo ngati tingathe kufulumizitsa, ndiye kumapeto kwa positi ndithudi ndidzawonjezera zambiri za zotsatira zomaliza.
UPD: Chifukwa cha upangiri wa anzanga, ndidakwanitsa kufulumizitsa kuwerenga. Anali:
159 ops (magome 644, mitsinje 4, gulu 5).
Zowonjezera:
.withLoadBalancingPolicy(NewTokenAwarePolicy(DCAwareRoundRobinPolicy.builder().build()))
Ndipo ndidasewera ndi kuchuluka kwa ulusi. Zotsatira zake ndi izi:
4 matebulo, 100 ulusi, batch = 1 (chidutswa ndi chidutswa): 301 ops
4 matebulo, 100 ulusi, batch = 10: 447 ops
4 matebulo, 100 ulusi, batch = 100: 625 ops
Pambuyo pake ndigwiritsanso ntchito malangizo ena okonzekera, ndikuyesa mayeso onse ndikuwonjezera zotsatira kumapeto kwa positi.
Source: www.habr.com