ʻAʻole kēia he hoʻohenehene, me he mea lā e hōʻike pololei ana kēia kiʻi kikoʻī i ke ʻano o kēia mau ʻikepili, a ma ka hopena e maopopo ke kumu:
Wahi a DB-Engines Ranking, ʻo nā ʻikepili kolamu NoSQL kaulana loa ʻo Cassandra (ma hope o CS) a me HBase (HB).
Ma ka makemake o ka hopena, ua loaʻa i kā mākou hui hoʻokele hoʻoili ʻikepili ma Sberbank a hana pū me HB. I loko o kēia manawa, ua aʻo maikaʻi mākou i kona ikaika a me kona nāwaliwali a aʻo i ke kuke ʻana. Eia naʻe, ʻo ka loaʻa ʻana o kahi ʻano ʻē aʻe i ke ʻano o CS e koi mau iā mākou e hōʻeha iki iā mākou iho me nā kānalua: ua hana mākou i ka koho kūpono? Eia kekahi, nā hopena , hana ʻia e DataStax, ua ʻōlelo lākou ua maʻalahi ʻo CS i ka HB me ka ʻaneʻane o ka helu ʻana. Ma ka ʻaoʻao ʻē aʻe, he pāʻina hoihoi ʻo DataStax, a ʻaʻole pono ʻoe e lawe i kā lākou ʻōlelo no ia mea. Pihoihoi pū mākou i ka liʻiliʻi o ka ʻike e pili ana i nā kūlana hoʻāʻo, no laila ua hoʻoholo mākou e ʻike iā mākou iho ʻo wai ke aliʻi o BigData NoSql, a ua lilo nā hopena i mea hoihoi loa.
Eia nō naʻe, ma mua o ka neʻe ʻana i nā hopena o nā hoʻokolohua i hana ʻia, pono e wehewehe i nā ʻano koʻikoʻi o nā hoʻonohonoho kaiapuni. ʻO ka ʻoiaʻiʻo hiki ke hoʻohana ʻia ʻo CS i kahi ʻano e ʻae ai i ka nalowale o ka ʻikepili. ʻO kēlā mau mea. ʻO kēia ka manawa hoʻokahi wale nō kikowaena (node) ke kuleana no ka ʻikepili o kekahi kī, a inā no kekahi kumu e hāʻule, a laila e nalowale ka waiwai o kēia kī. No nā hana he nui ʻaʻole koʻikoʻi kēia, akā no ka ʻoihana panakō ʻo kēia ka mea ʻokoʻa ma mua o ke kānāwai. I kā mākou hihia, he mea nui e loaʻa i kekahi mau kope o ka ʻikepili no ka mālama pono ʻana.
No laila, ua noʻonoʻo ʻia ke ʻano hana CS ma ke ʻano hoʻopiʻi triple, ʻo ia hoʻi. Ua hana ʻia ka hana ʻana o ka casespace me nā ʻāpana aʻe:
CREATE KEYSPACE ks WITH REPLICATION = {'class' : 'NetworkTopologyStrategy', 'datacenter1' : 3};
A laila, ʻelua mau ala e hōʻoia ai i ka pae i makemake ʻia o ke kūlike. Rula maʻamau:
NW + NR > RF
ʻO ia hoʻi, ʻoi aku ka nui o nā hōʻoia mai nā nodes i ka wā e kākau ai (NW) a me ka helu o nā hōʻoia mai nā nodes i ka wā e heluhelu ai (NR) ma mua o ka helu replication. I kā mākou hihia, RF = 3, ʻo ia ka mea kūpono nā koho aʻe:
2 + 2 > 3
3 + 1 > 3
No ka mea he mea koʻikoʻi ia no mākou e mālama i ka ʻikepili i hiki ke hilinaʻi ʻia, ua koho ʻia ka papahana 3 + 1. Eia kekahi, hana ʻo HB ma kahi loina like, ʻo ia hoʻi. e oi aku ka pololei o ia hoohalike ana.
Pono e hoʻomaopopo ʻia ua hana ʻo DataStax i ka ʻokoʻa i kā lākou aʻo ʻana, ua hoʻonohonoho lākou i ka RF = 1 no CS a me HB (no ka mea hope ma ka hoʻololi ʻana i nā hoʻonohonoho HDFS). He mea koʻikoʻi loa kēia no ka mea he nui ka hopena i ka hana CS i kēia hihia. No ka laʻana, hōʻike ke kiʻi ma lalo nei i ka piʻi ʻana o ka manawa e pono ai e hoʻouka i ka ʻikepili i CS:
Maʻaneʻi mākou e ʻike ai i kēia: ʻoi aku ka nui o nā kaula hoʻokūkū e kākau i ka ʻikepili, ʻoi aku ka lōʻihi. He mea maʻamau kēia, akā he mea nui e ʻoi aku ka kiʻekiʻe o ka hoʻohaʻahaʻa hana no RF=3. I nā huaʻōlelo ʻē aʻe, inā mākou e kākau i nā kaula 4 i loko o nā papa 5 i kēlā me kēia (20 ka nui), a laila nalowale ʻo RF=3 ma kahi o 2 mau manawa (150 kekona no RF=3 me 75 no RF=1). Akā inā mākou e hoʻonui i ka ukana ma ka hoʻouka ʻana i nā ʻikepili i loko o nā papa 8 me 5 mau kaula i kēlā me kēia (40 ka nui), a laila ʻo ka nalowale o RF=3 ua 2,7 mau manawa (375 kekona me 138).
Malia paha ʻo kēia ka mea huna o ka hoʻāʻo hoʻokō kūleʻa i hana ʻia e DataStax no CS, no ka mea, no ka HB ma kā mākou kū ʻana i ka hoʻololi ʻana i ka helu replication mai 2 a 3 ʻaʻohe hopena. ʻO kēlā mau. ʻAʻole nā disks ka bottleneck HB no kā mākou hoʻonohonoho. Eia nō naʻe, he nui nā pōʻino ʻē aʻe ma ʻaneʻi, no ka mea, pono e hoʻomaopopo ʻia ʻo kā mākou mana o HB i hoʻopaʻa iki ʻia a hoʻololi ʻia, ʻokoʻa loa nā kaiapuni, etc. He mea kūpono hoʻi e ʻike ʻaʻole paha wau i ʻike pehea e hoʻomākaukau pono ai i ka CS a aia kekahi mau ala ʻoi aku ka maikaʻi e hana pū me ia, a ke manaʻo nei au e ʻike mākou ma nā ʻōlelo. Akā ʻo ka mea mua.
Ua hana ʻia nā hoʻāʻo a pau ma kahi pūʻulu hāmeʻa i loaʻa i nā kikowaena 4, kēlā me kēia me kēia hoʻonohonoho:
CPU: Xeon E5-2680 v4 @ 2.40GHz 64 pae.
Nā diski: 12 ʻāpana SATA HDD
java mana: 1.8.0_111
CS Version: 3.11.5
cassandra.yml nā ʻāpanahelu_tokens: 256
hinted_handoff_enabled: ʻoiaʻiʻo
hinted_handoff_throttle_in_kb: 1024
max_hints_delivery_threads: 2
kuhikuhi_ kuhikuhi: /data10/cassandra/hints
hints_flush_period_in_ms: 10000
max_hints_file_size_in_mb: 128
batchlog_replay_throttle_in_kb: 1024
mea hōʻoia: AllowAllAuthenticator
mea mana: AllowAllAuthorizer
role_manager: CassandraRoleManager
roles_validity_in_ms: 2000
mana_mana_ma_ms: 2000
credentials_validity_in_ms: 2000
mea hoʻokaʻawale: org.apache.cassandra.dht.Murmur3Partitioner
ʻikepili_file_directories:
- /data1/cassandra/data # kēlā me kēia papa kuhikuhi dataN he disk kaʻawale
- /data2/cassandra/data
- /data3/cassandra/data
- /data4/cassandra/data
- /data5/cassandra/data
- /data6/cassandra/data
- /data7/cassandra/data
- /data8/cassandra/data
commitlog_directory: /data9/cassandra/commitlog
cdc_enabled: wahaheʻe
disk_failure_policy: kū
commit_failure_policy: kū
prepared_statements_cache_size_mb:
thrift_prepared_statements_cache_size_mb:
key_cache_size_in_mb:
key_cache_save_period: 14400
row_cache_size_in_mb: 0
row_cache_save_period: 0
counter_cache_size_in_mb:
counter_cache_save_period: 7200
saved_caches_directory: /data10/cassandra/saved_caches
commitlog_sync: manawa
commitlog_sync_period_in_ms: 10000
commitlog_segment_size_in_mb: 32
mea hoʻolako hua:
- inoa_papa: org.apache.cassandra.locator.SimpleSeedProvider
Kaumaha:
- nā hua: "*,*"
concurrent_reads: 256 # hoʻāʻo 64 - ʻaʻohe ʻokoʻa i ʻike ʻia
concurrent_writes: 256 # hoʻāʻo 64 - ʻaʻohe ʻokoʻa i ʻike ʻia
concurrent_counter_writes: 256 # hoʻāʻo 64 - ʻaʻohe ʻokoʻa i ʻike ʻia
concurrent_materialized_view_writes: 32
memtable_heap_space_in_mb: 2048 # hoʻāʻo 16 GB - ʻoi aku ka lohi
memtable_allocation_type: heap_buffers
index_summary_capacity_in_mb:
index_summary_resize_interval_in_minutes: 60
trickle_fsync: wahaheʻe
trickle_fsync_interval_in_kb: 10240
waihona_puka: 7000
ssl_storage_port: 7001
listen_address: *
leka uila: *
listen_on_broadcast_address: ʻoiaʻiʻo
internode_authenticator: org.apache.cassandra.auth.AllowAllInternodeAuthenticator
start_native_transport: ʻoiaʻiʻo
native_transport_port: 9042
start_rpc: ʻoiaʻiʻo
rpc_address: *
rpc_port: 9160
rpc_keepalive: ʻoiaʻiʻo
rpc_server_type: sync
thrift_framed_transport_size_in_mb: 15
incremental_backups: wahahee
snapshot_before_compaction: wahahee
auto_snapshot: ʻoiaʻiʻo
kolamu_index_size_in_kb: 64
column_index_cache_size_in_kb: 2
concurrent_compactors: 4
compaction_throughput_mb_per_sec: 1600
sstable_preemptive_open_interval_in_mb: 50
read_request_timeout_in_ms: 100000
range_request_timeout_in_ms: 200000
kākau_noi_manawa_ma_ms: 40000
counter_write_request_timeout_in_ms: 100000
cas_contention_timeout_in_ms: 20000
truncate_request_timeout_in_ms: 60000
request_timeout_in_ms: 200000
slow_query_log_timeout_in_ms: 500
cross_node_timeout: wahahee
endpoint_snitch: GossipingPropertyFileSnitch
dynamic_snitch_update_interval_in_ms: 100
dynamic_snitch_reset_interval_in_ms: 600000
dynamic_snitch_badness_threshold: 0.1
noi_scheduler: org.apache.cassandra.scheduler.NoScheduler
server_encryption_options:
internode_encryption: ʻaʻohe
client_encryption_options:
hiki: wahahee
internode_compression: dc
inter_dc_tcp_nodelay: wahaheʻe
tracetype_query_ttl: 86400
tracetype_repair_ttl: 604800
enable_user_defined_functions: wahahee
enable_scripted_user_defined_functions: wahahee
windows_timer_interval: 1
transparent_data_encryption_options:
hiki: wahahee
kupapaʻu_warn_threshold: 1000
paepae_puka_kupapau: 100000
ka nui o ka nui o ka hui: 200
pūʻulu_fail_threshold_in_kb: 250
unlogged_batch_across_partitions_warn_threshold: 10
compaction_large_partition_warning_threshold_mb: 100
gc_warn_threshold_in_ms: 1000
back_pressure_enabled: wahahee
enable_materialized_views: ʻoiaʻiʻo
enable_sasi_indexes: ʻoiaʻiʻo
Nā koho GC:
### Nā hoʻonohonoho CMS-XX:+HoʻohanaParNewGC
-XX:+HoʻohanaConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled
-XX:SurvivorRatio=8
-XX:MaxTenuringThreshold=1
-XX:CMSInitiatingOccupancyFraction=75
-XX:+HoʻohanaCMSInitiatingOccupancyOnly
-XX:CMSWaitDuration=10000
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecord mau
-XX:+CMSClassUnloadingEnabled
Ua hoʻokaʻawale ʻia ka hoʻomanaʻo jvm.options 16Gb (ua hoʻāʻo pū mākou i 32 Gb, ʻaʻohe ʻokoʻa i ʻike ʻia).
Ua hana ʻia nā papa me ke kauoha:
CREATE TABLE ks.t1 (id bigint PRIMARY KEY, title text) WITH compression = {'sstable_compression': 'LZ4Compressor', 'chunk_length_kb': 64};Manaʻo HB: 1.2.0-cdh5.14.2 (ma ka papa org.apache.hadoop.hbase.regionserver.HRegion ua kāpae mākou i ka MetricsRegion i alakaʻi iā GC inā ʻoi aku ka nui o nā ʻāpana ma mua o 1000 ma RegionServer)
Nā ʻāpana HBase paʻa ʻolezookeeper.session.timeout: 120000
hbase.rpc.timeout: 2 minuke (mau)
hbase.client.scanner.timeout.period: 2 minuke
hbase.master.handler.count: 10
hbase.regionserver.lease.period, hbase.client.scanner.timeout.period: 2 minute(s)
hbase.regionserver.handler.count: 160
hbase.regionserver.metahandler.count: 30
hbase.regionserver.logroll.period: 4 hola(s)
hbase.regionserver.maxlogs: 200
hbase.hregion.memstore.flush.size: 1 GiB
hbase.hregion.memstore.block.multiplier: 6
hbase.hstore.compactionThreshold: 5
hbase.hstore.blockingStoreFiles: 200
hbase.hregion.majorcompaction: 1 lā (mau)
HBase Service Advanced Configuration Snippet (Safety Valve) no hbase-site.xml:
hbase.regionserver.wal.codecorg.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec
hbase.master.namespace.init.timeout3600000
hbase.regionserver.optionalcacheflushinterval18000000
hbase.regionserver.thread.compaction.large12
hbase.regionserver.wal.enablecompressiontrue
hbase.hstore.compaction.max.size1073741824
hbase.server.compactchecker.interval.multiplier200
Nā koho hoʻonohonoho Java no HBase RegionServer:
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:ReservedCodeCacheSize=256m
hbase.snapshot.master.timeoutMillis: 2 mau minuke
hbase.snapshot.region.timeout: 2 mau minuke
hbase.snapshot.master.timeout.millis: 2 mau minuke
HBase REST Server Max Log Nui: 100 MiB
ʻO HBase REST Server Nā waihona waihona waihona kiʻekiʻe loa: 5
ʻO ka nui o ka moʻolelo ʻo HBase Thrift Server: 100 MiB
ʻO HBase Thrift Server ka nui o nā waihona waihona waihona: 5
Nui Nui Log Nui: 100 MiB
Nā waihona waihona waihona kiʻekiʻe loa: 5
Ka nui o ka Loko Loko ʻĀpanaServer: 100 MiB
ʻO nā hoʻihoʻi waihona waihona moʻohelu kiʻekiʻe loa o RegionServer: 5
HBase Active Master Detection Window: 4 minuke (mau)
dfs.client.hedged.read.threadpool.size: 40
dfs.client.hedged.read.threshold.millis: 10 millisecond(s)
hbase.rest.threads.min: 8
hbase.rest.threads.max: 150
ʻO nā mea wehewehe faila kaʻina kiʻekiʻe: 180000
hbase.thrift.minWorkerThreads: 200
hbase.master.executor.openregion.threads: 30
hbase.master.executor.closeregion.threads: 30
hbase.master.executor.serverops.threads: 60
hbase.regionserver.thread.compaction.small: 6
hbase.ipc.server.read.threadpool.size: 20
Nā Loiloi Hoʻoneʻe Māhele: 6
Ka nui o ka puʻu Java mea kūʻai ma nā Paita: 1 GiB
Pūʻulu Default Server HBase REST: 3 GiB
Pūʻulu Default Server HBase Thrift: 3 GiB
ʻO Java Heap Nui o HBase Master ma Bytes: 16 GiB
Ka nui o Java Heap o HBase RegionServer ma Bytes: 32 GiB
+Kānāwai
maxClientCnxns: 601
maxSessionTimeout: 120000
Ke hana nei i nā papa:
hbase org.apache.hadoop.hbase.util.RegionSplitter ns:t1 UniformSplit -c 64 -f cf
hoʻololi 'ns:t1', {NAME => 'cf', DATA_BLOCK_ENCODING => 'FAST_DIFF', COMPRESSION => 'GZ'}
Aia kekahi wahi koʻikoʻi ma ʻaneʻi - ʻaʻole ʻōlelo ka wehewehe ʻana o DataStax i ka nui o nā wahi i hoʻohana ʻia e hana i nā papa HB, ʻoiai he mea koʻikoʻi kēia no nā puke nui. No laila, no nā hoʻokolohua, koho ʻia ka nui = 64, e hiki ai ke mālama i ka 640 GB, ʻo ia hoʻi. pākaukau ʻano nui.
I ka manawa o ka hoʻāʻo ʻana, loaʻa iā HBase he 22 tausani mau papa a me 67 tausani mau ʻāina (he mea make kēia no ka mana 1.2.0 inā ʻaʻole no ka patch i ʻōlelo ʻia ma luna).
I kēia manawa no ke code. No ka mea, ʻaʻole maopopo ka mea hoʻonohonoho i ʻoi aku ka maikaʻi no kahi waihona kikoʻī, ua hana ʻia nā hoʻokolohua ma nā hui like ʻole. ʻO kēlā mau mea. i kekahi mau ho'āʻo, ua hoʻouka ʻia nā papa 4 i ka manawa like (ua hoʻohana ʻia nā 4 nodes a pau no ka pilina). Ma nā hoʻokolohua ʻē aʻe ua hana mākou me 8 mau papa ʻokoʻa. I kekahi mau hihia, he 100 ka nui o ka puʻupuʻu, a ʻo nā mea ʻē aʻe he 200 (ka hoʻohālikelike hui - e ʻike i ke code ma lalo). ʻO ka nui ʻikepili no ka waiwai he 10 bytes a i ʻole 100 bytes (dataSize). ʻO ka huina, 5 miliona mau moʻolelo i kākau ʻia a heluhelu ʻia i kēlā me kēia papa i kēlā me kēia manawa. I ka manawa like, ua kākau ʻia/heluhelu ʻia nā pae 5 i kēlā me kēia pākaukau (helu helu - thNum), ua hoʻohana kēlā me kēia i kāna mau kī ponoʻī (helu = 1 miliona):
if (opType.equals("insert")) {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
StringBuilder sb = new StringBuilder("BEGIN BATCH ");
for (int i = 0; i < batch; i++) {
String value = RandomStringUtils.random(dataSize, true, true);
sb.append("INSERT INTO ")
.append(tableName)
.append("(id, title) ")
.append("VALUES (")
.append(key)
.append(", '")
.append(value)
.append("');");
key++;
}
sb.append("APPLY BATCH;");
final String query = sb.toString();
session.execute(query);
}
} else {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
StringBuilder sb = new StringBuilder("SELECT * FROM ").append(tableName).append(" WHERE id IN (");
for (int i = 0; i < batch; i++) {
sb = sb.append(key);
if (i+1 < batch)
sb.append(",");
key++;
}
sb = sb.append(");");
final String query = sb.toString();
ResultSet rs = session.execute(query);
}
}
No laila, ua hāʻawi ʻia nā hana like no HB:
Configuration conf = getConf();
HTable table = new HTable(conf, keyspace + ":" + tableName);
table.setAutoFlush(false, false);
List<Get> lGet = new ArrayList<>();
List<Put> lPut = new ArrayList<>();
byte[] cf = Bytes.toBytes("cf");
byte[] qf = Bytes.toBytes("value");
if (opType.equals("insert")) {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
lPut.clear();
for (int i = 0; i < batch; i++) {
Put p = new Put(makeHbaseRowKey(key));
String value = RandomStringUtils.random(dataSize, true, true);
p.addColumn(cf, qf, value.getBytes());
lPut.add(p);
key++;
}
table.put(lPut);
table.flushCommits();
}
} else {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
lGet.clear();
for (int i = 0; i < batch; i++) {
Get g = new Get(makeHbaseRowKey(key));
lGet.add(g);
key++;
}
Result[] rs = table.get(lGet);
}
}
No ka mea ma HB pono e mālama ka mea kūʻai i ka hāʻawi like ʻana o ka ʻikepili, ua like ke ʻano o ka hana paʻakai kī:
public static byte[] makeHbaseRowKey(long key) {
byte[] nonSaltedRowKey = Bytes.toBytes(key);
CRC32 crc32 = new CRC32();
crc32.update(nonSaltedRowKey);
long crc32Value = crc32.getValue();
byte[] salt = Arrays.copyOfRange(Bytes.toBytes(crc32Value), 5, 7);
return ArrayUtils.addAll(salt, nonSaltedRowKey);
}
I kēia manawa ka mea hoihoi loa - nā hopena:
ʻO ka mea like ma ke ʻano pakuhi:
ʻO ka maikaʻi o ka HB he mea kahaha loa aia kahi kānalua aia kekahi ʻano bottleneck i ka hoʻonohonoho CS. Eia nō naʻe, ʻaʻole wikiwiki ʻo Googling a me ka ʻimi ʻana i nā ʻāpana ʻokoʻa loa (e like me concurrent_writes a i ʻole memtable_heap_space_in_mb). I ka manawa like, maʻemaʻe nā lāʻau a ʻaʻole hoʻohiki i kekahi mea.
Ua māhele like ʻia ka ʻikepili ma nā nodes, ua like like nā helu mai nā nodes a pau.
ʻO kēia ke ʻano o ka ʻikepili papa mai kekahi o nā nodesWahi kī: ks
Helu Heluhelu: 9383707
Heluhelu Latency: 0.04287025042448576 ms
Ka helu helu: 15462012
Kākau Latency: 0.1350068438699957 ms
Ke kali ʻana i nā ʻula: 0
Papa: t1
SST helu helu: 16
Wahi i hoʻohana ʻia (ola): 148.59 MiB
Wahi i hoʻohana ʻia (nui): 148.59 MiB
Wahi i hoʻohana ʻia e nā kiʻi paʻi (nui): 0 bytes
Hoʻohana ʻia ka hoʻomanaʻo ʻana o ka puʻu (nui): 5.17 MiB
SSTable Compression Ratio: 0.5720989576459437
Ka helu o nā ʻāpana (manaʻo): 3970323
Helu pūnaewele hoʻomanaʻo: 0
Nui ʻikepili hoʻopaʻa ʻia: 0 bytes
Hoʻohana ʻia ka hoʻomanaʻo hoʻopaʻa ʻia ma waho o ka puʻu: 0 bytes
Ka helu hoʻololi Memtable: 5
Helu heluhelu kūloko: 2346045
Heluhelu heluhelu kūloko: NaN ms
Helu kākau kūloko: 3865503
Ka lōʻihi kākau kūloko: NaN ms
Ke kali nei nā ʻūlū: 0
Pākēneka i hoʻoponopono ʻia: 0.0
ʻO ka kānana Bloom i nā hopena maikaʻi ʻole: 25
ʻO ka lākiō hoʻopunipuni Bloom: 0.00000
Wahi kānana pua i hoʻohana ʻia: 4.57 MiB
Hoʻohana ʻia ke kānana Bloom mai ka puʻu hoʻomanaʻo: 4.57 MiB
Hōʻuluʻulu hōʻuluʻulu manaʻo i hoʻohana ʻia: 590.02 KiB
Hoʻohana ʻia ka metadata hoʻopaʻa ʻia mai ka puʻu hoʻomanaʻo: 19.45 KiB
ʻO ka hapa liʻiliʻi o nā paita: 36
ʻO ka palena palena paʻa paʻa: 42
ʻO ka pānaʻi paʻi paʻi ʻia: 42
'Awelika mau pūnaewele ola no kēlā me kēia ʻāpana (ʻelima mau minuke hope loa): NaN
ʻO ka nui o nā cell ola ma kēlā me kēia ʻāpana (ʻelima mau minuke hope loa): 0
Nā pōhaku kupapaʻu maʻamau i kēlā me kēia ʻāpana (ʻelima mau minuke hope loa): NaN
Nā pōhaku kupapaʻu kiʻekiʻe loa no kēlā me kēia ʻāpana (ʻelima mau minuke hope loa): 0
Nā hoʻololi i hāʻule: 0 bytes
ʻO ka hoʻāʻo ʻana e hōʻemi i ka nui o ka pūʻulu (ʻo ka hoʻouna ʻana i kēlā me kēia) ʻaʻohe hopena, ʻoi aku ka ʻino. Hiki paha i ka ʻoiaʻiʻo ʻo kēia ka hana kiʻekiʻe loa no CS, no ka mea ua like nā hopena i loaʻa no CS me nā mea i loaʻa no DataStax - ma kahi o nā haneli haneli o nā hana i kekona. Eia kekahi, inā mākou e nānā i ka hoʻohana waiwai, e ʻike mākou e hoʻohana ana ʻo CS i nā CPU a me nā disks:
Hōʻike ke kiʻi i ka hoʻohana ʻana i ka wā e holo ai nā hoʻokolohua āpau i ka lālani no nā ʻikepili ʻelua.
E pili ana i ka pono heluhelu ikaika o HB. Ma ʻaneʻi hiki iā ʻoe ke ʻike no nā ʻikepili ʻelua, haʻahaʻa loa ka hoʻohana ʻana i ka disk i ka wā heluhelu (ʻo nā hoʻokolohua heluhelu ka ʻāpana hope o ka pōʻai hoʻāʻo no kēlā me kēia waihona, no ka laʻana no CS mai 15:20 a 15:40). I ka hihia o HB, maopopo ke kumu - ka hapa nui o ka ʻikepili e kau ana ma ka hoʻomanaʻo, i ka memstore, a ua hūnā ʻia kekahi i loko o ka blockcache. No CS, ʻaʻole maopopo loa ke ʻano o ka hana ʻana, akā ʻaʻole ʻike ʻia ka hana hou ʻana o ka disk, akā inā wale nō, ua hoʻāʻo ʻia e hiki ai i ka cache row_cache_size_in_mb = 2048 a hoʻonohonoho i ka caching = {'kī': 'ALL', 'rows_per_partition': ' 2000000'}, akā ua ʻoi aku ka ʻino.
He mea kūpono hoʻi e haʻi hou i kahi mea nui e pili ana i ka helu o nā wahi ma HB. I kā mākou hihia, ua kuhikuhi ʻia ka waiwai e like me 64. Inā ʻoe e hoʻemi a hoʻohālikelike iā ia, no ka laʻana, 4, a laila i ka heluhelu ʻana, hāʻule ka wikiwiki i 2 mau manawa. ʻO ke kumu, ʻoi aku ka wikiwiki o ka hoʻopiha ʻana o ka memstore a hoʻopau pinepine ʻia nā faila a i ka heluhelu ʻana, pono e hoʻoponopono hou ʻia nā faila, he hana paʻakikī loa ia no HB. Ma nā kūlana maoli, hiki ke mālama ʻia kēia ma ka noʻonoʻo ʻana ma o kahi hoʻolālā presplitting a compactification; ʻoiai, hoʻohana mākou i kahi pono kākau ponoʻī e hōʻiliʻili i ka ʻōpala a hoʻopaʻa mau i nā HFiles i ke kua. Hiki paha no nā ho'āʻo DataStax ua hoʻokaʻawale lākou i 1 ʻāpana wale nō ma ka pākaukau (ʻaʻole pololei) a ʻo ia ka mea e wehewehe ai i ke kumu i haʻahaʻa loa ai ka HB i kā lākou mau hoʻokolohua heluhelu.
Ua unuhi ʻia nā manaʻo hoʻoholo mua mai kēia. Ke manaʻo nei ʻaʻohe hewa nui i hana ʻia i ka wā o ka hoʻāʻo ʻana, a laila ua like ʻo Cassandra me kahi colossus me nā wāwae pālolo. ʻOi aku ka pololei, ʻoiai ʻo ia e kaulike ana ma ka wāwae hoʻokahi, e like me ke kiʻi ma ka hoʻomaka ʻana o ka ʻatikala, hōʻike ʻo ia i nā hopena maikaʻi loa, akā i ka hakakā ʻana ma lalo o nā ʻano like ʻole e nalowale ʻo ia. Ma ka manawa like, me ka noʻonoʻo ʻana i ka hoʻohana haʻahaʻa CPU ma kā mākou hāmeʻa, ua aʻo mākou e kanu i ʻelua RegionServer HBs no kēlā me kēia host a laila pālua i ka hana. ʻO kēlā mau mea. Ma ka noʻonoʻo ʻana i ka hoʻohana ʻana i nā kumuwaiwai, ʻoi aku ka pōʻino o ke kūlana no CS.
ʻOiaʻiʻo, ʻano synthetic kēia mau hoʻokolohua a ʻoi aku ka haʻahaʻa o ka nui o ka ʻikepili i hoʻohana ʻia ma aneʻi. Hiki paha inā hoʻololi mākou i terabytes, ʻokoʻa ke kūlana, akā no ka HB hiki iā mākou ke hoʻouka terabytes, no ka CS ua lilo kēia i pilikia. Hoʻolei pinepine ʻo ia i kahi OperationTimedOutException me kēia mau puke, ʻoiai ua hoʻonui ʻia nā palena no ke kali ʻana i kahi pane i nā manawa he nui i hoʻohālikelike ʻia i nā mea paʻamau.
Manaʻo wau ma o ka hui pū ʻana e ʻike mākou i nā bottlenecks o CS a inā hiki iā mākou ke wikiwiki, a laila ma ka hopena o ka pou e hoʻohui maoli wau i ka ʻike e pili ana i nā hopena hope loa.
UPD: Mahalo i ka ʻōlelo aʻoaʻo a nā hoa, ua hiki iaʻu ke wikiwiki i ka heluhelu. Ua:
159 ops (644 papa, 4 kahawai, batch 5).
Hoʻohui:
.withLoadBalancingPolicy(TokenAwarePolicy hou(DCAwareRoundRobinPolicy.builder().build()))
A pāʻani wau me ka nui o nā loina. ʻO kēia ka hopena:
4 papaʻaina, 100 milo, pūʻulu = 1 (ʻāpana ma ka ʻāpana): 301 ops
4 papaʻaina, 100 mau kaula, pūʻulu = 10: 447 ops
4 papaʻaina, 100 mau kaula, pūʻulu = 100: 625 ops
Ma hope e hoʻohana au i nā ʻōlelo aʻoaʻo hoʻokani ʻē aʻe, e holo i kahi pōʻai hoʻāʻo piha a hoʻohui i nā hopena ma ka hopena o ka pou.
Source: www.habr.com