E le o se tala malie, e foliga mai o lenei ata patino e sili ona saʻo le atagia mai o le aano o nei faʻamaumauga, ma i le faaiuga o le a manino pe aisea:
E tusa ai ma le DB-Engines Ranking, o faʻamaumauga pito sili ona taʻutaʻua o NoSQL o Cassandra (o le CS) ma le HBase (HB).
E tusa ai ma le manaʻoga o le faʻamoemoe, o la matou 'au faʻatautaia o faʻamaumauga i Sberbank ua uma ma galulue vavalalata ma HB. I le taimi lea, sa matou suʻesuʻeina lelei ona malosi ma vaivaiga ma aʻoaʻoina le auala e kuka ai. Ae ui i lea, o le i ai o se isi mea i le tulaga o le CS na faʻamalosia ai i matou e faʻasalaina i matou i sina masalosalo: pe na matou faia le filifiliga saʻo? E le gata i lea, o taunuuga , na faia e DataStax, na latou fai mai o le CS e faigofie ona taia le HB ma toetoe lava a faʻaumatia sikoa. I le isi itu, DataStax o se pati fiafia, ma e le tatau ona e faia a latou upu mo ia. Sa matou le mautonu foi i le itiiti ifo o faʻamatalaga e uiga i tulaga o suʻega, o lea na matou filifili ai e suʻe i matou lava poʻo ai le tupu o BigData NoSql, ma o faʻaiʻuga na maua na foliga mai e matua manaia.
Ae ui i lea, aʻo leʻi agai i luma i faʻaiʻuga o suʻega na faia, e manaʻomia le faʻamatalaina o itu taua o le faʻaogaina o le siosiomaga. O le mea moni o le CS e mafai ona faʻaaogaina i se faiga e mafai ai ona leiloa faʻamatalaga. O na. o le taimi lea e naʻo le tasi le server (node) e nafa ma faʻamatalaga o se ki faapitoa, ma afai mo se mafuaʻaga e le manuia, ona leiloa lea o le tau o lenei ki. Mo le tele o galuega e le taua tele, ae mo le vaega o teugatupe o le tuusaunoaga lea nai lo le tulafono. I la matou tulaga, e taua le i ai o ni kopi o faʻamaumauga mo le teuina faʻamaoni.
O le mea lea, na'o le fa'aogaina o le CS i le fa'asologa fa'alua fa'alua na iloiloina, i.e. O le fausiaina o le casespace na faʻatinoina i tulaga nei:
CREATE KEYSPACE ks WITH REPLICATION = {'class' : 'NetworkTopologyStrategy', 'datacenter1' : 3};
O le isi, e lua auala e faʻamautinoa ai le tulaga manaʻomia o le tutusa. Tulafono lautele:
NW + NR > RF
O lona uiga o le numera o faʻamaoniga mai nodes pe a tusitusi (NW) faʻatasi ai ma le numera o faʻamaoniga mai nodes pe a faitau (NR) e tatau ona sili atu nai lo le faʻasologa o mea. I la matou tulaga, RF = 3, o lona uiga o filifiliga nei e talafeagai:
2 + 2 > 3
3 + 1 > 3
Talu ai e taua tele mo i tatou le teuina o faʻamaumauga i le faʻamaoni e mafai ai, o le 3 + 1 polokalame na filifilia. E le gata i lea, o le HB e galue i luga o se mataupu faavae tutusa, i.e. o sea faatusatusaga o le a sili atu ona talafeagai.
E tatau ona maitauina na faia e DataStax le faʻafeagai i la latou suʻesuʻega, latou te seti RF = 1 mo CS ma HB (mo le mulimuli e ala i le suia o tulaga HDFS). O se itu taua tele ona o le a'afiaga i le fa'atinoga o le CS i lenei mataupu e tele. Mo se faʻataʻitaʻiga, o le ata o loʻo i lalo o loʻo faʻaalia ai le siʻitia o le taimi e manaʻomia e utaina ai faʻamatalaga i le CS:
O iinei tatou te vaʻai ai i mea nei: o le tele o filo tauva e tusia faʻamaumauga, o le umi e manaʻomia. E masani lava, ae e taua tele le faʻaleagaina o faʻatinoga mo RF = 3 e sili atu le maualuga. I se isi faaupuga, afai tatou te tusia 4 filo i 5 laulau taʻitasi (20 i le aofaʻi), ona leiloa lea o le RF = 3 pe a ma le 2 taimi (150 sekone mo RF = 3 versus 75 mo RF = 1). Ae afai tatou te faʻateleina le uta e ala i le faʻapipiʻiina o faʻamaumauga i luga o laulau 8 ma 5 filo i le tasi (40 i le aofaʻi), o le leiloa o le RF = 3 ua 2,7 taimi (375 sekone faʻasaga i le 138).
Masalo o se vaega lenei o le mealilo o le suʻega o uta manuia na faia e DataStax mo CS, aua mo le HB i le matou tulaga e suia ai le faʻasologa o mea mai le 2 i le 3 e leai se aoga. O na. tisiki e le o le HB bottleneck mo la tatou faatulagaga. Ae ui i lea, o loʻo i ai le tele o isi faʻalavelave iinei, aua e tatau ona maitauina o la matou faʻataʻitaʻiga o le HB na faʻapipiʻi laʻititi ma tweaked, o siosiomaga e matua ese lava, ma isi. E taua foi le matauina atonu ou te le iloa pe faapefea ona saunia saʻo le CS ma o loʻo i ai nisi auala sili atu ona aoga e galue ai, ma ou te faʻamoemoe o le a tatou mauaina i faʻamatalaga. Ae muamua mea muamua.
O suʻega uma na faia i luga o se fuifui meafaigaluega e aofia ai le 4 'auʻaunaga, taʻitasi ma le faʻatulagaga nei:
PPU: Xeon E5-2680 v4 @ 2.40GHz 64 filo.
Disks: 12 fasi SATA HDD
java version: 1.8.0_111
CS Version: 3.11.5
cassandra.yml parakalafanumera_tokens: 256
hinted_handoff_enabled: moni
fa'ailoa mai le fa'amalo i le kb: 1024
max_hints_delivery_threads: 2
hints_directory: /data10/cassandra/hints
hints_flush_period_in_ms: 10000
max_hints_file_size_in_mb: 128
batchlog_replay_throttle_in_kb: 1024
fa'amaoni: AllowAllAuthenticator
fa'atagaina: AllowAllAuthorizer
role_manager: CassandraRoleManager
roles_validity_in_ms: 2000
fa'atagaga_malo_i_ms: 2000
credentials_validity_in_ms: 2000
vaeluaga: org.apache.cassandra.dht.Murmur3Partitioner
data_file_directories:
- /data1/cassandra/data # fa'amaumauga ta'itasi dataN ose tisiki eseese
- /data2/cassandra/data
- /data3/cassandra/data
- /data4/cassandra/data
- /data5/cassandra/data
- /data6/cassandra/data
- /data7/cassandra/data
- /data8/cassandra/data
commitlog_directory: /data9/cassandra/commitlog
cdc_enabled: sese
disk_failure_policy: taofi
commit_failure_policy: taofi
prepared_statements_cache_size_mb:
thrift_prepared_statements_cache_size_mb:
key_cache_size_in_mb:
key_cache_save_period: 14400
row_cache_size_in_mb: 0
row_cache_save_period: 0
counter_cache_size_in_mb:
counter_cache_save_period: 7200
saved_caches_directory: /data10/cassandra/saved_caches
commitlog_sync: faavaitaimi
commitlog_sync_period_in_ms: 10000
commitlog_segment_size_in_mb: 32
seed_provider:
- igoa_vasega: org.apache.cassandra.locator.SimpleSeedProvider
faʻaiuga:
— fatu: "*,*"
concurrent_reads: 256 # taumafai 64 - leai se eseesega na matauina
concurrent_writes: 256 # taumafai 64 - leai se eseesega na matauina
concurrent_counter_writes: 256 # taumafai 64 - leai se eseesega na matauina
concurrent_materialized_view_writes: 32
memtable_heap_space_in_mb: 2048 # taumafai 16 GB - sa telegese
memtable_allocation_type: heap_buffers
index_summary_capacity_in_mb:
index_summary_resize_interval_in_minutes: 60
trickle_fsync: sese
trickle_fsync_interval_in_kb: 10240
teutaga_taulaga: 7000
ssl_storage_port: 7001
listen_address: *
tuatusi_faasalalauga: *
listen_on_broadcast_address: moni
internode_authenticator: org.apache.cassandra.auth.AllowAllInternodeAuthenticator
start_native_transport: moni
native_transport_port: 9042
start_rpc: moni
rpc_address: *
rpc_port: 9160
rpc_keepalive: moni
rpc_server_type: sync
fa'atauva'a_fe'avea'i_tele_i_mb: 15
incremental_backups: sese
snapshot_before_compaction: sese
auto_snapshot: moni
column_index_size_in_kb: 64
column_index_cache_size_in_kb: 2
concurrent_compactors: 4
compaction_throughput_mb_per_sec: 1600
sstable_preemptive_open_interval_in_mb: 50
faitau_request_timeout_in_ms: 100000
range_request_timeout_in_ms: 200000
write_request_timeout_in_ms: 40000
counter_write_request_timeout_in_ms: 100000
cas_contention_timeout_in_ms: 20000
truncate_request_timeout_in_ms: 60000
request_timeout_in_ms: 200000
slow_query_log_timeout_in_ms: 500
cross_node_timeout: sese
endpoint_snitch: GossipingPropertyFileSnitch
dynamic_snitch_update_interval_in_ms: 100
dynamic_snitch_reset_interval_in_ms: 600000
dynamic_snitch_badness_threshold: 0.1
request_scheduler: org.apache.cassandra.scheduler.NoScheduler
server_encryption_options:
internode_encryption: leai
client_encryption_options:
mafai: sese
internode_compression: dc
inter_dc_tcp_nodelay: sese
tracetype_query_ttl: 86400
tracetype_repair_ttl: 604800
enable_user_defined_functions: sese
enable_scripted_user_defined_functions: sese
windows_timer_interval: 1
transparent_data_encryption_options:
mafai: sese
tombstone_warn_threshold: 1000
tombstone_failure_threshold: 100000
lapopo'a_alaa'i_threshold_in_kb: 200
tele_vaega_fail_threshold_in_kb: 250
unlogged_batch_across_partitions_warn_threshold: 10
compaction_large_partition_warning_threshold_mb: 100
gc_warn_threshold_in_ms: 1000
back_pressure_enabled: sese
enable_materialized_views: moni
enable_sasi_indexes: moni
Fa'atonuga GC:
### Fa'atonu CMS-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled
-XX:SurvivorRatio=8
-XX:MaxTenuringThreshold=1
-XX:CMSIinitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyNa'o
-XX:CMSWaitDuration=10000
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecord Always
-XX:+CMSClassUnloadingEnabled
O le jvm.options manatua na tuʻuina 16Gb (sa matou taumafai foi 32 Gb, leai se eseesega na matauina).
O laulau na faia ma le faʻatonuga:
CREATE TABLE ks.t1 (id bigint PRIMARY KEY, title text) WITH compression = {'sstable_compression': 'LZ4Compressor', 'chunk_length_kb': 64};HB version: 1.2.0-cdh5.14.2 (i le vasega org.apache.hadoop.hbase.regionserver.HRegion na matou le aofia ai MetricsRegion lea na oʻo atu i le GC ina ua sili atu i le 1000 le numera o itulagi ile RegionServer)
Fa'amaufa'ailoga HBase e le fa'aletonuzookeeper.session.timeout: 120000
hbase.rpc.time out: 2 minute(s)
hbase.client.scanner.timeout.period: 2 minute(s)
hbase.master.handler.count: 10
hbase.regionserver.lease.period, hbase.client.scanner.timeout.period: 2 minute(s)
hbase.regionserver.handler.count: 160
hbase.regionserver.metahandler.count: 30
hbase.regionserver.logroll.period: 4 itula(s)
hbase.regionserver.maxlogs: 200
hbase.hregion.memstore.flush.size: 1 GiB
hbase.hregion.memstore.block.multiplier: 6
hbase.hstore.compactionThreshold: 5
hbase.hstore.blockingStoreFiles: 200
hbase.hregion.majorcompaction: 1 aso(s)
HBase Service Advanced Configuration Snippet (Saogalemu Valve) mo hbase-site.xml:
hbase.regionserver.wal.codecorg.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec
hbase.master.namespace.init.timeout3600000
hbase.regionserver.optionalcacheflushinterval18000000
hbase.regionserver.thread.compaction.large12
hbase.regionserver.wal.enablecompressiontrue
hbase.hstore.compaction.max.size1073741824
hbase.server.compactchecker.interval.multiplier200
Java Configuration Options mo le HBase RegionServer:
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:ReservedCodeCacheSize=256m
hbase.snapshot.master.timeoutMillis: 2 minute(s)
hbase.snapshot.region.time out: 2 minute(s)
hbase.snapshot.master.timeout.millis: 2 minute(s)
HBase REST Server Max Fa'amaumauga Fa'amau: 100 MiB
HBase REST Server Fa'asao Fa'amaumauga Maualuga Maualuga: 5
HBase Fa'atauga Fa'atauva'a Avanoa Fa'amau Fa'amau: 100 MiB
HBase Thrift Server Fa'asao Fa'amaumauga Maualuga Maualuga: 5
Master Max Log Tele: 100 MiB
Matai Sili Lagolago faila faila: 5
ItulagiServer Max Ogoa Tele: 100 MiB
RegionServer Fa'amauina Fa'amaumauga Fa'amaumauga Fa'amaumauga: 5
HBase Active Master Detection Window: 4 minute(s)
dfs.client.hedged.read.threadpool.size: 40
dfs.client.hedged.read.threshold.millis: 10 milisecond(s)
hbase.rest.threads.min: 8
hbase.rest.threads.max: 150
Tulaga Maualuga Fa'amatalaga Fa'amatalaga: 180000
hbase.thrift.minWorkerThreads: 200
hbase.master.executor.openregion.threads: 30
hbase.master.executor.closeregion.threads: 30
hbase.master.executor.serverops.threads: 60
hbase.regionserver.thread.compaction.small: 6
hbase.ipc.server.read.threadpool.size: 20
Itulagi Mover filo: 6
Tagata Fa'atau Java Fa'aputuga Tele i Paita: 1 GiB
HBase REST Server Vaega Fa'atonu: 3 GiB
HBase Fa'atauga Fa'atauva'a Vaega Fa'atonu: 3 GiB
Java Heap Size o le HBase Master i Bytes: 16 GiB
Java Heap Tele o le HBase RegionServer i Bytes: 32 GiB
+ Leoleo Manu
maxClientCnxns: 601
maxSessionTimeout: 120000
Fausia laulau:
hbase org.apache.hadoop.hbase.util.RegionSplitter ns:t1 UniformSplit -c 64 -f cf
sui 'ns:t1', {NAME => 'cf', DATA_BLOCK_ENCODING => 'FAST_DIFF', COMPRESSION => 'GZ'}
E tasi le itu taua iinei - o le DataStax faʻamatalaga e le o taʻu mai pe fia itulagi na faʻaaogaina e fai ai laulau HB, e ui lava e taua tele mo voluma tetele. O le mea lea, mo suʻega, numera = 64 na filifilia, lea e mafai ai ona teuina i luga ole 640 GB, i.e. laulau lapopoa.
I le taimi o le suʻega, HBase sa i ai 22 afe laulau ma 67 afe itulagi (semanu e faʻamaʻi mo le version 1.2.0 pe a le o le patch na taʻua i luga).
Lenei mo le code. Talu ai e leʻi manino poʻo fea faʻasalalauga e sili atu ona aoga mo se faʻamaumauga faʻapitoa, o suʻega na faia i tuʻufaʻatasiga eseese. O na. i nisi o suʻega, 4 laulau sa utaina i le taimi e tasi (uma 4 nodes na faʻaogaina mo le fesoʻotaʻiga). I isi suʻega sa matou galulue ma 8 laulau eseese. I nisi tulaga, o le aofaʻi o le aofaʻi o le 100, i isi 200 (faʻasologa o le vaega - vaʻai le numera i lalo). O le tele o faʻamatalaga mo le tau o le 10 paita poʻo le 100 paita (dataSize). I le aotelega, e 5 miliona faamaumauga na tusia ma faitau i laulau taitasi i taimi taitasi. I le taimi lava e tasi, e 5 filo na tusia / faitau i laulau taʻitasi (numera filo - thNum), na faʻaogaina e taʻitasi a latou lava ki (faitau = 1 miliona):
if (opType.equals("insert")) {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
StringBuilder sb = new StringBuilder("BEGIN BATCH ");
for (int i = 0; i < batch; i++) {
String value = RandomStringUtils.random(dataSize, true, true);
sb.append("INSERT INTO ")
.append(tableName)
.append("(id, title) ")
.append("VALUES (")
.append(key)
.append(", '")
.append(value)
.append("');");
key++;
}
sb.append("APPLY BATCH;");
final String query = sb.toString();
session.execute(query);
}
} else {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
StringBuilder sb = new StringBuilder("SELECT * FROM ").append(tableName).append(" WHERE id IN (");
for (int i = 0; i < batch; i++) {
sb = sb.append(key);
if (i+1 < batch)
sb.append(",");
key++;
}
sb = sb.append(");");
final String query = sb.toString();
ResultSet rs = session.execute(query);
}
}
E tusa ai, na tuʻuina atu galuega tutusa mo HB:
Configuration conf = getConf();
HTable table = new HTable(conf, keyspace + ":" + tableName);
table.setAutoFlush(false, false);
List<Get> lGet = new ArrayList<>();
List<Put> lPut = new ArrayList<>();
byte[] cf = Bytes.toBytes("cf");
byte[] qf = Bytes.toBytes("value");
if (opType.equals("insert")) {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
lPut.clear();
for (int i = 0; i < batch; i++) {
Put p = new Put(makeHbaseRowKey(key));
String value = RandomStringUtils.random(dataSize, true, true);
p.addColumn(cf, qf, value.getBytes());
lPut.add(p);
key++;
}
table.put(lPut);
table.flushCommits();
}
} else {
for (Long key = count * thNum; key < count * (thNum + 1); key += 0) {
lGet.clear();
for (int i = 0; i < batch; i++) {
Get g = new Get(makeHbaseRowKey(key));
lGet.add(g);
key++;
}
Result[] rs = table.get(lGet);
}
}
Talu ai i le HB e tatau i le tagata o tausia ona tausia le tufatufaina tutusa o faʻamaumauga, o le galuega autu o le masima e pei o lenei:
public static byte[] makeHbaseRowKey(long key) {
byte[] nonSaltedRowKey = Bytes.toBytes(key);
CRC32 crc32 = new CRC32();
crc32.update(nonSaltedRowKey);
long crc32Value = crc32.getValue();
byte[] salt = Arrays.copyOfRange(Bytes.toBytes(crc32Value), 5, 7);
return ArrayUtils.addAll(salt, nonSaltedRowKey);
}
Le taimi nei o le vaega sili ona manaia - o taunuuga:
Le mea lava e tasi ile fa'akalafa:
O le lelei o le HB e matua ofo lava o loʻo i ai se masalosaloga o loʻo i ai se ituaiga fagu i totonu o le seti CS. Peita'i, Googling ma su'esu'e mo fa'amaufa'ailoga sili ona manino (pei o concurrent_writes po'o memtable_heap_space_in_mb) e le'i fa'avavevave. I le taimi lava e tasi, o ogalaau e mama ma e le palauvale i se mea.
O faʻamaumauga na tufatufaina tutusa i luga o nodes, o fuainumera mai nodes uma e tutusa lava.
O le fa'atusa lea o fuainumera o laulau mai se tasi o nodesAvanoa autu: ks
Faitau Faitau: 9383707
Faitau Latency: 0.04287025042448576 ms
Tusi Faitau: 15462012
Tusi Latency: 0.1350068438699957 ms
Fa'atali Fufulu: 0
Laulau: t1
SST faitau aofa'i: 16
Avanoa fa'aoga (ola): 148.59 MiB
Avanoa fa'aoga (atoa): 148.59 MiB
Avanoa fa'aoga e fa'ata'ita'iga (toatoa): 0 paita
Fa'ato'a fa'aputuga manatua fa'aoga (toatoa): 5.17 MiB
SSTTable Compression Ratio: 0.5720989576459437
Numera o vaeluaga (fa'atatau): 3970323
Numera sela e mafai ona fa'aaogaina: 0
Tele fa'amaumauga e mafai ona manatua: 0 paita
Fa'ato'a fa'aputuga manatua fa'aaoga: 0 paita
Numera ki fa'amaufa'ailoga: 5
Faitauga fa'apitonu'u: 2346045
Fa'alotoifale leo faitau: NaN ms
Numera tusitusi fa'apitonu'u: 3865503
Fa'alenu'u tusitusi fa'alotoifale: NaN ms
Fa'atali fulumaga: 0
Pasene toe faaleleia: 0.0
Fugalaau faamama mea sese: 25
Fugalaau faamama fua faatatau sese: 0.00000
Fugalaau faamama avanoa e faaaoga: 4.57 MiB
Fugala'au faamama ese fa'aputuga manatua fa'aaoga: 4.57 MiB
Aotelega fa'asinomaga mai fa'aputuga manatua fa'aaoga: 590.02 KiB
Fa'amauina metadata mai fa'aputuga manatua fa'aaoga: 19.45 KiB
Vaeluaga fa'ama'i paita la'ititi: 36
Vaeluaga fa'ama'i paita maualuga: 42
Vaeluaga fa'ama'oti paita lona uiga: 42
Fuafua sela ola i le fasi pepa (lima minute mulimuli): NaN
Ole maualuga ole sela ola ile fasi (lima minute mulimuli): 0
Averesi o ma'a tu'ugamau i fasi pepa (lima minute mulimuli): NaN
Ma'a tu'ugamau aupito maualuga i fasi pepa (lima minute mulimuli): 0
Suiga Pa'u: 0 paita
O se taumafaiga e faʻaitiitia le tele o le vaega (e oʻo lava i le lafoina taʻitoʻatasi) e leai se aoga, na sili atu le leaga. Atonu o le mea moni o le mea moni lava lea o le maualuga o le faatinoga mo CS, talu ai o taunuuga na maua mo CS e tutusa ma mea na maua mo DataStax - e tusa ma le fiaselau afe o gaioiga i le sekone. E le gata i lea, afai tatou te vaʻavaʻai i le faʻaogaina o punaoa, o le a tatou iloa o le CS e faʻaaogaina le tele o PPU ma disks:
O le ata o lo'o fa'aalia ai le fa'aogaina i le taimi o su'ega uma i se laina mo fa'amaumauga uma e lua.
E tusa ai ma le malosi o le faitau tusi a le HB. O iinei e mafai ona e vaʻai ai mo faʻamaumauga uma e lua, o le faʻaogaina o tisiki i le taimi o le faitau e matua maualalo (faitau suʻega o le vaega mulimuli lea o le taamilosaga o suʻega mo faʻamaumauga taʻitasi, mo se faʻataʻitaʻiga mo le CS e mai le 15:20 i le 15:40). I le tulaga o le HB, e manino le mafuaʻaga - o le tele o faʻamatalaga o loʻo tautau i le mafaufau, i totonu o le memstore, ma o nisi o loʻo teuina i poloka poloka. Ae mo le CS, e le o manino pe faʻapefea ona galue, ae e le o vaʻaia foi le toe faʻaaogaina o tisiki, ae naʻo le tulaga, na faia se taumafaiga e mafai ai le cache row_cache_size_in_mb = 2048 ma seti caching = {'ki': 'ALL', 'rows_per_partition': ' 2000000'}, ae ua atili ai ona leaga.
E taua foi le toe taʻua o se mea taua e uiga i le numera o itulagi i le HB. I la matou tulaga, o le tau na faʻamaonia e pei o le 64. Afai e te faʻaititia ma faʻatusatusa i, mo se faʻataʻitaʻiga, 4, ona faitau loa lea o le saoasaoa e pa'ū i le 2 taimi. O le mafuaʻaga o le memstore o le a faʻatumu vave ma faila faila o le a sili atu ona fufulu ma pe a faitau, e manaʻomia le tele o faila e faʻatautaia, o se faʻalavelave faigata mo le HB. I tulaga moni, e mafai ona togafitia lenei mea e ala i le mafaufau e ala i se taʻiala faʻapipiʻi ma faʻapipiʻi; aemaise lava, matou te faʻaogaina se faʻaoga tusitusia e aoina ai lapisi ma faʻapipiʻi HFiles i taimi uma i tua. E mafai lava mo su'ega a DataStax latou te tu'uina atu na'o le 1 le itulagi i le laulau (e le sa'o) ma o le a fa'amanino ai le mafua'aga na matua maualalo ai le HB i a latou su'ega faitau.
O fa'ai'uga muamua nei o lo'o maua mai i lea mea. Faapea e leai ni mea sese tele na faia i le taimi o suʻega, ona foliga mai ai lea o Cassandra o se colossus ma vae o le omea. E sili atu le saʻo, aʻo ia paleni i luga o le tasi vae, e pei o le ata i le amataga o le tusiga, na ia faʻaalia ai ni taunuuga lelei, ae i se fusuaga i lalo o tulaga lava e tasi na te faiaina ai. I le taimi lava e tasi, i le amanaia o le maualalo o le faʻaogaina o le PPU i luga oa matou meafaigaluega, na matou aʻoaʻo ai e totō lua RegionServer HBs i le talimalo ma faʻaluaina ai le faʻatinoga. O na. I le amanaia o le faʻaaogaina o punaoa, o le tulaga mo CS e sili atu ona leaga.
O le mea moni, o nei suʻega e fai si faʻapipiʻiina ma o le aofaʻi o faʻamatalaga na faʻaaogaina iinei e laʻititi lava. E mafai pe afai tatou te fesuiaʻi i terabytes, o le a ese le tulaga, ae mo le HB e mafai ona tatou utaina terabytes, mo le CS na iʻu ina faʻafitauli. E masani ona lafo se OperationTimedOutException e oʻo lava i nei voluma, e ui lava o faʻamau mo le faʻatali mo se tali ua faʻateleina taimi faʻatusatusa i mea le lelei.
Ou te faʻamoemoe e ala i taumafaiga faʻatasi o le a tatou maua ai faʻamaufaʻailoga a le CS ma afai e mafai ona tatou faʻavaveina, ona i ai lea i le faaiuga o le pou o le a ou faʻaopoopoina faʻamatalaga e uiga i taunuuga mulimuli.
UPD: Faafetai i le fautuaga a uo, na mafai ona ou faatelevaveina le faitau. Sa:
159 ops (644 laulau, 4 vaitafe, vaega 5).
Faaopopo:
.withLoadBalancingPolicy(new TokenAwarePolicy(DCAwareRoundRobinPolicy.builder().build()))
Ma sa ou taalo solo i le numera o filo. O le taunuuga o mea nei:
4 laulau, 100 filo, vaega = 1 (vaega i lea fasi pepa): 301 ops
4 laulau, 100 filo, vaega = 10: 447 ops
4 laulau, 100 filo, vaega = 100: 625 ops
Mulimuli ane o le a ou faʻaogaina isi faʻamatalaga faʻalogo, faʻataʻitaʻi atoa le taamilosaga o suʻega ma faʻaopoopo iʻuga i le pito o le pou.
puna: www.habr.com