Pulogalamu ya ProHoster > Blog > Ulamuliro > Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

VictoriaMetrics, TimescaleDB ndi InfluxDB adafanizidwa mu nkhani yapita pa dataset yokhala ndi mabiliyoni a data amtundu wa 40K wapadera.

Zaka zingapo zapitazo panali nthawi ya Zabbix. Seva iliyonse yachitsulo yopanda kanthu inalibe zizindikiro zochulukirapo - kugwiritsa ntchito CPU, kugwiritsa ntchito RAM, kugwiritsa ntchito disk ndi kugwiritsa ntchito maukonde. Mwanjira iyi, ma metrics ochokera kuma seva masauzande amatha kulowa munthawi yapadera ya 40, ndipo Zabbix imatha kugwiritsa ntchito MySQL ngati kumbuyo kwa data yanthawi :)

Panopa yekha node_exporter ndi masinthidwe osasinthika amapereka ma metrics opitilira 500 pagulu lapakati. Pali zambiri ogulitsa kunja kwa ma database osiyanasiyana, maseva apaintaneti, makina a hardware, ndi zina zotere. Onse amapereka ma metric othandiza osiyanasiyana. Zonse ntchito zambiri ayambe kudziikira okha zizindikiro zosiyanasiyana. Pali Kubernetes yokhala ndi magulu ndi ma pod omwe amawulula ma metric ambiri. Izi zimapangitsa kuti maseva aziwonetsa masauzande masauzande a ma metrics apadera pa wolandira. Chifukwa chake mndandanda wanthawi wa 40K wapadera sulinso mphamvu yayikulu. Ikukhala yodziwika bwino ndipo iyenera kusamaliridwa mosavuta ndi TSDB yamakono pa seva imodzi.

Kodi pali mndandanda wanthawi zotani pakali pano? Mwina 400K kapena 4M? Kapena 40m? Tiyeni tifanizire ma TSDB amakono ndi manambala awa.

Kuyika benchmark

Mtengo wa TSBS ndi chida chabwino kwambiri chowonetsera ma TSDB. Zimakupatsani mwayi wopanga ma metrics opitilira muyeso podutsa nthawi yofunikira yogawidwa ndi 10 - mbendera. -mlingo (kale -scale-var). 10 ndi chiwerengero cha miyeso (metrics) yopangidwa pa wolandira aliyense kapena seva. Ma dataset otsatirawa adapangidwa pogwiritsa ntchito TSBS pa benchmark:

  • 400K nthawi yapadera, 60 yachiwiri yapakati pakati pa ma data, deta imatenga masiku atatu athunthu, ~ 3B chiwerengero chonse cha mfundo za data.
  • 4M wapadera nthawi mndandanda, 600 yachiwiri interval, deta span 3 masiku athunthu, ~ 1.7B chiwerengero cha mfundo deta.
  • Nthawi yapadera ya 40M, nthawi ya ola la 1, deta imatenga masiku atatu athunthu, ~ 3B chiwerengero chonse cha mfundo za data.

Makasitomala ndi seva zikuyenda pazochitika zodzipatulira n1-mulingo-16 mu Google Cloud. Zochitika izi zinali ndi masinthidwe awa:

  • ma CPU: 16
  • RAM: 60 GB
  • Kusungirako: Standard 1TB HDD. Amapereka 120 Mbps kuwerenga / kulemba kutulutsa, 750 kuwerenga ntchito pamphindi ndipo 1,5K imalemba pamphindikati.

Ma TSDB adachotsedwa pazithunzi zovomerezeka ndikuyendetsa mu docker ndi masanjidwe awa:

  • VictoriaMetrics:

    docker run -it --rm -v /mnt/disks/storage/vmetrics-data:/victoria-metrics-data -p 8080:8080 valyala/victoria-metrics

  • Makhalidwe a InfluxDB (-e) amafunikira kuti athandizire mphamvu zambiri. Onani zambiri mu zolemba):

    docker run -it --rm -p 8086:8086 
-e INFLUXDB_DATA_MAX_VALUES_PER_TAG=4000000 
-e INFLUXDB_DATA_CACHE_MAX_MEMORY_SIZE=100g 
-e INFLUXDB_DATA_MAX_SERIES_PER_DATABASE=0 
-v /mnt/disks/storage/influx-data:/var/lib/influxdb influxdb

  • TimescaleDB (masinthidwe otengedwa kuchokera izo file):

MEM=`free -m | grep "Mem" | awk β€˜{print $7}’`
let "SHARED=$MEM/4"
let "CACHE=2*$MEM/3"
let "WORK=($MEM-$SHARED)/30"
let "MAINT=$MEM/16"
let "WAL=$MEM/16"
docker run -it β€” rm -p 5432:5432 
--shm-size=${SHARED}MB 
-v /mnt/disks/storage/timescaledb-data:/var/lib/postgresql/data 
timescale/timescaledb:latest-pg10 postgres 
-cmax_wal_size=${WAL}MB 
-clog_line_prefix="%m [%p]: [%x] %u@%d" 
-clogging_collector=off 
-csynchronous_commit=off 
-cshared_buffers=${SHARED}MB 
-ceffective_cache_size=${CACHE}MB 
-cwork_mem=${WORK}MB 
-cmaintenance_work_mem=${MAINT}MB 
-cmax_files_per_process=100

Chojambulira deta chinayendetsedwa ndi ulusi 16 wofanana.

Nkhaniyi ili ndi zotsatira za ma benchmarks okha. Zotsatira za benchmark yosankhidwa zidzasindikizidwa m'nkhani ina.

400K nthawi yapaderadera

Tiyeni tiyambe ndi zinthu zosavuta - 400K. Zotsatira za benchmark:

  • VictoriaMetrics: 2,6M mfundo za data pamphindi; Kugwiritsa ntchito RAM: 3 GB; Kukula komaliza kwa data pa disk: 965 MB
  • InfluxDB: 1.2M mfundo za data pamphindi; Kugwiritsa ntchito RAM: 8.5 GB; kukula komaliza kwa data pa disk: 1.6 GB
  • Nthawi: 849K data point pamphindikati; Kugwiritsa ntchito RAM: 2,5 GB; Kukula komaliza kwa data pa disk: 50 GB

Monga mukuwonera pazotsatira zomwe zili pamwambapa, VictoriaMetrics imapambana pakuyika magwiridwe antchito ndi kuchuluka kwa compression. Nthawi imapambana mukugwiritsa ntchito RAM, koma imagwiritsa ntchito malo ambiri a disk - 29 bytes pa data point.

Pansipa pali ma graph ogwiritsira ntchito CPU pa TSDB iliyonse panthawi ya benchmark:

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi chojambula: VictoriaMetrics - CPU katundu panthawi yoyesa kuyesa kwapadera kwa 400K metric.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: InfluxDB - CPU katundu panthawi yoyeserera yapadera ya metric 400K.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzithunzi: TimescaleDB - CPU katundu panthawi yoyesa kuyesa kwapadera kwa 400K.

VictoriaMetrics imagwiritsa ntchito ma vCPU onse omwe alipo, pomwe InfluxDB imagwiritsa ntchito ~ 2 mwa 16 vCPU.

Timecale imangogwiritsa ntchito 3-4 mwa 16 vCPUs. Kuchuluka kwa iowait ndi kachitidwe mu TimescaleDB timescale graph ikuwonetsa kutsekeka mu kachitidwe kakang'ono ka input/output (I/O). Tiyeni tiwone ma graph ogwiritsira ntchito disk bandwidth:

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: VictoriaMetrics - Disk Bandwidth Usage mu Insertion Test for Unique Metrics 400K.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: InfluxDB - Disk Bandwidth Usage pa Insertion Test for Unique Metrics 400K.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: TimescaleDB - Disk Bandwidth Usage pa Insertion Test for Unique Metrics 400K.

VictoriaMetrics imalemba zambiri pa 20 Mbps ndi nsonga mpaka 45 Mbps. Nsonga zimagwirizana ndi kuphatikizika kwakukulu mumtengo NGO.

InfluxDB imalemba zambiri pa 160 MB / s, pomwe 1 TB drive zikhale zochepa lembani kutulutsa 120 MB/s.

TimescaleDB ili ndi malire kuti alembe kutulutsa kwa 120 Mbps, koma nthawi zina imaphwanya malire ndikufikira 220 Mbps pamitengo yapamwamba. Nsonga izi zimagwirizana ndi zigwa zosakwanira kugwiritsa ntchito CPU mu graph yapitayi.

Tiyeni tiwone ma graph ogwiritsira ntchito / zotuluka (I/O):

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzithunzi: VictoriaMetrics - Lowetsani kugwiritsa ntchito I/O kuyesa kwa 400K ma metric apadera.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: InfluxDB - Ikani kuyesa kwa I/O kugwiritsa ntchito ma metric apadera a 400K.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: TimescaleDB - Ikani kuyesa kwa I/O kugwiritsa ntchito ma metric apadera a 400K.

Tsopano zikuwonekeratu kuti TimescaleDB ikufikira malire ake a I/O, kotero siingagwiritse ntchito ma 12 vCPU otsalawo.

4M nthawi yapaderadera

Mndandanda wa nthawi ya 4M umawoneka wovuta pang'ono. Koma mpikisano wathu wapambana mayesowa. Zotsatira za benchmark:

  • VictoriaMetrics: 2,2M mfundo za data pamphindi; Kugwiritsa ntchito RAM: 6 GB; Kukula komaliza kwa data pa disk: 3 GB.
  • InfluxDB: 330K data point pamphindi; Kugwiritsa ntchito RAM: 20,5 GB; Kukula komaliza kwa data pa disk: 18,4 GB.
  • TimescaleDB: 480K data point pamphindikati; Kugwiritsa ntchito RAM: 2,5 GB; Kukula komaliza kwa data pa disk: 52 GB.

Kuchita kwa InfluxDB kwatsika kuchoka pa ma data 1,2M pa sekondi imodzi kwa nthawi ya 400K kufika pa 330K data points pa sekondi imodzi kwa 4M time series. Uku ndikuwonongeka kwakukulu kwa magwiridwe antchito poyerekeza ndi ena omwe akupikisana nawo. Tiyeni tiwone ma graph ogwiritsira ntchito CPU kuti timvetsetse chomwe chimayambitsa kutayika uku:

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzithunzi: VictoriaMetrics - Kugwiritsa ntchito CPU pakuyesa kuyika kwanthawi yapadera ya 4M.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: InfluxDB - Kugwiritsa ntchito CPU pakuyesa kuyika kwanthawi yapadera ya 4M.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: TimescaleDB - Kugwiritsa ntchito CPU pakuyesa kuyika kwanthawi yapadera ya 4M.

VictoriaMetrics imagwiritsa ntchito pafupifupi mphamvu zonse za processing unit (CPU). Kutsika kumapeto kumafanana ndi kusakanikirana kwa LSM kotsalira pambuyo poti deta yonse yayikidwa.

InfluxDB imagwiritsa ntchito 8 yokha ya 16 vCPU, pomwe TimsecaleDB imagwiritsa ntchito 4 ya 16 vCPU. Kodi ma graph awo amafanana chiyani? Kugawana kwakukulu iowait, zomwe zikuwonetsanso kutsekeka kwa I/O.

TimescaleDB ili ndi gawo lalikulu system. Tikuganiza kuti mphamvu yayikulu idayambitsa mafoni ambiri kapena ambiri zolakwika zazing'ono zamasamba.

Tiyeni tiwone ma graph a disk throughput:

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: VictoriaMetrics - Kugwiritsa ntchito disk bandwidth kuyika ma metric apadera a 4M.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: InfluxDB - Kugwiritsa ntchito disk bandwidth kuyika ma metric apadera a 4M.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: TimescaleDB - Kugwiritsa ntchito disk bandwidth kuyika ma metric apadera a 4M.

VictoriaMetrics idafikira malire a 120 MB/s pachimake, pomwe liwiro lapakati linali 40 MB/s. Zikuoneka kuti ma fusions angapo olemera a LSM adachitika pachimake.

InfluxDB imafinyanso kutulutsa kwapakati kwa 200 MB/s ndi nsonga zofikira 340 MB/s pa disk yokhala ndi malire a 120 MB/s :)

TimescaleDB siilinso disk limited. Zikuwoneka kuti zimachepetsedwa ndi chinthu china chokhudzana ndi kuchuluka kwakukulu систСмной CPU katundu.

Tiyeni tiwone ma graph ogwiritsira ntchito IO:

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi chojambula: VictoriaMetrics - Kugwiritsa ntchito I/O poyesa kuyika kwanthawi yapadera ya 4M.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzithunzi: InfluxDB - Kugwiritsa ntchito I/O pakuyesa kuyika kwanthawi yapadera ya 4M.

Benchmark ya TSDB yogwira ntchito kwambiri VictoriaMetrics vs TimescaleDB vs InfluxDB

Pamwambapa pali chithunzi: TimescaleDB - Kugwiritsa ntchito I/O pakuyesa kuyika kwanthawi yapadera ya 4M.

Magwiridwe a IO amafanana ndi disk bandwidth - InfluxDB ndi IO yochepa, pomwe VictoriaMetrics ndi TimescaleDB ali ndi zida za IO.

40M nthawi yapaderadera

Nthawi yapadera ya 40M inali yayikulu kwambiri kwa InfluxDB :)

Zotsatira za benchmark:

  • VictoriaMetrics: 1,7M mfundo za data pamphindi; Kugwiritsa ntchito RAM: 29 GB; Kugwiritsa ntchito malo a disk: 17 GB.
  • InfluxDB: Sizinathe chifukwa zimafunika kuposa 60GB ya RAM.
  • TimescaleDB: 330K data point pamphindi, kugwiritsa ntchito RAM: 2,5 GB; Kugwiritsa ntchito malo a disk: 84GB.

TimescaleDB ikuwonetsa kugwiritsidwa ntchito kotsika komanso kokhazikika kwa RAM pa 2,5 GB - chimodzimodzi ndi ma metric apadera a 4M ndi 400K.

VictoriaMetrics idakwera pang'onopang'ono pamlingo wa ma data 100k pa sekondi iliyonse mpaka mayina onse oyika 40M atasinthidwa. Kenaka adapeza chiwerengero chokhazikika cha 1,5-2,0M data point pamphindi, kotero zotsatira zomaliza zinali mfundo za 1,7M pamphindi.

Ma graph a nthawi yapadera ya 40M ndi ofanana ndi ma graph a nthawi yapadera ya 4M, kotero tiyeni tidumphe.

anapezazo

  • Ma TSDB amakono amatha kukonza zoyikapo mamiliyoni anthawi yapadera pa seva imodzi. M'nkhani yotsatira, tiyesa momwe ma TSDB amachitira bwino masankho pamipikisano yanthawi yapadera.
  • Kusakwanira kwa CPU kugwiritsa ntchito nthawi zambiri kumawonetsa kutsekeka kwa I/O. Zingasonyezenso kuti kutsekerezako ndi kowawa kwambiri, ndi ulusi wochepa chabe womwe umatha kuthamanga panthawi imodzi.
  • Botolo la I/O liripo, makamaka m'malo osungira omwe si a SSD monga zida za block network za operekera mitambo.
  • VictoriaMetrics imapereka kukhathamiritsa kwapang'onopang'ono, kotsika kwa I/O. Imapereka liwiro labwino kwambiri komanso chiΕ΅erengero chabwino kwambiri cha kuponderezana.

Tsitsani Chithunzi cha VictoriaMetrics single-server ndikuyesa pa data yanu. Binary yofananira ikupezeka pa GitHub.

Werengani zambiri za VictoriaMetrics mu izi nkhani.

Zosintha: zasindikizidwa Nkhani yoyerekeza kuyika kwa VictoriaMetrics ndi InfluxDB ndi zotsatira zobwerezedwa.

Kusintha #2: Werenganinso Nkhani yokhudza scalability VictoriaMetrics vs InfluxDB vs TimescaleDB.

Kusintha #3: VictoriaMetrics tsopano ndi gwero lotseguka!

Macheza a Telegraph: https://t.me/VictoriaMetrics_ru1

Source: www.habr.com

Kuwonjezera ndemanga