ื ื™ืฆืŸ Clickhouse ื•ื•ื™ ืึท ืคืึทืจื‘ื™ื™ึทื˜ ืคึฟืึทืจ ELK, Big Query ืื•ืŸ TimescaleDB

ืงืœื™ืงื›ืึธื•ืกืข ืื™ื– ืึทืŸ ืึธืคึฟืŸ ืžืงื•ืจ ืงืึธืœื•ืžื ืขืจ ื“ืึทื˜ืึทื‘ื™ื™ืก ืคืึทืจื•ื•ืึทืœื˜ื•ื ื’ ืกื™ืกื˜ืขื ืคึฟืึทืจ ืึธื ืœื™ื™ืŸ ืึทื ืึทืœื™ืกื™ืก ืึธื ืคึฟืจืขื’ ืคึผืจืึทืกืขืกื™ื ื’ (OLAP), ื‘ืืฉืืคืŸ ื“ื•ืจืš ื™ืึทื ื“ืขืงืก. ืขืก ืื™ื– ื’ืขื ื™ืฆื˜ ื“ื•ืจืš Yandex, CloudFlare, VK.com, Badoo ืื•ืŸ ืื ื“ืขืจืข ื‘ืึทื“ื™ื ื•ื ื’ืก ืึทืจื•ื ื“ื™ ื•ื•ืขืœื˜ ืฆื• ืงืจืึธื ื˜ืึทืงืข ื’ืจื•ื™ืก ืึทืžืึทื•ื ืฅ ืคื•ืŸ ื“ืึทื˜ืŸ (ื™ื ืกืขืจื˜ื™ื ื’ ื˜ื•ื™ื–ื ื˜ืขืจ ืคื•ืŸ ืจืึธื•ื– ืคึผืขืจ ืกืขืงื•ื ื“ืข ืึธื“ืขืจ ืคึผืขื˜ืึทื‘ื™ื˜ืขืก ืคื•ืŸ ื“ืึทื˜ืŸ ืกื˜ืึธืจื“ ืื•ื™ืฃ ื“ื™ืกืง).

ืื™ืŸ ืึท ืจืขื’ื•ืœืขืจ "ืฉื˜ืจื™ืงืœ" ื“ื‘ืžืก, ื‘ื™ื™ืฉืคื™ืœืŸ ืคื•ืŸ ื•ื•ืึธืก ื–ืขื ืขืŸ MySQL, Postgres, MS SQL Server, ื“ืึทื˜ืŸ ื–ืขื ืขืŸ ืกื˜ืึธืจื“ ืื™ืŸ ื“ื™ ืคืืœื’ืขื ื“ืข ืกื“ืจ:

ื ื™ืฆืŸ Clickhouse ื•ื•ื™ ืึท ืคืึทืจื‘ื™ื™ึทื˜ ืคึฟืึทืจ ELK, Big Query ืื•ืŸ TimescaleDB

ืื™ืŸ ื“ืขื ืคืึทืœ, ื•ื•ืึทืœื•ืขืก ืฉื™ื™ึทื›ื•ืช ืฆื• ืื™ื™ืŸ ืจื•ื“ืขืจืŸ ื–ืขื ืขืŸ ืคื™ื–ื™ืงืœื™ ืกื˜ืึธืจื“ ื ื™ืจื‘ื™ื™. ืื™ืŸ ืงืึธืœื•ืžื ืขืจ DBMS, ื•ื•ืึทืœื•ืขืก ืคื•ืŸ ืคืึทืจืฉื™ื“ืขื ืข ืฉืคืืœื˜ืŸ ื–ืขื ืขืŸ ืกื˜ืึธืจื“ ืกืขืคึผืขืจืึทื˜ืœื™, ืื•ืŸ ื“ืึทื˜ืŸ ืคื•ืŸ ืื™ื™ืŸ ื–ื™ื™ึทืœ ื–ืขื ืขืŸ ืกื˜ืึธืจื“ ืฆื•ื–ืึทืžืขืŸ:

ื ื™ืฆืŸ Clickhouse ื•ื•ื™ ืึท ืคืึทืจื‘ื™ื™ึทื˜ ืคึฟืึทืจ ELK, Big Query ืื•ืŸ TimescaleDB

ื‘ื™ื™ืฉืคื™ืœืŸ ืคื•ืŸ ืงืึธืœื•ืžื ืขืจ ื“ื‘ืžืก ื–ืขื ืขืŸ ื•ื•ืขืจื˜ื™ืงืึท, ืคึผืึทืจืึทืงืกืขืœ (ืึทืงื˜ื™ืึทืŸ ืžืึทื˜ืจื™ืฅ, ืึทืžืึทื–ืึธืŸ ืจืขื“ืฉื™ืคื˜), Sybase IQ, Exasol, Infobright, InfiniDB, MonetDB (VectorWise, Actian Vector), LucidDB, SAP HANA, Google Dremel, Google PowerDrill, Druid, kdb+.

ืคึผืึธืกื˜ ืคืึธืจื•ื•ืขืจื“ืขืจ ืคื™ืจืžืข Qwintry ืกื˜ืึทืจื˜ืขื“ ื ื™ืฆืŸ Clickhouse ืื™ืŸ 2018 ืคึฟืึทืจ ืจื™ืคึผืึธืจื˜ื™ื ื’ ืื•ืŸ ืื™ื– ื’ืขื•ื•ืขืŸ ื–ื™ื™ืขืจ ื™ืžืคึผืจืขืกื˜ ืžื™ื˜ ื–ื™ื™ืŸ ืคึผืึทืฉื˜ืขืก, ืกืงืึทืœืึทื‘ื™ืœื™ื˜ื™, SQL ืฉื˜ื™ืฆืŸ ืื•ืŸ ื’ื™ื›ืงื™ื™ึทื˜. ื“ื™ ื’ื™ื›ืงื™ื™ึทื˜ ืคื•ืŸ ื“ืขื DBMS ื‘ืึธืจื“ืขืจื“ ืžื™ื˜ ืžืึทื’ื™ืฉ.

ื™ื–

ืงืœื™ืงื›ืึธื•ืกืข ืื™ื– ืื™ื ืกื˜ืึทืœื™ืจืŸ ืื•ื™ืฃ ื•ื‘ื•ื ื˜ื• ืžื™ื˜ ืื™ื™ืŸ ื‘ืึทืคึฟืขืœ. ืื•ื™ื‘ ืื™ืจ ื•ื•ื™ืกืŸ SQL, ืื™ืจ ืงืขื ื˜ ื’ืœื™ื™ืš ืึธื ื”ื™ื™ื‘ืŸ ื ื™ืฆืŸ Clickhouse ืคึฟืึทืจ ื“ื™ื™ืŸ ื“ืึทืจืฃ. ืึธื‘ืขืจ, ื“ืึธืก ื˜ื•ื˜ ื ื™ืฉื˜ ืžื™ื™ื ืขืŸ ืึทื– ืื™ืจ ืงืขื ืขืŸ ื˜ืึธืŸ "ื•ื•ื™ื™ึทื–ืŸ ืฉืึทืคึฟืŸ ื˜ื™ืฉ" ืื™ืŸ MySQL ืื•ืŸ ื ืึธื›ืžืึทื›ืŸ ืื•ืŸ ืคึผืึทืคึผ ื“ื™ SQL ืื™ืŸ ืงืœื™ืงื›ืึธื•ืกืข.

ืงืึทืžืคึผืขืจื“ ืฆื• MySQL, ืขืก ื–ืขื ืขืŸ ื•ื•ื™ื›ื˜ื™ืง ื“ืึทื˜ืŸ ื˜ื™ืคึผ ื“ื™ืคืขืจืึทื ืกื™ื– ืื™ืŸ ื“ื™ ื˜ื™ืฉ ืกื˜ืฉืขืžืึท ื–ื•ืš, ืึทื–ื•ื™ ืื™ืจ ื•ื•ืขื˜ ื ืึธืš ื“ืึทืจืคึฟืŸ ืขื˜ืœืขื›ืข ืžืึธืœ ืฆื• ื˜ื•ื™ืฉืŸ ื“ื™ ื˜ื™ืฉ ืกื˜ืฉืขืžืึท ื–ื•ืš ืื•ืŸ ืœืขืจื ืขืŸ ื“ื™ ื˜ื™ืฉ ืขื ื“ื–ืฉืึทื ื– ืฆื• ื–ื™ื™ืŸ ื‘ืึทืงื•ื•ืขื.

ืงืœื™ืงื›ืึธื•ืกืข ืึทืจื‘ืขื˜ ื’ืจื•ื™ืก ืึธืŸ ืงื™ื™ืŸ ื ืึธืš ื•ื•ื™ื™ื›ื•ื•ืืจื’, ืึธื‘ืขืจ ืื•ื™ื‘ ืื™ืจ ื•ื•ื™ืœืŸ ืฆื• ื ื•ืฆืŸ ืจืขืคึผืœืึทืงื™ื™ืฉืึทืŸ, ืื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ื™ื ืกื˜ืึทืœื™ืจืŸ ZooKeeper. ืึธื ืคึฟืจืขื’ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืึทื ืึทืœื™ืกื™ืก ื•ื•ื™ื™ื–ื˜ ื•ื™ืกื’ืขืฆื™ื™ื›ื ื˜ ืจืขื–ื•ืœื˜ืึทื˜ืŸ - ื“ื™ ืกื™ืกื˜ืขื ื˜ื™ืฉืŸ ืึทื ื˜ื”ืึทืœื˜ืŸ ืึทืœืข ื“ื™ ืื™ื ืคึฟืึธืจืžืึทืฆื™ืข, ืื•ืŸ ืึทืœืข ื“ื™ ื“ืึทื˜ืŸ ืงืขื ืขืŸ ื–ื™ื™ืŸ ืจื™ื˜ืจื™ื•ื•ื“ ืžื™ื˜ ืึทืœื˜ ืื•ืŸ ื ื•ื“ื ืข ืกืงืœ.

ืคืึธืจืฉื˜ืขืœื•ื ื’

ื ื™ืฆืŸ Clickhouse ื•ื•ื™ ืึท ืคืึทืจื‘ื™ื™ึทื˜ ืคึฟืึทืจ ELK, Big Query ืื•ืŸ TimescaleDB

ื“ื™ ืงืœื™ืงื›ืึธื•ืกืข ื“ืึทื˜ืึทื‘ื™ื™ืก ื”ืื˜ ืึท ื–ื™ื™ืขืจ ืคึผืฉื•ื˜ ืคึผืœืึทืŸ - ืึทืœืข ื ืึธื•ื“ื– ืื™ืŸ ื“ืขื ืงื ื•ื™ืœ ื”ืึธื‘ืŸ ื“ื™ ื–ืขืœื‘ืข ืคืึทื ื’ืงืฉืึทื ืึทืœื™ื˜ื™ ืื•ืŸ ื ืึธืจ ื ื•ืฆืŸ ZooKeeper ืคึฟืึทืจ ืงืึธื•ืึธืจื“ืึทื ื™ื™ืฉืึทืŸ. ืžื™ืจ ื”ืึธื‘ืŸ ื’ืขื‘ื•ื™ื˜ ืึท ืงืœื™ื™ืŸ ืงื ื•ื™ืœ ืคื•ืŸ ืขื˜ืœืขื›ืข ื ืึธื•ื“ื– ืื•ืŸ ื“ื•ืจื›ื’ืขืงืึธื›ื˜ ื˜ืขืกื˜ื™ื ื’, ื‘ืขืฉืึทืก ื•ื•ืึธืก ืžื™ืจ ื’ืขืคึฟื•ื ืขืŸ ืึทื– ื“ื™ ืกื™ืกื˜ืขื ื”ืื˜ ื’ืึทื ืฅ ื™ืžืคึผืจืขืกื™ื•ื• ืคืึธืจืฉื˜ืขืœื•ื ื’, ื•ื•ืึธืก ืงืึธืจืึทืกืคึผืึทื ื“ื– ืฆื• ื“ื™ ืกื˜ื™ื™ื˜ื™ื“ ืึทื“ื•ื•ืึทื ื˜ื™ื“ื–ืฉื™ื– ืื™ืŸ ืึทื ืึทืœื™ืกื™ืก DBMS ื‘ืขื ื˜ืฉืžืึทืจืงืก. ืžื™ืจ ื‘ืึทืฉืœืึธืกืŸ ืฆื• ื ืขืžืขืŸ ืึท ื ืขืขื ื˜ืขืจ ืงื•ืง ืื™ืŸ ื“ืขื ื‘ืึทื’ืจื™ืฃ ื”ื™ื ื˜ืขืจ ClickHouse. ื“ืขืจ ืขืจืฉื˜ืขืจ ืฉื˜ืขืจื•ื ื’ ืคึฟืึทืจ ืคืึธืจืฉื•ื ื’ ืื™ื– ื’ืขื•ื•ืขืŸ ื“ื™ ืคืขืœืŸ ืคื•ืŸ ืžื›ืฉื™ืจื™ื ืื•ืŸ ื“ื™ ืงืœื™ื™ืŸ ืงืœื™ืงื›ืึธื•ืกืข ืงื”ืœ, ืึทื–ื•ื™ ืžื™ืจ ื“ืขืœื•ื•ื•ื“ ืื™ืŸ ื“ืขื ืคึผืœืึทืŸ ืคื•ืŸ ื“ืขื DBMS ืฆื• ืคึฟืึทืจืฉื˜ื™ื™ืŸ ื•ื•ื™ ืขืก ืึทืจื‘ืขื˜.

ClickHouse ืฉื˜ื™ืฆื˜ ื ื™ืฉื˜ ืจื™ืกื™ื•ื•ื™ื ื’ ื“ืึทื˜ืŸ ื’ืœื™ื™ืš ืคึฟื•ืŸ Kafka ื–ื™ื ื˜ ืขืก ืื™ื– ื ืึธืจ ืึท ื“ืึทื˜ืึทื‘ื™ื™ืก, ืึทื–ื•ื™ ืžื™ืจ ื’ืขืฉืจื™ื‘ืŸ ืื•ื ื“ื–ืขืจ ืื™ื™ื’ืขื ืข ืึทื“ืึทืคึผื˜ืขืจ ื“ื™ื ืกื˜ ืื™ืŸ Go. ืขืก ืœื™ื™ืขื ืขืŸ ืงืึทืคึผ'ืŸ ืคึผืจืึธื˜ืึธ ืงืึธื“ืขื“ ืึทืจื˜ื™ืงืœืขืŸ ืคื•ืŸ Kafka, ืงืึธื ื•ื•ืขืจื˜ืขื“ ื–ื™ื™ ืฆื• TSV ืื•ืŸ ื™ื ืกืขืจื˜ืึทื“ ื–ื™ื™ ืื™ืŸ ืงืœื™ืงืงื›ืึธื•ืกืข ืื™ืŸ ื‘ืึทื˜ืฉืึทื– ื“ื•ืจืš ื“ื™ ื”ื˜ื˜ืคึผ ืฆื•ื‘ื™ื ื“. ืžื™ืจ ืฉืคึผืขื˜ืขืจ ืจื™ืจืึธื•ื˜ ื“ืขื ื“ื™ื ืกื˜ ืฆื• ื ื•ืฆืŸ ื“ื™ Go ื‘ื™ื‘ืœื™ืึธื˜ืขืง ืื™ืŸ ืงืึทื ื“ื–ืฉืึทื ื’ืงืฉืึทืŸ ืžื™ื˜ ClickHouse ืก ืื™ื™ื’ืขื ืข ืฆื•ื‘ื™ื ื“ ืฆื• ืคึฟืึทืจื‘ืขืกืขืจืŸ ืคืึธืจืฉื˜ืขืœื•ื ื’. ื•ื•ืขืŸ ืขื•ื•ืึทืœื•ืึทื˜ื™ื ื’ ื“ื™ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคื•ืŸ ืจื™ืกื™ื•ื•ื™ื ื’ ืคึผืึทืงื™ืฅ, ืžื™ืจ ื“ื™ืกืงืึทื•ื•ืขืจื“ ืึท ื•ื•ื™ื›ื˜ื™ืง ื–ืึทืš - ืขืก ืคืืจืงืขืจื˜ ืื•ื™ืก ืึทื– ืคึฟืึทืจ ClickHouse ื“ืขื ืคืึธืจืฉื˜ืขืœื•ื ื’ ื“ืขืคึผืขื ื“ืก ืฉื˜ืืจืง ืื•ื™ืฃ ื“ื™ ื’ืจื™ื™ืก ืคื•ืŸ ื“ืขื ืคึผืึทืงืึทื˜, ื“ืึธืก ืื™ื– ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ืกื™ื™ืžืึทืœื˜ื™ื™ื ื™ืึทืกืœื™ ื™ื ืกืขืจื˜ืึทื“ ืจืึธื•ื–. ืฆื• ืคึฟืึทืจืฉื˜ื™ื™ืŸ ื•ื•ืึธืก ื“ืึธืก ื›ืึทืคึผืึทื ื–, ืžื™ืจ ื’ืขืงื•ืงื˜ ื•ื•ื™ ืงืœื™ืงื›ืึธื•ืกืข ืกื˜ืึธืจื– ื“ืึทื˜ืŸ.

ื“ื™ ื”ื•ื™ืคึผื˜ ืžืึธื˜ืึธืจ, ืึธื“ืขืจ ื’ืึทื ืฅ ืึท ืžืฉืคึผื—ื” ืคื•ืŸ ื˜ื™ืฉ ืขื ื“ื–ืฉืึทื ื–, ื’ืขื ื™ืฆื˜ ื“ื•ืจืš ClickHouse ืฆื• ืงืจืึธื ื“ืึทื˜ืŸ ืื™ื– MergeTree. ื“ืขืจ ืžืึธื˜ืึธืจ ืื™ื– ืงืึทื ืกืขืคึผื˜ืฉื•ืึทืœื™ ืขื ืœืขืš ืฆื• ื“ื™ LSM ืึทืœื’ืขืจื™ื“ืึทื ื’ืขื ื™ืฆื˜ ืื™ืŸ Google BigTable ืึธื“ืขืจ Apache Cassandra, ืึธื‘ืขืจ ื•ื™ืกืžื™ื™ื“ืŸ ืงืึทื ืกื˜ืจืึทืงื˜ื™ื ื’ ืึท ื™ื ื˜ืขืจืžื™ื“ื™ื™ื˜ ื–ื›ึผืจื•ืŸ ื˜ื™ืฉ ืื•ืŸ ืฉืจื™ื™ื‘ื˜ ื“ืึทื˜ืŸ ื’ืœื™ื™ึทืš ืฆื• ื“ื™ืกืง. ื“ืึธืก ื’ื™ื˜ ืขืก ื•ื™ืกื’ืขืฆื™ื™ื›ื ื˜ ืฉืจื™ื™ึทื‘ืŸ ื˜ืจื•ืคึผื•ื˜, ื•ื•ื™ื™ึทืœ ื™ืขื“ืขืจ ื™ื ืกืขืจื˜ืึทื“ ืคึผืึทืงืึทื˜ ืื™ื– ืื•ื™ืกื’ืขืฉื˜ืขืœื˜ ื‘ืœื•ื™ื– ื“ื•ืจืš ื“ื™ ืขืจืฉื˜ื™ืง ืฉืœื™ืกืœ, ืงืึทืžืคึผืจืขืกื˜ ืื•ืŸ ื’ืขืฉืจื™ื‘ืŸ ืฆื• ื“ื™ืกืง ืฆื• ืคืึธืจืขื ืึท ืึธืคึผืฉื ื™ื˜.

ื“ืขืจ ืึทื•ื•ืขืง ืคื•ืŸ ืึท ื–ื™ืงืึธืจืŸ ื˜ื™ืฉ ืึธื“ืขืจ ืงื™ื™ืŸ ื‘ืึทื’ืจื™ืฃ ืคื•ืŸ "ืคืจืขืฉื ืขืกืก" ืคื•ืŸ ื“ืึทื˜ืŸ ืื•ื™ืš ืžื™ื˜ืœ ืึทื– ื–ื™ื™ ืงืขื ืขืŸ ื‘ืœื•ื™ื– ื–ื™ื™ืŸ ืžื•ืกื™ืฃ; ื“ื™ ืกื™ืกื˜ืขื ืฉื˜ื™ืฆื˜ ื ื™ืฉื˜ ื˜ืฉืึทื ื’ื™ื ื’ ืึธื“ืขืจ ื“ื™ืœื™ื˜ื™ื ื’. ื“ืขืจื•ื•ื™ื™ึทืœ, ื“ืขืจ ื‘ืœื•ื™ื– ื•ื•ืขื’ ืฆื• ื•ื™ืกืžืขืงืŸ ื“ืึทื˜ืŸ ืื™ื– ืฆื• ื•ื™ืกืžืขืงืŸ ืขืก ื“ื•ืจืš ืงืึทืœืขื ื“ืึทืจ ื—ื•ื“ืฉ, ื•ื•ื™ื™ึทืœ ืกืขื’ืžืึทื ืฅ ืงื™ื™ื ืžืึธืœ ืึทืจื™ื‘ืขืจ ืึท ื—ื•ื“ืฉ ื’ืจืขื ืขืฅ. ื“ื™ ClickHouse ืžืึทื ืฉืึทืคึฟื˜ ืื™ื– ืึทืงื˜ื™ื•ื•ืœื™ ืืจื‘ืขื˜ืŸ ืฆื• ืžืึทื›ืŸ ื“ืขื ืฉื˜ืจื™ืš ืงื•ืกื˜ืึธืžื™ื–ืึทื‘ืœืข. ืื•ื™ืฃ ื“ื™ ืื ื“ืขืจืข ื”ืึทื ื˜, ืขืก ืžืื›ื˜ ืฉืจื™ื™ื‘ืŸ ืื•ืŸ ืฆื•ื ื•ื™ืคื’ื™ืกืŸ ืกืขื’ืžืึทื ืฅ ืงืึทื ื˜ืขื ืฉืึทืŸ-ืคืจื™ื™, ืึทื–ื•ื™ ื‘ืึทืงื•ืžืขืŸ ื˜ืจื•ืคึผื•ื˜ ื•ื•ืึธื’ ืœื™ื ืขืึทืจืœื™ ืžื™ื˜ ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ืงืึทื ืงืขืจืึทื ื˜ ื™ื ืกืขืจืฅ ื‘ื™ื– ื™ / ืึธ ืึธื“ืขืจ ื”ืึทืจืฅ ื–ืขื˜ื™ืงื•ื ื’ ืึทืงืขืจื–.
ืึธื‘ืขืจ, ื“ืึธืก ืื•ื™ืš ืžื™ื˜ืœ ืึทื– ื“ื™ ืกื™ืกื˜ืขื ืื™ื– ื ื™ืฉื˜ ืคึผืึทืกื™ืง ืคึฟืึทืจ ืงืœื™ื™ืŸ ืคึผืึทืงื™ืฅ, ืึทื–ื•ื™ Kafka ื‘ืึทื“ื™ื ื•ื ื’ืก ืื•ืŸ ื™ื ืกืขืจื˜ืขืจืก ื–ืขื ืขืŸ ื’ืขื ื™ืฆื˜ ืคึฟืึทืจ ื‘ืึทืคืขืจื™ื ื’. ื“ืขืจื ืึธืš, ClickHouse ืื™ืŸ ื“ืขืจ ื”ื™ื ื˜ืขืจื’ืจื•ื ื˜ ืงืขืกื™ื™ื“ืขืจ ื“ื•ืจื›ืคื™ืจืŸ ืกืขื’ืžืขื ื˜ ืฆื•ื ื•ื™ืคื’ื™ืกืŸ, ืึทื–ื•ื™ ืึทื– ืคื™ืœืข ืงืœื™ื™ืŸ ื˜ื™ื™ืœืŸ ืคื•ืŸ ืื™ื ืคึฟืึธืจืžืึทืฆื™ืข ื•ื•ืขื˜ ื–ื™ื™ืŸ ืงืึทืžื‘ื™ื™ื ื“ ืื•ืŸ ืจืขืงืึธืจื“ืขื“ ืžืขืจ ืžืึธืœ, ืึทื–ื•ื™ ื™ื ืงืจื™ืกื™ื ื’ ื“ื™ ืจืขืงืึธืจื“ื™ื ื’ ื™ื ื˜ืขื ืกื™ื˜ื™. ืึธื‘ืขืจ, ืฆื• ืคื™ืœืข ืึทื ืงืึทื ืขืงื˜ื™ื“ ืคึผืึทืจืฅ ื•ื•ืขื˜ ืคืึทืจืฉืึทืคืŸ ืึทื’ืจืขืกื™ื•ื• ื’ืขืจื’ืœ ืคื•ืŸ ื™ื ืกืขืจืฅ ืึทื–ื•ื™ ืœืึทื ื’ ื•ื•ื™ ื“ื™ ืฆื•ื ื•ื™ืคื’ื™ืกืŸ ื”ืืœื˜. ืžื™ืจ ื”ืึธื‘ืŸ ื’ืขืคึฟื•ื ืขืŸ ืึทื– ื“ืขืจ ื‘ืขืกื˜ืขืจ ืงืึธืžืคึผืจืึธืžื™ืก ืฆื•ื•ื™ืฉืŸ ืคืึทืงื˜ื™ืฉ-ืฆื™ื™ื˜ ื™ื ื“ื–ืฉืขืกื˜ืฉืึทืŸ ืื•ืŸ ื™ื ื“ื–ืฉืขืกื˜ืฉืึทืŸ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืื™ื– ืฆื• ืึทืจื™ื™ึทื ืฆื™ืขืŸ ืึท ืœื™ืžื™ื˜ืขื“ ื ื•ืžืขืจ ืคื•ืŸ ื™ื ืกืขืจืฅ ืคึผืขืจ ืกืขืงื•ื ื“ืข ืื™ืŸ ื“ื™ ื˜ื™ืฉ.

ื“ืขืจ ืฉืœื™ืกืœ ืฆื• ืœื™ื™ืขื ืขืŸ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคื•ืŸ ื“ื™ ื˜ื™ืฉ ืื™ื– ื™ื ื“ืขืงืกื™ื ื’ ืื•ืŸ ื“ื™ ืึธืจื˜ ืคื•ืŸ ื“ื™ ื“ืึทื˜ืŸ ืื•ื™ืฃ ื“ื™ืกืง. ื ื™ื˜ ืงื™ื™ืŸ ืขื ื™ืŸ ื•ื•ื™ ืฉื ืขืœ ื“ื™ ืคึผืจืึทืกืขืกื™ื ื’ ืื™ื–, ื•ื•ืขืŸ ื“ืขืจ ืžืึธื˜ืึธืจ ื“ืึทืจืฃ ื™ื‘ืขืจืงื•ืงืŸ ื˜ืขืจืึทื‘ื™ื˜ืขืก ืคื•ืŸ ื“ืึทื˜ืŸ ืคื•ืŸ ื“ื™ืกืง ืื•ืŸ ื ืึธืจ ื ื•ืฆืŸ ืึท ื˜ื™ื™ืœ ืคื•ืŸ ืขืก, ืขืก ื•ื•ืขื˜ ื ืขืžืขืŸ ืฆื™ื™ื˜. ClickHouse ืื™ื– ืึท ืงืึธืœื•ืžื ืขืจ ืงืจืึธื, ืึทื–ื•ื™ ื™ืขื“ืขืจ ืึธืคึผืฉื ื™ื˜ ื›ึผื•ืœืœ ืึท ื˜ืขืงืข ืคึฟืึทืจ ื™ืขื“ืขืจ ื–ื™ื™ึทืœ (ื–ื™ื™ึทืœ) ืžื™ื˜ ืื•ื™ืกื’ืขืฉื˜ืขืœื˜ ื•ื•ืึทืœื•ืขืก ืคึฟืึทืจ ื™ืขื“ืขืจ ืจื•ื“ืขืจืŸ. ื“ืขื ื•ื•ืขื’, ื’ืึทื ืฅ ืฉืคืืœื˜ืŸ ืคืขืœื ื“ื™ืง ืคื•ืŸ ื“ื™ ืึธื ืคึฟืจืขื’ ืงืขื ืขืŸ ื–ื™ื™ืŸ ืกืงื™ืคึผื˜ ืขืจืฉื˜ืขืจ, ืื•ืŸ ื“ืขืžืึธืœื˜ ืงื™ื™ืคืœ ืกืขืœื– ืงืขื ืขืŸ ื–ื™ื™ืŸ ืคึผืจืึทืกืขืกื˜ ืื™ืŸ ืคึผืึทืจืึทืœืขืœ ืžื™ื˜ ื•ื•ืขืงื˜ืึธืจื™ื–ืขื“ ื“ื•ืจื›ืคื™ืจื•ื ื’. ืฆื• ื•ื™ืกืžื™ื™ื“ืŸ ืึท ืคื•ืœ ื™ื‘ืขืจืงื•ืงืŸ, ื™ืขื“ืขืจ ืึธืคึผืฉื ื™ื˜ ื”ืื˜ ืึท ืงืœื™ื™ืŸ ืื™ื ื“ืขืงืก ื˜ืขืงืข.

ื’ืขื’ืขื‘ืŸ ืึทื– ืึทืœืข ืฉืคืืœื˜ืŸ ื–ืขื ืขืŸ ืื•ื™ืกื’ืขืฉื˜ืขืœื˜ ื“ื•ืจืš ื“ื™ "ืขืจืฉื˜ื™ืง ืฉืœื™ืกืœ", ื“ื™ ืื™ื ื“ืขืงืก ื˜ืขืงืข ื›ึผื•ืœืœ ื‘ืœื•ื™ื– ื“ื™ ืœืึทื‘ืขืœืก (ืงืึทืคึผื˜ืฉืขืจื“ ืจืึธื•ื–) ืคื•ืŸ ื™ืขื“ืขืจ ื ื˜ ืจื•ื“ืขืจืŸ ืฆื• ืงืขื ืขืŸ ื”ืึทืœื˜ืŸ ื–ื™ื™ ืื™ืŸ ื–ื›ึผืจื•ืŸ ืืคื™ืœื• ืคึฟืึทืจ ื–ื™ื™ืขืจ ื’ืจื•ื™ืก ื˜ื™ืฉืŸ. ืคึฟืึทืจ ื‘ื™ื™ึทืฉืคึผื™ืœ, ืื™ืจ ืงืขื ืขืŸ ืฉื˜ืขืœืŸ ื“ื™ ืคืขืœื™ืงื™ื™ึทื˜ ืกืขื˜ื˜ื™ื ื’ืก ืฆื• "ืฆื™ื™ื›ืŸ ื™ืขื“ืขืจ 8192 ืจื•ื“ืขืจืŸ", ืื•ืŸ ื“ืขืžืึธืœื˜ "ืงื ืึทืคึผ" ื™ื ื“ืขืงืกื™ื ื’ ืคื•ืŸ ืึท ื˜ื™ืฉ ืžื™ื˜ 1 ื˜ืจื™ืœื™ืึทืŸ. ืฉื•ืจื•ืช ื•ื•ืึธืก ืคึผืึทืกื™ืง ืœื™ื™ื›ื˜ ืื™ืŸ ื–ื›ึผืจื•ืŸ ื•ื•ืขื˜ ื ืขืžืขืŸ ื‘ืœื•ื™ื– 122 ืื•ืชื™ื•ืช.

ืกื™ืกื˜ืขื ืึทื ื˜ื•ื•ื™ืงืœื•ื ื’

ื“ื™ ืึทื ื˜ื•ื•ื™ืงืœื•ื ื’ ืื•ืŸ ืคึฟืึทืจื‘ืขืกืขืจื•ื ื’ ืคื•ืŸ Clickhouse ืงืขื ืขืŸ ื–ื™ื™ืŸ ื˜ืจื™ื™ืกื˜ ืื™ืŸ ื’ื™ื˜ื”ื•ื‘ ืจืขืคึผืึธ ืื•ืŸ ืžืึทื›ืŸ ื–ื™ื›ืขืจ ืึทื– ื“ืขืจ ืคึผืจืึธืฆืขืก ืคื•ืŸ "ื•ื•ืึทืงืกืŸ ืึทืจื•ื™ืฃ" ืึทืงืขืจื– ืื™ืŸ ืึทืŸ ื™ืžืคึผืจืขืกื™ื•ื• ื’ืึทื ื’.

ื ื™ืฆืŸ Clickhouse ื•ื•ื™ ืึท ืคืึทืจื‘ื™ื™ึทื˜ ืคึฟืึทืจ ELK, Big Query ืื•ืŸ TimescaleDB

ืคึผืึธืคึผื•ืœืึทืจื™ื˜ืขื˜

ื“ื™ ืคึผืึธืคึผื•ืœืึทืจื™ื˜ืขื˜ ืคื•ืŸ ืงืœื™ืงื›ืึธื•ืกืข ืกื™ืžื– ืฆื• ื•ื•ืึทืงืกืŸ ืขืงืกืคึผืึธื•ื ืขื ืฉืึทืœื™, ืกืคึผืขืฆื™ืขืœ ืื™ืŸ ื“ื™ ืจื•ืกื™ืฉ-ื’ืขืจืขื“ื˜ ืงื”ืœ. ืœืขืฆื˜ืข ื™ืึธืจ ืก ื”ื•ื™ืš ืžืึทืกืข 2018 ืงืึธื ืคึฟืขืจืขื ืฅ (ืžืึธืกืงื•ื•ืข, 8-9 ื ืื•ื•ืขืžื‘ืขืจ 2018) ื’ืขื•ื•ื™ื–ืŸ ืึทื– ืึทื–ืึท ืžืึธื ืกื˜ืขืจืก ื•ื•ื™ vk.com ืื•ืŸ Badoo ื ื•ืฆืŸ Clickhouse, ืžื™ื˜ ื•ื•ืึธืก ื–ื™ื™ ืึทืจื™ื™ึทื ืœื™ื™ื’ืŸ ื“ืึทื˜ืŸ (ืœืžืฉืœ, ืœืึธื’ืก) ืคื•ืŸ ื˜ืขื ืก ืคื•ืŸ ื˜ื•ื™ื–ื ื˜ืขืจ ืคื•ืŸ ืกืขืจื•ื•ืขืจืก ืกื™ื™ืžืึทืœื˜ื™ื™ื ื™ืึทืกืœื™. ืื™ืŸ ืึท 40 ืžื™ื ื•ื˜ ื•ื•ื™ื“ืขื ื™ื•ืจื™ ื ืึทืกืจืขื˜ื“ื™ื ืึธื•ื• ืคื•ืŸ ื“ื™ VKontakte ืžืึทื ืฉืึทืคึฟื˜ ืจืขื“ื˜ ื•ื•ืขื’ืŸ ื•ื•ื™ ื“ืึธืก ืื™ื– ื’ืขื˜ืืŸ. ื‘ืึทืœื“ ืžื™ืจ ื•ื•ืขืœืŸ ืคึผืึธืกื˜ืŸ ื“ื™ ื˜ืจืึทื ืกืงืจื™ืคึผื˜ ืื•ื™ืฃ Habr ืคึฟืึทืจ ื™ื– ืคื•ืŸ ืืจื‘ืขื˜ืŸ ืžื™ื˜ ื“ืขื ืžืึทื˜ืขืจื™ืึทืœ.

ืึทืคึผืคึผืœื™ืงืึทื˜ื™ืึธื ืก

ื ืึธืš ืกืคึผืขื ื“ื™ื ื’ ืขื˜ืœืขื›ืข ืžืึธืœ ืจื™ืกืขืจื˜ืฉื™ื ื’, ืื™ืš ื˜ืจืึทื›ื˜ืŸ ืขืก ื–ืขื ืขืŸ ื’ืขื‘ื™ื˜ืŸ ื•ื•ื• ClickHouse ืงืขืŸ ื–ื™ื™ืŸ ื ื•ืฆื™ืง ืึธื“ืขืจ ื’ืึธืจ ืคืึทืจื‘ื™ื™ึทื˜ืŸ ืื ื“ืขืจืข, ืžืขืจ ื˜ืจืื“ื™ืฆื™ืื ืขืœืŸ ืื•ืŸ ืคืึธืœืงืก ืกืึทืœื•ืฉืึทื ื– ืึทื–ืึท ื•ื•ื™ MySQL, PostgreSQL, ELK, Google Big Query, Amazon RedShift, TimescaleDB, Hadoop, MapReduce, Pinot ืื•ืŸ ื“ืจื•ื™ื“. ื“ื™ ืคืืœื’ืขื ื“ืข ื‘ืืฉืจื™ื™ื‘ื˜ ื“ื™ ื“ืขื˜ืึทื™ืœืก ืคื•ืŸ ื ื™ืฆืŸ ClickHouse ืฆื• ืคืึทืจื”ื™ื™ึทื ื˜ื™ืงืŸ ืึธื“ืขืจ ื’ืึธืจ ืคืึทืจื‘ื™ื™ึทื˜ืŸ ื“ื™ ืื•ื™ื‘ืŸ DBMS.

ืคืึทืจื‘ืจื™ื™ื˜ืขืจืŸ ื“ื™ ืงื™ื™ืคึผืึทื‘ื™ืœืึทื˜ื™ื– ืคื•ืŸ MySQL ืื•ืŸ PostgreSQL

ืœืขืฆื˜ื ืก ืžื™ืจ ื”ืึธื‘ืŸ ื˜ื™ื™ืœ ืจื™ืคึผืœื™ื™ืกื˜ MySQL ืžื™ื˜ ClickHouse ืคึฟืึทืจ ืื•ื ื“ื–ืขืจ ื ื•ื–ืœืขื˜ืขืจ ืคึผืœืึทื˜ืคืึธืจืžืข ืžืึทื•ื˜ื™ืง ื ื•ื–ืœืขื˜ืขืจ. ื“ืขืจ ืคึผืจืึธื‘ืœืขื ืื™ื– ื’ืขื•ื•ืขืŸ ืึทื– MySQL, ืจืขื›ื˜ ืฆื• ืึท ื ืขื‘ืขืš ืคึผืœืึทืŸ, ืื™ื– ื’ืขื•ื•ืขืŸ ืœืึธื’ื™ื ื’ ื™ืขื“ืขืจ ืื™ืžืขื™ืœ ื’ืขืฉื™ืงื˜ ืื•ืŸ ื™ืขื“ืขืจ ืœื™ื ืง ืื™ืŸ ื“ืขื ื‘ืœื™ืฆืคึผืึธืกื˜ ืžื™ื˜ ืึท Base64 ื”ืึทืฉ, ืงืจื™ื™ื™ื˜ื™ื ื’ ืึท ืจื™ื–ื™ืง MySQL ื˜ื™ืฉ (email_stats). ื ืึธืš ืฉื™ืงื˜ ื‘ืœื•ื™ื– 10 ืžื™ืœื™ืึธืŸ ื™ืžื™ื™ืœื– ืฆื• ืกืขืจื•ื•ื™ืก ืื‘ืื ืขื ื˜ืŸ, ื“ืขื ื˜ื™ืฉ ื”ืึธื˜ ืคืึทืจื ื•ืžืขืŸ 150 ื’ื™ื’ืื‘ื™ื™ื˜ ืคื•ืŸ ื˜ืขืงืข ืคึผืœืึทืฅ, ืื•ืŸ MySQL ืื ื’ืขื”ื•ื™ื‘ืŸ ืฆื• ื–ื™ื™ืŸ "ื ืึทืจื™ืฉ" ืื•ื™ืฃ ืคึผืฉื•ื˜ ืคึฟืจืื’ืŸ. ืฆื• ืคืึทืจืจื™ื›ื˜ืŸ ื“ื™ ื˜ืขืงืข ืคึผืœืึทืฅ ืึทืจื•ื™ืกื’ืขื‘ืŸ, ืžื™ืจ ื”ืฆืœื—ื” ื’ืขื•ื•ื™ื™ื ื˜ InnoDB ื˜ื™ืฉ ืงืึทืžืคึผืจืขืฉืึทืŸ ื•ื•ืึธืก ืจื™ื“ื•ืกื˜ ืขืก ืžื™ื˜ ืึท ืคืึทืงื˜ืึธืจ ืคื•ืŸ 4. ืึธื‘ืขืจ, ืขืก ื ืึธืš ื˜ื•ื˜ ื ื™ืฉื˜ ืžืึทื›ืŸ ื–ื™ื ืขืŸ ืฆื• ืงืจืึธื ืžืขืจ ื•ื•ื™ 20-30 ืžื™ืœื™ืึธืŸ ื™ืžื™ื™ืœื– ืื™ืŸ MySQL ื ืึธืจ ืคึฟืึทืจ ื“ื™ ืฆื•ืœื™ื‘ ืคื•ืŸ ืœื™ื™ืขื ืขืŸ ื’ืขืฉื™ื›ื˜ืข, ื–ื™ื ื˜ ืงื™ื™ืŸ ืคึผืฉื•ื˜ ืึธื ืคึฟืจืขื’ ื•ื•ืึธืก ืคึฟืึทืจ ืขื˜ืœืขื›ืข ืกื™ื‘ื” ื“ืึทืจืฃ ืฆื• ื˜ืึธืŸ ืึท ืคื•ืœ ื™ื‘ืขืจืงื•ืงืŸ ืจืขื–ื•ืœื˜ืึทื˜ืŸ ืื™ืŸ ื•ื™ืกื‘ื™ื™ึทื˜ืŸ ืื•ืŸ ืึท ืคึผืœืึทืฅ ืคื•ืŸ ืื™ืš / ืึธ ืœืืกื˜, ืœื•ื™ื˜ ื•ื•ืึธืก ืžื™ืจ ื”ืึธื‘ืŸ ืงืขืกื™ื™ื“ืขืจ ื‘ืึทืงื•ืžืขืŸ ื•ื•ืึธืจื ื™ื ื’ื– ืคื•ืŸ ื–ืึทื‘ื‘ื™ืงืก.

ื ื™ืฆืŸ Clickhouse ื•ื•ื™ ืึท ืคืึทืจื‘ื™ื™ึทื˜ ืคึฟืึทืจ ELK, Big Query ืื•ืŸ TimescaleDB

ืงืœื™ืงื›ืึธื•ืกืข ื ื™ืฆื˜ ืฆื•ื•ื™ื™ ืงืึทืžืคึผืจืขืฉืึทืŸ ืึทืœื’ืขืจื™ื“ืึทืžื– ื•ื•ืึธืก ืจืขื“ื•ืฆื™ืจืŸ ื“ื™ ื“ืึทื˜ืŸ ื‘ืึทื ื“ ืžื™ื˜ ื‘ืขืขืจืขืš 3-4 ืžืืœ, ืึธื‘ืขืจ ืื™ืŸ ื“ืขื ื‘ืึทื–ื•ื ื“ืขืจ ืคืึทืœ ื“ื™ ื“ืึทื˜ืŸ ื–ืขื ืขืŸ ืกืคึผืขืฆื™ืขืœ "ืงืึธืžืคึผืจืขืกืกืึทื‘ืึทืœ".

ื ื™ืฆืŸ Clickhouse ื•ื•ื™ ืึท ืคืึทืจื‘ื™ื™ึทื˜ ืคึฟืึทืจ ELK, Big Query ืื•ืŸ TimescaleDB

ืจื™ืคึผืœื™ื™ืกื™ื ื’ ELK

ื‘ืึทื–ื™ืจื˜ ืื•ื™ืฃ ืžื™ื™ืŸ ืื™ื™ื’ืขื ืข ื“ืขืจืคืึทืจื•ื ื’, ื“ื™ ELK ืึธื ืœื™ื™ื’ืŸ (ElasticSearch, Logstash ืื•ืŸ Kibana, ืื™ืŸ ื“ืขื ื‘ืึทื–ื•ื ื“ืขืจ ืคืึทืœ ElasticSearch) ืจื™ืงื•ื•ื™ื™ืขืจื– ืคื™ืœ ืžืขืจ ืจืขืกื•ืจืกืŸ ืฆื• ืœื•ื™ืคืŸ ื•ื•ื™ ื ื™ื™ื˜ื™ืง ืฆื• ืงืจืึธื ืœืึธื’ืก. ElasticSearch ืื™ื– ืึท ื’ืจื•ื™ืก ืžืึธื˜ืึธืจ ืื•ื™ื‘ ืื™ืจ ื“ืึทืจืคึฟืŸ ื’ื•ื˜ ืคื•ืœ-ื˜ืขืงืกื˜ ืงืœืึธืฅ ื–ื•ื›ืŸ (ื•ื•ืึธืก ืื™ืš ื˜ืึธืŸ ื ื™ื˜ ื˜ืจืึทื›ื˜ืŸ ืื™ืจ ื˜ืึทืงืข ื“ืึทืจืคึฟืŸ), ืึธื‘ืขืจ ืื™ืš ื‘ื™ืŸ ื•ื•ืึทื ื“ืขืจื™ื ื’ ื•ื•ืึธืก ืขืก ืื™ื– ื’ืขื•ื•ืืจืŸ ื“ื™ ื“ืข ืคืึทืงื˜ืึธ ื ืึธืจืžืึทืœ ืœืึธื’ื™ื ื’ ืžืึธื˜ืึธืจ. ื–ื™ื™ืŸ ื™ื ื“ื–ืฉืขืกื˜ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืงืึทืžื‘ื™ื™ื ื“ ืžื™ื˜ Logstash ื”ืื˜ ืื•ื ื“ื– ืคึผืจืึธื‘ืœืขืžืก ืืคื™ืœื• ืื•ื ื˜ืขืจ ื’ืึทื ืฅ ืœื™ื›ื˜ ืœืึธื•ื“ื– ืื•ืŸ ืคืืจืœืื ื’ื˜ ืื•ื ื“ื– ืฆื• ืœื™ื™ื’ืŸ ืžืขืจ ืื•ืŸ ืžืขืจ ื‘ืึทืจืึทืŸ ืื•ืŸ ื“ื™ืกืง ืคึผืœืึทืฅ. ื•ื•ื™ ืึท ื“ืึทื˜ืึทื‘ื™ื™ืก, Clickhouse ืื™ื– ื‘ืขืกืขืจ ื•ื•ื™ ElasticSearch ืคึฟืึทืจ ื“ื™ ืคืืœื’ืขื ื“ืข ืกื™ื‘ื•ืช:

  • ืกืงืœ ื“ื™ืึทืœืขืงื˜ ืฉื˜ื™ืฆืŸ;
  • ื“ืขืจ ื‘ืขืกื˜ืขืจ ื’ืจืึทื“ ืคื•ืŸ ืงืึทืžืคึผืจืขืฉืึทืŸ ืคื•ืŸ ืกื˜ืึธืจื“ ื“ืึทื˜ืŸ;
  • ืฉื˜ื™ืฆืŸ ืคึฟืึทืจ ืจืขื’ืขืงืก ืจืขื’ื•ืœืขืจ ืื•ื™ืกื“ืจื•ืง ืึธื ืคึฟืจืขื’ืŸ ืึทื ืฉื˜ืึธื˜ ืคื•ืŸ ืคื•ืœ ื˜ืขืงืกื˜ ืึธื ืคึฟืจืขื’ืŸ;
  • ื™ืžืคึผืจื•ื•ื•ื“ ืึธื ืคึฟืจืขื’ ืกืงืขื“ื–ืฉื•ืœื™ื ื’ ืื•ืŸ ื”ืขื›ืขืจ ืงื•ื™ืœืขืœื“ื™ืง ืคืึธืจืฉื˜ืขืœื•ื ื’.

ื“ืขืจื•ื•ื™ื™ึทืœ, ื“ื™ ื‘ื™ื’ืึทืกื˜ ืคึผืจืึธื‘ืœืขื ืึทื– ืขืจื™ื™ื–ืึทื– ื•ื•ืขืŸ ืงืึทืžืคึผืขืจื™ื ื’ ClickHouse ืžื™ื˜ ELK ืื™ื– ื“ื™ ืคืขืœืŸ ืคื•ืŸ ืกืึทืœื•ืฉืึทื ื– ืคึฟืึทืจ ื•ืคึผืœืึธืึทื“ื™ื ื’ ืœืึธื’ืก, ื•ื•ื™ ื’ืขื–ื•ื ื˜ ื•ื•ื™ ื“ื™ ืคืขืœืŸ ืคื•ืŸ ื“ืึทืงื™ื•ืžืขื ื˜ื™ื™ืฉืึทืŸ ืื•ืŸ ื˜ื•ื˜ืึธืจื™ืึทืœื– ืื•ื™ืฃ ื“ืขืจ ื˜ืขืžืข. ื“ืขืจืฆื•, ื™ืขื“ืขืจ ื‘ืึทื ื™ืฆืขืจ ืงืขื ืขืŸ ืงืึทื ืคื™ื’ื™ืขืจ ELK ื ื™ืฆืŸ ื“ื™ ื“ื™ื’ื™ื˜ืึทืœ ืืงืขืืŸ ืžืึทื ื•ืึทืœ, ื•ื•ืึธืก ืื™ื– ื–ื™ื™ืขืจ ื•ื•ื™ื›ื˜ื™ืง ืคึฟืึทืจ ื“ื™ ื’ื™ืš ื™ืžืคึผืœืึทืžืขื ื˜ื™ื™ืฉืึทืŸ ืคื•ืŸ ืึทื–ืึท ื˜ืขืงื ืึทืœืึทื“ื–ืฉื™ื–. ืขืก ืื™ื– ืึท ื“ืึทื˜ืึทื‘ื™ื™ืก ืžืึธื˜ืึธืจ, ืึธื‘ืขืจ ืขืก ืื™ื– ื ืึธืš ืงื™ื™ืŸ Filebeat ืคึฟืึทืจ ืงืœื™ืงืงื›ืึทื•ืก. ื™ื, ืขืก ืื™ื– ื“ืึธืจื˜ ื’ืœืึทื˜ ืื•ืŸ ืึท ืกื™ืกื˜ืขื ืคึฟืึทืจ ืืจื‘ืขื˜ืŸ ืžื™ื˜ ืœืึธื’ืก ืœืึธื’ื”ืึธื•ืกืข, ืขืก ืื™ื– ืึท ื’ืขืฆื™ื™ึทื’ ืงืœื™ืงืงื˜ื™ื™ืœ ืฆื• ืึทืจื™ื™ึทืŸ ืœืึธื’ ื˜ืขืงืข ื“ืึทื˜ืŸ ืื™ืŸ ืงืœื™ืงืงื›ืึทื•ืก, ืึธื‘ืขืจ ืึทืœืข ื“ืขื ื ืขืžื˜ ืžืขืจ ืฆื™ื™ื˜. ืึธื‘ืขืจ, ClickHouse ืื™ื– ื ืึธืš ื“ืขืจ ืคื™ืจืขืจ ืจืขื›ื˜ ืฆื• ื–ื™ื™ืŸ ืคึผืึทืฉื˜ืขืก, ืึทื–ื•ื™ ืืคื™ืœื• ื‘ื™ื’ื™ื ืขืจื– ืงืขื ืขืŸ ืœื™ื™ื›ื˜ ื™ื ืกื˜ืึทืœื™ืจืŸ ืขืก ืื•ืŸ ืึธื ื”ื™ื™ื‘ืŸ ื ื™ืฆืŸ ืขืก ื’ืึธืจ ืคืึทื ื’ืงืฉืึทื ืึทืœื™ ืื™ืŸ ื‘ืœื•ื™ื– 10 ืžื™ื ื•ื˜.

ืคึผืจื™ืคืขืจื™ื ื’ ืžื™ื ื™ืžืึทืœื™ืกื˜ ืกืึทืœื•ืฉืึทื ื–, ืื™ืš ื’ืขืคืจื•ืื•ื•ื˜ ื ื™ืฆืŸ FluentBit, ืึท ื’ืขืฆื™ื™ึทื’ ืคึฟืึทืจ ืฉื™ืคึผื™ื ื’ ืœืึธื’ืก ืžื™ื˜ ื–ื™ื™ืขืจ ืงืœื™ื™ืŸ ื–ื›ึผืจื•ืŸ, ืฆื•ื–ืึทืžืขืŸ ืžื™ื˜ ClickHouse, ื‘ืฉืขืช ืื™ืš ื˜ืจื™ื™ื ื’ ืฆื• ื•ื™ืกืžื™ื™ื“ืŸ ื ื™ืฆืŸ Kafka. ืึธื‘ืขืจ, ืžื™ื ืขืจื•ื•ืขืจื˜ื™ืง ื™ื ืงืึทืžืคึผืึทื˜ืึทื‘ื™ืœืึทื˜ื™ื– ื“ืึทืจืคึฟืŸ ืฆื• ื–ื™ื™ืŸ ืึทื“ืจืขืกื“, ืึทื–ืึท ื•ื•ื™ ื“ืึทื˜ืข ืคึฟืึธืจืžืึทื˜ ืคึผืจืึธื‘ืœืขืžืกืื™ื™ื“ืขืจ ื“ืึธืก ืงืขืŸ ื–ื™ื™ืŸ ื’ืขื˜ืืŸ ืึธืŸ ื“ื™ ืคึผืจืึทืงืกื™ ืฉื™ื›ื˜ืข ื•ื•ืึธืก ืงืึทื ื•ื•ืขืจืฅ ื“ื™ ื“ืึทื˜ืŸ ืคื•ืŸ FluentBit ืฆื• ClickHouse.

ื•ื•ื™ ืึทืŸ ืึธืœื˜ืขืจื ืึทื˜ื™ื•ื•, ืงื™ื‘ืึทื ืึท ืงืขื ืขืŸ ื–ื™ื™ืŸ ื’ืขื•ื•ื™ื™ื ื˜ ื•ื•ื™ ืึท ืงืœื™ืงื›ืึธื•ืกืข ื‘ืึทืงืขื ื“ ื’ืจืืคืื ื. ืคื•ืŸ ื•ื•ืึธืก ืื™ืš ืคึฟืึทืจืฉื˜ื™ื™ืŸ, ื“ืึธืก ืงืขืŸ ืคืึทืจืฉืึทืคืŸ ืคืึธืจืฉื˜ืขืœื•ื ื’ ื™ืฉื•ื– ื•ื•ืขืŸ ืจืขื ื“ืขืจื™ื ื’ ืจื™ื–ื™ืง ื ื•ืžืขืจ ืคื•ืŸ ื“ืึทื˜ืŸ ืคื•ื ืงื˜ืŸ, ืกืคึผืขืฆื™ืขืœ ืžื™ื˜ ืขืœื˜ืขืจืข ื•ื•ืขืจืกื™ืขืก ืคื•ืŸ Grafana. ืžื™ืจ ื”ืึธื‘ืŸ ื ื™ืฉื˜ ืคึผืจื•ึผื•ื•ื˜ ื“ืึธืก ื‘ื™ื™ Qwintry ื ืึธืš, ืึธื‘ืขืจ ื˜ืขื ื•ืช ื•ื•ืขื’ืŸ ื“ืขื ื“ืขืจืฉื™ื™ึทื ืขืŸ ืคื•ืŸ ืฆื™ื™ื˜ ืฆื• ืฆื™ื™ื˜ ืื•ื™ืฃ ื“ื™ ClickHouse ืฉื˜ื™ืฆืŸ ืงืึทื ืึทืœ ืื•ื™ืฃ ื˜ืขืœืขื’ืจืึทื.

ืคืึทืจื‘ื™ื™ึทื˜ ืคื•ืŸ Google Big Query ืื•ืŸ Amazon RedShift (ืœื™ื™ื–ื•ื ื’ ืคึฟืึทืจ ื’ืจื•ื™ืก ืงืึธืžืคึผืึทื ื™ืขืก)

ื“ื™ ื™ื“ืขืึทืœ ื ื•ืฆืŸ ืคืึทืœ ืคึฟืึทืจ BigQuery ืื™ื– ืฆื• ืžืึทืกืข 1 ื˜ื‘ ืคื•ืŸ JSON ื“ืึทื˜ืŸ ืื•ืŸ ืœื•ื™ืคืŸ ืึทื ืึทืœื™ืกื™ืก ืคึฟืจืื’ืŸ ืื•ื™ืฃ ืขืก. Big Query ืื™ื– ืึท ื•ื™ืกื’ืขืฆื™ื™ื›ื ื˜ ืคึผืจืึธื“ื•ืงื˜ ื•ื•ืขืžืขื ืก ืกืงืึทืœืึทื‘ื™ืœื™ื˜ื™ ืงืขื ืขืŸ ื ื™ื˜ ื–ื™ื™ืŸ ืึธื•ื•ื•ืขืจืกื˜ื™ื™ื˜ื™ื“. ื“ืึธืก ืื™ื– ืคื™ืœ ืžืขืจ ืงืึธืžืคึผืœื™ืฆื™ืจื˜ ื•ื•ื™ื™ื›ื•ื•ืืจื’ ื•ื•ื™ ClickHouse, ื•ื•ืึธืก ืœื•ื™ืคื˜ ืื•ื™ืฃ ืึทืŸ ื™ื ืขืจืœืขืš ืงื ื•ื™ืœ, ืึธื‘ืขืจ ืคึฟื•ืŸ ื“ืขื ืงืœื™ืขื ื˜ ืก ืคื•ื ื˜ ืคื•ืŸ ืžื™ื™ื ื•ื ื’ ืขืก ื”ืื˜ ืึท ืคึผืœืึทืฅ ืื™ืŸ ืคึผืจืึธืกื˜ ืžื™ื˜ ClickHouse. ื‘ื™ื’ืงื•ื•ืขืจื™ ืงืขืŸ ื’ืขืฉื•ื•ื™ื ื“ ื•ื•ืขืจืŸ ื˜ื™ื™ึทืขืจ ืึทืžืึธืœ ืื™ืจ ืึธื ื”ื™ื™ื‘ ืคึผื™ื™ื™ื ื’ ืคึผืขืจ ืกืขืœืขืงื˜, ืึทื–ื•ื™ ืขืก ืื™ื– ืึท ืืžืช ืกืึทืึทืก ืœื™ื™ื–ื•ื ื’ ืžื™ื˜ ืึทืœืข ื–ื™ื™ึทืŸ ืคึผืจืึธืก ืื•ืŸ ืงืึธื ืก.

ClickHouse ืื™ื– ื“ืขืจ ื‘ืขืกื˜ืขืจ ื‘ืจื™ืจื” ื•ื•ืขืŸ ืื™ืจ ืœื•ื™ืคืŸ ืึท ืคึผืœืึทืฅ ืคื•ืŸ ืงืึทืžืคึผื™ื•ื˜ื™ืฉืึทื ืึทืœื™ ื˜ื™ื™ึทืขืจ ืคึฟืจืื’ืŸ. ื“ื™ ืžืขืจ ืกืขืœืขืงื˜ ืงื•ื•ื™ืจื™ื– ืื™ืจ ืœื•ื™ืคืŸ ื™ืขื“ืขืจ ื˜ืึธื’, ื“ื™ ืžืขืจ ืขืก ืžืื›ื˜ ื–ื™ื ืขืŸ ืฆื• ืคืึทืจื‘ื™ื™ึทื˜ืŸ ื‘ื™ื’ ืึธื ืคึฟืจืขื’ ืžื™ื˜ ืงืœื™ืงื›ืึธื•ืกืข, ื•ื•ื™ื™ึทืœ ืึทื–ืึท ืึท ืคืึทืจื‘ื™ื™ึทื˜ ืงืขื ืขืŸ ืฉืคึผืึธืจืŸ ื˜ื•ื™ื–ื ื˜ืขืจ ืคื•ืŸ ื“ืึธืœืœืึทืจืก ื•ื•ืขืŸ ืขืก ืงื•ืžื˜ ืฆื• ืคื™ืœืข ื˜ืขืจืึทื‘ื™ื˜ืขืก ืคื•ืŸ ื“ืึทื˜ืŸ ืคึผืจืึทืกืขืกื˜. ื“ืึธืก ืื™ื– ื ื™ืฉื˜ ืึทืคึผืœื™ื™ื– ืฆื• ืกื˜ืึธืจื“ ื“ืึทื˜ืŸ, ื•ื•ืึธืก ืื™ื– ื’ืึทื ืฅ ื‘ื™ืœื™ืง ืฆื• ืคึผืจืึธืฆืขืก ืื™ืŸ ื’ืจื•ื™ืก ืงื•ื•ืขืจื™.

ืื™ืŸ ืึทืŸ ืึทืจื˜ื™ืงืœ ืคื•ืŸ ืึทืœื˜ื™ื ื™ื˜ื™ ืงืึธ-ื’ืจื™ื ื“ืขืจ ืืœืขืงืกืื ื“ืขืจ ื–ื™ื™ืฆืขื•ื• "ื™ื‘ืขืจื‘ื™ื™ึทื˜ ืฆื• ืงืœื™ืงื›ืึธื•ืกืข" ืจืขื“ื˜ ื•ื•ืขื’ืŸ ื“ื™ ื‘ืขื ืขืคื™ืฅ ืคื•ืŸ ืึทื–ืึท ืึท DBMS ืžื™ื’ืจืึทื˜ื™ืึธืŸ.

TimescaleDB ืคืึทืจื‘ื™ื™ึทื˜

TimescaleDB ืื™ื– ืึท PostgreSQL ืคืึทืจืœืขื ื’ืขืจื•ื ื’ ื•ื•ืึธืก ืึธืคึผื˜ื™ืžื™ื–ืขืก ืืจื‘ืขื˜ืŸ ืžื™ื˜ ืฆื™ื™ื˜ ืกืขืจื™ืข ืฆื™ื™ื˜ ืกืขืจื™ืข ืื™ืŸ ืึท ืจืขื’ื•ืœืขืจ ื“ืึทื˜ืึทื‘ื™ื™ืก (https://docs.timescale.com/v1.0/introduction, https://habr.com/ru/company/zabbix/blog/458530/).

ื›ืึธื˜ืฉ ืงืœื™ืงื›ืึธื•ืกืข ืื™ื– ื ื™ืฉื˜ ืึท ืขืจื ืกื˜ ืงืึธื ืงื•ืจืขื ื˜ ืื™ืŸ ื“ื™ ื ื™ืฉืข ืคื•ืŸ โ€‹โ€‹ืฆื™ื™ื˜ ืกืขืจื™ืข, ืึธื‘ืขืจ ืงืึธืœื•ืžื ืขืจ ืกื˜ืจื•ืงื˜ื•ืจ ืื•ืŸ ื•ื•ืขืงื˜ืึธืจ ืึธื ืคึฟืจืขื’ ื“ื•ืจื›ืคื™ืจื•ื ื’, ืขืก ืื™ื– ืคื™ืœ ืคืึทืกื˜ืขืจ ื•ื•ื™ TimescaleDB ืื™ืŸ ืจื•ื‘ึฟ ืงืึทืกืขืก ืคื•ืŸ ืึทื ืึทืœื™ืกื™ืก ืึธื ืคึฟืจืขื’ ืคึผืจืึทืกืขืกื™ื ื’. ืื™ืŸ ื“ืขืจ ื–ืขืœื‘ื™ืงืขืจ ืฆื™ื™ื˜, ื“ื™ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคื•ืŸ ื‘ืืงื•ืžืขืŸ ืคึผืขืงืœ ื“ืึทื˜ืŸ ืคื•ืŸ ClickHouse ืื™ื– ื‘ืขืขืจืขืš 3 ืžืืœ ื”ืขื›ืขืจ, ืื•ืŸ ืขืก ืื•ื™ืš ื ื™ืฆื˜ 20 ืžืืœ ื•ื•ื™ื™ื ื™ืงืขืจ ื“ื™ืกืง ืคึผืœืึทืฅ, ื•ื•ืึธืก ืื™ื– ื˜ืึทืงืข ื•ื•ื™ื›ื˜ื™ืง ืคึฟืึทืจ ืคึผืจืึทืกืขืกื™ื ื’ ื’ืจื•ื™ืก ื•ื•ืึทืœื™ื•ืžื– ืคื•ืŸ ื”ื™ืกื˜ืืจื™ืฉืข ื“ืึทื˜ืŸ: โ€จhttps://www.altinity.com/blog/ClickHouse-for-time-series.

ื ื™ื˜ ืขื ืœืขืš ClickHouse, ื“ืขืจ ื‘ืœื•ื™ื– ื•ื•ืขื’ ืฆื• ืจืึทื˜ืขื•ื•ืขืŸ ืขื˜ืœืขื›ืข ื“ื™ืกืง ืคึผืœืึทืฅ ืื™ืŸ TimescaleDB ืื™ื– ืฆื• ื ื•ืฆืŸ ZFS ืึธื“ืขืจ ืขื ืœืขืš ื˜ืขืงืข ืกื™ืกื˜ืขืžืขืŸ.

ืึทืคึผืงืึทืžื™ื ื’ ื“ืขืจื”ื™ื™ึทื ื˜ื™ืงื•ื ื’ืขืŸ ืฆื• ClickHouse ื•ื•ืขื˜ ืžืกืชึผืžื ื‘ืึทืงืขื ืขืŸ ื“ืขืœื˜ืึท ืงืึทืžืคึผืจืขืฉืึทืŸ, ื•ื•ืึธืก ื•ื•ืขื˜ ืžืึทื›ืŸ ืขืก ืืคื™ืœื• ืžืขืจ ืคึผืึทืกื™ืง ืคึฟืึทืจ ืคึผืจืึทืกืขืกื™ื ื’ ืื•ืŸ ืกื˜ืึธืจื™ื ื’ ืฆื™ื™ื˜ ืกืขืจื™ืข ื“ืึทื˜ืŸ. TimescaleDB ืงืขืŸ ื–ื™ื™ืŸ ืึท ื‘ืขืกืขืจ ื‘ืจื™ืจื” ื•ื•ื™ ื ืึทืงืขื˜ ClickHouse ืื™ืŸ ื“ื™ ืคืืœื’ืขื ื“ืข ืงืึทืกืขืก:

  • ืงืœื™ื™ืŸ ื™ื ืกื˜ืึทืœื™ื™ืฉืึทื ื– ืžื™ื˜ ื–ื™ื™ืขืจ ืงืœื™ื™ืŸ ื‘ืึทืจืึทืŸ (<3 GB);
  • ืึท ื’ืจื•ื™ืก ื ื•ืžืขืจ ืคื•ืŸ ืงืœื™ื™ืŸ ื™ื ืกืขืจืฅ ื•ื•ืึธืก ืื™ืจ ื˜ืึธืŸ ื ื™ืฉื˜ ื•ื•ืขืœืŸ ืฆื• ื‘ืึทืคืขืจ ืื™ืŸ ื’ืจื•ื™ืก ืคืจืึทื’ืžืึทื ืฅ;
  • ื‘ืขืกืขืจ ืงืึธื ืกื™ืกื˜ืขื ืกื™, ื™ื•ื ืึทืคืึธืจืžืึทื˜ื™ ืื•ืŸ ืึทืกื™ื“ ื‘ืื“ืขืจืคืขื ื™ืฉืŸ;
  • PostGIS ืฉื˜ื™ืฆืŸ;
  • ืคืึทืจื‘ื™ื ื“ืŸ ืžื™ื˜ ื™ื’ื–ื™ืกื˜ื™ื ื’ PostgreSQL ื˜ื™ืฉืŸ, ื–ื™ื ื˜ Timescale DB ืื™ื– ื‘ื™ื™ืกื™ืงืœื™ PostgreSQL.

ืคืึทืจืžืขืกื˜ ืžื™ื˜ Hadoop ืื•ืŸ MapReduce ืกื™ืกื˜ืขืžืขืŸ

Hadoop ืื•ืŸ ืื ื“ืขืจืข ืžืึทืคึผืจืขื“ื•ืกืข ืคึผืจืึธื“ื•ืงื˜ืŸ ืงืขื ืขืŸ ื“ื•ืจื›ืคื™ืจืŸ ืึท ืคึผืœืึทืฅ ืคื•ืŸ ืงืึธืžืคึผืœืขืงืก ื—ืฉื‘ื•ื ื•ืช, ืึธื‘ืขืจ ื–ื™ื™ ื˜ืขื ื“ ืฆื• ืœื•ื™ืคืŸ ืžื™ื˜ ืจื™ื–ื™ืง ืœื™ื™ื˜ืึทื ืกื™ื–. ClickHouse ืคื™ืงืกื™ื– ื“ืขื ืคึผืจืึธื‘ืœืขื ื“ื•ืจืš ืคึผืจืึทืกืขืกื™ื ื’ ื˜ืขืจืึทื‘ื™ื˜ืขืก ืคื•ืŸ ื“ืึทื˜ืŸ ืื•ืŸ ืคึผืจืึทื“ื•ืกื™ื ื’ ืจืขื–ื•ืœื˜ืึทื˜ืŸ ื›ึผืžืขื˜ ื˜ื™ื™ืงืขืฃ. ืึทื–ื•ื™, ClickHouse ืื™ื– ืคื™ืœ ืžืขืจ ืขืคืขืงื˜ื™ื•ื• ืื™ืŸ ืคึผืขืจืคืึธืจืžื™ื ื’ ืฉื ืขืœ, ื™ื ื˜ืขืจืึทืงื˜ื™ื•ื• ืึทื ืึทืœื™ืกื™ืก ืคืึธืจืฉื•ื ื’, ื•ื•ืึธืก ื–ืึธืœ ื–ื™ื™ืŸ ืคื•ืŸ ืื™ื ื˜ืขืจืขืก ืฆื• ื“ืึทื˜ืŸ ืกื™ื™ืึทื ื˜ื™ืก.

ืคืึทืจืžืขืกื˜ ืžื™ื˜ ืคึผื™ื ืึธื˜ ืื•ืŸ ื“ืจื•ื™ื“

ืงืœื™ืงื›ืึธื•ืกืข ืก ืงืœืึธื•ืกืึทืกื˜ ืงืึธืžืคึผืขื˜ื™ื˜ืึธืจืก ื–ืขื ืขืŸ ืงืึธืœื•ืžื ืขืจ, ืœื™ื ืขืึทืจืœื™ ืกืงืึทืœืึทื‘ืœืข ืึธืคึฟืŸ ืžืงื•ืจ ืคึผืจืึธื“ื•ืงื˜ืŸ ืคึผื™ื ืึธื˜ ืื•ืŸ ื“ืจื•ื™ื“. ืึท ื•ื™ืกื’ืขืฆื™ื™ื›ื ื˜ ืึทืจื‘ืขื˜ ืงืึทืžืคึผืขืจื™ื ื’ ื“ื™ ืกื™ืกื˜ืขืžืขืŸ ืื™ื– ืืจื•ื™ืก ืื™ืŸ ื“ืขื ืึทืจื˜ื™ืงืœ ืจืึธืžืึทื ืึท ืœืขื•ื•ืขื ื˜ืึธื•ื•ืึท ื“ืึทื˜ืขื“ ืคืขื‘ืจื•ืืจ 1, 2018

ื ื™ืฆืŸ Clickhouse ื•ื•ื™ ืึท ืคืึทืจื‘ื™ื™ึทื˜ ืคึฟืึทืจ ELK, Big Query ืื•ืŸ TimescaleDB

ื“ืขืจ ืึทืจื˜ื™ืงืœ ื“ืึทืจืฃ ืึทืคึผื“ื™ื™ื˜ื™ื ื’ - ืขืก ื–ืื’ื˜ ืึทื– ClickHouse ืฉื˜ื™ืฆื˜ ื ื™ืฉื˜ ื“ืขืจื”ื™ื™ึทื ื˜ื™ืงืŸ ืื•ืŸ ื•ื™ืกืžืขืงืŸ ืึทืคึผืขืจื™ื™ืฉืึทื ื–, ื•ื•ืึธืก ืื™ื– ื ื™ืฉื˜ ืœืขื’ืึทืžืจืข ืืžืช ืคึฟืึทืจ ื“ื™ ืœืขืฆื˜ืข ื•ื•ืขืจืกื™ืขืก.

ืžื™ืจ ื”ืึธื‘ืŸ ื ื™ืฉื˜ ืึท ืคึผืœืึทืฅ ืคื•ืŸ ื“ืขืจืคืึทืจื•ื ื’ ืžื™ื˜ ื“ื™ ื“ืึทื˜ืึทื‘ื™ื™ืกื™ื–, ืึธื‘ืขืจ ืื™ืš ื˜ืึธืŸ ื ื™ื˜ ื˜ืึทืงืข ื•ื•ื™ ื“ื™ ืงืึทืžืคึผืœืขืงืกื™ื˜ื™ ืคื•ืŸ ื“ื™ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ื•ื•ืึธืก ืื™ื– ืคืืจืœืื ื’ื˜ ืฆื• ืœื•ื™ืคืŸ Druid ืื•ืŸ Pinot - ื“ืึธืก ืื™ื– ืึท ื’ืึทื ืฅ ื‘ื™ื ื˜ืœ ืคื•ืŸ ืžืึธื•ื•ื™ื ื’ ืคึผืึทืจืฅ ืกืขืจืึทื•ื ื“ืึทื“ ื“ื•ืจืš ื“ื–ืฉืึทื•ื•ืึท ืื•ื™ืฃ ืึทืœืข ื–ื™ื™ื˜ืŸ.

Druid ืื•ืŸ Pinot ื–ืขื ืขืŸ ืึทืคึผืึทื˜ืฉื™ ื™ื ื’ืงื™ืึทื‘ื™ื™ื˜ืขืจ ืคึผืจืึทื“ื–ืฉืขืงืก, ื“ื™ ืคึผืจืึธื’ืจืขืก ืคื•ืŸ ื•ื•ืึธืก ืื™ื– ื‘ืื“ืขืงื˜ ืื™ืŸ ื“ืขื˜ืึทืœ ื“ื•ืจืš Apache ืื•ื™ืฃ ื–ื™ื™ึทืŸ GitHub ืคึผืจื•ื™ืขืงื˜ ื‘ืœืขื˜ืขืจ. ืคึผื™ื ืึธื˜ ืืจื•ื™ืก ืื™ืŸ ื“ื™ ื™ื ื’ืงื™ืึทื‘ื™ื™ื˜ืขืจ ืื™ืŸ ืืงื˜ืื‘ืขืจ 2018, ืื•ืŸ ื“ืจื•ื™ื“ ืื™ื– ื’ืขื‘ื•ื™ืจืŸ 8 ื—ื“ืฉื™ื ืคืจื™ืขืจ - ืื™ืŸ ืคืขื‘ืจื•ืืจ.

ื“ืขืจ ืžืึทื ื’ืœ ืคื•ืŸ ืื™ื ืคึฟืึธืจืžืึทืฆื™ืข ื•ื•ืขื’ืŸ ื•ื•ื™ AFS ืึทืจื‘ืขื˜ ืจื™ื™ื–ืึทื– ืขื˜ืœืขื›ืข, ืื•ืŸ ื˜ืึธืžืขืจ ื ืึทืจื™ืฉ, ืคึฟืจืื’ืŸ ืคึฟืึทืจ ืžื™ืจ. ืื™ืš ื•ื•ื•ื ื“ืขืจ ืฆื™ ื“ื™ ืคึผื™ื ืึธื˜ ืžื—ื‘ืจื™ื ื”ืึธื‘ืŸ ื‘ืืžืขืจืงื˜ ืึทื– ื“ื™ ืึทืคึผืึทื˜ืฉื™ ื•ื•ื™ืงื™ืคึผืขื“ื™ืข ืื™ื– ืžืขืจ ื’ื™ื ืฆื™ืง ืฆื• ื“ืจื•ื™ื“, ืื•ืŸ ืฆื™ ื“ื™ ืฉื˜ืขืœื•ื ื’ ืฆื• ื“ื™ ืงืึธื ืงื•ืจืขื ื˜ ื’ืขืคึฟื™ืจื˜ ืึท ื’ืขืคื™ืœ ืคื•ืŸ ืžืขืงืึทื ืข? ื•ื•ืขื˜ ื“ื™ ืึทื ื˜ื•ื•ื™ืงืœื•ื ื’ ืคื•ืŸ Druid ืคึผืึทืžืขืœืขืš ืื•ืŸ ื“ื™ ืึทื ื˜ื•ื•ื™ืงืœื•ื ื’ ืคื•ืŸ Pinot ืคืึทืจื’ื™ื›ืขืจืŸ ืื•ื™ื‘ ื“ื™ ื‘ืึทืงืขืจื– ืคื•ืŸ ื“ื™ ืขืจืฉื˜ืข ืคึผืœื•ืฆืœื™ื ื’ ื•ื•ืขืจืŸ ืื™ื ื˜ืขืจืขืกื™ืจื˜ ืื™ืŸ ื“ื™ ื™ืขื ืขืจ?

ื“ื™ืกืึทื“ื•ื•ืึทื ื˜ื™ื“ื–ืฉื™ื– ืคื•ืŸ ืงืœื™ืงื›ืึธื•ืกืข

ื•ืžืฆื™ื™ึทื˜ื™ืง: ื“ืึธืš, ื“ืึธืก ืื™ื– ื ืึธืš ื ื™ืฉื˜ ืึท ื ื•ื“ื ืข ื˜ืขื›ื ืึธืœืึธื’ื™ืข, ืึธื‘ืขืจ ืื™ืŸ ืงื™ื™ืŸ ืคืึทืœ, ื’ืึธืจื ื™ืฉื˜ ื•ื•ื™ ื“ืึธืก ืื™ื– ื’ืขื–ืขืŸ ืื™ืŸ ืื ื“ืขืจืข ืงืึธืœื•ืžื ืขืจ ื“ื‘ืžืก.

ืงืœื™ื™ืŸ ื™ื ืกืขืจืฅ ื˜ืึธืŸ ื ื™ืฉื˜ ืึทืจื‘ืขื˜ืŸ ื’ืขื–ื•ื ื˜ ืื™ืŸ ื”ื•ื™ืš ื’ื™ื›ืงื™ื™ึทื˜: ื™ื ืกืขืจืฅ ืžื•ื–ืŸ ื–ื™ื™ืŸ ืฉืคึผืึทืœื˜ืŸ ืื™ืŸ ื’ืจืขืกืขืจืข ืฉื˜ื™ืงืขืจ ื•ื•ื™ื™ึทืœ ื“ื™ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคื•ืŸ ืงืœื™ื™ืŸ ื™ื ืกืขืจืฅ ื“ื™ื’ืจื™ื™ื“ ืื™ืŸ ืคึผืจืึธืคึผืึธืจืฆื™ืข ืฆื• ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ืฉืคืืœื˜ืŸ ืื™ืŸ ื™ืขื“ืขืจ ืจื•ื“ืขืจืŸ. ื“ืึธืก ืื™ื– ื•ื•ื™ ืงืœื™ืงื›ืึธื•ืกืข ืกื˜ืึธืจื– ื“ืึทื˜ืŸ ืื•ื™ืฃ ื“ื™ืกืง - ื™ืขื“ืขืจ ื–ื™ื™ึทืœ ืจืขืคึผืจืึทื–ืขื ืฅ 1 ื˜ืขืงืข ืึธื“ืขืจ ืžืขืจ, ืึทื–ื•ื™ ืฆื• ืึทืจื™ื™ึทื ืœื™ื™ื’ืŸ 1 ืจื•ื“ืขืจืŸ ืžื™ื˜ 100 ืฉืคืืœื˜ืŸ, ืื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ืขืคึฟืขื ืขืŸ ืื•ืŸ ืฉืจื™ื™ึทื‘ืŸ ืื™ืŸ ืžื™ื ื“ืกื˜ืขืจ 100 ื˜ืขืงืขืก. ื“ืึธืก ืื™ื– ื•ื•ืึธืก ื‘ืึทืคืขืจื™ื ื’ ื™ื ืกืขืจืฅ ืจื™ืงื•ื•ื™ื™ืขืจื– ืึท ืžื™ื“ืึทืœืžืึทืŸ (ืกื™ื™ึทื“ืŸ ื“ืขืจ ืงืœื™ืขื ื˜ ื–ื™ืš ื’ื™ื˜ ื‘ืึทืคืขืจื™ื ื’) - ื™ื•ื–ืฉืึทื•ื•ืึทืœื™ ืงืึทืคืงืึท ืึธื“ืขืจ ืึท ืžื™ืŸ ืคื•ืŸ ืจื™ื™ ืคืึทืจื•ื•ืึทืœื˜ื•ื ื’ ืกื™ืกื˜ืขื. ืื™ืจ ืงืขื ื˜ ืื•ื™ืš ื ื•ืฆืŸ ื“ื™ ื‘ืึทืคืขืจ ื˜ื™ืฉ ืžืึธื˜ืึธืจ ืฆื• ื ืึธื›ืžืึทื›ืŸ ื’ืจื•ื™ืก ื˜ืฉืึทื ื’ืงืก ืคื•ืŸ ื“ืึทื˜ืŸ ืื™ืŸ MergeTree ื˜ื™ืฉืŸ.

ื˜ื™ืฉ ื“ื–ืฉืึธื™ื ืก ื–ืขื ืขืŸ ืœื™ืžื™ื˜ืขื“ ื“ื•ืจืš ื“ื™ ื‘ืึทืจืึทืŸ ืคื•ืŸ ื“ื™ ืกืขืจื•ื•ืขืจ, ืึธื‘ืขืจ ืื™ืŸ ืžื™ื ื“ืกื˜ืขืจ ื–ื™ื™ ื–ืขื ืขืŸ ื“ืึธืจื˜! ืœืžืฉืœ, Druid ืื•ืŸ Pinot ื˜ืึธืŸ ื ื™ื˜ ื”ืึธื‘ืŸ ืึทื–ืึท ืงืึทื ืขืงืฉืึทื ื– ืื™ืŸ ืึทืœืข, ื•ื•ื™ื™ึทืœ ื–ื™ื™ ื–ืขื ืขืŸ ืฉื•ื•ืขืจ ืฆื• ื™ื ืกื˜ืจื•ืžืขื ื˜ ื’ืœื™ื™ึทืš ืื™ืŸ ืคื•ื ืื ื“ืขืจื’ืขื˜ื™ื™ืœื˜ ืกื™ืกื˜ืขืžืขืŸ ื•ื•ืึธืก ืฉื˜ื™ืฆืŸ ื ื™ืฉื˜ ืžืึธื•ื•ื™ื ื’ ื’ืจื•ื™ืก ืฉื˜ื™ืงืขืจ ืคื•ืŸ ื“ืึทื˜ืŸ ืฆื•ื•ื™ืฉืŸ ื ืึธื•ื“ื–.

ืคื™ื™ื ื“ื™ื ื’ื–

ืžื™ืจ ืคึผืœืึทื ื™ืจืŸ ืฆื• ื•ื•ื™ื™ื“ืœื™ ื ื•ืฆืŸ ClickHouse ืื™ืŸ Qwintry ืื™ืŸ ื“ื™ ืงื•ืžืขื ื“ื™ืง ื™ืึธืจืŸ, ื•ื•ื™ื™ึทืœ ื“ื™ DBMS ื’ื™ื˜ ืึท ื•ื™ืกื’ืขืฆื™ื™ื›ื ื˜ ื•ื•ืึธื’ ืคื•ืŸ ืคืึธืจืฉื˜ืขืœื•ื ื’, ื ื™ื“ืขืจื™ืง ืึธื•ื•ื•ืขืจื›ืขื“, ืกืงืึทืœืึทื‘ื™ืœื™ื˜ื™ ืื•ืŸ ืคึผืึทืฉื˜ืขืก. ืื™ืš ื‘ื™ืŸ ืฉื™ื™ืŸ ื–ื™ื›ืขืจ ืึทื– ืขืก ื•ื•ืขื˜ ืึธื ื”ื™ื™ื‘ืŸ ืฆื• ืคืึทืจืฉืคึผืจื™ื™ื˜ืŸ ื’ืขืฉื•ื•ื™ื ื“ ืึทืžืึธืœ ื“ื™ ClickHouse ืงื”ืœ ืงื•ืžื˜ ืžื™ื˜ ืžืขืจ ื•ื•ืขื’ืŸ ืฆื• ื ื•ืฆืŸ ืขืก ืื™ืŸ ืงืœื™ื™ืŸ ืฆื• ืžื™ื˜ืœ-ืกื™ื™ื–ื“ ื™ื ืกื˜ืึทืœื™ื™ืฉืึทื ื–.

ืขื˜ืœืขื›ืข ืึทื“ืก ๐Ÿ™‚

ื“ืื ืง ืื™ืจ ืคึฟืึทืจ ืกื˜ื™ื™ื™ื ื’ ืžื™ื˜ ืื•ื ื“ื–. ืฆื™ ืื™ืจ ื•ื•ื™ ืื•ื ื“ื–ืขืจ ืึทืจื˜ื™ืงืœืขืŸ? ื•ื•ื™ืœืŸ ืฆื• ื–ืขืŸ ืžืขืจ ื˜ืฉื™ืงืึทื•ื•ืข ืื™ื ื”ืึทืœื˜? ืฉื˜ื™ืฆืŸ ืื•ื ื“ื– ื“ื•ืจืš ืคึผืœื™ื™ืกื™ื ื’ ืึท ืกื“ืจ ืึธื“ืขืจ ืจืขืงืึทืžืขื ื“ื™ื“ ืฆื• ืคืจืขื ื“ื–, ื•ื•ืึธืœืงืŸ ื•ื•ืคึผืก ืคึฟืึทืจ ื“ืขื•ื•ืขืœืึธืคึผืขืจืก ืคึฟื•ืŸ $ 4.99, ืึท ื™ื™ื ืฆื™ืง ืึทื ืึทืœืึธื’ ืคื•ืŸ ืคึผืึธื–ื™ืฆื™ืข ืกืขืจื•ื•ืขืจืก ื•ื•ืึธืก ืื™ื– ื™ื ื•ื•ืขื ื˜ืึทื“ ื“ื•ืจืš ืื•ื ื“ื– ืคึฟืึทืจ ืื™ืจ: ื“ืขืจ ื’ืื ืฆืขืจ ืืžืช ื•ื•ืขื’ืŸ VPS (KVM) E5-2697 v3 (6 ืงืึธืจืขืก) 10GB DDR4 480GB SSD 1Gbps ืคึฟื•ืŸ $19 ืึธื“ืขืจ ื•ื•ื™ ืฆื• ื˜ื™ื™ืœืŸ ืึท ืกืขืจื•ื•ืขืจ? (ื‘ื ื™ืžืฆื ืžื™ื˜ RAID1 ืื•ืŸ RAID10, ืึทืจื•ื™ืฃ ืฆื• 24 ืงืึธืจืขืก ืื•ืŸ ืึทืจื•ื™ืฃ ืฆื• 40GB DDR4).

Dell R730xd 2 ืžืืœ ื˜ืฉื™ืคึผืขืจ ืื™ืŸ Equinix Tier IV ื“ืึทื˜ืŸ ืฆืขื ื˜ืขืจ ืื™ืŸ ืืžืกื˜ืขืจื“ืื? ื ืึธืจ ื“ืึธ 2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100 TV ืคึฟื•ืŸ $199 ืื™ืŸ ื“ื™ ื ืขื˜ื”ืขืจืœืึทื ื“ืก! Dell R420 - 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB - ืคึฟื•ืŸ $99! ืœื™ื™ืขื ืขืŸ ื•ื•ืขื’ืŸ ื•ื•ื™ ืฆื• ื‘ื•ื™ืขืŸ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ืงืึธืจืคึผ. ืงืœืึทืก ืžื™ื˜ ื“ื™ ื ื•ืฆืŸ ืคื•ืŸ Dell R730xd E5-2650 v4 ืกืขืจื•ื•ืขืจืก ื•ื•ืขืจื˜ 9000 ืขื•ืจืึธืก ืคึฟืึทืจ ืึท ืคึผืขื ื™?

ืžืงื•ืจ: www.habr.com

ืœื™ื™ื’ืŸ ืึท ื‘ืึทืžืขืจืงื•ื ื’