ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื ืื˜ื™ืฅ. ื˜ืจืึทื ืกืœ.: ืื™ืŸ ื“ืขื ืึทืจื˜ื™ืงืœ, Banzai Cloud ืฉืึทืจืขืก ืึท ื‘ื™ื™ืฉืคึผื™ืœ ืคื•ืŸ ื•ื•ื™ ื–ื™ื™ืŸ ืžื ื”ื’ ืžื›ืฉื™ืจื™ื ืงืขื ืขืŸ ื–ื™ื™ืŸ ื’ืขื•ื•ื™ื™ื ื˜ ืฆื• ืžืึทื›ืŸ Kafka ื’ืจื™ื ื’ืขืจ ืฆื• ื ื•ืฆืŸ ืื™ืŸ Kubernetes. ื“ื™ ืคืืœื’ืขื ื“ืข ื™ื ืกื˜ืจืึทืงืฉืึทื ื– ืื™ืœื•ืกื˜ืจื™ืจืŸ ื•ื•ื™ ืื™ืจ ืงืขื ืขืŸ ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืึธืคึผื˜ื™ืžืึทืœ ื’ืจื™ื™ืก ืคื•ืŸ ื“ื™ื™ืŸ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ืื•ืŸ ืงืึทื ืคื™ื’ื™ืขืจ Kafka ื–ื™ืš ืฆื• ื“ืขืจื’ืจื™ื™ื›ืŸ ื“ื™ ืคืืจืœืื ื’ื˜ ื˜ืจื•ืคึผื•ื˜.

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

Apache Kafka ืื™ื– ืึท ืคื•ื ืื ื“ืขืจื’ืขื˜ื™ื™ืœื˜ ืกื˜ืจื™ืžื™ื ื’ ืคึผืœืึทื˜ืคืึธืจืžืข ืคึฟืึทืจ ืงืจื™ื™ื™ื˜ื™ื ื’ ืคืึทืจืœืึธื–ืœืขืš, ืกืงืึทืœืึทื‘ืœืข ืื•ืŸ ื”ื•ื™ืš-ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคืึทืงื˜ื™ืฉ-ืฆื™ื™ื˜ ืกื˜ืจื™ืžื™ื ื’ ืกื™ืกื˜ืขืžืขืŸ. ื–ื™ื™ืŸ ื™ืžืคึผืจืขืกื™ื•ื• ืงื™ื™ืคึผืึทื‘ื™ืœืึทื˜ื™ื– ืงืขื ืขืŸ ื–ื™ื™ืŸ ืขืงืกื˜ืขื ื“ืขื“ ืžื™ื˜ Kubernetes. ืคึฟืึทืจ ื“ืขื ืžื™ืจ ื”ืึธื‘ืŸ ื“ืขื•ื•ืขืœืึธืคึผืขื“ ืขืคึฟืŸ ืžืงื•ืจ ืงืึทืคืงืึท ืึธืคึผืขืจืึทื˜ืึธืจ ืื•ืŸ ืึท ื›ืœื™ ื’ืขืจื•ืคืŸ ืกื•ืคึผืขืจื˜ื•ื‘ืขืก. ื–ื™ื™ ืœืึธื–ืŸ ืื™ืจ ืฆื• ืœื•ื™ืคืŸ Kafka ืื•ื™ืฃ Kubernetes ืื•ืŸ ื ื•ืฆืŸ ื–ื™ื™ึทืŸ ืคืึทืจืฉื™ื“ืŸ ืคึฟืขื™ึดืงื™ื™ื˜ืŸ, ืึทื–ืึท ื•ื•ื™ ืคื™ื™ืŸ-ื˜ื•ื ื™ื ื’ ื“ื™ ืžืขืงืœืขืจ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ, ืžืขื˜ืจื™ืง-ื‘ืื–ื™ืจื˜ ืกืงื™ื™ืœื™ื ื’ ืžื™ื˜ ืจื™ื‘ืึทืœืึทื ืกื™ื ื’, ื’ืขืฉื˜ืขืœ ื•ื•ื™ืกื™ืงื™ื™ึทื˜, "ื•ื•ื™ื™ืš" (ื’ืจืึทืฆื™ืขื–) ืจืึธื•ืœื™ื ื’ ื“ืขืจื”ื™ื™ึทื ื˜ื™ืงื•ื ื’ืขืŸ, ืขื˜ืง.

ืคึผืจื•ึผื•ื•ื˜ ืกื•ืคึผืขืจื˜ื•ื‘ืขืก ืื™ืŸ ื“ื™ื™ืŸ ืงื ื•ื™ืœ:

curl https://getsupertubes.sh | sh ะธ supertubes install -a --no-democluster --kubeconfig <path-to-eks-cluster-kubeconfig-file>

ืึธื“ืขืจ ืงืึธื ื˜ืึทืงื˜ ื“ืึทืงื™ื•ืžืขื ื˜ื™ื™ืฉืึทืŸ. ืื™ืจ ืงืขื ื˜ ืื•ื™ืš ืœื™ื™ืขื ืขืŸ ื•ื•ืขื’ืŸ ืขื˜ืœืขื›ืข ืคื•ืŸ โ€‹โ€‹ื“ื™ ืงื™ื™ืคึผืึทื‘ื™ืœืึทื˜ื™ื– ืคื•ืŸ Kafka, ื“ื™ ืึทืจื‘ืขื˜ ืžื™ื˜ ื•ื•ืึธืก ืื™ื– ืึธื˜ืึทืžื™ื™ื˜ื™ื“ ื ื™ืฆืŸ ืกื•ืคึผืขืจื˜ื•ื‘ืขืก ืื•ืŸ Kafka ืึธืคึผืขืจืึทื˜ืึธืจ. ืžื™ืจ ื”ืึธื‘ืŸ ืฉื•ื™ืŸ ื’ืขืฉืจื™ื‘ืŸ ื•ื•ืขื’ืŸ ื–ื™ื™ ืื•ื™ืฃ ื“ืขื ื‘ืœืึธื’:

ื•ื•ืขืŸ ืื™ืจ ื‘ืึทืฉืœื™ืกืŸ ืฆื• ืฆืขื•ื•ื™ืงืœืขืŸ ืึท Kafka ืงื ื•ื™ืœ ืื•ื™ืฃ Kubernetes, ืื™ืจ ื•ื•ืขื˜ ืžืกืชึผืžื ื–ื™ื™ืŸ ืคื™ื™ืกื˜ ืžื™ื˜ ื“ื™ ืึทืจื•ื™ืกืจื•ืคืŸ ืฆื• ื‘ืึทืฉืœื™ืกืŸ ื“ื™ ืึธืคึผื˜ื™ืžืึทืœ ื’ืจื™ื™ืก ืคื•ืŸ ื“ื™ ืึทื ื“ืขืจืœื™ื™ื™ื ื’ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ืื•ืŸ ื“ื™ ื ื•ื™ื˜ ืฆื• ืคื™ื™ืŸ-ื˜ื•ื ืข ื“ื™ื™ืŸ Kafka ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ืฆื• ื˜ืจืขืคืŸ ื“ื™ ื˜ืจื•ืคึผื•ื˜ ืจืขืงื•ื•ื™ืจืขืžืขื ืฅ. ื“ื™ ืžืึทืงืกื™ืžื•ื ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคื•ืŸ ื™ืขื“ืขืจ ืžืขืงืœืขืจ ืื™ื– ื‘ืืฉืœืืกืŸ ื“ื•ืจืš ื“ื™ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคื•ืŸ ื“ื™ ืึทื ื“ืขืจืœื™ื™ื™ื ื’ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ืงืึทืžืคึผืึธื•ื ืึทื ืฅ, ืึทื–ืึท ื•ื•ื™ ื–ื›ึผืจื•ืŸ, ืคึผืจืึทืกืขืกืขืจ, ื“ื™ืกืง ื’ื™ื›ืงื™ื™ึทื˜, ื ืขืฅ ื‘ืึทื ื“ื•ื•ื™ื“ื˜, ืขื˜ืง.

ื™ื“ืขืึทืœืœื™, ื“ื™ ืžืขืงืœืขืจ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ื–ืึธืœ ื–ื™ื™ืŸ ืึทื–ื•ื™ ืึทื– ืึทืœืข ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ืขืœืขืžืขื ื˜ืŸ ื–ืขื ืขืŸ ื’ืขื ื™ืฆื˜ ืฆื• ื–ื™ื™ืขืจ ืžืึทืงืกื™ืžื•ื ืงื™ื™ืคึผืึทื‘ื™ืœืึทื˜ื™ื–. ืึธื‘ืขืจ, ืื™ืŸ ืคืึทืงื˜ื™ืฉ ืœืขื‘ืŸ ื“ืขื ืกืขื˜ืึทืคึผ ืื™ื– ื’ืึทื ืฅ ืงืึธืžืคึผืœื™ืฆื™ืจื˜. ืขืก ืื™ื– ืžืขืจ ืžืกืชึผืžื ืึทื– ื ื™ืฆืขืจืก ื•ื•ืขืœืŸ ืงืึทื ืคื™ื’ื™ืขืจ ื‘ืจืึธืงืขืจืก ืฆื• ืžืึทืงืกืึทืžื™ื™ื– ื“ื™ ื ื•ืฆืŸ ืคื•ืŸ ืื™ื™ืŸ ืึธื“ืขืจ ืฆื•ื•ื™ื™ ืงืึทืžืคึผืึธื•ื ืึทื ืฅ (ื“ื™ืกืง, ื–ื›ึผืจื•ืŸ ืึธื“ืขืจ ืคึผืจืึทืกืขืกืขืจ). ื‘ื›ืœืœ ื’ืขืจืขื“ื˜, ืึท ืžืขืงืœืขืจ ื•ื•ื™ื™ื–ื˜ ืžืึทืงืกื™ืžื•ื ืคืึธืจืฉื˜ืขืœื•ื ื’ ื•ื•ืขืŸ ื–ื™ื™ืŸ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ืึทืœืึทื•ื– ื“ื™ ืกืœืึธื•ืึทืกื˜ ืงืึธืžืคึผืึธื ืขื ื˜ ืฆื• ื–ื™ื™ืŸ ื’ืขื•ื•ื™ื™ื ื˜ ืฆื• ื–ื™ื™ืŸ ืคื•ืœืึทืกื˜ ืžืึธืก. ื“ืขื ื•ื•ืขื’ ืžื™ืจ ืงืขื ืขืŸ ื‘ืึทืงื•ืžืขืŸ ืึท ืคึผืจืึธืกื˜ ื’ืขื“ืึทื ืง ืคื•ืŸ ื“ื™ ืžืึทืกืข ืึทื– ืื™ื™ืŸ ืžืขืงืœืขืจ ืงืขื ืขืŸ ืฉืขืคึผืŸ.

ื˜ื”ืขืึธืจืขื˜ื™ืงืึทืœืœื™, ืžื™ืจ ืงืขื ืขืŸ ืื•ื™ืš ืึธืคึผืฉืึทืฆืŸ ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ื‘ืจืึธืงืขืจืก ืคืืจืœืื ื’ื˜ ืฆื• ืฉืขืคึผืŸ ืึท ื’ืขื’ืขื‘ืŸ ืžืึทืกืข. ืึธื‘ืขืจ, ืื™ืŸ ืคื™ืจ ืขืก ื–ืขื ืขืŸ ืึทื–ื•ื™ ืคื™ืœืข ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ืึธืคึผืฆื™ืขืก ืื™ืŸ ืคืึทืจืฉื™ื“ืขื ืข ืœืขื•ื•ืขืœืก ืึทื– ืขืก ืื™ื– ื–ื™ื™ืขืจ ืฉื•ื•ืขืจ (ืื•ื™ื‘ ื ื™ื˜ ืื•ืžืžืขื’ืœืขืš) ืฆื• ืึธืคึผืฉืึทืฆืŸ ื“ื™ ืคึผืึธื˜ืขื ืฆื™ืขืœ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคื•ืŸ ืึท ื‘ืึทื–ื•ื ื“ืขืจ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ. ืื™ืŸ ืื ื“ืขืจืข ื•ื•ืขืจื˜ืขืจ, ืขืก ืื™ื– ื–ื™ื™ืขืจ ืฉื•ื•ืขืจ ืฆื• ืคึผืœืึทืŸ ืึท ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ื‘ืื–ื™ืจื˜ ืื•ื™ืฃ ืขื˜ืœืขื›ืข ื’ืขื’ืขื‘ืŸ ืคืึธืจืฉื˜ืขืœื•ื ื’.

ืคึฟืึทืจ Supertubes ื ื™ืฆืขืจืก, ืžื™ืจ ื™ื•ื–ืฉืึทื•ื•ืึทืœื™ ื ืขืžืขืŸ ื“ื™ ืคืืœื’ืขื ื“ืข ืฆื•ื’ืึทื ื’: ืžื™ืจ ืึธื ื”ื™ื™ื‘ืŸ ืžื™ื˜ ืขื˜ืœืขื›ืข ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ (ื™ื ืคืจืึทืกื˜ื˜ืจืึทืงื˜ืฉืขืจ + ืกืขื˜ื˜ื™ื ื’ืก), ืื•ืŸ ืžืขืกื˜ืŸ ื–ื™ื™ึทืŸ ืคืึธืจืฉื˜ืขืœื•ื ื’, ืกื˜ืจื•ื™ืขืจืŸ ื“ื™ ืžืขืงืœืขืจ ืกืขื˜ื˜ื™ื ื’ืก ืื•ืŸ ืื™ื‘ืขืจื—ื–ืจืŸ ื“ืขื ืคึผืจืึธืฆืขืก ื•ื•ื™ื“ืขืจ. ื“ืึธืก ื›ืึทืคึผืึทื ื– ื‘ื™ื– ื“ื™ ืกืœืึธื•ืึทืกื˜ ืงืึธืžืคึผืึธื ืขื ื˜ ืคื•ืŸ ื“ื™ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ืื™ื– ื’ืึธืจ ื™ื•ื˜ืึทืœื™ื™ื–ื“.

ืื™ืŸ ื“ืขื ื•ื•ืขื’, ืžื™ืจ ื‘ืึทืงื•ืžืขืŸ ืึท ืงืœืึธืจ ื’ืขื“ืึทื ืง ืคื•ืŸ ื•ื•ื™ ืคื™ืœืข ื‘ืจืึธืงืขืจืก ืึท ืงื ื•ื™ืœ ื“ืึทืจืฃ ืฆื• ืฉืขืคึผืŸ ืึท ื–ื™ื›ืขืจ ืžืึทืกืข (ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ื‘ืจืึธืงืขืจืก ืื•ื™ืš ื“ืขืคึผืขื ื“ืก ืื•ื™ืฃ ืื ื“ืขืจืข ืกื™ื‘ื•ืช, ืึทื–ืึท ื•ื•ื™ ื“ื™ ืžื™ื ื™ืžื•ื ื ื•ืžืขืจ ืคื•ืŸ ืึธื ื–ืึธื’ ืจืขืคึผืœืึทืงืึทื– ืฆื• ืขื ืฉื•ืจ ืจื™ื–ื™ืœื™ืึทื ืก, ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ืฆืขื˜ื™ื™ืœื•ื ื’. ืžื ื”ื™ื’ื™ื ื•ื›ื•'). ืื™ืŸ ืึทื“ื™ืฉืึทืŸ, ืžื™ืจ ื‘ืึทืงื•ืžืขืŸ ื™ื ืกื™ื™ื˜ ืื™ืŸ ื•ื•ืึธืก ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ืงืึทืžืคึผืึธื•ื ืึทื ืฅ ื“ืึทืจืคืŸ ื•ื•ืขืจื˜ื™ืงืึทืœ ืกืงื™ื™ืœื™ื ื’.

ื“ืขืจ ืึทืจื˜ื™ืงืœ ื•ื•ืขื˜ ืจืขื“ืŸ ื•ื•ืขื’ืŸ ื“ื™ ืกื˜ืขืคึผืก ืžื™ืจ ื ืขืžืขืŸ ืฆื• ื‘ืึทืงื•ืžืขืŸ ื“ื™ ืžืขืจืกื˜ ืื•ื™ืก ืคื•ืŸ ื“ื™ ืกืœืึธื•ืึทืกื˜ ืงืึทืžืคึผืึธื•ื ืึทื ืฅ ืื™ืŸ ืขืจืฉื˜ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทื ื– ืื•ืŸ ืžืขืกื˜ืŸ ื“ื™ ื˜ืจื•ืคึผื•ื˜ ืคื•ืŸ ืึท Kafka ืงื ื•ื™ืœ. ื ื”ืขื›ืกื˜ ืจื™ื–ื™ืœื™ืึทื ื˜ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ืจื™ืงื•ื•ื™ื™ืขืจื– ื‘ื™ื™ึท ืžื™ื ื“ืกื˜ืขืจ ื“ืจื™ื™ ืคืœื™ืกื ื“ื™ืง ื‘ืจืึธืงืขืจืก (min.insync.replicas=3), ืคื•ื ืื ื“ืขืจื’ืขื˜ื™ื™ืœื˜ ืื™ื‘ืขืจ ื“ืจื™ื™ ืคืึทืจืฉื™ื“ืขื ืข ืึทืงืกืขืกืึทื‘ื™ืœื™ื˜ื™ ื–ืึธื ืขืก. ืฆื• ืงืึทื ืคื™ื’ื™ืขืจ, ื•ื•ืึธื’ ืื•ืŸ ืžืึธื ื™ื˜ืึธืจ ื“ื™ Kubernetes ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ, ืžื™ืจ ื ื•ืฆืŸ ืื•ื ื“ื–ืขืจ ืื™ื™ื’ืขื ืข ืงืึทื ื˜ื™ื™ื ืขืจ ืคืึทืจื•ื•ืึทืœื˜ื•ื ื’ ืคึผืœืึทื˜ืคืึธืจืžืข ืคึฟืึทืจ ื›ื™ื™ื‘ืจื™ื“ ื•ื•ืืœืงื ืก - ืจืขืจื  - ืœื™ื ื™ืข. ืขืก ืฉื˜ื™ืฆื˜ ืื•ื™ืฃ-ื”ืึทื ืึธื›ืข (ื ืึทืงืขื˜ ืžืขื˜ืึทืœ, VMware) ืื•ืŸ ืคื™ื ืฃ ื˜ื™ื™ืคึผืก ืคื•ืŸ ื•ื•ืืœืงื ืก (Alibaba, AWS, Azure, Google, Oracle), ื•ื•ื™ ื’ืขื–ื•ื ื˜ ื•ื•ื™ ืงื™ื™ืŸ ืงืึธืžื‘ื™ื ืึทืฆื™ืข ืคื•ืŸ โ€‹โ€‹โ€‹โ€‹ื–ื™ื™.

ื’ืขื“ืื ืงืขืŸ ื•ื•ืขื’ืŸ ืงืึทืคืงืึท ืงื ื•ื™ืœ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ืื•ืŸ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ

ืคึฟืึทืจ ื“ื™ ื‘ื™ื™ืฉืคื™ืœืŸ ืื•ื ื˜ืŸ, ืžื™ืจ ืื•ื™ืกื“ืขืจื•ื•ื™ื™ืœื˜ AWS ื•ื•ื™ ื“ื™ ื•ื•ืึธืœืงืŸ ืฉืคึผื™ื™ึทื–ืขืจ ืื•ืŸ EKS ื•ื•ื™ ื“ื™ Kubernetes ืคืึทืจืฉืคึผืจื™ื™ื˜ื•ื ื’. ื ืขื ืœืขืš ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ืงืขื ืขืŸ ื–ื™ื™ืŸ ื™ืžืคึผืœืึทืžืขื ืึทื“ ื ื™ืฆืŸ P.K.E. - Kubernetes ืคืึทืจืฉืคึผืจื™ื™ื˜ื•ื ื’ ืคึฟื•ืŸ Banzai Cloud, ืกืขืจื˜ืึทืคื™ื™ื“ ื“ื•ืจืš CNCF.

ื“ื™ืกืง

ืึทืžืึทื–ืึธืŸ ืึธืคืคืขืจืก ืคืึทืจืฉื™ื“ืŸ EBS ื‘ืึทื ื“ ื˜ื™ื™ืคึผืก. ืื™ืŸ ื“ื™ ื”ืึทืจืฅ ื’ืคึผืงืกื ื•ืžืงืก ะธ ื™ืึธืงืกื ื•ืžืงืก ืขืก ื–ืขื ืขืŸ SSD ื“ืจื™ื™ื•ื•ื–, ืึธื‘ืขืจ, ืฆื• ืขื ืฉื•ืจ ื”ื•ื™ืš ื˜ืจื•ืคึผื•ื˜ ื’ืคึผืงืกื ื•ืžืงืก ืงืึทื ืกื•ืžื– ืึทืงื™ื•ืžื™ืึทืœื™ื™ื˜ื™ื“ ืงืจืขื“ื™ืฅ (ืื™ืš / ืึธ ืงืจืขื“ื™ืฅ), ืึทื–ื•ื™ ืžื™ืจ ื‘ื™ืœื›ืขืจ ื“ืขื ื˜ื™ืคึผ ื™ืึธืงืกื ื•ืžืงืก, ื•ื•ืึธืก ืึธืคืคืขืจืก ืงืึธื ืกื™ืกื˜ืขื ื˜ ื”ื•ื™ืš ื˜ืจื•ืคึผื•ื˜.

ื˜ื™ื™ืคึผืก ืคื•ืŸ ื‘ื™ื™ืฉืคื™ืœืŸ

Kafka ืก ืคืึธืจืฉื˜ืขืœื•ื ื’ ืื™ื– ื”ืขื›ืกื˜ ืึธืคืขื ื’ื™ืง ืื•ื™ืฃ ื“ื™ ืึทืคึผืขืจื™ื™ื˜ื™ื ื’ ืกื™ืกื˜ืขื ืก ื‘ืœืึทื˜ ืงืึทืฉ, ืึทื–ื•ื™ ืžื™ืจ ื“ืึทืจืคึฟืŸ ื™ื ืกื˜ืึทื ืกื™ื– ืžื™ื˜ ื’ืขื ื•ื’ ื–ื›ึผืจื•ืŸ ืคึฟืึทืจ ื“ื™ ื‘ืจืึธืงืขืจืก (JVM) ืื•ืŸ ื‘ืœืึทื˜ ืงืึทืฉ. ื ืฉื˜ื™ื™ื’ืขืจ c5.2xlarge - ืึท ื’ื•ื˜ ืึธื ื”ื™ื™ื‘, ื–ื™ื ื˜ ืขืก ื”ืื˜ 16 ื’ื™ื’ืื‘ื™ื™ื˜ ืคื•ืŸ ื–ื›ึผืจื•ืŸ ืื•ืŸ ืึธืคึผื˜ื™ืžื™ื–ืขื“ ืฆื• ืึทืจื‘ืขื˜ืŸ ืžื™ื˜ EBS. ื–ื™ื™ืŸ ื›ื™ืกืึธืจืŸ ืื™ื– ืึทื– ืขืก ืื™ื– ื‘ืœื•ื™ื– ื˜ื•ื™ื’ืขื•ื•ื“ื™ืง ืฆื• ืฆื•ืฉื˜ืขืœืŸ ืžืึทืงืกื™ืžื•ื ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคึฟืึทืจ ื ื™ื˜ ืžืขืจ ื•ื•ื™ 30 ืžื™ื ื•ื˜ ื™ืขื“ืขืจ 24 ืฉืขื”. ืื•ื™ื‘ ื“ื™ื™ืŸ ื•ื•ืขืจืงืœืึธื•ื“ ืจื™ืงื•ื•ื™ื™ืขืจื– ืฉืคึผื™ืฅ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืื™ื‘ืขืจ ืึท ืœืขื ื’ืขืจืข ืฆื™ื™ื˜, ืื™ืจ ืงืขืŸ ื•ื•ืขืœืŸ ืฆื• ื‘ืึทื˜ืจืึทื›ื˜ืŸ ืื ื“ืขืจืข ื˜ื™ื™ืคึผืก. ืึทื– ืก ืคึผื•ื ืงื˜ ื•ื•ืึธืก ืžื™ืจ ื”ืื‘ืŸ, ืกื˜ืึธืคึผืคึผื™ื ื’ ื‘ื™ื™ึท c5.4xlarge. ืขืก ื’ื™ื˜ ื“ื™ ืžืึทืงืกื™ืžื•ื ื’ื™ื›ืงื™ื™ึทื˜ 593,75 ืžื‘/ืก. ืžืึทืงืกื™ืžื•ื ื˜ืจื•ืคึผื•ื˜ ืคื•ืŸ ืึทืŸ EBS ื‘ืึทื ื“ ื™ืึธืงืกื ื•ืžืงืก ื”ืขื›ืขืจ ื•ื•ื™ ื“ืขืจ ื‘ื™ื™ึทืฉืคึผื™ืœ c5.4xlarge, ืึทื–ื•ื™ ื“ื™ ืกืœืึธื•ืึทืกื˜ ืขืœืขืžืขื ื˜ ืคื•ืŸ ื“ื™ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ืื™ื– ืžืกืชึผืžื ื“ื™ ื™ / ืึธ ื˜ืจื•ืคึผื•ื˜ ืคื•ืŸ ื“ืขื ื˜ื™ืคึผ ื˜ื™ืคึผ (ื•ื•ืึธืก ืื•ื ื“ื–ืขืจ ืžืึทืกืข ื˜ืขืกืฅ ื–ืึธืœ ืื•ื™ืš ื‘ืึทืฉื˜ืขื˜ื™ืงืŸ).

ะกะตั‚ัŒ

ื“ื™ ื ืขืฅ ื˜ืจื•ืคึผื•ื˜ ืžื•ื–ืŸ ื–ื™ื™ืŸ ื’ืจื•ื™ืก ื’ืขื ื•ื’ ืงืึทืžืคึผืขืจื“ ืžื™ื˜ ื“ื™ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคื•ืŸ ื“ื™ VM ื‘ื™ื™ึทืฉืคึผื™ืœ ืื•ืŸ ื“ื™ืกืง, ืึทื ื“ืขืจืฉ ื“ื™ ื ืขืฅ ื•ื•ืขืจื˜ ืึท ื‘ืึทื˜ืึทืœื ืขืง. ืื™ืŸ ืื•ื ื“ื–ืขืจ ืคืึทืœ, ื“ื™ ื ืขืฅ ืฆื•ื‘ื™ื ื“ c5.4xlarge ืฉื˜ื™ืฆื˜ ืกืคึผื™ื“ื– ืคื•ืŸ ืึทืจื•ื™ืฃ ืฆื• 10 ื’ื™ื’ืื‘ื™ื™ื˜ / s, ื•ื•ืึธืก ืื™ื– ื‘ืื˜ื™ื™ื˜ื™ืง ื”ืขื›ืขืจ ื•ื•ื™ ื“ื™ ื™ / ืึธ ื˜ืจื•ืคึผื•ื˜ ืคื•ืŸ ืึท VM ื‘ื™ื™ึทืฉืคึผื™ืœ.

ืžืขืงืœืขืจ ื“ื™ืคึผืœื•ื™ืžืึทื ื˜

ื‘ืจืึธืงืขืจืก ื–ืึธืœ ื–ื™ื™ืŸ ื“ื™ืคึผืœื•ื™ื“ (ืกืงืขื“ื–ืฉื•ืœื“ ืื™ืŸ Kubernetes) ืฆื• ื“ืขื“ืึทืงื™ื™ื˜ืึทื“ ื ืึธื•ื“ื– ืฆื• ื•ื™ืกืžื™ื™ื“ืŸ ืงืึทืžืคึผื™ื˜ื™ื ื’ ืžื™ื˜ ืื ื“ืขืจืข ืคึผืจืึทืกืขืกืึทื– ืคึฟืึทืจ ืงืคึผื•, ื–ื›ึผืจื•ืŸ, ื ืขืฅ ืื•ืŸ ื“ื™ืกืง ืจืขืกื•ืจืกืŸ.

Java ื•ื•ืขืจืกื™ืข

ื“ื™ ืœืึทื“ื–ืฉื™ืงืึทืœ ื‘ืจื™ืจื” ืื™ื– Java 11 ื•ื•ื™ื™ึทืœ ืขืก ืื™ื– ืงืึทืžืคึผืึทื˜ืึทื‘ืึทืœ ืžื™ื˜ ื“ืึธืงืงืขืจ ืื™ืŸ ื“ืขื ื–ื™ื ืขืŸ ืึทื– ื“ื™ JVM ืจื™ื›ื˜ื™ืง ื“ื™ื˜ืขืจืžืึทื ื– ื“ื™ ืคึผืจืึทืกืขืกืขืจื– ืื•ืŸ ื–ื›ึผืจื•ืŸ ื‘ื ื™ืžืฆื ืฆื• ื“ืขื ืงืึทื ื˜ื™ื™ื ืขืจ ืื™ืŸ ื•ื•ืึธืก ื“ืขืจ ืžืขืงืœืขืจ ืื™ื– ืคืœื™ืกื ื“ื™ืง. ื•ื•ื™ื™ืœ ืึทื– ื“ื™ ืงืคึผื• ืœื™ืžืึทืฅ ื–ืขื ืขืŸ ื•ื•ื™ื›ื˜ื™ืง, ื“ื™ JVM ื™ื ืขืจืœืขืš ืื•ืŸ ื˜ืจืึทื ืกืคึผืขืจืึทื ื˜ืœื™ ืฉื˜ืขืœืŸ ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ GC ืคึฟืขื“ืขื ืื•ืŸ JIT ืคึฟืขื“ืขื. ืžื™ืจ ื”ืื‘ืŸ ื’ืขื ื•ืฆื˜ ื“ืขื ืงืืคืงื ื‘ื™ืœื“ banzaicloud/kafka:2.13-2.4.0, ื•ื•ืึธืก ื›ื•ืœืœ Kafka ื•ื•ืขืจืกื™ืข 2.4.0 (ืกืงืึทืœืึท 2.13) ืื•ื™ืฃ Java 11.

ืื•ื™ื‘ ืื™ืจ ื•ื•ื™ืœื˜ ืœืขืจื ืขืŸ ืžืขืจ ื•ื•ืขื’ืŸ Java / JVM ืื•ื™ืฃ Kubernetes, ื˜ืฉืขืง ืื•ื ื“ื–ืขืจ ืคืืœื’ืขื ื“ืข ืึทืจื˜ื™ืงืœืขืŸ:

ืžืขืงืœืขืจ ื–ื™ืงืึธืจืŸ ืกืขื˜ื˜ื™ื ื’ืก

ืขืก ื–ืขื ืขืŸ ืฆื•ื•ื™ื™ ืฉืœื™ืกืœ ืึทืกืคึผืขืงืฅ ืฆื• ืงืึทื ืคื™ื’ื™ืขืจ ืžืขืงืœืขืจ ื–ื›ึผืจื•ืŸ: ืกืขื˜ื˜ื™ื ื’ืก ืคึฟืึทืจ ื“ื™ JVM ืื•ืŸ ืคึฟืึทืจ ื“ื™ Kubernetes ืคึผืึธื“. ื“ื™ ื–ื™ืงืึธืจืŸ ืœื™ืžื™ื˜ ื‘ืึทืฉื˜ื™ืžื˜ ืคึฟืึทืจ ืึท ืคึผืึธื“ ืžื•ื–ืŸ ื–ื™ื™ืŸ ื’ืจืขืกืขืจ ื•ื•ื™ ื“ื™ ืžืึทืงืกื™ืžื•ื ืงื•ืคึผืข ื’ืจื™ื™ืก ืึทื–ื•ื™ ืึทื– ื“ื™ JVM ื”ืื˜ ืคึผืœืึทืฅ ืคึฟืึทืจ ื“ื™ Java ืžืขื˜ืึทืกืคึผืึทืกืข, ื•ื•ืึธืก ืจื™ื–ื™ื™ื“ื– ืื™ืŸ ื–ื™ื™ืŸ ืื™ื™ื’ืขื ืข ื–ื›ึผืจื•ืŸ, ืื•ืŸ ืคึฟืึทืจ ื“ื™ ืึธืคึผืขืจื™ื™ื˜ื™ื ื’ ืกื™ืกื˜ืขื ื‘ืœืึทื˜ ืงืึทืฉ, ื•ื•ืึธืก Kafka ืึทืงื˜ื™ื•ื•ืœื™ ื ื™ืฆื˜. ืื™ืŸ ืื•ื ื“ื–ืขืจ ื˜ืขืกืฅ ืžื™ืจ ืœืึธื ื˜ืฉื˜ ืงืึทืคืงืึท ื‘ืจืึธืงืขืจืก ืžื™ื˜ ืคึผืึทืจืึทืžืขื˜ืขืจืก -Xmx4G -Xms2G, ืื•ืŸ ื“ื™ ื–ื›ึผืจื•ืŸ ืฉื™ืขื•ืจ ืคึฟืึทืจ ื“ื™ ืคึผืึธื“ ืื™ื– ื’ืขื•ื•ืขืŸ 10 Gi. ื‘ื™ื˜ืข ื˜ืึธืŸ ืึทื– ื–ื™ืงืึธืจืŸ ืกืขื˜ื˜ื™ื ื’ืก ืคึฟืึทืจ ื“ื™ JVM ืงืขื ืขืŸ ื–ื™ื™ืŸ ื‘ืืงื•ืžืขืŸ ืื•ื™ื˜ืึธืžืึทื˜ื™ืฉ ื ื™ืฆืŸ -XX:MaxRAMPercentage ะธ -X:MinRAMPercentage, ื‘ืื–ื™ืจื˜ ืื•ื™ืฃ ื“ื™ ื–ื›ึผืจื•ืŸ ืฉื™ืขื•ืจ ืคึฟืึทืจ ื“ื™ ืคึผืึธื“.

ืžืขืงืœืขืจ ืคึผืจืึทืกืขืกืขืจ ืกืขื˜ื˜ื™ื ื’ืก

ืื™ืŸ ืึทืœื’ืขืžื™ื™ืŸ, ืื™ืจ ืงืขื ืขืŸ ืคึฟืึทืจื‘ืขืกืขืจืŸ ืคืึธืจืฉื˜ืขืœื•ื ื’ ื“ื•ืจืš ื™ื ืงืจื™ืกื™ื ื’ ืคึผืึทืจืึทืœืขืœื™ื–ืึทื ื“ื•ืจืš ื™ื ืงืจื™ืกื™ื ื’ ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ืคึฟืขื“ืขื ื’ืขื ื™ืฆื˜ ื“ื•ืจืš Kafka. ื“ื™ ืžืขืจ ืคึผืจืึทืกืขืกืขืจื– ื‘ื ื™ืžืฆื ืคึฟืึทืจ Kafka, ื“ื™ ื‘ืขืกืขืจ. ืื™ืŸ ืื•ื ื“ื–ืขืจ ืคึผืจืึธื‘ืข, ืžื™ืจ ืกื˜ืึทืจื˜ืขื“ ืžื™ื˜ ืึท ืฉื™ืขื•ืจ ืคื•ืŸ 6 ืคึผืจืึทืกืขืกืขืจื– ืื•ืŸ ื‘ื™ืกืœืขื›ื•ื•ื™ื™ึทื– (ื“ื•ืจืš ื™ื˜ืขืจื™ื™ืฉืึทื ื–) ืื•ื™ืคืฉื˜ื™ื™ืŸ ื–ื™ื™ืขืจ ื ื•ืžืขืจ ืฆื• 15. ืื™ืŸ ื“ืขืจืฆื•, ืžื™ืจ ืฉื˜ืขืœืŸ num.network.threads=12 ืื™ืŸ ื“ื™ ืžืขืงืœืขืจ ืกืขื˜ื˜ื™ื ื’ืก ืฆื• ืคืึทืจื’ืจืขืกืขืจืŸ ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ืคึฟืขื“ืขื ื•ื•ืึธืก ื‘ืึทืงื•ืžืขืŸ ื“ืึทื˜ืŸ ืคื•ืŸ ื“ื™ ื ืขืฅ ืื•ืŸ ืฉื™ืงืŸ ืขืก. ื’ืœื™ื™ืš ืึทื ื˜ื“ืขืงื˜ ืึทื– ื“ื™ ื ืื›ื’ื™ื™ืขืจ ื‘ืจืึธืงืขืจืก ืงืขืŸ ื ื™ืฉื˜ ื‘ืึทืงื•ืžืขืŸ ืจืขืคึผืœืึทืงืึทื– ื’ืขืฉื•ื•ื™ื ื“ ื’ืขื ื•ื’, ื–ื™ื™ ืื•ื™ืคืฉื˜ื™ื™ืŸ num.replica.fetchers ืฆื• 4 ืฆื• ืคืึทืจื’ืจืขืกืขืจืŸ ื“ื™ ื’ื™ื›ืงื™ื™ึทื˜ ืžื™ื˜ ื•ื•ืึธืก ื ืื›ื’ื™ื™ืขืจ ื‘ืจืึธืงืขืจืก ืจืขืคึผืœื™ืงื™ื™ื˜ื™ื“ ืึทืจื˜ื™ืงืœืขืŸ ืคื•ืŸ ืคื™ืจืขืจ.

ืžืึทืกืข ื“ื•ืจ ื˜ื•ืœ

ืื™ืจ ื–ืึธืœ ืขื ืฉื•ืจ ืึทื– ื“ื™ ืื•ื™ืกื’ืขืงืœื™ื‘ืŸ ืžืึทืกืข ื’ืขื ืขืจืึทื˜ืึธืจ ื˜ื•ื˜ ื ื™ืฉื˜ ืœื•ื™ืคืŸ ืื•ื™ืก ืคื•ืŸ ืงืึทืคึผืึทืฆื™ื˜ืขื˜ ืื™ื™ื“ืขืจ ื“ื™ Kafka ืงื ื•ื™ืœ (ื•ื•ืึธืก ืื™ื– ื‘ืขื ื˜ืฉืžืึทืจืงืขื“) ืจื™ื˜ืฉืึทื– ื–ื™ื™ืŸ ืžืึทืงืกื™ืžื•ื ืžืึทืกืข. ืื™ืŸ ืื ื“ืขืจืข ื•ื•ืขืจื˜ืขืจ, ืขืก ืื™ื– ื ื™ื™ื˜ื™ืง ืฆื• ื“ื•ืจื›ืคื™ืจืŸ ืึท ืคึผืจื™ืœื™ืžืึทื ืขืจื™ ืึทืกืขืกืžืึทื ื˜ ืคื•ืŸ ื“ื™ ืงื™ื™ืคึผืึทื‘ื™ืœืึทื˜ื™ื– ืคื•ืŸ ื“ื™ ืžืึทืกืข ื“ื•ืจื™ื ื’ ื’ืขืฆื™ื™ึทื’, ืื•ืŸ ืื•ื™ืš ืื•ื™ืกืงืœื™ื™ึทื‘ืŸ ื˜ื™ืคึผ ื˜ื™ื™ืคึผืก ืคึฟืึทืจ ืขืก ืžื™ื˜ ืึท ื’ืขื ื•ื’ ื ื•ืžืขืจ ืคื•ืŸ ืคึผืจืึทืกืขืกืขืจื– ืื•ืŸ ื–ื›ึผืจื•ืŸ. ืื™ืŸ ื“ืขื ืคืึทืœ, ืื•ื ื“ื–ืขืจ ื’ืขืฆื™ื™ึทื’ ื•ื•ืขื˜ ืคึผืจืึธื“ื•ืฆื™ืจืŸ ืžืขืจ ืžืึทืกืข ื•ื•ื™ ื“ื™ ืงืึทืคืงืึท ืงื ื•ื™ืœ ืงืขื ืขืŸ ืฉืขืคึผืŸ. ื ืึธืš ืคื™ืœืข ื™ืงืกืคึผืขืจืึทืžืึทื ืฅ, ืžื™ืจ ื’ืขื–ืขืฆื˜ ืื•ื™ืฃ ื“ืจื™ื™ึท ืงืึธืคึผื™ืขืก c5.4xlarge, ื™ืขื“ืขืจ ืคื•ืŸ ื•ื•ืึธืก ื”ืื˜ ืึท ื’ืขื ืขืจืึทื˜ืึธืจ ืคืœื™ืกื ื“ื™ืง.

ื‘ืขื ื˜ืฉืžืึทืจืงื™ื ื’

ืคืึธืจืฉื˜ืขืœื•ื ื’ ืžืขื–ืฉืขืจืžืึทื ื˜ ืื™ื– ืึท ื™ื˜ืขืจืึทื˜ื™ื•ื• ืคึผืจืึธืฆืขืก ื•ื•ืึธืก ื›ื•ืœืœ ื“ื™ ืคืืœื’ืขื ื“ืข ืกื˜ืึทื’ืขืก:

  • ื‘ืึทืฉื˜ืขื˜ื™ืงืŸ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ (EKS ืงื ื•ื™ืœ, Kafka ืงื ื•ื™ืœ, ืžืึทืกืข ื“ื•ืจ ื’ืขืฆื™ื™ึทื’, ืื•ืŸ ืคึผืจืึธืžืขื˜ื”ืขื•ืก ืื•ืŸ ื’ืจืึทืคืึทื ืึท);
  • ื“ื–ืฉืขื ืขืจื™ื™ื˜ื™ื ื’ ืึท ืžืึทืกืข ืคึฟืึทืจ ืึท ื–ื™ื›ืขืจ ืฆื™ื™ึทื˜ ืฆื• ืคื™ืœื˜ืขืจ ื˜ืจืึทืค - ื“ื™ื•ื•ื™ื™ื™ืฉืึทื ื– ืื™ืŸ ื“ื™ ื’ืขื–ืืžืœื˜ ืคืึธืจืฉื˜ืขืœื•ื ื’ ื™ื ื“ื™ืงืึทื˜ืึธืจืก;
  • ืึทื“ื–ืฉืึทืกื˜ื™ื ื’ ื“ื™ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ืื•ืŸ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ืคื•ืŸ ื“ื™ ืžืขืงืœืขืจ ื‘ืื–ื™ืจื˜ ืื•ื™ืฃ ื‘ืืžืขืจืงื˜ ืคืึธืจืฉื˜ืขืœื•ื ื’ ื™ื ื“ื™ืงืึทื˜ืึธืจืก;
  • ืจื™ืคึผื™ื˜ื™ื ื’ ื“ืขื ืคึผืจืึธืฆืขืก ื‘ื™ื– ื“ื™ ืคืืจืœืื ื’ื˜ ืžื“ืจื’ื” ืคื•ืŸ Kafka ืงื ื•ื™ืœ ื˜ืจื•ืคึผื•ื˜ ืื™ื– ืึทื˜ืฉื™ื•ื•ื“. ืื™ืŸ ื“ืขืจ ื–ืขืœื‘ื™ืงืขืจ ืฆื™ื™ื˜, ืขืก ืžื•ื–ืŸ ื–ื™ื™ืŸ ืงืึทื ืกื™ืกื˜ืึทื ื˜ืœื™ ืจืขืคึผืจืึธื“ื•ืกื™ื‘ืœืข ืื•ืŸ ื‘ืึทื•ื•ื™ื™ึทื–ืŸ ืžื™ื ื™ืžืึทืœ ื•ื•ืขืจื™ื™ื™ืฉืึทื ื– ืื™ืŸ ื˜ืจื•ืคึผื•ื˜.

ื“ืขืจ ื•ื•ื™ื™ึทื˜ืขืจ ืึธืคึผื˜ื™ื™ืœื•ื ื’ ื‘ืืฉืจื™ื™ื‘ื˜ ื“ื™ ืกื˜ืขืคึผืก ื•ื•ืึธืก ื–ืขื ืขืŸ ื“ื•ืจื›ื’ืขืงืึธื›ื˜ ื‘ืขืฉืึทืก ื“ื™ ืคึผืจืึธื‘ืข ืงื ื•ื™ืœ ื‘ืขื ื˜ืฉืžืึทืจืงื™ื ื’ ืคึผืจืึธืฆืขืก.

ืžื›ืฉื™ืจื™ื

ื“ื™ ืคืืœื’ืขื ื“ืข ืžื›ืฉื™ืจื™ื ื–ืขื ืขืŸ ื’ืขื ื™ืฆื˜ ืฆื• ื’ืขืฉื•ื•ื™ื ื“ ืฆืขื•ื•ื™ืงืœืขืŸ ืึท ื‘ืึทืกืขืœื™ื ืข ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ, ื“ื–ืฉืขื ืขืจื™ื™ื˜ ืœืึธื•ื“ื– ืื•ืŸ ืžืขืกื˜ืŸ ืคืึธืจืฉื˜ืขืœื•ื ื’:

  • Banzai Cloud Pipeline ืคึฟืึทืจ ืึธืจื’ืึทื ื™ื™ื–ื™ื ื’ ืึทืŸ EKS ืงื ื•ื™ืœ ืคึฟื•ืŸ Amazon c ืคึผืจืึธืžืขื˜ื”ืขื•ืก (ืฆื• ื–ืึทืžืœืขืŸ ืงืึทืคืงืึท ืื•ืŸ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ ืžืขื˜ืจื™ืงืก) ืื•ืŸ ื’ืจืืคืื ื (ืฆื• ื•ื•ื™ื–ืฉื•ื•ืึทืœื™ื™ื– ื“ื™ ืžืขื˜ืจื™ืงืก). ืžื™ ืจ ื”ืื‘ ืŸ ื–ื™ ืš ื’ืขื ื•ืฆื˜ ื™ื ืึทื’ืจื™ื™ื˜ื™ื“ ะฒ ืจืขืจื  - ืœื™ื ื™ืข ืกืขืจื•ื•ื™ืกืขืก ื•ื•ืึธืก ืฆื•ืฉื˜ืขืœืŸ ืคืขื“ืขืจื™ื™ื˜ื™ื“ ืžืึธื ื™ื˜ืึธืจื™ื ื’, ืกืขื ื˜ืจืึทืœื™ื™ื–ื“ ืงืœืึธืฅ ื–ืึทืžืœื•ื ื’, ื•ื•ืึทืœื ืขืจืึทื‘ื™ืœื™ื˜ื™ ืกืงืึทื ื™ื ื’, ื•ืžื’ืœื™ืง ืึธืคึผื–ื•ืš, ืคืึทืจื ืขืžื•ื ื’-ืžื™ื™ื ื•ื ื’ ื–ื™ื›ืขืจื”ื™ื™ื˜ ืื•ืŸ ืคื™ืœ ืžืขืจ.
  • ืกืึทื ื’ืจืขื ืขืœ - ืึท ื’ืขืฆื™ื™ึทื’ ืคึฟืึทืจ ืžืึทืกืข ื˜ืขืกื˜ื™ื ื’ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ.
  • Grafana ื“ืึทืฉื‘ืึธืจื“ื– ืคึฟืึทืจ ื•ื•ื™ื–ืฉื•ื•ืึทืœื™ื™ื–ื™ื ื’ Kafka ืžืขื˜ืจื™ืงืก ืื•ืŸ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ: Kubernetes Kafka, ื ืึธื“ืข ืขืงืกืคึผืึธืจื˜ืขืจ.
  • Supertubes CLI ืคึฟืึทืจ ื“ื™ ื™ื–ื™ืึทืกื˜ ื•ื•ืขื’ ืฆื• ืฉื˜ืขืœืŸ ืึท Kafka ืงื ื•ื™ืœ ืื•ื™ืฃ Kubernetes. ื–ืึธืึธืงืขืขืคึผืขืจ, ืงืึทืคืงืึท ืึธืคึผืขืจืึทื˜ืึธืจ, ืขื ื•ื•ื•ื™ ืื•ืŸ ืคื™ืœืข ืื ื“ืขืจืข ืงืึทืžืคึผืึธื•ื ืึทื ืฅ ื–ืขื ืขืŸ ืื™ื ืกื˜ืึทืœื™ืจืŸ ืื•ืŸ ืจืขื›ื˜ ืงืึทื ืคื™ื’ื™ืขืจื“ ืฆื• ืœื•ื™ืคืŸ ืึท ืคึผืจืึธื“ื•ืงืฆื™ืข-ื’ืจื™ื™ื˜ ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื•ื™ืฃ ืงื•ื‘ืขืจื ืขื˜ืขืก.
    • ืฆื• ื™ื ืกื˜ืึทืœื™ืจืŸ supertubes CLI ื ื•ืฆืŸ ื“ื™ ืฆื•ื’ืขืฉื˜ืขืœื˜ ื™ื ืกื˜ืจืึทืงืฉืึทื ื– ื“ืึธ.

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

EKS ืงื ื•ื™ืœ

ืฆื•ื’ืจื™ื™ื˜ืŸ ืึทืŸ EKS ืงื ื•ื™ืœ ืžื™ื˜ ื“ืขื“ืึทืงื™ื™ื˜ืึทื“ ืึทืจื‘ืขื˜ ื ืึธื•ื“ื– c5.4xlarge ืื™ืŸ ืคืึทืจืฉื™ื“ืขื ืข ืึทื•ื•ื™ื™ืœืึทื‘ื™ืœื™ื˜ื™ ื–ืึธื ืขืก ืคึฟืึทืจ ืคึผืึธื“ืก ืžื™ื˜ Kafka ื‘ืจืึธืงืขืจืก, ื•ื•ื™ ื’ืขื–ื•ื ื˜ ื•ื•ื™ ื“ืขื“ืึทืงื™ื™ื˜ืึทื“ ื ืึธื•ื“ื– ืคึฟืึทืจ ื“ื™ ืžืึทืกืข ื’ืขื ืขืจืึทื˜ืึธืจ ืื•ืŸ ืžืึธื ื™ื˜ืึธืจื™ื ื’ ื™ื ืคืจืึทืกื˜ืจืึทืงื˜ืฉืขืจ.

banzai cluster create -f https://raw.githubusercontent.com/banzaicloud/kafka-operator/master/docs/benchmarks/infrastructure/cluster_eks_202001.json

ืึทืžืึธืœ ื“ื™ EKS ืงื ื•ื™ืœ ืื™ื– ืึทืจื•ื™ืฃ ืื•ืŸ ืคืœื™ืกื ื“ื™ืง, ื’ืขื‘ืŸ ื–ื™ื™ืŸ ื™ื ืึทื’ืจื™ื™ื˜ื™ื“ ืžืึธื ื™ื˜ืึธืจื™ื ื’ ื“ื™ื ืกื˜ - ื–ื™ ื•ื•ืขื˜ ืฆืขื•ื•ื™ืงืœืขืŸ ืคึผืจืึธืžืขื˜ื”ืขื•ืก ืื•ืŸ ื’ืจืึทืคืึทื ืึท ืื™ืŸ ืึท ืงื ื•ื™ืœ.

ืงืึทืคืงืึท ืกื™ืกื˜ืขื ืงืึทืžืคึผืึธื•ื ืึทื ืฅ

ื™ื ืกื˜ืึทืœื™ืจืŸ ืงืึทืคืงืึท ืกื™ืกื˜ืขื ืงืึทืžืคึผืึธื•ื ืึทื ืฅ (ื–ืึธืึธืงืขืขืคึผืขืจ, ืงืึทืคืงืึท-ืึธืคึผืขืจืึทื˜ืึธืจ) ืื™ืŸ EKS ื ื™ืฆืŸ ืกื•ืคึผืขืจื˜ื•ื‘ืขืก CLI:

supertubes install -a --no-democluster --kubeconfig <path-to-eks-cluster-kubeconfig-file>

ืงืืคืงื ืืฉื›ื•ืœ

ื“ื•ืจืš ืคืขืœื™ืงื™ื™ึทื˜, EKS ื ื™ืฆื˜ EBS ื•ื•ืึทืœื™ื•ืžื– ืคื•ืŸ ื˜ื™ืคึผ ื’ืคึผืงืกื ื•ืžืงืก, ืึทื–ื•ื™ ืื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ืฉืึทืคึฟืŸ ืึท ื‘ืึทื–ื•ื ื“ืขืจ ืกื˜ืึธืจื™ื“ื–ืฉ ืงืœืึทืก ื‘ืื–ื™ืจื˜ ืื•ื™ืฃ ื•ื•ืึทืœื™ื•ืžื– ื™ืึธืงืกื ื•ืžืงืก ืคึฟืึทืจ ืงืึทืคืงืึท ืงื ื•ื™ืœ:

kubectl create -f - <<EOF
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: kubernetes.io/aws-ebs
parameters:
  type: io1
  iopsPerGB: "50"
  fsType: ext4
volumeBindingMode: WaitForFirstConsumer
EOF

ืฉื˜ืขืœืŸ ื“ื™ ืคึผืึทืจืึทืžืขื˜ืขืจ ืคึฟืึทืจ ื‘ืจืึธืงืขืจืก min.insync.replicas=3 ืื•ืŸ ืฆืขื•ื•ื™ืงืœืขืŸ ืžืขืงืœืขืจ ืคึผืึธื“ืก ืื•ื™ืฃ ื ืึธื•ื“ื– ืื™ืŸ ื“ืจื™ื™ ืคืึทืจืฉื™ื“ืขื ืข ืึทื•ื•ื™ื™ืœืึทื‘ื™ืœืึทื˜ื™ ื–ืึธื ืขืก:

supertubes cluster create -n kafka --kubeconfig <path-to-eks-cluster-kubeconfig-file> -f https://raw.githubusercontent.com/banzaicloud/kafka-operator/master/docs/benchmarks/infrastructure/kafka_202001_3brokers.yaml --wait --timeout 600

ื˜ืขืžืขืก

ืžื™ืจ ืœื•ื™ืคืŸ ื“ืจื™ื™ ืœืึธื“ืŸ ื’ืขื ืขืจืึทื˜ืึธืจ ื™ื ืกื˜ืึทื ืกื™ื– ืื™ืŸ ืคึผืึทืจืึทืœืขืœ. ื™ืขื“ืขืจ ืคื•ืŸ ื–ื™ื™ ืฉืจื™ื™ื‘ื˜ ืฆื• ื–ื™ื™ืขืจ ืื™ื™ื’ืŸ ื˜ืขืžืข, ื“ืึธืก ืื™ื–, ืžื™ืจ ื“ืึทืจืคึฟืŸ ื“ืจื™ื™ ื˜ืขืžืขืก ืื™ืŸ ื’ืึทื ืฅ:

supertubes cluster topic create -n kafka --kubeconfig <path-to-eks-cluster-kubeconfig-file> -f -<<EOF
apiVersion: kafka.banzaicloud.io/v1alpha1
kind: KafkaTopic
metadata:
  name: perftest1
spec:
  name: perftest1
  partitions: 12
  replicationFactor: 3
  retention.ms: '28800000'
  cleanup.policy: delete
EOF

supertubes cluster topic create -n kafka --kubeconfig <path-to-eks-cluster-kubeconfig-file> -f -<<EOF
apiVersion: kafka.banzaicloud.io/v1alpha1
kind: KafkaTopic
metadata:
    name: perftest2
spec:
  name: perftest2
  partitions: 12
  replicationFactor: 3
  retention.ms: '28800000'
  cleanup.policy: delete
EOF

supertubes cluster topic create -n kafka --kubeconfig <path-to-eks-cluster-kubeconfig-file> -f -<<EOF
apiVersion: kafka.banzaicloud.io/v1alpha1
kind: KafkaTopic
metadata:
  name: perftest3
spec:
  name: perftest3
  partitions: 12
  replicationFactor: 3
  retention.ms: '28800000'
  cleanup.policy: delete
EOF

ืคึฟืึทืจ ื™ืขื“ืขืจ ื˜ืขืžืข, ื“ื™ ืจืขืคึผืœืึทืงื™ื™ืฉืึทืŸ ืคืึทืงื˜ืึธืจ ืื™ื– 3 - ื“ื™ ืžื™ื ื™ืžื•ื ืจืขืงืึทืžืขื ื“ื™ื“ ื•ื•ืขืจื˜ ืคึฟืึทืจ ื”ืขื›ืกื˜ ื‘ื ื™ืžืฆื ืคึผืจืึธื“ื•ืงืฆื™ืข ืกื™ืกื˜ืขืžืขืŸ.

ืžืึทืกืข ื“ื•ืจ ื˜ื•ืœ

ืžื™ืจ ืœืึธื ื˜ืฉื˜ ื“ืจื™ื™ ืงืืคื™ืขืก ืคื•ืŸ ื“ื™ ืžืึทืกืข ื’ืขื ืขืจืึทื˜ืึธืจ (ื™ืขื“ืขืจ ืื™ื™ื ืขืจ ื’ืขืฉืจื™ื‘ืŸ ืื™ืŸ ืึท ื‘ืึทื–ื•ื ื“ืขืจ ื˜ืขืžืข). ืคึฟืึทืจ ืœืึธื“ืŸ ื’ืขื ืขืจืึทื˜ืึธืจ ืคึผืึธื“ืก, ืื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ืฉื˜ืขืœืŸ ื ืึธื“ืข ืงื™ืจื‘ื•ืช ืึทื–ื•ื™ ืึทื– ื–ื™ื™ ื–ืขื ืขืŸ ืกืงืขื“ื–ืฉื•ืœื“ ื‘ืœื•ื™ื– ืื•ื™ืฃ ื“ื™ ื ืึธื•ื“ื– ืึทืœืึทืงื™ื™ื˜ื™ื“ ืคึฟืึทืจ ื–ื™ื™:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: loadtest
  name: perf-load1
  namespace: kafka
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: loadtest
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: loadtest
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: nodepool.banzaicloud.io/name
                operator: In
                values:
                - loadgen
      containers:
      - args:
        - -brokers=kafka-0:29092,kafka-1:29092,kafka-2:29092,kafka-3:29092
        - -topic=perftest1
        - -required-acks=all
        - -message-size=512
        - -workers=20
        image: banzaicloud/perfload:0.1.0-blog
        imagePullPolicy: Always
        name: sangrenel
        resources:
          limits:
            cpu: 2
            memory: 1Gi
          requests:
            cpu: 2
            memory: 1Gi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30

ืขื˜ืœืขื›ืข ื ืงื•ื“ื•ืช ืฆื• ื˜ืึธืŸ:

  • ื“ื™ ืžืึทืกืข ื’ืขื ืขืจืึทื˜ืึธืจ ื“ื–ืฉืขื ืขืจื™ื™ืฅ ืึทืจื˜ื™ืงืœืขืŸ ืคื•ืŸ 512 ื‘ื™ื˜ืขืก ืื™ืŸ ืœืขื ื’ ืื•ืŸ ืคึผืึทื‘ืœื™ืฉืึทื– ื–ื™ื™ ืฆื• Kafka ืื™ืŸ ื‘ืึทื˜ืฉืึทื– ืคื•ืŸ 500 ืึทืจื˜ื™ืงืœืขืŸ.
  • ื ื™ืฆืŸ ืึท ืึทืจื’ื•ืžืขื ื˜ -required-acks=all ื“ื™ ื•ื™ืกื’ืึทื‘ืข ืื™ื– ื’ืขืจืขื›ื ื˜ ื•ื•ื™ ื’ืขืจืึธื˜ืŸ ื•ื•ืขืŸ ืึทืœืข ืกื™ื ื’ืงืจืึทื ื™ื™ื–ื“ ืจืขืคึผืœืึทืงืึทื– ืคื•ืŸ ื“ืขื ืึธื ื–ืึธื’ ื–ืขื ืขืŸ ื‘ืืงื•ืžืขืŸ ืื•ืŸ ื‘ืืฉื˜ืขื˜ื™ืงื˜ ื“ื•ืจืš ืงืึทืคืงืึท ื‘ืจืึธืงืขืจืก. ื“ืึธืก ืžื™ื™ื ื˜ ืึทื– ืื™ืŸ ื“ื™ ื‘ืขื ื˜ืฉืžืึทืจืง ืžื™ืจ ื’ืขืžืืกื˜ืŸ ื ื™ื˜ ื‘ืœื•ื™ื– ื“ื™ ื’ื™ื›ืงื™ื™ึทื˜ ืคื•ืŸ ืคื™ืจืขืจ ื•ื•ืึธืก ื‘ืึทืงื•ืžืขืŸ ืึทืจื˜ื™ืงืœืขืŸ, ืึธื‘ืขืจ ืื•ื™ืš ื–ื™ื™ืขืจ ืื ื”ืขื ื’ืขืจืก ืจืขืคึผืœื™ืงื™ื™ื˜ ืึทืจื˜ื™ืงืœืขืŸ. ื“ืขืจ ืฆื™ืœ ืคื•ืŸ ื“ืขื ืคึผืจืึธื‘ืข ืื™ื– ื ื™ืฉื˜ ืฆื• ืึธืคึผืฉืึทืฆืŸ ืงืึทื ืกื•ืžืขืจ ืœื™ื™ืขื ืขืŸ ื’ื™ื›ืงื™ื™ึทื˜ (ืงืึทื ืกื•ืžืขืจ) ืœืขืฆื˜ื ืก ื‘ืืงื•ืžืขืŸ ืึทืจื˜ื™ืงืœืขืŸ ื•ื•ืึธืก ื ืึธืš ื‘ืœื™ื™ื‘ืŸ ืื™ืŸ ื“ื™ ืึทืก ื‘ืœืึทื˜ ืงืึทืฉ, ืื•ืŸ ื–ื™ื™ืŸ ืคืึทืจื’ืœื™ื™ึทืš ืžื™ื˜ ื“ื™ ืœื™ื™ืขื ืขืŸ ื’ื™ื›ืงื™ื™ึทื˜ ืคื•ืŸ ืึทืจื˜ื™ืงืœืขืŸ ืกื˜ืึธืจื“ ืื•ื™ืฃ ื“ื™ืกืง.
  • ื“ื™ ืžืึทืกืข ื’ืขื ืขืจืึทื˜ืึธืจ ืœื•ื™ืคื˜ 20 ื˜ื•ืขืจืก ืื™ืŸ ืคึผืึทืจืึทืœืขืœ (-workers=20). ื™ืขื“ืข ืจ ืืจื‘ื™ื™ื˜ืขืจ ืื ื˜ื”ืืœื˜ 5 ืคืจืื“ื•ืฆื™ืจืขืจ , ืฐืขืœื› ืข ื˜ืฒืœ ืŸ ื“ืข ื ืืจื‘ืขื˜ืขืจ ืก ืคืืจื‘ื™ื ื“ื•ื  ื’ ืฆ ื• ื“ืข ื ืงืืคืงืึพืงื ื•ื™ืœ . ื•ื•ื™ ืึท ืจืขื–ื•ืœื˜ืึทื˜, ื™ืขื“ืขืจ ื’ืขื ืขืจืึทื˜ืึธืจ ื”ืื˜ 100 ืคึผืจืึธื“ื•ืกืขืจืก, ืื•ืŸ ื–ื™ื™ ืึทืœืข ืฉื™ืงืŸ ืึทืจื˜ื™ืงืœืขืŸ ืฆื• ื“ื™ ืงืึทืคืงืึท ืงื ื•ื™ืœ.

ืžืึธื ื™ื˜ืึธืจื™ื ื’ ื“ื™ ื’ืขื–ื•ื ื˜ ืคื•ืŸ ื“ืขื ืงื ื•ื™ืœ

ื‘ืขืฉืึทืก ืžืึทืกืข ื˜ืขืกื˜ื™ื ื’ ืคื•ืŸ ื“ื™ Kafka ืงื ื•ื™ืœ, ืžื™ืจ ืื•ื™ืš ืžืึธื ื™ื˜ืึธืจืขื“ ื–ื™ื™ืŸ ื’ืขื–ื•ื ื˜ ืฆื• ืขื ืฉื•ืจ ืึทื– ืขืก ื–ืขื ืขืŸ ืงื™ื™ืŸ ืคึผืึธื“ ืจื™ืกื˜ืึทืจื˜ื–, ืงื™ื™ืŸ ืึทื•ื˜-ืคื•ืŸ-ืกื™ื ืง ืจืขืคึผืœืึทืงืึทื– ืื•ืŸ ืžืึทืงืกื™ืžื•ื ื˜ืจื•ืคึผื•ื˜ ืžื™ื˜ ืžื™ื ื™ืžืึทืœ ืคืœืึทืงื˜ืฉื•ื™ื™ืฉืึทื ื–:

  • ื“ืขืจ ืžืึทืกืข ื’ืขื ืขืจืึทื˜ืึธืจ ืฉืจื™ื™ื‘ื˜ ื ืึธืจืžืึทืœ ืกื˜ืึทื˜ื™ืกื˜ื™ืง ื•ื•ืขื’ืŸ ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ืึทืจื˜ื™ืงืœืขืŸ ืืจื•ื™ืก ืื•ืŸ ื“ื™ ื˜ืขื•ืช ืงื•ืจืก. ื“ืขืจ ื˜ืขื•ืช ืงื•ืจืก ื–ืึธืœ ื‘ืœื™ื™ึทื‘ืŸ ื“ื™ ื–ืขืœื‘ืข 0,00%.
  • Cruise Control, ื“ื™ืคึผืœื•ื™ื“ ื“ื•ืจืš ืงืึทืคืงืึท-ืึธืคึผืขืจืึทื˜ืึธืจ, ื’ื™ื˜ ืึท ื“ืึทืฉื‘ืึธืจื“ ื•ื•ื• ืžื™ืจ ืงืขื ืขืŸ ืื•ื™ืš ืžืึธื ื™ื˜ืึธืจ ื“ื™ ืฉื˜ืึทื˜ ืคื•ืŸ ื“ื™ ืงื ื•ื™ืœ. ืฆื• ื–ืขืŸ ื“ืขื ื˜ืึทืคืœื™ืข ื˜ืึธืŸ:
    supertubes cluster cruisecontrol show -n kafka --kubeconfig <path-to-eks-cluster-kubeconfig-file>
  • ISR ืžื“ืจื’ื” (ื ื•ืžืขืจ ืคื•ืŸ "ืื™ืŸ-ืกื™ื ืง" ืจืขืคึผืœืึทืงืึทื–) ื™ื™ึทื ืฉืจื•ืžืคึผืŸ ืื•ืŸ ื™ืงืกืคึผืึทื ืฉืึทืŸ ื–ืขื ืขืŸ ื’ืœื™ื™ึทืš ืฆื• 0.

ืžืขื–ืฉืขืจืžืึทื ื˜ ืจืขื–ื•ืœื˜ืึทื˜ืŸ

3 ื‘ืจืึธืงืขืจืก, ืึธื ื–ืึธื’ ื’ืจื™ื™ืก - 512 ื‘ื™ื˜ืขืก

ืžื™ื˜ ืคึผืึทืจื˜ื™ืฉืึทื ื– ื™ื•ื•ืึทื ืœื™ ืคื•ื ืื ื“ืขืจื’ืขื˜ื™ื™ืœื˜ ืื™ื‘ืขืจ ื“ืจื™ื™ ื‘ืจืึธืงืขืจืก, ืžื™ืจ ื–ืขื ืขืŸ ื‘ื™ื›ื•ืœืช ืฆื• ื“ืขืจื’ืจื™ื™ื›ืŸ ืคืึธืจืฉื˜ืขืœื•ื ื’ ~ 500 ืžื‘ / s (ื‘ืขืขืจืขืš 990 ื˜ื•ื™ื–ื ื˜ ืึทืจื˜ื™ืงืœืขืŸ ืคึผืขืจ ืกืขืงื•ื ื“ืข):

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื“ื™ ื–ื™ืงืึธืจืŸ ืงืึทื ืกืึทืžืฉืึทืŸ ืคื•ืŸ ื“ื™ JVM ื•ื•ื™ืจื˜ื•ืึทืœ ืžืึทืฉื™ืŸ ื”ืื˜ ื ื™ืฉื˜ ื™ืงืกื™ื“ 2 ื’ื™ื’ืื‘ื™ื™ื˜:

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื“ื™ืกืง ื˜ืจื•ืคึผื•ื˜ ืจื™ื˜ืฉื˜ ื“ื™ ืžืึทืงืกื™ืžื•ื I/O ื ืึธื“ืข ื˜ืจื•ืคึผื•ื˜ ืื™ืŸ ืึทืœืข ื“ืจื™ื™ ืงืึทืกืขืก ืื•ื™ืฃ ื•ื•ืึธืก ื“ื™ ื‘ืจืึธืงืขืจืก ื–ืขื ืขืŸ ืคืœื™ืกื ื“ื™ืง:

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ืคึฟื•ืŸ ื“ื™ ื“ืึทื˜ืŸ ื•ื•ืขื’ืŸ ื–ื™ืงืึธืจืŸ ื‘ืึทื ื™ืฅ ื“ื•ืจืš ื ืึธื•ื“ื–, ืขืก ื’ื™ื™ื˜ ืึทื– ืกื™ืกื˜ืขื ื‘ืึทืคืขืจื™ื ื’ ืื•ืŸ ืงืึทื˜ืฉื™ื ื’ ื’ืขื ื•ืžืขืŸ ~ 10-15 ื’ื™ื’ืื‘ื™ื™ื˜:

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

3 ื‘ืจืึธืงืขืจืก, ืึธื ื–ืึธื’ ื’ืจื™ื™ืก - 100 ื‘ื™ื˜ืขืก

ื•ื•ืขืŸ ื“ื™ ืึธื ื–ืึธื’ ื’ืจื™ื™ืก ื“ื™ืงืจื™ืกืึทื–, ื“ื™ ื˜ืจื•ืคึผื•ื˜ ื˜ืจืืคื ืก ืžื™ื˜ ื‘ืขืขืจืขืš 15-20%: ื“ื™ ืฆื™ื™ื˜ ืคืืจื‘ืจืื›ื˜ ืคึผืจืึทืกืขืกื™ื ื’ ื™ืขื“ืขืจ ืึธื ื–ืึธื’ ืึทืคืขืงืฅ ืขืก. ืื™ืŸ ื“ืขืจืฆื•, ื“ื™ ืคึผืจืึทืกืขืกืขืจ ืžืึทืกืข ืื™ื– ื›ึผืžืขื˜ ื“ืึทื‘ืึทืœื“.

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื–ื™ื ื˜ ืžืขืงืœืขืจ ื ืึธื•ื“ื– ื ืึธืš ื”ืึธื‘ืŸ ืึทื ื™ื•ื–ื“ ืงืึธืจืขืก, ืคืึธืจืฉื˜ืขืœื•ื ื’ ืงืขื ืขืŸ ื–ื™ื™ืŸ ื™ืžืคึผืจื•ื•ื•ื“ ื“ื•ืจืš ื˜ืฉืึทื ื’ื™ื ื’ ื“ื™ Kafka ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ. ื“ืึธืก ืื™ื– ื ื™ืฉื˜ ืึทืŸ ื’ืจื™ื ื’ ืึทืจื‘ืขื˜, ืึทื–ื•ื™ ืฆื• ืคืึทืจื’ืจืขืกืขืจืŸ ื˜ืจื•ืคึผื•ื˜ ืขืก ืื™ื– ื‘ืขืกืขืจ ืฆื• ืึทืจื‘ืขื˜ืŸ ืžื™ื˜ ื’ืจืขืกืขืจืข ืึทืจื˜ื™ืงืœืขืŸ.

4 ื‘ืจืึธืงืขืจืก, ืึธื ื–ืึธื’ ื’ืจื™ื™ืก - 512 ื‘ื™ื˜ืขืก

ืื™ืจ ืงืขื ืขืŸ ืœื™ื™ื›ื˜ ืคืึทืจื’ืจืขืกืขืจืŸ ื“ื™ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคื•ืŸ ืึท Kafka ืงื ื•ื™ืœ ื“ื•ืจืš ืคืฉื•ื˜ ืึทื“ื™ื ื’ ื ื™ื™ึท ื‘ืจืึธืงืขืจืก ืื•ืŸ ื”ืึทืœื˜ืŸ ืึท ื•ื•ืึธื’ ืคื•ืŸ ืคึผืึทืจื˜ื™ืฉืึทื ื– (ื“ืึธืก ื™ื ืฉื•ืจื– ืึทื– ื“ื™ ืžืึทืกืข ืื™ื– ื™ื•ื•ืึทื ืœื™ ืคื•ื ืื ื“ืขืจื’ืขื˜ื™ื™ืœื˜ ืฆื•ื•ื™ืฉืŸ ื‘ืจืึธืงืขืจืก). ืื™ืŸ ืื•ื ื“ื–ืขืจ ืคืึทืœ, ื ืึธืš ืึทื“ื™ื ื’ ืึท ืžืขืงืœืขืจ, ื“ื™ ืงื ื•ื™ืœ ื˜ืจื•ืคึผื•ื˜ ื’ืขื•ื•ืืงืกืŸ ืฆื• ~ 580 ืžื‘ / s (~ 1,1 ืžื™ืœื™ืึธืŸ ืึทืจื˜ื™ืงืœืขืŸ ืคึผืขืจ ืกืขืงื•ื ื“ืข). ื“ืขืจ ื•ื•ื•ึผืงืก ืื™ื– ื’ืขื•ื•ืขืŸ ื•ื•ื™ื™ื ื™ืงืขืจ ื•ื•ื™ ื“ืขืจื•ื•ืึทืจื˜: ื“ืึธืก ืื™ื– ื“ืขืจ ื”ื•ื™ืคึผื˜ ื“ืขืจืงืœืขืจื˜ ื“ื•ืจืš ื“ื™ ื™ืžื‘ืึทืœืึทื ืก ืคื•ืŸ ืคึผืึทืจื˜ื™ืฉืึทื ื– (ื ื™ื˜ ืึทืœืข ื‘ืจืึธืงืขืจืก ืึทืจื‘ืขื˜ ืื™ืŸ ื“ื™ ืฉืคึผื™ืฅ ืคื•ืŸ ื–ื™ื™ืขืจ ืงื™ื™ืคึผืึทื‘ื™ืœืึทื˜ื™ื–).

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื–ื›ึผืจื•ืŸ ืงืึทื ืกืึทืžืฉืึทืŸ ืคื•ืŸ ื“ื™ JVM ืžืึทืฉื™ืŸ ืื™ื– ืคืืจื‘ืœื™ื‘ืŸ ืื•ื ื˜ืขืจ 2 GB:

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื“ื™ ืึทืจื‘ืขื˜ ืคื•ืŸ ื‘ืจืึธืงืขืจืก ืžื™ื˜ ื“ืจื™ื™ื•ื•ื– ืื™ื– ืึทืคืขืงื˜ืึทื“ ื“ื•ืจืš ื“ื™ ื™ืžื‘ืึทืœืึทื ืก ืคื•ืŸ ืคึผืึทืจื˜ื™ืฉืึทื ื–:

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ื‘ืึทืฉื˜ื™ืžืขืŸ ื“ื™ ืฆื•ื ืขืžืขืŸ ื’ืจื™ื™ืก ืคึฟืึทืจ ืึท ืงืึทืคืงืึท ืงื ื•ื™ืœ ืื™ืŸ ืงื•ื‘ืขืจื ืขื˜ืขืก

ืคื™ื™ื ื“ื™ื ื’ื–

ื“ื™ ื™ื˜ืขืจืึทื˜ื™ื•ื• ืฆื•ื’ืึทื ื’ ื•ื•ืึธืก ืื™ื– ื“ืขืจืœืื ื’ื˜ ืื•ื™ื‘ืŸ ืงืขื ืขืŸ ื–ื™ื™ืŸ ื™ืงืกืคึผืึทื ื“ื™ื“ ืฆื• ื“ืขืงืŸ ืžืขืจ ืงืึธืžืคึผืœื™ืฆื™ืจื˜ ืกื™ื ืขืจื™ืึธื•ื– ืžื™ื˜ ื”ื•ื ื“ืขืจื˜ืขืจ ืคื•ืŸ ืงืึธื ืกื•ืžืขืจืก, ืจืขืคึผืึทืจื˜ื™ืึธื ื™ื ื’, ืจืึธื•ืœื™ื ื’ ื“ืขืจื”ื™ื™ึทื ื˜ื™ืงื•ื ื’ืขืŸ, ืคึผืึธื“ ืจื™ืกื˜ืึทืจื˜, ืขื˜ืง. ืึทืœืข ื“ืขื ืึทืœืึทื•ื– ืื•ื ื“ื– ืฆื• ืึทืกืกืขืกืก ื“ื™ ืœื™ืžืึทืฅ ืคื•ืŸ ืงืึทืคืงืึท ืงื ื•ื™ืœ ืก ืงื™ื™ืคึผืึทื‘ื™ืœืึทื˜ื™ื– ืื™ืŸ ืคืึทืจืฉื™ื“ืŸ ื˜ื ืึธื™ื, ื™ื“ืขื ื˜ื™ืคื™ืฆื™ืจืŸ ื‘ืึทื˜ืึทืœื ืขืงืก ืื™ืŸ ื–ื™ื™ึทืŸ ืึธืคึผืขืจืึทืฆื™ืข ืื•ืŸ ื’ืขืคึฟื™ื ืขืŸ ื•ื•ืขื’ืŸ ืฆื• ืงืึทืžื‘ืึทื˜ ื–ื™ื™.

ืžื™ืจ ื“ื™ื–ื™ื™ื ื“ ืกื•ืคึผืขืจื˜ื•ื‘ืขืก ืฆื• ื’ืขืฉื•ื•ื™ื ื“ ืื•ืŸ ืœื™ื™ื›ื˜ ืฆืขื•ื•ื™ืงืœืขืŸ ืึท ืงื ื•ื™ืœ, ืงืึทื ืคื™ื’ื™ืขืจ ืขืก, ืœื™ื™ื’ืŸ / ืึทืจืึธืคึผื ืขืžืขืŸ ื‘ืจืึธืงืขืจืก ืื•ืŸ ื˜ืขืžืขืก, ืจื™ืกืคึผืึทื ื“ ืฆื• ืึทืœืขืจืฅ ืื•ืŸ ืขื ืฉื•ืจ ืึทื– Kafka ืื™ืŸ ืึทืœื’ืขืžื™ื™ืŸ ืึทืจื‘ืขื˜ ืจืขื›ื˜ ืื•ื™ืฃ Kubernetes. ืื•ื ื“ื–ืขืจ ืฆื™ืœ ืื™ื– ืฆื• ื”ืขืœืคึฟืŸ ืื™ืจ ืงืึทื ืกืึทื ื˜ืจื™ื™ื˜ ืื•ื™ืฃ ื“ื™ ื”ื•ื™ืคึผื˜ ืึทืจื‘ืขื˜ ("ื“ื–ืฉืขื ืขืจื™ืจืŸ" ืื•ืŸ "ืคืึทืจื ื•ืฆืŸ" ืงืึทืคืงืึท ืึทืจื˜ื™ืงืœืขืŸ), ืื•ืŸ ืœืึธื–ืŸ ืึทืœืข ื“ื™ ืฉื•ื•ืขืจ ืึทืจื‘ืขื˜ ืฆื• ืกื•ืคึผืขืจื˜ื•ื‘ืขืก ืื•ืŸ ื“ื™ ืงืึทืคืงืึท ืึธืคึผืขืจืึทื˜ืึธืจ.

ืื•ื™ื‘ ืื™ืจ ื–ืขื ื˜ ืื™ื ื˜ืขืจืขืกื™ืจื˜ ืื™ืŸ Banzai ืงืœืึธื•ื“ ื˜ืขืงื ืึทืœืึทื“ื–ืฉื™ื– ืื•ืŸ ืขืคึฟืŸ ืžืงื•ืจ ืคึผืจืึทื“ื–ืฉืขืงืก, ืึทื‘ืึธื ื™ืจืŸ ืฆื• ื“ื™ ืคื™ืจืžืข ืื™ืŸ ื’ื™ื˜ื”ื•ื‘, ืœื™ื ืงืขื“ื™ืŸ ืึธื“ืขืจ ื˜ื•ื•ื™ื˜ื˜ืขืจ.

ืคึผืก ืคื•ืŸ ืื™ื‘ืขืจื–ืขืฆืขืจ

ืœื™ื™ืขื ืขืŸ ืื•ื™ืš ืื•ื™ืฃ ืื•ื ื“ื–ืขืจ ื‘ืœืึธื’:

ืžืงื•ืจ: www.habr.com

ืœื™ื™ื’ืŸ ืึท ื‘ืึทืžืขืจืงื•ื ื’