ããã¯ãåæã·ã¹ãã ã«é¢ããäžé£ã®èšäºã®ç¬¬ XNUMX éšã§ã (
ä»æ¥ã§ã¯ãæ
éãªããŒã¿åŠçãšçµæã®è§£éããã»ãŒãã¹ãŠã®çš®é¡ã®ããžãã¹ã«åœ¹ç«ã€ããšã¯ããã¯ãçãã®äœå°ããããŸããã ãã®ç¹ã§ãåæã·ã¹ãã ã«ã¯ãã©ã¡ãŒã¿ããŸããŸãããŒããããããã«ãªããã¢ããªã±ãŒã·ã§ã³å
ã®ããªã¬ãŒãšãŠãŒã¶ãŒ ã€ãã³ãã®æ°ãå¢å ããŠããŸãã
ãã®ãããäŒæ¥ã¯åæããŠå¥å
šãªææ決å®ãè¡ãããã®çã®æ
å ±ãã¢ããªã¹ãã«ãŸããŸãæäŸããŠããŸãã äŒæ¥ã«ãšã£ãŠã®åæã·ã¹ãã ã®éèŠæ§ãéå°è©äŸ¡ãã¹ãã§ã¯ãªããã·ã¹ãã èªäœãä¿¡é Œæ§ãããå®å®ããŠããå¿
èŠããããŸãã
ã¯ã©ã€ã¢ã³ãã¢ããªã¹ã
顧客åæã¯ãäŒæ¥ãå
¬åŒ SDK ãéããŠèªç€Ÿã® Web ãµã€ããŸãã¯ã¢ããªã±ãŒã·ã§ã³ã«æ¥ç¶ããç¬èªã®ã³ãŒãããŒã¹ã«çµ±åããŠã€ãã³ã ããªã¬ãŒãéžæãããµãŒãã¹ã§ãã ãã®ã¢ãããŒãã«ã¯æãããªæ¬ ç¹ããããŸããéžæãããµãŒãã¹ã®å¶éã«ãããåéããããã¹ãŠã®ããŒã¿ãåžæã©ããã«åŠçãããªãå¯èœæ§ããããŸãã ããšãã°ãããã·ã¹ãã ã§ã¯ MapReduce ã¿ã¹ã¯ãå®è¡ããã®ã¯ç°¡åã§ã¯ãããŸããããå¥ã®ã·ã¹ãã ã§ã¯ã¢ãã«ãå®è¡ã§ããŸããã ãã XNUMX ã€ã®æ¬ ç¹ã¯ããµãŒãã¹ã®å®æç㪠(é«é¡ãª) è«æ±é¡ã§ãã
åžå Žã«ã¯å€ãã®é¡§å®¢åæãœãªã¥ãŒã·ã§ã³ããããŸãããã¢ããªã¹ãã¯é
ããæ©ãããããããã¿ã¹ã¯ã«é©ããåäžã®ãŠãããŒãµã« ãµãŒãã¹ãååšããªããšããäºå®ã«çŽé¢ããããšã«ãªããŸã (ããããã¹ãŠã®ãµãŒãã¹ã®äŸ¡æ Œã¯åžžã«äžæããŠããŸã)ã ãã®ãããªç¶æ³ã§ã¯ãå€ãã®äŒæ¥ã¯ãå¿
èŠãªã«ã¹ã¿ã èšå®ãšæ©èœããã¹ãŠåããç¬èªã®åæã·ã¹ãã ãäœæããããšã決å®ããŸãã
ãµãŒããŒã¢ããªã¹ã
ãµãŒããŒãµã€ãåæã¯ãäŒæ¥å
ã§èªç€Ÿã®ãµãŒããŒäžã« (éåžžã¯) ç¬èªã®åªåã§å°å
¥ã§ãããµãŒãã¹ã§ãã ãã®ã¢ãã«ã§ã¯ããã¹ãŠã®ãŠãŒã¶ãŒ ã€ãã³ããå
éšãµãŒããŒã«ä¿åããããããéçºè
ã¯ããŸããŸãªã¹ãã¬ãŒãž ããŒã¿ããŒã¹ãè©ŠããŠãæã䟿å©ãªã¢ãŒããã¯ãã£ãéžæã§ããŸãã ãŸããäžéšã®ã¿ã¹ã¯ã§ãµãŒãããŒãã£ã®ã¯ã©ã€ã¢ã³ãåæã䜿çšãããå Žåã§ããããã¯å¯èœã§ãã
ãµãŒããŒåŽåæ㯠XNUMX ã€ã®æ¹æ³ã§å±éã§ããŸãã ãŸãããªãŒãã³ãœãŒã¹ ãŠãŒãã£ãªãã£ãããã€ãéžæãããã·ã³ã«å±éããŠããžãã¹ ããžãã¯ãéçºããŸãã
ãããã¡
ã³ã³ãº
å¿
èŠãªãã®ã¯äœã§ãã«ã¹ã¿ãã€ãºã§ããŸã
ããã¯å€ãã®å Žåéåžžã«å°é£ã§ãããå¥ã®éçºè
ãå¿
èŠã«ãªããŸãã
XNUMX çªç®: èªåã§ãããã€ããã®ã§ã¯ãªããSaaS ãµãŒãã¹ (AmazonãGoogleãAzure) ãå©çšããŸãã SaaS ã«ã€ããŠã¯ã第 XNUMX åã§è©³ãã説æããŸãã
ãããã¡
ã³ã³ãº
äžçšåºŠã®éã§ã¯å®ããªããããããŸãããã倧å¹
ãªæé·ã§ã¯äŸç¶ãšããŠé«ããªããããã§ããã
ãã¹ãŠã®ãã©ã¡ãŒã¿ãå¶åŸ¡ããããšã¯ã§ããŸãã
管çã¯å®å
šã«ãµãŒãã¹ãããã€ããŒã®è©ã«ç§»ç®¡ãããŸã
ãµãŒãã¹ã®å
éšã«äœããããã¯åžžã«ç¥ãããŠããããã§ã¯ãããŸãã (å¿
èŠãªãå ŽåããããŸã)
ãµãŒããŒåæãåéããæ¹æ³
ã¯ã©ã€ã¢ã³ãåæã®äœ¿çšããããŠç¬èªã®åæãæ§ç¯ãããå Žåã¯ããŸãæ°ããã·ã¹ãã ã®ã¢ãŒããã¯ãã£ãããèããå¿ èŠããããŸãã 以äžã§ã¯ãäœãèæ ®ããå¿ èŠãããããåã¹ããããå¿ èŠãªçç±ãããã³äœ¿çšã§ããããŒã«ã段éçã«èª¬æããŸãã
1. ããŒã¿ã®åä¿¡
顧客åæã®å Žåãšåæ§ã«ãäŒæ¥ã®ã¢ããªã¹ãã¯ãŸããä»åŸèª¿æ»ãããã€ãã³ãã®çš®é¡ãéžæãããªã¹ãã«åéããŸãã éåžžããããã®ã€ãã³ãã¯ãã€ãã³ã ãã¿ãŒã³ããšåŒã°ããç¹å®ã®é åºã§çºçããŸãã
次ã«ãã¢ãã€ã« ã¢ããªã±ãŒã·ã§ã³ (Web ãµã€ã) ã«éåžžã®ãŠãŒã¶ãŒ (ããã€ã¹) ãšå€æ°ã®ãµãŒããŒããããšæ³åããŠãã ããã ã€ãã³ããããã€ã¹ãããµãŒããŒã«å®å
šã«è»¢éããã«ã¯ãäžéå±€ãå¿
èŠã§ãã ã¢ãŒããã¯ãã£ã«ãã£ãŠã¯ãè€æ°ã®ç°ãªãã€ãã³ã ãã¥ãŒãååšããå ŽåããããŸãã
ã«ãã
Quoraã«æçš¿ãã 2014 幎ãApache Kafka ã®äœæè ã¯ããå·çã«æé©åãããã·ã¹ãã ãã§ããããšãšãã«ãã«ã®äœåã奜ãã ã£ãããšããããã®ãœãããŠã§ã¢ã«ãã©ã³ã ã«ãã«ã«ã¡ãªãã§ååãä»ããããšã«ããŸããã âãŠã£ãããã£ã¢
ãã®äŸã§ã¯ãå€ãã®ããŒã¿ ãããã¥ãŒãµãŒãšããŒã¿ ã³ã³ã·ã¥ãŒã㌠(ããã€ã¹ãšãµãŒããŒ) ããããKafka ã¯ããããçžäºã«æ¥ç¶ããã®ã«åœ¹ç«ã¡ãŸãã æ¶è²»è
ã«ã€ããŠã¯ãäž»ãªäž»é¡ãšããŠæ¬¡ã®ã¹ãããã§ããã«è©³ãã説æããŸãã ããã§ã¯ãããŒã¿ãããã¥ãŒãµãŒ (ã€ãã³ã) ã®ã¿ãèæ
®ããŸãã
Kafka ã¯ãã¥ãŒãšããŒãã£ã·ã§ã³ã®æŠå¿µãã«ãã»ã«åããŠããŸããããã«ã€ããŠã¯ãä»ã®å Žæ (ããšãã°ã
ïŒåç
åæã«ãKafka ã䜿çšãããšããã£ã³ã¯ã§èªã¿åããã€ãã³ãã®ã¹ããªãŒã ããããããã§åŠçã§ããŸãã Kafka ã¯ãå¢å€§ããããŒãº (ã€ãã³ãã®å°çäœçœ®æ
å ±ãªã©) ã«åãããŠé©åã«æ¡åŒµã§ãããéåžžã«äŸ¿å©ãªããŒã«ã§ãã
é垞㯠XNUMX ã€ã®ã·ã£ãŒãã§ååã§ãããã¹ã±ãŒãªã³ã°ãããš (ãã€ãã®ããã«) ç¶æ³ã¯ããã«è€éã«ãªããŸãã ã¢ãŒããã¯ãã£ã¯ãã©ãŒã«ããã¬ã©ã³ãã§ããå¿
èŠããããããéçšç°å¢ã§ç©çã·ã£ãŒãã XNUMX ã€ã ã䜿çšããããšæã人ã¯ããããããªãã§ãããã Kafka ã«å ããŠããã XNUMX ã€ã®ããç¥ããããœãªã¥ãŒã·ã§ã³ãšã㊠RabbitMQ ããããŸãã å®çšŒåç°å¢ã§ã¯ã€ãã³ãåæçšã®ãã¥ãŒãšããŠäœ¿çšããŸããã§ãã (ãã®ãããªçµéšãããå Žåã¯ãã³ã¡ã³ãã§æããŠãã ãã!)ã ãã ããAWS Kinesis ã䜿çšããŸããã
次ã®ã¹ãããã«é²ãåã«ãã·ã¹ãã ã®ãã XNUMX ã€ã®è¿œå ã¬ã€ã€ãŒã§ããçã®ãã° ã¹ãã¬ãŒãžã«ã€ããŠèšåããå¿
èŠããããŸãã ããã¯å¿
é ã®ã¬ã€ã€ãŒã§ã¯ãããŸããããäœãåé¡ãçºçã㊠Kafka ã®ã€ãã³ã ãã¥ãŒããªã»ãããããå Žåã«åœ¹ç«ã¡ãŸãã çã®ãã°ãä¿åããã«ã¯ãè€éã§é«äŸ¡ãªãœãªã¥ãŒã·ã§ã³ã¯å¿
èŠãªããæ£ããé åºã§ã©ãã (ããŒã ãã©ã€ãäžã§ãã£ãŠã) ã«æžã蟌ãã ãã§æžã¿ãŸãã
2. ã€ãã³ãã¹ããªãŒã ã®åŠç
ãã¹ãŠã®ã€ãã³ããæºåããé©åãªãã¥ãŒã«é
眮ããåŸãåŠçã¹ãããã«é²ã¿ãŸãã ããã§ã¯ãæãäžè¬ç㪠XNUMX ã€ã®åŠçãªãã·ã§ã³ã«ã€ããŠèª¬æããŸãã
æåã®ãªãã·ã§ã³ã¯ãApache ã·ã¹ãã 㧠Spark ã¹ããªãŒãã³ã°ãæå¹ã«ããããšã§ãã ãã¹ãŠã® Apache 補åã¯ããã¡ã€ã« ã¬ããªã«ãåããå®å
šãªãã¡ã€ã« ã·ã¹ãã ã§ãã HDFS äžã«ååšããŸãã Spark Streaming ã¯ãã¹ããªãŒãã³ã° ããŒã¿ãåŠçããé©åã«æ¡åŒµã§ãã䜿ããããããŒã«ã§ãã ãã ããç¶æããã®ãé£ããå ŽåããããŸãã
ãã XNUMX ã€ã®ãªãã·ã§ã³ã¯ãç¬èªã®ã€ãã³ã ãã³ãã©ãŒãæ§ç¯ããããšã§ãã ãããè¡ãã«ã¯ãããšãã°ãPython ã¢ããªã±ãŒã·ã§ã³ãäœæããDocker ã§ãã«ãããŠãKafka ãã¥ãŒã«ãµãã¹ã¯ã©ã€ãããå¿
èŠããããŸãã ããªã¬ãŒã Docker ãã³ãã©ãŒã«å°çãããšãåŠçãéå§ãããŸãã ãã®æ¹æ³ã§ã¯ãã¢ããªã±ãŒã·ã§ã³ãåžžã«å®è¡ããŠããå¿
èŠããããŸãã
äžèšã®ãªãã·ã§ã³ã®ãããããéžæãããšä»®å®ããŠãåŠçèªäœã«é²ã¿ãŸãããã ããã»ããµã¯ãããŒã¿ã®æå¹æ§ããã§ãã¯ããã¬ããŒãžããå£ãããã€ãã³ãããã£ã«ã¿ãªã³ã°ããããšããå§ããå¿
èŠããããŸãã æ€èšŒã«ã¯é垞䜿çšããŸã
3. ããŒã¿ããŒã¹
XNUMX çªç®ã®ã¹ãããã¯ãæ£èŠåãããã€ãã³ããç¶æããããšã§ãã æ¢è£œã®åæã·ã¹ãã ã䜿çšããå Žåã¯ãé »ç¹ã«ã¢ã¯ã»ã¹ããå¿
èŠãããããã䟿å©ãªããŒã¿ããŒã¹ãéžæããããšãéèŠã§ãã
ããŒã¿ãåºå®ã¹ããŒã ã«ããŸãé©åããå Žåã¯ã次ã®ãããããéžæã§ããŸãã
éæ§é åããŒã¿ã®å Žåã¯ãããšãã°ãNoSQL ã䜿çšã§ããŸãã
ããšãã°ããã£ãšåçŽãªãã®ãäœæããããšãã§ããŸãã
4. éèš
ãã¹ãŠã®ã€ãã³ããæ
éã«ä¿åããããå°çããããããããã¹ãŠã®éèŠãªæ
å ±ãåéããããŒã¿ããŒã¹ãæŽæ°ããŸãã ç§ãã¡ã¯äžççã«ãé¢é£ããããã·ã¥ããŒããšææšãååŸããããšèããŠããŸãã ããšãã°ãã€ãã³ããããŠãŒã¶ãŒ ãããã¡ã€ã«ãåéããäœããã®æ¹æ³ã§è¡åã枬å®ããŸãã ã€ãã³ãã¯éçŽãåéããã(ãŠãŒã¶ãŒ ããŒãã«ã«) å床ä¿åãããŸãã åæã«ãã¢ã°ãªã²ãŒã¿ãŒã³ãŒãã£ããŒã¿ãŒã«ãã£ã«ã¿ãŒãæ¥ç¶ããŠãç¹å®ã®çš®é¡ã®ã€ãã³ãããã®ã¿ãŠãŒã¶ãŒãåéã§ããããã«ã·ã¹ãã ãæ§ç¯ããããšãã§ããŸãã
ãã®åŸãããŒã å
ã®èª°ããé«åºŠãªåæã®ã¿ãå¿
èŠãšããå Žåã¯ãå€éšåæã·ã¹ãã ãæ¥ç¶ã§ããŸãã ããã¯ã¹ããã«ãããäžåºŠåè¬ã§ããŸãã ãã ããéåžžã«é«äŸ¡ã§ããããããã¹ãŠã®ãŠãŒã¶ãŒ ã€ãã³ããããã«éä¿¡ãããããã§ã¯ãªããå¿
èŠãªãã®ã ããéä¿¡ãããŸãã ãããè¡ãã«ã¯ãçã®ã€ãã³ããç§ãã¡èªèº«ã以åã«éçŽãããã®ãå€éšã·ã¹ãã ãAPIããŸãã¯åºåãã©ãããã©ãŒã ã«è»¢éããã³ãŒãã£ããŒã¿ãŒãäœæããå¿
èŠããããŸãã
5. ããã³ããšã³ã
äœæããã·ã¹ãã ã«ããã³ããšã³ããæ¥ç¶ããå¿
èŠããããŸãã è¯ãäŸã¯ãµãŒãã¹ã§ã
- ãŠãŒã¶ãŒã¯ SQL ã¯ãšãªãäœæããŸãã
- ããã«å¿ããŠåœŒã¯ãµã€ã³ãåãåããŸãã
- ããã«å¯ŸããŠãæ°ããèŠèŠåããäœæãããèªåçšã«ä¿åã§ããçŸããã°ã©ããåŸãããŸãã
ãµãŒãã¹å
ã®ããžã¥ã¢ã©ã€ãŒãŒã·ã§ã³ã¯èªåæŽæ°ãããã¢ãã¿ãªã³ã°ãã«ã¹ã¿ãã€ãºããŠè¿œè·¡ã§ããŸãã Redash ã¯èªå·±ãã¹ãåã®å Žåã¯ç¡æã§ãããSaaS ã®å Žåã¯æé¡ 50 ãã«ããããŸãã
ãŸãšã
äžèšã®æé ããã¹ãŠå®äºãããããµãŒããŒåæãäœæããŸãã ãã¹ãŠãèªåã§æ§æããå¿
èŠããããããããã¯åã«é¡§å®¢åæãæ¥ç¶ããã»ã©åçŽã§ã¯ãªãããšã«æ³šæããŠãã ããã ãããã£ãŠãç¬èªã®ã·ã¹ãã ãäœæããåã«ãæ¬æ Œçãªåæã·ã¹ãã ã®å¿
èŠæ§ãšãããã«å²ãåœãŠããããªãœãŒã¹ãæ¯èŒãã䟡å€ããããŸãã
èšç®ãè¡ã£ãçµæãã³ã¹ããé«ãããããšãããã£ãå Žåã¯ã次ã®ããŒãã§ããµãŒããŒåŽåæã®å®äŸ¡ãªããŒãžã§ã³ãäœæããæ¹æ³ã«ã€ããŠèª¬æããŸãã
èªãã§ãããŠããããšãïŒ ã³ã¡ã³ãã§ã質åããã ããã°å¹žãã§ãã
åºæïŒ habr.com