Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes
Wannan labarin zai taimake ka ka fahimci yadda ma'auni na nauyi ke aiki a Kubernetes, abin da ke faruwa lokacin haɓaka haɗin kai na dogon lokaci, da kuma dalilin da ya sa ya kamata ka yi la'akari da daidaitawar abokin ciniki idan kana amfani da HTTP / 2, gRPC, RSockets, AMQP, ko wasu ka'idoji masu tsawo. . 

Kadan game da yadda ake sake rarraba zirga-zirga a Kubernetes 

Kubernetes yana ba da abstractions masu dacewa guda biyu don ƙaddamar da aikace-aikacen: Sabis da Ayyuka.

Ƙaddamarwa suna bayyana yadda da kuma nawa kwafin aikace-aikacenku ya kamata su gudana a kowane lokaci. Ana tura kowace aikace-aikacen azaman Pod kuma an sanya adireshin IP.

Sabis suna kama da aiki zuwa ma'aunin nauyi. An ƙirƙira su don rarraba zirga-zirga a cikin fastoci da yawa.

Bari mu ga yadda yake kama.

  1. A cikin zanen da ke ƙasa zaku iya ganin misalai guda uku na aikace-aikacen iri ɗaya da ma'aunin nauyi:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  2. Ana kiran ma'auni mai ɗaukar nauyi Sabis kuma an sanya shi adireshin IP. Ana tura duk wani buƙatun mai shigowa zuwa ɗaya daga cikin kwas ɗin:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  3. Yanayin turawa yana ƙayyade adadin lokuta na aikace-aikacen. Kusan ba za ku taɓa faɗaɗa kai tsaye a ƙarƙashin:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  4. Ana sanya kowane faifan adireshin IP ɗinsa:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

Yana da amfani a yi tunanin ayyuka azaman tarin adiresoshin IP. Duk lokacin da ka sami damar sabis ɗin, ana zaɓar ɗaya daga cikin adiresoshin IP daga jerin kuma ana amfani da su azaman adireshin wurin.

Ga alama haka.

  1. Ana karɓar buƙatar curl 10.96.45.152 zuwa sabis:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  2. Sabis ɗin yana zaɓar ɗaya daga cikin adiresoshin fasfo guda uku a matsayin wurin da zai nufa:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  3. Ana karkatar da zirga-zirga zuwa takamaiman fasfo:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

Idan aikace-aikacenku ya ƙunshi gaba da baya, to za ku sami duka sabis da turawa ga kowane.

Lokacin da gaban gaba ya yi buƙatuwa ga bangon baya, baya buƙatar sanin ainihin kwas ɗin kwas ɗin na baya: ana iya samun ɗaya, goma, ko ɗari.

Har ila yau, gaban gaba bai san komai ba game da adiresoshin kwas ɗin da ke hidimar baya.

Lokacin da gaban gaba ya yi buƙatu zuwa ga baya, yana amfani da adireshin IP na sabis na baya, wanda baya canzawa.

Wannan shi ne yadda abin yake.

  1. Ƙarƙashin 1 yana buƙatar ɓangaren baya na ciki. Maimakon zaɓar takamaiman ɗaya don abin baya, yana yin buƙatu zuwa sabis:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  2. Sabis ɗin yana zaɓar ɗaya daga cikin kwas ɗin baya a matsayin adireshin inda ake nufi:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  3. Traffic yana tafiya daga Pod 1 zuwa Pod 5, wanda sabis ɗin ya zaɓa:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  4. Ƙarƙashin 1 bai san ainihin adadin kwas ɗin kamar ƙasa da 5 ke ɓoye a bayan sabis ɗin ba:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

Amma ta yaya daidai sabis ɗin ke rarraba buƙatun? Da alama ana amfani da ma'auni na zagaye-robin? Bari mu gane shi. 

Daidaitawa a cikin ayyukan Kubernetes

Babu sabis na Kubernetes. Babu tsari don sabis ɗin da aka sanya adireshin IP da tashar jiragen ruwa.

Kuna iya tabbatar da wannan ta shiga cikin kowane kumburi a cikin tari da gudanar da umarnin netstat -ntlp.

Ba za ku iya ma sami adireshin IP da aka keɓe ga sabis ɗin ba.

Adireshin IP na sabis ɗin yana cikin sashin sarrafawa, a cikin mai sarrafawa, kuma an yi rikodin shi a cikin bayanan - da sauransu. Adireshin iri ɗaya yana amfani da wani sashi - kube-proxy.
Kube-proxy yana karɓar jerin adiresoshin IP don duk ayyuka kuma yana haifar da saitin ƙa'idodin iptables akan kowane kumburi a cikin tari.

Waɗannan ƙa'idodin sun ce: "Idan muka ga adireshin IP na sabis ɗin, muna buƙatar canza adireshin wurin da buƙatun kuma aika zuwa ɗaya daga cikin kwas ɗin."

Adireshin IP ɗin sabis ɗin ana amfani dashi azaman wurin shigarwa kawai kuma ba'a yin amfani da shi ta kowane tsari yana sauraron waccan adireshin IP da tashar jiragen ruwa.

Mu kalli wannan

  1. Yi la'akari da gungu na nodes uku. Kowane kumburi yana da kwasfa:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  2. Dauren fentin launin beige wani bangare ne na sabis. Saboda babu sabis ɗin a matsayin tsari, ana nuna shi da launin toka:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  3. Pod na farko yana buƙatar sabis kuma dole ne ya je ɗaya daga cikin abubuwan da ke da alaƙa:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  4. Amma sabis ɗin ba ya wanzu, tsarin ba ya wanzu. Ta yaya yake aiki?

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  5. Kafin buƙatun ya bar kumburi, yana wucewa ta ka'idodin iptables:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  6. Dokokin iptables sun san cewa sabis ɗin ba ya wanzu kuma ya maye gurbin adireshin IP ɗin sa tare da ɗaya daga cikin adiresoshin IP na kwas ɗin da ke da alaƙa da waccan sabis ɗin:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  7. Buƙatar tana karɓar ingantaccen adireshin IP azaman adireshin inda ake nufi kuma ana sarrafa shi akai-akai:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  8. Dangane da topology na cibiyar sadarwa, buƙatar ƙarshe ta kai ga fasfo:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

Shin iptables na iya ɗaukar ma'auni?

A'a, ana amfani da iptables don tacewa kuma ba a tsara su don daidaitawa ba.

Koyaya, yana yiwuwa a rubuta saitin ƙa'idodin da ke aiki kamar pseudo-balancer.

Kuma wannan shine ainihin abin da aka aiwatar a Kubernetes.

Idan kuna da kwasfa uku, kube-proxy zai rubuta dokoki masu zuwa:

  1. Zaɓi sub na farko tare da yuwuwar 33%, in ba haka ba je zuwa doka ta gaba.
  2. Zaɓi na biyu tare da yuwuwar 50%, in ba haka ba je zuwa doka ta gaba.
  3. Zaɓi na uku a ƙarƙashin.

Wannan tsarin yana haifar da zaɓar kowane kwasfa tare da yuwuwar 33%.

Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

Kuma babu tabbacin cewa za a zaɓi Pod 2 na gaba bayan Pod 1.

Примечание: iptables yana amfani da tsarin ƙididdiga tare da rarraba bazuwar. Don haka, daidaitawar algorithm yana dogara ne akan zaɓin bazuwar.

Yanzu da kuka fahimci yadda ayyuka ke aiki, bari mu kalli mafi kyawun yanayin sabis.

Haɗin haɗin da aka daɗe a cikin Kubernetes ba sa ƙima ta tsohuwa

Kowane buƙatun HTTP daga gaban gaba zuwa bayan baya ana amfani da shi ta hanyar haɗin TCP daban, wanda aka buɗe kuma yana rufe.

Idan gaban gaba ya aika buƙatun 100 a sakan daya zuwa ƙarshen baya, to ana buɗe haɗin TCP daban-daban 100 kuma an rufe su.

Kuna iya rage lokacin sarrafa buƙatun da kaya ta buɗe haɗin TCP guda ɗaya da amfani da shi don duk buƙatun HTTP masu zuwa.

Ka'idar HTTP tana da fasalin da ake kira HTTP keep-alive, ko sake amfani da haɗi. A wannan yanayin, ana amfani da haɗin TCP guda ɗaya don aikawa da karɓar buƙatun HTTP da yawa da martani:

Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

Ba a kunna wannan fasalin ta tsohuwa: duka uwar garken da abokin ciniki dole ne a saita su daidai.

Saitin kanta abu ne mai sauƙi kuma mai sauƙi ga yawancin harsunan shirye-shirye da mahalli.

Ga wasu hanyoyin haɗin kai zuwa misalai a cikin harsuna daban-daban:

Me zai faru idan muka yi amfani da rayayyu a cikin sabis na Kubernetes?
Bari mu ɗauka cewa duka gaba da baya suna goyan bayan ci gaba da rayuwa.

Muna da kwafin gaba ɗaya da kwafi uku na bayan baya. Gaban gaba yana yin buƙatun farko kuma yana buɗe haɗin TCP zuwa ƙarshen baya. Buƙatar ta kai ga sabis ɗin, an zaɓi ɗaya daga cikin kwas ɗin baya azaman adireshin wurin. Ƙarshen baya yana aika amsa, kuma gaba yana karɓa.

Ba kamar yanayin da aka saba ba inda aka rufe haɗin TCP bayan karɓar amsa, yanzu ana buɗe shi don ƙarin buƙatun HTTP.

Me zai faru idan gaban gaba ya aika ƙarin buƙatun zuwa ƙarshen baya?

Don tura waɗannan buƙatun, za a yi amfani da haɗin TCP mai buɗewa, duk buƙatun za su je baya ɗaya inda buƙatun farko ya tafi.

Shin bai kamata iptables su sake rarraba zirga-zirgar ba?

Ba a wannan yanayin ba.

Lokacin da aka ƙirƙiri haɗin TCP, yana tafiya ta ƙa'idodin iptables, waɗanda ke zaɓar takamaiman baya inda zirga-zirgar za ta tafi.

Tunda duk buƙatun na gaba suna kan haɗin TCP da aka riga aka buɗe, ba a ƙara kiran ƙa'idodin iptables.

Bari mu ga yadda yake kama.

  1. Pod na farko yana aika buƙatu zuwa sabis:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  2. Kun riga kun san abin da zai faru a gaba. Babu sabis ɗin, amma akwai ƙa'idodin iptables waɗanda zasu aiwatar da buƙatar:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  3. Za a zaɓi ɗaya daga cikin kwas ɗin baya a matsayin adireshin inda ake nufi:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  4. Buƙatar ta kai ga kwafsa. A wannan gaba, za a kafa haɗin TCP mai tsayi tsakanin kwas ɗin biyu:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  5. Duk wani buƙatu na gaba daga faifan farko zai bi ta hanyar haɗin da aka riga aka kafa:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

Sakamakon shine lokutan amsawa da sauri kuma mafi girma kayan aiki, amma kuna rasa ikon sikelin bayanan baya.

Ko da kuna da kwasfa biyu a bayan baya, tare da haɗin kai akai-akai, zirga-zirga koyaushe zai tafi ɗaya daga cikinsu.

Za a iya gyara wannan?

Tun da Kubernetes bai san yadda ake daidaita haɗin kai ba, wannan aikin ya ratsa gare ku.

Sabis ɗin tarin adiresoshin IP ne da tashar jiragen ruwa da ake kira wuraren ƙarewa.

Aikace-aikacenku na iya samun jerin wuraren ƙarshe daga sabis ɗin kuma ku yanke shawarar yadda ake rarraba buƙatun tsakanin su. Kuna iya buɗe haɗin kai mai tsayi zuwa kowane fasfo da daidaita buƙatun tsakanin waɗannan haɗin yanar gizo ta amfani da zagaye-robin.

Ko nema ƙarin hadaddun daidaita algorithms.

Lambar gefen abokin ciniki wanda ke da alhakin daidaitawa yakamata ya bi wannan ma'ana:

  1. Samo jerin abubuwan ƙarshe daga sabis ɗin.
  2. Buɗe haɗin kai mai tsayi don kowane ƙarshen ƙarshen.
  3. Lokacin da ake buƙatar yin buƙata, yi amfani da ɗayan buɗewar haɗin gwiwa.
  4. Sabunta jerin abubuwan ƙarshe akai-akai, ƙirƙira sababbi ko rufe tsoffin haɗin kai idan lissafin ya canza.

Wannan shi ne abin da zai kasance.

  1. Maimakon kwaf ɗin farko da ke aika buƙatun zuwa sabis ɗin, zaku iya daidaita buƙatun a gefen abokin ciniki:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  2. Kuna buƙatar rubuta lambar da ke tambayar waɗanne fastoci ke cikin sabis ɗin:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  3. Da zarar kana da lissafin, ajiye shi a gefen abokin ciniki kuma yi amfani da shi don haɗawa da kwas ɗin:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

  4. Kai ne ke da alhakin ma'auni algorithm:

    Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

Yanzu tambayar ta taso: shin wannan matsalar ta shafi HTTP keep-alive?

Daidaita lodin gefen abokin ciniki

HTTP ba shine kawai yarjejeniya ba wanda zai iya amfani da haɗin TCP na dindindin.

Idan aikace-aikacenku yana amfani da ma'ajin bayanai, to ba a buɗe haɗin TCP a duk lokacin da kuke buƙatar yin buƙatu ko dawo da daftari daga bayanan. 

Madadin haka, an buɗe kuma ana amfani da haɗin TCP na dindindin zuwa bayanan bayanai.

Idan an shigar da bayanan ku akan Kubernetes kuma ana ba da damar shiga azaman sabis, to zaku ci karo da irin waɗannan matsalolin da aka bayyana a sashin da ya gabata.

Kwafin bayanai ɗaya za a fi lodi fiye da sauran. Kube-proxy da Kubernetes ba za su taimaka wajen daidaita haɗin kai ba. Dole ne ku kula don daidaita tambayoyin zuwa bayananku.

Ya danganta da wane ɗakin karatu da kuke amfani da shi don haɗawa da bayanan bayanai, kuna iya samun zaɓuɓɓuka daban-daban don magance wannan matsalar.

A ƙasa akwai misali na samun damar gungu na bayanai na MySQL daga Node.js:

var mysql = require('mysql');
var poolCluster = mysql.createPoolCluster();

var endpoints = /* retrieve endpoints from the Service */

for (var [index, endpoint] of endpoints) {
  poolCluster.add(`mysql-replica-${index}`, endpoint);
}

// Make queries to the clustered MySQL database

Akwai wasu ƙa'idodi da yawa waɗanda ke amfani da haɗin kai na TCP:

  • WebSockets da amintattun WebSockets
  • HTTP / 2
  • gRPC
  • RSockets
  • AMQP

Ya kamata ka riga ka saba da yawancin waɗannan ka'idoji.

Amma idan waɗannan ka'idoji sun shahara sosai, me yasa ba a sami daidaiton daidaitawa ba? Me yasa dabarar abokin ciniki ke buƙatar canzawa? Shin akwai maganin Kubernetes na asali?

Kube-proxy da iptables an ƙirƙira su don rufe mafi yawan lokuta masu amfani yayin turawa zuwa Kubernetes. Wannan don dacewa ne.

Idan kana amfani da sabis na gidan yanar gizo wanda ke fallasa API ɗin REST, kuna cikin sa'a - a wannan yanayin, ba a amfani da haɗin TCP na dindindin, zaku iya amfani da kowane sabis na Kubernetes.

Amma da zarar kun fara amfani da haɗin TCP na dindindin, dole ne ku gano yadda ake rarraba kaya daidai-da-wane a cikin bayan gida. Kubernetes baya ƙunshi shirye-shiryen da aka yi don wannan harka.

Koyaya, tabbas akwai zaɓuɓɓuka waɗanda zasu iya taimakawa.

Daidaita haɗin kai na dogon lokaci a cikin Kubernetes

Akwai nau'ikan sabis guda huɗu a cikin Kubernetes:

  1. ClusterIP
  2. NodePort
  3. LoadBalancer
  4. Rashin kai

Sabis guda uku na farko suna aiki bisa ga adireshin IP mai kama-da-wane, wanda kube-proxy ke amfani dashi don gina ƙa'idodin iptables. Amma ainihin tushen duk sabis sabis ne mara kai.

Sabis ɗin mara kai ba shi da kowane adireshin IP da ke da alaƙa da shi kuma yana ba da hanya kawai don dawo da jerin adiresoshin IP da tashar jiragen ruwa na kwas ɗin (madaidaicin wuraren) masu alaƙa da shi.

Duk ayyuka sun dogara ne akan sabis ɗin mara kai.

Sabis ɗin ClusterIP sabis ne marar kai tare da wasu ƙarin abubuwa: 

  1. Layer na gudanarwa yana sanya masa adireshin IP.
  2. Kube-proxy yana haifar da mahimman ka'idodin iptables.

Ta wannan hanyar zaku iya yin watsi da kube-proxy kuma kuyi amfani da jerin abubuwan ƙarshen da aka samo daga sabis ɗin mara kai don ɗaukar ma'auni na aikace-aikacenku.

Amma ta yaya za mu iya ƙara irin wannan dabaru ga duk aikace-aikacen da aka tura a cikin tari?

Idan an riga an tura aikace-aikacen ku, wannan aikin na iya zama kamar ba zai yiwu ba. Koyaya, akwai madadin zaɓi.

Sabis Mesh zai taimake ku

Wataƙila kun riga kun lura cewa dabarun daidaita lodin gefen abokin ciniki daidai ne.

Lokacin da aikace-aikacen ya fara, shi:

  1. Yana samun jerin adiresoshin IP daga sabis ɗin.
  2. Yana buɗewa da kula da tafkin haɗin gwiwa.
  3. Lokaci-lokaci yana sabunta tafkin ta ƙara ko cire wuraren ƙarshe.

Da zarar aikace-aikacen yana son yin buƙata, shi:

  1. Yana zaɓar haɗin da ke akwai ta amfani da wasu dabaru (misali zagaye-robin).
  2. Yana aiwatar da buƙatar.

Waɗannan matakan suna aiki don haɗin yanar gizo na WebSockets, gRPC, da AMQP.

Kuna iya raba wannan dabarar zuwa wani ɗakin karatu na daban kuma kuyi amfani da shi a cikin aikace-aikacenku.

Koyaya, zaku iya amfani da meshes sabis kamar Istio ko Linkerd maimakon.

Sabis Mesh yana haɓaka aikace-aikacenku tare da tsari wanda:

  1. Neman adiresoshin IP na sabis ta atomatik.
  2. Gwajin haɗin kai kamar WebSockets da gRPC.
  3. Daidaita buƙatun ta amfani da madaidaicin yarjejeniya.

Sabis Mesh yana taimakawa sarrafa zirga-zirga a cikin tari, amma yana da matukar amfani. Wasu zaɓuɓɓuka suna amfani da ɗakunan karatu na ɓangare na uku kamar Netflix Ribbon ko proxies na shirye-shirye kamar Manzo.

Me zai faru idan kun yi watsi da batutuwan daidaitawa?

Kuna iya zaɓar kada ku yi amfani da daidaita nauyi kuma har yanzu ba ku lura da kowane canje-canje ba. Bari mu kalli wasu ƴan yanayin aiki.

Idan kuna da ƙarin abokan ciniki fiye da sabobin, wannan ba babbar matsala ba ce.

Bari mu ce akwai abokan ciniki guda biyar waɗanda ke haɗa zuwa sabobin biyu. Ko da babu daidaitawa, za a yi amfani da sabobin biyu:

Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

Ƙila ba za a rarraba haɗin kai daidai gwargwado ba: watakila abokan ciniki huɗu sun haɗa su zuwa uwar garken guda ɗaya, amma akwai kyakkyawar dama cewa za a yi amfani da duka sabobin.

Abin da ya fi matsala shi ne yanayin sabanin haka.

Idan kuna da ƙarancin abokan ciniki da ƙarin sabar, ƙila ba za a yi amfani da albarkatun ku ba kuma yuwuwar cikas za ta bayyana.

Bari mu ce akwai abokan ciniki biyu da sabar guda biyar. A cikin mafi kyawun yanayin, za a sami haɗin kai biyu na dindindin zuwa sabobin biyu cikin biyar.

Sauran sabobin za su kasance marasa aiki:

Load ɗin daidaitawa da haɓaka haɗin kai na dogon lokaci a cikin Kubernetes

Idan waɗannan sabar guda biyu ba za su iya ɗaukar buƙatun abokin ciniki ba, ƙima a kwance ba zai taimaka ba.

ƙarshe

An tsara ayyukan Kubernetes don yin aiki a mafi yawan daidaitattun yanayin aikace-aikacen yanar gizo.

Koyaya, da zarar ka fara aiki tare da ƙa'idodin aikace-aikacen da ke amfani da haɗin kai na TCP, kamar bayanan bayanai, gRPC ko WebSockets, sabis ɗin ba su dace ba. Kubernetes baya samar da hanyoyin ciki don daidaita haɗin TCP na dindindin.

Wannan yana nufin dole ne ka rubuta aikace-aikace tare da daidaitawa-gefen abokin ciniki a zuciya.

Tafsirin da ƙungiyar ta shirya Kubernetes aaS daga Mail.ru.

Me kuma za a karanta a kan batun:

  1. Matakai uku na autoscaling a Kubernetes da yadda ake amfani da su yadda ya kamata
  2. Kubernetes a cikin ruhin satar fasaha tare da samfuri don aiwatarwa.
  3. Tashar mu ta Telegram game da canjin dijital.

source: www.habr.com

Add a comment