ʻOki ʻana i nā kaula: neʻe ʻana mai ka Puppet Enterprise i Ansible Tower. Mahele 1

Ua hōʻemi ka National Environmental Satellite Data Information Service (NESDIS) i kāna mau kumukūʻai hoʻonohonoho hoʻonohonoho no Red Hat Enterprise Linux (RHEL) e 35% ma o ka neʻe ʻana mai Puppet Enterprise i Ansible Tower. Ma kēia wikiō "pehea mākou i hana ai", wehewehe ʻo Michael Rau i ka hihia no kēia neʻe ʻana, kaʻana like ʻana i nā ʻōlelo aʻoaʻo a me nā haʻawina i aʻo ʻia mai ka neʻe ʻana mai kahi SCM i kekahi.

Mai kēia wikiō ʻoe e aʻo ai:

  • pehea e hōʻoia ai i ka hoʻokele ʻana i ka hiki ke hoʻololi mai ka Puppet Enterprise i Ansible Tower;
  • he aha nā hoʻolālā e hoʻohana ai e hoʻololi i ka hoʻololi e like me ka hiki;
  • nā ʻōlelo aʻoaʻo no ka unuhi ʻana i ka PE i loko o ka Ansible Playbook;
  • Manaʻo no ka hoʻokomo pono ʻana i ka hale kiaʻi Ansible.

ʻOki ʻana i nā kaula: neʻe ʻana mai ka Puppet Enterprise i Ansible Tower. Mahele 1

Aloha ʻoukou a pau, ʻo Michael Rau koʻu inoa, he Senior Systems Engineer wau ma ActioNet, e hana ana no ka lawelawe ʻo National Oceanic and Atmospheric Administration (NOAA) NESDIS. I kēia lā e kamaʻilio mākou e pili ana i ka ʻoki kaula - koʻu ʻike ponoʻī o ka neʻe ʻana mai ka Puppet Enterprise i Ansible Tower. ʻO ke kumumanaʻo o kēia hōʻike ʻana ʻo ia ka "nānā i koʻu mau ʻōpala" i waiho ʻia ma hope o koʻu hana ʻana i kēia hoʻololi ma mua o ka makahiki. Makemake au e kaʻana like i nā mea aʻu i aʻo ai ma kēia kaʻina hana. No laila ke lawe ʻoe i kahi mea e like me kēia, me ka hoʻohana ʻana i kaʻu ʻike, hiki iā ʻoe ke hana i ka hoʻololi me ka ʻole o ka hana hou.

ʻIke ʻoe i nā paheʻe like me kēia ma ka hoʻomaka ʻana o kēlā me kēia hōʻike ma Ansible Fest. Hōʻike kēia paheʻe i ka mōʻaukala o ka automation o kaʻu hui. ʻAʻole wau he mea hou i kēia no ka mea ua hoʻohana wau i ka Puppet/Puppet Enterprise mai ka makahiki 2007. Ua hoʻomaka wau e hana pū me Ansible ma 2016, a e like me nā mea hoʻohana ʻē aʻe o kēia huahana, ua hoihoi au i ka hiki ke "hoʻopunipuni" me ka hoʻohana ʻana i ka laina kauoha a me nā palapala maʻalahi (nā puke pāʻani). I ka hopena o 2017, ua hoʻokokoke au i koʻu hoʻokele e pili ana i nā kumu ikaika no ka neʻe ʻana i Ansible Tower. Ma kahi minuke e haʻi wau iā ʻoe e pili ana i nā kumu i hoʻoikaika iaʻu e hana i kēia hana. Ma hope o ka loaʻa ʻana o ka ʻae o ka hoʻokele, ua hala kekahi mau mahina hou e hoʻopau ai i ka hoʻolālā, a ua hoʻololi au i Ianuali-Feberuari o kēia makahiki. No laila, ua haʻalele loa mākou i ka Puppet no ka Ansible, a he mea nui ia.

ʻOki ʻana i nā kaula: neʻe ʻana mai ka Puppet Enterprise i Ansible Tower. Mahele 1

ʻO ka mea hoihoi loa iaʻu e pili ana iā Ansible ʻo ka hiki ke kākau a hoʻohana i nā kuleana a me nā puke pāʻani. He maikaʻi nā kuleana no ka hana ʻana i nā hana ʻokoʻa akā pili a hoʻokomo i nā ʻikepili āpau e pili ana i kēlā mau hana ma kahi hoʻokahi. ʻO kahi puke pāʻani he syntax YAML, faila palapala e wehewehe ana i nā hana no hoʻokahi a ʻoi aku paha nā pūʻali. Ke haʻi aku nei au i nā mea hoʻohana e pili ana i kēia mau hiʻohiʻona, ma mua o nā mea hoʻomohala polokalamu. Hāʻawi ʻo Ansible Tower iā ʻoe i ka hiki ke ʻōlelo, "ʻaʻole, ʻaʻole ʻoe e komo i ka shell, akā hāʻawi wau iā ʻoe i ka hiki ke holo i nā kaʻina hana a pau a hoʻomaka hou i ka lawelawe inā pono ʻoe." E haʻi aku wau iā ʻoe e pili ana i ke kaiapuni hana a me nā mea hana a mākou e hoʻohana ai.

ʻOki ʻana i nā kaula: neʻe ʻana mai ka Puppet Enterprise i Ansible Tower. Mahele 1

He LAN federal kēia, 7 mau pūnaewele kino i hoʻopili ʻia ma ke ao MPLS, 140 RHEL mau kikowaena, 99% o ia mau mea virtual (vSphere), SuperMicro hardware, NexentaStore mālama pūnaewele, kahi hoʻonohonoho o Cisco, Arista a me Cumulus hoʻololi a me Fortinet UTM hui hoʻoweliweli hoʻokele. nā mea hana ma kēlā me kēia pūnaewele.

ʻO ke ʻano o ka pūnaewele federal e pono iaʻu e hoʻohana i nā hana palekana ʻike āpau i hāʻawi ʻia e ke kānāwai. Pono ʻoe e hoʻomanaʻo ʻaʻole kākoʻo ʻo Puppet Enterprise i ka hapa nui o nā hāmeʻa a mākou e hoʻohana ai. Ke koi ʻia nei mākou e hoʻohana i nā lako kālā no ka mea he pilikia nā keʻena aupuni i ke kālā ʻana i kēia mea lilo. ʻO ia ke kumu e kūʻai ai mākou i ka lako SuperMicro a hōʻuluʻulu i kā mākou mau mea hana mai nā ʻāpana pākahi, ʻo ka mālama ʻana i hōʻoia ʻia e nā ʻaelike aupuni. Hoʻohana mākou i Linux a ʻo kēia kekahi o nā kumu nui no ka hoʻololi ʻana iā Ansible.

ʻO kā mākou moʻolelo me ka Puppet penei.

ʻOki ʻana i nā kaula: neʻe ʻana mai ka Puppet Enterprise i Ansible Tower. Mahele 1

I ka makahiki 2007, loaʻa iā mākou kahi pūnaewele liʻiliʻi o 20-25 nodes, kahi a mākou i kau ai i ka Puppet. ʻO ke kumu, ʻo kēia mau nodes he "pahu" RedHat wale nō. I ka makahiki 2010, ua hoʻomaka mākou e hoʻohana i ka pūnaewele Puppet Dashboard no 45 mau nodes. Ke hoʻomau nei ka hoʻonui ʻana o ka pūnaewele, ua neʻe mākou i ka PE 2014 i ka makahiki 3.3, e hana ana i kahi hoʻololi piha me kahi kākau hou ʻana no 75 nodes. Pono kēia e hana no ka mea makemake ʻo Puppet e hoʻololi i nā lula o ka pāʻani, a ma kēia hihia ua hoʻololi loa lākou i ka ʻōlelo. I hoʻokahi makahiki ma hope mai, i ka pau ʻana o ke kākoʻo no ka mana 3 o ka Puppet Enterprise, ua koi ʻia mākou e neʻe i PE 2015.2. Pono mākou e kākau hou i ka hōʻike no nā kikowaena hou a kūʻai i kahi laikini me kahi mālama o 100 nodes, ʻoiai i kēlā manawa he 85 wale nō kā mākou nodes.

2 makahiki wale nō i hala, a ua hana hou mākou i nā hana he nui e neʻe i ka mana hou PE 2016.4. Ua kūʻai mākou i laikini no 300 nodes, loaʻa iā 130 wale nō. Pono mākou e hana hou i nā hoʻololi nui i ka hōʻike no ka mea he ʻano syntax ʻokoʻa ka mana hou o ka ʻōlelo ma mua o ka ʻōlelo o ka mana 2015. ʻO ka hopena, ua hoʻololi kā mākou SCM mai ka mana mana SVN i Bitbucket (Git). ʻO kēia kā mākou "pili" me ka Puppet.

No laila, pono wau e wehewehe i ka hoʻokele no ke aha mākou e neʻe ai i kahi SCM ʻē aʻe me ka hoʻohana ʻana i kēia mau manaʻo. ʻO ka mua ke kumukūʻai kiʻekiʻe o ka lawelawe. Ua kamaʻilio wau me nā poʻe ma RedHat a ʻōlelo lākou ʻo ke kumukūʻai o ka holo ʻana i kahi pūnaewele node 300 me Ansible Tower ka hapalua o ke kumukūʻai o ka Puppet Enterprise. Inā kūʻai ʻoe i Ansible Engine, e like ke kumukūʻai, akā e loaʻa iā ʻoe nā hiʻohiʻona he nui ma mua o PE. No ka mea he ʻoihana mokuʻāina mākou i kālā ʻia mai ka waihona kālā federal, he manaʻo ikaika loa kēia.

ʻOki ʻana i nā kaula: neʻe ʻana mai ka Puppet Enterprise i Ansible Tower. Mahele 1

ʻO ka pane ʻelua he versatility. Kākoʻo ʻo Puppet i nā lako lako i loaʻa kahi ʻelele Puppet. 'O ia ho'i, pono e ho'okomo 'ia kekahi 'elele ma nā ho'ololi a pau, a 'o ia ka mana hou loa. A inā kākoʻo kekahi o kāu mau hoʻololi i hoʻokahi mana, a kākoʻo kekahi i kekahi, pono ʻoe e hoʻokomo i kahi mana hou o ka ʻelele PE ma luna o lākou i hiki iā lākou ke hana i ka ʻōnaehana SCM hoʻokahi.

He ʻokoʻa ka hana ʻana o ka ʻōnaehana Ansible Tower no ka mea ʻaʻohe ona mea hana, akā aia nā modules e kākoʻo i nā hoʻololi Cisco a me nā hoʻololi ʻē aʻe. Kākoʻo kēia SCM iā Qubes OS, Linux a me 4.NET UTM. Kākoʻo pū ʻo Ansible Tower i nā mea mālama mālama pūnaewele ʻo NexentaStore e pili ana i ka kernel Illumos, kahi ʻōnaehana hoʻohana Unix i wehe ʻia. He kākoʻo liʻiliʻi loa kēia, akā hana ʻo Ansible Tower.

ʻO ke kolu o ka hoʻopaʻapaʻa, he mea koʻikoʻi ia noʻu a no kā mākou hoʻokele, ʻo ia ka maʻalahi o ka hoʻohana. Ua hoʻohana au i 10 mau makahiki i ka haku ʻana i nā modules Puppet a me ka code manifest, akā ua aʻo au iā Ansible i loko o hoʻokahi pule no ka mea ʻoi aku ka maʻalahi o kēia SCM e hana pū me. Inā holo ʻoe i nā faila hiki ke hoʻokō ʻia, ʻoiaʻiʻo, ke ʻole ʻoe e hana pono ʻole, a laila hana pū nā mea lawelawe akamai a pane aku iā lākou. He maʻalahi ke aʻo a me ka wikiwiki o nā puke pāʻani ma YAML. ʻO ka poʻe i lohe ʻole iā YAML ma mua, hiki ke heluhelu wale i nā palapala a hoʻomaopopo maʻalahi i ka hana ʻana.

ʻO kaʻoiaʻiʻo, hana ʻo Puppet i kāu hana ma ke ʻano he mea hoʻomohala i ʻoi aku ka paʻakikī no ka mea e pili ana i ka hoʻohana ʻana i ka Puppet Master. ʻO ia wale nō ka mīkini i ʻae ʻia e kamaʻilio me nā ʻelele Puppet. Inā ua hoʻololi ʻoe i ka hōʻike a makemake ʻoe e hoʻāʻo i kāu code, pono ʻoe e kākau hou i ke code no ka Puppet Master, ʻo ia hoʻi, hoʻonohonoho i ka faila Puppet Master /etc/hosts e hoʻopili i nā mea kūʻai aku a hoʻomaka i ka lawelawe Puppet Server. Ma hope wale nō e hiki ai iā ʻoe ke hoʻāʻo i ka hana o nā lako pūnaewele ma kahi hoʻokipa. He hana ʻeha kēia.
ʻOi aku ka maʻalahi o nā mea a pau ma Ansible. ʻO nā mea a pau āu e hana ai, ʻo ka hoʻomohala ʻana i ke code no kahi mīkini hiki ke kamaʻilio ma o SSH me ka mea hoʻokipa ma lalo o ka hoʻāʻo. ʻOi aku ka maʻalahi o kēia hana.

ʻO ka pōmaikaʻi nui aʻe o Ansible Tower ka hiki ke hoʻohana i kāu ʻōnaehana kākoʻo i kēia manawa a mālama i kāu hoʻonohonoho lako lako. Ke hoʻohana nei kēia SCM i nā ʻike āpau e pili ana i kāu ʻenehana a me kāu lako, nā mīkini virtual, nā kikowaena, a me nā mea ʻē aʻe me ka ʻole o nā hana hou. Hiki iā ia ke kamaʻilio me kāu mau kikowaena RH Satellite, inā loaʻa iā ʻoe, a hāʻawi iā ʻoe i nā hoʻohui ʻaʻole loa e loaʻa iā ʻoe me ka Puppet.

ʻO kekahi mea nui ka mana kikoʻī. ʻIke ʻoe he ʻōnaehana modular ʻo Puppet, he noi mea kūʻai aku-server, no laila pono ʻoe e wehewehe i nā ʻano o kāu mau mīkini āpau i kahi hōʻike lōʻihi. I kēia hihia, pono e hoʻāʻo ʻia ke kūlana o kēlā me kēia mea o ka ʻōnaehana i kēlā me kēia hapalua hola - ʻo ia ka manawa paʻamau. Penei ka hana o ka Puppet.

Mālama ʻo Tower iā ʻoe mai kēlā. Hiki iā ʻoe ke holo i nā kaʻina hana like ʻole ma nā mea hana like ʻole me ka ʻole o ka palena; hiki iā ʻoe ke hana i nā hana maʻamau, holo i nā kaʻina hana koʻikoʻi ʻē aʻe, hoʻonohonoho i kahi ʻōnaehana palekana, a hana pū me nā ʻikepili. Hiki iā ʻoe ke hana i nā mea paʻakikī i ka Puppet Enterprise. No laila, inā ʻoe i hoʻonohonoho iā ia ma hoʻokahi host, e lōʻihi ka manawa no ka hoʻololi ʻana i ka hopena i nā mea hoʻokipa i koe. Ma Ansible, hoʻololi nā hoʻololi a pau i ka manawa like.

ʻO ka hope, e nānā kākou i ka module palekana. Hoʻokomo ʻo Ansible Tower iā ia me ka kupaianaha, me ka pololei a me ka mālama. Hiki iā ʻoe ke hāʻawi i nā mea hoʻohana i ke komo ʻana i nā lawelawe kikoʻī a i ʻole i nā pūʻali kikoʻī. Hana wau i kēia me kaʻu mau limahana i maʻa i ka hana ma Windows, e kaupalena ana i ko lākou komo ʻana i ka shell Linux. Ke hōʻoia nei au e loaʻa iā lākou ka hale kiaʻi i hiki iā lākou ke hana wale i ka hana a holo wale i nā lawelawe e pili ana iā lākou.

ʻOki ʻana i nā kaula: neʻe ʻana mai ka Puppet Enterprise i Ansible Tower. Mahele 1

E nānā kākou i nā mea āu e hana ai ma mua o ka hoʻololi ʻana i ka hale kiaʻi ʻo Ansible. ʻO ka mea mua, pono ʻoe e hoʻomākaukau i kāu mau mea hana. Inā ʻaʻole i loko o ka waihona ʻikepili kekahi mau mea o kāu ʻoihana, pono ʻoe e hoʻohui iā lākou ma laila. Aia nā ʻōnaehana ʻaʻole e hoʻololi i ko lākou mau hiʻohiʻona a no laila ʻaʻole i loko o ka waihona Puppet, akā inā ʻaʻole ʻoe e hoʻohui iā lākou ma mua o ka neʻe ʻana i Tower, e nalowale ana ʻoe i kekahi mau pono. He ʻikepili mua paha kēia, akā pono e loaʻa ka ʻike e pili ana i nā lako a pau āu. No laila, pono ʻoe e kākau i kahi palapala ʻenehana ikaika e hoʻohuli maʻalahi i nā hoʻololi ʻenehana āpau i loko o ka waihona, a laila ʻike ʻo Ansible i nā mea hoʻokipa e noho ma ka ʻōnaehana hou. ʻAʻole pono ʻoe e haʻi i kēia SCM i nā mea hoʻokipa āu i hoʻohui ai a ʻaʻole i noho hou nā pūʻali, no ka mea, e ʻike maʻalahi ia i kēia. ʻOi aku ka nui o ka ʻikepili i loko o ka waihona, ʻoi aku ka maikaʻi a me ka maʻalahi o Ansible. He hana ia me he mea lā e heluhelu wale ana i ka barcode status hardware mai kahi waihona.

E hoʻolōʻihi i ka manawa e kamaʻāina ai i ka laina kauoha ma Ansible. E holo i kekahi mau kauoha maʻamau no ka hoʻāʻo ʻana i ka palapala ʻenehana, e kākau a holo i kekahi mau palapala pāʻani maʻalahi akā pono, e hoʻohana i nā maʻamau Jinja2 inā kūpono. E ho'āʻo e kākau i kahi kuleana a me ka palapala no kahi kaʻina hana paʻakikī a me nā ʻanuʻu he nui me ka hoʻohana ʻana i ka hoʻonohonoho ʻenehana maʻamau. E pāʻani me kēia mau mea, e hoʻāʻo pehea e hana ai. Ma kēia ala ʻoe e aʻo ai pehea e hoʻohana ai i nā mea hana hale waihona puke i hoʻohana ʻia ma Tower. Ua ʻōlelo mua wau ua lawe iaʻu ma kahi o 3 mau mahina e hoʻomākaukau ai no ka hoʻololi. Manaʻo wau e pili ana i kaʻu ʻike, hiki iā ʻoe ke hana wikiwiki. Mai noʻonoʻo ʻoe i kēia manawa, no ka mea ma hope e ʻike ʻoe i nā pono āpau o ka hana i hana ʻia.

A laila, pono ʻoe e hoʻoholo i ka mea āu e manaʻo ai mai Ansible Tower, he aha ka mea e hana ai kēia ʻōnaehana iā ʻoe.

ʻOki ʻana i nā kaula: neʻe ʻana mai ka Puppet Enterprise i Ansible Tower. Mahele 1

Pono ʻoe e hoʻokau i ka ʻōnaehana ma nā lako hana ʻole, ma nā mīkini virtual kūlohelohe? A i ʻole makemake ʻoe e mālama i nā kūlana hana mua a me nā hoʻonohonoho o nā mea hana e kū nei? He mea koʻikoʻi kēia no nā hui aupuni, no laila pono ʻoe e hōʻoia e hiki iā ʻoe ke neʻe a hoʻokau iā Ansible ma kāu hoʻonohonoho ʻana. E ʻike i nā kaʻina hana hoʻokele maʻamau āu e makemake ai e hoʻokaʻawale. E ʻike inā pono ʻoe e kau i nā noi a me nā lawelawe kikoʻī ma ka ʻōnaehana hou. E hana i kahi papa inoa o nā mea āu e makemake ai e hana a hoʻonohonoho mua i ia.

A laila hoʻomaka e kākau i ka code script a me nā kuleana e hiki ai i nā hana āu e manaʻo ai e hoʻopau. E hoʻohui iā lākou i Projects, kahi hōʻiliʻili kūpono o nā puke pāʻani kūpono. E pili ana kēlā me kēia Papahana i kahi waihona ʻokoʻa ʻo Git a i ʻole kahi waihona ʻokoʻa e pili ana i ka mana code āu e hoʻohana ai. Hiki iā ʻoe ke hoʻokele i nā palapala pāʻani a me nā papa kuhikuhi puke pāʻani ma ke kau lima ʻana iā lākou ma ke Alanui Project Base ma ke kikowaena Tower, a i ʻole ma ke kau ʻana i ka playbook i nā ʻōnaehana kumu kumu (SCM) i kākoʻo ʻia e Tower, me Git, Subversion, Mercurial, a me Red Hat Nā ʻike. I loko o hoʻokahi Pāhana hiki iā ʻoe ke kau i nā palapala he nui āu e makemake ai. No ka laʻana, ua hana au i hoʻokahi Papahana maʻamau kahi aʻu i kau ai i kahi palapala no nā mea kumu RedHat, kahi palapala no ka Linux core, a me nā palapala no ke koena o nā baselines. No laila, i loko o hoʻokahi papahana aia nā ʻano hana a me nā hiʻohiʻona i mālama ʻia mai kahi waihona Git.

ʻO ka holo ʻana i kēia mau mea āpau ma o ka laina kauoha he ala maikaʻi ia e hoʻāʻo ai i kā lākou hana. E hoʻomākaukau kēia iā ʻoe no ka hoʻokomo ʻana i ka hale kiaʻi.

E kamaʻilio liʻiliʻi kāu e pili ana i ka hoʻololi ʻana i ka hōʻike Puppet, no ka mea, ua lōʻihi koʻu manawa ma kēia a hiki i koʻu ʻike ʻana i ka mea pono e hana ʻia.

ʻOki ʻana i nā kaula: neʻe ʻana mai ka Puppet Enterprise i Ansible Tower. Mahele 1

E like me kaʻu i ʻōlelo ai ma mua, mālama ʻo Puppet i nā hoʻonohonoho āpau a me nā koho hāmeʻa i hoʻokahi hōʻike lōʻihi, a mālama kēia hōʻike i nā mea āpau e hana ai kēia SCM. I ka hana ʻana i ka hoʻololi, ʻaʻole pono ʻoe e hoʻopili i kāu mau hana āpau i kahi papa inoa hoʻokahi; akā, e noʻonoʻo e pili ana i ke ʻano o ka ʻōnaehana hou: nā kuleana, nā palapala, nā hōʻailona, ​​​​nā hui a me nā mea e hele ai i laila. Pono e hui pū ʻia kekahi o nā ʻenehana pūnaewele autonomous i mau pūʻulu i hiki ke hana ʻia nā palapala. Hiki ke hoʻohui ʻia i nā ʻano hana paʻakikī e pili ana i ka nui o nā kumuwaiwai, me nā papa paʻa ponoʻī. Ma mua o ka neʻe ʻana, pono ʻoe e hoʻoholo i kēia. Inā ʻoe e hana nei i nā kuleana nui a i ʻole nā ​​hiʻohiʻona i kūpono ʻole ma ka pale hoʻokahi, pono ʻoe e hoʻohana i nā hōʻailona e hiki ke hopu i nā ʻāpana kikoʻī o ka ʻōnaehana.

18:00

ʻOki ʻana i nā kaula: neʻe ʻana mai ka Puppet Enterprise i Ansible Tower. Mahele 2

Kekahi mau hoʻolaha 🙂

Mahalo no kou noho pū ʻana me mākou. Makemake ʻoe i kā mākou ʻatikala? Makemake ʻoe e ʻike i nā mea hoihoi hou aʻe? E kākoʻo iā mākou ma ke kau ʻana i kahi kauoha a i ʻole ka ʻōlelo ʻana i nā hoaaloha, cloud VPS no nā mea hoʻomohala mai $4.99, he analogue kū hoʻokahi o nā kikowaena hoʻokomo-level, i hana ʻia e mākou no ʻoe: ʻO ka ʻoiaʻiʻo holoʻokoʻa e pili ana iā VPS (KVM) E5-2697 v3 (6 Cores) 10GB DDR4 480GB SSD 1Gbps mai $ 19 a pehea e kaʻana like ai i kahi kikowaena? (loaʻa me RAID1 a me RAID10, a hiki i 24 cores a hiki i 40GB DDR4).

ʻO Dell R730xd 2 mau manawa maʻalahi ma Equinix Tier IV kikowaena data ma Amsterdam? Eia wale nō 2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100 TV mai $199 ma Netherlands! Dell R420 - 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB - mai $99! Heluhelu e pili ana Pehea e kūkulu ai i ka ʻoihana ʻoihana. papa me ka hoʻohana 'ana o Dell R730xd E5-2650 v4 kikowaena waiwai 9000 euros no ka peni?

Source: www.habr.com

Pākuʻi i ka manaʻo hoʻopuka