Hōʻike kēia mea i ka mea hana maʻalahi a me ka wikiwiki loa, ka hana āu e ʻike ai ma KDPV. ʻO ka mea mahalo, ua hoʻolālā ʻia ke koholā e hoʻokipa ʻia ma kahi kikowaena git mamao. Nā kikoʻī ma lalo o ka ʻoki.
Pehea i hoʻololi ai ka mea hana ʻike ʻikepili a Airbnb i koʻu ola
Laki wau i ka hana ʻana i kekahi mau pilikia leʻaleʻa i kaʻu ʻoihana: Ua aʻo au i ka makemakika thread i ka wā e loaʻa ai koʻu kēkelē ma MIT, hana ma nā kumu hoʻohālike a me kahi papahana open source. ma Wayfair, a me ka hoʻokō ʻana i nā hiʻohiʻona ʻimi homepage hou a me nā hoʻomaikaʻi CUPED ma Airbnb. Akā ʻaʻole i nani kēia mau hana a pau - ʻoiaʻiʻo, hoʻohana pinepine wau i ka hapa nui o koʻu manawa e ʻimi, aʻo a hōʻoia i ka ʻikepili. ʻOiai he kūlana mau kēia ma ka hana, ʻaʻole i manaʻo iaʻu he pilikia kēia a hiki i ka Airbnb, kahi i hoʻoponopono ʻia me ka hoʻohana ʻana i kahi mea hana ʻike data - .
Ma hea e loaʻa ai iaʻu {{data}}? ʻIkepili.
He aha ke ʻano o kēia kolamu? ʻIkepili.
Pehea 'o {{metric}} i kēia lā? ʻIkepili.
He aha ke ʻano o ke ola? IN ʻIkepili, paha.
ʻAe, loaʻa iā ʻoe ke kiʻi. ʻO ka ʻimi ʻana i ka ʻikepili a me ka hoʻomaopopo ʻana i ke ʻano o ia mea, pehea i hana ʻia ai, a pehea e hoʻohana ai ia mea a pau he mau minuke, ʻaʻole hola. Hiki iaʻu ke hoʻolilo i koʻu manawa i ka hana ʻana i nā inferences maʻalahi, a i ʻole nā algorithm hou, (... a i ʻole ka pane ʻana i nā nīnau maʻamau e pili ana i ka ʻikepili) ma mua o ka ʻimi ʻana i nā memo, ke kākau ʻana i nā nīnau SQL repetitive, a me ka haʻi ʻana i nā hoa hana ma Slack e hoʻāʻo e hana hou i ka pōʻaiapili, kahi mea ʻē aʻe. ua loaa mua.
He aha ka pilikia?
Ua ʻike au ʻaʻole i loaʻa i ka hapa nui o koʻu mau hoa i kahi mea hana. Makemake nā ʻoihana liʻiliʻi e hāʻawi i nā kumuwaiwai nui i ka hana ʻana a me ka mālama ʻana i kahi hāmeʻa platform e like me Dataportal. A ʻoiai he nui nā haʻina open source i loaʻa, ua hoʻolālā ʻia lākou i ka nui, e hana paʻakikī i ka hoʻonohonoho ʻana a me ka mālama ʻana me ka ʻole o kahi ʻenekini DevOps. No laila ua hoʻoholo wau e hana i kahi mea hou.
ʻO Whale: he mea hana ʻike ʻikepili maʻalahi
A ʻae, ma ka maʻalahi a hiki i ke kumu o ka naʻaupō, ʻo wau ka maʻalahi a hiki i ka naʻaupō. ʻElua wale nō māhele o ke koholā:
- ʻO kahi waihona Python e hōʻiliʻili ana i nā metadata a hoʻohālikelike iā ia ma MarkDown.
- He laina kauoha Rust no ka huli ʻana i kēia ʻikepili.
Mai ka manaʻo o ka ʻōnaehana kūloko no ka mālama ʻana, he nui wale nā faila kikokikona a me kahi papahana e hoʻonui i ka kikokikona. ʻO ia, no laila he mea liʻiliʻi ka hoʻokipa ʻana ma kahi kikowaena git e like me Github. ʻAʻohe ʻōlelo nīnau hou e aʻo ai, ʻaʻohe mea hoʻokele hoʻokele, ʻaʻohe waihona. Ua ʻike ʻia ʻo Git i nā mea a pau, no laila ua manuahi ka hoʻonohonoho ʻana a me ka hui pū ʻana. E nānā pono kākou i ka hana .
Hōʻike piha ʻia ʻo git based GUI
Hoʻolālā ʻia ʻo Whale e holo i ke kai o kahi kikowaena git mamao. He hana maʻamau: wehewehe i kekahi mau pili, kope i ka palapala ʻo Github Actions (a i ʻole e kākau i hoʻokahi no kāu kahua CI/CD i koho ʻia) a loaʻa iā ʻoe kahi hāmeʻa ʻike ʻikepili pūnaewele i ka manawa. Hiki iā ʻoe ke ʻimi, nānā, palapala, a kaʻana like i kāu pāpaʻi ma Github.
ʻO kahi laʻana o kahi papa stub i hana ʻia me ka hoʻohana ʻana i nā Hana Github. Hōʻikeʻike hana piha .
Huli CLI wikiwiki o ka uila o kāu waihona
Noho ʻo Whale a hanu i ka laina kauoha, e hāʻawi ana i nā hulina ikaika, milliseconds ma kāu mau papa. ʻOiai me nā miliona o nā papaʻaina, ua hiki iā mākou ke hana i ke kohola i mea hoʻokani maikaʻi loa ma ka hoʻohana ʻana i kekahi mau mīkini caching akamai, a me ke kūkulu hou ʻana i ke kua i Rust. ʻAʻole ʻoe e ʻike i kahi lag hulina [hello, Google DS].
ʻO ka hōʻike ʻana o ke koholā, e ʻimi i nā papa ʻaina miliona.
Ka helu ʻakomi o nā anana [ma ka beta]
ʻO kekahi o kaʻu mea punahele liʻiliʻi loa ma ke ʻano he ʻepekema data e holo pinepine ana i nā nīnau like e nānā i ka maikaʻi o ka ʻikepili i hoʻohana ʻia. Kākoʻo ʻo Whale i ka hiki ke wehewehe i nā ana ma SQL maʻalahi e hoʻonohonoho ʻia e holo pū me kāu metadata scraping pipelines. E wehewehe i ka poloka metrics ma ka format YAML i loko o ka papa stub, a holo 'akomi 'o Whale ma ka papa kuhikuhi a holo i nā nīnau i pūnana ma loko o nā ana.
```metrics
metric-name:
sql: |
select count(*) from table
```
Hoʻohui pū ʻia me Github, ʻo ia hoʻi, hiki i ke koholā ke lilo i kumu maʻalahi o ka ʻoiaʻiʻo no nā wehewehe metric. Hoʻopaʻa pū ʻo Whale i nā waiwai me kahi hōʻailona manawa ma ka papa kuhikuhi "~/". whale/metrics" inā makemake ʻoe e hana i kekahi ʻano pakuhi a i ʻole ka noiʻi hohonu.
ʻO ka Future
Ma hope o ke kamaʻilio ʻana me nā mea hoʻohana o kā mākou mau mana koholā i hoʻokuʻu mua ʻia, ua ʻike mākou he pono nā kānaka i ka hana ākea. No ke aha he mea paahana huli papa? No ke aha ʻaʻole kahi mea hana huli metric? No ke aha e nānā ʻole ai? No ke aha ʻaʻole kahi mea hana nīnau SQL? ʻOiai ua manaʻo mua ʻia ʻo whale v1 he mea hana hoa CLI maʻalahi Dataportal/Amundsen, ua ulu aʻe ia i kahi papa hana kūʻokoʻa piha, a ke manaʻolana nei mākou e lilo ia i ʻāpana koʻikoʻi o ka mea hana a ka ʻepekema data.
Inā makemake ʻoe e ʻike i ka hoʻomohala ʻana, e hui pū me mākou , wehe Puka ma , a i ʻole e hoʻopili pololei . Loaʻa iā mākou kekahi mau hiʻohiʻona maikaʻi - Jinja templates, bookmarks, search filters, Slack alerts, Jupyter integration, a me kahi CLI dashboard no metrics - akā makemake mākou i kāu hoʻokomo.
hopena
Hoʻokumu ʻia a kākoʻo ʻia ʻo Whale e Dataframe, kahi hoʻomaka i hauʻoli wau i ka hoʻokumu ʻana me nā poʻe ʻē aʻe. ʻOiai ua hoʻolālā ʻia ke koholā no nā ʻepekema data, ua hoʻolālā ʻia ʻo Dataframe no nā hui ʻepekema data. No ka poʻe makemake e launa pū, e ʻoluʻolu , e hoʻohui mākou iā ʻoe i ka papa inoa kali.
A ma ke code promo HABR, hiki iā ʻoe ke loaʻa i kahi 10% hou aʻe i ke kumukūʻai i hōʻike ʻia ma ka hae.
Nā papa hou aʻe
Nā ʻatikala
Source: www.habr.com