Namhlanje sipapasha icandelo lesibini lokuguqulelwa kwezinto malunga nendlela iDropbox elungelelanise ngayo ulawulo lohlobo lwemigca yezigidi ezininzi zekhowudi yePython.
β
Uhlobo olusemthethweni lwenkxaso (PEP 484)
Senze imifuniselo yethu yokuqala enzulu nge-mypy eDropbox ngexesha leVeki yokuHacka ngo-2014. IVeki yokuHacka sisiganeko seveki enye esibanjwe yiDropbox. Ngeli xesha, abasebenzi banokusebenza nantoni na abayifunayo! Ezinye zeeprojekthi zetekhnoloji ezidumileyo zeDropbox zaqala kwimisitho efana nale. Ngenxa yolu vavanyo, sigqibe kwelokuba i-mypy ibonakala iyathembisa, nangona iprojekthi ingekakulungeli ukusetyenziswa ngokubanzi.
Ngeli xesha, umbono wokulinganisa iinkqubo zokubonisa uhlobo lwePython wawusemoyeni. Njengoko benditshilo, kuba iPython 3.0 ibinokwenzeka ukuba kusetyenziswe uhlobo lwezichasiselo zemisebenzi, kodwa ezi ibiziintetho nje ezingavakaliyo, ngaphandle kwe-syntax echaziweyo kunye nesemantics. Ngexesha lokwenziwa kwenkqubo, ezi zichasiselo, ubukhulu becala, bezivele zingahoywa. Emva kweVeki yokuHacka, saqala ukusebenza ekumiseleni iisemantics. Lo msebenzi ukhokelele ekuveleni (UGuido van Rossum, uΕukasz Langa kunye nam siye sasebenzisana kolu xwebhu).
Iinjongo zethu zinokujongwa macala omabini. Okokuqala, sasinethemba lokuba yonke i-ecosystem yePython inokusebenzisa indlela eqhelekileyo yokusebenzisa iingcebiso zohlobo (igama elisetyenziswe kwiPython njengelilingana "nohlobo lwezichasiselo"). Oku, xa kujongwa imingcipheko enokwenzeka, kungangcono kunokusebenzisa iindlela ezininzi ezingahambelaniyo. Okwesibini, besifuna ukuxoxa ngokuphandle ngeendlela zohlobo lwenkcazo kunye namalungu amaninzi oluntu lwePython. Lo mnqweno wawulawulwa ngokuyinxenye kukuba asifuni ukubonakala "njengabawexuki" kwiingcamango ezisisiseko zolwimi emehlweni obuninzi beeprogram zePython. Lulwimi oluchwetheziweyo olutshintshatshintshayo, olwaziwa ngokuba "yidada lokuchwetheza". Kuluntu, kwasekuqaleni, isimo sengqondo esikrokrelayo malunga nombono wokuchwetheza okusisigxina asinakunceda kodwa sivele. Kodwa loo mvakalelo yada yaphela emva kokuba kucacile ukuba ukuchwetheza okumileyo akuyi kuba sisinyanzelo (kwaye emva kokuba abantu bebonile ukuba iluncedo ngokwenene).
Uhlobo lwe-syntax yengcebiso eyathi yamkelwa ekugqibeleni yayifana kakhulu nenkxaso ye-mypy ngelo xesha. I-PEP 484 yakhululwa kunye nePython 3.5 ngo-2015. IPython yayingaselulo ulwimi oluchwethezwayo. Ndiyathanda ukucinga ngesi siganeko njengesiganeko esibalulekileyo kwimbali yePython.
Ukuqala kokufuduka
Ekupheleni kuka-2015, iDropbox yenza iqela labantu abathathu ukuba basebenze kwi-mypy. Baquka uGuido van Rossum, uGreg Price kunye noDavid Fisher. Ukususela ngaloo mzuzu, imeko yaqala ukukhula ngokukhawuleza okukhulu. Umqobo wokuqala ekukhuleni kwe-mypy yayikukusebenza. Njengoko ndikhankanyile ngasentla, kwiintsuku zokuqala zeprojekthi ndacinga ngokuguqulela ukuphunyezwa kwe-mypy ku-C, kodwa le ngcamango yagqithwa kuluhlu okwangoku. Besibambekile ekuqhubeni inkqubo sisebenzisa itoliki yeCPython, engakhawulezi ngokwaneleyo kwizixhobo ezifana ne-mypy. (Iprojekthi yePyPy, enye indlela yokuphunyezwa kwePython kunye ne-JIT compiler, ayizange isincede nathi.)
Ngethamsanqa, uphuculo oluthile lwe-algorithmic luze kuncedo lwethu apha. I-"accelerator" yokuqala enamandla yayikukuphunyezwa kokuhlolwa okunyukayo. Ingcamango emva kolu phuculo yayilula: ukuba zonke izixhomekeke kwimodyuli azitshintshanga ukususela kwixesha elidlulileyo le-mypy, ngoko sinokusebenzisa idatha egcinwe ngexesha elidlulileyo ngelixa sisebenza kunye nokuxhomekeka. Besifuna kuphela ukujonga uhlobo kwiifayile ezilungisiweyo nakwiifayile ezixhomekeke kuzo. I-Mypy iye yaya phambili kancinci: ukuba ujongano lwangaphandle lwemodyuli alutshintshanga, i-mypy icinge ukuba ezinye iimodyuli ezingenise ngaphandle le modyuli akufuneki zikhangelwe kwakhona.
Ukukhangela okongeziweyo kusincedile kakhulu xa sichaza amanani amakhulu ekhowudi ekhoyo. Ingongoma kukuba le nkqubo ihlala ibandakanya ukuphindaphindwa okuninzi kwe-mypy njengoko izichasiselo zongezwa ngokuthe ngcembe kwikhowudi kwaye ziphuculwe ngokuthe ngcembe. Ukubaleka kokuqala kwe-mypy kwakusacotha kakhulu kuba yayinokuxhomekeka okuninzi ukujonga. Emva koko, ukuphucula imeko, siphumeze indlela yokugcinwa kwe-caching ekude. Ukuba i-mypy ibhaqa ukuba i-cache yendawo inokwenzeka ukuba iphelelwe lixesha, ikhuphela umfanekiso okhawulezayo we-cache yangoku ye-codebase yonke ukusuka kwindawo yogcino esembindini. Emva koko yenza uqwalaselo lokwandiswa ngokusebenzisa le snapshot. Oku kusithathele elinye inyathelo elikhulu ekwandiseni ukusebenza kwe-mypy.
Eli yayilixesha lokwamkelwa ngokukhawuleza nangendalo kohlobo lokujonga kwiDropbox. Ekupheleni kuka-2016, sele sinamalunga nama-420000 imigca yekhowudi yePython enezihlomelo zohlobo. Abasebenzisi abaninzi babenomdla wokukhangela uhlobo. Amaqela amaninzi ophuhliso asebenzisa iDropbox mypy.
Yonke into yayikhangeleka ilungile ngoko, kodwa kwakusekuninzi ekufuneka sikwenzile. Saqala ukwenza uphando lwangaphakathi lwangaphakathi ngamaxesha athile ukuze sichonge iindawo eziyingxaki zeprojekthi kwaye siqonde ukuba yeyiphi imiba efuna ukusonjululwa kuqala (esi senzo sisasetyenziswa kwinkampani namhlanje). Eyona nto ibalulekileyo, njengoko kwacacayo, yayiyimisebenzi emibini. Okokuqala, sasidinga uhlobo olungaphezulu lokugubungela ikhowudi, okwesibini, sifuna i-mypy ukuba isebenze ngokukhawuleza. Kwakucace gca ukuba umsebenzi wethu wokukhawulezisa i-mypy kwaye uyiphumeze kwiiprojekthi zenkampani wawusekude nokugqiba. Thina, sikuqonda ngokupheleleyo ukubaluleka kwale misebenzi mibini, saqalisa ukuyisombulula.
Imveliso engakumbi!
Iitshekhi ezongeziweyo zenze i-mypy ngokukhawuleza, kodwa isixhobo besingakhawulezi ngokwaneleyo. Iitshekhi ezininzi ezongeziweyo zithathe malunga nomzuzu. Isizathu soku ibiziimpahla ezithunyelwa ngaphandle ngomjikelo. Oku mhlawumbi akuyi kumangalisa nabani na osebenze ngeekhowudi ezinkulu ezibhalwe kwiPython. Sasineseti zamakhulu eemodyuli, nganye kuzo yazingenisa ngaphandle ngokungangqalanga zonke ezinye. Ukuba nayiphi na ifayile kwiluphu yokungenisa iye yatshintshwa, i-mypy kwafuneka iqhubekekise zonke iifayile kuloo loop, kwaye rhoqo naziphi na iimodyuli ezithatha ngaphandle iimodyuli ukusuka kuloo loop. Omnye umjikelo ololo hlobo yayidume kakubi βi-dependency tangleβ eyabangela ingxaki enkulu kwiDropbox. Emva kokuba esi sakhiwo siqulethe iimodyuli ezininzi, ngelixa zingeniswa ngaphandle, ngokuthe ngqo okanye ngokungathanga ngqo, iimvavanyo ezininzi, zazisetyenziselwa ikhowudi yokuvelisa.
Sithathele ingqalelo into enokwenzeka "yokukhulula" ukuxhomekeka kwisetyhula, kodwa besingenazo izixhobo zokukwenza oko. Kwakukho ikhowudi eninzi kakhulu esasingaqhelananga nayo. Ngenxa yoko, siye seza nenye indlela. Sagqiba ekubeni senze umsebenzi we-mypy ngokukhawuleza nangona kukho "i-tangles yokuxhomekeka". Sifezekise le njongo sisebenzisa i-mypy daemon. I-daemon yinkqubo yomncedisi eyenza iimpawu ezimbini ezinomdla. Okokuqala, igcina ulwazi malunga ne-codebase yonke kwimemori. Oku kuthetha ukuba ngalo lonke ixesha uqhuba i-mypy, awunyanzelekanga ukuba ulayishe idatha egciniweyo enxulumene namawaka axhomekeke kumazwe angaphandle. Okwesibini, ngononophelo, kwinqanaba leeyunithi ezincinci zesakhiwo, uhlalutya ukuxhomekeka phakathi kwemisebenzi kunye namanye amaziko. Umzekelo, ukuba umsebenzi foo ibiza umsebenzi bar, ngoko kukho ukuxhomekeka foo ukusuka bar. Xa ifayile itshintsha, i-daemon kuqala, kwindawo yodwa, iqhuba kuphela ifayile etshintshiweyo. Emva koko ijonga iinguqu ezibonakalayo zangaphandle kuloo fayile, ezinje ngokutshintshwa kotyikityo lomsebenzi. I-daemon isebenzisa ulwazi oluneenkcukacha malunga nokurhweba ngaphandle kuphela ukujonga kabini loo misebenzi eneneni isebenzisa umsebenzi olungisiweyo. Ngokuqhelekileyo, ngale ndlela, kufuneka ukhangele imisebenzi embalwa kakhulu.
Ukuphumeza konke oku bekungelula, kuba ukuphunyezwa kwe-mypy yokuqala bekugxile kakhulu ekuqhubeni ifayile enye ngexesha. Kwafuneka sijongane neemeko ezininzi zomda, ukwenzeka kwakufuna ukuhlolwa okuphindaphindiweyo kwiimeko apho kukho into etshintshileyo kwikhowudi. Umzekelo, oku kwenzeka xa iklasi yabelwe iklasi entsha yesiseko. Sakuba senze into ebesiyifuna, siye sakwazi ukunciphisa ixesha lokwenziwa kovavanyo olongezelelekileyo ukuya kwimizuzwana embalwa. Oku kwabonakala kululoyiso olukhulu kuthi.
Imveliso engakumbi!
Kunye ne-caching ekude endixoxe ngayo ngasentla, i-mypy daemon iphantse yazisombulula ngokupheleleyo iingxaki ezivelayo xa umdwelisi wenkqubo ehlala eqhuba ukujonga uhlobo, esenza utshintsho kwinani elincinci leefayile. Nangona kunjalo, ukusebenza kwenkqubo kwimeko yosetyenziso oluncinci bekusekude kakhulu. Ukuqaliswa okucocekileyo kwe-mypy kunokuthatha imizuzu eyi-15. Kwaye oku bekungaphezulu kunokuba besiya kuvuya. Iveki nganye imeko yaba mbi ngakumbi njengoko abadwelisi beprogram baqhubeka bebhala ikhowudi entsha kwaye bongeza izichasiselo kwikhowudi ekhoyo. Abasebenzisi bethu basalambele ukusebenza ngakumbi, kodwa sonwabile ukudibana nabo phakathi.
Sagqiba ekubeni sibuyele kwenye yeembono zangaphambili malunga ne-mypy. Oko kukuthi, ukuguqula ikhowudi yePython ibe yikhowudi yeC. Ukuzama i-Cython (inkqubo ekuvumela ukuba uguqulele ikhowudi ebhalwe kwi-Python kwikhowudi ye-C) ayizange isinike nayiphi na isantya esibonakalayo, ngoko ke siye sagqiba ekubeni sivuselele ingcamango yokubhala umqambi wethu. Ekubeni i-codebase ye-mypy (ebhalwe kwi-Python) sele iqulethe zonke iinkcazo zohlobo oluyimfuneko, sicinge ukuba kuya kuba yinto efanelekileyo ukuzama ukusebenzisa ezi nkcazo ukukhawulezisa inkqubo. Ndakha ngokukhawuleza iprototype ukuvavanya le ngcamango. Ibonise ukonyuka okungaphezulu kwe-10 ekusebenzeni kwii-micro-benchmarks ezahlukeneyo. Ingcamango yethu yayikukuqulunqa iimodyuli zePython kwiimodyuli zeC usebenzisa i-Cython, kunye nokuguqula uhlobo lwezichasiselo kwi-run-time check checks (ngokuqhelekileyo uhlobo lwezichasiselo aluhoywa ngexesha lokuqhuba kwaye lusetyenziswe kuphela kwiinkqubo zokujonga uhlobo). Ngokwenene sasicwangcise ukuguqulela ukuphunyezwa kwe-mypy ukusuka kwiPython ukuya kulwimi olwaluyilelwe ukuba luchwethezwe ngokwestatiki, oluya kujongeka (kwaye, ubukhulu becala, lusebenza) kanye njengePython. (Olu hlobo lokufuduka kolwimi olunqamlezayo luye lwaba yinto yesithethe seprojekthi ye-mypy. Ukuphunyezwa kwe-mypy yasekuqaleni yabhalwa kwi-Alore, emva koko kwakukho i-syntactic hybrid yeJava kunye nePython).
Ukugxila kwi-API yolwandiso lwe-CPython kwakungundoqo ekungaphulukani amandla okulawula iprojekthi. Akuzange kufuneke siphumeze umatshini wenyani okanye nawaphi na amathala eencwadi adingwa yi-mypy. Ukongeza, siya kuba nokufikelela kuyo yonke i-ecosystem yePython kunye nazo zonke izixhobo (ezifana ne-pytest). Oku kuthetha ukuba sinokuqhubeka sisebenzisa ikhowudi yePython etolikileyo ngexesha lophuhliso, okusivumela ukuba siqhubeke sisebenza kunye nomzekelo okhawulezayo wokwenza utshintsho lwekhowudi kunye nokuvavanya, kunokulinda ukuba ikhowudi iqokelele. Kwakubonakala ngathi senza umsebenzi omkhulu wokuhlala kwizitulo ezibini, ngoko kuthetha, kwaye sasiyithanda.
Umqambi, esiwubize ngokuba yi-mypyc (ekubeni isebenzisa i-mypy njengesiphelo sangaphambili sokuhlalutya iintlobo), yajika yaba yiprojekthi ephumelele kakhulu. Lilonke, siphumelele malunga ne-4x yesantya sokubaleka rhoqo kwe-mypy ngaphandle kwe-caching. Ukuphuhlisa ingundoqo yeprojekthi ye-mypyc yathatha iqela elincinci likaMichael Sullivan, u-Ivan Levkivsky, uHugh Hahn, kunye nam malunga neenyanga zekhalenda ze-4. Lo myinge womsebenzi wawumncinci kakhulu kunoko bekuya kufuneka ukuba ubhale kwakhona i-mypy, umzekelo, kwi-C ++ okanye i-Go. Yaye kwafuneka senze iinguqulelo ezimbalwa kakhulu kulo msebenzi kunokuba bekuya kufuneka sizenze xa siphinda siwubhala ngolunye ulwimi. Sikwanethemba lokuba sinokuzisa i-mypyc kwinqanaba apho abanye abadwelisi benkqubo beDropbox banokuyisebenzisa ukuqokelela kunye nokukhawulezisa ikhowudi yabo.
Ukufezekisa eli nqanaba lomsebenzi, kuye kwafuneka sisebenzise izisombululo zobunjineli ezinomdla. Ngaloo ndlela, umqambi unokukhawulezisa imisebenzi emininzi ngokusebenzisa ngokukhawuleza, ukwakhiwa kwe-C. Kwaye umnxeba onjalo ukhawuleza kakhulu kunokubiza umsebenzi otolikiweyo. Eminye imisebenzi, enje ngokhangelo lwesichazi-magama, isabandakanyeka ekusebenziseni iifowuni eziqhelekileyo ze-C-API ezivela kwi-CPython, eziye zakhawuleza ngokukhawuleza xa zihlanganiswa. Siye sakwazi ukuphelisa umthwalo owongezelelweyo kwinkqubo eyenziwe ngokutolika, kodwa oku kule meko kwanika inzuzo encinci kuphela ngokusebenza.
Ukuchonga eyona misebenzi "ecothayo", senze ikhowudi yeprofayili. Sixhobe ngale datha, sizame ukwenza i-tweak mypyc ukuze ivelise ikhowudi yeC ekhawulezayo kwimisebenzi enjalo, okanye sibhale kwakhona ikhowudi yePython ehambelanayo sisebenzisa imisebenzi ekhawulezayo (kwaye ngamanye amaxesha besingenaso isisombululo esaneleyo saloo nto okanye enye ingxaki) . Ukubhala kwakhona ikhowudi yePython kwakudla ngokuba sisisombululo esilula kwingxaki kunokuba umqambi enze inguqu efanayo ngokuzenzekelayo. Ngexesha elide, sasifuna ukwenza ngokuzenzekelayo uninzi lwezi nguqulelo, kodwa ngelo xesha sasigxile ekukhawuleziseni i-mypy ngomzamo omncinci. Kwaye ekuqhubeleni phambili kule njongo, sinqumle iikona ezininzi.
Iza kuqhubekaβ¦
Bafundi abathandekayo! Zazithini iimbono zakho ngeprojekthi ye-mypy xa wawufunda ngobukho bayo?
umthombo: www.habr.com