{"id":38479,"date":"2019-10-31T22:23:56","date_gmt":"2019-10-31T19:23:56","guid":{"rendered":"https:\/\/prohoster.info\/blog\/put-k-proverke-tipov-4-millionov-strok-python-koda-chast-2\/"},"modified":"2019-10-31T22:23:56","modified_gmt":"2019-10-31T19:23:56","slug":"put-k-proverke-tipov-4-millionov-strok-python-koda-chast-2","status":"publish","type":"post","link":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/put-k-proverke-tipov-4-millionov-strok-python-koda-chast-2","title":{"rendered":"Cesta k ov\u011b\u0159ov\u00e1n\u00ed typ\u016f 4 milion\u016f \u0159\u00e1dk\u016f k\u00f3du v Pythonu. \u010c\u00e1st 2","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Dnes publikujeme druhou \u010d\u00e1st p\u0159ekladu materi\u00e1lu o tom, jak v Dropboxu organizovali kontrolu typ\u016f n\u011bkolika milion\u016f \u0159\u00e1dk\u016f Python k\u00f3du.<\/p>\n

\"Cesta<\/a><\/noindex><\/p>\n

\u2192 \u010c\u00edst prvn\u00ed \u010d\u00e1st<\/a><\/noindex>
\n<\/a><\/noindex><\/p>\n

Ofici\u00e1ln\u00ed podpora typ\u016f (PEP 484)<\/h2>\n

\nNa Hack Weeku 2014 jsme provedli na\u0161e prvn\u00ed v\u00e1\u017en\u00e9 experimenty s mypy v Dropboxu. Hack Week je akce, kterou Dropbox po\u0159\u00e1d\u00e1 jednou za rok. B\u011bhem tohoto t\u00fddne maj\u00ed zam\u011bstnanci mo\u017enost pracovat na \u010demkoli! N\u011bkter\u00e9 z nejzn\u00e1m\u011bj\u0161\u00edch technick\u00fdch projekt\u016f Dropboxu za\u010daly pr\u00e1v\u011b na t\u011bchto akc\u00edch. Z t\u011bchto experiment\u016f jsme dosp\u011bli k z\u00e1v\u011bru, \u017ee mypy vypad\u00e1 slibn\u011b, a\u010dkoliv tento projekt je\u0161t\u011b nebyl p\u0159ipraven k \u0161irok\u00e9mu pou\u017eit\u00ed.<\/p>\n

V t\u00e9 dob\u011b se ve vzduchu vzn\u00e1\u0161ela my\u0161lenka na standardizaci syst\u00e9m\u016f pro poskytov\u00e1n\u00ed typov\u00fdch n\u00e1pov\u011bd v Pythonu. Jak jsem ji\u017e zm\u00ednil, od Pythonu 3.0 bylo mo\u017en\u00e9 pou\u017e\u00edvat typov\u00e9 anotace pro funkce, ale \u0161lo pouze o libovoln\u00e9 v\u00fdrazy, bez definovan\u00e9ho syntaxe a s\u00e9mantiky. B\u011bhem prov\u00e1d\u011bn\u00ed programu byli v\u011bt\u0161inou tyto anotace jednodu\u0161e ignorov\u00e1ny. Po Hack Weeku jsme za\u010dali pracovat na standardizaci s\u00e9mantiky. Tato pr\u00e1ce vedla k p\u0159ijet\u00ed PEP 484<\/a><\/noindex> (na tomto dokumentu se pod\u00edleli Guido van Rossum, Luk\u00e1\u0161 Langa a j\u00e1).<\/p>\n

Na\u0161e motivy lze posuzovat ze dvou stran. Za prv\u00e9, doufali jsme, \u017ee cel\u00e1 ekosyst\u00e9m Pythonu by mohl p\u0159istoupit k jednotn\u00e9mu p\u0159\u00edstupu k pou\u017e\u00edv\u00e1n\u00ed typov\u00fdch n\u00e1pov\u011bd (type hints \u2014 term\u00edn pou\u017eit\u00e9 v Pythonu jako analog \u201etypov\u00fdch anotac\u00ed\u201c). To by, s ohledem na mo\u017en\u00e9 rizika, bylo lep\u0161\u00ed ne\u017e pou\u017e\u00edv\u00e1n\u00ed mnoha vz\u00e1jemn\u011b neslu\u010diteln\u00fdch p\u0159\u00edstup\u016f. Za druh\u00e9, cht\u011bli jsme otev\u0159en\u011b diskutovat o mechanismech typov\u00e9ho anotov\u00e1n\u00ed s mnoha z\u00e1stupci komunity Python. \u010c\u00e1ste\u010dn\u011b bylo toto p\u0159\u00e1n\u00ed motivov\u00e1no t\u00edm, \u017ee jsme necht\u011bli vypadat jako \u201eodpadl\u00edci\u201c od z\u00e1kladn\u00edch my\u0161lenek jazyka v o\u010d\u00edch \u0161irok\u00e9 ve\u0159ejnosti Python program\u00e1tor\u016f. Je to dynamicky typovan\u00fd jazyk, zn\u00e1m\u00fd \u201ekachen\u00edm typizace\u201c. V komunit\u011b, na za\u010d\u00e1tku, nemohlo nevzniknout n\u011bkolika podez\u0159el\u00e9 n\u00e1zory na my\u0161lenku statick\u00e9 typizace. Ale tento postoj nakonec oslabil \u2014 pot\u00e9, co bylo jasn\u00e9, \u017ee statick\u00e1 typizace nebude povinn\u00e1 (a pot\u00e9, co lid\u00e9 pochopili, \u017ee je skute\u010dn\u011b u\u017eite\u010dn\u00e1).<\/p>\n

P\u0159ijat\u00e1 syntaxe typov\u00fdch n\u00e1vrh\u016f byla velmi podobn\u00e1 t\u00e9, kterou tehdy podporoval mypy. Dokument PEP 484 byl vyd\u00e1n spolu s Pythonem 3.5 v roce 2015. Python ji\u017e nebyl jazykem, kter\u00fd podporoval pouze dynamick\u00e9 typov\u00e1n\u00ed. R\u00e1d o t\u00e9to ud\u00e1losti p\u0159em\u00fd\u0161l\u00edm jako o v\u00fdznamn\u00e9m miln\u00edku v historii Pythonu.<\/p>\n

Za\u010d\u00e1tek migrace<\/h2>\n

\nNa konci roku 2015 byla v Dropboxu pro pr\u00e1ci na mypy vytvo\u0159ena t\u00fdm t\u0159\u00ed lid\u00ed. Pat\u0159ili sem Guido van Rossum, Greg Price a David Fisher. Od tohoto okam\u017eiku se situace za\u010dala velmi rychle vyv\u00edjet. Prvn\u00edm probl\u00e9mem na cest\u011b za r\u016fstem mypy byla v\u00fdkonnost. Jak jsem ji\u017e nazna\u010dil v\u00fd\u0161e, v ran\u00e9 f\u00e1zi v\u00fdvoje projektu jsem p\u0159em\u00fd\u0161lel o p\u0159ekladu implementace mypy do jazyka C, ale tato my\u0161lenka byla prozat\u00edm vy\u0159azena. Z\u016fstali jsme na tom, \u017ee k spu\u0161t\u011bn\u00ed syst\u00e9mu byl pou\u017eit interpretr CPython, kter\u00fd nem\u00e1 dostate\u010dnou rychlost pro n\u00e1stroje jako mypy. (Projekt PyPy, alternativn\u00ed implementace Pythonu s JIT kompil\u00e1torem, n\u00e1m tak\u00e9 nepomohl.)<\/p>\n

Na\u0161t\u011bst\u00ed n\u00e1m sem p\u0159i\u0161ly na pomoc n\u011bkter\u00e9 algoritmick\u00e9 vylep\u0161en\u00ed. Prvn\u00edm mocn\u00fdm \"zrychlova\u010dem\" byla implementace inkrement\u00e1ln\u00ed kontroly. My\u0161lenka tohoto vylep\u0161en\u00ed byla jednoduch\u00e1: pokud se v\u0161echny z\u00e1vislosti modulu od posledn\u00edho spu\u0161t\u011bn\u00ed mypy nezm\u011bnily, m\u016f\u017eeme b\u011bhem pr\u00e1ce se z\u00e1vislostmi pou\u017e\u00edt data, kter\u00e1 byla ukl\u00e1d\u00e1na b\u011bhem p\u0159edchoz\u00edho sezen\u00ed. Museli jsme pouze prov\u00e9st kontrolu typ\u016f ve zm\u011bn\u011bn\u00fdch souborech a v t\u011bch souborech, kter\u00e9 na n\u011b z\u00e1visely. Mypy \u0161el dokonce o n\u011bco d\u00e1l: pokud se extern\u00ed rozhran\u00ed modulu nezm\u011bnilo \u2013 mypy pova\u017eoval, \u017ee nen\u00ed nutn\u00e9 znovu kontrolovat jin\u00e9 moduly, kter\u00e9 tento modul importuj\u00ed.<\/p>\n

Inkrement\u00e1ln\u00ed kontrola n\u00e1m velmi pomohla p\u0159i anotaci velk\u00fdch objem\u016f existuj\u00edc\u00edho k\u00f3du. Tento proces obvykle zahrnuje mnoho iterativn\u00edch spu\u0161t\u011bn\u00ed mypy, proto\u017ee anotace se do k\u00f3du postupn\u011b p\u0159id\u00e1vaj\u00ed a zlep\u0161uj\u00ed. Prvn\u00ed spu\u0161t\u011bn\u00ed mypy bylo st\u00e1le velmi pomal\u00e9, proto\u017ee b\u011bhem jeho prov\u00e1d\u011bn\u00ed bylo nutn\u00e9 zkontrolovat mnoho z\u00e1vislost\u00ed. Abychom situaci zlep\u0161ili, implementovali jsme mechanismus d\u00e1lkov\u00e9ho caching. Pokud mypy zjist\u00ed, \u017ee m\u00edstn\u00ed cache je pravd\u011bpodobn\u011b zastaral\u00e1, na\u010dte aktu\u00e1ln\u00ed snapshot cache pro cel\u00fd k\u00f3d ze centralizovan\u00e9ho repozit\u00e1\u0159e. Pot\u00e9 provede inkrement\u00e1ln\u00ed kontrolu s pou\u017eit\u00edm tohoto snapshotu. To n\u00e1s posunulo zase o v\u00fdznamn\u00fd krok vp\u0159ed k zv\u00fd\u0161en\u00ed v\u00fdkonu mypy.<\/p>\n

Bylo to obdob\u00ed rychl\u00e9ho a p\u0159irozen\u00e9ho zav\u00e1d\u011bn\u00ed syst\u00e9mu kontroly typ\u016f v Dropboxu. Na konci roku 2016 jsme ji\u017e m\u011bli zhruba 420000 \u0159\u00e1dk\u016f Python k\u00f3du s anotacemi typ\u016f. Mnoho u\u017eivatel\u016f se nad\u0161en\u011b zapojilo do kontroly typ\u016f. V Dropboxu mypy vyu\u017e\u00edvalo \u010d\u00edm d\u00e1l v\u00edce t\u00fdm\u016f v\u00fdvoj\u00e1\u0159\u016f.<\/p>\n

V\u0161echno tehdy vypadalo dob\u0159e, ale je\u0161t\u011b n\u00e1s \u010dekalo mnoho pr\u00e1ce. Za\u010dali jsme prov\u00e1d\u011bt pravideln\u00e9 intern\u00ed dotazov\u00e1n\u00ed u\u017eivatel\u016f, abychom zjistili problematick\u00e9 oblasti projektu a pochopili, kter\u00e9 ot\u00e1zky je t\u0159eba \u0159e\u0161it jako prvn\u00ed (tato praxe se pou\u017e\u00edv\u00e1 ve spole\u010dnosti dodnes). Nejv\u00fdznamn\u011bj\u0161\u00edmi \u00fakoly byly, jak se uk\u00e1zalo, dv\u011b \u00fakoly. Prvn\u00ed \u2014 pot\u0159ebovali jsme v\u011bt\u0161\u00ed pokryt\u00ed k\u00f3du typy, druh\u00e1 \u2014 mypy pot\u0159eboval fungovat rychleji. Bylo zcela jasn\u00e9, \u017ee na\u0161e pr\u00e1ce na zrychlen\u00ed mypy a jeho zaveden\u00ed do projekt\u016f spole\u010dnosti byla je\u0161t\u011b daleko od dokon\u010den\u00ed. V pln\u00e9 m\u00ed\u0159e si uv\u011bdomuj\u00edc d\u016fle\u017eitost t\u011bchto dvou \u00fakol\u016f, pustili jsme se do jejich \u0159e\u0161en\u00ed.<\/p>\n

V\u00edce v\u00fdkonu!<\/h2>\n

\nInkrement\u00e1ln\u00ed kontroly urychlily mypy, ale tento n\u00e1stroj st\u00e1le nebyl dostate\u010dn\u011b rychl\u00fd. Mnoho inkrement\u00e1ln\u00edch kontrol trvalo kolem minuty. P\u0159\u00ed\u010dinou byla cyklick\u00e1 importov\u00e1n\u00ed. To pravd\u011bpodobn\u011b nep\u0159ekvap\u00ed ty, kdo pracovali s velk\u00fdmi k\u00f3dov\u00fdmi z\u00e1kladnami napsan\u00fdmi v Pythonu. M\u011bli jsme sady ze stovek modul\u016f, z nich\u017e ka\u017ed\u00fd nep\u0159\u00edmo importoval v\u0161echny ostatn\u00ed. Pokud byl jak\u00fdkoli soubor v cyklu import\u016f zm\u011bn\u011bn, mypy musel zpracovat v\u0161echny soubory, kter\u00e9 do tohoto cyklu pat\u0159ily, a \u010dasto i jak\u00e9koli moduly importuj\u00edc\u00ed moduly z tohoto cyklu. Jedn\u00edm z t\u011bchto cykl\u016f byla notoricky zn\u00e1m\u00e1 \u201eklubko z\u00e1vislost\u00ed\u201c, kter\u00e9 zp\u016fsobilo mnoho nep\u0159\u00edjemnost\u00ed v Dropboxu. Jednou tato struktura obsahovala n\u011bkolik set modul\u016f, p\u0159i\u010dem\u017e ji importovalo, p\u0159\u00edmo nebo nep\u0159\u00edmo, mnoho test\u016f a byla pou\u017e\u00edv\u00e1na i v produk\u010dn\u00edm k\u00f3du.<\/p>\n

Zva\u017eovali jsme mo\u017enost \u201erozpleten\u00ed\u201c cyklick\u00fdch z\u00e1vislost\u00ed, ale nem\u011bli jsme prost\u0159edky na to, abychom to ud\u011blali. Bylo tam p\u0159\u00edli\u0161 mnoho k\u00f3du, se kter\u00fdm jsme nebyli obezn\u00e1meni. Nakonec jsme se dostali k alternativn\u00edmu p\u0159\u00edstupu. Rozhodli jsme se, \u017ee mypy bude fungovat rychle i v p\u0159\u00edpad\u011b \u201eklubk\u016f z\u00e1vislost\u00ed\u201c. Tento c\u00edl jsme dos\u00e1hli pomoc\u00ed d\u00e9mona mypy. D\u00e9mona - to je serverov\u00fd proces, kter\u00fd implementuje dv\u011b zaj\u00edmav\u00e9 funkce. Za prv\u00e9 - uchov\u00e1v\u00e1 v pam\u011bti informace o cel\u00e9 k\u00f3dov\u00e9 z\u00e1kladn\u011b. To znamen\u00e1, \u017ee p\u0159i ka\u017ed\u00e9m spu\u0161t\u011bn\u00ed mypy nen\u00ed t\u0159eba na\u010d\u00edtat vyrovn\u00e1vac\u00ed data vztahuj\u00edc\u00ed se k tis\u00edc\u016fm importovan\u00fdch z\u00e1vislost\u00ed. Za druh\u00e9 - d\u016fkladn\u011b analyzuje z\u00e1vislosti mezi funkcemi a jin\u00fdmi entitami na \u00farovni mal\u00fdch struktur\u00e1ln\u00edch jednotek. Nap\u0159\u00edklad, pokud funkce foo<\/code> vyvol\u00e1v\u00e1 funkci bar<\/code>, pak existuje z\u00e1vislost foo<\/code> od bar<\/code>. Kdy\u017e se soubor zm\u011bn\u00ed - d\u00e9mon nejprve izolovan\u011b zpracov\u00e1v\u00e1 pouze zm\u011bn\u011bn\u00fd soubor. Pot\u00e9 se pod\u00edv\u00e1 na zm\u011bny tohoto souboru, kter\u00e9 jsou viditeln\u00e9 zven\u010d\u00ed, jako jsou nap\u0159\u00edklad zm\u011bn\u011bn\u00e9 podpisy funkc\u00ed. D\u00e9mon pou\u017e\u00edv\u00e1 podrobn\u00e9 informace o importech pouze pro op\u011btovn\u00e9 ov\u011b\u0159en\u00ed t\u011bch funkc\u00ed, kter\u00e9 skute\u010dn\u011b pou\u017e\u00edvaj\u00ed zm\u011bn\u011bnou funkci. Obvykle p\u0159i tomto p\u0159\u00edstupu je t\u0159eba kontrolovat velmi m\u00e1lo funkc\u00ed.<\/p>\n

Realizace toho v\u0161eho nebyla snadn\u00e1, proto\u017ee p\u016fvodn\u00ed implementace mypy byla siln\u011b zam\u011b\u0159ena na zpracov\u00e1n\u00ed jednoho souboru najednou. Museli jsme se vypo\u0159\u00e1dat s mnoha okrajov\u00fdmi situacemi, jejich\u017e vznik vy\u017eadoval opakovan\u00e9 kontroly, kdykoli do\u0161lo k jak\u00e9koliv zm\u011bn\u011b v k\u00f3du. Nap\u0159\u00edklad se to st\u00e1v\u00e1, kdy\u017e je t\u0159\u00edd\u011b p\u0159i\u0159azen nov\u00fd z\u00e1kladn\u00ed t\u0159\u00edda. Po tom, co jsme ud\u011blali to, co jsme cht\u011bli, jsme dok\u00e1zali sn\u00ed\u017eit \u010das prov\u00e1d\u011bn\u00ed v\u011bt\u0161iny inkrement\u00e1ln\u00edch kontrol na n\u011bkolik sekund. P\u0159i\u0161lo n\u00e1m to jako velk\u00e9 v\u00edt\u011bzstv\u00ed.<\/p>\n

Je\u0161t\u011b v\u011bt\u0161\u00ed v\u00fdkon!<\/h2>\n

\nSpole\u010dn\u011b s d\u00e1lkov\u00fdm ke\u0161ov\u00e1n\u00edm, o kter\u00e9m jsem se zm\u00ednil v\u00fd\u0161e, d\u00e9mon mypy prakticky \u00fapln\u011b vy\u0159e\u0161il probl\u00e9my, kter\u00e9 vznikaly, kdy\u017e program\u00e1tor \u010dasto spou\u0161t\u011bl kontrolu typ\u016f, p\u0159i\u010dem\u017e prov\u00e1d\u011bl zm\u011bny v mal\u00e9m po\u010dtu soubor\u016f. Nicm\u00e9n\u011b v\u00fdkon syst\u00e9mu v nejm\u00e9n\u011b p\u0159\u00edzniv\u00e9m variant\u011b jeho pou\u017eit\u00ed byl st\u00e1le daleko od optim\u00e1ln\u00edho. \u010cist\u00e9 spu\u0161t\u011bn\u00ed mypy mohlo trvat v\u00edce ne\u017e 15 minut. A to bylo mnohem v\u00edce, ne\u017e bychom byli ochotni akceptovat. Situace se ka\u017ed\u00fdm t\u00fddnem zhor\u0161ovala, proto\u017ee program\u00e1to\u0159i pokra\u010dovali v psan\u00ed nov\u00e9ho k\u00f3du a p\u0159id\u00e1v\u00e1n\u00ed anotac\u00ed k existuj\u00edc\u00edmu k\u00f3du. Na\u0161i u\u017eivatel\u00e9 st\u00e1le tou\u017eili po v\u011bt\u0161\u00edm v\u00fdkonu a my jsme s pot\u011b\u0161en\u00edm byli p\u0159ipraveni jim vyj\u00edt vst\u0159\u00edc.<\/p>\n

Rozhodli jsme se vr\u00e1tit k jedn\u00e9 z ran\u00fdch my\u0161lenek ohledn\u011b mypy. A to konkr\u00e9tn\u011b k p\u0159evodu k\u00f3du Pythonu do k\u00f3du C. Experimenty s Cythonem (co\u017e je syst\u00e9m, kter\u00fd umo\u017e\u0148uje p\u0159ev\u00e1d\u011bt k\u00f3d napsan\u00fd v Pythonu do k\u00f3du C) n\u00e1m nep\u0159inesly \u017e\u00e1dn\u00e9 viditeln\u00e9 zrychlen\u00ed, tak\u017ee jsme se rozhodli o\u017eivit my\u0161lenku naps\u00e1n\u00ed vlastn\u00edho kompil\u00e1toru. Vzhledem k tomu, \u017ee k\u00f3dov\u00e1 z\u00e1kladna mypy (napsan\u00e1 v Pythonu) ji\u017e obsahovala v\u0161echny pot\u0159ebn\u00e9 typov\u00e9 anotace, p\u0159i\u0161lo n\u00e1m smyslupln\u00e9 vyzkou\u0161et tyto anotace k urychlen\u00ed chodu syst\u00e9mu. Rychle jsem vytvo\u0159il prototyp pro ov\u011b\u0159en\u00ed t\u00e9to my\u0161lenky. Uk\u00e1zal v\u00edce ne\u017e desetin\u00e1sobn\u00fd n\u00e1r\u016fst v\u00fdkonu na r\u016fzn\u00fdch mikrobenchmarc\u00edch. Na\u0161\u00ed my\u0161lenkou bylo kompilovat moduly Pythonu na moduly C pomoc\u00ed Cythona a p\u0159ev\u00e1d\u011bt typov\u00e9 anotace na typov\u00e9 kontroly prov\u00e1d\u011bn\u00e9 b\u011bhem b\u011bhu programu (typov\u00e9 anotace se obvykle b\u011bhem b\u011bhu program\u016f ignoruj\u00ed a vyu\u017e\u00edvaj\u00ed se pouze kontroln\u00edmi syst\u00e9my typ\u016f). V podstat\u011b jsme pl\u00e1novali p\u0159ev\u00e9st implementaci mypy z Pythonu do jazyka, kter\u00fd byl staticky typovan\u00fd, a kter\u00fd by vypadal (a v\u011bt\u0161inou fungoval) p\u0159esn\u011b jako Python. (Tento typ mezijazykov\u00e9 migrace se stal jakousi tradic\u00ed projektu mypy. P\u016fvodn\u00ed implementace mypy byla naps\u00e1na v Alore, pot\u00e9 byl syntaktick\u00fd hybrid Javy a Pythonu).<\/p>\n

Orientace na API roz\u0161\u00ed\u0159en\u00ed CPython byla kl\u00ed\u010dem k tomu, abychom neztratili p\u0159\u00edle\u017eitosti v oblasti \u0159\u00edzen\u00ed projekt\u016f. Neminulo n\u00e1s pot\u0159eba implementovat virtu\u00e1ln\u00ed stroj nebo jak\u00e9koli knihovny, kter\u00e9 by mypy pot\u0159ebovalo. Nav\u00edc by n\u00e1m st\u00e1le byla p\u0159\u00edstupn\u00e1 cel\u00e1 ekosyst\u00e9m Pythonu, byly by dostupn\u00e9 v\u0161echny n\u00e1stroje (jako je pytest). To znamenalo, \u017ee bychom mohli pokra\u010dovat v pou\u017e\u00edv\u00e1n\u00ed interpretovan\u00e9ho k\u00f3du Pythonu b\u011bhem v\u00fdvoje, co\u017e by n\u00e1m umo\u017enilo nad\u00e1le pracovat s velmi rychlou metodou \u00faprav k\u00f3du a jeho testov\u00e1n\u00ed, m\u00edsto abychom \u010dekali na kompilaci k\u00f3du. Vypadalo to, \u017ee se n\u00e1m skv\u011ble da\u0159\u00ed, jak se \u0159\u00edk\u00e1, sed\u011bt na dvou \u017eidl\u00edch, a to se n\u00e1m l\u00edbilo.<\/p>\n

Kompl\u00e1tor, kter\u00fd jsme nazvali mypyc (proto\u017ee jako frontend pou\u017e\u00edv\u00e1 mypy pro anal\u00fdzu typ\u016f), se uk\u00e1zal jako velmi \u00fasp\u011b\u0161n\u00fd projekt. Celkov\u011b jsme dos\u00e1hli p\u0159ibli\u017en\u011b \u010dty\u0159n\u00e1sobn\u00e9ho zrychlen\u00ed \u010dast\u00fdch spu\u0161t\u011bn\u00ed mypy bez pou\u017eit\u00ed ke\u0161ov\u00e1n\u00ed. V\u00fdvoj j\u00e1dra projektu mypyc zabral mal\u00e9mu t\u00fdmu, do kter\u00e9ho pat\u0159ili Michael Sullivan, Ivan Levkyvskyi, Hugh Han a j\u00e1, p\u0159ibli\u017en\u011b 4 kalend\u00e1\u0159n\u00ed m\u011bs\u00edce. Tento objem pr\u00e1ce byl mnohem men\u0161\u00ed, ne\u017e by byl pot\u0159ebn\u00fd pro p\u0159eps\u00e1n\u00ed mypy nap\u0159\u00edklad do C++ nebo Go. A tak\u00e9 jsme museli prov\u00e9st mnohem m\u00e9n\u011b zm\u011bn v projektu, ne\u017e kolik by bylo pot\u0159eba p\u0159i p\u0159episov\u00e1n\u00ed do jin\u00e9ho jazyka. Doufali jsme tak\u00e9, \u017ee dostaneme mypyc na \u00farove\u0148, na kter\u00e9 by ho mohli pou\u017e\u00edvat i jin\u00ed program\u00e1to\u0159i z Dropboxu k tomu, aby kompilovali a zrychlovali sv\u016fj k\u00f3d.<\/p>\n

Abychom dos\u00e1hli takov\u00e9 \u00farovn\u011b v\u00fdkonu, museli jsme aplikovat n\u011bkter\u00e1 zaj\u00edmav\u00e1 in\u017een\u00fdrsk\u00e1 \u0159e\u0161en\u00ed. Kompl\u00e1tor m\u016f\u017ee urychlit vykon\u00e1v\u00e1n\u00ed mnoha operac\u00ed d\u00edky pou\u017eit\u00ed rychl\u00fdch n\u00edzko\u00farov\u0148ov\u00fdch konstrukc\u00ed C. Nap\u0159\u00edklad vol\u00e1n\u00ed zkompilovan\u00e9 funkce se p\u0159ekl\u00e1d\u00e1 na vol\u00e1n\u00ed C funkce. Takov\u00e9 vol\u00e1n\u00ed se vykon\u00e1v\u00e1 mnohem rychleji ne\u017e vol\u00e1n\u00ed interpretovan\u00e9 funkce. N\u011bkter\u00e9 operace, jako nap\u0159\u00edklad vyhled\u00e1v\u00e1n\u00ed ve slovn\u00edc\u00edch, st\u00e1le spo\u010d\u00edvaly v b\u011b\u017en\u00e9m vol\u00e1n\u00ed C-API z CPython, kter\u00e9 po kompilaci bylo jen o n\u011bco rychlej\u0161\u00ed. Dok\u00e1zali jsme se zbavit dodate\u010dn\u00e9 z\u00e1t\u011b\u017ee na syst\u00e9m, kterou vytv\u00e1\u0159ela interpretace, ale to v tomto p\u0159\u00edpad\u011b p\u0159ineslo jen mal\u00e9 v\u00fdhody v oblasti v\u00fdkonu.<\/p>\n

Pro identifikaci nejb\u011b\u017en\u011bj\u0161\u00edch \"pomal\u00fdch\" operac\u00ed jsme provedli profilov\u00e1n\u00ed k\u00f3du. Armed with the obtained data, we tried either to tweak mypyc so that it would generate faster C code for such operations, or rewrite the corresponding Python code using faster operations (sometimes we simply didn't have a straightforward solution for a specific problem). Rewriting Python code often proved to be an easier solution to the problem than implementing the same transformation automatically in the compiler. In the long term, we wanted to automate many of these transformations, but at that moment we were focused on speeding up mypy with minimal effort. And in pursuit of this goal, we cut a few corners.<\/p>\n

Pokra\u010dov\u00e1n\u00ed p\u0159\u00ed\u0161t\u011b\u2026<\/p>\n

V\u00e1\u017een\u00ed \u010dten\u00e1\u0159i!<\/b> Jak\u00e9 dojmy ve v\u00e1s vyvolal projekt mypy, kdy\u017e jste se o n\u011bm dozv\u011bd\u011bli?<\/p>\n

\"Cesta<\/a><\/noindex>
\n\"Cesta<\/a><\/noindex>
\n
Zdroj: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u0421\u0435\u0433\u043e\u0434\u043d\u044f \u043f\u0443\u0431\u043b\u0438\u043a\u0443\u0435\u043c \u0432\u0442\u043e\u0440\u0443\u044e \u0447\u0430\u0441\u0442\u044c \u043f\u0435\u0440\u0435\u0432\u043e\u0434\u0430 \u043c\u0430\u0442\u0435\u0440\u0438\u0430\u043b\u0430 \u043e \u0442\u043e\u043c, \u043a\u0430\u043a \u0432 Dropbox \u043e\u0440\u0433\u0430\u043d\u0438\u0437\u043e\u0432\u044b\u0432\u0430\u043b\u0438 \u043a\u043e\u043d\u0442\u0440\u043e\u043b\u044c \u0442\u0438\u043f\u043e\u0432 \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u0438\u0445 \u043c\u0438\u043b\u043b\u0438\u043e\u043d\u043e\u0432 \u0441\u0442\u0440\u043e\u043a Python-\u043a\u043e\u0434\u0430. \u2192 \u0427\u0438\u0442\u0430\u0442\u044c \u043f\u0435\u0440\u0432\u0443\u044e \u0447\u0430\u0441\u0442\u044c \u041e\u0444\u0438\u0446\u0438\u0430\u043b\u044c\u043d\u0430\u044f \u043f\u043e\u0434\u0434\u0435\u0440\u0436\u043a\u0430 \u0442\u0438\u043f\u043e\u0432 (PEP 484) \u041c\u044b \u043f\u0440\u043e\u0432\u0435\u043b\u0438 \u043f\u0435\u0440\u0432\u044b\u0435 \u0441\u0435\u0440\u044c\u0451\u0437\u043d\u044b\u0435 \u044d\u043a\u0441\u043f\u0435\u0440\u0438\u043c\u0435\u043d\u0442\u044b \u0441 mypy \u0432 Dropbox \u0432\u043e \u0432\u0440\u0435\u043c\u044f Hack Week 2014. Hack Week \u2014 \u044d\u0442\u043e \u043c\u0435\u0440\u043e\u043f\u0440\u0438\u044f\u0442\u0438\u0435, \u043f\u0440\u043e\u0432\u043e\u0434\u0438\u043c\u043e\u0435 Dropbox \u0432 \u0442\u0435\u0447\u0435\u043d\u0438\u0435 \u043e\u0434\u043d\u043e\u0439 \u043d\u0435\u0434\u0435\u043b\u0438. \u0412 \u044d\u0442\u043e \u0432\u0440\u0435\u043c\u044f […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":28879,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-38479","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Cesta k ov\u011b\u0159en\u00ed typ\u016f 4 milion\u016f \u0159\u00e1dk\u016f Python k\u00f3du. \u010c\u00e1st 2 | ProHoster","description":"Dnes publikujeme druhou \u010d\u00e1st p\u0159ekladu materi\u00e1lu o tom, jak v Dropboxu organizovali kontrolu typ\u016f n\u011bkolika milion\u016f \u0159\u00e1dk\u016f Python k\u00f3du.","canonical_url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/put-k-proverke-tipov-4-millionov-strok-python-koda-chast-2","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"cs_CZ","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041f\u0443\u0442\u044c \u043a \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u0435 \u0442\u0438\u043f\u043e\u0432 4 \u043c\u0438\u043b\u043b\u0438\u043e\u043d\u043e\u0432 \u0441\u0442\u0440\u043e\u043a Python-\u043a\u043e\u0434\u0430. \u0427\u0430\u0441\u0442\u044c 2 | ProHoster","og:description":"\u0421\u0435\u0433\u043e\u0434\u043d\u044f \u043f\u0443\u0431\u043b\u0438\u043a\u0443\u0435\u043c \u0432\u0442\u043e\u0440\u0443\u044e \u0447\u0430\u0441\u0442\u044c \u043f\u0435\u0440\u0435\u0432\u043e\u0434\u0430 \u043c\u0430\u0442\u0435\u0440\u0438\u0430\u043b\u0430 \u043e \u0442\u043e\u043c, \u043a\u0430\u043a \u0432 Dropbox \u043e\u0440\u0433\u0430\u043d\u0438\u0437\u043e\u0432\u044b\u0432\u0430\u043b\u0438 \u043a\u043e\u043d\u0442\u0440\u043e\u043b\u044c \u0442\u0438\u043f\u043e\u0432 \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u0438\u0445 \u043c\u0438\u043b\u043b\u0438\u043e\u043d\u043e\u0432 \u0441\u0442\u0440\u043e\u043a Python-\u043a\u043e\u0434\u0430.","og:url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/put-k-proverke-tipov-4-millionov-strok-python-koda-chast-2","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:23:56+00:00","article:modified_time":"2019-10-31T19:23:56+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"38479","title":null,"description":null,"keywords":null,"keyphrases":{"focus":[],"additional":[]},"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-23 22:13:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 01:08:32","updated":"2026-08-11 12:50:31","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/38479","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/comments?post=38479"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/38479\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media\/28879"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media?parent=38479"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/categories?post=38479"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/tags?post=38479"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}