{"id":37955,"date":"2019-10-31T22:20:47","date_gmt":"2019-10-31T19:20:47","guid":{"rendered":"https:\/\/prohoster.info\/blog\/linejnaya-regressiya-i-metody-eyo-vosstanovleniya\/"},"modified":"2019-10-31T22:20:47","modified_gmt":"2019-10-31T19:20:47","slug":"linejnaya-regressiya-i-metody-eyo-vosstanovleniya","status":"publish","type":"post","link":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/linejnaya-regressiya-i-metody-eyo-vosstanovleniya","title":{"rendered":"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/60ca67872405e9f15b151e958f04260d.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Zdroj: <noindex><a rel=\"nofollow\" href=\"https:\/\/xkcd.com\/1725\/\">xkcd<\/a><\/noindex><\/i><\/p>\n<p>Line\u00e1rn\u00ed regrese je jedn\u00edm z z\u00e1kladn\u00edch algoritm\u016f pro mnoho oblast\u00ed spojen\u00fdch s anal\u00fdzou dat. D\u016fvod je jasn\u00fd. Je to velmi jednoduch\u00fd a srozumiteln\u00fd algoritmus, co\u017e p\u0159isp\u00edv\u00e1 k jeho \u0161irok\u00e9mu vyu\u017e\u00edv\u00e1n\u00ed ji\u017e po des\u00edtky, ne-li stovky let. My\u0161lenka spo\u010d\u00edv\u00e1 v p\u0159edpokladu line\u00e1rn\u00ed z\u00e1vislosti jedn\u00e9 prom\u011bnn\u00e9 na souboru dal\u0161\u00edch prom\u011bnn\u00fdch, a pak se pokou\u0161\u00edme tuto z\u00e1vislost obnovit.<\/p>\n<p>V tomto \u010dl\u00e1nku se v\u0161ak nebudeme zab\u00fdvat aplikac\u00ed line\u00e1rn\u00ed regrese pro \u0159e\u0161en\u00ed praktick\u00fdch \u00faloh. Budeme se v\u011bnovat zaj\u00edmav\u00fdm rys\u016fm implementace distribuovan\u00fdch algoritm\u016f pro jej\u00ed obnovu, se kter\u00fdmi jsme se setkali p\u0159i psan\u00ed modulu strojov\u00e9ho u\u010den\u00ed v <noindex><a rel=\"nofollow\" href=\"https:\/\/ignite.apache.org\/\">Apache Ignite<\/a><\/noindex>. Trocha z\u00e1kladn\u00ed matematiky, princip\u016f strojov\u00e9ho u\u010den\u00ed a distribuovan\u00fdch v\u00fdpo\u010dt\u016f pom\u016f\u017ee objasnit, jak obnovit line\u00e1rn\u00ed regresi, i kdy\u017e jsou data rozd\u011blena mezi tis\u00edce uzl\u016f.<br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h3>O \u010dem je \u0159e\u010d?<\/h3>\n<p>\nStoj\u00edme p\u0159ed \u00fakolem obnovit line\u00e1rn\u00ed z\u00e1vislost. Jako vstupn\u00ed data m\u00e1me mno\u017estv\u00ed vektor\u016f \u00fadajn\u011b nez\u00e1visl\u00fdch prom\u011bnn\u00fdch, p\u0159i\u010dem\u017e ka\u017ed\u00e9mu z nich je p\u0159i\u0159azena hodnota z\u00e1visl\u00e9 prom\u011bnn\u00e9. Tato data lze p\u0159edstavit ve form\u011b dvou matic:<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/2ffbdd09fdc5efaf287fbb4935033302.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\nNyn\u00ed, kdy\u017e p\u0159edpokl\u00e1d\u00e1me z\u00e1vislost, a nav\u00edc je\u0161t\u011b line\u00e1rn\u00ed, zap\u00ed\u0161eme na\u0161i domn\u011bnku ve form\u011b sou\u010dinu matic (pro zjednodu\u0161en\u00ed z\u00e1pisu se zde a d\u00e1le p\u0159edpokl\u00e1d\u00e1, \u017ee voln\u00fd \u010dlen rovnice je skryt za <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/21e90cb829e0bc2e12c836f7810c1b1a.png\" style=\"display:block;margin: 0 auto;\" \/>, a posledn\u00ed sloupec matice <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/25c0af2e153ae371e71588efe3bc2ee2.png\" style=\"display:block;margin: 0 auto;\" \/> obsahuje jedni\u010dky):<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/b1c3c8d2332ae27eedeef675178dee4f.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\nVelmi to p\u0159ipom\u00edn\u00e1 syst\u00e9m line\u00e1rn\u00edch rovnic, \u017ee? P\u0159ipom\u00edn\u00e1, ale \u0159e\u0161en\u00ed tohoto syst\u00e9mu rovnic pravd\u011bpodobn\u011b neexistuje. P\u0159\u00ed\u010dinou je \u0161um, kter\u00fd se vyskytuje t\u00e9m\u011b\u0159 v ka\u017ed\u00fdch re\u00e1ln\u00fdch datech. Tak\u00e9 d\u016fvodem m\u016f\u017ee b\u00fdt absence line\u00e1rn\u00ed z\u00e1vislosti jako takov\u00e9, s n\u00ed\u017e lze bojovat zaveden\u00edm dal\u0161\u00edch prom\u011bnn\u00fdch, kter\u00e9 neline\u00e1rn\u011b z\u00e1vis\u00ed na p\u016fvodn\u00edch. Zva\u017eme n\u00e1sleduj\u00edc\u00ed p\u0159\u00edklad:<br \/>\n<img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/d449b8f931e91cc7b33634ee8d4a4329.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Zdroj: <noindex><a rel=\"nofollow\" href=\"https:\/\/en.wikipedia.org\/wiki\/Linear_regression\">Wikipedia<\/a><\/noindex><\/i><\/p>\n<p>Jedn\u00e1 se o jednoduch\u00fd p\u0159\u00edklad line\u00e1rn\u00ed regrese, kter\u00fd demonstruje z\u00e1vislost jedn\u00e9 prom\u011bnn\u00e9 (na ose <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/fdea97dff413fb13444d7fd8ab65ca0b.png\" style=\"display:block;margin: 0 auto;\" \/>) na druh\u00e9 prom\u011bnn\u00e9 (na ose <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/edf9cb265e4e4655cd84fb8739843952.png\" style=\"display:block;margin: 0 auto;\" \/>). Aby uveden\u00fd syst\u00e9m line\u00e1rn\u00edch rovnic m\u011bl \u0159e\u0161en\u00ed, mus\u00ed le\u017eet v\u0161echny body p\u0159esn\u011b na jedn\u00e9 p\u0159\u00edmce. Ale to tak nen\u00ed. A nele\u017e\u00ed na jedn\u00e9 p\u0159\u00edmce pr\u00e1v\u011b kv\u016fli \u0161umu (nebo kv\u016fli tomu, \u017ee p\u0159edpoklad o existenci line\u00e1rn\u00ed z\u00e1vislosti byl chybn\u00fd). Tak\u017ee k obnoven\u00ed line\u00e1rn\u00ed z\u00e1vislosti na skute\u010dn\u00fdch datech je obvykle nutn\u00e9 zav\u00e9st je\u0161t\u011b jeden p\u0159edpoklad: vstupn\u00ed \u00fadaje obsahuj\u00ed \u0161um a tento \u0161um m\u00e1 <noindex><a rel=\"nofollow\" href=\"https:\/\/en.wikipedia.org\/wiki\/Normal_distribution\">norm\u00e1ln\u00ed rozd\u011blen\u00ed<\/a><\/noindex>. Lze \u010dinit p\u0159edpoklady i o jin\u00fdch typech rozd\u011blen\u00ed \u0161umu, ale ve v\u011bt\u0161in\u011b p\u0159\u00edpad\u016f se zva\u017euje pr\u00e1v\u011b norm\u00e1ln\u00ed rozd\u011blen\u00ed, o kter\u00e9m bude d\u00e1le \u0159e\u010d.<\/p>\n<h3>Metoda maxim\u00e1ln\u00ed v\u011brohodnosti<\/h3>\n<p>\nTak\u017ee jsme p\u0159edpokl\u00e1dali p\u0159\u00edtomnost n\u00e1hodn\u00e9ho norm\u00e1ln\u011b rozd\u011blen\u00e9ho \u0161umu. Jak se tedy v t\u00e9to situaci chovat? Na to existuje a je \u0161iroce vyu\u017e\u00edv\u00e1na v matematice <noindex><a rel=\"nofollow\" href=\"https:\/\/en.wikipedia.org\/wiki\/Maximum_likelihood_estimation\">metoda maxim\u00e1ln\u00ed v\u011brohodnosti<\/a><\/noindex>. Stru\u010dn\u011b \u0159e\u010deno, jej\u00ed podstatou je v\u00fdb\u011br <noindex><a rel=\"nofollow\" href=\"https:\/\/en.wikipedia.org\/wiki\/Likelihood_function\">funkce v\u011brohodnosti<\/a><\/noindex> a jej\u00ed n\u00e1sledn\u00e1 maximalizace.<\/p>\n<p>Vra\u0165me se k obnoven\u00ed line\u00e1rn\u00ed z\u00e1vislosti na datech se norm\u00e1ln\u00edm \u0161umem. V\u0161imn\u011bme si, \u017ee p\u0159edpokl\u00e1dan\u00e1 line\u00e1rn\u00ed z\u00e1vislost je matematick\u00fdm o\u010dek\u00e1v\u00e1n\u00edm <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/73cd24a6605bce1a4f38339ee8c61613.png\" style=\"display:block;margin: 0 auto;\" \/> existuj\u00edc\u00edho norm\u00e1ln\u00edho rozd\u011blen\u00ed. Z\u00e1rove\u0148 je pravd\u011bpodobnost, \u017ee <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/caaf32b1af58d748244acbab640bbab8.png\" style=\"display:block;margin: 0 auto;\" \/> nab\u00fdv\u00e1 ur\u010dit\u00e9 hodnoty, za p\u0159edpokladu, \u017ee existuj\u00ed pozorovan\u00e9 <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/9cda87721bba4812b7cec96f204ff5f6.png\" style=\"display:block;margin: 0 auto;\" \/>, vypad\u00e1 n\u00e1sledovn\u011b:<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/7acbd6bff263d52773617904d3249a96.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\nNyn\u00ed dosad\u00edme m\u00edsto <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/d5a167a87fa416e938678b2a80b353dc.png\" style=\"display:block;margin: 0 auto;\" \/> a <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/330ad17be2629a5159b513ba95a96c72.png\" style=\"display:block;margin: 0 auto;\" \/> n\u00e1m pot\u0159ebn\u00e9 prom\u011bnn\u00e9:<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/57531f93ee0d0050b1fbfc64419f44ad.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\nZb\u00fdv\u00e1 pouze naj\u00edt vektor <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/801ebcd43ae0bc0cf54bc0f68bdc21da.png\" style=\"display:block;margin: 0 auto;\" \/>, p\u0159i kter\u00e9m je tato pravd\u011bpodobnost maxim\u00e1ln\u00ed. Abychom takovou funkci maximalizovali, je pohodln\u00e9 nejprve ji zlogaritmovat (logaritmus funkce dos\u00e1hne maxima ve stejn\u00e9 lokaci jako samotn\u00e1 funkce):<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/7485448a44f8ee201254fea4208deb54.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\nCo\u017e se zase shrnuje na minimalizaci n\u00e1sleduj\u00edc\u00ed funkce:<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/0957864e5dfbfc1147c24784df67af20.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\nMimochodem, tomu se \u0159\u00edk\u00e1 metoda <noindex><a rel=\"nofollow\" href=\"https:\/\/en.wikipedia.org\/wiki\/Linear_least_squares\">nejmen\u0161\u00edch \u010dtverc\u016f<\/a><\/noindex>. \u010casto jsou v\u0161echny v\u00fd\u0161e uveden\u00e9 \u00favahy opomenuty a jednodu\u0161e se tento metoda pou\u017e\u00edv\u00e1.<\/p>\n<h3>QR rozklad<\/h3>\n<p>\nMinimum uveden\u00e9 funkce lze nal\u00e9zt, pokud najdeme bod, ve kter\u00e9m je gradient t\u00e9to funkce roven nule. A gradient bude zaps\u00e1n n\u00e1sleduj\u00edc\u00edm zp\u016fsobem:<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/1581c46f8fa625826ae2f76ec561dc3f.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/en.wikipedia.org\/wiki\/QR_decomposition\">QR rozklad<\/a><\/noindex> je maticovou metodou \u0159e\u0161en\u00ed \u00falohy minimalizace pou\u017e\u00edvanou v metod\u011b nejmen\u0161\u00edch \u010dtverc\u016f. V tomto ohledu p\u0159ep\u00ed\u0161eme rovnici v maticov\u00e9m tvaru:<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/4ee809a96550577df855fbd57049a41c.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\nTak\u017ee rozkl\u00e1d\u00e1me matici <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/2fa2b32bef5246132da5b3b9bc713ab5.png\" style=\"display:block;margin: 0 auto;\" \/> na matice <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/fd8c620c602ae49e8d6e39ef8b551d0b.png\" style=\"display:block;margin: 0 auto;\" \/> a <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/32804631a9a2f8bff35d0bcf7c29bfc6.png\" style=\"display:block;margin: 0 auto;\" \/> a prov\u00e1d\u00edme \u0159adu transformac\u00ed (samotn\u00fd algoritmus QR rozkladu zde bude projedn\u00e1n jen okrajov\u011b, pouze jeho vyu\u017eit\u00ed v souvislosti s dan\u00fdm \u00fakolem):<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/56cfb4ed9126e632cba51940c0d9afe6.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\nMatice <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/106bb4d6704eb9f42078555c662a8b97.png\" style=\"display:block;margin: 0 auto;\" \/> je ortogon\u00e1ln\u00ed. To n\u00e1m umo\u017e\u0148uje zbavit se sou\u010dinu <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/618077c012f7b81f23756b9c1e54eb73.png\" style=\"display:block;margin: 0 auto;\" \/>:<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/1715fb2e77d8be75e68e4791997aaa44.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\nA pokud nahrad\u00edme <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/926c51120ec8608bf81dc8f36fcc3ef4.png\" style=\"display:block;margin: 0 auto;\" \/> na <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/9e578153ea2dc36d0eb86be4eedf4899.png\" style=\"display:block;margin: 0 auto;\" \/>, vyjde <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/8cd914db7317c01dc5449c70b36dba73.png\" style=\"display:block;margin: 0 auto;\" \/>. Vzhledem k tomu, \u017ee <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/88fa4e81fc2bedf16e41e90e320a5647.png\" style=\"display:block;margin: 0 auto;\" \/> je horn\u00ed troj\u00faheln\u00edkov\u00e1 matice, vypad\u00e1 to n\u00e1sledovn\u011b:<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/689eaaece2a497c5bd02582e3e672a41.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\nToto lze \u0159e\u0161it pomoc\u00ed substitu\u010dn\u00ed metody. Prvek <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/e163597c97a539238731c31ca5ce011b.png\" style=\"display:block;margin: 0 auto;\" \/> se nach\u00e1z\u00ed jako <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/211d86418c9a38f164f490b1a7b5fb71.png\" style=\"display:block;margin: 0 auto;\" \/>, p\u0159edchoz\u00ed prvek <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/ed3742b3a7cfb14bad801485a8cf01df.png\" style=\"display:block;margin: 0 auto;\" \/> se nach\u00e1z\u00ed jako <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/b3e582f565e1060b47af4008a98c3524.png\" style=\"display:block;margin: 0 auto;\" \/> a tak d\u00e1le.<\/p>\n<p>Zde je t\u0159eba poznamenat, \u017ee slo\u017eitost v\u00fdsledn\u00e9ho algoritmu d\u00edky pou\u017eit\u00ed QR rozkladu je rovna <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/c5c20145b7b165120d9979c2e2ca4711.png\" style=\"display:block;margin: 0 auto;\" \/>. P\u0159itom, i kdy\u017e operace n\u00e1soben\u00ed matic je dob\u0159e paralelizovateln\u00e1, napsat efektivn\u00ed distribuovanou verzi tohoto algoritmu se nezd\u00e1 mo\u017en\u00e9.<\/p>\n<h3>Gradientn\u00ed sp\u00e1d<\/h3>\n<p>\nKdy\u017e hovo\u0159\u00edme o minimalizaci ur\u010dit\u00e9 funkce, v\u017edy stoj\u00ed za to vzpomenout na metodu (stochastick\u00e9ho) gradientn\u00edho sp\u00e1du. Je to jednoduch\u00e1 a efektivn\u00ed metoda minimalizace, zalo\u017een\u00e1 na iterativn\u00edm v\u00fdpo\u010dtu gradientu funkce v bod\u011b a n\u00e1sledn\u00e9m posunu sm\u011brem, kter\u00fd je opa\u010dn\u00fd k gradientu. Ka\u017ed\u00fd takov\u00fd krok p\u0159ibli\u017euje \u0159e\u0161en\u00ed k minimu. Gradient vypad\u00e1 st\u00e1le takto:<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/d195806787197323312be7c0b9d8b240.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>Tato metoda je tak\u00e9 dob\u0159e paralelizovateln\u00e1 a distribuovateln\u00e1 d\u00edky line\u00e1rn\u00edm vlastnostem oper\u00e1tora gradientu. V\u0161imn\u011bme si, \u017ee ve v\u00fd\u0161e uveden\u00e9m vzorci pod symbolem sumy jsou nez\u00e1visl\u00e9 \u010dleny. Jin\u00fdmi slovy, m\u016f\u017eeme vypo\u010d\u00edtat gradient nez\u00e1visle pro v\u0161echny indexy <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/0c10d574f70b318b6562fb444e460fa7.png\" style=\"display:block;margin: 0 auto;\" \/> od prvn\u00edho do <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/c9b39d76c5f724af137b5db3053e1a60.png\" style=\"display:block;margin: 0 auto;\" \/>, sou\u010dasn\u011b lze vypo\u010d\u00edtat gradient pro indexy od <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/1a9407fcb4ec67a171463d36dca30a80.png\" style=\"display:block;margin: 0 auto;\" \/> do <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/f1e29c5b16d68d6377108f53803a9cd0.png\" style=\"display:block;margin: 0 auto;\" \/>. Pak se se\u010dtou z\u00edskan\u00e9 gradienty. V\u00fdsledkem s\u010d\u00edt\u00e1n\u00ed bude stejn\u00fd, jako kdybychom okam\u017eit\u011b vypo\u010d\u00edtali gradient pro indexy od prvn\u00edho do <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/c5ec5004c59a5343f22894ef676606e3.png\" style=\"display:block;margin: 0 auto;\" \/>. T\u00edmto zp\u016fsobem, pokud jsou data rozd\u011blena mezi n\u011bkolik \u010d\u00e1st\u00ed dat, m\u016f\u017ee b\u00fdt gradient vypo\u010d\u00edt\u00e1n nez\u00e1visle na ka\u017ed\u00e9 \u010d\u00e1sti, a pot\u00e9 mohou b\u00fdt v\u00fdsledky t\u011bchto v\u00fdpo\u010dt\u016f se\u010dteny, aby se z\u00edskal kone\u010dn\u00fd v\u00fdsledek:<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/073ce5e3a6688e3a1ae7774aaff1a843.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>Z hlediska implementace se to vejde do paradigmatu <noindex><a rel=\"nofollow\" href=\"https:\/\/en.wikipedia.org\/wiki\/MapReduce\">MapReduce<\/a><\/noindex>. V ka\u017ed\u00e9m kroku gradientn\u00edho sp\u00e1du je ka\u017ed\u00e9mu uzlu dat zasl\u00e1na \u00faloha pro v\u00fdpo\u010det gradientu, pot\u00e9 jsou vypo\u010d\u00edtan\u00e9 gradienty shrom\u00e1\u017ed\u011bny dohromady a v\u00fdsledek jejich s\u010d\u00edt\u00e1n\u00ed se vyu\u017e\u00edv\u00e1 k vylep\u0161en\u00ed v\u00fdsledku.<\/p>\n<p>I kdy\u017e je implementace jednoduch\u00e1 a umo\u017e\u0148uje prov\u00e1d\u011bn\u00ed v paradigmatu MapReduce, gradientn\u00ed sestup m\u00e1 tak\u00e9 sv\u00e9 nev\u00fdhody. Obzvl\u00e1\u0161t\u011b po\u010det krok\u016f pot\u0159ebn\u00fdch k dosa\u017een\u00ed konvergence je zna\u010dn\u011b vy\u0161\u0161\u00ed ve srovn\u00e1n\u00ed s jin\u00fdmi, v\u00edce specializovan\u00fdmi metodami.<\/p>\n<h3>LSQR<\/h3>\n<p>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/web.stanford.edu\/group\/SOL\/software\/lsqr\/\">LSQR<\/a><\/noindex> je dal\u0161\u00ed metoda \u0159e\u0161en\u00ed dan\u00e9ho \u00fakolu, kter\u00e1 je vhodn\u00e1 jak pro obnovu line\u00e1rn\u00ed regrese, tak pro \u0159e\u0161en\u00ed syst\u00e9m\u016f line\u00e1rn\u00edch rovnic. Jej\u00edm hlavn\u00edm rysem je, \u017ee kombinuje v\u00fdhody maticov\u00fdch metod a iterativn\u00edho p\u0159\u00edstupu. Implementace t\u00e9to metody lze naj\u00edt v knihovn\u011b <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.scipy.org\/doc\/scipy-0.14.0\/reference\/generated\/scipy.sparse.linalg.lsqr.html\">SciPy<\/a><\/noindex>, tak i v <noindex><a rel=\"nofollow\" href=\"http:\/\/matlab.izmiran.ru\/help\/techdoc\/ref\/lsqr.html\">MATLAB<\/a><\/noindex>. Popis t\u00e9to metody zde nebude uveden (m\u016f\u017eete jej nal\u00e9zt v \u010dl\u00e1nku <noindex><a rel=\"nofollow\" href=\"https:\/\/web.stanford.edu\/group\/SOL\/software\/lsqr\/lsqr-toms82a.pdf\">LSQR: An algorithm for sparse linear equations and sparse least squares<\/a><\/noindex>). M\u00edsto toho bude demonstrov\u00e1n p\u0159\u00edstup, kter\u00fd umo\u017e\u0148uje p\u0159izp\u016fsobit LSQR pro prov\u00e1d\u011bn\u00ed v distribuovan\u00e9m prost\u0159ed\u00ed.<\/p>\n<p>Z\u00e1kladem metody LSQR je <noindex><a rel=\"nofollow\" href=\"http:\/\/www.netlib.org\/utk\/people\/JackDongarra\/etemplates\/node198.html\">procedura bidiagonalizace<\/a><\/noindex>. To je iterativn\u00ed procedura, jej\u00ed\u017e ka\u017ed\u00e1 iterace se skl\u00e1d\u00e1 z n\u00e1sleduj\u00edc\u00edch krok\u016f:<br \/>\n<img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/3c2f7b5c6f57830e9b522023a8e72a48.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nPokud v\u0161ak vych\u00e1z\u00edme z toho, \u017ee matice je horizont\u00e1ln\u011b partitionov\u00e1na, pak ka\u017edou iteraci lze p\u0159edstavit jako dva kroky MapReduce. T\u00edmto zp\u016fsobem se da\u0159\u00ed minimalizovat p\u0159enos dat b\u011bhem ka\u017ed\u00e9 z iterac\u00ed (pouze vektory o d\u00e9lce odpov\u00eddaj\u00edc\u00ed po\u010dtu nezn\u00e1m\u00fdch): <img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/e86c3c422629b78bc574f66db8a6139d.png\" style=\"display:block;margin: 0 auto;\" \/> Pr\u00e1v\u011b tento p\u0159\u00edstup se pou\u017e\u00edv\u00e1 p\u0159i implementaci line\u00e1rn\u00ed regrese v<\/p>\n<p><img decoding=\"async\" alt=\"Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy\" src=\"\/wp-content\/uploads\/2019\/09\/6b67654b29ed24283c4b04d66c05ea5c.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nApache Ignite ML <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/apache\/ignite\/blob\/master\/modules\/ml\/src\/main\/java\/org\/apache\/ignite\/ml\/math\/isolve\/lsqr\/AbstractLSQR.java\">Existuje mnoho algoritm\u016f pro obnovu line\u00e1rn\u00ed regrese, ale ne v\u0161echny mohou b\u00fdt pou\u017e\u00edv\u00e1ny za jak\u00fdchkoli podm\u00ednek. Nap\u0159\u00edklad QR rozklad je skv\u011bl\u00fd pro p\u0159esn\u00e9 \u0159e\u0161en\u00ed na mal\u00fdch datov\u00fdch sad\u00e1ch. Gradientn\u00ed sestup je snadno implementovateln\u00fd a umo\u017e\u0148uje rychle naj\u00edt p\u0159ibli\u017en\u00e9 \u0159e\u0161en\u00ed. A LSQR kombinuje nejlep\u0161\u00ed vlastnosti p\u0159edchoz\u00edch dvou algoritm\u016f, proto\u017ee m\u016f\u017ee b\u00fdt distribuov\u00e1n, rychleji konverguje ve srovn\u00e1n\u00ed s gradientn\u00edm sestupem a tak\u00e9 umo\u017e\u0148uje p\u0159ed\u010dasn\u00e9 zastaven\u00ed algoritmu na rozd\u00edl od QR rozkladu p\u0159i hled\u00e1n\u00ed p\u0159ibli\u017en\u00e9ho \u0159e\u0161en\u00ed.<\/a><\/noindex>.<\/p>\n<h3>Z\u00e1v\u011br<\/h3>\n<p>\n\ud83e\udd47Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy | ProHoster<br \/>\n<br \/>Zdroj: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/465743\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0418\u0441\u0442\u043e\u0447\u043d\u0438\u043a: xkcd \u041b\u0438\u043d\u0435\u0439\u043d\u0430\u044f \u0440\u0435\u0433\u0440\u0435\u0441\u0441\u0438\u044f \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u043e\u0434\u043d\u0438\u043c \u0438\u0437 \u0431\u0430\u0437\u043e\u0432\u044b\u0445 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c\u043e\u0432 \u0434\u043b\u044f \u043c\u043d\u043e\u0433\u0438\u0445 \u043e\u0431\u043b\u0430\u0441\u0442\u0435\u0439, \u0441\u0432\u044f\u0437\u0430\u043d\u043d\u044b\u0445 \u0441 \u0430\u043d\u0430\u043b\u0438\u0437\u043e\u043c \u0434\u0430\u043d\u043d\u044b\u0445. \u041f\u0440\u0438\u0447\u0438\u043d\u0430 \u044d\u0442\u043e\u043c\u0443 \u043e\u0447\u0435\u0432\u0438\u0434\u043d\u0430. \u042d\u0442\u043e \u043e\u0447\u0435\u043d\u044c \u043f\u0440\u043e\u0441\u0442\u043e\u0439 \u0438 \u043f\u043e\u043d\u044f\u0442\u043d\u044b\u0439 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c, \u0447\u0442\u043e \u0441\u043f\u043e\u0441\u043e\u0431\u0441\u0442\u0432\u0443\u0435\u0442 \u0435\u0433\u043e \u0448\u0438\u0440\u043e\u043a\u043e\u043c\u0443 \u043f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0438\u044e \u0443\u0436\u0435 \u043c\u043d\u043e\u0433\u0438\u0435 \u0434\u0435\u0441\u044f\u0442\u043a\u0438, \u0435\u0441\u043b\u0438 \u043d\u0435 \u0441\u043e\u0442\u043d\u0438, \u043b\u0435\u0442. \u0418\u0434\u0435\u044f \u0437\u0430\u043a\u043b\u044e\u0447\u0430\u0435\u0442\u0441\u044f \u0432 \u0442\u043e\u043c, \u0447\u0442\u043e \u043c\u044b \u043f\u0440\u0435\u0434\u043f\u043e\u043b\u0430\u0433\u0430\u0435\u043c \u043b\u0438\u043d\u0435\u0439\u043d\u0443\u044e \u0437\u0430\u0432\u0438\u0441\u0438\u043c\u043e\u0441\u0442\u044c \u043e\u0434\u043d\u043e\u0439 \u043f\u0435\u0440\u0435\u043c\u0435\u043d\u043d\u043e\u0439 \u043e\u0442 \u043d\u0430\u0431\u043e\u0440\u0430 \u0434\u0440\u0443\u0433\u0438\u0445 \u043f\u0435\u0440\u0435\u043c\u0435\u043d\u043d\u044b\u0445, \u0430 \u043f\u043e\u0442\u043e\u043c \u043f\u044b\u0442\u0430\u0435\u043c\u0441\u044f [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":28483,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-37955","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.3 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0418\u0441\u0442\u043e\u0447\u043d\u0438\u043a:\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/linejnaya-regressiya-i-metody-eyo-vosstanovleniya\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.3\" \/>\n\t\t<meta property=\"og:locale\" content=\"cs_CZ\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041b\u0438\u043d\u0435\u0439\u043d\u0430\u044f \u0440\u0435\u0433\u0440\u0435\u0441\u0441\u0438\u044f \u0438 \u043c\u0435\u0442\u043e\u0434\u044b \u0435\u0451 \u0432\u043e\u0441\u0441\u0442\u0430\u043d\u043e\u0432\u043b\u0435\u043d\u0438\u044f | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0418\u0441\u0442\u043e\u0447\u043d\u0438\u043a:\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/linejnaya-regressiya-i-metody-eyo-vosstanovleniya\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-10-31T19:20:47+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2019-10-31T19:20:47+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Line\u00e1rn\u00ed regrese a metody jej\u00ed obnovy | ProHoster","description":"Zdroj:","canonical_url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/linejnaya-regressiya-i-metody-eyo-vosstanovleniya","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"cs_CZ","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041b\u0438\u043d\u0435\u0439\u043d\u0430\u044f \u0440\u0435\u0433\u0440\u0435\u0441\u0441\u0438\u044f \u0438 \u043c\u0435\u0442\u043e\u0434\u044b \u0435\u0451 \u0432\u043e\u0441\u0441\u0442\u0430\u043d\u043e\u0432\u043b\u0435\u043d\u0438\u044f | ProHoster","og:description":"\u0418\u0441\u0442\u043e\u0447\u043d\u0438\u043a:","og:url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/linejnaya-regressiya-i-metody-eyo-vosstanovleniya","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:20:47+00:00","article:modified_time":"2019-10-31T19:20:47+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"37955","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-23 19:55:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 01:17:23","updated":"2026-01-23 19:55:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/37955","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/comments?post=37955"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/37955\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media\/28483"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media?parent=37955"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/categories?post=37955"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/tags?post=37955"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}