{"id":37955,"date":"2019-10-31T22:20:47","date_gmt":"2019-10-31T19:20:47","guid":{"rendered":"https:\/\/prohoster.info\/blog\/linejnaya-regressiya-i-metody-eyo-vosstanovleniya\/"},"modified":"2019-10-31T22:20:47","modified_gmt":"2019-10-31T19:20:47","slug":"linejnaya-regressiya-i-metody-eyo-vosstanovleniya","status":"publish","type":"post","link":"https:\/\/prohoster.info\/da\/blog\/administrirovanie\/linejnaya-regressiya-i-metody-eyo-vosstanovleniya","title":{"rendered":"Line\u00e6r regression og metoder til dens genopretning","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Line\u00e6r
\nKilde: xkcd<\/a><\/noindex><\/i><\/p>\n

Line\u00e6r regression er en af de grundl\u00e6ggende algoritmer inden for mange omr\u00e5der relateret til dataanalyse. \u00c5rsagen hertil er \u00e5benlys. Det er en meget simpel og forst\u00e5elig algoritme, hvilket g\u00f8r den bredt anvendelig i mange \u00e5rtier, hvis ikke hundreder, nu. Ideen er, at vi antager en line\u00e6r afh\u00e6ngighed af \u00e9n variabel fra et s\u00e6t af andre variable, og derefter fors\u00f8ger vi at genskabe denne afh\u00e6ngighed.<\/p>\n

Men i denne artikel vil vi ikke fokusere p\u00e5 anvendelsen af line\u00e6r regression til l\u00f8sning af praktiske opgaver. Her vil vi se p\u00e5 interessante egenskaber ved implementeringen af distribuerede algoritmer til dens genskabelse, som vi st\u00f8dte p\u00e5, da vi skrev en maskinl\u00e6ringsmodul i Apache Ignite<\/a><\/noindex>. Noget grundl\u00e6ggende matematik, grundl\u00e6ggende maskinl\u00e6ringsprincipper og distribueret computing vil hj\u00e6lpe med at forst\u00e5, hvordan man kan genskabe line\u00e6r regression, selvom dataene er fordelt mellem tusindvis af noder.
\n<\/a><\/noindex><\/p>\n

Hvad handler det om?<\/h3>\n

\nVi st\u00e5r over for opgaven at genskabe den line\u00e6re afh\u00e6ngighed. Som inputdata gives et s\u00e6t vektorer af formodede uafh\u00e6ngige variable, hvor hver af dem svarer til en v\u00e6rdi af den afh\u00e6ngige variabel. Disse data kan repr\u00e6senteres som to matricer:<\/p>\n

\"Line\u00e6r<\/p>\n

\nNu, da der antages en afh\u00e6ngighed, og det er en line\u00e6r en, lad os skrive vores antagelse som et matrixprodukt (for at forenkle notationen antages det her og fremad, at den frie led i ligningen er skjult bag \"Line\u00e6r, og den sidste kolonne i matricen \"Line\u00e6r indeholder etere):<\/p>\n

\"Line\u00e6r<\/p>\n

\nDet ligner meget et system af line\u00e6re ligninger, g\u00f8r det ikke? Det ligner, men s\u00e5dant et system af ligninger vil sandsynligvis ikke have nogen l\u00f8sninger. \u00c5rsagen til dette er st\u00f8j, der praktisk talt findes i alle virkelige data. En anden \u00e5rsag kan v\u00e6re frav\u00e6ret af en line\u00e6r afh\u00e6ngighed som s\u00e5dan, som man kan fors\u00f8ge at tackle ved at indf\u00f8re yderligere variable, der ikke-line\u00e6rt afh\u00e6nger af de oprindelige. Lad os overveje f\u00f8lgende eksempel:
\n\"Line\u00e6r
\nKilde: Wikipedia<\/a><\/noindex><\/i><\/p>\n

Dette er et simpelt eksempel p\u00e5 line\u00e6r regression, der viser afh\u00e6ngigheden af \u00e9n variabel (p\u00e5 aksen \"Line\u00e6r) fra en anden variabel (p\u00e5 aksen \"Line\u00e6r). For the system of linear equations to have a solution, all points must lie exactly on one line. But this is not the case. They do not lie on one line precisely because of noise (or because the assumption of linear dependence was incorrect). Thus, to recover linear dependence from real data, it is usually necessary to introduce one more assumption: the input data contains noise and this noise has a normal distribution<\/a><\/noindex>. Other types of noise distributions can also be assumed, but in the overwhelming majority of cases, it is specifically the normal distribution that is considered, which will be discussed further.<\/p>\n

Maximum likelihood-metoden<\/h3>\n

\nSo, we have assumed the presence of randomly normally distributed noise. What should we do in this situation? Mathematics has a widely used approach for this. maksimale sandsynlighedsmetode<\/a><\/noindex>. In short, it involves selecting sandsynlighedsfunktionen<\/a><\/noindex> and subsequently maximizing it.<\/p>\n

Returning to the restoration of linear dependence from data with normal noise, note that the presumed linear dependence represents the mathematical expectation \"Line\u00e6r of the existing normal distribution. At the same time, the probability that \"Line\u00e6r takes a certain value, given the presence of observed \"Line\u00e6r, looks as follows:<\/p>\n

\"Line\u00e6r<\/p>\n

\nNow, let's substitute the necessary variables instead of \"Line\u00e6r og \"Line\u00e6r :<\/p>\n

\"Line\u00e6r<\/p>\n

\nWe only need to find the vector \"Line\u00e6r, for which this probability is maximized. To maximize such a function, it is convenient to first logarithm it (the logarithm of the function will reach its maximum at the same point as the function itself):<\/p>\n

\"Line\u00e6r<\/p>\n

\nWhich, in turn, reduces to minimizing the following function:<\/p>\n

\"Line\u00e6r<\/p>\n

\nBy the way, this is called the method of least squares<\/a><\/noindex>. Often, all of the above reasoning is omitted, and this method is simply used.<\/p>\n

QR decomposition<\/h3>\n

\nThe minimum of the above function can be found by locating the point at which the gradient of this function is zero. The gradient will be written as follows:<\/p>\n

\"Line\u00e6r<\/p>\n

\nQR decomposition<\/a><\/noindex> is a matrix method for solving the minimization problem used in the least squares method. In this regard, let's rewrite the equation in matrix form:<\/p>\n

\"Line\u00e6r<\/p>\n

\nThus, we decompose the matrix \"Line\u00e6r into matrices \"Line\u00e6r og \"Line\u00e6r og udf\u00f8rer en r\u00e6kke transformationer (selve QR-faktoriseringen vil ikke blive diskuteret her, kun dens anvendelse i forhold til den stillede opgave):<\/p>\n

\"Line\u00e6r<\/p>\n

\nMatrix \"Line\u00e6r er ortogonal. Dette giver os mulighed for at undg\u00e5 produktet \"Line\u00e6r:<\/p>\n

\"Line\u00e6r<\/p>\n

\nOg hvis vi erstatter \"Line\u00e6r og f\u00e5 en fungerende kommando. \"Line\u00e6r, s\u00e5 f\u00e5r vi \"Line\u00e6r. I betragtning af at \"Line\u00e6r er en \u00f8verste trekantet matrix, ser det s\u00e5ledes ud:<\/p>\n

\"Line\u00e6r<\/p>\n

\nDet kan l\u00f8ses ved substitutionsmetoden. Elementet \"Line\u00e6r er fundet som \"Line\u00e6r, det forrige element \"Line\u00e6r er fundet som \"Line\u00e6r og s\u00e5 videre.<\/p>\n

Her skal det bem\u00e6rkes, at kompleksiteten af den resulterende algoritme ved brug af QR-faktorisering er lig med \"Line\u00e6r. Selvom matrixmultiplikation kan parallelliseres godt, er det ikke muligt at skrive en effektiv distribueret version af denne algoritme.<\/p>\n

Gradientfald<\/h3>\n

\nN\u00e5r man taler om optimering af en given funktion, er det altid v\u00e6rd at huske metoden for (stochastic) gradientfald. Det er en simpel og effektiv metode til at minimere, baseret p\u00e5 iterativ beregning af gradienten af funktionen p\u00e5 et punkt og derefter flytning i retning modsat gradienten. Hvert s\u00e5dant skridt bringer l\u00f8sningen t\u00e6ttere p\u00e5 minimum. Gradientet ser dermed stadig s\u00e5dan ud:<\/p>\n

\"Line\u00e6r<\/p>\n

Denne metode kan desuden lettet parallelliseres og distribueres takket v\u00e6re de line\u00e6re egenskaber ved gradientoperatoren. Bem\u00e6rk, at de uafh\u00e6ngige led i summen er indeholdt i formlen ovenfor. Med andre ord kan vi beregne gradienten uafh\u00e6ngigt for alle indekser \"Line\u00e6r fra den f\u00f8rste til \"Line\u00e6r, samtidig med dette kan vi beregne gradienten for indekserne fra \"Line\u00e6r op til \"Line\u00e6r. Derefter kan de opn\u00e5ede gradienter l\u00e6gges sammen. Resultatet af summen vil v\u00e6re det samme, som hvis vi havde beregnet gradienten med det samme for indekserne fra den f\u00f8rste til \"Line\u00e6r. Dermed, hvis dataene er fordelt mellem flere datadelinger, kan gradienten beregnes uafh\u00e6ngigt p\u00e5 hver del, og derefter kan resultaterne af disse beregninger summeres for at opn\u00e5 det endelige resultat:<\/p>\n

\"Line\u00e6r<\/p>\n

Set fra implementeringssynspunktet falder dette ind under paradigmet MapReduce<\/a><\/noindex>. I hvert skridt af gradientfaldet sendes der en opgave til hver dataknode for at beregne gradienten, derefter samles de beregnede gradienter, og resultatet af deres summation bruges til at forbedre resultatet.<\/p>\n

P\u00e5 trods af den enkle implementering og muligheden for at udf\u00f8re i MapReduce-paradigmet har gradientfaldet ogs\u00e5 sine ulemper. Is\u00e6r er antallet af skridt, der kr\u00e6ves for at n\u00e5 konvergens, betydeligt h\u00f8jere sammenlignet med andre, mere specialiserede metoder.<\/p>\n

LSQR<\/h3>\n

\nLSQR<\/a><\/noindex> er endnu en metode til at l\u00f8se det stillede problem, der er velegnet til b\u00e5de genopbygning af line\u00e6r regression og l\u00f8sning af systemer af line\u00e6re ligninger. Dens hovedkarakteristika er, at den kombinerer fordelene fra matrixmetoder og den iterative tilgang. Implementeringer af denne metode kan findes i biblioteket SciPy<\/a><\/noindex>. Vi kan g\u00f8re dette, da vi har ledige porte (eller vi frig\u00f8r dem p\u00e5 forh\u00e5nd) i switchene MATLAB<\/a><\/noindex>. Beskrivelsen af denne metode vil ikke blive pr\u00e6senteret her (den kan findes i artiklen LSQR: An algorithm for sparse linear equations and sparse least squares<\/a><\/noindex>). I stedet vil der blive vist en tilgang til at tilpasse LSQR til udf\u00f8relse i et distribueret milj\u00f8.<\/p>\n

Kernen i LSQR-metoden er bidiagonalisationsproceduren<\/a><\/noindex>. Dette er en iterativ procedure, hvor hver iteration best\u00e5r af f\u00f8lgende trin:
\n\"Line\u00e6r
\n
\nMen hvis vi antager, at matrixen er vandret partitioneret, kan hver iteration pr\u00e6senteres som to MapReduce-trin. P\u00e5 denne m\u00e5de lykkes det at minimere datasendinger under hver iteration (kun vektorer med l\u00e6ngde svarende til antallet af ukendte): \"Line\u00e6r Netop denne tilgang anvendes ved implementeringen af line\u00e6r regression i<\/p>\n

\"Line\u00e6r
\n
\nApache Ignite ML Der findes mange algoritmer til genopbygning af line\u00e6r regression, men ikke alle kan anvendes under alle omst\u00e6ndigheder. QR-faktorisering egner sig glimrende til pr\u00e6cise l\u00f8sninger p\u00e5 sm\u00e5 datam\u00e6ngder. Gradientfaldet implementeres nemt og muligg\u00f8r hurtigt at finde en tiln\u00e6rmelsesl\u00f8sning. LSQR kombinerer de bedste egenskaber fra de to tidligere algoritmer, da det kan distribueres, konvergerer hurtigere i forhold til gradientfaldet og tillader tidlig stop af algoritmen i mods\u00e6tning til QR-faktorisering for at finde en tiln\u00e6rmelsesl\u00f8sning.<\/a><\/noindex>.<\/p>\n

Konklusion<\/h3>\n

\nBeta-versionen af Vivaldi-browseren er tilg\u00e6ngelig til Android
\n
Kilde: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u0418\u0441\u0442\u043e\u0447\u043d\u0438\u043a: xkcd \u041b\u0438\u043d\u0435\u0439\u043d\u0430\u044f \u0440\u0435\u0433\u0440\u0435\u0441\u0441\u0438\u044f \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u043e\u0434\u043d\u0438\u043c \u0438\u0437 \u0431\u0430\u0437\u043e\u0432\u044b\u0445 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c\u043e\u0432 \u0434\u043b\u044f \u043c\u043d\u043e\u0433\u0438\u0445 \u043e\u0431\u043b\u0430\u0441\u0442\u0435\u0439, \u0441\u0432\u044f\u0437\u0430\u043d\u043d\u044b\u0445 \u0441 \u0430\u043d\u0430\u043b\u0438\u0437\u043e\u043c \u0434\u0430\u043d\u043d\u044b\u0445. \u041f\u0440\u0438\u0447\u0438\u043d\u0430 \u044d\u0442\u043e\u043c\u0443 \u043e\u0447\u0435\u0432\u0438\u0434\u043d\u0430. \u042d\u0442\u043e \u043e\u0447\u0435\u043d\u044c \u043f\u0440\u043e\u0441\u0442\u043e\u0439 \u0438 \u043f\u043e\u043d\u044f\u0442\u043d\u044b\u0439 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c, \u0447\u0442\u043e \u0441\u043f\u043e\u0441\u043e\u0431\u0441\u0442\u0432\u0443\u0435\u0442 \u0435\u0433\u043e \u0448\u0438\u0440\u043e\u043a\u043e\u043c\u0443 \u043f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0438\u044e \u0443\u0436\u0435 \u043c\u043d\u043e\u0433\u0438\u0435 \u0434\u0435\u0441\u044f\u0442\u043a\u0438, \u0435\u0441\u043b\u0438 \u043d\u0435 \u0441\u043e\u0442\u043d\u0438, \u043b\u0435\u0442. \u0418\u0434\u0435\u044f \u0437\u0430\u043a\u043b\u044e\u0447\u0430\u0435\u0442\u0441\u044f \u0432 \u0442\u043e\u043c, \u0447\u0442\u043e \u043c\u044b \u043f\u0440\u0435\u0434\u043f\u043e\u043b\u0430\u0433\u0430\u0435\u043c \u043b\u0438\u043d\u0435\u0439\u043d\u0443\u044e \u0437\u0430\u0432\u0438\u0441\u0438\u043c\u043e\u0441\u0442\u044c \u043e\u0434\u043d\u043e\u0439 \u043f\u0435\u0440\u0435\u043c\u0435\u043d\u043d\u043e\u0439 \u043e\u0442 \u043d\u0430\u0431\u043e\u0440\u0430 \u0434\u0440\u0443\u0433\u0438\u0445 \u043f\u0435\u0440\u0435\u043c\u0435\u043d\u043d\u044b\u0445, \u0430 \u043f\u043e\u0442\u043e\u043c \u043f\u044b\u0442\u0430\u0435\u043c\u0441\u044f […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":28483,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-37955","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Line\u00e6r regression og metoder til dens genopbygning | ProHoster","description":"Kilde:","canonical_url":"https:\/\/prohoster.info\/da\/blog\/administrirovanie\/linejnaya-regressiya-i-metody-eyo-vosstanovleniya","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"da_DK","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041b\u0438\u043d\u0435\u0439\u043d\u0430\u044f \u0440\u0435\u0433\u0440\u0435\u0441\u0441\u0438\u044f \u0438 \u043c\u0435\u0442\u043e\u0434\u044b \u0435\u0451 \u0432\u043e\u0441\u0441\u0442\u0430\u043d\u043e\u0432\u043b\u0435\u043d\u0438\u044f | ProHoster","og:description":"\u0418\u0441\u0442\u043e\u0447\u043d\u0438\u043a:","og:url":"https:\/\/prohoster.info\/da\/blog\/administrirovanie\/linejnaya-regressiya-i-metody-eyo-vosstanovleniya","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:20:47+00:00","article:modified_time":"2019-10-31T19:20:47+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"37955","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-23 19:55:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 01:17:23","updated":"2026-01-23 19:55:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/posts\/37955","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/comments?post=37955"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/posts\/37955\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/media\/28483"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/media?parent=37955"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/categories?post=37955"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/tags?post=37955"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}