{"id":53777,"date":"2019-12-10T00:00:00","date_gmt":"2019-12-09T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/privodim-uravnenie-linejnoj-regressii-v-matrichnyj-vid"},"modified":"2020-02-18T14:01:42","modified_gmt":"2020-02-18T11:01:42","slug":"privodim-uravnenie-linejnoj-regressii-v-matrichnyj-vid","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/news\/privodim-uravnenie-linejnoj-regressii-v-matrichnyj-vid","title":{"rendered":"We present the equation of linear regression in matrix form.","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"We
\n
\nThe purpose of this article is to support novice data scientists. In the previous article<\/a><\/noindex> simple terms, we analyzed three methods for solving the linear regression equation: analytical solution, gradient descent, and stochastic gradient descent. For the analytical solution, we used the formula \"We. In this article, as the title suggests, we will justify the use of this formula or, in other words, derive it ourselves. <\/p>\n

Why it makes sense to pay special attention to the formula \"We? <\/p>\n

It is with the matrix equation that introductions to linear regression typically begin. However, detailed derivations of how the formula was derived are rarely encountered.
\n<\/a><\/noindex>
\nFor example, in Yandex's machine learning courses, when students are introduced to regularization, they are advised to use functions from the library sklearn<\/i>, yet there is no mention of the matrix representation of the algorithm. This is the moment when some students may feel inclined to delve deeper into the issue\u2014writing code without using the built-in functions. To achieve this, one must first represent the equation with the regularizer in matrix form. This article will indeed help those who wish to master such skills. Let's get started.<\/p>\n

Initial Conditions<\/h2>\n

<\/p>\n

Target Indicators<\/h3>\n

\nWe have a series of values for the target indicator. For example, the target indicator could be the price of an asset: oil, gold, wheat, or the dollar, etc. By a series of values of the target indicator, we mean the number of observations. Such observations could be, for instance, the monthly oil prices over a year, which means we will have 12 values for the target indicator. Let's start by introducing notations. We will denote each value of the target indicator as \"We. Overall, we have \"We observations, and thus we can represent our observations as \"We.<\/p>\n

Regressors<\/h3>\n

\nLet's assume that there are factors that, to a certain extent, explain the values of the target variable. For example, the exchange rate of the dollar\/ruble is strongly influenced by the price of oil, the Federal Reserve rate, and others. Such factors are called regressors. In this case, each value of the target variable should correspond to a value of the regressor, meaning that if we have 12 target values for each month in 2018, we should also have 12 regressor values for the same period. We denote the values of each regressor as \"We. Suppose that in our case there are \"We regressors (i.e., \"We factors that influence the values of the target variable). This means our regressors can be represented as follows: for the 1st regressor (for example, the price of oil): \"We, for the 2nd regressor (for example, the Federal Reserve rate): \"We, for the \u201c\"We-th\u201d regressor: \"We<\/p>\n

The dependence of the target variables on the regressors<\/h3>\n

\nLet\u2019s assume that the dependence of the target variable \"We from regressors \u201c\"We-th\u201d observation can be expressed through the linear regression equation of the form:<\/p>\n

\"We<\/p>\n

, where \"We \u2014 \u201c\"We-th\u201d value of the regressor from 1 to \"We,<\/p>\n

\"We - the number of regressors from 1 to \"We<\/p>\n

\"We - the coefficients that represent the magnitude by which the calculated target variable will change on average with a change in the regressor. <\/p>\n

In other words, we determine a \"unique\" coefficient for each regressor (except for \"We), then we multiply the coefficients by the values of the regressors for the \" \"We, then we multiply the coefficients by the values of the regressors \u201c\"We-th\u201d observation, resulting in some approximation \u201c\"We-th\u201d target metric.<\/p>\n

, for which the values of our approximating function \"Wewill be positioned as close as possible to the values of the target variables. \"We Assessment of the quality of the approximating function<\/p>\n

We will define the quality of the approximating function using the method of least squares. The quality assessment function will take the following form:<\/h3>\n

\nWe need to find values for the coefficients $w$, for which the value<\/p>\n

\"We<\/p>\n

will be minimal. \"We We convert the equation into matrix form<\/p>\n

Vector representation<\/h2>\n

<\/p>\n

To start, to make life easier, we should pay attention to the linear regression equation and notice that the first coefficient<\/h3>\n

\nTo start with, to make your life easier, you should pay attention to the linear regression equation and notice that the first coefficient \"We is not multiplied by any regressors. Consequently, when we convert the data into matrix form, this circumstance will seriously complicate the calculations. In this regard, it is proposed to introduce another regressor for the first coefficient \"We and set it to one. More precisely, set each \u201c\"We-th\u201d value of this regressor to one \u2014 after all, multiplying by one will not change our results, and in terms of matrix multiplication rules, it will significantly reduce our difficulties. <\/p>\n

Now, for a time, to simplify the material, let's assume that we only have one \u201c\"We-th\u201d observation. Then, we represent the values of the regressors \u201c\"We-th\u201d observation as a vector. \"We. The vector \"We has a dimension of \"We, which means \"We rows and 1 column:<\/p>\n

\"We<\/p>\n

The sought coefficients can be represented as a vector \"We, which has a dimension of \"We:<\/p>\n

\"We<\/p>\n

The linear regression equation for the \u201c\"We-th\u201d observation will take the form: <\/p>\n

\"We<\/p>\n

The function for evaluating the quality of the linear model will take the form:<\/p>\n

\"We<\/p>\n

Note that according to the rules of matrix multiplication, we needed to transpose the vector \"We.<\/p>\n

Matrix representation<\/h3>\n

\nAs a result of multiplying the vectors, we will get a number: \"We, which was expected. This number is the approximation of the \u201c\"We-th\u201d target metric. But we need the approximation not of just one target metric value, but of all. To do this, let's write all \u201c\"We-th\u201d regressors in matrix format. \"We. The resulting matrix has a dimension of \"We:<\/p>\n

\"We<\/p>\n

Now the linear regression equation will take the form:<\/p>\n

\"We<\/p>\n

Let's denote the values of the target indicators (all \"We) as a vector \"We with a dimension of \"We:<\/p>\n

\"We<\/p>\n

Now we can write the equation evaluating the quality of the linear model in matrix format:<\/p>\n

\"We<\/p>\n

In fact, from this formula, we subsequently obtain the well-known formula \"We<\/p>\n

How is this done? Parentheses are expanded, differentiation is performed, the resulting expressions are transformed, etc., and this is exactly what we will now engage in.<\/p>\n

Matrix transformations<\/h2>\n

<\/p>\n

Let\u2019s expand the parentheses<\/h3>\n

\n\"We<\/p>\n

\"We<\/p>\n

Prepare the equation for differentiation<\/h3>\n

\nFor this, we will perform some transformations. In the subsequent calculations, it will be more convenient if the vector \"We is presented at the beginning of each product in the equation.<\/p>\n

Transformation 1<\/h4>\n

\n\"We<\/p>\n

How did this happen? To answer this question, it's enough to look at the sizes of the multiplied matrices and see that we obtain a number or otherwise \"We.<\/p>\n

Let's write down the dimensions of the matrix expressions.<\/p>\n

\"We<\/p>\n

\"We<\/p>\n

\"We<\/p>\n

Transformation 2<\/h4>\n

\n\"We<\/p>\n

Let's outline similarly to transformation 1<\/p>\n

\"We<\/p>\n

\"We<\/p>\n

On the output, we obtain an equation that we need to differentiate:
\n\"We<\/p>\n

Let's differentiate the quality assessment function of the model<\/h2>\n

\nWe will differentiate with respect to the vector \"We:<\/p>\n

\"We<\/p>\n

\n\"We<\/p>\n

\"We<\/p>\n

\"We<\/p>\n

There should be no questions as to why \"We but we will consider the operations for determining the derivatives in the two other expressions in more detail.<\/p>\n

Differentiation 1<\/h3>\n

\nLet's break down the differentiation: \"We<\/p>\n

To determine the derivative of a matrix or vector, we need to look at their contents. Let's take a look:<\/p>\n

\"We<\/p>\n

\"We<\/p>\n

\"We \"We<\/p>\n

We will denote the product of matrices \"We as a matrix \"We. The matrix \"We is square and moreover, it is symmetric. These properties will be useful later, let's remember them. The matrix \"We has a dimension of \"We:<\/p>\n

\"We<\/p>\n

Now our task is to correctly multiply the vectors by the matrix and avoid getting 'two times two equals five,' so let's focus and be extremely careful.<\/p>\n

\"We<\/p>\n

\"We<\/p>\n

\"We<\/p>\n

\"We<\/p>\n

However, we ended up with an intricate expression! In fact, we have obtained a number\u2014a scalar. And now, we are truly transitioning to differentiation. We need to find the derivative of the obtained expression with respect to each coefficient \"We and we will get an output vector of size \"We. Just in case, I will describe the procedures step by step:<\/p>\n

1) differentiate with respect to \"We, we will obtain: \"We<\/p>\n

2) differentiate with respect to \"We, we will obtain: \"We<\/p>\n

3) differentiate with respect to \"We, we will obtain: \"We<\/p>\n

On output\u2014the promised vector of size \"We:<\/p>\n

\"We<\/p>\n

If we look closely at the vector, we can notice that the left and corresponding right elements of the vector can be grouped in such a way that ultimately, from the presented vector, we can extract a vector \"We of size \"We. For example, \"We (the left element of the top row of the vector) \"We (the right element of the top row of the vector) can be represented as \"We, and \"We \u2014 as \"We and so on for each row. Let's group them:<\/p>\n

\"We<\/p>\n

We will factor out the vector \"We and we will obtain:<\/p>\n

\"We<\/p>\n

Now, let's take a closer look at the resulting matrix. The matrix represents the sum of two matrices \"We:<\/p>\n

\"We<\/p>\n

Let's remember that some time ago, we pointed out one important property of the matrix \"We \u2014 it is symmetric. Based on this property, we can confidently state that the expression \"We is equal to \"We. This is easy to verify by expanding the matrix multiplication element-wise. \"WeWe won't do this here, those interested can conduct the check themselves. <\/p>\n

Let's return to our expression. After our transformations, it turned out just as we wanted to see it:<\/p>\n

\"We<\/p>\n

So, we have managed the first differentiation. Now, let's move on to the second expression.<\/p>\n

Differentiation 2<\/h3>\n

\n\"We<\/p>\n

Let's take the beaten path. It will be much shorter than the previous one, so don't stray far from the screen.<\/p>\n

Let's break down the vectors and the matrix element by element:<\/p>\n

\"We<\/p>\n

\"We<\/p>\n

\"We<\/p>\n

For now, let's remove the two from the calculations \u2014 it doesn't play a significant role, and we'll put it back later. We'll multiply the vectors by the matrix. First, we will multiply the matrix \"We by the vector \"We, here we have no restrictions. We'll get a vector of size \"We:<\/p>\n

\"We<\/p>\n

Next, we will perform the following operation \u2014 multiply the vector \"We by the resulting vector. We will end up with a number:<\/p>\n

\"We<\/p>\n

This is what we will differentiate. We will get a vector of dimension \"We:<\/p>\n

\"We<\/p>\n

Does this remind you of something? That's right! This is the product of the matrix \"We by the vector \"We. <\/p>\n

Thus, the second differentiation is successfully completed.<\/p>\n

In conclusion<\/h2>\n

\nNow we know how the equality was derived \"We.<\/p>\n

Finally, let's outline a quick way to transform the basic formulas.<\/p>\n

We will assess the model's quality according to the least squares method:<\/b>
\n\"We<\/p>\n

\"We<\/p>\n

Let's differentiate the obtained expression:<\/b>
\n\"We \"We<\/p>\n

\"We<\/p>\n

Literature<\/h2>\n

\nInternet sources:<\/b><\/u><\/p>\n

1) habr.com\/ru\/post\/278513<\/a><\/noindex><\/i>
\n2) habr.com\/ru\/company\/ods\/blog\/322076<\/a><\/noindex><\/i>
\n3) habr.com\/ru\/post\/307004<\/a><\/noindex><\/i>
\n4) nabatchikov.com\/blog\/view\/matrix_der<\/a><\/noindex><\/i><\/p>\n

Textbooks, problem sets:<\/b><\/u><\/p>\n

1) Lecture notes on higher mathematics: complete course \/ D.T. Pismennyi \u2013 4th ed. \u2013 Moscow: Iris-Press, 2006
\n2) Applied regression analysis \/ N. Draper, G. Smith \u2013 2nd ed. \u2013 Moscow: Finance and Statistics, 1986 (translated from English)
\n3) Problems on solving matrix equations:
\nfunction-x.ru\/matrix_equations.html<\/a><\/noindex>
\nmathprofi.ru\/deistviya_s_matricami.html<\/a><\/noindex><\/i>
\n
Source: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u0426\u0435\u043b\u044c \u0441\u0442\u0430\u0442\u044c\u0438 \u2014 \u043e\u043a\u0430\u0437\u0430\u043d\u0438\u0435 \u043f\u043e\u0434\u0434\u0435\u0440\u0436\u043a\u0438 \u043d\u0430\u0447\u0438\u043d\u0430\u044e\u0449\u0438\u043c \u0434\u0430\u0442\u0430\u0441\u0430\u0439\u043d\u0442\u0438\u0441\u0442\u0430\u043c. \u0412 \u043f\u0440\u0435\u0434\u044b\u0434\u0443\u0449\u0435\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u043c\u044b \u043d\u0430 \u043f\u0430\u043b\u044c\u0446\u0430\u0445 \u0440\u0430\u0437\u043e\u0431\u0440\u0430\u043b\u0438 \u0442\u0440\u0438 \u0441\u043f\u043e\u0441\u043e\u0431\u0430 \u0440\u0435\u0448\u0435\u043d\u0438\u044f \u0443\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u044f \u043b\u0438\u043d\u0435\u0439\u043d\u043e\u0439 \u0440\u0435\u0433\u0440\u0435\u0441\u0441\u0438\u0438: \u0430\u043d\u0430\u043b\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u043e\u0435 \u0440\u0435\u0448\u0435\u043d\u0438\u0435, \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043d\u044b\u0439 \u0441\u043f\u0443\u0441\u043a, \u0441\u0442\u043e\u0445\u0430\u0441\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u0433\u0440\u0430\u0434\u0438\u0435\u043d\u0442\u043d\u044b\u0439 \u0441\u043f\u0443\u0441\u043a. \u0422\u043e\u0433\u0434\u0430 \u0434\u043b\u044f \u0430\u043d\u0430\u043b\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u043e\u0433\u043e \u0440\u0435\u0448\u0435\u043d\u0438\u044f \u043c\u044b \u043f\u0440\u0438\u043c\u0435\u043d\u0438\u043b\u0438 \u0444\u043e\u0440\u043c\u0443\u043b\u0443 . \u0412 \u044d\u0442\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435, \u043a\u0430\u043a \u0441\u043b\u0435\u0434\u0443\u0435\u0442 \u0438\u0437 \u0437\u0430\u0433\u043e\u043b\u043e\u0432\u043a\u0430, \u043c\u044b \u043e\u0431\u043e\u0441\u043d\u0443\u0435\u043c \u043f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0438\u0435 \u0434\u0430\u043d\u043d\u043e\u0439 \u0444\u043e\u0440\u043c\u0443\u043b\u044b \u0438\u043b\u0438 \u0434\u0440\u0443\u0433\u0438\u043c\u0438 \u0441\u043b\u043e\u0432\u0430\u043c\u0438, \u0441\u0430\u043c\u043e\u0441\u0442\u043e\u044f\u0442\u0435\u043b\u044c\u043d\u043e \u0435\u0435 \u0432\u044b\u0432\u0435\u0434\u0435\u043c. \u041f\u043e\u0447\u0435\u043c\u0443 \u0438\u043c\u0435\u0435\u0442 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-53777","post","type-post","status-publish","format-standard","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47We bring the linear regression equation into matrix form | ProHoster","description":"The purpose of the article is to provide support.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/news\/privodim-uravnenie-linejnoj-regressii-v-matrichnyj-vid","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041f\u0440\u0438\u0432\u043e\u0434\u0438\u043c \u0443\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u0435 \u043b\u0438\u043d\u0435\u0439\u043d\u043e\u0439 \u0440\u0435\u0433\u0440\u0435\u0441\u0441\u0438\u0438 \u0432 \u043c\u0430\u0442\u0440\u0438\u0447\u043d\u044b\u0439 \u0432\u0438\u0434 | ProHoster","og:description":"\u0426\u0435\u043b\u044c \u0441\u0442\u0430\u0442\u044c\u0438 \u2014 \u043e\u043a\u0430\u0437\u0430\u043d\u0438\u0435 \u043f\u043e\u0434\u0434\u0435\u0440\u0436\u043a\u0438.","og:url":"https:\/\/prohoster.info\/en\/blog\/news\/privodim-uravnenie-linejnoj-regressii-v-matrichnyj-vid","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-12-09T21:00:00+00:00","article:modified_time":"2020-02-18T11:01:42+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"53777","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 08:43:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 20:18:29","updated":"2026-01-24 08:43:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/53777","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=53777"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/53777\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=53777"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=53777"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=53777"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}