Nkhaniyi ikufotokoza njira zingapo zodziwira masamu a mzere wosavuta (wowirikiza).
Njira zonse zothetsera equation zomwe takambiranazi zimachokera ku njira ya ma squares ochepa. Taganizirani njira zotsatirazi:
- Njira yothetsera
- Kutsika kwa Gradient
- Kutsika kwa Stochastic gradient
Pa njira iliyonse yothetsera equation ya mzere wowongoka, nkhaniyo imapereka ntchito zosiyanasiyana, zomwe zimagawidwa m'magulu omwe amalembedwa popanda kugwiritsa ntchito laibulale. Chiwerengero ndi omwe amagwiritsa ntchito powerengera Chiwerengero. Amakhulupirira kuti mwaluso ntchito Chiwerengero idzachepetsa ndalama zamakompyuta.
Khodi yonse yoperekedwa m'nkhaniyo imalembedwa m'chinenerocho python 2.7 kugwiritsa ntchito Buku la Jupyter. Khodi yochokera ndi fayilo yokhala ndi data yachitsanzo imayikidwa
Nkhaniyi imayang'ana kwambiri oyamba kumene komanso omwe ayamba kale kuphunzira pang'onopang'ono kuphunzira gawo lalikulu kwambiri mu nzeru zopangira - kuphunzira pamakina.
Kuti timvetse mfundoyi, timagwiritsa ntchito chitsanzo chosavuta.
Chitsanzo mikhalidwe
Tili ndi zikhalidwe zisanu zomwe zimadziwika ndi kudalira Y ΠΎΡ X (Gulu 1):
Table No. 1 βChitsanzo cha zinthuβ
Tidzaganiza kuti zikhalidwe ndi mwezi wa chaka, ndi - ndalama mwezi uno. M'mawu ena, ndalama zimadalira mwezi wa chaka, ndi - chizindikiro chokha chomwe ndalama zimadalira.
Chitsanzo ndi chotero-choncho, ponse pakuwona kudalira kovomerezeka kwa ndalama pa mwezi wa chaka, komanso kuchokera ku chiwerengero cha zikhalidwe - ndizochepa kwambiri. Komabe, kuphweka koteroko kumapangitsa kuti zikhale zotheka, monga akunena, kufotokoza, osati momasuka nthawi zonse, zinthu zomwe oyamba kumene amatengera. Komanso kuphweka kwa manambala kudzalola omwe akufuna kuthetsa chitsanzo pamapepala popanda ndalama zambiri zogwirira ntchito.
Tiyeni tiyerekeze kuti kudalira koperekedwa mu chitsanzo kungathe kuyerekezedwa bwino ndi masamu a equation ya mzere wosavuta (wowirikiza) wa mawonekedwe:
kumene ndi mwezi womwe ndalama zidalandilidwa; - ndalama zogwirizana ndi mwezi, ΠΈ ndi ma regression coefficients a mzere woyerekeza.
Onani kuti coefficient nthawi zambiri amatchedwa otsetsereka kapena gradient ya mzere woyerekeza; imayimira ndalama zomwe a zikasintha .
Mwachiwonekere, ntchito yathu mu chitsanzo ndikusankha ma coefficients mu equation ΠΈ , pomwe kupatuka kwa ndalama zomwe tawerengera mwezi uliwonse kuchokera pamayankho owona, i.e. Makhalidwe omwe aperekedwa pachitsanzo adzakhala ochepa.
Njira yocheperako
Malingana ndi njira yaing'ono ya ma squares, kupatukako kuyenera kuwerengeredwa ndi squaring. Njirayi imakuthandizani kuti mupewe kuchotseratu zopatuka ngati ali ndi zizindikiro zotsutsana. Mwachitsanzo, ngati nthawi ina, kupatuka ndiko +5 (kuphatikiza zisanu), ndi zina -5 (kuchotsa zisanu), ndiye kuchuluka kwa zopatukako kuzichotsa ndikufika ku 0 (ziro). Ndizotheka kuti musaphatikizepo kupotoka, koma kugwiritsa ntchito malo a modulus ndiye kuti zopatuka zonse zizikhala zabwino ndikuunjikana. Sitidzakambirana mwatsatanetsatane mfundoyi, koma zingosonyeza kuti kuti mawerengedwe azitha kuwerengera, ndi chizolowezi kupotoza.
Umu ndi momwe fomula imawonekera momwe tingadziwire kuchuluka kwapang'onopang'ono (zolakwika):
kumene ndi ntchito yoyerekeza mayankho owona (ndiko kuti, ndalama zomwe tidawerengera),
ndi mayankho owona (ndalama zomwe zaperekedwa pachitsanzo),
ndiye chitsanzo (chiwerengero cha mwezi womwe kupatukako kwatsimikiziridwa)
Tiyeni tisiyanitse ntchitoyo, tifotokoze ma equation ang'onoang'ono, ndikukhala okonzeka kupita ku yankho la kusanthula. Koma choyamba, tiyeni titenge ulendo waufupi wokhudza kusiyanitsa ndi kukumbukira tanthauzo la geometric la mawuwo.
Kusiyanitsa
Kusiyanitsa ndiko kugwira ntchito kopeza chochokera ku ntchito.
Kodi chotulukapo chimagwiritsidwa ntchito chiyani? Zomwe zimachokera ku ntchito zimasonyeza kuchuluka kwa kusintha kwa ntchitoyo ndipo imatiuza momwe imayendera. Ngati zotumphukira pamfundo yomwe wapatsidwa zili zabwino, ndiye kuti ntchitoyo imawonjezeka; apo ayi, ntchitoyo imachepa. Ndipo mtengo wamtengo wapatali wamtundu wamtundu uliwonse, umakwera kwambiri kusintha kwa ntchito, komanso kutsetsereka kwa graph ya ntchito.
Mwachitsanzo, pansi pa zikhalidwe za Cartesian coordinate system, mtengo wa zotumphukira pamfundo M (0,0) ndi wofanana ndi + 25 zikutanthauza kuti pa nthawi yopatsidwa, pamene mtengo umasinthidwa kumanja ndi gawo wamba, mtengo kumawonjezeka ndi 25 mayunitsi ochiritsira. Pa grafu ikuwoneka ngati kukwera kwakukulu kwamtengo wapatali kuchokera ku mfundo yopatsidwa.
Chitsanzo china. Mtengo wotengera ndi wofanana -0,1 zikutanthauza kuti akasamutsidwa pagawo limodzi lokhazikika, mtengo amachepetsa ndi 0,1 kokha unit ochiritsira. Nthawi yomweyo, pa graph ya ntchitoyo, titha kuwona kutsetsereka kotsika komwe kumawonekera. Kujambula fanizo ndi phiri, zimakhala ngati tikutsika pang'onopang'ono potsetsereka kuchokera paphiri, mosiyana ndi chitsanzo cham'mbuyomo, pomwe timayenera kukwera nsonga zazitali kwambiri :)
Choncho, pambuyo kusiyanitsa ntchito mwa zovuta ΠΈ , timatanthauzira ma equation osiyanitsa a dongosolo loyamba. Pambuyo pozindikira ma equation, tidzalandira dongosolo la ma equation awiri, pothana ndi zomwe titha kusankha ma coefficients awa. ΠΈ , pomwe zikhalidwe za zotengera zomwe zimagwirizana pamfundo zomwe zapatsidwa zimasintha ndi pang'ono kwambiri, ndipo ngati yankho la kusanthula silisintha konse. Mwa kuyankhula kwina, ntchito yolakwika pa ma coefficients omwe apezeka idzafika pang'onopang'ono, chifukwa zitsulo zotsalira pang'ono pazigawozi zidzakhala zofanana ndi zero.
Chifukwa chake, molingana ndi malamulo amasiyanidwe, gawo lotengera gawo la dongosolo la 1 pokhudzana ndi coefficient. adzatenga fomu:
1st Order of partial derivative equation molingana ndi adzatenga fomu:
Zotsatira zake, tidalandira kachitidwe ka ma equation omwe ali ndi njira yowunikira yosavuta:
kuyamba{equation*}
kuyamba{cases}
ndi + bsumlimits_{i=1}^nx_i - sumlimits_{i=1}^ny_i = 0
sumlimits_{i=1}^nx_i(a +bsumlimits_{i=1}^nx_i - sumlimits_{i=1}^ny_i) = 0
mapeto {zochitika}
mapeto{equation*}
Tisanathe kuthetsa equation, tiyeni tiyikenso, fufuzani ngati kutsitsa kuli kolondola, ndikusintha deta.
Kutsegula ndikusintha data
Tiyenera kuzindikira kuti chifukwa chakuti yankho la analytics, ndipo kenako kutsika kwa gradient ndi stochastic gradient, tidzagwiritsa ntchito codeyi mumitundu iwiri: kugwiritsa ntchito laibulale. Chiwerengero ndipo popanda kuzigwiritsa ntchito, ndiye kuti tidzafunika masanjidwe oyenera a data (onani code).
Kutsitsa kwa data ndikukonza code
# ΠΈΠΌΠΏΠΎΡΡΠΈΡΡΠ΅ΠΌ Π²ΡΠ΅ Π½ΡΠΆΠ½ΡΠ΅ Π½Π°ΠΌ Π±ΠΈΠ±Π»ΠΈΠΎΡΠ΅ΠΊΠΈ
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import math
import pylab as pl
import random
# Π³ΡΠ°ΡΠΈΠΊΠΈ ΠΎΡΠΎΠ±ΡΠ°Π·ΠΈΠΌ Π² Jupyter
%matplotlib inline
# ΡΠΊΠ°ΠΆΠ΅ΠΌ ΡΠ°Π·ΠΌΠ΅Ρ Π³ΡΠ°ΡΠΈΠΊΠΎΠ²
from pylab import rcParams
rcParams['figure.figsize'] = 12, 6
# ΠΎΡΠΊΠ»ΡΡΠΈΠΌ ΠΏΡΠ΅Π΄ΡΠΏΡΠ΅ΠΆΠ΄Π΅Π½ΠΈΡ Anaconda
import warnings
warnings.simplefilter('ignore')
# Π·Π°Π³ΡΡΠ·ΠΈΠΌ Π·Π½Π°ΡΠ΅Π½ΠΈΡ
table_zero = pd.read_csv('data_example.txt', header=0, sep='t')
# ΠΏΠΎΡΠΌΠΎΡΡΠΈΠΌ ΠΈΠ½ΡΠΎΡΠΌΠ°ΡΠΈΡ ΠΎ ΡΠ°Π±Π»ΠΈΡΠ΅ ΠΈ Π½Π° ΡΠ°ΠΌΡ ΡΠ°Π±Π»ΠΈΡΡ
print table_zero.info()
print '********************************************'
print table_zero
print '********************************************'
# ΠΏΠΎΠ΄Π³ΠΎΡΠΎΠ²ΠΈΠΌ Π΄Π°Π½Π½ΡΠ΅ Π±Π΅Π· ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΡ NumPy
x_us = []
[x_us.append(float(i)) for i in table_zero['x']]
print x_us
print type(x_us)
print '********************************************'
y_us = []
[y_us.append(float(i)) for i in table_zero['y']]
print y_us
print type(y_us)
print '********************************************'
# ΠΏΠΎΠ΄Π³ΠΎΡΠΎΠ²ΠΈΠΌ Π΄Π°Π½Π½ΡΠ΅ Ρ ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅ΠΌ NumPy
x_np = table_zero[['x']].values
print x_np
print type(x_np)
print x_np.shape
print '********************************************'
y_np = table_zero[['y']].values
print y_np
print type(y_np)
print y_np.shape
print '********************************************'
Kuwonetseratu
Tsopano, titatha, choyamba, kukweza deta, kachiwiri, kuyang'ana kulondola kwa kutsitsa ndikuyika deta, tidzachita zowonetseratu. Njira yomwe imagwiritsidwa ntchito nthawi zambiri ndi iyi pawiri malaibulale Nyanja. Mu chitsanzo chathu, chifukwa cha chiwerengero chochepa, palibe chifukwa chogwiritsa ntchito laibulale Nyanja. Tidzagwiritsa ntchito laibulale yokhazikika Matlotlib ndipo tangoyang'anani pa chiwembucho.
Scatterplot kodi
print 'ΠΡΠ°ΡΠΈΠΊ β1 "ΠΠ°Π²ΠΈΡΠΈΠΌΠΎΡΡΡ Π²ΡΡΡΡΠΊΠΈ ΠΎΡ ΠΌΠ΅ΡΡΡΠ° Π³ΠΎΠ΄Π°"'
plt.plot(x_us,y_us,'o',color='green',markersize=16)
plt.xlabel('$Months$', size=16)
plt.ylabel('$Sales$', size=16)
plt.show()
Tchati Na. 1 βKudalira ndalama za mwezi wapachakaβ
Njira yothetsera
Tiyeni tigwiritse ntchito zida zomwe zimakonda kwambiri python ndi kuthetsa dongosolo la equations:
kuyamba{equation*}
kuyamba{cases}
ndi + bsumlimits_{i=1}^nx_i - sumlimits_{i=1}^ny_i = 0
sumlimits_{i=1}^nx_i(a +bsumlimits_{i=1}^nx_i - sumlimits_{i=1}^ny_i) = 0
mapeto {zochitika}
mapeto{equation*}
Malinga ndi lamulo la Cramer tipeza general determinant, komanso determinants ndi ndi mwa , pambuyo pake, kugawa determinant ndi kwa general determinant - pezani coefficient , mofananamo timapeza coefficient .
Analytical solution code
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ Π΄Π»Ρ ΡΠ°ΡΡΠ΅ΡΠ° ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ² a ΠΈ b ΠΏΠΎ ΠΏΡΠ°Π²ΠΈΠ»Ρ ΠΡΠ°ΠΌΠ΅ΡΠ°
def Kramer_method (x,y):
# ΡΡΠΌΠΌΠ° Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ (Π²ΡΠ΅ ΠΌΠ΅ΡΡΡΠ°)
sx = sum(x)
# ΡΡΠΌΠΌΠ° ΠΈΡΡΠΈΠ½Π½ΡΡ
ΠΎΡΠ²Π΅ΡΠΎΠ² (Π²ΡΡΡΡΠΊΠ° Π·Π° Π²Π΅ΡΡ ΠΏΠ΅ΡΠΈΠΎΠ΄)
sy = sum(y)
# ΡΡΠΌΠΌΠ° ΠΏΡΠΎΠΈΠ·Π²Π΅Π΄Π΅Π½ΠΈΡ Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ Π½Π° ΠΈΡΡΠΈΠ½Π½ΡΠ΅ ΠΎΡΠ²Π΅ΡΡ
list_xy = []
[list_xy.append(x[i]*y[i]) for i in range(len(x))]
sxy = sum(list_xy)
# ΡΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ
list_x_sq = []
[list_x_sq.append(x[i]**2) for i in range(len(x))]
sx_sq = sum(list_x_sq)
# ΠΊΠΎΠ»ΠΈΡΠ΅ΡΡΠ²ΠΎ Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ
n = len(x)
# ΠΎΠ±ΡΠΈΠΉ ΠΎΠΏΡΠ΅Π΄Π΅Π»ΠΈΡΠ΅Π»Ρ
det = sx_sq*n - sx*sx
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΠΈΡΠ΅Π»Ρ ΠΏΠΎ a
det_a = sx_sq*sy - sx*sxy
# ΠΈΡΠΊΠΎΠΌΡΠΉ ΠΏΠ°ΡΠ°ΠΌΠ΅ΡΡ a
a = (det_a / det)
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΠΈΡΠ΅Π»Ρ ΠΏΠΎ b
det_b = sxy*n - sy*sx
# ΠΈΡΠΊΠΎΠΌΡΠΉ ΠΏΠ°ΡΠ°ΠΌΠ΅ΡΡ b
b = (det_b / det)
# ΠΊΠΎΠ½ΡΡΠΎΠ»ΡΠ½ΡΠ΅ Π·Π½Π°ΡΠ΅Π½ΠΈΡ (ΠΏΡΠΎΠΎΠ²Π΅ΡΠΊΠ°)
check1 = (n*b + a*sx - sy)
check2 = (b*sx + a*sx_sq - sxy)
return [round(a,4), round(b,4)]
# Π·Π°ΠΏΡΡΡΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ ΠΈ Π·Π°ΠΏΠΈΡΠ΅ΠΌ ΠΏΡΠ°Π²ΠΈΠ»ΡΠ½ΡΠ΅ ΠΎΡΠ²Π΅ΡΡ
ab_us = Kramer_method(x_us,y_us)
a_us = ab_us[0]
b_us = ab_us[1]
print ' 33[1m' + ' 33[4m' + "ΠΠΏΡΠΈΠΌΠ°Π»ΡΠ½ΡΠ΅ Π·Π½Π°ΡΠ΅Π½ΠΈΡ ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ² a ΠΈ b:" + ' 33[0m'
print 'a =', a_us
print 'b =', b_us
print
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ Π΄Π»Ρ ΠΏΠΎΠ΄ΡΡΠ΅ΡΠ° ΡΡΠΌΠΌΡ ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΈΠ±ΠΎΠΊ
def errors_sq_Kramer_method(answers,x,y):
list_errors_sq = []
for i in range(len(x)):
err = (answers[0] + answers[1]*x[i] - y[i])**2
list_errors_sq.append(err)
return sum(list_errors_sq)
# Π·Π°ΠΏΡΡΡΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ ΠΈ Π·Π°ΠΏΠΈΡΠ΅ΠΌ Π·Π½Π°ΡΠ΅Π½ΠΈΠ΅ ΠΎΡΠΈΠ±ΠΊΠΈ
error_sq = errors_sq_Kramer_method(ab_us,x_us,y_us)
print ' 33[1m' + ' 33[4m' + "Π‘ΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ" + ' 33[0m'
print error_sq
print
# Π·Π°ΠΌΠ΅ΡΠΈΠΌ Π²ΡΠ΅ΠΌΡ ΡΠ°ΡΡΠ΅ΡΠ°
# print ' 33[1m' + ' 33[4m' + "ΠΡΠ΅ΠΌΡ Π²ΡΠΏΠΎΠ»Π½Π΅Π½ΠΈΡ ΡΠ°ΡΡΠ΅ΡΠ° ΡΡΠΌΠΌΡ ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ:" + ' 33[0m'
# % timeit error_sq = errors_sq_Kramer_method(ab,x_us,y_us)
Nazi zomwe tili nazo:
Chifukwa chake, zikhalidwe za ma coefficients zapezeka, kuchuluka kwapang'onopang'ono kwapang'onopang'ono kwakhazikitsidwa. Tiyeni tijambule mzere wowongoka pa histogram yobalalitsa molingana ndi ma coefficients opezeka.
Regression line kodi
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ Π΄Π»Ρ ΡΠΎΡΠΌΠΈΡΠΎΠ²Π°Π½ΠΈΡ ΠΌΠ°ΡΡΠΈΠ²Π° ΡΠ°ΡΡΡΠ΅ΡΠ½ΡΡ
Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ Π²ΡΡΡΡΠΊΠΈ
def sales_count(ab,x,y):
line_answers = []
[line_answers.append(ab[0]+ab[1]*x[i]) for i in range(len(x))]
return line_answers
# ΠΏΠΎΡΡΡΠΎΠΈΠΌ Π³ΡΠ°ΡΠΈΠΊΠΈ
print 'ΠΡΡΠΈΠΊβ2 "ΠΡΠ°Π²ΠΈΠ»ΡΠ½ΡΠ΅ ΠΈ ΡΠ°ΡΡΠ΅ΡΠ½ΡΠ΅ ΠΎΡΠ²Π΅ΡΡ"'
plt.plot(x_us,y_us,'o',color='green',markersize=16, label = '$True$ $answers$')
plt.plot(x_us, sales_count(ab_us,x_us,y_us), color='red',lw=4,
label='$Function: a + bx,$ $where$ $a='+str(round(ab_us[0],2))+',$ $b='+str(round(ab_us[1],2))+'$')
plt.xlabel('$Months$', size=16)
plt.ylabel('$Sales$', size=16)
plt.legend(loc=1, prop={'size': 16})
plt.show()
Tchati nambala 2 βMayankho olondola ndi owerengekaβ
Mutha kuyang'ana graph yopatuka mwezi uliwonse. Kwa ife, sitipeza phindu lililonse kuchokera ku izo, koma tidzakwaniritsa chidwi chathu cha momwe mzere wosavuta wa regression equation umadziwonetsera kudalira kwa ndalama pa mwezi wa chaka.
Chizindikiro chapatuka
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ Π΄Π»Ρ ΡΠΎΡΠΌΠΈΡΠΎΠ²Π°Π½ΠΈΡ ΠΌΠ°ΡΡΠΈΠ²Π° ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ Π² ΠΏΡΠΎΡΠ΅Π½ΡΠ°Ρ
def error_per_month(ab,x,y):
sales_c = sales_count(ab,x,y)
errors_percent = []
for i in range(len(x)):
errors_percent.append(100*(sales_c[i]-y[i])/y[i])
return errors_percent
# ΠΏΠΎΡΡΡΠΎΠΈΠΌ Π³ΡΠ°ΡΠΈΠΊ
print 'ΠΡΠ°ΡΠΈΠΊβ3 "ΠΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΡ ΠΏΠΎ-ΠΌΠ΅ΡΡΡΠ½ΠΎ, %"'
plt.gca().bar(x_us, error_per_month(ab_us,x_us,y_us), color='brown')
plt.xlabel('Months', size=16)
plt.ylabel('Calculation error, %', size=16)
plt.show()
Tchati Na. 3 βZopatuka, %β
Osati angwiro, koma tinamaliza ntchito yathu.
Tiyeni tilembe ntchito yomwe, kuti tidziwe ma coefficients ΠΈ amagwiritsa ntchito laibulale Chiwerengero, molondola kwambiri, tidzalemba ntchito ziwiri: imodzi pogwiritsa ntchito pseudoinverse matrix (yosavomerezeka pochita, popeza ndondomekoyi ndi yovuta komanso yosakhazikika), inayo imagwiritsa ntchito matrix equation.
Nadi ya Analytical Solution (NumPy)
# Π΄Π»Ρ Π½Π°ΡΠ°Π»Π° Π΄ΠΎΠ±Π°Π²ΠΈΠΌ ΡΡΠΎΠ»Π±Π΅Ρ Ρ Π½Π΅ ΠΈΠ·ΠΌΠ΅Π½ΡΡΡΠΈΠΌΡΡ Π·Π½Π°ΡΠ΅Π½ΠΈΠ΅ΠΌ Π² 1.
# ΠΠ°Π½Π½ΡΠΉ ΡΡΠΎΠ»Π±Π΅Ρ Π½ΡΠΆΠ΅Π½ Π΄Π»Ρ ΡΠΎΠ³ΠΎ, ΡΡΠΎΠ±Ρ Π½Π΅ ΠΎΠ±ΡΠ°Π±Π°ΡΡΠ²Π°ΡΡ ΠΎΡΠ΄Π΅Π»ΡΠ½ΠΎ ΠΊΠΎΡΡΡΠΈΡΠ΅Π½Ρ a
vector_1 = np.ones((x_np.shape[0],1))
x_np = table_zero[['x']].values # Π½Π° Π²ΡΡΠΊΠΈΠΉ ΡΠ»ΡΡΠ°ΠΉ ΠΏΡΠΈΠ²Π΅Π΄Π΅ΠΌ Π² ΠΏΠ΅ΡΠ²ΠΈΡΠ½ΡΠΉ ΡΠΎΡΠΌΠ°Ρ Π²Π΅ΠΊΡΠΎΡ x_np
x_np = np.hstack((vector_1,x_np))
# ΠΏΡΠΎΠ²Π΅ΡΠΈΠΌ ΡΠΎ, ΡΡΠΎ Π²ΡΠ΅ ΡΠ΄Π΅Π»Π°Π»ΠΈ ΠΏΡΠ°Π²ΠΈΠ»ΡΠ½ΠΎ
print vector_1[0:3]
print x_np[0:3]
print '***************************************'
print
# Π½Π°ΠΏΠΈΡΠ΅ΠΌ ΡΡΠ½ΠΊΡΠΈΡ, ΠΊΠΎΡΠΎΡΠ°Ρ ΠΎΠΏΡΠ΅Π΄Π΅Π»ΡΠ΅Ρ Π·Π½Π°ΡΠ΅Π½ΠΈΡ ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ² a ΠΈ b Ρ ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅ΠΌ ΠΏΡΠ΅Π²Π΄ΠΎΠΎΠ±ΡΠ°ΡΠ½ΠΎΠΉ ΠΌΠ°ΡΡΠΈΡΡ
def pseudoinverse_matrix(X, y):
# Π·Π°Π΄Π°Π΅ΠΌ ΡΠ²Π½ΡΠΉ ΡΠΎΡΠΌΠ°Ρ ΠΌΠ°ΡΡΠΈΡΡ ΠΏΡΠΈΠ·Π½Π°ΠΊΠΎΠ²
X = np.matrix(X)
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΡΠ΅ΠΌ ΡΡΠ°Π½ΡΠΏΠΎΠ½ΠΈΡΠΎΠ²Π°Π½Π½ΡΡ ΠΌΠ°ΡΡΠΈΡΡ
XT = X.T
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΡΠ΅ΠΌ ΠΊΠ²Π°Π΄ΡΠ°ΡΠ½ΡΡ ΠΌΠ°ΡΡΠΈΡΡ
XTX = XT*X
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΡΠ΅ΠΌ ΠΏΡΠ΅Π²Π΄ΠΎΠΎΠ±ΡΠ°ΡΠ½ΡΡ ΠΌΠ°ΡΡΠΈΡΡ
inv = np.linalg.pinv(XTX)
# Π·Π°Π΄Π°Π΅ΠΌ ΡΠ²Π½ΡΠΉ ΡΠΎΡΠΌΠ°Ρ ΠΌΠ°ΡΡΠΈΡΡ ΠΎΡΠ²Π΅ΡΠΎΠ²
y = np.matrix(y)
# Π½Π°Ρ
ΠΎΠ΄ΠΈΠΌ Π²Π΅ΠΊΡΠΎΡ Π²Π΅ΡΠΎΠ²
return (inv*XT)*y
# Π·Π°ΠΏΡΡΡΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ
ab_np = pseudoinverse_matrix(x_np, y_np)
print ab_np
print '***************************************'
print
# Π½Π°ΠΏΠΈΡΠ΅ΠΌ ΡΡΠ½ΠΊΡΠΈΡ, ΠΊΠΎΡΠΎΡΠ°Ρ ΠΈΡΠΏΠΎΠ»ΡΠ·ΡΠ΅Ρ Π΄Π»Ρ ΡΠ΅ΡΠ΅Π½ΠΈΡ ΠΌΠ°ΡΡΠΈΡΠ½ΠΎΠ΅ ΡΡΠ°Π²Π½Π΅Π½ΠΈΠ΅
def matrix_equation(X,y):
a = np.dot(X.T, X)
b = np.dot(X.T, y)
return np.linalg.solve(a, b)
# Π·Π°ΠΏΡΡΡΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ
ab_np = matrix_equation(x_np,y_np)
print ab_np
Tiyeni tifananize nthawi yomwe idagwiritsidwa ntchito pozindikira ma coefficients ΠΈ , molingana ndi njira zitatu zoperekedwa.
Khodi yowerengera nthawi yowerengera
print ' 33[1m' + ' 33[4m' + "ΠΡΠ΅ΠΌΡ Π²ΡΠΏΠΎΠ»Π½Π΅Π½ΠΈΡ ΡΠ°ΡΡΠ΅ΡΠ° ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ² Π±Π΅Π· ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΡ Π±ΠΈΠ±Π»ΠΈΠΎΡΠ΅ΠΊΠΈ NumPy:" + ' 33[0m'
% timeit ab_us = Kramer_method(x_us,y_us)
print '***************************************'
print
print ' 33[1m' + ' 33[4m' + "ΠΡΠ΅ΠΌΡ Π²ΡΠΏΠΎΠ»Π½Π΅Π½ΠΈΡ ΡΠ°ΡΡΠ΅ΡΠ° ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ² Ρ ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅ΠΌ ΠΏΡΠ΅Π²Π΄ΠΎΠΎΠ±ΡΠ°ΡΠ½ΠΎΠΉ ΠΌΠ°ΡΡΠΈΡΡ:" + ' 33[0m'
%timeit ab_np = pseudoinverse_matrix(x_np, y_np)
print '***************************************'
print
print ' 33[1m' + ' 33[4m' + "ΠΡΠ΅ΠΌΡ Π²ΡΠΏΠΎΠ»Π½Π΅Π½ΠΈΡ ΡΠ°ΡΡΠ΅ΡΠ° ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ² Ρ ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅ΠΌ ΠΌΠ°ΡΡΠΈΡΠ½ΠΎΠ³ΠΎ ΡΡΠ°Π²Π½Π΅Π½ΠΈΡ:" + ' 33[0m'
%timeit ab_np = matrix_equation(x_np, y_np)
Ndi deta yaying'ono, ntchito "yodzilemba yokha" imatuluka patsogolo, yomwe imapeza ma coefficients pogwiritsa ntchito njira ya Cramer.
Tsopano mutha kupita ku njira zina zopezera ma coefficients ΠΈ .
Kutsika kwa Gradient
Choyamba, tiyeni tifotokoze chomwe gradient ndi. Mwachidule, gradient ndi gawo lomwe limawonetsa momwe ntchito ikukulirakulira. Mwa fanizo ndi kukwera phiri, kumene gradient nkhope ndi pamene kukwera pamwamba pa phiri. Kukulitsa chitsanzo ndi phirili, timakumbukira kuti kwenikweni timafunikira kutsika kotsetsereka kwambiri kuti tifike kumtunda mwamsanga, ndiko kuti, osachepera - malo omwe ntchitoyo simawonjezeka kapena kuchepa. Panthawi imeneyi, chotulukacho chidzakhala chofanana ndi zero. Chifukwa chake, sitifunikira gradient, koma antigradient. Kuti mupeze antigradient muyenera kuchulukitsa gradient ndi -1 (kuchotsa chimodzi).
Tiyeni tiyang'ane pa mfundo yakuti ntchito ikhoza kukhala ndi minima angapo, ndipo atatsikira mu imodzi mwa izo pogwiritsa ntchito ndondomeko yomwe ili pansipa, sitingathe kupeza zina zochepa, zomwe zingakhale zotsika kuposa zomwe zapezeka. Tiyeni tipumule, izi sizowopsa kwa ife! Kwa ife tikuchita ndi zochepa zochepa, popeza ntchito yathu pa graph ndi parabola wokhazikika. Ndipo monga tonse tiyenera kudziwa bwino kuchokera ku maphunziro athu a masamu akusukulu, parabola ili ndi gawo limodzi lochepa.
Titadziwa chifukwa chake timafunikira chowongolera, komanso kuti gradient ndi gawo, ndiye kuti, vekitala yokhala ndi zolumikizira zoperekedwa, zomwe ndizofanana ndendende ma coefficients. ΠΈ tikhoza kugwiritsa ntchito gradient kutsika.
Ndisanayambe, ndikupangira kuti ndiwerenge ziganizo zingapo za algorithm yotsika:
- Timazindikira m'njira yabodza-mwachisawawa ma coefficients ΠΈ . Mu chitsanzo chathu, tiwona ma coefficients pafupi ndi zero. Izi ndizofala, koma nkhani iliyonse ikhoza kukhala ndi machitidwe ake.
- Kuchokera ku coordinate chotsani mtengo wa 1st Order of partial derivative pamfundoyi . Choncho, ngati zotumphukira zili zabwino, ndiye kuti ntchitoyo imawonjezeka. Choncho, pochotsa mtengo wa chotengeracho, tidzayenda mosiyana ndi kukula, ndiko kuti, kutsika. Ngati chotengeracho chili choyipa, ndiye kuti ntchitoyo panthawiyi imachepa ndipo pochotsa mtengo wamtengo wapatali timasunthira kumbali yotsika.
- Timagwira ntchito yofanana ndi coordinate : chotsani mtengo wa zotumphukira pang'ono pamfundoyo .
- Kuti musalumphe pazing'onozing'ono ndikuwulukira mumlengalenga, m'pofunika kukhazikitsa kukula kwa sitepe yopita kumtunda. Nthawi zambiri, mutha kulemba nkhani yonse ya momwe mungakhazikitsire masitepe molondola komanso momwe mungasinthire panthawi yotsika kuti muchepetse ndalama zowerengera. Koma tsopano tili ndi ntchito yosiyana pang'ono patsogolo pathu, ndipo tidzakhazikitsa kukula kwa sitepe pogwiritsa ntchito njira ya sayansi ya "poke" kapena, monga amanenera m'mawu wamba, empirically.
- Kamodzi ife tachokera makonzedwe anapatsidwa ΠΈ Chotsani zikhalidwe zomwe zimachokera, timapeza ma coordinates atsopano ΠΈ . Timatenga sitepe yotsatira (kuchotsa), kale kuchokera kumagulu owerengeka. Ndipo kotero kuzungulira kumayamba mobwerezabwereza, mpaka kuyanjana kofunikira kukwaniritsidwa.
Zonse! Tsopano takonzeka kupita kukasaka chigwa chakuya kwambiri cha Mariana Trench. Tiyeni tiyambe.
Code ya kutsika kwa gradient
# Π½Π°ΠΏΠΈΡΠ΅ΠΌ ΡΡΠ½ΠΊΡΠΈΡ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠ³ΠΎ ΡΠΏΡΡΠΊΠ° Π±Π΅Π· ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΡ Π±ΠΈΠ±Π»ΠΈΠΎΡΠ΅ΠΊΠΈ NumPy.
# Π€ΡΠ½ΠΊΡΠΈΡ Π½Π° Π²Ρ
ΠΎΠ΄ ΠΏΡΠΈΠ½ΠΈΠΌΠ°Π΅Ρ Π΄ΠΈΠ°ΠΏΠ°Π·ΠΎΠ½Ρ Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ x,y, Π΄Π»ΠΈΠ½Ρ ΡΠ°Π³Π° (ΠΏΠΎ ΡΠΌΠΎΠ»ΡΠ°Π½ΠΈΡ=0,1), Π΄ΠΎΠΏΡΡΡΠΈΠΌΡΡ ΠΏΠΎΠ³ΡΠ΅ΡΠ½ΠΎΡΡΡ(tolerance)
def gradient_descent_usual(x_us,y_us,l=0.1,tolerance=0.000000000001):
# ΡΡΠΌΠΌΠ° Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ (Π²ΡΠ΅ ΠΌΠ΅ΡΡΡΠ°)
sx = sum(x_us)
# ΡΡΠΌΠΌΠ° ΠΈΡΡΠΈΠ½Π½ΡΡ
ΠΎΡΠ²Π΅ΡΠΎΠ² (Π²ΡΡΡΡΠΊΠ° Π·Π° Π²Π΅ΡΡ ΠΏΠ΅ΡΠΈΠΎΠ΄)
sy = sum(y_us)
# ΡΡΠΌΠΌΠ° ΠΏΡΠΎΠΈΠ·Π²Π΅Π΄Π΅Π½ΠΈΡ Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ Π½Π° ΠΈΡΡΠΈΠ½Π½ΡΠ΅ ΠΎΡΠ²Π΅ΡΡ
list_xy = []
[list_xy.append(x_us[i]*y_us[i]) for i in range(len(x_us))]
sxy = sum(list_xy)
# ΡΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ
list_x_sq = []
[list_x_sq.append(x_us[i]**2) for i in range(len(x_us))]
sx_sq = sum(list_x_sq)
# ΠΊΠΎΠ»ΠΈΡΠ΅ΡΡΠ²ΠΎ Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ
num = len(x_us)
# Π½Π°ΡΠ°Π»ΡΠ½ΡΠ΅ Π·Π½Π°ΡΠ΅Π½ΠΈΡ ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ², ΠΎΠΏΡΠ΅Π΄Π΅Π»Π΅Π½Π½ΡΠ΅ ΠΏΡΠ΅Π²Π΄ΠΎΡΠ»ΡΡΠ°ΠΉΠ½ΡΠΌ ΠΎΠ±ΡΠ°Π·ΠΎΠΌ
a = float(random.uniform(-0.5, 0.5))
b = float(random.uniform(-0.5, 0.5))
# ΡΠΎΠ·Π΄Π°Π΅ΠΌ ΠΌΠ°ΡΡΠΈΠ² Ρ ΠΎΡΠΈΠ±ΠΊΠ°ΠΌΠΈ, Π΄Π»Ρ ΡΡΠ°ΡΡΠ° ΠΈΡΠΏΠΎΠ»ΡΠ·ΡΠ΅ΠΌ Π·Π½Π°ΡΠ΅Π½ΠΈΡ 1 ΠΈ 0
# ΠΏΠΎΡΠ»Π΅ Π·Π°Π²Π΅ΡΡΠ΅Π½ΠΈΡ ΡΠΏΡΡΠΊΠ° ΡΡΠ°ΡΡΠΎΠ²ΡΠ΅ Π·Π½Π°ΡΠ΅Π½ΠΈΡ ΡΠ΄Π°Π»ΠΈΠΌ
errors = [1,0]
# Π·Π°ΠΏΡΡΠΊΠ°Π΅ΠΌ ΡΠΈΠΊΠ» ΡΠΏΡΡΠΊΠ°
# ΡΠΈΠΊΠ» ΡΠ°Π±ΠΎΡΠ°Π΅Ρ Π΄ΠΎ ΡΠ΅Ρ
ΠΏΠΎΡ, ΠΏΠΎΠΊΠ° ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠ΅ ΠΏΠΎΡΠ»Π΅Π΄Π½Π΅ΠΉ ΠΎΡΠΈΠ±ΠΊΠΈ ΡΡΠΌΠΌΡ ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡ ΠΏΡΠ΅Π΄ΡΠ΄ΡΡΠ΅ΠΉ, Π½Π΅ Π±ΡΠ΄Π΅Ρ ΠΌΠ΅Π½ΡΡΠ΅ tolerance
while abs(errors[-1]-errors[-2]) > tolerance:
a_step = a - l*(num*a + b*sx - sy)/num
b_step = b - l*(a*sx + b*sx_sq - sxy)/num
a = a_step
b = b_step
ab = [a,b]
errors.append(errors_sq_Kramer_method(ab,x_us,y_us))
return (ab),(errors[2:])
# Π·Π°ΠΏΠΈΡΠ΅ΠΌ ΠΌΠ°ΡΡΠΈΠ² Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ
list_parametres_gradient_descence = gradient_descent_usual(x_us,y_us,l=0.1,tolerance=0.000000000001)
print ' 33[1m' + ' 33[4m' + "ΠΠ½Π°ΡΠ΅Π½ΠΈΡ ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ² a ΠΈ b:" + ' 33[0m'
print 'a =', round(list_parametres_gradient_descence[0][0],3)
print 'b =', round(list_parametres_gradient_descence[0][1],3)
print
print ' 33[1m' + ' 33[4m' + "Π‘ΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ:" + ' 33[0m'
print round(list_parametres_gradient_descence[1][-1],3)
print
print ' 33[1m' + ' 33[4m' + "ΠΠΎΠ»ΠΈΡΠ΅ΡΡΠ²ΠΎ ΠΈΡΠ΅ΡΠ°ΡΠΈΠΉ Π² Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠΌ ΡΠΏΡΡΠΊΠ΅:" + ' 33[0m'
print len(list_parametres_gradient_descence[1])
print
Tinadumphira pansi pa Mariana Trench ndipo kumeneko tidapeza ma coefficient ofanana ΠΈ , zomwe nβzimene zinali kuyembekezera.
Tiyeni tidutsenso, nthawi ino yokha, galimoto yathu yakuya yakuzama idzadzazidwa ndi matekinoloje ena, omwe ndi laibulale. Chiwerengero.
Khodi ya kutsika kwa gradient (NumPy)
# ΠΏΠ΅ΡΠ΅Π΄ ΡΠ΅ΠΌ ΠΎΠΏΡΠ΅Π΄Π΅Π»ΠΈΡΡ ΡΡΠ½ΠΊΡΠΈΡ Π΄Π»Ρ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠ³ΠΎ ΡΠΏΡΡΠΊΠ° Ρ ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅ΠΌ Π±ΠΈΠ±Π»ΠΈΠΎΡΠ΅ΠΊΠΈ NumPy,
# Π½Π°ΠΏΠΈΡΠ΅ΠΌ ΡΡΠ½ΠΊΡΠΈΡ ΠΎΠΏΡΠ΅Π΄Π΅Π»Π΅Π½ΠΈΡ ΡΡΠΌΠΌΡ ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ ΡΠ°ΠΊΠΆΠ΅ Ρ ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅ΠΌ NumPy
def error_square_numpy(ab,x_np,y_np):
y_pred = np.dot(x_np,ab)
error = y_pred - y_np
return sum((error)**2)
# Π½Π°ΠΏΠΈΡΠ΅ΠΌ ΡΡΠ½ΠΊΡΠΈΡ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠ³ΠΎ ΡΠΏΡΡΠΊΠ° Ρ ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅ΠΌ Π±ΠΈΠ±Π»ΠΈΠΎΡΠ΅ΠΊΠΈ NumPy.
# Π€ΡΠ½ΠΊΡΠΈΡ Π½Π° Π²Ρ
ΠΎΠ΄ ΠΏΡΠΈΠ½ΠΈΠΌΠ°Π΅Ρ Π΄ΠΈΠ°ΠΏΠ°Π·ΠΎΠ½Ρ Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ x,y, Π΄Π»ΠΈΠ½Ρ ΡΠ°Π³Π° (ΠΏΠΎ ΡΠΌΠΎΠ»ΡΠ°Π½ΠΈΡ=0,1), Π΄ΠΎΠΏΡΡΡΠΈΠΌΡΡ ΠΏΠΎΠ³ΡΠ΅ΡΠ½ΠΎΡΡΡ(tolerance)
def gradient_descent_numpy(x_np,y_np,l=0.1,tolerance=0.000000000001):
# ΡΡΠΌΠΌΠ° Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ (Π²ΡΠ΅ ΠΌΠ΅ΡΡΡΠ°)
sx = float(sum(x_np[:,1]))
# ΡΡΠΌΠΌΠ° ΠΈΡΡΠΈΠ½Π½ΡΡ
ΠΎΡΠ²Π΅ΡΠΎΠ² (Π²ΡΡΡΡΠΊΠ° Π·Π° Π²Π΅ΡΡ ΠΏΠ΅ΡΠΈΠΎΠ΄)
sy = float(sum(y_np))
# ΡΡΠΌΠΌΠ° ΠΏΡΠΎΠΈΠ·Π²Π΅Π΄Π΅Π½ΠΈΡ Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ Π½Π° ΠΈΡΡΠΈΠ½Π½ΡΠ΅ ΠΎΡΠ²Π΅ΡΡ
sxy = x_np*y_np
sxy = float(sum(sxy[:,1]))
# ΡΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ
sx_sq = float(sum(x_np[:,1]**2))
# ΠΊΠΎΠ»ΠΈΡΠ΅ΡΡΠ²ΠΎ Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ
num = float(x_np.shape[0])
# Π½Π°ΡΠ°Π»ΡΠ½ΡΠ΅ Π·Π½Π°ΡΠ΅Π½ΠΈΡ ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ², ΠΎΠΏΡΠ΅Π΄Π΅Π»Π΅Π½Π½ΡΠ΅ ΠΏΡΠ΅Π²Π΄ΠΎΡΠ»ΡΡΠ°ΠΉΠ½ΡΠΌ ΠΎΠ±ΡΠ°Π·ΠΎΠΌ
a = float(random.uniform(-0.5, 0.5))
b = float(random.uniform(-0.5, 0.5))
# ΡΠΎΠ·Π΄Π°Π΅ΠΌ ΠΌΠ°ΡΡΠΈΠ² Ρ ΠΎΡΠΈΠ±ΠΊΠ°ΠΌΠΈ, Π΄Π»Ρ ΡΡΠ°ΡΡΠ° ΠΈΡΠΏΠΎΠ»ΡΠ·ΡΠ΅ΠΌ Π·Π½Π°ΡΠ΅Π½ΠΈΡ 1 ΠΈ 0
# ΠΏΠΎΡΠ»Π΅ Π·Π°Π²Π΅ΡΡΠ΅Π½ΠΈΡ ΡΠΏΡΡΠΊΠ° ΡΡΠ°ΡΡΠΎΠ²ΡΠ΅ Π·Π½Π°ΡΠ΅Π½ΠΈΡ ΡΠ΄Π°Π»ΠΈΠΌ
errors = [1,0]
# Π·Π°ΠΏΡΡΠΊΠ°Π΅ΠΌ ΡΠΈΠΊΠ» ΡΠΏΡΡΠΊΠ°
# ΡΠΈΠΊΠ» ΡΠ°Π±ΠΎΡΠ°Π΅Ρ Π΄ΠΎ ΡΠ΅Ρ
ΠΏΠΎΡ, ΠΏΠΎΠΊΠ° ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠ΅ ΠΏΠΎΡΠ»Π΅Π΄Π½Π΅ΠΉ ΠΎΡΠΈΠ±ΠΊΠΈ ΡΡΠΌΠΌΡ ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡ ΠΏΡΠ΅Π΄ΡΠ΄ΡΡΠ΅ΠΉ, Π½Π΅ Π±ΡΠ΄Π΅Ρ ΠΌΠ΅Π½ΡΡΠ΅ tolerance
while abs(errors[-1]-errors[-2]) > tolerance:
a_step = a - l*(num*a + b*sx - sy)/num
b_step = b - l*(a*sx + b*sx_sq - sxy)/num
a = a_step
b = b_step
ab = np.array([[a],[b]])
errors.append(error_square_numpy(ab,x_np,y_np))
return (ab),(errors[2:])
# Π·Π°ΠΏΠΈΡΠ΅ΠΌ ΠΌΠ°ΡΡΠΈΠ² Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ
list_parametres_gradient_descence = gradient_descent_numpy(x_np,y_np,l=0.1,tolerance=0.000000000001)
print ' 33[1m' + ' 33[4m' + "ΠΠ½Π°ΡΠ΅Π½ΠΈΡ ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ² a ΠΈ b:" + ' 33[0m'
print 'a =', round(list_parametres_gradient_descence[0][0],3)
print 'b =', round(list_parametres_gradient_descence[0][1],3)
print
print ' 33[1m' + ' 33[4m' + "Π‘ΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ:" + ' 33[0m'
print round(list_parametres_gradient_descence[1][-1],3)
print
print ' 33[1m' + ' 33[4m' + "ΠΠΎΠ»ΠΈΡΠ΅ΡΡΠ²ΠΎ ΠΈΡΠ΅ΡΠ°ΡΠΈΠΉ Π² Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠΌ ΡΠΏΡΡΠΊΠ΅:" + ' 33[0m'
print len(list_parametres_gradient_descence[1])
print
Ma coefficient values ΠΈ wosasinthika.
Tiyeni tiwone momwe cholakwikacho chinasinthira pakutsika kwa gradient, ndiye kuti, momwe kuchuluka kwa masikweya amitundu inasinthira ndi sitepe iliyonse.
Code yokonzera kuchuluka kwa masikweya anayi
print 'ΠΡΠ°ΡΠΈΠΊβ4 "Π‘ΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ ΠΏΠΎ-ΡΠ°Π³ΠΎΠ²ΠΎ"'
plt.plot(range(len(list_parametres_gradient_descence[1])), list_parametres_gradient_descence[1], color='red', lw=3)
plt.xlabel('Steps (Iteration)', size=16)
plt.ylabel('Sum of squared deviations', size=16)
plt.show()
Graph No. 4 βKuchuluka kwa masikweya apatuka panthawi yotsikaβ
Pa graph tikuwona kuti ndi sitepe iliyonse cholakwikacho chimachepa, ndipo pambuyo pa kubwereza kobwerezabwereza timawona mzere pafupifupi wopingasa.
Pomaliza, tiyeni tiyerekeze kusiyana kwa nthawi yogwiritsira ntchito code:
Khodi yotsimikizira nthawi yowerengera kutsika kwa gradient
print ' 33[1m' + ' 33[4m' + "ΠΡΠ΅ΠΌΡ Π²ΡΠΏΠΎΠ»Π½Π΅Π½ΠΈΡ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠ³ΠΎ ΡΠΏΡΡΠΊΠ° Π±Π΅Π· ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΡ Π±ΠΈΠ±Π»ΠΈΠΎΡΠ΅ΠΊΠΈ NumPy:" + ' 33[0m'
%timeit list_parametres_gradient_descence = gradient_descent_usual(x_us,y_us,l=0.1,tolerance=0.000000000001)
print '***************************************'
print
print ' 33[1m' + ' 33[4m' + "ΠΡΠ΅ΠΌΡ Π²ΡΠΏΠΎΠ»Π½Π΅Π½ΠΈΡ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠ³ΠΎ ΡΠΏΡΡΠΊΠ° Ρ ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅ΠΌ Π±ΠΈΠ±Π»ΠΈΠΎΡΠ΅ΠΊΠΈ NumPy:" + ' 33[0m'
%timeit list_parametres_gradient_descence = gradient_descent_numpy(x_np,y_np,l=0.1,tolerance=0.000000000001)
Mwina tikuchita cholakwika, koma ndi ntchito yosavuta "yolemba kunyumba" yomwe sigwiritsa ntchito laibulale. Chiwerengero imaposa nthawi yowerengera ntchito pogwiritsa ntchito laibulale Chiwerengero.
Koma sitinayime njii, koma tikupita ku kuphunzira njira ina yosangalatsa yothanirana ndi mizere yosavuta yosinthira equation. Kumanani!
Kutsika kwa Stochastic gradient
Kuti mumvetse msanga mfundo ya ntchito ya stochastic gradient m'badwo, ndi bwino kudziwa kusiyana kwake ndi kutsika wamba gradient. Ife, pankhani ya kutsika kwa gradient, mu equations of derivatives of ΠΈ adagwiritsa ntchito ziwerengero zamakhalidwe onse ndi mayankho owona omwe amapezeka pachitsanzo (ndiko kuti, kuchuluka kwa zonse ΠΈ ). Pakutsika kwa stochastic gradient, sitidzagwiritsa ntchito zikhalidwe zonse zomwe zilipo pachitsanzo, koma m'malo mwake, sankhani zomwe zimatchedwa index index ndikugwiritsa ntchito mfundo zake.
Mwachitsanzo, ngati ndondomeko yatsimikiziridwa kukhala nambala 3 (zitatu), ndiye timatenga zikhalidwe ΠΈ , kenako timalowetsa zikhalidwezo m'magawo otengedwa ndikusankha ma coordinates atsopano. Kenako, titatsimikiza makonzedwewo, timasankhanso mwachisawawa mndandanda wa zitsanzo, m'malo mwa zikhalidwe zomwe zikugwirizana ndi indexyo muzosiyana pang'ono, ndikuzindikira makonzedwewo mwanjira yatsopano. ΠΈ ndi zina. mpaka kusanganikirana kukhale kobiriwira. Poyang'ana koyamba, zingawoneke ngati izi sizingagwire ntchito konse, koma zimatero. Ndizowona kuti ndizoyenera kudziwa kuti cholakwikacho sichimachepa ndi sitepe iliyonse, koma pali chizolowezi.
Kodi maubwino amtundu wa stochastic gradient ndi otani? Ngati kukula kwathu kwachitsanzo kuli kwakukulu kwambiri ndipo kumayesedwa muzinthu masauzande ambiri, ndiye kuti ndizosavuta kukonza, tinene, chikwi chimodzi mwachisawawa, m'malo mwachitsanzo chonse. Apa ndipamene kutsika kwa stochastic gradient kumayambira. Kwa ife, ndithudi, sitidzawona kusiyana kwakukulu.
Tiyeni tiwone khodi.
Khodi ya kutsika kwa stochastic gradient
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ ΡΡΠΎΡ
.Π³ΡΠ°Π΄.ΡΠ°Π³Π°
def stoch_grad_step_usual(vector_init, x_us, ind, y_us, l):
# Π²ΡΠ±ΠΈΡΠ°Π΅ΠΌ Π·Π½Π°ΡΠ΅Π½ΠΈΠ΅ ΠΈΠΊΡ, ΠΊΠΎΡΠΎΡΠΎΠ΅ ΡΠΎΠΎΡΠ²Π΅ΡΡΡΠ²ΡΠ΅Ρ ΡΠ»ΡΡΠ°ΠΉΠ½ΠΎΠΌΡ Π·Π½Π°ΡΠ΅Π½ΠΈΡ ΠΏΠ°ΡΠ°ΠΌΠ΅ΡΡΠ° ind
# (ΡΠΌ.Ρ-ΡΠΈΡ stoch_grad_descent_usual)
x = x_us[ind]
# ΡΠ°ΡΡΡΠΈΡΡΠ²ΡΠ°Π΅ΠΌ Π·Π½Π°ΡΠ΅Π½ΠΈΠ΅ y (Π²ΡΡΡΡΠΊΡ), ΠΊΠΎΡΠΎΡΠ°Ρ ΡΠΎΠΎΡΠ²Π΅ΡΡΡΠ²ΡΠ΅Ρ Π²ΡΠ±ΡΠ°Π½Π½ΠΎΠΌΡ Π·Π½Π°ΡΠ΅Π½ΠΈΡ x
y_pred = vector_init[0] + vector_init[1]*x_us[ind]
# Π²ΡΡΠΈΡΠ»ΡΠ΅ΠΌ ΠΎΡΠΈΠ±ΠΊΡ ΡΠ°ΡΡΠ΅ΡΠ½ΠΎΠΉ Π²ΡΡΡΡΠΊΠΈ ΠΎΡΠ½ΠΎΡΠΈΡΠ΅Π»ΡΠ½ΠΎ ΠΏΡΠ΅Π΄ΡΡΠ°Π²Π»Π΅Π½Π½ΠΎΠΉ Π² Π²ΡΠ±ΠΎΡΠΊΠ΅
error = y_pred - y_us[ind]
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΡΠ΅ΠΌ ΠΏΠ΅ΡΠ²ΡΡ ΠΊΠΎΠΎΡΠ΄ΠΈΠ½Π°ΡΡ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ° ab
grad_a = error
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΡΠ΅ΠΌ Π²ΡΠΎΡΡΡ ΠΊΠΎΠΎΡΠ΄ΠΈΠ½Π°ΡΡ ab
grad_b = x_us[ind]*error
# Π²ΡΡΠΈΡΠ»ΡΠ΅ΠΌ Π½ΠΎΠ²ΡΠΉ Π²Π΅ΠΊΡΠΎΡ ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ²
vector_new = [vector_init[0]-l*grad_a, vector_init[1]-l*grad_b]
return vector_new
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ ΡΡΠΎΡ
.Π³ΡΠ°Π΄.ΡΠΏΡΡΠΊΠ°
def stoch_grad_descent_usual(x_us, y_us, l=0.1, steps = 800):
# Π΄Π»Ρ ΡΠ°ΠΌΠΎΠ³ΠΎ Π½Π°ΡΠ°Π»Π° ΡΠ°Π±ΠΎΡΡ ΡΡΠ½ΠΊΡΠΈΠΈ Π·Π°Π΄Π°Π΄ΠΈΠΌ Π½Π°ΡΠ°Π»ΡΠ½ΡΠ΅ Π·Π½Π°ΡΠ΅Π½ΠΈΡ ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ²
vector_init = [float(random.uniform(-0.5, 0.5)), float(random.uniform(-0.5, 0.5))]
errors = []
# Π·Π°ΠΏΡΡΡΠΈΠΌ ΡΠΈΠΊΠ» ΡΠΏΡΡΠΊΠ°
# ΡΠΈΠΊΠ» ΡΠ°ΡΡΠΈΡΠ°Π½ Π½Π° ΠΎΠΏΡΠ΅Π΄Π΅Π»Π΅Π½Π½ΠΎΠ΅ ΠΊΠΎΠ»ΠΈΡΠ΅ΡΡΠ²ΠΎ ΡΠ°Π³ΠΎΠ² (steps)
for i in range(steps):
ind = random.choice(range(len(x_us)))
new_vector = stoch_grad_step_usual(vector_init, x_us, ind, y_us, l)
vector_init = new_vector
errors.append(errors_sq_Kramer_method(vector_init,x_us,y_us))
return (vector_init),(errors)
# Π·Π°ΠΏΠΈΡΠ΅ΠΌ ΠΌΠ°ΡΡΠΈΠ² Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ
list_parametres_stoch_gradient_descence = stoch_grad_descent_usual(x_us, y_us, l=0.1, steps = 800)
print ' 33[1m' + ' 33[4m' + "ΠΠ½Π°ΡΠ΅Π½ΠΈΡ ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ² a ΠΈ b:" + ' 33[0m'
print 'a =', round(list_parametres_stoch_gradient_descence[0][0],3)
print 'b =', round(list_parametres_stoch_gradient_descence[0][1],3)
print
print ' 33[1m' + ' 33[4m' + "Π‘ΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ:" + ' 33[0m'
print round(list_parametres_stoch_gradient_descence[1][-1],3)
print
print ' 33[1m' + ' 33[4m' + "ΠΠΎΠ»ΠΈΡΠ΅ΡΡΠ²ΠΎ ΠΈΡΠ΅ΡΠ°ΡΠΈΠΉ Π² ΡΡΠΎΡ
Π°ΡΡΠΈΡΠ΅ΡΠΊΠΎΠΌ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠΌ ΡΠΏΡΡΠΊΠ΅:" + ' 33[0m'
print len(list_parametres_stoch_gradient_descence[1])
Timayang'ana mosamala ma coefficients ndikudzifunsa tokha funso "Kodi izi zitha bwanji?" Tili ndi ma coefficient ena ΠΈ . Mwinamwake kutsika kwa stochastic gradient kwapeza magawo abwino kwambiri a equation? Tsoka ilo ayi. Ndikokwanira kuyang'ana kuchuluka kwapang'onopang'ono ndikuwona kuti ndi ma coefficients atsopano, cholakwikacho ndi chachikulu. Sitikufulumira kutaya mtima. Tiyeni tipange chithunzi cha kusintha kwa zolakwika.
Khodi yokonzera kuchuluka kwa masikweya anayi opatuka mumtsika wa stochastic gradient
print 'ΠΡΠ°ΡΠΈΠΊ β5 "Π‘ΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ ΠΏΠΎ-ΡΠ°Π³ΠΎΠ²ΠΎ"'
plt.plot(range(len(list_parametres_stoch_gradient_descence[1])), list_parametres_stoch_gradient_descence[1], color='red', lw=2)
plt.xlabel('Steps (Iteration)', size=16)
plt.ylabel('Sum of squared deviations', size=16)
plt.show()
Graph No. 5 "Kuchuluka kwa masikweya apatuke panthawi yotsika kwa stochastic gradient"
Kuyang'ana pa ndandanda, zonse zimagwera m'malo mwake ndipo tsopano tikonza zonse.
Ndiye chinachitika ndi chiyani? Zotsatirazi zinachitika. Tikasankha mwachisawawa mwezi, ndiye kuti ndi mwezi wosankhidwa womwe algorithm yathu ikufuna kuchepetsa cholakwika pakuwerengera ndalama. Kenaka timasankha mwezi wina ndikubwereza kuwerengera, koma timachepetsa cholakwika cha mwezi wachiwiri wosankhidwa. Tsopano kumbukirani kuti miyezi iwiri yoyambirira imapatuka kuchokera pamzere wa mzere wosavuta wa regression equation. Izi zikutanthauza kuti iliyonse mwa miyezi iwiriyi ikasankhidwa, pochepetsa zolakwika za aliyense wa iwo, ma algorithm athu amawonjezera kulakwitsa kwachitsanzo chonsecho. Ndiye titani? Yankho ndi losavuta: muyenera kuchepetsa sitepe yotsika. Kupatula apo, pochepetsa gawo lotsika, cholakwikacho chidzasiyanso "kulumpha" mmwamba ndi pansi. Kapena m'malo mwake, cholakwika cha "kudumpha" sichidzasiya, koma sichidzachita mwamsanga :) Tiyeni tiwone.
Khodi yoyendetsa SGD ndi ma increments ang'onoang'ono
# Π·Π°ΠΏΡΡΡΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ, ΡΠΌΠ΅Π½ΡΡΠΈΠ² ΡΠ°Π³ Π² 100 ΡΠ°Π· ΠΈ ΡΠ²Π΅Π»ΠΈΡΠΈΠ² ΠΊΠΎΠ»ΠΈΡΠ΅ΡΡΠ²ΠΎ ΡΠ°Π³ΠΎΠ² ΡΠΎΠΎΡΠ²Π΅ΡΡΠ²ΡΡΡΠ΅
list_parametres_stoch_gradient_descence = stoch_grad_descent_usual(x_us, y_us, l=0.001, steps = 80000)
print ' 33[1m' + ' 33[4m' + "ΠΠ½Π°ΡΠ΅Π½ΠΈΡ ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ² a ΠΈ b:" + ' 33[0m'
print 'a =', round(list_parametres_stoch_gradient_descence[0][0],3)
print 'b =', round(list_parametres_stoch_gradient_descence[0][1],3)
print
print ' 33[1m' + ' 33[4m' + "Π‘ΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ:" + ' 33[0m'
print round(list_parametres_stoch_gradient_descence[1][-1],3)
print
print ' 33[1m' + ' 33[4m' + "ΠΠΎΠ»ΠΈΡΠ΅ΡΡΠ²ΠΎ ΠΈΡΠ΅ΡΠ°ΡΠΈΠΉ Π² ΡΡΠΎΡ
Π°ΡΡΠΈΡΠ΅ΡΠΊΠΎΠΌ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠΌ ΡΠΏΡΡΠΊΠ΅:" + ' 33[0m'
print len(list_parametres_stoch_gradient_descence[1])
print 'ΠΡΠ°ΡΠΈΠΊ β6 "Π‘ΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ ΠΏΠΎ-ΡΠ°Π³ΠΎΠ²ΠΎ"'
plt.plot(range(len(list_parametres_stoch_gradient_descence[1])), list_parametres_stoch_gradient_descence[1], color='red', lw=2)
plt.xlabel('Steps (Iteration)', size=16)
plt.ylabel('Sum of squared deviations', size=16)
plt.show()
Graph No. 6 "Kuchuluka kwa masikweya apatuka panthawi yotsika kwambiri (masitepe 80)"
Ma coefficients asintha, koma sali abwino. Mwachinyengo, izi zitha kukonzedwa motere. Timasankha, mwachitsanzo, muzobwereza 1000 zomaliza zowerengera za coefficients zomwe zolakwika zochepa zidapangidwa. Zowona, chifukwa cha izi tidzayeneranso kulemba zolemba za ma coefficients okha. Sitidzachita izi, koma tcherani khutu ku ndondomekoyi. Zikuwoneka zosalala ndipo cholakwika chikuwoneka kuti chikucheperachepera. Kwenikweni izi sizowona. Tiyeni tiwone zobwereza 1000 zoyambirira ndikuziyerekeza ndi zomaliza.
Khodi ya tchati cha SGD (masitepe 1000 oyamba)
print 'ΠΡΠ°ΡΠΈΠΊ β7 "Π‘ΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ ΠΏΠΎ-ΡΠ°Π³ΠΎΠ²ΠΎ. ΠΠ΅ΡΠ²ΡΠ΅ 1000 ΠΈΡΠ΅ΡΠ°ΡΠΈΠΉ"'
plt.plot(range(len(list_parametres_stoch_gradient_descence[1][:1000])),
list_parametres_stoch_gradient_descence[1][:1000], color='red', lw=2)
plt.xlabel('Steps (Iteration)', size=16)
plt.ylabel('Sum of squared deviations', size=16)
plt.show()
print 'ΠΡΠ°ΡΠΈΠΊ β7 "Π‘ΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ ΠΏΠΎ-ΡΠ°Π³ΠΎΠ²ΠΎ. ΠΠΎΡΠ»Π΅Π΄Π½ΠΈΠ΅ 1000 ΠΈΡΠ΅ΡΠ°ΡΠΈΠΉ"'
plt.plot(range(len(list_parametres_stoch_gradient_descence[1][-1000:])),
list_parametres_stoch_gradient_descence[1][-1000:], color='red', lw=2)
plt.xlabel('Steps (Iteration)', size=16)
plt.ylabel('Sum of squared deviations', size=16)
plt.show()
Graph No. 7 "Sum of squared deviations SGD (masitepe 1000 oyamba)"
Graph No. 8 βSum of squared deviations SGD (masitepe 1000 otsiriza)β
Kumayambiriro kwa kutsika, tikuwona kuchepa kofananako komanso kuchepa kwakukulu kwa zolakwika. M'mawu omaliza, tikuwona kuti cholakwikacho chimazungulira ndikuzungulira mtengo wa 1,475 ndipo nthawi zina chimafanana ndi mtengo wokwanira, koma umakwerabe ... coefficients ΠΈ , ndiyeno sankhani zomwe zolakwikazo ndizochepa. Komabe, tinali ndi vuto lalikulu kwambiri: tidayenera kutenga masitepe 80 (onani code) kuti tipeze mfundo zomwe zili pafupi kwambiri. Ndipo izi zikutsutsana kale ndi lingaliro lakupulumutsa nthawi yowerengera ndi kutsika kwa stochastic gradient kutengera kutsika kwa gradient. Ndi chiyani chomwe chingawongoleredwe ndikuwongoleredwa? Sizovuta kuzindikira kuti muzobwereza zoyamba timatsika molimba mtima, choncho, tiyenera kusiya sitepe yaikulu muzobwereza zoyamba ndikuchepetsa sitepe pamene tikupita patsogolo. Sitidzachita izi m'nkhaniyi - ndi yaitali kwambiri. Iwo omwe akufuna akhoza kudziganizira okha momwe angachitire izi, sizovuta :)
Tsopano tiyeni tichite kutsika kwa stochastic gradient pogwiritsa ntchito laibulale Chiwerengero (ndipo tisapunthwe pa miyala yomwe tidaitchula kale)
Code of Stochastic Gradient Descent (NumPy)
# Π΄Π»Ρ Π½Π°ΡΠ°Π»Π° Π½Π°ΠΏΠΈΡΠ΅ΠΌ ΡΡΠ½ΠΊΡΠΈΡ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠ³ΠΎ ΡΠ°Π³Π°
def stoch_grad_step_numpy(vector_init, X, ind, y, l):
x = X[ind]
y_pred = np.dot(x,vector_init)
err = y_pred - y[ind]
grad_a = err
grad_b = x[1]*err
return vector_init - l*np.array([grad_a, grad_b])
# ΠΎΠΏΡΠ΅Π΄Π΅Π»ΠΈΠΌ ΡΡΠ½ΠΊΡΠΈΡ ΡΡΠΎΡ
Π°ΡΡΠΈΡΠ΅ΡΠΊΠΎΠ³ΠΎ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠ³ΠΎ ΡΠΏΡΡΠΊΠ°
def stoch_grad_descent_numpy(X, y, l=0.1, steps = 800):
vector_init = np.array([[np.random.randint(X.shape[0])], [np.random.randint(X.shape[0])]])
errors = []
for i in range(steps):
ind = np.random.randint(X.shape[0])
new_vector = stoch_grad_step_numpy(vector_init, X, ind, y, l)
vector_init = new_vector
errors.append(error_square_numpy(vector_init,X,y))
return (vector_init), (errors)
# Π·Π°ΠΏΠΈΡΠ΅ΠΌ ΠΌΠ°ΡΡΠΈΠ² Π·Π½Π°ΡΠ΅Π½ΠΈΠΉ
list_parametres_stoch_gradient_descence = stoch_grad_descent_numpy(x_np, y_np, l=0.001, steps = 80000)
print ' 33[1m' + ' 33[4m' + "ΠΠ½Π°ΡΠ΅Π½ΠΈΡ ΠΊΠΎΡΡΡΠΈΡΠΈΠ΅Π½ΡΠΎΠ² a ΠΈ b:" + ' 33[0m'
print 'a =', round(list_parametres_stoch_gradient_descence[0][0],3)
print 'b =', round(list_parametres_stoch_gradient_descence[0][1],3)
print
print ' 33[1m' + ' 33[4m' + "Π‘ΡΠΌΠΌΠ° ΠΊΠ²Π°Π΄ΡΠ°ΡΠΎΠ² ΠΎΡΠΊΠ»ΠΎΠ½Π΅Π½ΠΈΠΉ:" + ' 33[0m'
print round(list_parametres_stoch_gradient_descence[1][-1],3)
print
print ' 33[1m' + ' 33[4m' + "ΠΠΎΠ»ΠΈΡΠ΅ΡΡΠ²ΠΎ ΠΈΡΠ΅ΡΠ°ΡΠΈΠΉ Π² ΡΡΠΎΡ
Π°ΡΡΠΈΡΠ΅ΡΠΊΠΎΠΌ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠΌ ΡΠΏΡΡΠΊΠ΅:" + ' 33[0m'
print len(list_parametres_stoch_gradient_descence[1])
print
Makhalidwe adakhala ngati ofanana ngati akutsika osagwiritsa ntchito Chiwerengero. Komabe, izi ndi zomveka.
Tiyeni tiwone kuti kutsika kwa stochastic gradient kunatitengera nthawi yayitali bwanji.
Khodi yodziwitsa nthawi yowerengera SGD (masitepe 80 zikwi)
print ' 33[1m' + ' 33[4m' +
"ΠΡΠ΅ΠΌΡ Π²ΡΠΏΠΎΠ»Π½Π΅Π½ΠΈΡ ΡΡΠΎΡ
Π°ΡΡΠΈΡΠ΅ΡΠΊΠΎΠ³ΠΎ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠ³ΠΎ ΡΠΏΡΡΠΊΠ° Π±Π΅Π· ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΡ Π±ΠΈΠ±Π»ΠΈΠΎΡΠ΅ΠΊΠΈ NumPy:"
+ ' 33[0m'
%timeit list_parametres_stoch_gradient_descence = stoch_grad_descent_usual(x_us, y_us, l=0.001, steps = 80000)
print '***************************************'
print
print ' 33[1m' + ' 33[4m' +
"ΠΡΠ΅ΠΌΡ Π²ΡΠΏΠΎΠ»Π½Π΅Π½ΠΈΡ ΡΡΠΎΡ
Π°ΡΡΠΈΡΠ΅ΡΠΊΠΎΠ³ΠΎ Π³ΡΠ°Π΄ΠΈΠ΅Π½ΡΠ½ΠΎΠ³ΠΎ ΡΠΏΡΡΠΊΠ° Ρ ΠΈΡΠΏΠΎΠ»ΡΠ·ΠΎΠ²Π°Π½ΠΈΠ΅ΠΌ Π±ΠΈΠ±Π»ΠΈΠΎΡΠ΅ΠΊΠΈ NumPy:"
+ ' 33[0m'
%timeit list_parametres_stoch_gradient_descence = stoch_grad_descent_numpy(x_np, y_np, l=0.001, steps = 80000)
Kupitilira m'nkhalango, mitambo imakhala yakuda: kachiwiri, ndondomeko "yodzilemba" imasonyeza zotsatira zabwino kwambiri. Zonsezi zikusonyeza kuti payenera kukhala njira zochenjera kwambiri zogwiritsira ntchito laibulale Chiwerengero, zomwe zimafulumizitsadi ntchito zowerengera. Mβnkhaniyi sitiphunzira za iwo. Padzakhala china choti muganizire mu nthawi yanu yopuma :)
Tifotokoza mwachidule
Ndisanafotokoze mwachidule, ndikufuna kuyankha funso limene mwachionekere linachokera kwa oΕ΅erenga athu okondedwa. Chifukwa chiyani, kwenikweni, "kuzunzika" kotereku ndi mitsinje, chifukwa chiyani tifunika kuyenda kukwera ndi kutsika phiri (makamaka kutsika) kuti tipeze malo otsika kwambiri, ngati tili ndi chida champhamvu komanso chosavuta m'manja mwathu. njira yowunikira, yomwe imatitumiza nthawi yomweyo kumalo abwino?
Yankho la funsoli lili pamwamba. Tsopano tayang'ana chitsanzo chophweka, momwe yankho lenileni liri zimadalira chizindikiro chimodzi . Simukuwona izi nthawi zambiri m'moyo, ndiye tiyerekeze kuti tili ndi 2, 30, 50 kapena zizindikiro zambiri. Tiwonjeze pa izi masauzande, kapena masauzande masauzande azinthu pamalingaliro aliwonse. Pankhaniyi, yankho lowunikira silingathe kupirira mayeso ndikulephera. Kenako, kutsika kwa gradient ndi kusiyanasiyana kwake kudzatifikitsa pang'onopang'ono ku cholinga - kuchepa kwa ntchitoyo. Ndipo musadandaule za liwiro - mwina tiyang'ana njira zomwe zingatilole kukhazikitsa ndikuwongolera kutalika kwa masitepe (ndiko kuti, liwiro).
Ndipo tsopano mwachidule kwenikweni mwachidule.
Choyamba, ndikuyembekeza kuti zomwe zafotokozedwa m'nkhaniyi zithandiza kuyambitsa "asayansi a data" kumvetsetsa momwe mungathetsere ma equation osavuta (osati okha).
Chachiwiri, tinayang'ana njira zingapo zothetsera equation. Tsopano, malingana ndi mmene zinthu zilili, tikhoza kusankha imene ili yoyenera kuthetsa vutolo.
Chachitatu, tawona mphamvu ya zoikamo zowonjezera, zomwe ndi kutalika kwa sitepe yotsika. Izi sizinganyalanyazidwe. Monga tafotokozera pamwambapa, kuti muchepetse mtengo wowerengera, kutalika kwa masitepe kuyenera kusinthidwa pakutsika.
Chachinayi, kwa ife, ntchito "zolembera kunyumba" zimasonyeza zotsatira zabwino kwambiri za nthawi yowerengera. Izi mwina ndi chifukwa chosagwiritsa ntchito mwaukadaulo kwambiri laibulaleyo Chiwerengero. Koma zikhale choncho, mfundo yotsatirayi imadzisonyeza yokha. Kumbali imodzi, nthawi zina ndikofunikira kufunsa malingaliro okhazikika, ndipo kumbali ina, sikoyenera kusokoneza chilichonse - m'malo mwake, nthawi zina njira yosavuta yothetsera vuto ndiyothandiza kwambiri. Ndipo popeza cholinga chathu chinali kusanthula njira zitatu zothanirana ndi mizere yosavuta, kugwiritsa ntchito "zolemba tokha" kunali kokwanira kwa ife.
Literature (kapena zina zotero)
1. Kutsika kwa mzere
2. Njira yocheperako
3. Kuchokera
4. Zabwino
5. Kutsika kwa gradient
6. Laibulale ya NumPy