SciPy (wotchulidwa kuti sai pie) ndi phukusi la masamu pogwiritsa ntchito Numpy Python extension. Ndi SciPy, gawo lanu lolumikizana la Python limakhala sayansi yathunthu yofananira komanso malo ovuta opangira makina monga MATLAB, IDL, Octave, R-Lab, ndi SciLab. Lero ndikufuna kulankhula mwachidule za momwe mungagwiritsire ntchito ma aligorivimu odziwika bwino mu phukusi la scipy.optimize. Thandizo latsatanetsatane komanso laposachedwa pakugwiritsa ntchito ntchito litha kupezeka nthawi zonse pogwiritsa ntchito lamulo la help() kapena kugwiritsa ntchito Shift+Tab.
Mau oyamba
Kuti mudzipulumutse nokha ndi owerenga kuti asafufuze ndikuwerenga magwero oyambira, maulalo ofotokozera njira azikhala pa Wikipedia. Monga lamulo, chidziwitsochi ndi chokwanira kumvetsetsa njirazo mwachizoloΕ΅ezi ndi zikhalidwe za ntchito yawo. Kuti mumvetse tanthauzo la njira zamasamu, tsatirani ulalo wa zofalitsa zovomerezeka, zomwe zitha kupezeka kumapeto kwa nkhani iliyonse kapena mukusaka kwanu komwe mumakonda.
Chifukwa chake, gawo la scipy.optimize limaphatikizapo kukhazikitsa njira zotsatirazi:
- Kuchepetsa koyenera komanso kopanda malire kwa ntchito za scalar zamitundu ingapo (minim) pogwiritsa ntchito ma aligorivimu osiyanasiyana (Nelder-Mead simplex, BFGS, Newton conjugate gradients, ΠΈ )
- Kukhathamiritsa kwapadziko lonse (mwachitsanzo: , )
- Kuchepetsa zotsalira (zocheperako_makwerero) ndi ma aligorivimu okhotakhota pogwiritsa ntchito masikweya ochepa (curve_fit)
- Kuchepetsa magwiridwe antchito amtundu umodzi (minim_scalar) ndikufufuza mizu (root_scalar)
- Multidimensional solvers of system of equations (muzu) pogwiritsa ntchito ma aligorivimu osiyanasiyana (hybrid Powell, kapena njira zazikulu monga ).
M'nkhaniyi tiona chinthu choyamba pa mndandanda wonsewu.
Kuchepetsa mopanda malire kwa scalar ntchito yamitundu ingapo
Ntchito yocheperako kuchokera ku phukusi la scipy.optimize imapereka mawonekedwe owoneka bwino kuti athe kuthana ndi zovuta zocheperako komanso zopanda malire za ntchito za scalar zamitundu ingapo. Kuti tiwonetse momwe zimagwirira ntchito, tidzafunika ntchito yoyenera yamitundu ingapo, yomwe tidzayichepetsa m'njira zosiyanasiyana.
Pazifukwa izi, ntchito ya Rosenbrock ya mitundu ya N ndi yangwiro, yomwe ili ndi mawonekedwe:
Ngakhale kuti ntchito ya Rosenbrock ndi matrices ake a Jacobi ndi Hessian (zochokera koyamba ndi zachiwiri, motsatira) zafotokozedwa kale mu phukusi la scipy.optimize, tidzalongosola tokha.
import numpy as np
def rosen(x):
"""The Rosenbrock function"""
return np.sum(100.0*(x[1:]-x[:-1]**2.0)**2.0 + (1-x[:-1])**2.0, axis=0)Kuti timveke bwino, tiyeni tijambule mu 3D zamitundu iwiri ya Rosenbrock.
Zojambulajambula
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
# ΠΠ°ΡΡΡΠ°ΠΈΠ²Π°Π΅ΠΌ 3D Π³ΡΠ°ΡΠΈΠΊ
fig = plt.figure(figsize=[15, 10])
ax = fig.gca(projection='3d')
# ΠΠ°Π΄Π°Π΅ΠΌ ΡΠ³ΠΎΠ» ΠΎΠ±Π·ΠΎΡΠ°
ax.view_init(45, 30)
# Π‘ΠΎΠ·Π΄Π°Π΅ΠΌ Π΄Π°Π½Π½ΡΠ΅ Π΄Π»Ρ Π³ΡΠ°ΡΠΈΠΊΠ°
X = np.arange(-2, 2, 0.1)
Y = np.arange(-1, 3, 0.1)
X, Y = np.meshgrid(X, Y)
Z = rosen(np.array([X,Y]))
# Π ΠΈΡΡΠ΅ΠΌ ΠΏΠΎΠ²Π΅ΡΡ
Π½ΠΎΡΡΡ
surf = ax.plot_surface(X, Y, Z, cmap=cm.coolwarm)
plt.show()
Kudziwa pasadakhale kuti osachepera ndi 0 pa , tiyeni tiwone zitsanzo za momwe tingadziwire mtengo wochepa wa ntchito ya Rosenbrock pogwiritsa ntchito njira zosiyanasiyana za scipy.optimize.
Njira ya Nelder-Mead simplex
Pakhale mfundo yoyambira x0 mu danga la 5-dimensional. Tiyeni tipeze gawo lochepera la ntchito ya Rosenbrock yomwe ili pafupi kwambiri ndi iyo pogwiritsa ntchito algorithm (algorithm imatchulidwa ngati mtengo wa parameter):
from scipy.optimize import minimize
x0 = np.array([1.3, 0.7, 0.8, 1.9, 1.2])
res = minimize(rosen, x0, method='nelder-mead',
options={'xtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 339
Function evaluations: 571
[1. 1. 1. 1. 1.]Njira yosavuta ndiyo njira yosavuta yochepetsera ntchito yofotokozedwa momveka bwino komanso yosalala bwino. Sichifunikira kuwerengera zotumphukira za ntchito; ndikwanira kutchula zikhalidwe zake zokha. Njira ya Nelder-Mead ndi chisankho chabwino pazovuta zosavuta zochepetsera. Komabe, popeza sichigwiritsa ntchito kuyerekezera kwa gradient, zingatengere nthawi kuti mupeze zochepa.
Njira ya Powell
Algorithm ina yokhathamiritsa momwe magwiridwe antchito okha amawerengedwa ndi . Kuti mugwiritse ntchito, muyenera kukhazikitsa njira = 'powell' mu minim ntchito.
x0 = np.array([1.3, 0.7, 0.8, 1.9, 1.2])
res = minimize(rosen, x0, method='powell',
options={'xtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 19
Function evaluations: 1622
[1. 1. 1. 1. 1.]Broyden-Fletcher-Goldfarb-Shanno (BFGS) aligorivimu
Kupeza msanga convergence yankho, ndondomeko amagwiritsa ntchito gradient ya cholinga. Ma gradient amatha kufotokozedwa ngati ntchito kapena kuwerengeredwa pogwiritsa ntchito mitundu yoyambira. Mulimonsemo, njira ya BFGS nthawi zambiri imafuna mafoni ocheperako kuposa njira ya simplex.
Tiyeni tipeze zomwe zimachokera ku ntchito ya Rosenbrock mu mawonekedwe owunikira:
Mawuwa ndi ovomerezeka pazotsatira zamitundu yonse kupatula yoyamba ndi yomaliza, yomwe imatanthauzidwa kuti:
Tiyeni tiwone ntchito ya Python yomwe imawerengera gradient iyi:
def rosen_der (x):
xm = x [1: -1]
xm_m1 = x [: - 2]
xm_p1 = x [2:]
der = np.zeros_like (x)
der [1: -1] = 200 * (xm-xm_m1 ** 2) - 400 * (xm_p1 - xm ** 2) * xm - 2 * (1-xm)
der [0] = -400 * x [0] * (x [1] -x [0] ** 2) - 2 * (1-x [0])
der [-1] = 200 * (x [-1] -x [-2] ** 2)
return derNtchito yowerengera gradient imatchulidwa ngati mtengo wa jac parameter ya minim function, monga momwe zilili pansipa.
res = minimize(rosen, x0, method='BFGS', jac=rosen_der, options={'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 25
Function evaluations: 30
Gradient evaluations: 30
[1.00000004 1.0000001 1.00000021 1.00000044 1.00000092]Conjugate gradient algorithm (Newton)
Algorithm ndi njira yosinthidwa ya Newton.
Njira ya Newton idakhazikitsidwa pakuyerekeza ntchito mdera lanu ndi polynomial ya digiri yachiwiri:
kumene ndiye matrix a zotumphukira zachiwiri (Hessian matrix, Hessian).
Ngati Hessian ndi yotsimikizika yotsimikizika, ndiye kuti kucheperako kwa ntchitoyi kutha kupezeka pofananiza zero gradient ya mawonekedwe a quadratic kukhala ziro. Zotsatira zake zidzakhala mawu akuti:
Hessian yosiyana imawerengedwa pogwiritsa ntchito njira ya conjugate gradient. Chitsanzo chogwiritsa ntchito njirayi kuti muchepetse ntchito ya Rosenbrock chaperekedwa pansipa. Kuti mugwiritse ntchito njira ya Newton-CG, muyenera kufotokoza ntchito yomwe imawerengera Hessian.
Ntchito ya Hessian ya Rosenbrock mu mawonekedwe owunikira ndi ofanana ndi:
kumene ΠΈ , fotokozani matrix .
Zinthu zotsalira zomwe sizinali zero za matrix ndizofanana ndi:
Mwachitsanzo, mu malo azithunzi zisanu N = 5, matrix a Hessian a ntchito ya Rosenbrock ali ndi mawonekedwe a gulu:
Khodi yomwe imawerengera Hessian iyi pamodzi ndi code yochepetsera ntchito ya Rosenbrock pogwiritsa ntchito njira ya conjugate gradient (Newton):
def rosen_hess(x):
x = np.asarray(x)
H = np.diag(-400*x[:-1],1) - np.diag(400*x[:-1],-1)
diagonal = np.zeros_like(x)
diagonal[0] = 1200*x[0]**2-400*x[1]+2
diagonal[-1] = 200
diagonal[1:-1] = 202 + 1200*x[1:-1]**2 - 400*x[2:]
H = H + np.diag(diagonal)
return H
res = minimize(rosen, x0, method='Newton-CG',
jac=rosen_der, hess=rosen_hess,
options={'xtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 24
Function evaluations: 33
Gradient evaluations: 56
Hessian evaluations: 24
[1. 1. 1. 0.99999999 0.99999999]Chitsanzo ndi tanthawuzo la ntchito ya mankhwala a Hessian ndi vector yosagwirizana
M'mabvuto adziko lapansi, kugwiritsa ntchito makompyuta ndikusunga matrix onse a Hessian kungafune nthawi yofunikira komanso zokumbukira. Pankhaniyi, palibe chifukwa chofotokozera matrix a Hessian okha, chifukwa njira yochepetsera imafuna vekitala yokha yofanana ndi mankhwala a Hessian ndi vector ina yosagwirizana. Chifukwa chake, kuchokera pamawonekedwe owerengera, ndikwabwino kufotokozera mwachangu ntchito yomwe imabweretsa zotsatira za chinthu cha Hessian ndi vector yosagwirizana.
Ganizirani za ntchito ya hess, yomwe imatenga vekitala yochepetsera ngati mtsutso woyamba, ndi vector yokhazikika ngati mtsutso wachiwiri (pamodzi ndi mfundo zina zantchitoyo kuti zichepe). Kwa ife, kuwerengera mankhwala a Hessian wa ntchito ya Rosenbrock ndi vekitala yosagwirizana sikovuta kwambiri. Ngati p ndi vekitala mosinthasintha, ndiye mankhwala ali ndi mawonekedwe:
Ntchito yomwe imawerengera chopangidwa ndi Hessian ndi vector yokhazikika imadutsa ngati mtengo wa mtsutso wa hessp ku minimize ntchito:
def rosen_hess_p(x, p):
x = np.asarray(x)
Hp = np.zeros_like(x)
Hp[0] = (1200*x[0]**2 - 400*x[1] + 2)*p[0] - 400*x[0]*p[1]
Hp[1:-1] = -400*x[:-2]*p[:-2]+(202+1200*x[1:-1]**2-400*x[2:])*p[1:-1]
-400*x[1:-1]*p[2:]
Hp[-1] = -400*x[-2]*p[-2] + 200*p[-1]
return Hp
res = minimize(rosen, x0, method='Newton-CG',
jac=rosen_der, hessp=rosen_hess_p,
options={'xtol': 1e-8, 'disp': True})
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 24
Function evaluations: 33
Gradient evaluations: 56
Hessian evaluations: 66Conjugate gradient trust region algorithm (Newton)
Kusakhazikika bwino kwa matrix a Hessian komanso mayendedwe olakwika osakira kungapangitse algorithm ya Newton ya conjugate gradient kukhala yosagwira ntchito. Zikatero, zokonda zimaperekedwa (trust-region) conjugate Newton gradients.
Chitsanzo ndi tanthauzo la matrix a Hessian:
res = minimize(rosen, x0, method='trust-ncg',
jac=rosen_der, hess=rosen_hess,
options={'gtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 20
Function evaluations: 21
Gradient evaluations: 20
Hessian evaluations: 19
[1. 1. 1. 1. 1.]Chitsanzo ndi ntchito ya Hessian ndi vekitala yokhazikika:
res = minimize(rosen, x0, method='trust-ncg',
jac=rosen_der, hessp=rosen_hess_p,
options={'gtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 20
Function evaluations: 21
Gradient evaluations: 20
Hessian evaluations: 0
[1. 1. 1. 1. 1.]Njira zamtundu wa Krylov
Mofanana ndi njira ya trust-ncg, njira zamtundu wa Krylov ndizoyenera kuthetsa mavuto akuluakulu chifukwa amagwiritsa ntchito mankhwala a matrix-vector okha. Chofunikira chawo ndikuthana ndi vuto mdera la chidaliro lomwe lili ndi malo ochepa a Krylov. Pazovuta zosadziwika bwino, ndi bwino kugwiritsa ntchito njirayi, chifukwa imagwiritsa ntchito chiwerengero chochepa cha maulendo osagwirizana chifukwa cha chiwerengero chochepa cha mankhwala a matrix-vector pa subproblem, poyerekeza ndi njira ya trust-ncg. Kuphatikiza apo, yankho la quadratic subproblem limapezeka molondola kuposa kugwiritsa ntchito njira ya trust-ncg.
Chitsanzo ndi tanthauzo la matrix a Hessian:
res = minimize(rosen, x0, method='trust-krylov',
jac=rosen_der, hess=rosen_hess,
options={'gtol': 1e-8, 'disp': True})
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 19
Function evaluations: 20
Gradient evaluations: 20
Hessian evaluations: 18
print(res.x)
[1. 1. 1. 1. 1.]
Chitsanzo ndi ntchito ya Hessian ndi vekitala yokhazikika:
res = minimize(rosen, x0, method='trust-krylov',
jac=rosen_der, hessp=rosen_hess_p,
options={'gtol': 1e-8, 'disp': True})
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 19
Function evaluations: 20
Gradient evaluations: 20
Hessian evaluations: 0
print(res.x)
[1. 1. 1. 1. 1.]
Algorithm yofikira yankho mdera la chidaliro
Njira zonse (Newton-CG, trust-ncg ndi trust-krylov) ndizoyenera kuthetsa mavuto akuluakulu (ndi masauzande osiyanasiyana). Izi ndichifukwa choti maziko a conjugate gradient algorithm amatanthauza kutsimikizika kwa matrix a Hessian. Yankho likupezeka mobwereza bwereza, popanda kufalikira kwachi Hessian. Popeza mumangofunika kufotokozera ntchito yamtundu wa Hessian ndi vector yokhazikika, algorithm iyi ndiyabwino kwambiri pogwira ntchito ndi matrices a sparse (band diagonal). Izi zimapereka ndalama zochepa zokumbukira komanso kusunga nthawi.
Pazovuta zapakatikati, mtengo wosungira ndikuyika Hessian siwovuta. Izi zikutanthauza kuti ndizotheka kupeza yankho muzobwereza zocheperako, kuthetsa ma subproblems a dera la trust pafupifupi ndendende. Kuti muchite izi, ma equation ena osagwirizana amathetsedwa mobwerezabwereza pagawo lililonse la quadratic. Yankho lotere nthawi zambiri limafunikira 3 kapena 4 Cholesky kuwonongeka kwa matrix a Hessian. Zotsatira zake, njirayo imasinthasintha pang'ono ndipo imafuna kuwerengera kocheperako poyerekeza ndi njira zina zachigawo zodalirika. Algorithm iyi imangofunika kudziwa matrix a Hessian athunthu ndipo sizigwirizana ndi kuthekera kogwiritsa ntchito ntchito ya Hessian ndi vector yokhazikika.
Chitsanzo ndi kuchepetsa ntchito ya Rosenbrock:
res = minimize(rosen, x0, method='trust-exact',
jac=rosen_der, hess=rosen_hess,
options={'gtol': 1e-8, 'disp': True})
res.xOptimization terminated successfully.
Current function value: 0.000000
Iterations: 13
Function evaluations: 14
Gradient evaluations: 13
Hessian evaluations: 14
array([1., 1., 1., 1., 1.])Mwina tiyima pamenepo. M'nkhani yotsatira ndiyesera kunena zinthu zosangalatsa kwambiri za kuchepetsa zovomerezeka, kugwiritsa ntchito kuchepetsa kuthetsa mavuto okhudzana ndi kuyerekezera, kuchepetsa ntchito yamtundu umodzi, minimizers mopanda malire, ndikupeza mizu ya dongosolo la equations pogwiritsa ntchito scipy.optimize phukusi.
Source:
Source: www.habr.com