ʻO SciPy (ʻōlelo ʻia ʻo sai pie) kahi palapala noi makemakika e pili ana i ka extension Numpy Python. Me SciPy, lilo kāu hui Python interactive i ka ʻepekema ʻikepili piha a me ka ʻōnaehana prototyping ʻōnaehana paʻakikī e like me MATLAB, IDL, Octave, R-Lab, a me SciLab. I kēia lā makemake wau e kamaʻilio pōkole e pili ana i ka hoʻohana ʻana i kekahi mau algorithm optimization kaulana i loko o ka pūʻolo scipy.optimize. Hiki ke loaʻa mau ke kōkua kikoʻī a me ka manawa hou ma ka hoʻohana ʻana i nā hana me ka hoʻohana ʻana i ke kauoha kōkua () a i ʻole ke hoʻohana ʻana i ka Shift+Tab.
Hōʻike
I mea e hoʻopakele ai iā ʻoe iho a me ka poʻe heluhelu mai ka ʻimi ʻana a me ka heluhelu ʻana i nā kumu kumu mua, aia nā loulou i nā wehewehe o nā ʻano hana ma Wikipedia. Ma keʻano he kūlana, ua lawa kēiaʻike e hoʻomaopopo i nāʻano ma nā'ōlelo maʻamau a me nā kūlana no kā lākou noi. No ka hoʻomaopopo ʻana i ke ʻano o nā ʻano hana makemakika, e hahai i nā loulou i nā puke mana hou, hiki ke loaʻa ma ka hope o kēlā me kēia ʻatikala a i ʻole ma kāu ʻenekini punahele.
No laila, aia ka module scipy.optimize i ka hoʻokō ʻana i nā kaʻina hana:
- ʻO ka hōʻemi ʻana a me ka palena ʻole o nā hana scalar o kekahi mau mea hoʻololi (minim) me ka hoʻohana ʻana i nā algorithms like ʻole (Nelder-Mead simplex, BFGS, Newton conjugate gradients, и )
- ʻO ka loiloi honua (no ka laʻana: , )
- E hoemi ana i na koena (least_squares) a me nā ʻōlelo algorithm e hoʻohana ana i nā ʻāpana liʻiliʻi liʻiliʻi loa (curve_fit)
- Hoʻemi i nā hana scalar o hoʻokahi loli (minim_scalar) a me ka ʻimi ʻana i nā aʻa (root_scalar)
- Nā mea hoʻoponopono multidimensional o ka ʻōnaehana o nā hoohalike (aʻa) me ka hoʻohana ʻana i nā algorithms like ʻole (hybrid Powell, a i ʻole nā ʻano hana nui e like me ).
Ma kēiaʻatikala e noʻonoʻo mākou i ka mea mua wale nō mai kēia papa inoa holoʻokoʻa.
ʻO ka hoʻemi ʻole ʻana o kahi hana scalar o kekahi mau ʻano
Hāʻawi ka hana liʻiliʻi mai ka pūʻolo scipy.optimize i kahi kikowaena maʻamau no ka hoʻoponopono ʻana i nā pilikia hōʻemi kūlana a me ka palena ʻole o nā hana scalar o kekahi mau ʻano. No ka hōʻike ʻana pehea e hana ai, pono mākou i kahi hana kūpono o nā ʻano like ʻole, a mākou e hōʻemi ai i nā ʻano like ʻole.
No kēia mau kumu, kūpono ka hana Rosenbrock o nā ʻano N, nona ke ʻano:
ʻOiai ʻo ka hana ʻo Rosenbrock a me kāna mau matrices Jacobi a me Hessian (ʻo ka derivatives mua a me ka lua, kēlā me kēia) ua wehewehe ʻia i loko o ka scipy.optimize package, e wehewehe mākou iā mākou iho.
import numpy as np
def rosen(x):
"""The Rosenbrock function"""
return np.sum(100.0*(x[1:]-x[:-1]**2.0)**2.0 + (1-x[:-1])**2.0, axis=0)No ka akaka, e kahakiʻi i ka 3D i nā waiwai o ka hana Rosenbrock o ʻelua mau ʻano.
Kiʻi kuhi
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
# Настраиваем 3D график
fig = plt.figure(figsize=[15, 10])
ax = fig.gca(projection='3d')
# Задаем угол обзора
ax.view_init(45, 30)
# Создаем данные для графика
X = np.arange(-2, 2, 0.1)
Y = np.arange(-1, 3, 0.1)
X, Y = np.meshgrid(X, Y)
Z = rosen(np.array([X,Y]))
# Рисуем поверхность
surf = ax.plot_surface(X, Y, Z, cmap=cm.coolwarm)
plt.show()
ʻO ka ʻike mua ʻana he 0 ka liʻiliʻi ma , e nānā i nā hiʻohiʻona pehea e hoʻoholo ai i ka waiwai liʻiliʻi o ka hana Rosenbrock me ka hoʻohana ʻana i nā kaʻina hana scipy.optimize.
ʻO ke ala maʻalahi ʻo Nelder-Mead
E waiho i kahi kiko mua x0 ma ke akea 5. E ʻimi kākou i ka palena liʻiliʻi o ka hana Rosenbrock kokoke loa iā ia me ka hoʻohana ʻana i ka algorithm (Ua kuhikuhi ʻia ka algorithm ma ke ʻano he waiwai o ke ʻano hana):
from scipy.optimize import minimize
x0 = np.array([1.3, 0.7, 0.8, 1.9, 1.2])
res = minimize(rosen, x0, method='nelder-mead',
options={'xtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 339
Function evaluations: 571
[1. 1. 1. 1. 1.]ʻO ke ala maʻalahi ke ala maʻalahi e hōʻemi i kahi hana i wehewehe ʻia a maʻalahi hoʻi. ʻAʻole koi ʻia ka helu ʻana i nā derivatives o kahi hana; ua lawa ia e kuhikuhi wale i kāna mau waiwai. ʻO ke ala Nelder-Mead kahi koho maikaʻi no nā pilikia hoʻemi maʻalahi. Eia naʻe, no ka mea ʻaʻole ia e hoʻohana i nā koho gradient, lōʻihi paha ka lōʻihi o ka loaʻa ʻana o ka liʻiliʻi.
ʻO ke ala ʻo Powell
ʻO kekahi algorithm optimization kahi i helu ʻia ai nā waiwai hana . No ka hoʻohana ʻana, pono ʻoe e hoʻonohonoho i ke ala = 'powell' i ka hana minim.
x0 = np.array([1.3, 0.7, 0.8, 1.9, 1.2])
res = minimize(rosen, x0, method='powell',
options={'xtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 19
Function evaluations: 1622
[1. 1. 1. 1. 1.]Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm
No ka loaʻa ʻana o ka convergence wikiwiki i kahi hoʻonā, ke kaʻina hana hoʻohana i ka gradient o ka hana pahuhopu. Hiki ke kuhikuhi ʻia ka gradient ma ke ʻano he hana a helu ʻia me ka hoʻohana ʻana i nā ʻokoʻa kauoha mua. I kēlā me kēia hihia, ʻoi aku ka liʻiliʻi o ke ala BFGS ma mua o ke ʻano simplex.
E ʻimi kākou i ka derivative o ka hana Rosenbrock ma ke ʻano analytical:
He kūpono kēia ʻōlelo no nā derivatives o nā mea hoʻololi a pau koe ka mua a me ka hope, i wehewehe ʻia penei:
E nānā kākou i ka hana Python e helu ana i kēia gradient:
def rosen_der (x):
xm = x [1: -1]
xm_m1 = x [: - 2]
xm_p1 = x [2:]
der = np.zeros_like (x)
der [1: -1] = 200 * (xm-xm_m1 ** 2) - 400 * (xm_p1 - xm ** 2) * xm - 2 * (1-xm)
der [0] = -400 * x [0] * (x [1] -x [0] ** 2) - 2 * (1-x [0])
der [-1] = 200 * (x [-1] -x [-2] ** 2)
return derHoʻomaopopo ʻia ka hana helu gradient e like me ka waiwai o ka ʻāpana jac o ka hana liʻiliʻi, e like me ka mea i hōʻike ʻia ma lalo nei.
res = minimize(rosen, x0, method='BFGS', jac=rosen_der, options={'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 25
Function evaluations: 30
Gradient evaluations: 30
[1.00000004 1.0000001 1.00000021 1.00000044 1.00000092]Hoʻopili i ka algorithm gradient (Newton)
ʻO Algorithm ʻO ke ʻano hana a Newton i hoʻololi ʻia.
Hoʻokumu ʻia ke ʻano o Newton ma ka hoʻohālikelike ʻana i kahi hana ma kahi kūloko e kahi polynomial o ke degere ʻelua:
kahi ʻo ia ka matrix o nā huaʻōlelo ʻelua (Hessian matrix, Hessian).
Inā pololei ka Hessian, a laila hiki ke ʻike ʻia ka palena liʻiliʻi o kēia hana ma ka hoʻohālikelike ʻana i ka gradient ʻole o ke ʻano quadratic me ka ʻole. ʻO ka hopena ka hua'ōlelo:
Ua helu ʻia ka Hessian inverse me ke ʻano gradient conjugate. ʻO kahi laʻana o ka hoʻohana ʻana i kēia ʻano hana e hōʻemi i ka hana Rosenbrock i hāʻawi ʻia ma lalo nei. No ka hoʻohana ʻana i ke ʻano Newton-CG, pono ʻoe e kuhikuhi i kahi hana e helu ana i ka Hessian.
ʻO ka Hessian o ka hana Rosenbrock ma ke ʻano analytical like me:
kahi и , wehewehe i ka matrix .
ʻO nā mea i koe ʻole-zero o ka matrix ua like me:
No ka laʻana, i loko o kahi ākea ʻelima N = 5, ʻo ka matrix Hessian no ka hana Rosenbrock ke ʻano o kahi hui:
Code e helu ana i kēia Hessian me ke code no ka hoʻemi ʻana i ka hana Rosenbrock me ka hoʻohana ʻana i ke ʻano conjugate gradient (Newton):
def rosen_hess(x):
x = np.asarray(x)
H = np.diag(-400*x[:-1],1) - np.diag(400*x[:-1],-1)
diagonal = np.zeros_like(x)
diagonal[0] = 1200*x[0]**2-400*x[1]+2
diagonal[-1] = 200
diagonal[1:-1] = 202 + 1200*x[1:-1]**2 - 400*x[2:]
H = H + np.diag(diagonal)
return H
res = minimize(rosen, x0, method='Newton-CG',
jac=rosen_der, hess=rosen_hess,
options={'xtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 24
Function evaluations: 33
Gradient evaluations: 56
Hessian evaluations: 24
[1. 1. 1. 0.99999999 0.99999999]ʻO kahi laʻana me ka wehewehe ʻana i ka hana huahana o ka Hessian a me kahi vector arbitrary
I nā pilikia o ka honua maoli, hiki i ka helu ʻana a me ka mālama ʻana i ka matrix Hessian holoʻokoʻa ke koi i ka manawa nui a me nā kumu hoʻomanaʻo. I kēia hihia, ʻaʻohe pono e kuhikuhi i ka matrix Hessian ponoʻī, no ka mea ʻO ke kaʻina hana hoʻohaʻahaʻa e koi wale i kahi vector e like me ka huahana o ka Hessian me kahi vector ʻokoʻa ʻē aʻe. No laila, mai ka helu helu ʻana, ʻoi aku ka maikaʻi o ka wehewehe koke ʻana i kahi hana e hoʻihoʻi i ka hopena o ka huahana o ka Hessian me kahi vector arbitrary.
E noʻonoʻo i ka hana hess, e lawe ana i ka vector minimization ma ke ʻano he hoʻopaʻapaʻa mua, a me ka vector arbitrary e like me ka hoʻopaʻapaʻa ʻelua (me nā ʻōlelo hoʻopaʻapaʻa ʻē aʻe o ka hana e hōʻemi ʻia). I kā mākou hihia, ʻaʻole paʻakikī loa ka helu ʻana i ka huahana o ka Hessian o ka hana Rosenbrock me kahi vector kuʻuna. Ina p he mea hoʻokuʻu wale, a laila ka huahana loaʻa ke ʻano:
ʻO ka hana e helu ana i ka huahana o ka Hessian a me kahi vector kuʻuna ʻole ʻia e like me ka waiwai o ka hoʻopaʻapaʻa hessp i ka hana liʻiliʻi:
def rosen_hess_p(x, p):
x = np.asarray(x)
Hp = np.zeros_like(x)
Hp[0] = (1200*x[0]**2 - 400*x[1] + 2)*p[0] - 400*x[0]*p[1]
Hp[1:-1] = -400*x[:-2]*p[:-2]+(202+1200*x[1:-1]**2-400*x[2:])*p[1:-1]
-400*x[1:-1]*p[2:]
Hp[-1] = -400*x[-2]*p[-2] + 200*p[-1]
return Hp
res = minimize(rosen, x0, method='Newton-CG',
jac=rosen_der, hessp=rosen_hess_p,
options={'xtol': 1e-8, 'disp': True})
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 24
Function evaluations: 33
Gradient evaluations: 56
Hessian evaluations: 66E hoʻohui i ka gradient hilinaʻi ʻāpana algorithm (Newton)
ʻO ka maikaʻi ʻole o ka Hessian matrix a me nā kuhikuhi ʻimi hewa ʻole hiki ke hoʻopōmaikaʻi i ka algorithm gradient conjugate o Newton. Ma ia mau hihia, hāʻawi ʻia ka makemake (ʻāina hilinaʻi) hoʻohui i nā ʻanuʻu Newton.
Ka laʻana me ka wehewehe ʻana o ka matrix Hessian:
res = minimize(rosen, x0, method='trust-ncg',
jac=rosen_der, hess=rosen_hess,
options={'gtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 20
Function evaluations: 21
Gradient evaluations: 20
Hessian evaluations: 19
[1. 1. 1. 1. 1.]ʻO ka laʻana me ka hana huahana o ka Hessian a me kahi vector arbitrary:
res = minimize(rosen, x0, method='trust-ncg',
jac=rosen_der, hessp=rosen_hess_p,
options={'gtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 20
Function evaluations: 21
Gradient evaluations: 20
Hessian evaluations: 0
[1. 1. 1. 1. 1.]ʻO nā ʻano ʻano Krylov
E like me ke ʻano hilinaʻi-ncg, kūpono nā ʻano Krylov-type no ka hoʻoponopono ʻana i nā pilikia nui no ka mea hoʻohana wale lākou i nā huahana matrix-vector. ʻO kā lākou mea nui e hoʻoponopono i kahi pilikia ma kahi ʻāina hilinaʻi i kaupalena ʻia e kahi subspace Krylov i ʻoki ʻia. No nā pilikia maopopo ʻole, ʻoi aku ka maikaʻi o ka hoʻohana ʻana i kēia ʻano hana, no ka mea, hoʻohana ʻo ia i kahi helu liʻiliʻi o nā ʻike ʻole linear ma muli o ka liʻiliʻi o nā huahana matrix-vector i kēlā me kēia subproblem, i hoʻohālikelike ʻia me ke ʻano hilinaʻi-ncg. Eia kekahi, ʻoi aku ka pololei o ka hoʻonā i ka subproblem quadratic ma mua o ka hoʻohana ʻana i ke ala hilinaʻi-ncg.
Ka laʻana me ka wehewehe ʻana o ka matrix Hessian:
res = minimize(rosen, x0, method='trust-krylov',
jac=rosen_der, hess=rosen_hess,
options={'gtol': 1e-8, 'disp': True})
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 19
Function evaluations: 20
Gradient evaluations: 20
Hessian evaluations: 18
print(res.x)
[1. 1. 1. 1. 1.]
ʻO ka laʻana me ka hana huahana o ka Hessian a me kahi vector arbitrary:
res = minimize(rosen, x0, method='trust-krylov',
jac=rosen_der, hessp=rosen_hess_p,
options={'gtol': 1e-8, 'disp': True})
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 19
Function evaluations: 20
Gradient evaluations: 20
Hessian evaluations: 0
print(res.x)
[1. 1. 1. 1. 1.]
Algorithm no ka hoʻonā pili ma ka ʻāina hilinaʻi
ʻO nāʻano hana a pau (Newton-CG, trust-ncg a me trust-krylov) ua kūpono no ka hoʻoponoponoʻana i nā pilikia nui (me nā tausani o nā loli). Ma muli o ka manaʻo o ka conjugate gradient algorithm e pili ana i kahi hoʻoholo pili o ka matrix Hessian inverse. Loaʻa ka hoʻonā i ka iteratively, me ka hoʻonui ʻole o ka Hessian. No ka mea pono ʻoe e wehewehe i kahi hana no ka huahana o kahi Hessian a me kahi vector arbitrary, maikaʻi loa kēia algorithm no ka hana ʻana me nā matrices sparse (band diagonal). Hāʻawi kēia i nā kumukūʻai hoʻomanaʻo haʻahaʻa a me ka mālama manawa nui.
No nā pilikia liʻiliʻi, ʻaʻole koʻikoʻi ke kumukūʻai o ka mālama ʻana a me ka helu ʻana i ka Hessian. 'O ia ho'i, hiki ke loa'a i ka ho'onā ma nā 'ano li'ili'i li'ili'i, e ho'oponopono i nā pilikia o ka 'āina hilina'i kokoke pololei. No ka hana ʻana i kēia, ua hoʻoholo ʻia kekahi mau hoohalike nonlinear no kēlā me kēia subproblem quadratic. ʻO ia ʻano hoʻonā maʻamau e koi i 3 a i ʻole 4 Cholesky decompositions o ka matrix Hessian. ʻO ka hopena, hoʻololi ke ʻano i ka liʻiliʻi liʻiliʻi a koi ʻia ka liʻiliʻi o nā helu hana pahuhopu ma mua o nā ʻano hana ʻāina hilinaʻi i hoʻokō ʻia. Hōʻike wale kēia algorithm i ka hoʻoholo ʻana o ka matrix Hessian piha a ʻaʻole kākoʻo i ka hiki ke hoʻohana i ka hana huahana o ka Hessian a me kahi vector arbitrary.
Ka laʻana me ka hōʻemi ʻana i ka hana Rosenbrock:
res = minimize(rosen, x0, method='trust-exact',
jac=rosen_der, hess=rosen_hess,
options={'gtol': 1e-8, 'disp': True})
res.xOptimization terminated successfully.
Current function value: 0.000000
Iterations: 13
Function evaluations: 14
Gradient evaluations: 13
Hessian evaluations: 14
array([1., 1., 1., 1., 1.])E kū paha kāua i laila. Ma ka ʻatikala aʻe, e hoʻāʻo wau e haʻi i nā mea hoihoi loa e pili ana i ka hoʻemi ʻana i ke kūlana, ka noi ʻana o ka hoʻemi ʻana i ka hoʻoponopono ʻana i nā pilikia pili, e hōʻemi ana i kahi hana o hoʻokahi mea hoʻololi, nā mea liʻiliʻi liʻiliʻi, a me ka ʻimi ʻana i nā kumu o kahi ʻōnaehana o ka hoʻohana ʻana i ka scipy.optimize. pūʻolo.
Source:
Source: www.habr.com