SciPy (akpọ sai pie) bụ ngwungwu ngwa mgbakọ na mwepụ dabere na ndọtị Numpy Python. Site na SciPy, nnọkọ Python gị na-aghọ otu sayensị data zuru oke yana gburugburu ebe nleba anya sistemụ dị ka MATLAB, IDL, Octave, R-Lab, na SciLab. Taa, achọrọ m ikwu okwu nkenke maka otu esi eji ụfọdụ algorithms kachasị ama ama na ngwugwu scipy.optimize. Enwere ike nweta enyemaka zuru ezu na nke ọhụrụ maka iji ọrụ mgbe niile site na iji iwu enyemaka () ma ọ bụ iji Shift + Tab.
Okwu Mmalite
Iji chekwaa onwe gị na ndị na-agụ akwụkwọ site n'ịchọ na ịgụ akwụkwọ ndị bụ isi, njikọ na nkọwa nke ụzọ ga-adị na Wikipedia. Dị ka a na-achị, ozi a zuru ezu iji ghọta ụzọ n'ozuzu ya na ọnọdụ maka ngwa ha. Iji ghọta ụzọ mgbakọ na mwepụ dị mkpa, soro njikọ ndị a na-ebipụta akwụkwọ ikike, nke enwere ike ịchọta na njedebe nke edemede ọ bụla ma ọ bụ na nchọta ọkacha mmasị gị.
Ya mere, modul scipy.optimize gụnyere mmejuputa usoro ndị a:
- Mbelata ọrụ scalar nke ọtụtụ mgbanwe (obere) n'ọnọdụ na enweghị ọnọdụ site na iji algọridim dị iche iche (Nelder-Mead simplex, BFGS, Newton conjugate gradients, и )
- Nkwalite zuru ụwa ọnụ (dịka ọmụmaatụ: , )
- Na-ebelata ihe ndị fọdụrụnụ (least_squares) yana algọridim dabara adaba na-eji akuku pere mpe (curve_fit)
- Ibelata ọrụ scalar nke otu mgbanwe (minim_scalar) na ịchọ mgbọrọgwụ (root_scalar)
- Multidimensional solvers nke usoro nke nha anya (mgbọrọgwụ) na-eji dị iche iche algọridim (ngwakọ Powell, ma ọ bụ nnukwu usoro dị ka ).
N'isiokwu a, anyị ga-atụle naanị ihe mbụ dị na ndepụta a dum.
Mbelata ọrụ scalar nke ọtụtụ mgbanwe na-enweghị atụ
Ọrụ kacha nta sitere na ngwugwu scipy.optimize na-enye interface zuru oke maka idozi nsogbu mbelata ọnọdụ na enweghị ọnọdụ nke ọrụ scalar nke ọtụtụ mgbanwe. Iji gosi otú o si arụ ọrụ, anyị ga-achọ ọrụ kwesịrị ekwesị nke ọtụtụ mgbanwe, nke anyị ga-ebelata n'ụzọ dị iche iche.
Maka ebumnuche ndị a, ọrụ Rosenbrock nke N variables zuru oke, nke nwere ụdị:
N'agbanyeghị eziokwu ahụ na ọrụ Rosenbrock na Jacobi na Hessian matrices (nke mbụ na nke abụọ, n'otu n'otu) akọwaworị na ngwugwu scipy.optimize, anyị ga-akọwa ya n'onwe anyị.
import numpy as np
def rosen(x):
"""The Rosenbrock function"""
return np.sum(100.0*(x[1:]-x[:-1]**2.0)**2.0 + (1-x[:-1])**2.0, axis=0)Maka idoanya, ka anyị see na 3D ụkpụrụ nke ọrụ Rosenbrock nke mgbanwe abụọ.
Koodu eserese
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
# Настраиваем 3D график
fig = plt.figure(figsize=[15, 10])
ax = fig.gca(projection='3d')
# Задаем угол обзора
ax.view_init(45, 30)
# Создаем данные для графика
X = np.arange(-2, 2, 0.1)
Y = np.arange(-1, 3, 0.1)
X, Y = np.meshgrid(X, Y)
Z = rosen(np.array([X,Y]))
# Рисуем поверхность
surf = ax.plot_surface(X, Y, Z, cmap=cm.coolwarm)
plt.show()
Ịmara tupu oge eruo na opekempe bụ 0 na 
, Ka anyị leba anya n'ihe atụ nke otu esi achọpụta uru kacha nta nke ọrụ Rosenbrock site na iji usoro scipy dị iche iche.
Usoro Nelder-Mead simplex
Ka e nwee isi mmalite x0 na oghere akụkụ ise. Ka anyị chọta ebe kacha nta nke ọrụ Rosenbrock kacha nso ya site na iji algọridim (a kọwara algọridim dị ka uru nke usoro usoro):
from scipy.optimize import minimize
x0 = np.array([1.3, 0.7, 0.8, 1.9, 1.2])
res = minimize(rosen, x0, method='nelder-mead',
options={'xtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 339
Function evaluations: 571
[1. 1. 1. 1. 1.]Usoro simplex bụ ụzọ kachasị mfe iji belata ọrụ akọwapụtara nke ọma na nke dabara adaba. Ọ dịghị achọ ịgbakọ mpụta nke otu ọrụ zuru ezu ịkọwapụta naanị ụkpụrụ ya. Usoro Nelder-Mead bụ ezigbo nhọrọ maka nsogbu mbelata dị mfe. Agbanyeghị, ebe ọ naghị eji atụmatụ gradient, ọ nwere ike were ogologo oge ịchọta nke kacha nta.
Usoro Powell
Algọridim njikarịcha ọzọ nke a na-agbakọ naanị ụkpụrụ ọrụ bụ . Iji jiri ya, ịkwesịrị ịtọ usoro = 'powell' na ọrụ minim.
x0 = np.array([1.3, 0.7, 0.8, 1.9, 1.2])
res = minimize(rosen, x0, method='powell',
options={'xtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 19
Function evaluations: 1622
[1. 1. 1. 1. 1.]Broyden-Fletcher-Goldfarb-Shanno (BFGS) algọridim
Iji nweta ngwa ngwa convergence na ngwọta, usoro na-eji gradient nke ebumnobi ọrụ. Enwere ike ịkọwa gradient dị ka ọrụ ma ọ bụ gbakọọ site na iji ndịiche usoro mbụ. N'ọnọdụ ọ bụla, usoro BFGS na-achọkarị oku ọrụ ole na ole karịa usoro simplex.
Ka anyị chọta ihe mgbapụta nke ọrụ Rosenbrock n'ụdị nyocha:
Okwu a bara uru maka mwepụta nke mgbanwe niile ma ewezuga nke mbụ na nke ikpeazụ, nke akọwara dị ka:
Ka anyị lelee ọrụ Python na-agbakọ gradient a:
def rosen_der (x):
xm = x [1: -1]
xm_m1 = x [: - 2]
xm_p1 = x [2:]
der = np.zeros_like (x)
der [1: -1] = 200 * (xm-xm_m1 ** 2) - 400 * (xm_p1 - xm ** 2) * xm - 2 * (1-xm)
der [0] = -400 * x [0] * (x [1] -x [0] ** 2) - 2 * (1-x [0])
der [-1] = 200 * (x [-1] -x [-2] ** 2)
return derA na-akọwapụta ọrụ mgbako gradient dị ka uru nke paramita jac nke ọrụ minim, dị ka egosiri n'okpuru.
res = minimize(rosen, x0, method='BFGS', jac=rosen_der, options={'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 25
Function evaluations: 30
Gradient evaluations: 30
[1.00000004 1.0000001 1.00000021 1.00000044 1.00000092]Conjugate gradient algọridim (Newton)
Algorithm bụ usoro Newton gbanwetụrụ.
Usoro Newton gbadoro ụkwụ na ime ka otu ọrụ dị n'ime mpaghara site na ọnụọgụ nke ogo nke abụọ:
ebe 
bụ matriks nke usoro nke abụọ (Hessian matrix, Hessian).
Ọ bụrụ na Hessian bụ ihe ziri ezi, mgbe ahụ enwere ike ịchọta opekempe mpaghara ọrụ a site n'ịkọba gradient efu nke ụdị quadratic ka ọ bụrụ efu. Nsonaazụ ga-abụ okwu a:
A na-agbakọ Hessian ntụgharị site na iji usoro gradient conjugate. Enyere ihe atụ nke iji usoro a iji belata ọrụ Rosenbrock n'okpuru. Iji jiri usoro Newton-CG, ị ga-akọwarịrị ọrụ na-agbakọ Hessian.
Ọrụ Hessian nke Rosenbrock n'ụdị nyocha hà nhata:
ebe 
и 
, kọwaa matriks 
.
Ihe ndị fọdụrụ na-abụghị efu nke matriks hà nhata:
Dịka ọmụmaatụ, na oghere akụkụ ise N = 5, matrix Hessian maka ọrụ Rosenbrock nwere ụdị nke band:
Koodu na-agbakọ Hessian a yana koodu maka ibelata ọrụ Rosenbrock site na iji usoro conjugate gradient (Newton):
def rosen_hess(x):
x = np.asarray(x)
H = np.diag(-400*x[:-1],1) - np.diag(400*x[:-1],-1)
diagonal = np.zeros_like(x)
diagonal[0] = 1200*x[0]**2-400*x[1]+2
diagonal[-1] = 200
diagonal[1:-1] = 202 + 1200*x[1:-1]**2 - 400*x[2:]
H = H + np.diag(diagonal)
return H
res = minimize(rosen, x0, method='Newton-CG',
jac=rosen_der, hess=rosen_hess,
options={'xtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 24
Function evaluations: 33
Gradient evaluations: 56
Hessian evaluations: 24
[1. 1. 1. 0.99999999 0.99999999]Ihe atụ nwere nkọwa nke ọrụ ngwaahịa nke Hessian na vector aka ike
N'ime nsogbu ụwa n'ezie, ịgbakọ na ịchekwa matriks Hessian niile nwere ike ịchọ oge dị ukwuu na akụrụngwa ebe nchekwa. N'okwu a, ọ dịghị mkpa n'ezie ịkọwapụta matrix Hessian n'onwe ya, n'ihi na Usoro mbelata chọrọ naanị vector nhata ngwaahịa nke Hessian nwere vector ọzọ aka ike. Ya mere, site n'echiche mgbakọ na mwepụ, ọ ka mma ozugbo ịkọwapụta ọrụ na-eweghachi nsonaazụ ngwaahịa nke Hessian na vector aka ike.
Tụlee ọrụ hess, nke na-ewere vector mbelata dị ka arụmụka mbụ, yana vector aka ike dị ka arụmụka nke abụọ (yana arụmụka ndị ọzọ nke ọrụ a ga-ebelata). N'ọnọdụ anyị, ịgbakọ ngwaahịa nke Hessian nke ọrụ Rosenbrock na vector aka ike adịghị esiri ike. Ọ bụrụ p bụ vector aka ike, mgbe ahụ ngwaahịa 
dị ka:
A na-agafe ọrụ na-agbakọ ngwaahịa nke Hessian na vector aka ike dị ka uru nke arụmụka hessp iji wedata ọrụ:
def rosen_hess_p(x, p):
x = np.asarray(x)
Hp = np.zeros_like(x)
Hp[0] = (1200*x[0]**2 - 400*x[1] + 2)*p[0] - 400*x[0]*p[1]
Hp[1:-1] = -400*x[:-2]*p[:-2]+(202+1200*x[1:-1]**2-400*x[2:])*p[1:-1]
-400*x[1:-1]*p[2:]
Hp[-1] = -400*x[-2]*p[-2] + 200*p[-1]
return Hp
res = minimize(rosen, x0, method='Newton-CG',
jac=rosen_der, hessp=rosen_hess_p,
options={'xtol': 1e-8, 'disp': True})
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 24
Function evaluations: 33
Gradient evaluations: 56
Hessian evaluations: 66Conjugate gradient Trust region algọridim (Newton)
Ndozigharị nke matriks Hessian adịghị mma yana ntụzịaka ọchụchọ na-ezighi ezi nwere ike ime ka Newton conjugate gradient algọridim ghara ịdị irè. N'ọnọdụ ndị dị otú ahụ, a na-enye mmasị (mpaghara ntụkwasị obi) jikọtara Newton gradients.
Ọmụmaatụ na nkọwapụta nke matrix Hessian:
res = minimize(rosen, x0, method='trust-ncg',
jac=rosen_der, hess=rosen_hess,
options={'gtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 20
Function evaluations: 21
Gradient evaluations: 20
Hessian evaluations: 19
[1. 1. 1. 1. 1.]Ọmụmatụ na ọrụ ngwaahịa nke Hessian na vector aka ike:
res = minimize(rosen, x0, method='trust-ncg',
jac=rosen_der, hessp=rosen_hess_p,
options={'gtol': 1e-8, 'disp': True})
print(res.x)Optimization terminated successfully.
Current function value: 0.000000
Iterations: 20
Function evaluations: 21
Gradient evaluations: 20
Hessian evaluations: 0
[1. 1. 1. 1. 1.]Krylov ụdị usoro
Dị ka usoro ntụkwasị obi-ncg, ụzọ ụdị Krylov dabara nke ọma maka idozi nsogbu buru ibu n'ihi na ha na-eji naanị ngwaahịa matrix-vector. Ihe kachasị mkpa ha bụ iji dozie nsogbu na mpaghara ntụkwasị obi nke obere mpaghara Krylov gbajiri agbaji. Maka nsogbu ndị a na-ejighị n'aka, ọ ka mma iji usoro a, ebe ọ bụ na ọ na-eji ọnụ ọgụgụ dị nta nke ntanaka na-adịghị mma n'ihi ọnụ ọgụgụ dị nta nke ngwaahịa matrix-vector per subproblem, ma e jiri ya tụnyere usoro ntụkwasị obi-ncg. Na mgbakwunye, a na-achọta ihe ngwọta maka nsogbu quadratic nke ọma karịa iji usoro ntụkwasị obi-ncg.
Ọmụmaatụ na nkọwapụta nke matrix Hessian:
res = minimize(rosen, x0, method='trust-krylov',
jac=rosen_der, hess=rosen_hess,
options={'gtol': 1e-8, 'disp': True})
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 19
Function evaluations: 20
Gradient evaluations: 20
Hessian evaluations: 18
print(res.x)
[1. 1. 1. 1. 1.]
Ọmụmatụ na ọrụ ngwaahịa nke Hessian na vector aka ike:
res = minimize(rosen, x0, method='trust-krylov',
jac=rosen_der, hessp=rosen_hess_p,
options={'gtol': 1e-8, 'disp': True})
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 19
Function evaluations: 20
Gradient evaluations: 20
Hessian evaluations: 0
print(res.x)
[1. 1. 1. 1. 1.]
Algorithm maka ngwọta dị nso na mpaghara ntụkwasị obi
Usoro niile (Newton-CG, Trust-ncg na Trust-krylov) dabara nke ọma maka idozi nsogbu buru ibu (na ọtụtụ puku mgbanwe). Nke a bụ n'ihi n'eziokwu na conjugate gradient algọridim na-egosi mkpebi siri ike nke matrix Hessian inverse. A na-achọta azịza ya ugboro ugboro, na-enweghị mgbasawanye nke Hessian. Ebe ọ bụ na naanị ị chọrọ ịkọwa ọrụ maka ngwaahịa nke Hessian na vector aka ike, algọridim a dị mma karịsịa maka ịrụ ọrụ na matrices sparse (band diagonal). Nke a na-enye ọnụ ahịa ebe nchekwa dị ala yana nchekwa oge dị ukwuu.
Maka nsogbu ndị nwere ọkara, ọnụ ahịa ịchekwa na ịmepụta Hessian adịghị oke egwu. Nke a pụtara na ọ ga-ekwe omume ịnweta ngwọta na ntanye ntanye, na-edozi nsogbu ndị dị na mpaghara ntụkwasị obi fọrọ nke nta ka ọ bụrụ kpọmkwem. Iji mee nke a, a na-edozi ụfọdụ nhata na-abụghị nke ọnla n'otu oge maka nsogbu dị n'akụkụ anọ ọ bụla. Ngwọta dị otú ahụ na-achọkarị 3 ma ọ bụ 4 cholesky decompositions nke Hessian matrix. N'ihi nke a, usoro a na-agbakọta na ntanye ole na ole ma na-achọ mgbako ọrụ ebumnobi ole na ole karịa usoro mpaghara ntụkwasị obi ndị ọzọ etinyere. Algọridim a na-egosi naanị mkpebi nke matriks Hessian zuru oke ma ọ kwadoghị ikike iji ọrụ ngwaahịa nke Hessian na vector aka ike.
Ọmụmaatụ na mbelata ọrụ Rosenbrock:
res = minimize(rosen, x0, method='trust-exact',
jac=rosen_der, hess=rosen_hess,
options={'gtol': 1e-8, 'disp': True})
res.xOptimization terminated successfully.
Current function value: 0.000000
Iterations: 13
Function evaluations: 14
Gradient evaluations: 13
Hessian evaluations: 14
array([1., 1., 1., 1., 1.])Anyị ga-akwụsị ebe ahụ. N'isiokwu na-esonụ, m ga-agbalị ịgwa ndị kasị akpali ihe banyere ọnọdụ mbelata, ngwa nke mbelata na-edozi nsogbu nso nso, ibelata a ọrụ nke otu mgbanwe, aka ike minimizers, na-achọta mgbọrọgwụ nke a usoro nke equations iji scipy.optimize. ngwugwu.
isi:
isi: www.habr.com
