如何在带有边界的python优化中找到全局最小值?

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了如何在带有边界的python优化中找到全局最小值?相关的知识,希望对你有一定的参考价值。

我有一个包含64个变量的Python函数,我尝试在最小化函数中使用L-BFGS-B方法对其进行优化,但是这种方法非常依赖于初始猜测,并且未能找到全局最小值。

但我喜欢它为变量设置界限的能力。是否存在一种方法/函数来查找全局最小值,同时具有变量的边界?

答案

这可以用scipy.optimize.basinhopping完成。流域购物是一种旨在找到目标函数的全局最小值的函数。它使用函数scipy.optimize.minimize进行重复最小化,并在每次最小化后在坐标空间中进行随机步骤。通过使用实现边界的最小化器之一(例如L-BFGS-B),流域购物仍然可以遵守边界。以下是一些显示如何执行此操作的代码

# an example function with multiple minima
def f(x): return x.dot(x) + sin(np.linalg.norm(x) * np.pi)

# the starting point
x0 = [10., 10.]

# the bounds
xmin = [1., 1.]
xmax = [11., 11.]

# rewrite the bounds in the way required by L-BFGS-B
bounds = [(low, high) for low, high in zip(xmin, xmax)]

# use method L-BFGS-B because the problem is smooth and bounded
minimizer_kwargs = dict(method="L-BFGS-B", bounds=bounds)
res = basinhopping(f, x0, minimizer_kwargs=minimizer_kwargs)
print res

上面的代码适用于一个简单的情况,但是如果盆地随机位移例程将你带到那里,你仍然可以最终进入一个禁区。幸运的是,可以通过使用关键字take_step传递自定义步骤来重写

class RandomDisplacementBounds(object):
    """random displacement with bounds"""
    def __init__(self, xmin, xmax, stepsize=0.5):
        self.xmin = xmin
        self.xmax = xmax
        self.stepsize = stepsize

    def __call__(self, x):
        """take a random step but ensure the new position is within the bounds"""
        while True:
            # this could be done in a much more clever way, but it will work for example purposes
            xnew = x + np.random.uniform(-self.stepsize, self.stepsize, np.shape(x))
            if np.all(xnew < self.xmax) and np.all(xnew > self.xmin):
                break
        return xnew

# define the new step taking routine and pass it to basinhopping
take_step = RandomDisplacementBounds(xmin, xmax)
result = basinhopping(f, x0, niter=100, minimizer_kwargs=minimizer_kwargs,
                      take_step=take_step)
print result
另一答案

一些常识性建议,用于调试和可视化函数上的任何优化器:

您的目标函数和约束是否合理? 如果目标函数是f() + g()的总和,则在x中单独打印所有"fx-opt.nptxt"(下图);如果f()是总和的99%而g()是1%,则调查。

约束:x_i中有多少成分xfinal被困在边界,x_i <= lo_i>= hi_i


How bumpy is your function on a global scale ?
Run with several random startpoints, and save the results to analyze / plot:
title = "%s  n %d  ntermhess %d  nsample %d  seed %d" % (  # all params!
    __file__, n, ntermhess, nsample, seed )
print title
...
np.random.seed(seed)  # for reproducible runs
np.set_printoptions( threshold=100, edgeitems=10, linewidth=100,
        formatter = dict( float = lambda x: "%.3g" % x ))  # float arrays %.3g

lo, hi = bounds.T  # vecs of numbers or +- np.inf
print "lo:", lo
print "hi:", hi

fx = []  # accumulate all the final f, x
for jsample in range(nsample):
        # x0 uniformly random in box lo .. hi --
    x0 = lo + np.random.uniform( size=n ) * (hi - lo)

    x, f, d = fmin_l_bfgs_b( func, x0, approx_grad=1,
                m=ntermhess, factr=factr, pgtol=pgtol )
    print "f: %g  x: %s  x0: %s" % (f, x, x0)
    fx.append( np.r_[ f, x ])

fx = np.array(fx)  # nsample rows, 1 + dim cols
np.savetxt( "fx-opt.nptxt", fx, fmt="%8.3g", header=title )  # to analyze / plot

ffinal = fx[:,0]
xfinal = fx[:,1:]
print "final f values, sorted:", np.sort(ffinal)
jbest = ffinal.argmin()
print "best x:", xfinal[jbest]

如果某些ffinal值看起来相当不错,那么在那些附近尝试更多随机起始点 - 这肯定比纯随机更好。

如果x是曲线,或任何真实的,绘制最好的几个x0xfinal。 (经验法则是qamplexswpoi尺寸的样本~5 * d或10 * d。太慢,太多?减少d / maxiter,减少maxeval - 你不需要ftol 1e-6进行探索。)

如果需要可重复的结果,则必须在ftol以及派生文件和图中列出所有相关参数。否则,你会问“这是从哪里来的?”


How bumpy is your function on epsilon scale ~ 10^-6 ?
Methods that approximate a gradient sometimes return their last estimate, but if not:
title

但是,如果在优化器退出之前梯度估计很差/不稳定,您将看不到。然后你必须保存所有中间from scipy.optimize._numdiff import approx_derivative # 3-point, much better than ## from scipy.optimize import approx_fprime for eps in [1e-3, 1e-6]: grad = approx_fprime( x, func, epsilon=eps ) print "approx_fprime eps %g: %s" % (eps, grad) 来观看它们;在python中很容易 - 问一下这是不是很清楚。

在一些问题区域,通常从声称的[f, x, approx_fprime]备份和重新启动。例如,如果您在乡村公路上迷路,首先找到一条主要道路,然后从那里重新开始。


Summary:
don't expect any black-box optimizer to work on a function that's large-scale bumpy, or epsilon-scale bumpy, or both.
Invest in test scaffolding, and in ways to see what the optimizer is doing.
另一答案

非常感谢你的详细回复,但是作为我对python的新手,我不知道如何在我的程序中实现代码,但这是我尝试优化:

xmin

优值函数将x0与其他一些值组合在一起,形成8条曲线的6个控制点,然后计算它们的长度,曲率半径等。它将最终的优点作为这些参数与一些权重的线性组合。

我使用低精度的x0=np.array((10, 13, f*2.5, 0.08, 10, f*1.5, 0.06, 20, 10, 14, f*2.5, 0.08, 10, f*1.75, 0.07, 20, 10, 15, f*2.5, 0.08, 10, f*2, 0.08, 20, 10, 16, f*2.5, 0.08, 10, f*2.25, 0.09, 20, 10, 17, f*2.5, -0.08, 10, f*2.5, -0.06, 20, 10, 18, f*2.5, -0.08, 10, f*2.75,-0.07, 20, 10, 19, f*2.5, -0.08, 10, f*3, -0.08, 20, 10, 20, f*2.5, -0.08, 10, f*3.25,-0.09, 20)) # boundary for each variable, each element in this restricts the corresponding element above bnds=((1,12), (1,35), (0,f*6.75), (-0.1, 0.1),(1,35), (0,f*6.75), (-0.1, 0.1),(13, 35), (1,12), (1,35), (0,f*6.75), (-0.1, 0.1),(1,35), (0,f*6.75), (-0.1, 0.1),(13, 35), (1,12), (1,35), (0,f*6.75), (-0.1, 0.1),(1,35), (0,f*6.75), (-0.1, 0.1),(13, 35), (1,12), (1,35), (0,f*6.75), (-0.1, 0.1),(1,35), (0,f*6.75), (-0.1, 0.1),(13, 35), (1,12), (1,35), (0,f*6.75), (-0.1, 0.1),(1,35), (0,f*6.75), (-0.1, 0.1),(13, 35), (1,12), (1,35), (0,f*6.75), (-0.1, 0.1),(1,35), (0,f*6.75), (-0.1, 0.1),(13, 35), (1,12), (1,35), (0,f*6.75), (-0.1, 0.1),(1,35), (0,f*6.75), (-0.1, 0.1),(13, 35), (1,12), (1,35), (0,f*6.75), (-0.1, 0.1),(1,35), (0,f*6.75), (-0.1, 0.1),(13, 35), ) from scipy.optimize import basinhopping from scipy.optimize import minimize merit=a*meritoflength + b*meritofROC + c*meritofproximity +d*(distancetoceiling+distancetofloor)+e*heightorder minimizer_kwargs = {"method": "L-BFGS-B", "bounds": bnds, "tol":1e0} ret = basinhopping(merit_function, x0, minimizer_kwargs=minimizer_kwargs, niter=10, T=0.01) zoom = ret['x'] res = minimize(merit_function, zoom, method = 'L-BFGS-B', bounds=bnds, tol=1e-5) print res 来找到一些最小值,然后使用basinhopping来提高最低值的精度。

附:我正在运行的平台是Enthoght canopy 1.3.0,numpy 1.8.0 scipy 0.13.2 mac 10.8.3

以上是关于如何在带有边界的python优化中找到全局最小值?的主要内容,如果未能解决你的问题,请参考以下文章

如何从 DataFrame,Python-3 中找到前 N 个最小值

在连续数据中查找全局最小值

Mathematica 中 Minimize函数无法找到全局最小值时的解决方法

从旋转的矩形计算边界框坐标

如何有效地找到 NumPy 中平滑多维数组的局部最小值?

如何在python中找到输入数字的平均值、最小值、最大值和范围?