Advanced Optimization（高级优化）

Posted 2020-10-03 晓语听风

高级优化在octave中有封装函数，需要代价函数和每个theta对应代价函数的偏导实现，算是一个比较高级的梯度下降算法，免除人·工设定学习率

Note: [7:35 - \'100\' should be 100 instead. The value provided should be an integer and not a character string.]

"Conjugate gradient", "BFGS", and "L-BFGS" are more sophisticated, faster ways to optimize θ that can be used instead of gradient descent. We suggest that you should not write these more sophisticated algorithms yourself (unless you are an expert in numerical computing) but use the libraries instead, as they\'re already tested and highly optimized. Octave provides them.

We first need to provide a function that evaluates the following two functions for a given input value θ:

We can write a single function that returns both of these:

function [jVal, gradient] = costFunction(theta)
  jVal = [...code to compute J(theta)...];
  gradient = [...code to compute derivative of J(theta)...];
end

Then we can use octave\'s "fminunc()" optimization algorithm along with the "optimset()" function that creates an object containing the options we want to send to "fminunc()". (Note: the value for MaxIter should be an integer, not a character string - errata in the video at 7:30)

options = optimset(\'GradObj\', \'on\', \'MaxIter\', 100);
initialTheta = zeros(2,1);
   [optTheta, functionVal, exitFlag] = fminunc(@costFunction, initialTheta, options);

　We give to the function "fminunc()" our cost function, our initial vector of theta values, and the "options" object that we created beforehand.