Machine Learning Week 3-advanced-optimization

Posted 2020-10-26 Barry

costfunction代码如下

function[jVal,gradient]=costFunction(theta)
jVal=(theta(1)-5)^2+(theta(2)-5)^2
gradient=zeros(2,1)
gradient(1)=2*(theta(1)-5);
gradient(2)=2*(theta(2)-5);

　　运行如下代码段

options=optimset(‘GradObj‘,‘on‘,‘MaxIter‘,100)
initialTheta=zeros(2,1)
[optTheta,functionVal,exitFlag]=fminunc(@costFunction,initialTheta,options)

　　结果如下

 1 jVal =
 2 
 3     50
 4 
 5 
 6 gradient =
 7 
 8      0
 9      0
10 
11 
12 jVal =
13 
14    50.0000
15 
16 
17 gradient =
18 
19      0
20      0
21 
22 
23 jVal =
24 
25    50.0000
26 
27 
28 gradient =
29 
30      0
31      0
32 
33 
34 jVal =
35 
36      0
37 
38 
39 gradient =
40 
41      0
42      0
43 
44 
45 jVal =
46 
47    5.5511e-15
48 
49 
50 gradient =
51 
52      0
53      0
54 
55 
56 jVal =
57 
58    5.5511e-15
59 
60 
61 gradient =
62 
63      0
64      0
65 
66 
67 Local minimum found.
68 
69 Optimization completed because the size of the gradient is less than
70 the default value of the optimality tolerance.
71 
72 <stopping criteria details>
73 
74 
75 optTheta =
76 
77      5
78      5
79 
80 
81 functionVal =
82 
83      0
84 
85 
86 exitFlag =
87 
88      1

exitflag帮助你了解是否收敛

 

 

fminunc（）函数

fminunc finds a local minimum of a function of several variables.
    X = fminunc(FUN,X0) starts at X0 and attempts to find a local minimizer
    X of the function FUN. FUN accepts input X and returns a scalar
    function value F evaluated at X. X0 can be a scalar, vector or matrix. 
 
    X = fminunc(FUN,X0,OPTIONS) minimizes with the default optimization
    parameters replaced by values in OPTIONS, an argument created with the
    OPTIMOPTIONS function.  See OPTIMOPTIONS for details. Use the
    SpecifyObjectiveGradient option to specify that FUN also returns a
    second output argument G that is the partial derivatives of the
    function df/dX, at the point X. Use the HessianFcn option to specify
    that FUN also returns a third output argument H that is the 2nd partial
    derivatives of the function (the Hessian) at the point X. The Hessian
    is only used by the trust-region algorithm.
 
    X = fminunc(PROBLEM) finds the minimum for PROBLEM. PROBLEM is a
    structure with the function FUN in PROBLEM.objective, the start point
    in PROBLEM.x0, the options structure in PROBLEM.options, and solver
    name ‘fminunc‘ in PROBLEM.solver. Use this syntax to solve at the 
    command line a problem exported from OPTIMTOOL. 
 
    [X,FVAL] = fminunc(FUN,X0,...) returns the value of the objective 
    function FUN at the solution X.
 
    [X,FVAL,EXITFLAG] = fminunc(FUN,X0,...) returns an EXITFLAG that
    describes the exit condition. Possible values of EXITFLAG and the
    corresponding exit conditions are listed below. See the documentation
    for a complete description.
 
      1  Magnitude of gradient small enough. 
      2  Change in X too small.
      3  Change in objective function too small.
      5  Cannot decrease function along search direction.
      0  Too many function evaluations or iterations.
     -1  Stopped by output/plot function.
     -3  Problem seems unbounded. 
    
    [X,FVAL,EXITFLAG,OUTPUT] = fminunc(FUN,X0,...) returns a structure 
    OUTPUT with the number of iterations taken in OUTPUT.iterations, the 
    number of function evaluations in OUTPUT.funcCount, the algorithm used 
    in OUTPUT.algorithm, the number of CG iterations (if used) in
    OUTPUT.cgiterations, the first-order optimality (if used) in
    OUTPUT.firstorderopt, and the exit message in OUTPUT.message.
 
    [X,FVAL,EXITFLAG,OUTPUT,GRAD] = fminunc(FUN,X0,...) returns the value 
    of the gradient of FUN at the solution X.
 
    [X,FVAL,EXITFLAG,OUTPUT,GRAD,HESSIAN] = fminunc(FUN,X0,...) returns the 
    value of the Hessian of the objective function FUN at the solution X.
 
    Examples
      FUN can be specified using @:
         X = fminunc(@myfun,2)
 
    where myfun is a MATLAB function such as:
 
        function F = myfun(x)
        F = sin(x) + 3;
 
      To minimize this function with the gradient provided, modify
      the function myfun so the gradient is the second output argument:
         function [f,g] = myfun(x)
          f = sin(x) + 3;
          g = cos(x);
      and indicate the gradient value is available by creating options with
      OPTIONS.SpecifyObjectiveGradient set to true (using OPTIMOPTIONS):
         options = optimoptions(‘fminunc‘,‘SpecifyObjectiveGradient‘,true);
         x = fminunc(@myfun,4,options);
 
      FUN can also be an anonymous function:
         x = fminunc(@(x) 5*x(1)^2 + x(2)^2,[5;1])
 
    If FUN is parameterized, you can use anonymous functions to capture the
    problem-dependent parameters. Suppose you want to minimize the 
    objective given in the function myfun, which is parameterized by its 
    second argument c. Here myfun is a MATLAB file function such as
 
      function [f,g] = myfun(x,c)
 
      f = c*x(1)^2 + 2*x(1)*x(2) + x(2)^2; % function
      g = [2*c*x(1) + 2*x(2)               % gradient
           2*x(1) + 2*x(2)];
 
    To optimize for a specific value of c, first assign the value to c. 
    Then create a one-argument anonymous function that captures that value 
    of c and calls myfun with two arguments. Finally, pass this anonymous 
    function to fminunc:
 
      c = 3;                              % define parameter first
      options = optimoptions(‘fminunc‘,‘SpecifyObjectiveGradient‘,true); % indicate gradient is provided 
      x = fminunc(@(x) myfun(x,c),[1;1],options)