Optimization: Introduction
Posted juliaai123
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Optimization: Introduction相关的知识,希望对你有一定的参考价值。
CanChen
ggchen@mail.ustc.edu.cn
Recently I am following Optimization lectured by Prof.Zhouwang Yang at USTC. Opt lies at the heart of ML.
Why I choose his course? I am familiar with USTC courses and have a motive inside myself to acquire every knowledge in USTC courses‘ slides.
Now I have already followed most of this course and so I decide to write some blogs chapter by chapter to deepen my understanding and record my learning process in the meantime.
The first blog is introduction.
Definition
We use x to denote decision variable, S to denote feasible region, f to denote objective function.
Then optimization is to minimize f(x) in the scope S.
If S is available in the whole space, then this is an unconstrained optimization problem.
Otherwise, this is a linear programming problem if S only contains linear constraints and an non-linear programming problem if S contains nonlinear constraints.
Opt conditions
Unconstrain
The condition is very simple and we just need to know its first-order and/or second-order derivates if they exist.
Equation Constraint
Lagrange mutiplier. One thing that confused me when I first saw the lagrange equation is that the equation‘s derivate on the mutiplier is zero. Then I realied it is the original equation constraint in fact.
Inequation Constraint
Lagrange KKT. The key to understand KKT is determine whether inequation becomes equation when it meets the optimal point.
以上是关于Optimization: Introduction的主要内容,如果未能解决你的问题,请参考以下文章
Index Condition Pushdown Optimization
Adaptive Particle Swarm Optimization
Improving DNNs Hyperparameter tuning-Regularization and Optimization(week2)Optimization Methods(示例代码