目录：Matrix Differential Calculus with Applications in Statistics and Econometrics,3rd_[Magnus2019](代码

Posted 2021-12-25 ourweiguan

目录：Matrix Differential Calculus with Applications in Statistics and Econometrics,3rd_[Magnus2019]

Title   -16
Contents    -14
Preface -6
Part One — Matrices 1
1 Basic properties of vectors and matrices  3
    1.1 Introduction    3
    1.2 Sets    3
    1.3 Matrices: addition and multiplication   4
    1.4 The transpose of a matrix   6
    1.5 Square matrices 6
    1.6 Linear forms and quadratic forms    7
    1.7 The rank of a matrix    9
    1.8 The inverse 10
    1.9 The determinant 10
    1.10 The trace  11
    1.11 Partitioned matrices   12
    1.12 Complex matrices   14
    1.13 Eigenvalues and eigenvectors   14
    1.14 Schur’s decomposition theorem  17
    1.15 The Jordan decomposition   18
    1.16 The singular-value decomposition   20
    1.17 Further results concerning eigenvalues 20
    1.18 Positive (semi)definite matrices   23
    1.19 Three further results for positive definite matrices   25
    1.20 A useful result    26
    1.21 Symmetric matrix functions 27
    Miscellaneous exercises 28
    Bibliographical notes   30
2 Kronecker products, vec operator, and Moore-Penrose inverse   31
    2.1 Introduction    31
    2.2 The Kronecker product   31
    2.3 Eigenvalues of a Kronecker product  33
    2.4 The vec operator    34
    2.5 The Moore-Penrose (MP) inverse  36
    2.6 Existence and uniqueness of the MP inverse  37
    2.7 Some properties of the MP inverse   38
    2.8 Further properties  39
    2.9 The solution of linear equation systems 41
    Miscellaneous exercises 43
    Bibliographical notes   45
3 Miscellaneous matrix results  47
    3.1 Introduction    47
    3.2 The adjoint matrix  47
    3.3 Proof of Theorem 3.1    49
    3.4 Bordered determinants   51
    3.5 The matrix equation AX = 0  51
    3.6 The Hadamard product    52
    3.7 The commutation matrix K mn 54
    3.8 The duplication matrix D n  56
    3.9 Relationship between D n+1 and D n , I  58
    3.10 Relationship between D n+1 and D n , II    59
    3.11 Conditions for a quadratic form to be positive (negative) subject to linear constraints    60
    12 Necessary and sufficient conditions for r(A : B) = r(A) + r(B)63
    13 The bordered Gramian matrix  65
    14 The equations X 1 A + X 2 B ′ = G 1 ,X 1 B = G 2 67
    Miscellaneous exercises 69
    Bibliographical notes   70
Part Two — Differentials: the theory
4 Mathematical preliminaries    73
    4.1 Introduction    73
    4.2 Interior points and accumulation points 73
    4.3 Open and closed sets    75
    4.4 The Bolzano-Weierstrass theorem 77
    4.5 Functions   78
    4.6 The limit of a function 79
    4.7 Continuous functions and compactness    80
    4.8 Convex sets 81
    4.9 Convex and concave functions    83
    Bibliographical notes   86
5 Differentials and differentiability   87
    5.1 Introduction    87
    5.2 Continuity  88
    5.3 Differentiability and linear approximation  90
    5.4 The differential of a vector function   91
    5.5 Uniqueness of the differential  93
    5.6 Continuity of differentiable functions  94
    5.7 Partial derivatives 95
    5.8 The first identification theorem    96
    5.9 Existence of the differential, I    97
    5.10 Existence of the differential, II  99
    5.11 Continuous differentiability   100
    5.12 The chain rule 100
    5.13 Cauchy invariance  102
    5.14 The mean-value theorem for real-valued functions   103
    5.15 Differentiable matrix functions    104
    5.16 Some remarks on notation   106
    5.17 Complex differentiation    108
    Miscellaneous exercises 110
    Bibliographical notes   110
6 The second differential   111
    6.1 Introduction    111
    6.2 Second-order partial derivatives    111
    6.3 The Hessian matrix  112
    6.4 Twice differentiability and second-order approximation, I   113
    6.5 Definition of twice differentiability   114
    6.6 The second differential 115
    6.7 Symmetry of the Hessian matrix  117
    6.8 The second identification theorem   119
    6.9 Twice differentiability and second-order approximation, II  119
    6.10 Chain rule for Hessian matrices    121
    6.11 The analog for second differentials    123
    6.12 Taylor’s theorem for real-valued functions 124
    6.13 Higher-order differentials 125
    6.14 Real analytic functions    125
    6.15 Twice differentiable matrix functions  126
    Bibliographical notes   127
7 Static optimization   129
    7.1 Introduction    129
    7.2 Unconstrained optimization  130
    7.3 The existence of absolute extrema   131
    7.4 Necessary conditions for a local minimum    132
    7.5 Sufficient conditions for a local minimum: first-derivative test    134
    7.6 Sufficient conditions for a local minimum: second-derivative test   136
    7.7 Characterization of differentiable convex functions 138
    7.8 Characterization of twice differentiable convex functions   141
    7.9 Sufficient conditions for an absolute minimum   142
    7.10 Monotonic transformations  143
    7.11 Optimization subject to constraints    144
    7.12 Necessary conditions for a local minimum under constraints 145
    7.13 Sufficient conditions for a local minimum under constraints    149
    7.14 Sufficient conditions for an absolute minimum under constraints    154
    7.15 A note on constraints in matrix form   155
    7.16 Economic interpretation of Lagrange multipliers    155
    Appendix: the implicit function theorem 157
    Bibliographical notes   159
Part Three — Differentials: the practice    161
8 Some important differentials  163
    8.1 Introduction    163
    8.2 Fundamental rules of differential calculus  163
    8.3 The differential of a determinant   165
    8.4 The differential of an inverse  168
    8.5 Differential of the Moore-Penrose inverse   169
    8.6 The differential of the adjoint matrix  172
    8.7 On differentiating eigenvalues and eigenvectors 174
    8.8 The continuity of eigenprojections  176
    8.9 The differential of eigenvalues and eigenvectors: symmetric case    180
    8.10 Two alternative expressions for dλ 183
    8.11 Second differential of the eigenvalue function 185
    Miscellaneous exercises 186
    Bibliographical notes   189
9 First-order differentials and Jacobian matrices   191
    9.1 Introduction    191
    9.2 Classification  192
    9.3 Derisatives 192
    9.4 Derivatives 194
    9.5 Identification of Jacobian matrices 196
    9.6 The first identification table  197
    9.7 Partitioning of the derivative  197
    9.8 Scalar functions of a scalar    198
    9.9 Scalar functions of a vector    198
    9.10 Scalar functions of a matrix, I: trace 199
    9.11 Scalar functions of a matrix, II: determinant  201
    9.12 Scalar functions of a matrix, III: eigenvalue  202
    9.13 Two examples of vector functions   203
    9.14 Matrix functions   204
    9.15 Kronecker products 206
    9.16 Some other problems    208
    9.17 Jacobians of transformations   209
    Bibliographical notes   210
10 Second-order differentials and Hessian matrices  211
    10.1 Introduction   211
    10.2 The second identification table    211
    10.3 Linear and quadratic forms 212
    10.4 A useful theorem   213
    10.5 The determinant function   214
    10.6 The eigenvalue function    215
    10.7 Other examples 215
    10.8 Composite functions    217
    10.9 The eigenvector function   218
    10.10 Hessian of matrix functions, I    219
    10.11 Hessian of matrix functions, II   219
    Miscellaneous exercises 220
Part Four — Inequalities    223
11 Inequalities 225
    11.1 Introduction   225
    11.2 The Cauchy-Schwarz inequality  226
    11.3 Matrix analogs of the Cauchy-Schwarz inequality    227
    11.4 The theorem of the arithmetic and geometric means  228
    11.5 The Rayleigh quotient  230
    11.6 Concavity of λ 1 and convexity of λ n  232
    11.7 Variational description of eigenvalues 232
    11.8 Fischer’s min-max theorem  234
    11.9 Monotonicity of the eigenvalues    236
    11.10 The Poincar′ e separation theorem 236
    11.11 Two corollaries of Poincar′ e’s theorem   237
    11.12 Further consequences of the Poincar′ e theorem    238
    11.13 Multiplicative version    239
    11.14 The maximum of a bilinear form    241
    11.15 Hadamard’s inequality 242
    11.16 An interlude: Karamata’s inequality   242
    11.17 Karamata’s inequality and eigenvalues 244
    11.18 An inequality concerning positive semidefinite matrices   245
    11.19 A representation theorem for (Papi) 1/p   246
    11.20 A representation theorem for (trA p ) 1/p 247
    11.21 H?lder’s inequality   248
    11.22 Concavity of log|A|   250
    11.23 Minkowski’s inequality    251
    11.24 Quasilinear representation of |A| 1/n 253
    11.25 Minkowski’s determinant theorem   255
    11.26 Weighted means of order p 256
    11.27 Schl¨ omilch’s inequality 258
    11.28 Curvature properties of M p (x,a) 259
    11.29 Least squares 260
    11.30 Generalized least squares 261
    11.31 Restricted least squares  262
    11.32 Restricted least squares: matrix version  264
    Miscellaneous exercises 265
    Bibliographical notes   269
Part Five — The linear model    271
12 Statistical preliminaries    273
    12.1 Introduction   273
    12.2 The cumulative distribution function   273
    12.3 The joint density function 274
    12.4 Expectations   274
    12.5 Variance and covariance    275
    12.6 Independence of two random variables   277
    12.7 Independence of n random variables 279
    12.8 Sampling   279
    12.9 The one-dimensional normal distribution    279
    12.10 The multivariate normal distribution  280
    12.11 Estimation    282
    Miscellaneous exercises 282
    Bibliographical notes   283
13 The linear regression model  285
    13.1 Introduction   285
    13.2 Affine minimum-trace unbiased estimation   286
    13.3 The Gauss-Markov theorem   287
    13.4 The method of least squares    290
    13.5 Aitken’s theorem   291
    13.6 Multicollinearity  293
    13.7 Estimable functions    295
    13.8 Linear constraints: the case M(R ′ ) ? M(X ′ ) 296
    13.9 Linear constraints: the general case   300
    13.10 Linear constraints: the case M(R ′ ) ∩ M(X ′ ) = 0  302
    13.11 A singular variance matrix: the case M(X) ? M(V ) 304
    13.12 A singular variance matrix: the case r(X'V+ X) = r(X) 305
    13.13 A singular variance matrix: the general case, I   307
    13.14 Explicit and implicit linear constraints  307
    13.15 The general linear model, I   310
    13.16 A singular variance matrix: the general case, II  311
    13.17 The general linear model, II  314
    13.18 Generalized least squares 315
    13.19 Restricted least squares  316
    Miscellaneous exercises 318
    Bibliographical notes   319
14 Further topics in the linear model   321
    14.1 Introduction   321
    14.2 Best quadratic unbiased estimation of σ 2  322
    14.3 The best quadratic and positive unbiased estimator of σ 2  322
    14.4 The best quadratic unbiased estimator of σ 2   324
    14.5 Best quadratic invariant estimation of σ 2 326
    14.6 The best quadratic and positive invariant estimator of σ 2 327
    14.7 The best quadratic invariant estimator of σ 2  329
    14.8 Best quadratic unbiased estimation: multivariate normal case   330
    14.9 Bounds for the bias of the least-squares estimator of σ 2 , I  332
    14.10 Bounds for the bias of the least-squares estimator of σ 2 , II    333
    14.11 The prediction of disturbances    335
    14.12 Best linear unbiased predictors with scalar variance matrix   336
    14.13 Best linear unbiased predictors with fixed variance matrix, I 338
    14.14 Best linear unbiased predictors with fixed variance matrix, II    340
    14.15 Local sensitivity of the posterior mean   341
    14.16 Local sensitivity of the posterior precision  342
    Bibliographical notes   344
Part Six — Applications to maximum likelihood estimation    345
15 Maximum likelihood estimation    347
    15.1 Introduction   347
    15.2 The method of maximum likelihood (ML)  347
    15.3 ML estimation of the multivariate normal distribution  348
    15.4 Symmetry: implicit versus explicit treatment   350
    15.5 The treatment of positive definiteness 351
    15.6 The information matrix 352
    15.7 ML estimation of the multivariate normal distribution:distinct means   354
    15.8 The multivariate linear regression model   354
    15.9 The errors-in-variables model  357
    15.10 The nonlinear regression model with normal errors 359
    15.11 Special case: functional independence of mean and variance parameters 361
    15.12 Generalization of Theorem 15.6    362
    Miscellaneous exercises 364
    Bibliographical notes   365
16 Simultaneous equations   367
    16.1 Introduction   367
    16.2 The simultaneous equations model   367
    16.3 The identification problem 369
    16.4 Identification with linear constraints on B and Γ only 371
    16.5 Identification with linear constraints on B, Γ, and Σ  371
    16.6 Nonlinear constraints  373
    16.7 FIML: the information matrix (general case)    374
    16.8 FIML: asymptotic variance matrix (special case)    376
    16.9 LIML: first-order conditions   378
    16.10 LIML: information matrix  381
    16.11 LIML: asymptotic variance matrix  383
    Bibliographical notes   388
17 Topics in psychometrics  389
    17.1 Introduction   389
    17.2 Population principal components    390
    17.3 Optimality of principal components 391
    17.4 A related result   392
    17.5 Sample principal components    393
    17.6 Optimality of sample principal components  395
    17.7 One-mode component analysis    395
    17.8 One-mode component analysis and sample principal components    398
    17.9 Two-mode component analysis    399
    17.10 Multimode component analysis  400
    17.11 Factor analysis   404
    17.12 A zigzag routine  407
    17.13 A Newton-Raphson routine  408
    17.14 Kaiser’s varimax method   412
    17.15 Canonical correlations and variates in the population 414
    17.16 Correspondence analysis   417
    17.17 Linear discriminant analysis  418
    Bibliographical notes   419
Part Seven — Summary    421
18 Matrix calculus: the essentials  423
    18.1 Introduction   423
    18.2 Differentials  424
    18.3 Vector calculus    426
    18.4 Optimization   429
    18.5 Least squares  431
    18.6 Matrix calculus    432
    18.7 Interlude on linear and quadratic forms    434
    18.8 The second differential    434
    18.9 Chain rule for second differentials    436
    18.10 Four examples 438
    18.11 The Kronecker product and vec operator    439
    18.12 Identification    441
    18.13 The commutation matrix    442
    18.14 From second differential to Hessian   443
    18.15 Symmetry and the duplication matrix   444
    18.16 Maximum likelihood    445
    Further reading 448
Bibliography    449
Index of symbols    467
Subject index   471