Contents
Preface viii
1 Introduction 1
1 1 What is Econometrics? 1
1 2 The Probability Approach to Econometrics 1
1 3 Econometric Terms and Notation 2
1 4 Observational Data 3
1 5 Standard Data Structures 4
1 6 Sources for Economic Data 5
1 7 Econometric Software 7
1 8 Reading the Manuscript 7
1 9 Common Symbols 8
2 Conditional Expectation and Projection 9
2 1 Introduction 9
2 2 The Distribution of Wages 9
2 3 Conditional Expectation 11
2 4 Log Differences* 13
2 5 Conditional Expectation Function 14
2 6 Continuous Variables 15
2 7 Law of Iterated Expectations 16
2 8 CEF Error 18
2 9 Intercept-Only Model 19
2 10 Regression Variance 19
2 11 Best Predictor 20
2 12 Conditional Variance 21
2 13 Homoskedasticity and Heteroskedasticity 23
2 14 Regression Derivative 23
2 15 Linear CEF 24
2 16 Linear CEF with Nonlinear Effects 25
2 17 Linear CEF with Dummy Variables 26
2 18 Best Linear Predictor 28
2 19 Linear Predictor Error Variance 34
2 20 Regression Coefficients 35
2 21 Regression Sub-Vectors 35
2 22 Coefficient Decomposition 36
2 23 Omitted Variable Bias 37
2 24 Best Linear Approximation 38
2 25 Normal Regression 38
2 26 Regression to the Mean 39
2 27 Reverse Regression 40
2 28 Limitations of the Best Linear Predictor 41
i
CONTENTS ii
2 29 Random Coefficient Model 41
2 30 Causal Effects 43
2 31 Expectation: Mathematical Details* 47
2 32 Existence and Uniqueness of the Conditional Expectation* 49
2 33 Identification* 50 2 34 Technical Proofs* 51
Exercises 55
3 The Algebra of Least Squares 57
3 1 Introduction 57
3 2 Random Samples 57
3 3 Sample Means 58
3 4 Least Squares Estimator 58
3 5 Solving for Least Squares with One Regressor 59
3 6 Solving for Least Squares with Multiple Regressors 60
3 7 Illustration 62
3 8 Least Squares Residuals 62
3 9 Model in Matrix Notation 63
3 10 Projection Matrix 65
3 11 Orthogonal Projection 66
3 12 Estimation of Error Variance 67
3 13 Analysis of Variance 68
3 14 Regression Components 68
3 15 Residual Regression 70
3 16 Prediction Errors 71
3 17 Influential Observations 72
3 18 Normal Regression Model 74
3 19 CPS Data Set 76
3 20 Programming 78 3 21 Technical Proofs* 82
Exercises 83
4 Least Squares Regression 86
4 1 Introduction 86
4 2 Sample Mean 86
4 3 Linear Regression Model 87
4 4 Mean of Least-Squares Estimator 88
4 5 Variance of Least Squares Estimator 89
4 6 Gauss-Markov Theorem 91
4 7 Residuals 92
4 8 Estimation of Error Variance 93
4 9 Mean-Square Forecast Error 95
4 10 Covariance Matrix Estimation Under Homoskedasticity 96
4 11 Covariance Matrix Estimation Under Heteroskedasticity 97
4 12 Standard Errors 100
4 13 Computation 101
4 14 Measures of Fit 102
4 15 Empirical Example 103
4 16 Multicollinearity 105 4 17 Normal Regression Model 108 Exercises 110
CONTENTS iii
5 An Introduction to Large Sample Asymptotics 112
5 1 Introduction 112
5 2 Asymptotic Limits 112
5 3 Convergence in Probability 114
5 4 Weak Law of Large Numbers 115
5 5 Almost Sure Convergence and the Strong Law* 116
5 6 Vector-Valued Moments 117
5 7 Convergence in Distribution 118
5 8 Higher Moments 120
5 9 Functions of Moments 121
5 10 Delta Method 123
5 11 Stochastic Order Symbols 124
5 12 Uniform Stochastic Bounds* 126
5 13 Semiparametric Efficiency 127 5 14 Technical Proofs* 130
Exercises 134
6 Asymptotic Theory for Least Squares 135
6 1 Introduction 135
6 2 Consistency of Least-Squares Estimator 136
6 3 Asymptotic Normality 137
6 4 Joint Distribution 142
6 5 Consistency of Error Variance Estimators 144
6 6 Homoskedastic Covariance Matrix Estimation 145
6 7 Heteroskedastic Covariance Matrix Estimation 145
6 8 Summary of Covariance Matrix Notation 147
6 9 Alternative Covariance Matrix Estimators* 148
6 10 Functions of Parameters 149
6 11 Asymptotic Standard Errors 151
6 12 t statistic 153
6 13 Confidence Intervals 154
6 14 Regression Intervals 155
6 15 Forecast Intervals 157
6 16 Wald Statistic 158
6 17 Homoskedastic Wald Statistic 159
6 18 Confidence Regions 159
6 19 Semiparametric Efficiency in the Projection Model 160
6 20 Semiparametric Efficiency in the Homoskedastic Regression Model* 162
6 21 Uniformly Consistent Residuals* 164 6 22 Asymptotic Leverage* 165
Exercises 166
7 Restricted Estimation 169
7 1 Introduction 169
7 2 Constrained Least Squares 170
7 3 Exclusion Restriction 171
7 4 Minimum Distance 172
7 5 Asymptotic Distribution 173
7 6 Efficient Minimum Distance Estimator 174
7 7 Exclusion Restriction Revisited 175
7 8 Variance and Standard Error Estimation 177
7 9 Misspecification 177
CONTENTS iv
7 10 Nonlinear Constraints 179
7 11 Inequality Restrictions 180
7 12 Constrained MLE 181 7 13 Technical Proofs* 181
Exercises 183
8 Hypothesis Testing 185
8 1 Hypotheses 185
8 2 Acceptance and Rejection 186
8 3 Type I Error 187
8 4 t tests 187
8 5 Type II Error and Power 188
8 6 Statistical Significance 189
8 7 P-Values 190
8 8 t-ratios and the Abuse of Testing 192
8 9 Wald Tests 193
8 10 Homoskedastic Wald Tests 194
8 11 Criterion-Based Tests 195
8 12 Minimum Distance Tests 195
8 13 Minimum Distance Tests Under Homoskedasticity 196
8 14 F Tests 197
8 15 Likelihood Ratio Test 199
8 16 Problems with Tests of NonLinear Hypotheses 199
8 17 Monte Carlo Simulation 203
8 18 Confidence Intervals by Test Inversion 205
8 19 Power and Test Consistency 206
8 20 Asymptotic Local Power 207
8 21 Asymptotic Local Power, Vector Case 210 8 22 Technical Proofs* 212
Exercises 213
9 Regression Extensions 215
9 1 NonLinear Least Squares 215
9 2 Generalized Least Squares 218
9 3 Testing for Heteroskedasticity 221
9 4 Testing for Omitted NonLinearity 221
9 5 Least Absolute Deviations 222 9 6 Quantile Regression 224
Exercises 227
10 The Bootstrap 229
10 1 Definition of the Bootstrap 229
10 2 The Empirical Distribution Function 229
10 3 Nonparametric Bootstrap 231
10 4 Bootstrap Estimation of Bias and Variance 231
10 5 Percentile Intervals 232
10 6 Percentile-t Equal-Tailed Interval 234
10 7 Symmetric Percentile-t Intervals 234
10 8 Asymptotic Expansions 235
10 9 One-Sided Tests 237
10 10Symmetric Two-Sided Tests 238
10 11Percentile Confidence Intervals 239
CONTENTS v
10 12Bootstrap Methods for Regression Models 240
Exercises 242
11 NonParametric Regression 243
11 1 Introduction 243
11 2 Binned Estimator 243
11 3 Kernel Regression 245
11 4 Local Linear Estimator 246
11 5 Nonparametric Residuals and Regression Fit 247
11 6 Cross-Validation Bandwidth Selection 249
11 7 Asymptotic Distribution 252
11 8 Conditional Variance Estimation 255
11 9 Standard Errors 255
11 10Multiple Regressors 256
12 Series Estimation 259
12 1 Approximation by Series 259
12 2 Splines 259
12 3 Partially Linear Model 261
12 4 Additively Separable Models 261
12 5 Uniform Approximations 261
12 6 Runge’s Phenomenon 263 12 7 Approximating Regression 263 12 8 Residuals and Regression Fit 266
12 9 Cross-Validation Model Selection 266
12 10Convergence in Mean-Square 267
12 11Uniform Convergence 268
12 12Asymptotic Normality 269
12 13Asymptotic Normality with Undersmoothing 270
12 14Regression Estimation 271
12 15Kernel Versus Series Regression 272
12 16Technical Proofs 272
13 Generalized Method of Moments 278
13 1 Overidentified Linear Model 278
13 2 GMM Estimator 279
13 3 Distribution of GMM Estimator 280
13 4 Estimation of the Efficient Weight Matrix 281
13 5 GMM: The General Case 282
13 6 Over-Identification Test 282
13 7 Hypothesis Testing: The Distance Statistic 283
13 8 Conditional Moment Restrictions 284 13 9 Bootstrap GMM Inference 285
Exercises 287
14 Empirical Likelihood 289
14 1 Non-Parametric Likelihood 289
14 2 Asymptotic Distribution of EL Estimator 291
14 3 Overidentifying Restrictions 292
14 4 Testing 293
14 5 Numerical Computation 294
CONTENTS vi
15 Endogeneity 296
15 1 Instrumental Variables 297
15 2 Reduced Form 298
15 3 Identification 299
15 4 Estimation 299
15 5 Special Cases: IV and 2SLS 299
15 6 Bekker Asymptotics 301 15 7 Identification Failure 302
Exercises 304
16 Univariate Time Series 306
16 1 Stationarity and Ergodicity 306
16 2 Autoregressions 308
16 3 Stationarity of AR(1) Process 309
16 4 Lag Operator 309
16 5 Stationarity of AR(k) 310
16 6 Estimation 310
16 7 Asymptotic Distribution 311
16 8 Bootstrap for Autoregressions 312
16 9 Trend Stationarity 312
16 10Testing for Omitted Serial Correlation 313
16 11Model Selection 314
16 12Autoregressive Unit Roots 314
17 Multivariate Time Series 316
17 1 Vector Autoregressions (VARs) 316
17 2 Estimation 317
17 3 Restricted VARs 317
17 4 Single Equation from a VAR 317
17 5 Testing for Omitted Serial Correlation 318
17 6 Selection of Lag Length in an VAR 318
17 7 Granger Causality 319
17 8 Cointegration 319
17 9 Cointegrated VARs 320
18 Limited Dependent Variables 322
18 1 Binary Choice 322
18 2 Count Data 323
18 3 Censored Data 324
18 4 Sample Selection 325
19 Panel Data 327
19 1 Individual-Effects Model 327
19 2 Fixed Effects 327
19 3 Dynamic Panel Regression 329
20 Nonparametric Density Estimation 330
20 1 Kernel Density Estimation 330
20 2 Asymptotic MSE for Kernel Estimates 332
CONTENTS vii
A Matrix Algebra 335
A 1 Notation 335 A 2 Matrix Addition 336
A 3 Matrix Multiplication 336 A 4 Trace 337 A 5 Rank and Inverse 338 A 6 Determinant 339 A 7 Eigenvalues 340 A 8 Positive Definiteness 341 A 9 Singular Values 342 A 10 Matrix Calculus 342 A 11 Kronecker Products and the Vec Operator 343 A 12 Vector Norms 343 A 13 Matrix Norms 347
A 14 Matrix Inequalities
348
B Probability 351
B 1 Foundations 351 B 2 Random Variables 353 B 3 Expectation 353 B 4 Gamma Function 354 B 5 Common Distributions 355 B 6 Multivariate Random Variables 357
B 7 Conditional Distributions and Expectation 359
B 8 Transformations 361 B 9 Normal and Related Distributions 362 B 10 Inequalities 364
B 11 Maximum Likelihood 367
C Numerical Optimization 372
C 1 Grid Search 372 C 2 Gradient Methods 372 C 3 Derivative-Free Methods 374
