Title: Penalized likelihood and model averaging
Authors: Zhou, Jianhong (周建红)
Abstract: Penalized likelihood and model averaging are alternatives to model selection. The
former has the attractive feature of model selection, and its parameter estimation can be
achieved by a single minimization problem with computational cost growing polynomially
with the sample size. The latter compromises across all the competing models so
that it can incorporate model uncertainty into the estimation process. Model averaging
also can avoid selecting very poor models, which in turn holds the promise of reducing
estimation risks.
Penalized likelihood and model averaging have advantages over model selection
in some aspects, and they have been developed in parallel. This prompts the question
whether there exists any relationship between penalized likelihood and model averaging.
However, to the best of my knowledge, there is a paucity of literature focused on this
question. An creditable exception is made by Ullah et al. (2013). They propose a new
generalized ridge estimator which is one of the penalized likelihoods, and prove that
this estimator is algebraically identical to the model average estimator.
In this thesis, we choose the Tobit model and Poisson regression model to study
penalized likelihood, model averaging and the relationship between them. The Tobit
model is now a standard approach to model censored dependent variables. Based on
the seminal contribution of Tobin (1958), many extensions of the original Tobit model
have been developed, and numerous applications of these models have appeared in economics
since the 1970s. The Poisson regression model is a widely used econometric
and statistical tool for studying the relationship between a Poisson-type response variable
and a set of explanatory variables. We arrange this thesis in the following manner:
Chapter 1 presents the rationale and introduction. We also introduce the Tobit model,
the Poisson regression model, and the methodology of the performance measures.
Chapter 2 surveys the relevant literature on model selection, penalized likelihood,
and model averaging. Additionally, under the statistical framework, we discuss the
relationship between model selection and penalized likelihood, and the relationship
between pretest and the WALS (weighted-average least squares) estimator. We also
present model averaging, i.e., Bayesian model averaging and frequestist model averaging
under the same statistical framework.
Chapter 3 considers the Tobit I model. First, we propose one-step sparse estimates,
and their adjusted implication is also suggested. We conduct a simulation study to
investigate the performance of one-step sparse estimates with LASSO (least absolute
shrinkage and selection operator) and ALASSO (adaptive LASSO) penalties. For simplicity,
we call them LASSO estimator and ALASSO estimator. The results demonstrate
that, compared with LASSO estimators, ALASSO estimators usually have advantages
on getting more accurate results in terms of mean square error, and providing a more
accurate number of correct zeros. However, they have a disadvantage on producing a
more accurate number of correct nonzeros. Second, we explore the relationships among
one-step sparse estimates, model selection, and model averaging. The results by both a
simulation study and empirical example, demonstrate that none of these three methods
consistently performs better than the others. Model averaging tends to get more accurate
results when there is a high noise level to the model; otherwise, none of these three
methods has an obvious superiority.
Chapter 4 considers the Tobit II model. We propose a procedure combining the
Heckman two-step procedure with penalized regression for parameter estimation and
variable selection. We investigate the finite samples results of this procedure with
LASSO and ALASSO penalties by designing a simulation study. The conclusion is
similar with that of Chapter 3. Next, we study the relationships among this procedure,
model selection, and model averaging. From the results by both simulation study and
empirical example, it can be seen that each method has its own advantage. Especially
in cases with a high level of noise, model averaging has competitive strength over other
methods.
Chapter 5 develops a model averaging procedure based on an unbiased estimator of
the expected Kullback-Leibler distance for the Poisson regression. Simulation studies
show that, compared with other commonly used model selection and model average
estimators, the proposed model average estimator performs better in certain situations,
especially when the model is nonsparse. In all the cases we simulated, the proposed
method never produces the worst results. Our proposed method is further applied to a
real data example, demonstrating the advantage of our method.
Finally, Chapter 6 summarizes this thesis and presents possible directions for future
study.
Notes: CityU Call Number: QA276 .Z45 2015; xviii, 173 pages : illustrations 30 cm; Thesis (Ph.D.)--City University of Hong Kong, 2015.; Includes bibliographical references (pages 153-166)
↧