Graphical Investigation of Ridge Estimators When the Eigenvalues of the Matrix (X'X) are Skewed

Author(s): C. A. Uzuke, J. I. Mbegbu

Methods of estimating the ridge parameter in ridge regression analysis are available in the literature. This paper proposed some methods based on the works of Lawless and Wang (1976) and Khalaf and Shurkur (2005). A simulation study was conducted and mean square error (MSE) criterion was used to compare the performances of the proposed estimators and some other existing ones. It was observed that the performance of the these estimators depend on the variance of the random error , the correlation among the explanatory variables , the sample size and the number of explanatory variables . The increase in the number of explanatory variables and increase in the sample size reduces the MSE of the estimators even when the correlation between the explanatory variables are high, but for small sample size, MSE increases as the values of increases. One of the proposed methods outperforms all the other existing and proposed methods considered in terms of MSE values.

Eigenvalues, Mean Square Error, Multicollinearity, Ridge regression, Skewness.

1. Alkassab M. M and Qwaider O. Q (2010) A Comparison Between Unbiased Ridge and Least Square Regression Methods Using Simulation Technique. Journal of Modern Applied Statistical Methods vol 9 (2) pp 488-494.
2. Alkhamisi, M., Khalaf, S. and Shukur, G. (2006) Some Modifications for Choosing Ridge Parameters. Communications in Statistics – Theory and Methods vol 35 (11), pp 2005-2020
3. Ayres F. (1962) Theory and Problems of Matrices. Schaum Outline Series McGraw Hill NewYork.
4. Bata F.M., Ramanathan T. V. and Gore S. D (2008) The Efficiency of Modified Jackknife and Ridge –type regression estimators. A comparison Surveys in Mathematics and its Applications Vol. 3 pp 111-122
5. Cule E. and De Iorio M. (2013) Ridge Regression in Prediction Problems: Automatic Choice of the Ridge Parameter. Genetic Epidemiology vol 37, pp 704–714, Wiley Periodicals, Inc.
6. Dorugade A. V (2014) On Comparison of some Ridge Parameters in Ridge Regression. Journal of Applied Statistics vol, 15 (1) pp 33 – 45
7. Hoerl A. E. and Kennard, R. W. (1970a). Ridge Regression: Biased Estimation for Non-Orthogonal Problems. Technometrics vol. 12 pp 55 – 67
8. Hoerl A. E., Kennard R. W. and Baldwin, K. F. (1975) Ridge Regression: Some Simulations. Communications in Statistics, vol. 4, pp 105-123
9. Khalaf G. (2012a), A Proposed Ridge Parameter to Improve the Least Square Estimation Journal of Modern Applied Statistical Methods. Vol 11 (2), pp443-449.
10. Khalaf G. (2012b) Improved Estimator in the Presence of Multicollinearity. Journal of Modern Applied Statistical Methods Vol. 11, (1) pp152-157.
11. Khalaf G. and Shukur G. (2005) Choosing Ridge Regression Problems for Regression Problems. Communications in Statistics – Theory and Methods vol, 34, pp 1177-1182.
12. Kibria B. M. G (2003). Performance of Some New Ridge Regression Estimators Communications in Statistics – Simulation and Computation vol 32, (2), pp 419-435.
13. Lawless J. F. and Wang, P. (1976). A simulation Study of Ridge and Other Regression Estimators Communications in Statistics A, vol. 5 pp 307 – 323.
14. McDonald G. C. and Galarneau D. I. (1975). A Monte Carlo Evaluation of Some Ridge-Type Estimators. Journal of the American Statistical Association, vol. 70, pp 407 – 416.
15. Muniz, G., Kibria, B. M. G. and Shukur, G. (2010) On Developing Ridge Regression Parameters: A Graphical Investigation, Working Paper.
16. Newhouse, J. P. and Oman, S. D. (1971). An Evaluation of Ridge Estimators. Rand Corporation, vol. 15, no. 1,P-716-PR.
17. Pasha, G. R. and Ali Sha M. A. (2004) Application of Ridge Regression to Multicolinear Data, Journal of Research (Science) Bahauddin Zakariya University, Multan Pakistan pp 97-106
18. Ross S. M (2009) Introduction to Probability and Statistics for Engineers and Scientists 4th Ed. Associated Press P.267
19. Saleh, A. K. Md. E and Kibria, B. M. G. (1993). Performance of Some New Preliminary Test Ridge Regression Estimators and their Properties. Communications in Statistics- Theory and Methods, vol. 22, pp 2747 – 2764.
20. Schmidt D. F and Makalic E. (2009) “Universal Model for the Exponential Distribution” IEEE Transaction on Information Theory Vol 55 (7) pp. 3087 -3090
21. Spiegel L. and Stephen M. L (2008) Linear Algebra Schaum Outline Series
22. Szekely, G. J and Mori, T. F. (2001) A Characteristic Measure of Asymmetry and its Application for Testing Diagonal Symmetry, Communications in Statistics – Theory and Methods vol. 30(8), pp 1633-1639
23. Uzuke C. A. Mbegbu J. I and Nwosu C. R (2015) Peformance of Kibria, Khalaf and Shurkur method when the eigenvalues are skewed, Communications in Statistics-Simulation and Computation DOI: 10.1080/03610918.2015.1035444.
24. von Hippel P. T. (2005) Mean, Median and Skew: Correcting a Textbook Rule,. Journal of Statistics Education vol. 13 (2) pp 136-141
25. Zang J. and Ibrahim, M. (2005). A simulation Study on SPSS Ridge Regression and Ordinary Least Square Regression Procedures for Multicollinearity Data. Journal of Applied Statistics, vol. 32, pp 571- 586