Abstract

It is clear that with ever improving computational power and endless data, there have been more breakthroughs in Machine Learning. Some practices have clearly emerged as promising while building a neural network. A performance metric to judge the model, is to see if it is in the wrong side of bias or variance. While building a classifier, cases with high bias, and high variance crop up. This paper shall attempt to shed some light on the problem of bias-variance, and how to solve them, with some approaches to perform Regularization.

References

• Sebastian Raschka 2016 Model evaluation, model selection, and algorithm selection in machine learning.
• Bengio, Yoshua, and Yves Grandvalet. 2004. “No Unbiased Estimator of the Variance of K-Fold Cross-Validation.” J. Mach. Learn. Res. 5 (December). JMLR.org: 1089–1105.
• Stanford University, Coursera 2018 Machine Learning Computing Systems.
• Google 2018 Machine Learning Crash Course.
• Tan, Pang-Ning, Michael Steinbach, and Vipin Kumar. 2005 Classification: Basic Concepts, Decision Trees, and Model Evaluation.” In Introduction to Data Mining. Boston: Pearson Addison Wesley.
• Hawkins, Douglas M., Subhash C. Basak, and Denise Mills. 2003. “Assessing Model Fit by Cross-Validation.” Journal of Chemical Information and Computer Sciences 43 (2). American Chemical Society: 579–86.
• Anders Krogh, John A. Hertz. A Simple Weight Decay Can Improve Generalization.
• Juergen Schmidhuber 2014 Deep Learning: An Overview
• Andrew Ng. Feature selection, L1 vs. L2 regularization
• Coursera 2018 deeplearning.ai