Modern portfolio theory claims that diversification into non-correlated or negatively correlated activities reduces the overall risk of a portfolio. Considering the total income of a bank as a portfolio of interest income and non-interest income, this paper investigates how the variability of interest income and non- interest income, and covariance between interest income and non-interest income influence the various risk factors of banks. We set out a study in the Indian context. We have extracted data for the period 2005-2017 and employed an extended version of Ridge, Lasso and Elastics Net regression to take care of multi-collinearly in our data. We have considered 10-fold cross-validation techniques to get optimal values of tuning parameters for Ridge, Lasso, and Elastics Net regression (which is a convex combination of ridge and the LASSO). We have compared different regression techniques by comparing RMSE and R2. We observe that non-interest income is positively correlated with interest income in the Indian context, but it does stabilize variance, idiosyncratic risk & market risk (Beta) of Indian Banks.