Regarding OP's comment below, one way to assess the model with a test set is by comparing in-model to out-of-model mean squared error (MSE). If the dependance of y on x is non-linear, and the x's are in order, then you could get a negative R-sq with the test set. I noticed that you use the first three quarters of your data as the training set, rather than taking a random sample (as in this example). This is equivalent to saying that the test set is modeled better using it's mean, than using the model derived from the training set. If, for example, the model is a very poor predictor with the test set, then the residuals can actually be larger than the total variation in test set. value less than 0 (when defined this way). In this contrived example there is not much difference, but it is very possible to have an R-sq. # fraction of variability explained by the model # NOT the fraction of variability explained by the model SS.regression <- sum((test.pred - mean(test.y))^2) SS.residual <- sum((test.y - test.pred)^2) Summary(fit)$r.squared # both are = R.squaredīut this does not work with the test set (e.g., when you make predictions from a model). SS.regression/SS.total # fraction of variation explained by the modelġ-SS.residual/SS.total # same thing, for model frame ONLY!!! SS.regression <- sum((fitted(fit)-mean(df$y))^2) So R.sq is the fraction of variability in the dataset that is explained by the model, and will always be between 0 and 1. The conventional definition is:įor the training set, and the training set ONLY, The second problem is the definition of R-squared. Using lm(.) this way allows you, for example to generate predictions without all the matrix multiplication. Now fit has the model based on the training set. Train <- sample(1:nrow(df),0.75*nrow(df)) # random sample of 75% of data So, assuming your data is in two vectors x and y, set.seed(1) # for reproducible example lm(.) is meant to be used with a data frame, with the formula expressions referencing columns in the df. First, this is not a good way to use lm(.).
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |