Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First R^2-value of n-fold cross-validation incorrect #117

Closed
olafdimigen opened this issue Jun 23, 2021 · 2 comments
Closed

First R^2-value of n-fold cross-validation incorrect #117

olafdimigen opened this issue Jun 23, 2021 · 2 comments

Comments

@olafdimigen
Copy link
Member

Computation of cross-validated r2, tested with 3 folds (i.e., 2 fold events), spaced at 1/3 and 2/3 of cont. data.
Function returns 3 values, but first one is negative:

R2 = uf_checkmodelfit(EEG,'method','crossValR2','fold_event',{'foldmarker'},'channel',1)

Resulting in the following R2: [-0.583 0.1999 0.2045] % note negative value in R1(1)

@behinger
Copy link
Member

behinger commented Jun 23, 2021

while R² can never be <0, cross-validated R² can be. This was initially very unintuitive for me.
So it is possible, but seems unlikely with your other two values.

Could you add more folds (randomly placed, just for testing) to further debug it?

@olafdimigen
Copy link
Member Author

Good point, thanks. Looked at it further and this was indeed a consequence of very unevenly spaced fold events in my testing data. Closing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants