Model Evaluation

Posted 2020-09-28 付小同

 

1. Confusion Matrix

 

Fact\Predict	Class A	Class B
Class A	True Positive	False Negative
Class B	False Positive	True Nagative

 

 

 

A confusion table for Class A

 

Positive/ Negative: if target class is A, then the predict A is Positve, Others are negative.

True (P/N): if Predict = Fact, then it‘s True.

 

2. Measures based on Confusion Matrix

 a. Accuracy = TN+TP/ALL

　　comments: not good measure when data are unbalanced.

b. True Positive Rate/ recall/ sensitivity =  TP / TP + FN

　　comments: use it when Positive results are important

c. True Negative Rate =  TN / TN + FP

 

R for Confusion Matrix:

library(SDMTools)

confusion.matrix(svmmodel.truth,svmmodel.class)

 

3. ROC curve (bio-classification)

y: sensitivity

x: specificity

 

The bigger the Area of ROC is, the more accurate the model is.